[pgpool-general: 55] Re: [Pgpool-general] seemingly hung pgpool process consuming 100% CPU

Tatsuo Ishii ishii at postgresql.org
Thu Dec 8 16:10:05 JST 2011


How is your PostgreSQL server configuration?

Before you said you have 1 master and 2 standbys. Is this still
correct? It seems you only have two servers from the gdb trace and I
would like to make sure that.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

> Nope, no SSL enabled anywhere.
> 
> On Tue, Dec 6, 2011 at 6:49 PM, Tatsuo Ishii <ishii at postgresql.org> wrote:
>> Can you tell me if you are enabling SSL between frontend and pgpool
>> AND/OR pgpool and PostgreSQL?
>> --
>> Tatsuo Ishii
>> SRA OSS, Inc. Japan
>> English: http://www.sraoss.co.jp/index_en.php
>> Japanese: http://www.sraoss.co.jp
>>
>>> Lonni,
>>>
>>> First of all, pgpool-general at pgfoundry has moved to  pgpool-general at pgpool.net.
>>> Please subscribe here:
>>>
>>> http://www.pgpool.net/mailman/listinfo/pgpool-general
>>>
>>> From: Lonni J Friedman <netllama at gmail.com>
>>> Subject: Re: [Pgpool-general] seemingly hung pgpool process consuming 100% CPU
>>> Date: Tue, 6 Dec 2011 16:23:41 -0800
>>> Message-ID: <CAP=oouHQACD6ELcHOZz+3Oz8NkbbgjK3gSRbcbHrPKoi_DRP8g at mail.gmail.com>
>>>
>>>> On Wed, Nov 23, 2011 at 10:51 PM, Tatsuo Ishii <ishii at sraoss.co.jp> wrote:
>>>>>> On Wed, Nov 23, 2011 at 10:42 PM, Tatsuo Ishii <ishii at sraoss.co.jp> wrote:
>>>>>>>> Not wanting to be impatient, but I'm very concerned about this
>>>>>>>> problem, since its impossible to predict when it will occur.  Is there
>>>>>>>> additional information that I can provide to investigate this further?
>>>>>>>
>>>>>>> I really need to know where pgpool is looping.
>>>>>>
>>>>>> OK, how can I capture that information?
>>>>>
>>>>> You already attached to the pgpool process. So just type "n" (for
>>>>> "next") will tell you next line to execute. If pgpool really loops,
>>>>> "n" should show the same line after some repeating "n".
>>>>
>>>> OK, this reproduced again.  Here's the output:
>>>> #######
>>>> (gdb) bt
>>>> #0  0x0000000000419305 in pool_process_query (frontend=0x413e1f0,
>>>> backend=0x24d3520, reset_request=<value optimized out>) at
>>>> pool_process_query.c:111
>>>> #1  0x000000000040ae42 in do_child (unix_fd=3, inet_fd=<value
>>>> optimized out>) at child.c:354
>>>> #2  0x00000000004054c5 in fork_a_child (unix_fd=3, inet_fd=4, id=126)
>>>> at main.c:1072
>>>> #3  0x0000000000407b1c in main (argc=<value optimized out>,
>>>> argv=<value optimized out>) at main.c:549
>>>> (gdb) cont
>>>> Continuing.
>>>> ^C
>>>> Program received signal SIGINT, Interrupt.
>>>> pool_ssl_pending (cp=0x413e1f0) at pool_ssl.c:247
>>>> 247     {
>>>> (gdb) n
>>>> 248             if (cp->ssl_active > 0 && SSL_pending(cp->ssl) > 0)
>>>> (gdb) n
>>>> 251     }
>>>> (gdb) n
>>>> is_cache_empty (frontend=0x413e1f0, backend=0x24d3520) at
>>>> pool_process_query.c:3232
>>>> 3232            if (!pool_read_buffer_is_empty(frontend))
>>>> (gdb) n
>>>> 3235            for (i=0;i<NUM_BACKENDS;i++)
>>>> (gdb) n
>>>> 3237                    if (!VALID_BACKEND(i))
>>>> (gdb) n
>>>> 3244                    if (pool_ssl_pending(CONNECTION(backend, i)))
>>>> (gdb) n
>>>> 3247                    if (CONNECTION(backend, i)->len > 0)
>>>> (gdb) n
>>>> 3237                    if (!VALID_BACKEND(i))
>>>> (gdb) n
>>>> 3244                    if (pool_ssl_pending(CONNECTION(backend, i)))
>>>> (gdb) n
>>>> 3247                    if (CONNECTION(backend, i)->len > 0)
>>>> (gdb) n
>>>> 3252    }
>>>> (gdb) n
>>>> 3235            for (i=0;i<NUM_BACKENDS;i++)
>>>> (gdb) n
>>>> 3252    }
>>>> (gdb) n
>>>> pool_process_query (frontend=0x413e1f0, backend=0x24d3520,
>>>> reset_request=<value optimized out>) at pool_process_query.c:361
>>>> 361                             if
>>>> (!pool_read_buffer_is_empty(frontend) && !pool_is_query_in_progress())
>>>> (gdb) n
>>>> 379                             if
>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>> pool_is_query_in_progress())
>>>> (gdb) n
>>>> 379                             if
>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>> pool_is_query_in_progress())
>>>> (gdb) n
>>>> 388                     if (got_sighup)
>>>> (gdb) n
>>>> 111             state = 0;
>>>> (gdb) n
>>>> 116                     if (state == 0 && reset_request)
>>>> (gdb) n
>>>> 159                     check_stop_request();
>>>> (gdb) n
>>>> 165                     if (*InRecovery > 0 &&
>>>> pool_config->client_idle_limit_in_recovery == -1)
>>>> (gdb) n
>>>> 179                     if (is_cache_empty(frontend, backend) &&
>>>> !pool_is_query_in_progress())
>>>> (gdb) n
>>>> 361                             if
>>>> (!pool_read_buffer_is_empty(frontend) && !pool_is_query_in_progress())
>>>> (gdb) n
>>>> 379                             if
>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>> pool_is_query_in_progress())
>>>> (gdb) n
>>>> 379                             if
>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>> pool_is_query_in_progress())
>>>> (gdb) n
>>>> 388                     if (got_sighup)
>>>> (gdb) n
>>>> 111             state = 0;
>>>> (gdb) n
>>>> 116                     if (state == 0 && reset_request)
>>>> (gdb) n
>>>> 159                     check_stop_request();
>>>> (gdb) n
>>>> 165                     if (*InRecovery > 0 &&
>>>> pool_config->client_idle_limit_in_recovery == -1)
>>>> (gdb) n
>>>> 179                     if (is_cache_empty(frontend, backend) &&
>>>> !pool_is_query_in_progress())
>>>> (gdb) n
>>>> 361                             if
>>>> (!pool_read_buffer_is_empty(frontend) && !pool_is_query_in_progress())
>>>> (gdb) n
>>>> 379                             if
>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>> pool_is_query_in_progress())
>>>> (gdb) n
>>>> 379                             if
>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>> pool_is_query_in_progress())
>>>> (gdb) n
>>>> 388                     if (got_sighup)
>>>> (gdb) n
>>>> 111             state = 0;
>>>> (gdb) n
>>>> 116                     if (state == 0 && reset_request)
>>>> (gdb) n
>>>> 159                     check_stop_request();
>>>> (gdb) n
>>>> 165                     if (*InRecovery > 0 &&
>>>> pool_config->client_idle_limit_in_recovery == -1)
>>>> (gdb) n
>>>> 179                     if (is_cache_empty(frontend, backend) &&
>>>> !pool_is_query_in_progress())
>>>> (gdb) n
>>>> 361                             if
>>>> (!pool_read_buffer_is_empty(frontend) && !pool_is_query_in_progress())
>>>> (gdb) n
>>>> 379                             if
>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>> pool_is_query_in_progress())
>>>> (gdb) n
>>>> 379                             if
>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>> pool_is_query_in_progress())
>>>> (gdb) n
>>>> 388                     if (got_sighup)
>>>> (gdb) n
>>>> 111             state = 0;
>>>> (gdb) n
>>>> 116                     if (state == 0 && reset_request)
>>>> (gdb) n
>>>> 159                     check_stop_request();
>>>> (gdb) n
>>>> 165                     if (*InRecovery > 0 &&
>>>> pool_config->client_idle_limit_in_recovery == -1)
>>>> (gdb) n
>>>> 179                     if (is_cache_empty(frontend, backend) &&
>>>> !pool_is_query_in_progress())
>>>> (gdb) n
>>>> 361                             if
>>>> (!pool_read_buffer_is_empty(frontend) && !pool_is_query_in_progress())
>>>> (gdb) n
>>>> 379                             if
>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>> pool_is_query_in_progress())
>>>> (gdb) n
>>>> 379                             if
>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>> pool_is_query_in_progress())
>>>> (gdb) bt
>>>> #0  pool_process_query (frontend=0x413e1f0, backend=0x24d3520,
>>>> reset_request=<value optimized out>) at pool_process_query.c:379
>>>> #1  0x000000000040ae42 in do_child (unix_fd=3, inet_fd=<value
>>>> optimized out>) at child.c:354
>>>> #2  0x00000000004054c5 in fork_a_child (unix_fd=3, inet_fd=4, id=126)
>>>> at main.c:1072
>>>> #3  0x0000000000407b1c in main (argc=<value optimized out>,
>>>> argv=<value optimized out>) at main.c:549
>>>> (gdb) q
>>>> A debugging session is active.
>>>>
>>>>         Inferior 1 [process 22143] will be detached.
>>>> #######
>>>>
>>>>
>>>> Does this clarify where the problem exists?  If so, is it fixed in 3.1.1?
>>>>
>>>> thanks
>>>
>>> Thanks. I will look into this. I'm sure this is not fixed in 3.1.1
>>> since the issue above has not been addressed yet.


More information about the pgpool-general mailing list