[pgpool-general: 56] Re: [Pgpool-general] seemingly hung pgpool process consuming 100% CPU
Lonni J Friedman
netllama at gmail.com
Thu Dec 8 16:42:10 JST 2011
On Wed, Dec 7, 2011 at 11:10 PM, Tatsuo Ishii <ishii at postgresql.org> wrote:
> How is your PostgreSQL server configuration?
Did you want a copy of my postgresql.conf ?
>
> Before you said you have 1 master and 2 standbys. Is this still
> correct? It seems you only have two servers from the gdb trace and I
> would like to make sure that.
Yes, we still have 3 servers. pcp_node_count reports 3.
> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> English: http://www.sraoss.co.jp/index_en.php
> Japanese: http://www.sraoss.co.jp
>
>> Nope, no SSL enabled anywhere.
>>
>> On Tue, Dec 6, 2011 at 6:49 PM, Tatsuo Ishii <ishii at postgresql.org> wrote:
>>> Can you tell me if you are enabling SSL between frontend and pgpool
>>> AND/OR pgpool and PostgreSQL?
>>> --
>>> Tatsuo Ishii
>>> SRA OSS, Inc. Japan
>>> English: http://www.sraoss.co.jp/index_en.php
>>> Japanese: http://www.sraoss.co.jp
>>>
>>>> Lonni,
>>>>
>>>> First of all, pgpool-general at pgfoundry has moved to pgpool-general at pgpool.net.
>>>> Please subscribe here:
>>>>
>>>> http://www.pgpool.net/mailman/listinfo/pgpool-general
>>>>
>>>> From: Lonni J Friedman <netllama at gmail.com>
>>>> Subject: Re: [Pgpool-general] seemingly hung pgpool process consuming 100% CPU
>>>> Date: Tue, 6 Dec 2011 16:23:41 -0800
>>>> Message-ID: <CAP=oouHQACD6ELcHOZz+3Oz8NkbbgjK3gSRbcbHrPKoi_DRP8g at mail.gmail.com>
>>>>
>>>>> On Wed, Nov 23, 2011 at 10:51 PM, Tatsuo Ishii <ishii at sraoss.co.jp> wrote:
>>>>>>> On Wed, Nov 23, 2011 at 10:42 PM, Tatsuo Ishii <ishii at sraoss.co.jp> wrote:
>>>>>>>>> Not wanting to be impatient, but I'm very concerned about this
>>>>>>>>> problem, since its impossible to predict when it will occur. Is there
>>>>>>>>> additional information that I can provide to investigate this further?
>>>>>>>>
>>>>>>>> I really need to know where pgpool is looping.
>>>>>>>
>>>>>>> OK, how can I capture that information?
>>>>>>
>>>>>> You already attached to the pgpool process. So just type "n" (for
>>>>>> "next") will tell you next line to execute. If pgpool really loops,
>>>>>> "n" should show the same line after some repeating "n".
>>>>>
>>>>> OK, this reproduced again. Here's the output:
>>>>> #######
>>>>> (gdb) bt
>>>>> #0 0x0000000000419305 in pool_process_query (frontend=0x413e1f0,
>>>>> backend=0x24d3520, reset_request=<value optimized out>) at
>>>>> pool_process_query.c:111
>>>>> #1 0x000000000040ae42 in do_child (unix_fd=3, inet_fd=<value
>>>>> optimized out>) at child.c:354
>>>>> #2 0x00000000004054c5 in fork_a_child (unix_fd=3, inet_fd=4, id=126)
>>>>> at main.c:1072
>>>>> #3 0x0000000000407b1c in main (argc=<value optimized out>,
>>>>> argv=<value optimized out>) at main.c:549
>>>>> (gdb) cont
>>>>> Continuing.
>>>>> ^C
>>>>> Program received signal SIGINT, Interrupt.
>>>>> pool_ssl_pending (cp=0x413e1f0) at pool_ssl.c:247
>>>>> 247 {
>>>>> (gdb) n
>>>>> 248 if (cp->ssl_active > 0 && SSL_pending(cp->ssl) > 0)
>>>>> (gdb) n
>>>>> 251 }
>>>>> (gdb) n
>>>>> is_cache_empty (frontend=0x413e1f0, backend=0x24d3520) at
>>>>> pool_process_query.c:3232
>>>>> 3232 if (!pool_read_buffer_is_empty(frontend))
>>>>> (gdb) n
>>>>> 3235 for (i=0;i<NUM_BACKENDS;i++)
>>>>> (gdb) n
>>>>> 3237 if (!VALID_BACKEND(i))
>>>>> (gdb) n
>>>>> 3244 if (pool_ssl_pending(CONNECTION(backend, i)))
>>>>> (gdb) n
>>>>> 3247 if (CONNECTION(backend, i)->len > 0)
>>>>> (gdb) n
>>>>> 3237 if (!VALID_BACKEND(i))
>>>>> (gdb) n
>>>>> 3244 if (pool_ssl_pending(CONNECTION(backend, i)))
>>>>> (gdb) n
>>>>> 3247 if (CONNECTION(backend, i)->len > 0)
>>>>> (gdb) n
>>>>> 3252 }
>>>>> (gdb) n
>>>>> 3235 for (i=0;i<NUM_BACKENDS;i++)
>>>>> (gdb) n
>>>>> 3252 }
>>>>> (gdb) n
>>>>> pool_process_query (frontend=0x413e1f0, backend=0x24d3520,
>>>>> reset_request=<value optimized out>) at pool_process_query.c:361
>>>>> 361 if
>>>>> (!pool_read_buffer_is_empty(frontend) && !pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 379 if
>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>> pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 379 if
>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>> pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 388 if (got_sighup)
>>>>> (gdb) n
>>>>> 111 state = 0;
>>>>> (gdb) n
>>>>> 116 if (state == 0 && reset_request)
>>>>> (gdb) n
>>>>> 159 check_stop_request();
>>>>> (gdb) n
>>>>> 165 if (*InRecovery > 0 &&
>>>>> pool_config->client_idle_limit_in_recovery == -1)
>>>>> (gdb) n
>>>>> 179 if (is_cache_empty(frontend, backend) &&
>>>>> !pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 361 if
>>>>> (!pool_read_buffer_is_empty(frontend) && !pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 379 if
>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>> pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 379 if
>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>> pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 388 if (got_sighup)
>>>>> (gdb) n
>>>>> 111 state = 0;
>>>>> (gdb) n
>>>>> 116 if (state == 0 && reset_request)
>>>>> (gdb) n
>>>>> 159 check_stop_request();
>>>>> (gdb) n
>>>>> 165 if (*InRecovery > 0 &&
>>>>> pool_config->client_idle_limit_in_recovery == -1)
>>>>> (gdb) n
>>>>> 179 if (is_cache_empty(frontend, backend) &&
>>>>> !pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 361 if
>>>>> (!pool_read_buffer_is_empty(frontend) && !pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 379 if
>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>> pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 379 if
>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>> pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 388 if (got_sighup)
>>>>> (gdb) n
>>>>> 111 state = 0;
>>>>> (gdb) n
>>>>> 116 if (state == 0 && reset_request)
>>>>> (gdb) n
>>>>> 159 check_stop_request();
>>>>> (gdb) n
>>>>> 165 if (*InRecovery > 0 &&
>>>>> pool_config->client_idle_limit_in_recovery == -1)
>>>>> (gdb) n
>>>>> 179 if (is_cache_empty(frontend, backend) &&
>>>>> !pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 361 if
>>>>> (!pool_read_buffer_is_empty(frontend) && !pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 379 if
>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>> pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 379 if
>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>> pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 388 if (got_sighup)
>>>>> (gdb) n
>>>>> 111 state = 0;
>>>>> (gdb) n
>>>>> 116 if (state == 0 && reset_request)
>>>>> (gdb) n
>>>>> 159 check_stop_request();
>>>>> (gdb) n
>>>>> 165 if (*InRecovery > 0 &&
>>>>> pool_config->client_idle_limit_in_recovery == -1)
>>>>> (gdb) n
>>>>> 179 if (is_cache_empty(frontend, backend) &&
>>>>> !pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 361 if
>>>>> (!pool_read_buffer_is_empty(frontend) && !pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 379 if
>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>> pool_is_query_in_progress())
>>>>> (gdb) n
>>>>> 379 if
>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>> pool_is_query_in_progress())
>>>>> (gdb) bt
>>>>> #0 pool_process_query (frontend=0x413e1f0, backend=0x24d3520,
>>>>> reset_request=<value optimized out>) at pool_process_query.c:379
>>>>> #1 0x000000000040ae42 in do_child (unix_fd=3, inet_fd=<value
>>>>> optimized out>) at child.c:354
>>>>> #2 0x00000000004054c5 in fork_a_child (unix_fd=3, inet_fd=4, id=126)
>>>>> at main.c:1072
>>>>> #3 0x0000000000407b1c in main (argc=<value optimized out>,
>>>>> argv=<value optimized out>) at main.c:549
>>>>> (gdb) q
>>>>> A debugging session is active.
>>>>>
>>>>> Inferior 1 [process 22143] will be detached.
>>>>> #######
>>>>>
>>>>>
>>>>> Does this clarify where the problem exists? If so, is it fixed in 3.1.1?
>>>>>
>>>>> thanks
>>>>
>>>> Thanks. I will look into this. I'm sure this is not fixed in 3.1.1
>>>> since the issue above has not been addressed yet.
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
L. Friedman netllama at gmail.com
LlamaLand https://netllama.linux-sxs.org
More information about the pgpool-general
mailing list