[pgpool-general: 57] Re: [Pgpool-general] seemingly hung pgpool process consuming 100% CPU
Tatsuo Ishii
ishii at postgresql.org
Thu Dec 8 17:06:11 JST 2011
> On Wed, Dec 7, 2011 at 11:10 PM, Tatsuo Ishii <ishii at postgresql.org> wrote:
>> How is your PostgreSQL server configuration?
>
> Did you want a copy of my postgresql.conf ?
Yes, please. Also please show me the output of "show pool_status;".
>> Before you said you have 1 master and 2 standbys. Is this still
>> correct? It seems you only have two servers from the gdb trace and I
>> would like to make sure that.
>
> Yes, we still have 3 servers. pcp_node_count reports 3.
Ok.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp
>>> Nope, no SSL enabled anywhere.
>>>
>>> On Tue, Dec 6, 2011 at 6:49 PM, Tatsuo Ishii <ishii at postgresql.org> wrote:
>>>> Can you tell me if you are enabling SSL between frontend and pgpool
>>>> AND/OR pgpool and PostgreSQL?
>>>> --
>>>> Tatsuo Ishii
>>>> SRA OSS, Inc. Japan
>>>> English: http://www.sraoss.co.jp/index_en.php
>>>> Japanese: http://www.sraoss.co.jp
>>>>
>>>>> Lonni,
>>>>>
>>>>> First of all, pgpool-general at pgfoundry has moved to pgpool-general at pgpool.net.
>>>>> Please subscribe here:
>>>>>
>>>>> http://www.pgpool.net/mailman/listinfo/pgpool-general
>>>>>
>>>>> From: Lonni J Friedman <netllama at gmail.com>
>>>>> Subject: Re: [Pgpool-general] seemingly hung pgpool process consuming 100% CPU
>>>>> Date: Tue, 6 Dec 2011 16:23:41 -0800
>>>>> Message-ID: <CAP=oouHQACD6ELcHOZz+3Oz8NkbbgjK3gSRbcbHrPKoi_DRP8g at mail.gmail.com>
>>>>>
>>>>>> On Wed, Nov 23, 2011 at 10:51 PM, Tatsuo Ishii <ishii at sraoss.co.jp> wrote:
>>>>>>>> On Wed, Nov 23, 2011 at 10:42 PM, Tatsuo Ishii <ishii at sraoss.co.jp> wrote:
>>>>>>>>>> Not wanting to be impatient, but I'm very concerned about this
>>>>>>>>>> problem, since its impossible to predict when it will occur. Is there
>>>>>>>>>> additional information that I can provide to investigate this further?
>>>>>>>>>
>>>>>>>>> I really need to know where pgpool is looping.
>>>>>>>>
>>>>>>>> OK, how can I capture that information?
>>>>>>>
>>>>>>> You already attached to the pgpool process. So just type "n" (for
>>>>>>> "next") will tell you next line to execute. If pgpool really loops,
>>>>>>> "n" should show the same line after some repeating "n".
>>>>>>
>>>>>> OK, this reproduced again. Here's the output:
>>>>>> #######
>>>>>> (gdb) bt
>>>>>> #0 0x0000000000419305 in pool_process_query (frontend=0x413e1f0,
>>>>>> backend=0x24d3520, reset_request=<value optimized out>) at
>>>>>> pool_process_query.c:111
>>>>>> #1 0x000000000040ae42 in do_child (unix_fd=3, inet_fd=<value
>>>>>> optimized out>) at child.c:354
>>>>>> #2 0x00000000004054c5 in fork_a_child (unix_fd=3, inet_fd=4, id=126)
>>>>>> at main.c:1072
>>>>>> #3 0x0000000000407b1c in main (argc=<value optimized out>,
>>>>>> argv=<value optimized out>) at main.c:549
>>>>>> (gdb) cont
>>>>>> Continuing.
>>>>>> ^C
>>>>>> Program received signal SIGINT, Interrupt.
>>>>>> pool_ssl_pending (cp=0x413e1f0) at pool_ssl.c:247
>>>>>> 247 {
>>>>>> (gdb) n
>>>>>> 248 if (cp->ssl_active > 0 && SSL_pending(cp->ssl) > 0)
>>>>>> (gdb) n
>>>>>> 251 }
>>>>>> (gdb) n
>>>>>> is_cache_empty (frontend=0x413e1f0, backend=0x24d3520) at
>>>>>> pool_process_query.c:3232
>>>>>> 3232 if (!pool_read_buffer_is_empty(frontend))
>>>>>> (gdb) n
>>>>>> 3235 for (i=0;i<NUM_BACKENDS;i++)
>>>>>> (gdb) n
>>>>>> 3237 if (!VALID_BACKEND(i))
>>>>>> (gdb) n
>>>>>> 3244 if (pool_ssl_pending(CONNECTION(backend, i)))
>>>>>> (gdb) n
>>>>>> 3247 if (CONNECTION(backend, i)->len > 0)
>>>>>> (gdb) n
>>>>>> 3237 if (!VALID_BACKEND(i))
>>>>>> (gdb) n
>>>>>> 3244 if (pool_ssl_pending(CONNECTION(backend, i)))
>>>>>> (gdb) n
>>>>>> 3247 if (CONNECTION(backend, i)->len > 0)
>>>>>> (gdb) n
>>>>>> 3252 }
>>>>>> (gdb) n
>>>>>> 3235 for (i=0;i<NUM_BACKENDS;i++)
>>>>>> (gdb) n
>>>>>> 3252 }
>>>>>> (gdb) n
>>>>>> pool_process_query (frontend=0x413e1f0, backend=0x24d3520,
>>>>>> reset_request=<value optimized out>) at pool_process_query.c:361
>>>>>> 361 if
>>>>>> (!pool_read_buffer_is_empty(frontend) && !pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 379 if
>>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>>> pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 379 if
>>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>>> pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 388 if (got_sighup)
>>>>>> (gdb) n
>>>>>> 111 state = 0;
>>>>>> (gdb) n
>>>>>> 116 if (state == 0 && reset_request)
>>>>>> (gdb) n
>>>>>> 159 check_stop_request();
>>>>>> (gdb) n
>>>>>> 165 if (*InRecovery > 0 &&
>>>>>> pool_config->client_idle_limit_in_recovery == -1)
>>>>>> (gdb) n
>>>>>> 179 if (is_cache_empty(frontend, backend) &&
>>>>>> !pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 361 if
>>>>>> (!pool_read_buffer_is_empty(frontend) && !pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 379 if
>>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>>> pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 379 if
>>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>>> pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 388 if (got_sighup)
>>>>>> (gdb) n
>>>>>> 111 state = 0;
>>>>>> (gdb) n
>>>>>> 116 if (state == 0 && reset_request)
>>>>>> (gdb) n
>>>>>> 159 check_stop_request();
>>>>>> (gdb) n
>>>>>> 165 if (*InRecovery > 0 &&
>>>>>> pool_config->client_idle_limit_in_recovery == -1)
>>>>>> (gdb) n
>>>>>> 179 if (is_cache_empty(frontend, backend) &&
>>>>>> !pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 361 if
>>>>>> (!pool_read_buffer_is_empty(frontend) && !pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 379 if
>>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>>> pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 379 if
>>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>>> pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 388 if (got_sighup)
>>>>>> (gdb) n
>>>>>> 111 state = 0;
>>>>>> (gdb) n
>>>>>> 116 if (state == 0 && reset_request)
>>>>>> (gdb) n
>>>>>> 159 check_stop_request();
>>>>>> (gdb) n
>>>>>> 165 if (*InRecovery > 0 &&
>>>>>> pool_config->client_idle_limit_in_recovery == -1)
>>>>>> (gdb) n
>>>>>> 179 if (is_cache_empty(frontend, backend) &&
>>>>>> !pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 361 if
>>>>>> (!pool_read_buffer_is_empty(frontend) && !pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 379 if
>>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>>> pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 379 if
>>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>>> pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 388 if (got_sighup)
>>>>>> (gdb) n
>>>>>> 111 state = 0;
>>>>>> (gdb) n
>>>>>> 116 if (state == 0 && reset_request)
>>>>>> (gdb) n
>>>>>> 159 check_stop_request();
>>>>>> (gdb) n
>>>>>> 165 if (*InRecovery > 0 &&
>>>>>> pool_config->client_idle_limit_in_recovery == -1)
>>>>>> (gdb) n
>>>>>> 179 if (is_cache_empty(frontend, backend) &&
>>>>>> !pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 361 if
>>>>>> (!pool_read_buffer_is_empty(frontend) && !pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 379 if
>>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>>> pool_is_query_in_progress())
>>>>>> (gdb) n
>>>>>> 379 if
>>>>>> (!pool_read_buffer_is_empty(MASTER(backend)) ||
>>>>>> pool_is_query_in_progress())
>>>>>> (gdb) bt
>>>>>> #0 pool_process_query (frontend=0x413e1f0, backend=0x24d3520,
>>>>>> reset_request=<value optimized out>) at pool_process_query.c:379
>>>>>> #1 0x000000000040ae42 in do_child (unix_fd=3, inet_fd=<value
>>>>>> optimized out>) at child.c:354
>>>>>> #2 0x00000000004054c5 in fork_a_child (unix_fd=3, inet_fd=4, id=126)
>>>>>> at main.c:1072
>>>>>> #3 0x0000000000407b1c in main (argc=<value optimized out>,
>>>>>> argv=<value optimized out>) at main.c:549
>>>>>> (gdb) q
>>>>>> A debugging session is active.
>>>>>>
>>>>>> Inferior 1 [process 22143] will be detached.
>>>>>> #######
>>>>>>
>>>>>>
>>>>>> Does this clarify where the problem exists? If so, is it fixed in 3.1.1?
>>>>>>
>>>>>> thanks
>>>>>
>>>>> Thanks. I will look into this. I'm sure this is not fixed in 3.1.1
>>>>> since the issue above has not been addressed yet.
>
>
>
> --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> L. Friedman netllama at gmail.com
> LlamaLand https://netllama.linux-sxs.org
More information about the pgpool-general
mailing list