[pgpool-general: 1246] Re: PGPool Stops Accepting Connections Without Errors

[Nick Webb]

On Thu, Dec 6, 2012 at 4:45 PM, Tatsuo Ishii <ishii at postgresql.org> wrote:
> We recently fixed the case if pcp_attach_node/pcp_detach_node are
> repeated, pgpool children are not forked thus cannot accept
> connections from frontend. For 3.1-stable tree:
>

Tatsuo,

Yes, sorry I didn't mention that, I'm already running with that
release (I just got 3.1.5 when doing ./pgpool -version).

I have the latest from git 3.1 stable as of 12/2/2012 (4 days ago).

Nick

>> Not sure this is a problem with PGPool or not, but every 3-4 days our
>> PGPool instance just stops accepting postgres connections, they all
>> time out.  When this situation occurs, the system is not particularly
>> busy, and CPU, memory, and number of current connections are all
>> normal.  What's more, connecting to the backends directly from the
>> PGPool host also works fine, and is quite snappy (via psql), but no
>> connections to PGPool work.  Restart pgpool and things are back up and
>> running in seconds.
>>
>> Looking at the log, there are no errors or warnings, not sure where to
>> look next.  I checked all the usual Linux log suspects, and nothing
>> else was going on (like maxing out some system resource, etc.).
>>
>> Here is the log immediately before and after the "hang up", which
>> happened approximately 19:42 UTC:
>> Dec  6 19:23:58 PGLB pgpool[3210]: statement: SELECT
>> Dec  6 19:23:58 PGLB pgpool[3210]: DB node id: 1 backend pid: 22221
>> statement: SELECT
>> Dec  6 19:23:58 PGLB pgpool[3210]: statement: SELECT
>> Dec  6 19:23:58 PGLB pgpool[3210]: DB node id: 1 backend pid: 22221
>> statement: SELECT
>> Dec  6 19:23:58 PGLB pgpool3210]: statement: SELECT
>> Dec  6 19:23:58 PGLB pgpool[3210]: DB node id: 1 backend pid: 22221
>> statement: SELECT
>> Dec  6 19:24:24 PGLB pgpool[26744]: statement:  DISCARD ALL
>> Dec  6 19:24:24 PGLB pgpool[26744]: DB node id: 0 backend pid: 27232
>> statement:  DISCARD ALL
>> Dec  6 19:24:24 PGLB pgpool[26744]: DB node id: 1 backend pid: 20837
>> statement:  DISCARD ALL
>> Dec  6 19:24:24 PGLB pgpool[26744]: connection received: host=3.myhost.com=43275
>> Dec  6 19:42:06 PGLB pgpool[26782]: statement:  DISCARD ALL
>> Dec  6 19:42:06 PGLB pgpool[26782]: DB node id: 0 backend pid: 28378
>> statement:  DISCARD ALL
>> Dec  6 19:42:06 PGLB pgpool[26782]: DB node id: 1 backend pid: 21991
>> statement:  DISCARD ALL
>> Dec  6 19:42:06 PGLB pgpool[26782]: connection received:
>> host=www.myhost.com port=57636
>>
>> [ note the lack of entries after this point, until the restart,
>> despite many attempted connections ]
>>
>> Dec  6 19:43:59 PGLB pgpool[26276]: received fast shutdown request
>> Dec  6 19:44:00 PGLB pgpool[13237]: connection received:
>> host=2.myhost.com port=5973
>> Dec  6 19:44:00 PGLB pgpool[13234]: pgpool-II successfully started.
>> version 3.1.5 (hatsuiboshi)
>>
>> Next time I'll run a "show pool_pools", but it's hard for me to
>> believe that all the slots were full.
>>
>> Thanks,
>>
>> Nick
>> --
>> Nick Webb
>> Principal Systems Engineer
>> Red Wire Services, LLC
>> T: 206.829.8621 / F: 206.826.1223
>>
>> 1725 SW Roxbury St. STE 5
>> Seattle, WA 98106
>>
>> http://www.redwireservices.com
>>
>> http://linkedin.com/in/NickWebb
>> _______________________________________________
>> pgpool-general mailing list
>> pgpool-general at pgpool.net
>> http://www.pgpool.net/mailman/listinfo/pgpool-general