[pgpool-general: 421] Re: pg-pool-3.0.4. -unable to create child processes

[Tatsuo Ishii]

I suspect you hit a bug fixed in 3.0.7:

      - Fix bug that the process exits before unlocking semaphore by a
        signal interrupt. (Tatsuo Ishii)
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

> Dear list,
> 
> we just encountered a production issue with pg-pool-3.0.4.: pg-pool killed all idle childprocesses which were waiting for a connection request and failed to create new ones.
> 
> I am unsure why the childs exited:
> 
> 
> 1)    As we have child_life_time = 300 I guess the childs exited normally after a 5 minute period.
> 
> 2)    We did a Config reload on pg_pool, and that could mean also that waiting child processes are killed
> 
> but pg_pool did not create new ones. So we were left with ~110 active pg-pool processes (which still seemed to work) but we could make no new connections to pgppol (can't tell if the connections have been queued or not).
> This is what the syslog says:
> 
> May  7 15:43:13 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 15:43:13 LOG:   pid 938: reload config files.
> May  7 15:43:13 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 15:43:13 LOG:   pid 2707: reload config files.
> May  7 16:09:29 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 16:09:17 ERROR: pid 938: fork() failed. reason: Resource temporarily unavailable
> May  7 16:09:34 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 16:09:34 LOG:   pid 29194: connect_inet_domain_socket_by_port: exit request has been sent
> May  7 16:09:34 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 16:09:34 ERROR: pid 29194: connection to <backendserver ha1 here> (5432) failed
> May  7 16:09:34 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 16:09:34 ERROR: pid 29194: new_connection: create_cp() failed
> May  7 16:09:34 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 16:09:34 LOG:   pid 29194: notice_backend_error: 0 fail over request from pid 29194
> May  7 16:30:22 pgpool-PrepaidDB-HA1 heartbeat: [2425]: info: pgpool-prepaiddb-ha1 wants to go standby [all]
> 
> We also noticed some high load on the server which may be related to the issue.
> 
> Any idea what the the cause of this issue is? Do you recommend setting child_life_time = 0? I think there are some memory leak issues in the 3.0.4 versions.
> 3.0.4 is the latest official version in the Ubuntu repository and we would rather not go to self-compiled versions for reasons of maintenance and compliance.
> 
> Best regards,
> Ludwig
> 
> petaFuel GmbH
> Ludwig Adam
> Geschäftsführer
> 
> Münchner Strasse 4
> 85354 Freising
> 
> Tel.: +49 (0) 8161 4060 - 202               Mobil: +49 (0) 179-22 911 02
> Fax: +49 (0) 8161 4060 - 401               eMail: ludwig.adam at petafuel.de<mailto:ludwig.adam at petafuel.de>
> 
> Geschäftsführer: Ludwig Adam, Dr. Peter Schönweitz
> Amtsgericht München, HRB 133773
> www.petaFuel.de
>