[pgpool-general: 427] Re: FW: Re: Re: pg-pool-3.0.4. -unable to create child processes

Ludwig Adam ludwig.adam at petafuel.de
Wed May 9 16:44:18 JST 2012


Dear Tatsuo,

sorry, meant no offense- I was referring to the 227 GB.
Let's see what other members of this list have experienced.
Anyway,  do you have suggestions if and how we can work around the semaphore locking bug in version <3.0.7?

Thanks, 
Ludwig

petaFuel GmbH
Ludwig Adam
Geschäftsführer

Geschäftsführer: Ludwig Adam, Dr. Peter Schönweitz
Amtsgericht München, HRB 133773

Tel.: +49 (0) 8161 4060 - 202               Mobil: +49 (0) 179-22 911 02

> -----Ursprüngliche Nachricht-----
> Von: Tatsuo Ishii [mailto:ishii at postgresql.org]
> Gesendet: Dienstag, 8. Mai 2012 23:24
> An: Ludwig Adam
> Cc: pgpool-general at pgpool.net
> Betreff: Re: [pgpool-general: 425] FW: Re: Re: pg-pool-3.0.4. -unable to
> create child processes
> 
> That was *not* an estimation. That was from the output of "cat /proc/process-
> id/status". Of course it might be a matter of way to translate the number. I
> would like to hear opinions from Linux experts...
> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> English: http://www.sraoss.co.jp/index_en.php
> Japanese: http://www.sraoss.co.jp
> 
> > Dear Tatsuo,
> >
> > this seems to be a very high estimation. In our environment one child has
> max pool size of 1, i . e. It manages one connection to the backend. Memory
> usage of a idle process according to ps aux says 0.1 percent per child, which
> translates to roughly 1 meg if I am not mistaken...
> >
> > Thanks,
> > Ludwig
> >
> > Mobil gesendet.
> >
> > -----Original Message-----
> > From: Tatsuo Ishii [ishii at postgresql.org]
> > Received: Dienstag, 08 Mai 2012, 17:37
> > To: Ludwig Adam [ludwig.adam at petafuel.de]
> > CC: pgpool-general at pgpool.net [pgpool-general at pgpool.net]
> > Subject: Re: [pgpool-general: 419] Re: pg-pool-3.0.4. -unable to
> > create child processes
> >
> > On my x86_64 linux machine, a pgpool process's VMSize is
> > 56692+(68*num_init_children) (unit is kb). num_init_children = 1500
> > case, it will be 158692 kb. So total VMSize is 158692 * 1500 =
> > 238038000, which is about 227GB. I'm not sure how much of 227GB is
> > actually required since some of memory is shared among multiple
> > process, but I guess 1GB mem is too small.
> > --
> > Tatsuo Ishii
> > SRA OSS, Inc. Japan
> > English: http://www.sraoss.co.jp/index_en.php
> > Japanese: http://www.sraoss.co.jp
> >
> >> Dear list,
> >>
> >> just to provide you an update on the systems spec, as I have seen that the
> failed fork may be limited to some memory constraints / limits:
> >>
> >> We have set the num_init_childs to 1500 (from which ~500 are connected
> at low system load and around 900 at peak times currently).
> >> The allocated RAM for the machine is 1 GB- what's your recommendation
> for the relation of allocated RAM and # of pgpool childs?
> >>
> >> I would suspect a memory leak on the connected, non-idle processes (as
> we are still running 3.04) but the kind of error message does not make really
> sense- why should just a fork not be possible and also we had no child
> processes at all running.
> >>
> >>
> >> Ludwig
> >>
> >>
> >> petaFuel GmbH
> >> Ludwig Adam
> >> Geschäftsführer
> >>
> >> Geschäftsführer: Ludwig Adam, Dr. Peter Schönweitz Amtsgericht
> >> München, HRB 133773
> >>
> >> Tel.: +49 (0) 8161 4060 - 202               Mobil: +49 (0) 179-22 911 02
> >>
> >> Von: Ludwig Adam
> >> Gesendet: Montag, 7. Mai 2012 17:15
> >> An: pgpool-general at pgpool.net
> >> Cc: EDV
> >> Betreff: pg-pool-3.0.4. -unable to create child processes
> >>
> >> Dear list,
> >>
> >> we just encountered a production issue with pg-pool-3.0.4.: pg-pool killed
> all idle childprocesses which were waiting for a connection request and failed
> to create new ones.
> >>
> >> I am unsure why the childs exited:
> >>
> >>
> >> 1)    As we have child_life_time = 300 I guess the childs exited normally
> after a 5 minute period.
> >>
> >> 2)    We did a Config reload on pg_pool, and that could mean also that
> waiting child processes are killed
> >>
> >> but pg_pool did not create new ones. So we were left with ~110 active pg-
> pool processes (which still seemed to work) but we could make no new
> connections to pgppol (can't tell if the connections have been queued or not).
> >> This is what the syslog says:
> >>
> >> May  7 15:43:13 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 15:43:13
> LOG:   pid 938: reload config files.
> >> May  7 15:43:13 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 15:43:13
> LOG:   pid 2707: reload config files.
> >> May  7 16:09:29 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 16:09:17
> ERROR: pid 938: fork() failed. reason: Resource temporarily unavailable
> >> May  7 16:09:34 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 16:09:34
> LOG:   pid 29194: connect_inet_domain_socket_by_port: exit request has
> been sent
> >> May  7 16:09:34 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 16:09:34
> >> ERROR: pid 29194: connection to <backendserver ha1 here> (5432) failed
> May  7 16:09:34 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 16:09:34
> ERROR: pid 29194: new_connection: create_cp() failed
> >> May  7 16:09:34 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 16:09:34
> LOG:   pid 29194: notice_backend_error: 0 fail over request from pid 29194
> >> May  7 16:30:22 pgpool-PrepaidDB-HA1 heartbeat: [2425]: info:
> >> pgpool-prepaiddb-ha1 wants to go standby [all]
> >>
> >> We also noticed some high load on the server which may be related to the
> issue.
> >>
> >> Any idea what the the cause of this issue is? Do you recommend setting
> child_life_time = 0? I think there are some memory leak issues in the 3.0.4
> versions.
> >> 3.0.4 is the latest official version in the Ubuntu repository and we would
> rather not go to self-compiled versions for reasons of maintenance and
> compliance.
> >>
> >> Best regards,
> >> Ludwig
> >>
> >> petaFuel GmbH
> >> Ludwig Adam
> >> Geschäftsführer
> >>
> >> Münchner Strasse 4
> >> 85354 Freising
> >>
> >> Tel.: +49 (0) 8161 4060 - 202               Mobil: +49 (0) 179-22 911 02
> >> Fax: +49 (0) 8161 4060 - 401               eMail:
> ludwig.adam at petafuel.de<mailto:ludwig.adam at petafuel.de>
> >>
> >> Geschäftsführer: Ludwig Adam, Dr. Peter Schönweitz Amtsgericht
> >> München, HRB 133773 www.petaFuel.de<http://www.petaFuel.de>
> >>


More information about the pgpool-general mailing list