[pgpool-general: 6704] Re: pgpool 3.4 becomes unresponsive under load

[Pierre Timmermans]

Hi Kirstine
Something very confusing about pgpool is that when there is not enought child process available (i.e.  there is no connection in the pool available for a new request) it does not give an error, it just queue the connection request and there is nothing in the log. Furthermore the relation max_pool and num_init_children is not very clear and hard to understand unless you are in the easy case with a limited number of schema. 
So increasing num_init_children might indeed be the solution, especially if you see that your 40 child process are used.
Pierre 

    On Monday, September 9, 2019, 08:12:54 PM GMT+2, Travis Kirstine <tkirstine at firstbasesolutions.com> wrote:  
 
  
I  have pgpool 3.4 running in front of a pg 9.4 two node cluster running in streaming replication mode with connection pooling.  Under moderate load pgpool will become unresponsive and will need to be restarted.  The system is being used with a PHP / Python based web applications that can generate a number of concurrent selects / inserts (20 -100 queries per second).  The database seems to be able to handle the load however pgpool will fail under stress.  The database had 9 unique database / user combinations and I have  pgpool configured with:

 

num_init_children = 40
max_pool = 5
child_life_time = 300
child_max_connections = 0
connection_life_time = 0
client_idle_limit = 600




listen_backlog_multiplier = 10




Postgresql is configured with 250 max connections.  I could increase the num_init_children and max_pool however I'm to sure if this would resolve the issue.  The server supports up to 100,000 open files and appears to have around 15,000 open files when operational.





The issue I'm having is that it is not clear what could be causing the issue as there is no indication in the logs what is failing.





Any advise on how to trouble shoot this issue would be greatly appreciated 








_______________________________________________
pgpool-general mailing list
pgpool-general at pgpool.net
http://www.pgpool.net/mailman/listinfo/pgpool-general
  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20190909/dbb52052/attachment.html>