View Issue Details
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0000388||Pgpool-II||Bug||public||2018-03-18 19:47||2018-06-13 13:17|
|Target Version||Fixed in Version|
|Summary||0000388: pgpool-II 3.6.1 child process remain waiting for a non-existing client socket to send queries|
A few days ago, I created a simple tool to monitor the output of 'show pool_processes' command of my pgpool.
From the create_time field of the output, I find some pgpool child processes are occupied by my application for days. This is weird because all other connections of my application occupy the other child processes for just minutes.
Then I try to find out what happened to these child processes.
I use the `ps` tool to find out which client the pgpool child process connected to.
The output of ps likes:
root 7154 0.0 0.0 110664 6624 ? S 3月17 0:00 pgpool: mydbuser mydbname 10.2.145.151(54433) idle
It shows that the pid 7154 process is connected to 10.2.145.151(54433) and waiting for client to send queries.
However after I check my application cluster, I find that this client address is not exist anymore. I also use `ping 10.2.145.151` and `ss -a | grep 10.2.145.151` command on my pgool machine and they also indicate that there is no such host and no socket connected to such host.
This may means that the tcp keepalive of operation system works properly, however the pgpool child process are doing something else and do not notice that the client socket has already been closed and it doesn't accept new connection either.
I also check all the connections, which connected from the idle pgpool child process, on all backend postgres nodes, and they also state that they are waiting the pgpool child process to send queries.
Does anyone has any idea about this situation? Any suggestion to find out more detail? How to prevent or how do this happen?
Thank you very much.
I'am using pgpool-II version 3.6.1 with CentOS Linux release 7.2.1511.
The backend postgresql is 9.6.1
Here is some of my pgpool configurations:
num_init_children = 720
max_pool = 1
child_life_time = 300
child_max_connections = 60
connection_life_time = 0
client_idle_limit = 3600
Here is system tcpkeepalive setting:
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 9
|Tags||No tags attached.|
Pgpool-II 3.6.1 is very old. The latest 3.6.x version is 3.6.10. You might want to upgrade.
To track the problem, I need pgpool log for the process and/or gdb back trace.
||No feedback over 1 month. I'm going to close this issue if no feedback is provided.|
|2018-03-18 19:47||ruian||New Issue|
|2018-04-06 11:06||t-ishii||Assigned To||=> t-ishii|
|2018-04-06 11:06||t-ishii||Status||new => assigned|
|2018-05-07 15:40||t-ishii||Note Added: 0002010|
|2018-05-07 15:40||t-ishii||Status||assigned => feedback|
|2018-06-11 16:06||t-ishii||Note Added: 0002043|
|2018-06-13 13:17||t-ishii||Status||feedback => closed|