View Issue Details

IDProjectCategoryView StatusLast Update
0000387pgpool-HABugpublic2018-04-06 11:07
Reporterruian Assigned To 
PrioritynormalSeverityminorReproducibilityrandom
Status closedResolutionopen 
PlatformLinuxOSCentOSOS Version7.2.1511
Summary0000387: pgpool-II 3.6.1 child process remain waiting for a non-existing client socket to send queries
DescriptionHi everyone,

A few days ago, I created a simple tool to monitor the output of 'show pool_processes' command of my pgpool.

From the create_time field of the output, I find some pgpool child processes are occupied by my application for days. This is weird because all other connections of my application occupy the other child processes for just minutes.

Then I try to find out what happened to these child processes.
I use the `ps` tool to find out which client the pgpool child process connected to.
The output of ps likes:

root 7154 0.0 0.0 110664 6624 ? S 3月17 0:00 pgpool: mydbuser mydbname 10.2.145.151(54433) idle

It shows that the pid 7154 process is connected to 10.2.145.151(54433) and waiting for client to send queries.

However after I check my application cluster, I find that this client address is not exist anymore. I also use `ping 10.2.145.151` and `ss -a | grep 10.2.145.151` command on my pgool machine and they also indicate that there is no such host and no socket connected to such host.

This may means that the tcp keepalive of operation system works properly, however the pgpool child process are doing something else and do not notice that the client socket has already been closed and it doesn't accept new connection either.

I also check all the connections, which connected from the idle pgpool child process, on all backend postgres nodes, and they also state that they are waiting the pgpool child process to send queries.



Does anyone has any idea about this situation? Any suggestion to find out more detail? How to prevent or how do this happen?

Thank you very much.



I'am using pgpool-II version 3.6.1 with CentOS Linux release 7.2.1511.
The backend postgresql is 9.6.1

Here is some of my pgpool configurations:
num_init_children = 720
max_pool = 1
child_life_time = 300
child_max_connections = 60
connection_life_time = 0
client_idle_limit = 3600

Here is system tcpkeepalive setting:
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 9
TagsNo tags attached.

Activities

ruian

2018-03-18 19:50

reporter   ~0001971

Sorry for reporting to the wrong place. This should be related to pgpool-ll.

t-ishii

2018-04-06 11:07

developer   ~0001991

Issue moved to 388. So closed.

Issue History

Date Modified Username Field Change
2018-03-18 19:31 ruian New Issue
2018-03-18 19:50 ruian Note Added: 0001971
2018-04-06 11:07 t-ishii Note Added: 0001991
2018-04-06 11:07 t-ishii Status new => closed