View Issue Details

IDProjectCategoryView StatusLast Update
0000388Pgpool-IIBugpublic2018-06-13 13:17
ReporterruianAssigned Tot-ishii 
PrioritynormalSeverityminorReproducibilityrandom
Status closedResolutionopen 
PlatformLinuxOSCentOSOS Version7.2.1511
Product Version3.6.1 
Target VersionFixed in Version 
Summary0000388: pgpool-II 3.6.1 child process remain waiting for a non-existing client socket to send queries
DescriptionHi everyone,

A few days ago, I created a simple tool to monitor the output of 'show pool_processes' command of my pgpool.

From the create_time field of the output, I find some pgpool child processes are occupied by my application for days. This is weird because all other connections of my application occupy the other child processes for just minutes.

Then I try to find out what happened to these child processes.
I use the `ps` tool to find out which client the pgpool child process connected to.
The output of ps likes:

root 7154 0.0 0.0 110664 6624 ? S 3月17 0:00 pgpool: mydbuser mydbname 10.2.145.151(54433) idle

It shows that the pid 7154 process is connected to 10.2.145.151(54433) and waiting for client to send queries.

However after I check my application cluster, I find that this client address is not exist anymore. I also use `ping 10.2.145.151` and `ss -a | grep 10.2.145.151` command on my pgool machine and they also indicate that there is no such host and no socket connected to such host.

This may means that the tcp keepalive of operation system works properly, however the pgpool child process are doing something else and do not notice that the client socket has already been closed and it doesn't accept new connection either.

I also check all the connections, which connected from the idle pgpool child process, on all backend postgres nodes, and they also state that they are waiting the pgpool child process to send queries.



Does anyone has any idea about this situation? Any suggestion to find out more detail? How to prevent or how do this happen?

Thank you very much.



I'am using pgpool-II version 3.6.1 with CentOS Linux release 7.2.1511.
The backend postgresql is 9.6.1

Here is some of my pgpool configurations:
num_init_children = 720
max_pool = 1
child_life_time = 300
child_max_connections = 60
connection_life_time = 0
client_idle_limit = 3600

Here is system tcpkeepalive setting:
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 9
TagsNo tags attached.

Activities

t-ishii

2018-05-07 15:40

developer   ~0002010

Pgpool-II 3.6.1 is very old. The latest 3.6.x version is 3.6.10. You might want to upgrade.

To track the problem, I need pgpool log for the process and/or gdb back trace.

t-ishii

2018-06-11 16:06

developer   ~0002043

No feedback over 1 month. I'm going to close this issue if no feedback is provided.

Issue History

Date Modified Username Field Change
2018-03-18 19:47 ruian New Issue
2018-04-06 11:06 t-ishii Assigned To => t-ishii
2018-04-06 11:06 t-ishii Status new => assigned
2018-05-07 15:40 t-ishii Note Added: 0002010
2018-05-07 15:40 t-ishii Status assigned => feedback
2018-06-11 16:06 t-ishii Note Added: 0002043
2018-06-13 13:17 t-ishii Status feedback => closed