[pgpool-general: 2280] Strange watchdog - trusted server issue

[Sam Wouters]

Hi,

I have a running pgpool-II (3.3.1 on ubuntu-lts, package from postgresql 
repo) cluster, consisting of three nodes with watchdog enabled.
After a random period of time (a couple hours), watchdog goes into down 
state, with below log lines. This happenes consequently and on different 
clusters, no network issues that I know off (checked tcpdumps etc).
The log also says that ping to the trusted servers succeeds, but 
nevertheless you get the "failed to connect to any trusted servers"?

Any help in debugging this issue would be very much appreciated....

Sam

<LOG SNIPPET>
Nov 14 13:34:11 pgdb92-id03 pgpool[10107]: wd_hb_send: send 224 byte packet
Nov 14 13:34:11 pgdb92-id03 pgpool[10107]: wd_hb_sender: send heartbeat 
signal to 10.201.101.92:9694
Nov 14 13:34:11 pgdb92-id03 pgpool[10106]: wd_hb_recv: received 224 byte 
packet
Nov 14 13:34:11 pgdb92-id03 pgpool[10106]: wd_hb_receiver: received 
heartbeat signal from 10.201.101.92:5432
Nov 14 13:34:11 pgdb92-id03 pgpool[10108]: exec_ping: succeed to ping 
10.201.101.91
Nov 14 13:34:11 pgdb92-id03 pgpool[10108]: get_result: ping data: PING 
10.201.101.91 (10.201.101.91) 56(84) bytes of data.#012
Nov 14 13:34:11 pgdb92-id03 pgpool[10108]: exec_ping: succeed to ping 
10.201.101.92
Nov 14 13:34:11 pgdb92-id03 pgpool[10108]: get_result: ping data: PING 
10.201.101.92 (10.201.101.92) 56(84) bytes of data.#012#012--- 
10.201.101.92 ping statistics ---#0123 packets transmitted, 3 received, 
0% packet loss, time 2006ms#012rtt min/avg/max/mdev = 
0.000/0.000/0.002/0.001 ms
Nov 14 13:34:11 pgdb92-id03 pgpool[10108]: wd_lifecheck: failed to 
connect to any trusted servers
Nov 14 13:34:11 pgdb92-id03 pgpool[10108]: wd_IP_down: not delegate IP 
holder
</LOG SNIPPET>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20131114/3de8edab/attachment.html>