[pgpool-general: 1050] Issue with watchdog after failover

[Tim Verhoeven]

Hi,

I've been setting up pgpool-II 3.2 together with Postgresql 9.2 on
CentOS 6. I've got the streaming replication and failover & recovery
working fine (using repmgr in the process). So the final step was to
setup 2 pgpool instances with the new watchdog functionality.

But I have a problem after the failover of pgpool. See the log below

Sep 27 16:11:56 test01 pgpool[17847]: wd_init: start watchdog
Sep 27 16:11:56 test01 pgpool[17847]: pgpool-II successfully started.
version 3.2.0 (namameboshi)
Sep 27 16:11:56 test01 pgpool[17847]: find_primary_node: primary node id is 0
Sep 27 16:11:57 test01 pgpool[17854]: watchdog: lifecheck started
Sep 27 16:14:17 test01 pgpool[17853]: wd_escalation: escalated to master pgpool
Sep 27 16:14:19 test01 pgpool[17853]: wd_escalation:  escaleted to
delegate_IP holder
Sep 27 16:14:48 test01 pgpool[17854]: wd_lifecheck: lifecheck failed 3
times. pgpool seems not to be working
Sep 27 16:15:00 test01 pgpool[17854]: wd_lifecheck: lifecheck failed 3
times. pgpool seems not to be working
Sep 27 16:15:12 test01 pgpool[17854]: wd_lifecheck: lifecheck failed 3
times. pgpool seems not to be working
Sep 27 16:15:24 test01 pgpool[17854]: wd_lifecheck: lifecheck failed 3
times. pgpool seems not to be working
Sep 27 16:15:33 test01 pgpool[17847]: received smart shutdown request

So this instance of pgpool start in backup mode and everything is
working fine (16:11). Then at 16:14 I shutdown the other pgpool
instance to test the failover, initially it works fine. This instance
activates the clustered IP and accepts connections. But then starting
at 16:14:48 the lifecheck fails for some strange reason and pgpool
blocks incoming connections. So I then shut it down.

What could be wrong here? What can I do to debug this problem?

Thanks,
Tim


-- 
Tim Verhoeven - tim.verhoeven.be at gmail.com - 0479 / 88 11 83

Hoping the problem  magically goes away  by ignoring it is the
"microsoft approach to programming" and should never be allowed.
(Linus Torvalds)