[pgpool-general: 5750] Re: pg_terminate_backend -> postmaster on DB node 1 was shutdown by administrative command

[Alberto Rodríguez García]

Hello,

I think that my issue is related to Avi's issue.

I am running a multi data center setup with pgpool-II version 3.6.2
(subaruboshi) and postgresql 9.6.2

2 datacenter + 1 datacenter as witness.

2 pgsql per datacenter and 2 pgpool per data center sharing 1 VIP per
datacenter.

I am using pgpool,the streaming replication from postgresql and repmgr for
master failover and witness server.

Last friday I got this from different pgpool services in different
datacenters but using same master backend:

....
2017-09-22 18:26:40: pid 15131: LOG:  received degenerate backend request
for node_id: 2 from pid [15131]
2017-09-22 18:26:40: pid 7215: LOG:  watchdog received the failover command
from local pgpool-II on IPC interface
2017-09-22 18:26:40: pid 7215: LOG:  watchdog is processing the failover
command [DEGENERATE_BACKEND_REQUEST] received from local pgpool-II on IPC
interface
2017-09-22 18:26:40: pid 7215: LOG:  forwarding the failover request
[DEGENERATE_BACKEND_REQUEST] to all alive nodes
2017-09-22 18:26:40: pid 7215: DETAIL:  watchdog cluster currently has 1
connected remote nodes
2017-09-22 18:26:40: pid 7215: LOG:  new IPC connection received
2017-09-22 18:26:40: pid 7215: LOG:  watchdog received the failover command
from local pgpool-II on IPC interface
2017-09-22 18:26:40: pid 7215: LOG:  watchdog is processing the failover
command [DEGENERATE_BACKEND_REQUEST] received from local pgpool-II on IPC
interface
2017-09-22 18:26:40: pid 7215: LOG:  failover command
[DEGENERATE_BACKEND_REQUEST] from local pgpool-II on IPC interface is
ignored
....

The result of this was that pgpool removed the master from the load
balancer, leaving only the standby, so the setup was in "read-only".

I have attached part of the log file for more information and also I
include the pgpool.conf

What can I do to prevent this situation with pgpool?

Thank you

Kind regards.

Alberto Rodriguez.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20170925/c1666097/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pgpool_active_server_dc2.log
Type: text/x-log
Size: 24903 bytes
Desc: not available
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20170925/c1666097/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pgpool_server_dc2.conf
Type: application/octet-stream
Size: 51524 bytes
Desc: not available
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20170925/c1666097/attachment-0001.obj>