[pgpool-general: 5772] Re: Pgpool on CentOS7 refuse to come up

Lucas Luengas lucasluengas at gmail.com
Fri Oct 13 15:58:30 JST 2017


Hello.

You can use pgpool log file on Centos 7 using syslog.

In pgpool.conf, edit line with log_destination parameter:

log_destination = 'syslog'

In /etc/rsyslog.conf in section [RULES] add this line:

local0.*                                /var/log/pgpool.log

Alson, in /etc/rsyslog.conf add ";local0.none" to line:

*.info;mail.none;authpriv.none;cron.none;local0.none
 /var/log/messages

Then, restart pgpool service and rsyslog service.

systemctl restart pgpool
systemctl restart rsyslog

Regards.

On Tue, Apr 18, 2017 at 5:43 PM, Avi Weinberg <AviW at gilat.com> wrote:

> Hi all,
>
>
>
> We upgraded to CentOS7 and pgpool with the configuration that works on 6.8
> refuse to come up.  Your input is most welcome.
>
>
>
> In addition, is it possible to change pgpool configuration so even with
> CentOS7 we will have pgpool log file in addition or instead of the sys
> messages.
>
>
>
> Thanks
>
> Avi
>
>
>
>
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19154
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19155
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19156
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19157
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19158
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19159
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19161
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19162
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19163
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19164
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19165
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19166
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19167
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19169
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19170
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19171
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19172
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19173
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19174
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> LOG:  execute command: /etc/pgpool/failover.sh 1 172.18.255.42 5432
> /postgres_db/data -1 1 "" 1 "" "" > AAA_failover_called.log
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  new IPC connection received
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to release [FAILOVER] lock for failover ID 0
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has released the [FAILOVER] lock for failover ID 0
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> LOG:  failover: no follow backends are degenerated
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  new IPC connection received
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to release [FOLLOW MASTER] lock for failover ID 0
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has released the [FOLLOW MASTER] lock for failover ID 0
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> LOG:  failover: set new primary node: -1
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19183:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19184:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19185:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19186:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19187:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19188:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19189:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19190:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19191:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19192:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19193:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19194:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19195:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19196:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19197:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19198:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19199:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19200:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19201:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19202:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19203:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19204:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19205:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19206:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19176:
> LOG:  worker process received restart request
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  new IPC connection received
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to resign from a lock holder for failover ID 0
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has resigned from the lock holder
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19207:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: failover done. shutdown host
> 172.18.255.42(5432)2017-04-18 15:55:29: pid 19123: LOG:  failover done.
> shutdown host 172.18.255.42(5432)
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19144:
> LOG:  set SO_REUSEPORT option to the socket
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19144:
> LOG:  creating watchdog heartbeat receive socket.
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19144:
> DETAIL:  set SO_REUSEPORT
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19145:
> LOG:  set SO_REUSEPORT option to the socket
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19145:
> LOG:  creating socket for sending heartbeat
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19145:
> DETAIL:  set SO_REUSEPORT
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19175:
> LOG:  restart request received in pcp child process
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19175:
> DEBUG:  PCP child receives shutdown request signal 15, Forwarding to all
> children
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19175:
> DEBUG:  PCP child receives smart shutdown request
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  PCP child 19175 exits with status 0 in failover()
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  fork a new PCP child pid 19208 in failover()
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19208:
> DEBUG:  I am PCP child with pid:19208
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  reaper handler
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19148 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19148 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19148 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19149 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19149 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19149 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19150 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19150 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19150 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19151 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19151 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19151 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19152 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19152 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19152 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19153 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19153 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19153 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19154 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19154 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19154 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19155 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19155 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19155 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19156 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19156 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19156 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19157 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19157 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19157 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19158 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19158 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19158 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19159 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19159 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19159 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19161 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19161 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19161 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19162 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19162 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19162 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19163 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19163 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19163 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19164 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19164 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19164 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19165 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19165 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19165 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19166 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19166 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19166 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19167 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19167 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19167 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19169 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19169 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19169 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19170 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19170 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19170 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19171 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19171 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19171 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19172 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19172 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19172 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19173 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19173 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19173 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19174 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19174 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19174 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  worker child process with pid: 19176 exits with status 256
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  fork a new worker child process with pid: 19209
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  reaper handler: exiting normally
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19209:
> DEBUG:  I am 19209
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19209:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:31 postgres_server1 ntpd[2889]: Listen normally on 6
> bond1.18:0 172.18.255.40 UDP 123
>
> Apr 18 15:55:31 postgres_server1 pgpool: 2017-04-18 15:55:31: pid 19139:
> DEBUG:  watchdog exec interface up/down command: 'sudo arping -U $_IP_$ -w
> 1 -I bond1.18' succeeded
>
> Apr 18 15:55:32 postgres_server1 pgpool: 2017-04-18 15:55:32: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19139:
> DEBUG:  watchdog ping process for host "172.18.255.40" exited successfully
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19139:
> DEBUG:  watchdog ping
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19139:
> DETAIL:  ping data: PING 172.18.255.40 (172.18.255.40) 56(84) bytes of data.
>
> Apr 18 15:55:33 postgres_server1 pgpool: --- 172.18.255.40 ping statistics
> ---
>
> Apr 18 15:55:33 postgres_server1 pgpool: 3 packets transmitted, 3
> received, 0% packet loss, time 1998ms
>
> Apr 18 15:55:33 postgres_server1 pgpool: rtt min/avg/max/mdev =
> 0.024/0.026/0.030/0.006 ms
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19139:
> DEBUG:  watchdog succeeded to ping a host "172.18.255.40"
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19139:
> LOG:  successfully acquired the delegate IP:"172.18.255.40"
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19139:
> DETAIL:  'if_up_cmd' returned with success
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19124:
> DEBUG:  watchdog process signal handler
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19124:
> LOG:  watchdog escalation process with pid: 19139 exit with SUCCESS.
>
> Apr 18 15:55:34 postgres_server1 pgpool: 2017-04-18 15:55:34: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:36 postgres_server1 pgpool: 2017-04-18 15:55:36: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:36 postgres_server1 pgpool: 2017-04-18 15:55:36: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:55:36 postgres_server1 pgpool: 2017-04-18 15:55:36: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:55:36 postgres_server1 pgpool: 2017-04-18 15:55:36: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:55:38 postgres_server1 pgpool: 2017-04-18 15:55:38: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:41 postgres_server1 pgpool: 2017-04-18 15:55:41: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:55:42 postgres_server1 pgpool: 2017-04-18 15:55:42: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:44 postgres_server1 pgpool: 2017-04-18 15:55:44: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:46 postgres_server1 pgpool: 2017-04-18 15:55:46: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:46 postgres_server1 pgpool: 2017-04-18 15:55:46: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:55:46 postgres_server1 pgpool: 2017-04-18 15:55:46: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:55:46 postgres_server1 pgpool: 2017-04-18 15:55:46: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:55:48 postgres_server1 pgpool: 2017-04-18 15:55:48: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:51 postgres_server1 pgpool: 2017-04-18 15:55:51: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:55:52 postgres_server1 pgpool: 2017-04-18 15:55:52: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:54 postgres_server1 pgpool: 2017-04-18 15:55:54: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:56 postgres_server1 pgpool: 2017-04-18 15:55:56: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:56 postgres_server1 pgpool: 2017-04-18 15:55:56: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:55:56 postgres_server1 pgpool: 2017-04-18 15:55:56: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:55:56 postgres_server1 pgpool: 2017-04-18 15:55:56: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:55:58 postgres_server1 pgpool: 2017-04-18 15:55:58: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:01 postgres_server1 pgpool: 2017-04-18 15:56:01: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:56:02 postgres_server1 pgpool: 2017-04-18 15:56:02: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:04 postgres_server1 pgpool: 2017-04-18 15:56:04: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:06 postgres_server1 pgpool: 2017-04-18 15:56:06: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:06 postgres_server1 pgpool: 2017-04-18 15:56:06: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:56:06 postgres_server1 pgpool: 2017-04-18 15:56:06: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:56:06 postgres_server1 pgpool: 2017-04-18 15:56:06: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:56:08 postgres_server1 pgpool: 2017-04-18 15:56:08: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:11 postgres_server1 pgpool: 2017-04-18 15:56:11: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:56:12 postgres_server1 pgpool: 2017-04-18 15:56:12: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:14 postgres_server1 pgpool: 2017-04-18 15:56:14: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:16 postgres_server1 pgpool: 2017-04-18 15:56:16: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:16 postgres_server1 pgpool: 2017-04-18 15:56:16: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:56:16 postgres_server1 pgpool: 2017-04-18 15:56:16: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:56:16 postgres_server1 pgpool: 2017-04-18 15:56:16: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:56:18 postgres_server1 pgpool: 2017-04-18 15:56:18: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:21 postgres_server1 pgpool: 2017-04-18 15:56:21: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:56:22 postgres_server1 pgpool: 2017-04-18 15:56:22: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:24 postgres_server1 pgpool: 2017-04-18 15:56:24: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:26 postgres_server1 pgpool: 2017-04-18 15:56:26: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:26 postgres_server1 pgpool: 2017-04-18 15:56:26: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:56:26 postgres_server1 pgpool: 2017-04-18 15:56:26: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:56:26 postgres_server1 pgpool: 2017-04-18 15:56:26: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:56:28 postgres_server1 pgpool: 2017-04-18 15:56:28: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:31 postgres_server1 pgpool: 2017-04-18 15:56:31: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:56:32 postgres_server1 pgpool: 2017-04-18 15:56:32: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:34 postgres_server1 pgpool: 2017-04-18 15:56:34: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:36 postgres_server1 pgpool: 2017-04-18 15:56:36: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:36 postgres_server1 pgpool: 2017-04-18 15:56:36: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:56:36 postgres_server1 pgpool: 2017-04-18 15:56:36: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:56:36 postgres_server1 pgpool: 2017-04-18 15:56:36: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:56:38 postgres_server1 pgpool: 2017-04-18 15:56:38: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:41 postgres_server1 pgpool: 2017-04-18 15:56:41: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:56:42 postgres_server1 pgpool: 2017-04-18 15:56:42: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:44 postgres_server1 pgpool: 2017-04-18 15:56:44: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:45 postgres_server1 systemd: pgpool.service start operation
> timed out. Terminating.
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19123:
> LOG:  received smart shutdown request
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19123:
> LOG:  shutdown request. closing listen socket
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19144:
> DEBUG:  watchdog heartbeat receiver child receives shutdown request signal
> 15
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19140:
> DEBUG:  lifecheck child receives shutdown request signal 15, forwarding to
> all children
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19124:
> LOG:  Watchdog is shutting down
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19145:
> DEBUG:  watchdog heartbeat sender child receives shutdown request signal 15
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19140:
> DEBUG:  lifecheck child receives smart shutdown request
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19311:
> LOG:  watchdog: de-escalation started
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19140:
> DEBUG:  lifecheck child receives shutdown request signal 15, forwarding to
> all children
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19208:
> DEBUG:  PCP child receives shutdown request signal 15, Forwarding to all
> children
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19140:
> DEBUG:  lifecheck child receives smart shutdown request
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19208:
> DEBUG:  PCP child receives smart shutdown request
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19208:
> DEBUG:  PCP child receives shutdown request signal 15, Forwarding to all
> children
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19208:
> DEBUG:  PCP child receives smart shutdown request
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19123:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 15:56:45 postgres_server1 systemd: Failed to start Pgpool-II.
>
> Apr 18 15:56:45 postgres_server1 systemd: Unit pgpool.service entered
> failed state.
>
> Apr 18 15:56:45 postgres_server1 systemd: pgpool.service failed.
>
> Apr 18 16:00:01 postgres_server1 systemd: Started Session 13 of user root.
>
> Apr 18 16:00:01 postgres_server1 systemd: Starting Session 13 of user root.
>
> Apr 18 16:01:01 postgres_server1 systemd: Started Session 14 of user root.
>
> Apr 18 16:01:01 postgres_server1 systemd: Starting Session 14 of user root.
>
> Apr 18 16:03:57 postgres_server1 systemd: Starting Pgpool-II...
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> DEBUG:  loading hba configuration
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> DETAIL:  loading file :"/etc/pgpool-II/pool_hba.conf" for client
> authentication configuration file
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> LOG:  reading status file: 0 th backend is set to down status
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> DEBUG:  pool_coninfo_size: num_init_children (25) * max_pool (4) *
> MAX_NUM_BACKENDS (128) * sizeof(ConnectionInfo) (136) = 1740800 bytes
> requested for shared memory
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> DEBUG:  ProcessInfo: num_init_children (25) * sizeof(ProcessInfo) (32) =
> 800 bytes requested for shared memory
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> DEBUG:  Request info are: sizeof(POOL_REQUEST_INFO) 5304 bytes requested
> for shared memory
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> DEBUG:  Recovery management area: sizeof(int) 4 bytes requested for shared
> memory
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> LOG:  waiting for watchdog to initialize
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> LOG:  setting the local watchdog node name to "172.18.255.41:9999 Linux
> postgres_server1"
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> LOG:  watchdog cluster is configured with 1 remote nodes
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> LOG:  watchdog remote node:0 on 172.18.255.42:9000
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> LOG:  interface monitoring is disabled in watchdog
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> INFO:  IPC socket path: "/tmp/.s.PGPOOLWD_CMD.9000"
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> LOG:  watchdog node state changed from [DEAD] to [LOADING]
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> LOADING
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> DEBUG:  error in outbond connection to 172.18.255.42:9000
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> DETAIL:  Connection refused
>
> Apr 18 16:04:02 postgres_server1 pgpool: 2017-04-18 16:04:02: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = LOADING
>
> Apr 18 16:04:02 postgres_server1 pgpool: 2017-04-18 16:04:02: pid 19479:
> LOG:  watchdog node state changed from [LOADING] to [JOINING]
>
> Apr 18 16:04:02 postgres_server1 pgpool: 2017-04-18 16:04:02: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> JOINING
>
> Apr 18 16:04:02 postgres_server1 pgpool: 2017-04-18 16:04:02: pid 19479:
> DEBUG:  error in outbond connection to 172.18.255.42:9000
>
> Apr 18 16:04:02 postgres_server1 pgpool: 2017-04-18 16:04:02: pid 19479:
> DETAIL:  Connection refused
>
> Apr 18 16:04:06 postgres_server1 pgpool: 2017-04-18 16:04:06: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> JOINING
>
> Apr 18 16:04:06 postgres_server1 pgpool: 2017-04-18 16:04:06: pid 19479:
> LOG:  watchdog node state changed from [JOINING] to [INITIALIZING]
>
> Apr 18 16:04:06 postgres_server1 pgpool: 2017-04-18 16:04:06: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> INITIALIZING
>
> Apr 18 16:04:07 postgres_server1 pgpool: 2017-04-18 16:04:07: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State =
> INITIALIZING
>
> Apr 18 16:04:07 postgres_server1 pgpool: 2017-04-18 16:04:07: pid 19479:
> LOG:  I am the only alive node in the watchdog cluster
>
> Apr 18 16:04:07 postgres_server1 pgpool: 2017-04-18 16:04:07: pid 19479:
> HINT:  skiping stand for coordinator state
>
> Apr 18 16:04:07 postgres_server1 pgpool: 2017-04-18 16:04:07: pid 19479:
> LOG:  watchdog node state changed from [INITIALIZING] to [MASTER]
>
> Apr 18 16:04:07 postgres_server1 pgpool: 2017-04-18 16:04:07: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> MASTER
>
> Apr 18 16:04:07 postgres_server1 pgpool: 2017-04-18 16:04:07: pid 19479:
> LOG:  I am announcing my self as master/coordinator watchdog node
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  declare coordinator command finished with status:[COMMAND TIMEED
> OUT]
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  I am the cluster leader node
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DETAIL:  our declare coordinator message is accepted by all nodes
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  I am the cluster leader node. Starting escalation process
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  watchdog process is initialized
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  escalation process started with PID:19481
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  new IPC connection received
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  received the get data request from local pgpool-II on IPC interface
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> NOTICE:  error writing to IPC socket
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19481:
> LOG:  watchdog: escalation started
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> DEBUG:  I am watchdog lifecheck child with pid:19482
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  1:0: Unexpected
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  I am the master watchdog node
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  using the local backend node status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  new IPC connection received
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:549
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> LOG:  2 watchdog nodes are configured for lifecheck
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> LOG:  watchdog nodes ID:0 Name:"172.18.255.41:9999 Linux postgres_server1"
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> DETAIL:  Host:"172.18.255.41" WD Port:9000 pgpool-II port:9999
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> LOG:  watchdog nodes ID:1 Name:"Not_Set"
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> DETAIL:  Host:"172.18.255.42" WD Port:9000 pgpool-II port:9999
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> DEBUG:  watchdog checking life check is ready
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> DETAIL:  pgpool:1 at "172.18.255.42:9999" has not send the heartbeat
> signal yet
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  Setting up socket for 0.0.0.0:9999
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  Setting up socket for :::9999
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  perhaps failed to create INET domain socket
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  socket(::) failed: "Address family not supported by protocol"
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19490:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19491:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19492:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19493:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19494:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19495:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19496:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19497:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19498:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19499:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19500:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19501:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19502:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19503:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19504:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19505:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19506:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19508:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19509:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19510:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19511:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19512:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19513:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19514:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19515:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19481:
> DEBUG:  watchdog exec interface up/down command: 'sudo ifconfig bond1.18:0
> inet $_IP_$ netmask 255.255.255.0' succeeded
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19516:
> DEBUG:  I am PCP child with pid:19516
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  pgpool-II successfully started. version 3.6.2 (subaruboshi)
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  find_primary_node: checking backend no 0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  find_primary_node: checking backend no 1
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19517:
> DEBUG:  I am 19517
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19517:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  find_primary_node: no primary node found
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19517:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19517:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19517:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  Backend DB node 1 status is 2
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  Trying to make persistent DB connection to backend node 1 having
> status 2
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  doing health check against database:template1 user:postgres
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  Backend DB node 1 status is 2
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  Trying to make persistent DB connection to backend node 1 having
> status 2
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  setting backend node 1 status to NODE DOWN
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  received degenerate backend request for node_id: 1 from pid [19478]
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  new IPC connection received
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  watchdog received the failover command from local pgpool-II on IPC
> interface
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  watchdog is processing the failover command
> [DEGENERATE_BACKEND_REQUEST] received from local pgpool-II on IPC interface
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  I am the only pgpool-II node in the watchdog cluster
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DETAIL:  no need to propagate the failover command
> [DEGENERATE_BACKEND_REQUEST]
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler called
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kind: 1 flags: 1 node_count: 1 index:0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  new IPC connection received
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to become a lock holder for failover ID: 0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> the lock holder
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  starting to select new master node
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  starting degeneration. shutdown host 172.18.255.42(5432)
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  failover: no valid backends node found
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover/failback request details: STREAM: 1 reqkind: 1 detail: 1
> node_id: 1
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  Restart all children
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19490
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19491
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19492
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19493
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19494
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19495
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19496
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19497
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19498
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19499
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19500
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19501
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19502
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19503
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19504
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19505
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19506
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19508
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19509
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19510
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19511
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19512
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19513
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19514
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19515
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  execute command: /etc/pgpool/failover.sh 1 172.18.255.42 5432
> /postgres_db/data -1 1 "" 1 "" "" > AAA_failover_called.log
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  new IPC connection received
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to release [FAILOVER] lock for failover ID 0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has released the [FAILOVER] lock for failover ID 0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  failover: no follow backends are degenerated
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  new IPC connection received
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to release [FOLLOW MASTER] lock for failover ID 0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has released the [FOLLOW MASTER] lock for failover ID 0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  failover: set new primary node: -1
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19525:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19526:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19527:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19528:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19529:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19530:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19531:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19532:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19533:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19534:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19535:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19536:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19537:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19538:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19539:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19540:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19541:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19542:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19543:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19544:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19545:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19546:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19547:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19548:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  new IPC connection received
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19517:
> LOG:  worker process received restart request
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to resign from a lock holder for failover ID 0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has resigned from the lock holder
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:04:11 postgres_server1 pgpool: failover done. shutdown host
> 172.18.255.42(5432)2017-04-18 16:04:11: pid 19549: DEBUG:  initializing
> backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  failover done. shutdown host 172.18.255.42(5432)
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19486:
> LOG:  set SO_REUSEPORT option to the socket
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19486:
> LOG:  creating watchdog heartbeat receive socket.
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19486:
> DETAIL:  set SO_REUSEPORT
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19487:
> LOG:  set SO_REUSEPORT option to the socket
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19487:
> LOG:  creating socket for sending heartbeat
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19487:
> DETAIL:  set SO_REUSEPORT
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19516:
> LOG:  restart request received in pcp child process
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19516:
> DEBUG:  PCP child receives shutdown request signal 15, Forwarding to all
> children
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19516:
> DEBUG:  PCP child receives smart shutdown request
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  PCP child 19516 exits with status 0 in failover()
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  fork a new PCP child pid 19550 in failover()
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19550:
> DEBUG:  I am PCP child with pid:19550
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  reaper handler
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19490 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19490 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19490 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19491 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19491 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19491 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19492 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19492 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19492 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19493 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19493 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19493 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19494 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19494 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19494 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19495 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19495 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19495 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19496 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19496 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19496 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19497 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19497 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19497 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19498 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19498 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19498 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19499 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19499 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19499 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19500 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19500 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19500 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19501 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19501 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19501 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19502 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19502 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19502 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19503 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19503 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19503 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19504 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19504 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19504 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19505 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19505 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19505 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19506 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19506 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19506 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19508 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19508 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19508 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19509 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19509 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19509 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19510 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19510 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19510 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19511 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19511 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19511 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19512 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19512 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19512 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19513 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19513 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19513 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19514 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19514 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19514 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19515 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19515 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19515 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  worker child process with pid: 19517 exits with status 256
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  fork a new worker child process with pid: 19551
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  reaper handler: exiting normally
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19551:
> DEBUG:  I am 19551
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19551:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:13 postgres_server1 pgpool: 2017-04-18 16:04:13: pid 19481:
> DEBUG:  watchdog exec interface up/down command: 'sudo arping -U $_IP_$ -w
> 1 -I bond1.18' succeeded
>
> Apr 18 16:04:14 postgres_server1 pgpool: 2017-04-18 16:04:14: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19481:
> DEBUG:  watchdog ping process for host "172.18.255.40" exited successfully
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19481:
> DEBUG:  watchdog ping
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19481:
> DETAIL:  ping data: PING 172.18.255.40 (172.18.255.40) 56(84) bytes of data.
>
> Apr 18 16:04:15 postgres_server1 pgpool: --- 172.18.255.40 ping statistics
> ---
>
> Apr 18 16:04:15 postgres_server1 pgpool: 3 packets transmitted, 3
> received, 0% packet loss, time 2000ms
>
> Apr 18 16:04:15 postgres_server1 pgpool: rtt min/avg/max/mdev =
> 0.026/0.029/0.035/0.006 ms
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19481:
> DEBUG:  watchdog succeeded to ping a host "172.18.255.40"
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19481:
> LOG:  successfully acquired the delegate IP:"172.18.255.40"
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19481:
> DETAIL:  'if_up_cmd' returned with success
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19479:
> DEBUG:  watchdog process signal handler
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19479:
> LOG:  watchdog escalation process with pid: 19481 exit with SUCCESS.
>
> Apr 18 16:04:16 postgres_server1 pgpool: 2017-04-18 16:04:16: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:18 postgres_server1 pgpool: 2017-04-18 16:04:18: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:18 postgres_server1 pgpool: 2017-04-18 16:04:18: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:04:18 postgres_server1 pgpool: 2017-04-18 16:04:18: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:04:18 postgres_server1 pgpool: 2017-04-18 16:04:18: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:04:20 postgres_server1 pgpool: 2017-04-18 16:04:20: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:23 postgres_server1 pgpool: 2017-04-18 16:04:23: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:04:24 postgres_server1 pgpool: 2017-04-18 16:04:24: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:26 postgres_server1 pgpool: 2017-04-18 16:04:26: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:28 postgres_server1 pgpool: 2017-04-18 16:04:28: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:28 postgres_server1 pgpool: 2017-04-18 16:04:28: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:04:28 postgres_server1 pgpool: 2017-04-18 16:04:28: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:04:28 postgres_server1 pgpool: 2017-04-18 16:04:28: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:04:30 postgres_server1 pgpool: 2017-04-18 16:04:30: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:33 postgres_server1 pgpool: 2017-04-18 16:04:33: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:04:34 postgres_server1 pgpool: 2017-04-18 16:04:34: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:36 postgres_server1 pgpool: 2017-04-18 16:04:36: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:38 postgres_server1 pgpool: 2017-04-18 16:04:38: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:38 postgres_server1 pgpool: 2017-04-18 16:04:38: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:04:38 postgres_server1 pgpool: 2017-04-18 16:04:38: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:04:38 postgres_server1 pgpool: 2017-04-18 16:04:38: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:04:40 postgres_server1 pgpool: 2017-04-18 16:04:40: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:43 postgres_server1 pgpool: 2017-04-18 16:04:43: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:04:44 postgres_server1 pgpool: 2017-04-18 16:04:44: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:46 postgres_server1 pgpool: 2017-04-18 16:04:46: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:48 postgres_server1 pgpool: 2017-04-18 16:04:48: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:48 postgres_server1 pgpool: 2017-04-18 16:04:48: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:04:48 postgres_server1 pgpool: 2017-04-18 16:04:48: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:04:48 postgres_server1 pgpool: 2017-04-18 16:04:48: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:04:50 postgres_server1 pgpool: 2017-04-18 16:04:50: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:53 postgres_server1 pgpool: 2017-04-18 16:04:53: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:04:54 postgres_server1 pgpool: 2017-04-18 16:04:54: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:56 postgres_server1 pgpool: 2017-04-18 16:04:56: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:58 postgres_server1 pgpool: 2017-04-18 16:04:58: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:58 postgres_server1 pgpool: 2017-04-18 16:04:58: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:04:58 postgres_server1 pgpool: 2017-04-18 16:04:58: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:04:58 postgres_server1 pgpool: 2017-04-18 16:04:58: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:05:00 postgres_server1 pgpool: 2017-04-18 16:05:00: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:03 postgres_server1 pgpool: 2017-04-18 16:05:03: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:05:04 postgres_server1 pgpool: 2017-04-18 16:05:04: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:06 postgres_server1 pgpool: 2017-04-18 16:05:06: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:08 postgres_server1 pgpool: 2017-04-18 16:05:08: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:08 postgres_server1 pgpool: 2017-04-18 16:05:08: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:05:08 postgres_server1 pgpool: 2017-04-18 16:05:08: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:05:08 postgres_server1 pgpool: 2017-04-18 16:05:08: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:05:10 postgres_server1 pgpool: 2017-04-18 16:05:10: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:13 postgres_server1 pgpool: 2017-04-18 16:05:13: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:05:14 postgres_server1 pgpool: 2017-04-18 16:05:14: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:16 postgres_server1 pgpool: 2017-04-18 16:05:16: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:18 postgres_server1 pgpool: 2017-04-18 16:05:18: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:18 postgres_server1 pgpool: 2017-04-18 16:05:18: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:05:18 postgres_server1 pgpool: 2017-04-18 16:05:18: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:05:18 postgres_server1 pgpool: 2017-04-18 16:05:18: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:05:20 postgres_server1 pgpool: 2017-04-18 16:05:20: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:23 postgres_server1 pgpool: 2017-04-18 16:05:23: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:05:24 postgres_server1 pgpool: 2017-04-18 16:05:24: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:26 postgres_server1 pgpool: 2017-04-18 16:05:26: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:27 postgres_server1 systemd: pgpool.service start operation
> timed out. Terminating.
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19478:
> LOG:  received smart shutdown request
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19478:
> LOG:  shutdown request. closing listen socket
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19486:
> DEBUG:  watchdog heartbeat receiver child receives shutdown request signal
> 15
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19479:
> LOG:  Watchdog is shutting down
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19482:
> DEBUG:  lifecheck child receives shutdown request signal 15, forwarding to
> all children
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19487:
> DEBUG:  watchdog heartbeat sender child receives shutdown request signal 15
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19482:
> DEBUG:  lifecheck child receives smart shutdown request
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19550:
> DEBUG:  PCP child receives shutdown request signal 15, Forwarding to all
> children
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19550:
> DEBUG:  PCP child receives smart shutdown request
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19566:
> LOG:  watchdog: de-escalation started
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19482:
> DEBUG:  lifecheck child receives shutdown request signal 15, forwarding to
> all children
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19482:
> DEBUG:  lifecheck child receives smart shutdown request
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19566:
> DEBUG:  watchdog exec interface up/down command: 'sudo ifconfig bond1.18:0
> down' succeeded
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19566:
> LOG:  successfully released the delegate IP:"172.18.255.40"
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19566:
> DETAIL:  'if_down_cmd' returned with success
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19478:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 16:05:27 postgres_server1 systemd: Failed to start Pgpool-II.
>
> Apr 18 16:05:27 postgres_server1 systemd: Unit pgpool.service entered
> failed state.
>
> Apr 18 16:05:27 postgres_server1 systemd: pgpool.service failed.
>
> Apr 18 16:05:29 postgres_server1 ntpd[2889]: Deleting interface #6
> bond1.18:0, 172.18.255.40#123, interface stats: received=0, sent=0,
> dropped=0, active_time=598 secs
>
> Apr 18 16:09:29 postgres_server1 systemd: Starting Pgpool-II...
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> DEBUG:  loading hba configuration
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> DETAIL:  loading file :"/etc/pgpool-II/pool_hba.conf" for client
> authentication configuration file
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> LOG:  reading status file: 0 th backend is set to down status
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> DEBUG:  pool_coninfo_size: num_init_children (25) * max_pool (4) *
> MAX_NUM_BACKENDS (128) * sizeof(ConnectionInfo) (136) = 1740800 bytes
> requested for shared memory
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> DEBUG:  ProcessInfo: num_init_children (25) * sizeof(ProcessInfo) (32) =
> 800 bytes requested for shared memory
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> DEBUG:  Request info are: sizeof(POOL_REQUEST_INFO) 5304 bytes requested
> for shared memory
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> DEBUG:  Recovery management area: sizeof(int) 4 bytes requested for shared
> memory
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> LOG:  waiting for watchdog to initialize
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> LOG:  setting the local watchdog node name to "172.18.255.41:9999 Linux
> postgres_server1"
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> LOG:  watchdog cluster is configured with 1 remote nodes
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> LOG:  watchdog remote node:0 on 172.18.255.42:9000
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> LOG:  interface monitoring is disabled in watchdog
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> INFO:  IPC socket path: "/tmp/.s.PGPOOLWD_CMD.9000"
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> LOG:  watchdog node state changed from [DEAD] to [LOADING]
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> LOADING
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> DEBUG:  error in outbond connection to 172.18.255.42:9000
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> DETAIL:  Connection refused
>
> Apr 18 16:09:34 postgres_server1 pgpool: 2017-04-18 16:09:34: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = LOADING
>
> Apr 18 16:09:34 postgres_server1 pgpool: 2017-04-18 16:09:34: pid 19631:
> LOG:  watchdog node state changed from [LOADING] to [JOINING]
>
> Apr 18 16:09:34 postgres_server1 pgpool: 2017-04-18 16:09:34: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> JOINING
>
> Apr 18 16:09:34 postgres_server1 pgpool: 2017-04-18 16:09:34: pid 19631:
> DEBUG:  error in outbond connection to 172.18.255.42:9000
>
> Apr 18 16:09:34 postgres_server1 pgpool: 2017-04-18 16:09:34: pid 19631:
> DETAIL:  Connection refused
>
> Apr 18 16:09:38 postgres_server1 pgpool: 2017-04-18 16:09:38: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> JOINING
>
> Apr 18 16:09:38 postgres_server1 pgpool: 2017-04-18 16:09:38: pid 19631:
> LOG:  watchdog node state changed from [JOINING] to [INITIALIZING]
>
> Apr 18 16:09:38 postgres_server1 pgpool: 2017-04-18 16:09:38: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> INITIALIZING
>
> Apr 18 16:09:39 postgres_server1 pgpool: 2017-04-18 16:09:39: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State =
> INITIALIZING
>
> Apr 18 16:09:39 postgres_server1 pgpool: 2017-04-18 16:09:39: pid 19631:
> LOG:  I am the only alive node in the watchdog cluster
>
> Apr 18 16:09:39 postgres_server1 pgpool: 2017-04-18 16:09:39: pid 19631:
> HINT:  skiping stand for coordinator state
>
> Apr 18 16:09:39 postgres_server1 pgpool: 2017-04-18 16:09:39: pid 19631:
> LOG:  watchdog node state changed from [INITIALIZING] to [MASTER]
>
> Apr 18 16:09:39 postgres_server1 pgpool: 2017-04-18 16:09:39: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> MASTER
>
> Apr 18 16:09:39 postgres_server1 pgpool: 2017-04-18 16:09:39: pid 19631:
> LOG:  I am announcing my self as master/coordinator watchdog node
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  declare coordinator command finished with status:[COMMAND TIMEED
> OUT]
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  I am the cluster leader node
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DETAIL:  our declare coordinator message is accepted by all nodes
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  I am the cluster leader node. Starting escalation process
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  watchdog process is initialized
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  escalation process started with PID:19643
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  new IPC connection received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19643:
> LOG:  watchdog: escalation started
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  received the get data request from local pgpool-II on IPC interface
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> NOTICE:  error writing to IPC socket
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  1:0: Unexpected
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  I am the master watchdog node
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  using the local backend node status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> DEBUG:  I am watchdog lifecheck child with pid:19644
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  new IPC connection received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:549
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> LOG:  2 watchdog nodes are configured for lifecheck
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> LOG:  watchdog nodes ID:0 Name:"172.18.255.41:9999 Linux postgres_server1"
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> DETAIL:  Host:"172.18.255.41" WD Port:9000 pgpool-II port:9999
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> LOG:  watchdog nodes ID:1 Name:"Not_Set"
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> DETAIL:  Host:"172.18.255.42" WD Port:9000 pgpool-II port:9999
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> DEBUG:  watchdog checking life check is ready
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> DETAIL:  pgpool:1 at "172.18.255.42:9999" has not send the heartbeat
> signal yet
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  Setting up socket for 0.0.0.0:9999
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  Setting up socket for :::9999
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  perhaps failed to create INET domain socket
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  socket(::) failed: "Address family not supported by protocol"
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19652:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19653:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19654:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19655:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19656:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19657:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19658:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19659:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19660:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19661:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19662:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19663:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19664:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19665:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19666:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19667:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19668:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19670:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19671:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19672:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19673:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  network event received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DETAIL:  deleted = NO Link change event = NO
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = NETWORK IP IS ASSIGNED Current
> State = MASTER
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  network event received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DETAIL:  deleted = YES Link change event = NO
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = NETWORK IP IS REMOVED Current
> State = MASTER
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19674:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19675:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  network IP is removed but system still has a valid IP is assigned
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  IP = 172.19.255.41
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  IP = 172.18.255.41
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  IP = 172.18.255.40
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19676:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19677:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  network event received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DETAIL:  deleted = NO Link change event = NO
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = NETWORK IP IS ASSIGNED Current
> State = MASTER
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19643:
> DEBUG:  watchdog exec interface up/down command: 'sudo ifconfig bond1.18:0
> inet $_IP_$ netmask 255.255.255.0' succeeded
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19678:
> DEBUG:  I am PCP child with pid:19678
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  pgpool-II successfully started. version 3.6.2 (subaruboshi)
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  find_primary_node: checking backend no 0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  find_primary_node: checking backend no 1
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19680:
> DEBUG:  I am 19680
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19680:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  find_primary_node: no primary node found
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19680:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19680:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19680:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  starting health check
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  Backend DB node 1 status is 2
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  Trying to make persistent DB connection to backend node 1 having
> status 2
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  doing health check against database:template1 user:postgres
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  Backend DB node 1 status is 2
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  Trying to make persistent DB connection to backend node 1 having
> status 2
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  setting backend node 1 status to NODE DOWN
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  received degenerate backend request for node_id: 1 from pid [19630]
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  new IPC connection received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  watchdog received the failover command from local pgpool-II on IPC
> interface
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  watchdog is processing the failover command
> [DEGENERATE_BACKEND_REQUEST] received from local pgpool-II on IPC interface
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  I am the only pgpool-II node in the watchdog cluster
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DETAIL:  no need to propagate the failover command
> [DEGENERATE_BACKEND_REQUEST]
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler called
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kind: 1 flags: 1 node_count: 1 index:0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  new IPC connection received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to become a lock holder for failover ID: 0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> the lock holder
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  starting to select new master node
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  starting degeneration. shutdown host 172.18.255.42(5432)
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  failover: no valid backends node found
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover/failback request details: STREAM: 1 reqkind: 1 detail: 1
> node_id: 1
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  Restart all children
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19652
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19653
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19654
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19655
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19656
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19657
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19658
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19659
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19660
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19661
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19662
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19663
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19664
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19665
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19666
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19667
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19668
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19670
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19671
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19672
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19673
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19674
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19675
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19676
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19677
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  execute command: /etc/pgpool/failover.sh 1 172.18.255.42 5432
> /postgres_db/data -1 1 "" 1 "" "" > AAA_failover_called.log
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  new IPC connection received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to release [FAILOVER] lock for failover ID 0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has released the [FAILOVER] lock for failover ID 0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  failover: no follow backends are degenerated
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  new IPC connection received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to release [FOLLOW MASTER] lock for failover ID 0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has released the [FOLLOW MASTER] lock for failover ID 0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  failover: set new primary node: -1
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19687:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19688:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19689:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19690:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19691:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19692:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19693:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19694:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19695:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19696:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19697:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19698:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19699:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19700:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19701:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19702:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19703:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19704:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19705:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19706:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19707:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19708:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19709:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19710:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19680:
> LOG:  worker process received restart request
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  new IPC connection received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to resign from a lock holder for failover ID 0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has resigned from the lock holder
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:09:43 postgres_server1 pgpool: failover done. shutdown host
> 172.18.255.42(5432)2017-04-18 16:09:43: pid 19630: LOG:  failover done.
> shutdown host 172.18.255.42(5432)
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19711:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19648:
> LOG:  set SO_REUSEPORT option to the socket
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19648:
> LOG:  creating watchdog heartbeat receive socket.
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19648:
> DETAIL:  set SO_REUSEPORT
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19649:
> LOG:  set SO_REUSEPORT option to the socket
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19649:
> LOG:  creating socket for sending heartbeat
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19649:
> DETAIL:  set SO_REUSEPORT
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19678:
> LOG:  restart request received in pcp child process
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19678:
> DEBUG:  PCP child receives shutdown request signal 15, Forwarding to all
> children
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19678:
> DEBUG:  PCP child receives smart shutdown request
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  PCP child 19678 exits with status 0 in failover()
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  fork a new PCP child pid 19712 in failover()
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  starting health check
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  reaper handler
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19712:
> DEBUG:  I am PCP child with pid:19712
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19652 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19652 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19652 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19653 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19653 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19653 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19654 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19654 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19654 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19655 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19655 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19655 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19656 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19656 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19656 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19657 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19657 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19657 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19658 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19658 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19658 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19659 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19659 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19659 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19660 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19660 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19660 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19661 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19661 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19661 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19662 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19662 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19662 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19663 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19663 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19663 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19664 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19664 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19664 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19665 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19665 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19665 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19666 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19666 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19666 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19667 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19667 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19667 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19668 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19668 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19668 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19670 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19670 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19670 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19671 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19671 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19671 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19672 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19672 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19672 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19673 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19673 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19673 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19674 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19674 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19674 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19675 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19675 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19675 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19676 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19676 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19676 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19677 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19677 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19677 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  worker child process with pid: 19680 exits with status 256
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  fork a new worker child process with pid: 19713
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  reaper handler: exiting normally
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19713:
> DEBUG:  I am 19713
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19713:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:44 postgres_server1 ntpd[2889]: Listen normally on 7
> bond1.18:0 172.18.255.40 UDP 123
>
> Apr 18 16:09:45 postgres_server1 pgpool: 2017-04-18 16:09:45: pid 19643:
> DEBUG:  watchdog exec interface up/down command: 'sudo arping -U $_IP_$ -w
> 1 -I bond1.18' succeeded
>
> Apr 18 16:09:46 postgres_server1 pgpool: 2017-04-18 16:09:46: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19643:
> DEBUG:  watchdog ping process for host "172.18.255.40" exited successfully
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19643:
> DEBUG:  watchdog ping
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19643:
> DETAIL:  ping data: PING 172.18.255.40 (172.18.255.40) 56(84) bytes of data.
>
> Apr 18 16:09:47 postgres_server1 pgpool: --- 172.18.255.40 ping statistics
> ---
>
> Apr 18 16:09:47 postgres_server1 pgpool: 3 packets transmitted, 3
> received, 0% packet loss, time 1999ms
>
> Apr 18 16:09:47 postgres_server1 pgpool: rtt min/avg/max/mdev =
> 0.026/0.027/0.029/0.004 ms
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19643:
> DEBUG:  watchdog succeeded to ping a host "172.18.255.40"
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19643:
> LOG:  successfully acquired the delegate IP:"172.18.255.40"
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19643:
> DETAIL:  'if_up_cmd' returned with success
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19631:
> DEBUG:  watchdog process signal handler
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19631:
> LOG:  watchdog escalation process with pid: 19643 exit with SUCCESS.
>
> Apr 18 16:09:48 postgres_server1 pgpool: 2017-04-18 16:09:48: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:09:50 postgres_server1 pgpool: 2017-04-18 16:09:50: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:09:50 postgres_server1 pgpool: 2017-04-18 16:09:50: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:09:50 postgres_server1 pgpool: 2017-04-18 16:09:50: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:09:50 postgres_server1 pgpool: 2017-04-18 16:09:50: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:09:52 postgres_server1 pgpool: 2017-04-18 16:09:52: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:09:54 postgres_server1 pgpool: 2017-04-18 16:09:54: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:09:55 postgres_server1 pgpool: 2017-04-18 16:09:55: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:09:56 postgres_server1 pgpool: 2017-04-18 16:09:56: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:09:58 postgres_server1 pgpool: 2017-04-18 16:09:58: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:00 postgres_server1 pgpool: 2017-04-18 16:10:00: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:00 postgres_server1 pgpool: 2017-04-18 16:10:00: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:10:00 postgres_server1 pgpool: 2017-04-18 16:10:00: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:10:00 postgres_server1 pgpool: 2017-04-18 16:10:00: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:10:01 postgres_server1 systemd: Started Session 15 of user root.
>
> Apr 18 16:10:01 postgres_server1 systemd: Starting Session 15 of user root.
>
> Apr 18 16:10:02 postgres_server1 pgpool: 2017-04-18 16:10:02: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:04 postgres_server1 pgpool: 2017-04-18 16:10:04: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:05 postgres_server1 pgpool: 2017-04-18 16:10:05: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:10:06 postgres_server1 pgpool: 2017-04-18 16:10:06: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:08 postgres_server1 pgpool: 2017-04-18 16:10:08: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:10 postgres_server1 pgpool: 2017-04-18 16:10:10: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:10 postgres_server1 pgpool: 2017-04-18 16:10:10: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:10:10 postgres_server1 pgpool: 2017-04-18 16:10:10: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:10:10 postgres_server1 pgpool: 2017-04-18 16:10:10: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:10:12 postgres_server1 pgpool: 2017-04-18 16:10:12: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19630:
> DEBUG:  starting health check
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19630:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19630:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19630:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:10:15 postgres_server1 pgpool: 2017-04-18 16:10:15: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:10:16 postgres_server1 pgpool: 2017-04-18 16:10:16: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:18 postgres_server1 pgpool: 2017-04-18 16:10:18: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:20 postgres_server1 pgpool: 2017-04-18 16:10:20: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:20 postgres_server1 pgpool: 2017-04-18 16:10:20: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:10:20 postgres_server1 pgpool: 2017-04-18 16:10:20: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:10:20 postgres_server1 pgpool: 2017-04-18 16:10:20: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:10:22 postgres_server1 pgpool: 2017-04-18 16:10:22: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:24 postgres_server1 pgpool: 2017-04-18 16:10:24: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:25 postgres_server1 pgpool: 2017-04-18 16:10:25: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:10:26 postgres_server1 pgpool: 2017-04-18 16:10:26: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:28 postgres_server1 pgpool: 2017-04-18 16:10:28: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:30 postgres_server1 pgpool: 2017-04-18 16:10:30: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:30 postgres_server1 pgpool: 2017-04-18 16:10:30: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:10:30 postgres_server1 pgpool: 2017-04-18 16:10:30: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:10:30 postgres_server1 pgpool: 2017-04-18 16:10:30: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:10:32 postgres_server1 pgpool: 2017-04-18 16:10:32: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:34 postgres_server1 pgpool: 2017-04-18 16:10:34: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:35 postgres_server1 pgpool: 2017-04-18 16:10:35: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:10:36 postgres_server1 pgpool: 2017-04-18 16:10:36: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:38 postgres_server1 pgpool: 2017-04-18 16:10:38: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:40 postgres_server1 pgpool: 2017-04-18 16:10:40: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:40 postgres_server1 pgpool: 2017-04-18 16:10:40: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:10:40 postgres_server1 pgpool: 2017-04-18 16:10:40: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:10:40 postgres_server1 pgpool: 2017-04-18 16:10:40: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:10:42 postgres_server1 pgpool: 2017-04-18 16:10:42: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19630:
> DEBUG:  starting health check
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19630:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19630:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19630:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:10:45 postgres_server1 pgpool: 2017-04-18 16:10:45: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:10:46 postgres_server1 pgpool: 2017-04-18 16:10:46: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:48 postgres_server1 pgpool: 2017-04-18 16:10:48: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:50 postgres_server1 pgpool: 2017-04-18 16:10:50: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:50 postgres_server1 pgpool: 2017-04-18 16:10:50: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:10:50 postgres_server1 pgpool: 2017-04-18 16:10:50: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:10:50 postgres_server1 pgpool: 2017-04-18 16:10:50: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:10:52 postgres_server1 pgpool: 2017-04-18 16:10:52: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:54 postgres_server1 pgpool: 2017-04-18 16:10:54: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:55 postgres_server1 pgpool: 2017-04-18 16:10:55: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:10:56 postgres_server1 pgpool: 2017-04-18 16:10:56: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:58 postgres_server1 pgpool: 2017-04-18 16:10:58: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:59 postgres_server1 systemd: pgpool.service start operation
> timed out. Terminating.
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19630:
> LOG:  received smart shutdown request
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19630:
> LOG:  shutdown request. closing listen socket
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19648:
> DEBUG:  watchdog heartbeat receiver child receives shutdown request signal
> 15
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19644:
> DEBUG:  lifecheck child receives shutdown request signal 15, forwarding to
> all children
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19649:
> DEBUG:  watchdog heartbeat sender child receives shutdown request signal 15
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19631:
> LOG:  Watchdog is shutting down
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19644:
> DEBUG:  lifecheck child receives smart shutdown request
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19644:
> DEBUG:  lifecheck child receives shutdown request signal 15, forwarding to
> all children
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19644:
> DEBUG:  lifecheck child receives smart shutdown request
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19712:
> DEBUG:  PCP child receives shutdown request signal 15, Forwarding to all
> children
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19712:
> DEBUG:  PCP child receives smart shutdown request
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19630:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 16:10:59 postgres_server1 systemd: Failed to start Pgpool-II.
>
> Apr 18 16:10:59 postgres_server1 systemd: Unit pgpool.service entered
> failed state.
>
> Apr 18 16:10:59 postgres_server1 systemd: pgpool.service failed.
>
> Apr 18 16:12:44 postgres_server1 systemd-logind: New session 16 of user
> root.
>
> Apr 18 16:12:44 postgres_server1 systemd: Started Session 16 of user root.
>
> Apr 18 16:12:44 postgres_server1 systemd: Starting Session 16 of user root.
> IMPORTANT - This email and any attachments is intended for the above named
> addressee(s), and may contain information which is confidential or
> privileged. If you are not the intended recipient, please inform the sender
> immediately and delete this email: you should not copy or use this e-mail
> for any purpose nor disclose its contents to any person.
>
> _______________________________________________
> pgpool-general mailing list
> pgpool-general at pgpool.net
> http://www.pgpool.net/mailman/listinfo/pgpool-general
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20171013/57a1e3ca/attachment-0001.html>


More information about the pgpool-general mailing list