[pgpool-general: 5427] Re: Pgpool on CentOS7 refuse to come up

Muhammad Usama m.usama at gmail.com
Wed Apr 19 04:37:11 JST 2017


There was a recent fix committed to the pgpool.sysconfig and pgpool.service
files, that could cause such behaviours. Can you try using the latest
version of these two files from the git head.
You can find the *pgpool.sysconfig* and *pgpool.service* files in
*src/redhat/* directory of pgpool-II source.

Thanks
Best regards
Muhammad Usama

On Tue, Apr 18, 2017 at 8:43 PM, Avi Weinberg <AviW at gilat.com> wrote:

> Hi all,
>
>
>
> We upgraded to CentOS7 and pgpool with the configuration that works on 6.8
> refuse to come up.  Your input is most welcome.
>
>
>
> In addition, is it possible to change pgpool configuration so even with
> CentOS7 we will have pgpool log file in addition or instead of the sys
> messages.
>
>
>
> Thanks
>
> Avi
>
>
>
>
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19154
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19155
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19156
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19157
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19158
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19159
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19161
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19162
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19163
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19164
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19165
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19166
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19167
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19169
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19170
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19171
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19172
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19173
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DEBUG:  failover handler
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> DETAIL:  kill process with PID:19174
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> LOG:  execute command: /etc/pgpool/failover.sh 1 172.18.255.42 5432
> /postgres_db/data -1 1 "" 1 "" "" > AAA_failover_called.log
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  new IPC connection received
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to release [FAILOVER] lock for failover ID 0
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has released the [FAILOVER] lock for failover ID 0
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> LOG:  failover: no follow backends are degenerated
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  new IPC connection received
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to release [FOLLOW MASTER] lock for failover ID 0
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has released the [FOLLOW MASTER] lock for failover ID 0
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19123:
> LOG:  failover: set new primary node: -1
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19183:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19184:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19185:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19186:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19187:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19188:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19189:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19190:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19191:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19192:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19193:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19194:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19195:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19196:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19197:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19198:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19199:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19200:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19201:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19202:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19203:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19204:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19205:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19206:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19176:
> LOG:  worker process received restart request
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  new IPC connection received
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to resign from a lock holder for failover ID 0
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has resigned from the lock holder
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19124:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 15:55:29 postgres_server1 pgpool: 2017-04-18 15:55:29: pid 19207:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:29 postgres_server1 pgpool: failover done. shutdown host
> 172.18.255.42(5432)2017-04-18 15:55:29: pid 19123: LOG:  failover done.
> shutdown host 172.18.255.42(5432)
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19144:
> LOG:  set SO_REUSEPORT option to the socket
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19144:
> LOG:  creating watchdog heartbeat receive socket.
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19144:
> DETAIL:  set SO_REUSEPORT
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19145:
> LOG:  set SO_REUSEPORT option to the socket
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19145:
> LOG:  creating socket for sending heartbeat
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19145:
> DETAIL:  set SO_REUSEPORT
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19175:
> LOG:  restart request received in pcp child process
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19175:
> DEBUG:  PCP child receives shutdown request signal 15, Forwarding to all
> children
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19175:
> DEBUG:  PCP child receives smart shutdown request
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  PCP child 19175 exits with status 0 in failover()
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  fork a new PCP child pid 19208 in failover()
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19208:
> DEBUG:  I am PCP child with pid:19208
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  reaper handler
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19148 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19148 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19148 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19149 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19149 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19149 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19150 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19150 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19150 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19151 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19151 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19151 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19152 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19152 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19152 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19153 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19153 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19153 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19154 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19154 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19154 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19155 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19155 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19155 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19156 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19156 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19156 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19157 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19157 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19157 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19158 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19158 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19158 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19159 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19159 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19159 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19161 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19161 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19161 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19162 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19162 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19162 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19163 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19163 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19163 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19164 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19164 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19164 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19165 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19165 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19165 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19166 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19166 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19166 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19167 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19167 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19167 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19169 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19169 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19169 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19170 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19170 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19170 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19171 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19171 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19171 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19172 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19172 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19172 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19173 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19173 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19173 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  child process with pid: 19174 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19174 exits with status 0
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  child process with pid: 19174 exited with success and will not be
> restarted
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  worker child process with pid: 19176 exits with status 256
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> LOG:  fork a new worker child process with pid: 19209
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19123:
> DEBUG:  reaper handler: exiting normally
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19209:
> DEBUG:  I am 19209
>
> Apr 18 15:55:30 postgres_server1 pgpool: 2017-04-18 15:55:30: pid 19209:
> DEBUG:  initializing backend status
>
> Apr 18 15:55:31 postgres_server1 ntpd[2889]: Listen normally on 6
> bond1.18:0 172.18.255.40 UDP 123
>
> Apr 18 15:55:31 postgres_server1 pgpool: 2017-04-18 15:55:31: pid 19139:
> DEBUG:  watchdog exec interface up/down command: 'sudo arping -U $_IP_$ -w
> 1 -I bond1.18' succeeded
>
> Apr 18 15:55:32 postgres_server1 pgpool: 2017-04-18 15:55:32: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19139:
> DEBUG:  watchdog ping process for host "172.18.255.40" exited successfully
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19139:
> DEBUG:  watchdog ping
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19139:
> DETAIL:  ping data: PING 172.18.255.40 (172.18.255.40) 56(84) bytes of data.
>
> Apr 18 15:55:33 postgres_server1 pgpool: --- 172.18.255.40 ping statistics
> ---
>
> Apr 18 15:55:33 postgres_server1 pgpool: 3 packets transmitted, 3
> received, 0% packet loss, time 1998ms
>
> Apr 18 15:55:33 postgres_server1 pgpool: rtt min/avg/max/mdev =
> 0.024/0.026/0.030/0.006 ms
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19139:
> DEBUG:  watchdog succeeded to ping a host "172.18.255.40"
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19139:
> LOG:  successfully acquired the delegate IP:"172.18.255.40"
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19139:
> DETAIL:  'if_up_cmd' returned with success
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19124:
> DEBUG:  watchdog process signal handler
>
> Apr 18 15:55:33 postgres_server1 pgpool: 2017-04-18 15:55:33: pid 19124:
> LOG:  watchdog escalation process with pid: 19139 exit with SUCCESS.
>
> Apr 18 15:55:34 postgres_server1 pgpool: 2017-04-18 15:55:34: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:36 postgres_server1 pgpool: 2017-04-18 15:55:36: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:36 postgres_server1 pgpool: 2017-04-18 15:55:36: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:55:36 postgres_server1 pgpool: 2017-04-18 15:55:36: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:55:36 postgres_server1 pgpool: 2017-04-18 15:55:36: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:55:38 postgres_server1 pgpool: 2017-04-18 15:55:38: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:40 postgres_server1 pgpool: 2017-04-18 15:55:40: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:41 postgres_server1 pgpool: 2017-04-18 15:55:41: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:55:42 postgres_server1 pgpool: 2017-04-18 15:55:42: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:44 postgres_server1 pgpool: 2017-04-18 15:55:44: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:46 postgres_server1 pgpool: 2017-04-18 15:55:46: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:46 postgres_server1 pgpool: 2017-04-18 15:55:46: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:55:46 postgres_server1 pgpool: 2017-04-18 15:55:46: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:55:46 postgres_server1 pgpool: 2017-04-18 15:55:46: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:55:48 postgres_server1 pgpool: 2017-04-18 15:55:48: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:50 postgres_server1 pgpool: 2017-04-18 15:55:50: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:55:51 postgres_server1 pgpool: 2017-04-18 15:55:51: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:55:52 postgres_server1 pgpool: 2017-04-18 15:55:52: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:54 postgres_server1 pgpool: 2017-04-18 15:55:54: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:56 postgres_server1 pgpool: 2017-04-18 15:55:56: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:55:56 postgres_server1 pgpool: 2017-04-18 15:55:56: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:55:56 postgres_server1 pgpool: 2017-04-18 15:55:56: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:55:56 postgres_server1 pgpool: 2017-04-18 15:55:56: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:55:58 postgres_server1 pgpool: 2017-04-18 15:55:58: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:00 postgres_server1 pgpool: 2017-04-18 15:56:00: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:01 postgres_server1 pgpool: 2017-04-18 15:56:01: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:56:02 postgres_server1 pgpool: 2017-04-18 15:56:02: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:04 postgres_server1 pgpool: 2017-04-18 15:56:04: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:06 postgres_server1 pgpool: 2017-04-18 15:56:06: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:06 postgres_server1 pgpool: 2017-04-18 15:56:06: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:56:06 postgres_server1 pgpool: 2017-04-18 15:56:06: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:56:06 postgres_server1 pgpool: 2017-04-18 15:56:06: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:56:08 postgres_server1 pgpool: 2017-04-18 15:56:08: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:10 postgres_server1 pgpool: 2017-04-18 15:56:10: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:11 postgres_server1 pgpool: 2017-04-18 15:56:11: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:56:12 postgres_server1 pgpool: 2017-04-18 15:56:12: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:14 postgres_server1 pgpool: 2017-04-18 15:56:14: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:16 postgres_server1 pgpool: 2017-04-18 15:56:16: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:16 postgres_server1 pgpool: 2017-04-18 15:56:16: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:56:16 postgres_server1 pgpool: 2017-04-18 15:56:16: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:56:16 postgres_server1 pgpool: 2017-04-18 15:56:16: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:56:18 postgres_server1 pgpool: 2017-04-18 15:56:18: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:20 postgres_server1 pgpool: 2017-04-18 15:56:20: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:21 postgres_server1 pgpool: 2017-04-18 15:56:21: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:56:22 postgres_server1 pgpool: 2017-04-18 15:56:22: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:24 postgres_server1 pgpool: 2017-04-18 15:56:24: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:26 postgres_server1 pgpool: 2017-04-18 15:56:26: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:26 postgres_server1 pgpool: 2017-04-18 15:56:26: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:56:26 postgres_server1 pgpool: 2017-04-18 15:56:26: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:56:26 postgres_server1 pgpool: 2017-04-18 15:56:26: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:56:28 postgres_server1 pgpool: 2017-04-18 15:56:28: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:30 postgres_server1 pgpool: 2017-04-18 15:56:30: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:31 postgres_server1 pgpool: 2017-04-18 15:56:31: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:56:32 postgres_server1 pgpool: 2017-04-18 15:56:32: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:34 postgres_server1 pgpool: 2017-04-18 15:56:34: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:36 postgres_server1 pgpool: 2017-04-18 15:56:36: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:36 postgres_server1 pgpool: 2017-04-18 15:56:36: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:56:36 postgres_server1 pgpool: 2017-04-18 15:56:36: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:56:36 postgres_server1 pgpool: 2017-04-18 15:56:36: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:56:38 postgres_server1 pgpool: 2017-04-18 15:56:38: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19123:
> DEBUG:  starting health check
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19123:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19123:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19123:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:40 postgres_server1 pgpool: 2017-04-18 15:56:40: pid 19123:
> DEBUG:  health check: clearing alarm
>
> Apr 18 15:56:41 postgres_server1 pgpool: 2017-04-18 15:56:41: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 15:56:42 postgres_server1 pgpool: 2017-04-18 15:56:42: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:44 postgres_server1 pgpool: 2017-04-18 15:56:44: pid 19145:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 15:56:45 postgres_server1 systemd: pgpool.service start operation
> timed out. Terminating.
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19123:
> LOG:  received smart shutdown request
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19123:
> LOG:  shutdown request. closing listen socket
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19144:
> DEBUG:  watchdog heartbeat receiver child receives shutdown request signal
> 15
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19140:
> DEBUG:  lifecheck child receives shutdown request signal 15, forwarding to
> all children
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19124:
> LOG:  Watchdog is shutting down
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19145:
> DEBUG:  watchdog heartbeat sender child receives shutdown request signal 15
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19140:
> DEBUG:  lifecheck child receives smart shutdown request
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19124:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19124:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19124:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19311:
> LOG:  watchdog: de-escalation started
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19140:
> DEBUG:  lifecheck child receives shutdown request signal 15, forwarding to
> all children
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19208:
> DEBUG:  PCP child receives shutdown request signal 15, Forwarding to all
> children
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19140:
> DEBUG:  lifecheck child receives smart shutdown request
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19208:
> DEBUG:  PCP child receives smart shutdown request
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19208:
> DEBUG:  PCP child receives shutdown request signal 15, Forwarding to all
> children
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19208:
> DEBUG:  PCP child receives smart shutdown request
>
> Apr 18 15:56:45 postgres_server1 pgpool: 2017-04-18 15:56:45: pid 19123:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 15:56:45 postgres_server1 systemd: Failed to start Pgpool-II.
>
> Apr 18 15:56:45 postgres_server1 systemd: Unit pgpool.service entered
> failed state.
>
> Apr 18 15:56:45 postgres_server1 systemd: pgpool.service failed.
>
> Apr 18 16:00:01 postgres_server1 systemd: Started Session 13 of user root.
>
> Apr 18 16:00:01 postgres_server1 systemd: Starting Session 13 of user root.
>
> Apr 18 16:01:01 postgres_server1 systemd: Started Session 14 of user root.
>
> Apr 18 16:01:01 postgres_server1 systemd: Starting Session 14 of user root.
>
> Apr 18 16:03:57 postgres_server1 systemd: Starting Pgpool-II...
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> DEBUG:  loading hba configuration
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> DETAIL:  loading file :"/etc/pgpool-II/pool_hba.conf" for client
> authentication configuration file
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> LOG:  reading status file: 0 th backend is set to down status
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> DEBUG:  pool_coninfo_size: num_init_children (25) * max_pool (4) *
> MAX_NUM_BACKENDS (128) * sizeof(ConnectionInfo) (136) = 1740800 bytes
> requested for shared memory
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> DEBUG:  ProcessInfo: num_init_children (25) * sizeof(ProcessInfo) (32) =
> 800 bytes requested for shared memory
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> DEBUG:  Request info are: sizeof(POOL_REQUEST_INFO) 5304 bytes requested
> for shared memory
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> DEBUG:  Recovery management area: sizeof(int) 4 bytes requested for shared
> memory
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19478:
> LOG:  waiting for watchdog to initialize
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> LOG:  setting the local watchdog node name to "172.18.255.41:9999 Linux
> postgres_server1"
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> LOG:  watchdog cluster is configured with 1 remote nodes
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> LOG:  watchdog remote node:0 on 172.18.255.42:9000
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> LOG:  interface monitoring is disabled in watchdog
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> INFO:  IPC socket path: "/tmp/.s.PGPOOLWD_CMD.9000"
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> LOG:  watchdog node state changed from [DEAD] to [LOADING]
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> LOADING
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> DEBUG:  error in outbond connection to 172.18.255.42:9000
>
> Apr 18 16:03:57 postgres_server1 pgpool: 2017-04-18 16:03:57: pid 19479:
> DETAIL:  Connection refused
>
> Apr 18 16:04:02 postgres_server1 pgpool: 2017-04-18 16:04:02: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = LOADING
>
> Apr 18 16:04:02 postgres_server1 pgpool: 2017-04-18 16:04:02: pid 19479:
> LOG:  watchdog node state changed from [LOADING] to [JOINING]
>
> Apr 18 16:04:02 postgres_server1 pgpool: 2017-04-18 16:04:02: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> JOINING
>
> Apr 18 16:04:02 postgres_server1 pgpool: 2017-04-18 16:04:02: pid 19479:
> DEBUG:  error in outbond connection to 172.18.255.42:9000
>
> Apr 18 16:04:02 postgres_server1 pgpool: 2017-04-18 16:04:02: pid 19479:
> DETAIL:  Connection refused
>
> Apr 18 16:04:06 postgres_server1 pgpool: 2017-04-18 16:04:06: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> JOINING
>
> Apr 18 16:04:06 postgres_server1 pgpool: 2017-04-18 16:04:06: pid 19479:
> LOG:  watchdog node state changed from [JOINING] to [INITIALIZING]
>
> Apr 18 16:04:06 postgres_server1 pgpool: 2017-04-18 16:04:06: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> INITIALIZING
>
> Apr 18 16:04:07 postgres_server1 pgpool: 2017-04-18 16:04:07: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State =
> INITIALIZING
>
> Apr 18 16:04:07 postgres_server1 pgpool: 2017-04-18 16:04:07: pid 19479:
> LOG:  I am the only alive node in the watchdog cluster
>
> Apr 18 16:04:07 postgres_server1 pgpool: 2017-04-18 16:04:07: pid 19479:
> HINT:  skiping stand for coordinator state
>
> Apr 18 16:04:07 postgres_server1 pgpool: 2017-04-18 16:04:07: pid 19479:
> LOG:  watchdog node state changed from [INITIALIZING] to [MASTER]
>
> Apr 18 16:04:07 postgres_server1 pgpool: 2017-04-18 16:04:07: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> MASTER
>
> Apr 18 16:04:07 postgres_server1 pgpool: 2017-04-18 16:04:07: pid 19479:
> LOG:  I am announcing my self as master/coordinator watchdog node
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  declare coordinator command finished with status:[COMMAND TIMEED
> OUT]
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  I am the cluster leader node
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DETAIL:  our declare coordinator message is accepted by all nodes
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  I am the cluster leader node. Starting escalation process
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  watchdog process is initialized
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  escalation process started with PID:19481
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  new IPC connection received
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  received the get data request from local pgpool-II on IPC interface
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> NOTICE:  error writing to IPC socket
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19481:
> LOG:  watchdog: escalation started
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> DEBUG:  I am watchdog lifecheck child with pid:19482
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  1:0: Unexpected
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  I am the master watchdog node
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  using the local backend node status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  new IPC connection received
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:549
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> LOG:  2 watchdog nodes are configured for lifecheck
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> LOG:  watchdog nodes ID:0 Name:"172.18.255.41:9999 Linux postgres_server1"
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> DETAIL:  Host:"172.18.255.41" WD Port:9000 pgpool-II port:9999
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> LOG:  watchdog nodes ID:1 Name:"Not_Set"
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> DETAIL:  Host:"172.18.255.42" WD Port:9000 pgpool-II port:9999
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> DEBUG:  watchdog checking life check is ready
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19482:
> DETAIL:  pgpool:1 at "172.18.255.42:9999" has not send the heartbeat
> signal yet
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  Setting up socket for 0.0.0.0:9999
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  Setting up socket for :::9999
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  perhaps failed to create INET domain socket
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  socket(::) failed: "Address family not supported by protocol"
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19490:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19491:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19492:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19493:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19494:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19495:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19496:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19497:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19498:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19499:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19500:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19501:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19502:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19503:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19504:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19505:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19506:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19508:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19509:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19510:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19511:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19512:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19513:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19514:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19515:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19481:
> DEBUG:  watchdog exec interface up/down command: 'sudo ifconfig bond1.18:0
> inet $_IP_$ netmask 255.255.255.0' succeeded
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19516:
> DEBUG:  I am PCP child with pid:19516
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  pgpool-II successfully started. version 3.6.2 (subaruboshi)
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  find_primary_node: checking backend no 0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  find_primary_node: checking backend no 1
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19517:
> DEBUG:  I am 19517
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19517:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  find_primary_node: no primary node found
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19517:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19517:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19517:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  Backend DB node 1 status is 2
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  Trying to make persistent DB connection to backend node 1 having
> status 2
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  doing health check against database:template1 user:postgres
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  Backend DB node 1 status is 2
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  Trying to make persistent DB connection to backend node 1 having
> status 2
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  setting backend node 1 status to NODE DOWN
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  received degenerate backend request for node_id: 1 from pid [19478]
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  new IPC connection received
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  watchdog received the failover command from local pgpool-II on IPC
> interface
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  watchdog is processing the failover command
> [DEGENERATE_BACKEND_REQUEST] received from local pgpool-II on IPC interface
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  I am the only pgpool-II node in the watchdog cluster
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DETAIL:  no need to propagate the failover command
> [DEGENERATE_BACKEND_REQUEST]
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler called
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kind: 1 flags: 1 node_count: 1 index:0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  new IPC connection received
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to become a lock holder for failover ID: 0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> the lock holder
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  starting to select new master node
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  starting degeneration. shutdown host 172.18.255.42(5432)
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  failover: no valid backends node found
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover/failback request details: STREAM: 1 reqkind: 1 detail: 1
> node_id: 1
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  Restart all children
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19490
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19491
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19492
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19493
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19494
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19495
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19496
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19497
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19498
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19499
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19500
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19501
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19502
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19503
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19504
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19505
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19506
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19508
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19509
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19510
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19511
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19512
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19513
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19514
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DEBUG:  failover handler
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> DETAIL:  kill process with PID:19515
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  execute command: /etc/pgpool/failover.sh 1 172.18.255.42 5432
> /postgres_db/data -1 1 "" 1 "" "" > AAA_failover_called.log
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  new IPC connection received
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to release [FAILOVER] lock for failover ID 0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has released the [FAILOVER] lock for failover ID 0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  failover: no follow backends are degenerated
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  new IPC connection received
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to release [FOLLOW MASTER] lock for failover ID 0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has released the [FOLLOW MASTER] lock for failover ID 0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  failover: set new primary node: -1
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19525:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19526:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19527:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19528:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19529:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19530:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19531:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19532:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19533:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19534:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19535:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19536:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19537:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19538:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19539:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19540:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19541:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19542:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19543:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19544:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19545:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19546:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19547:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19548:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  new IPC connection received
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19517:
> LOG:  worker process received restart request
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to resign from a lock holder for failover ID 0
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has resigned from the lock holder
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19479:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:04:11 postgres_server1 pgpool: failover done. shutdown host
> 172.18.255.42(5432)2017-04-18 16:04:11: pid 19549: DEBUG:  initializing
> backend status
>
> Apr 18 16:04:11 postgres_server1 pgpool: 2017-04-18 16:04:11: pid 19478:
> LOG:  failover done. shutdown host 172.18.255.42(5432)
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19486:
> LOG:  set SO_REUSEPORT option to the socket
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19486:
> LOG:  creating watchdog heartbeat receive socket.
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19486:
> DETAIL:  set SO_REUSEPORT
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19487:
> LOG:  set SO_REUSEPORT option to the socket
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19487:
> LOG:  creating socket for sending heartbeat
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19487:
> DETAIL:  set SO_REUSEPORT
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19516:
> LOG:  restart request received in pcp child process
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19516:
> DEBUG:  PCP child receives shutdown request signal 15, Forwarding to all
> children
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19516:
> DEBUG:  PCP child receives smart shutdown request
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  PCP child 19516 exits with status 0 in failover()
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  fork a new PCP child pid 19550 in failover()
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19550:
> DEBUG:  I am PCP child with pid:19550
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  reaper handler
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19490 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19490 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19490 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19491 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19491 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19491 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19492 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19492 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19492 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19493 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19493 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19493 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19494 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19494 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19494 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19495 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19495 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19495 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19496 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19496 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19496 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19497 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19497 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19497 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19498 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19498 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19498 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19499 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19499 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19499 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19500 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19500 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19500 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19501 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19501 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19501 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19502 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19502 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19502 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19503 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19503 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19503 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19504 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19504 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19504 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19505 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19505 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19505 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19506 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19506 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19506 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19508 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19508 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19508 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19509 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19509 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19509 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19510 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19510 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19510 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19511 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19511 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19511 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19512 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19512 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19512 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19513 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19513 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19513 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19514 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19514 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19514 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  child process with pid: 19515 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19515 exits with status 0
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  child process with pid: 19515 exited with success and will not be
> restarted
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  worker child process with pid: 19517 exits with status 256
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> LOG:  fork a new worker child process with pid: 19551
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19478:
> DEBUG:  reaper handler: exiting normally
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19551:
> DEBUG:  I am 19551
>
> Apr 18 16:04:12 postgres_server1 pgpool: 2017-04-18 16:04:12: pid 19551:
> DEBUG:  initializing backend status
>
> Apr 18 16:04:13 postgres_server1 pgpool: 2017-04-18 16:04:13: pid 19481:
> DEBUG:  watchdog exec interface up/down command: 'sudo arping -U $_IP_$ -w
> 1 -I bond1.18' succeeded
>
> Apr 18 16:04:14 postgres_server1 pgpool: 2017-04-18 16:04:14: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19481:
> DEBUG:  watchdog ping process for host "172.18.255.40" exited successfully
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19481:
> DEBUG:  watchdog ping
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19481:
> DETAIL:  ping data: PING 172.18.255.40 (172.18.255.40) 56(84) bytes of data.
>
> Apr 18 16:04:15 postgres_server1 pgpool: --- 172.18.255.40 ping statistics
> ---
>
> Apr 18 16:04:15 postgres_server1 pgpool: 3 packets transmitted, 3
> received, 0% packet loss, time 2000ms
>
> Apr 18 16:04:15 postgres_server1 pgpool: rtt min/avg/max/mdev =
> 0.026/0.029/0.035/0.006 ms
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19481:
> DEBUG:  watchdog succeeded to ping a host "172.18.255.40"
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19481:
> LOG:  successfully acquired the delegate IP:"172.18.255.40"
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19481:
> DETAIL:  'if_up_cmd' returned with success
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19479:
> DEBUG:  watchdog process signal handler
>
> Apr 18 16:04:15 postgres_server1 pgpool: 2017-04-18 16:04:15: pid 19479:
> LOG:  watchdog escalation process with pid: 19481 exit with SUCCESS.
>
> Apr 18 16:04:16 postgres_server1 pgpool: 2017-04-18 16:04:16: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:18 postgres_server1 pgpool: 2017-04-18 16:04:18: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:18 postgres_server1 pgpool: 2017-04-18 16:04:18: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:04:18 postgres_server1 pgpool: 2017-04-18 16:04:18: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:04:18 postgres_server1 pgpool: 2017-04-18 16:04:18: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:04:20 postgres_server1 pgpool: 2017-04-18 16:04:20: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:22 postgres_server1 pgpool: 2017-04-18 16:04:22: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:23 postgres_server1 pgpool: 2017-04-18 16:04:23: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:04:24 postgres_server1 pgpool: 2017-04-18 16:04:24: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:26 postgres_server1 pgpool: 2017-04-18 16:04:26: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:28 postgres_server1 pgpool: 2017-04-18 16:04:28: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:28 postgres_server1 pgpool: 2017-04-18 16:04:28: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:04:28 postgres_server1 pgpool: 2017-04-18 16:04:28: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:04:28 postgres_server1 pgpool: 2017-04-18 16:04:28: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:04:30 postgres_server1 pgpool: 2017-04-18 16:04:30: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:32 postgres_server1 pgpool: 2017-04-18 16:04:32: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:33 postgres_server1 pgpool: 2017-04-18 16:04:33: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:04:34 postgres_server1 pgpool: 2017-04-18 16:04:34: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:36 postgres_server1 pgpool: 2017-04-18 16:04:36: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:38 postgres_server1 pgpool: 2017-04-18 16:04:38: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:38 postgres_server1 pgpool: 2017-04-18 16:04:38: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:04:38 postgres_server1 pgpool: 2017-04-18 16:04:38: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:04:38 postgres_server1 pgpool: 2017-04-18 16:04:38: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:04:40 postgres_server1 pgpool: 2017-04-18 16:04:40: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:42 postgres_server1 pgpool: 2017-04-18 16:04:42: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:43 postgres_server1 pgpool: 2017-04-18 16:04:43: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:04:44 postgres_server1 pgpool: 2017-04-18 16:04:44: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:46 postgres_server1 pgpool: 2017-04-18 16:04:46: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:48 postgres_server1 pgpool: 2017-04-18 16:04:48: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:48 postgres_server1 pgpool: 2017-04-18 16:04:48: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:04:48 postgres_server1 pgpool: 2017-04-18 16:04:48: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:04:48 postgres_server1 pgpool: 2017-04-18 16:04:48: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:04:50 postgres_server1 pgpool: 2017-04-18 16:04:50: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:52 postgres_server1 pgpool: 2017-04-18 16:04:52: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:04:53 postgres_server1 pgpool: 2017-04-18 16:04:53: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:04:54 postgres_server1 pgpool: 2017-04-18 16:04:54: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:56 postgres_server1 pgpool: 2017-04-18 16:04:56: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:58 postgres_server1 pgpool: 2017-04-18 16:04:58: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:04:58 postgres_server1 pgpool: 2017-04-18 16:04:58: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:04:58 postgres_server1 pgpool: 2017-04-18 16:04:58: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:04:58 postgres_server1 pgpool: 2017-04-18 16:04:58: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:05:00 postgres_server1 pgpool: 2017-04-18 16:05:00: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:02 postgres_server1 pgpool: 2017-04-18 16:05:02: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:03 postgres_server1 pgpool: 2017-04-18 16:05:03: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:05:04 postgres_server1 pgpool: 2017-04-18 16:05:04: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:06 postgres_server1 pgpool: 2017-04-18 16:05:06: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:08 postgres_server1 pgpool: 2017-04-18 16:05:08: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:08 postgres_server1 pgpool: 2017-04-18 16:05:08: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:05:08 postgres_server1 pgpool: 2017-04-18 16:05:08: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:05:08 postgres_server1 pgpool: 2017-04-18 16:05:08: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:05:10 postgres_server1 pgpool: 2017-04-18 16:05:10: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:12 postgres_server1 pgpool: 2017-04-18 16:05:12: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:13 postgres_server1 pgpool: 2017-04-18 16:05:13: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:05:14 postgres_server1 pgpool: 2017-04-18 16:05:14: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:16 postgres_server1 pgpool: 2017-04-18 16:05:16: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:18 postgres_server1 pgpool: 2017-04-18 16:05:18: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:18 postgres_server1 pgpool: 2017-04-18 16:05:18: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:05:18 postgres_server1 pgpool: 2017-04-18 16:05:18: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:05:18 postgres_server1 pgpool: 2017-04-18 16:05:18: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:05:20 postgres_server1 pgpool: 2017-04-18 16:05:20: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19478:
> DEBUG:  starting health check
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19478:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19478:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19478:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:22 postgres_server1 pgpool: 2017-04-18 16:05:22: pid 19478:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:05:23 postgres_server1 pgpool: 2017-04-18 16:05:23: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:05:24 postgres_server1 pgpool: 2017-04-18 16:05:24: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:26 postgres_server1 pgpool: 2017-04-18 16:05:26: pid 19487:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:05:27 postgres_server1 systemd: pgpool.service start operation
> timed out. Terminating.
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19478:
> LOG:  received smart shutdown request
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19478:
> LOG:  shutdown request. closing listen socket
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19486:
> DEBUG:  watchdog heartbeat receiver child receives shutdown request signal
> 15
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19479:
> LOG:  Watchdog is shutting down
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19482:
> DEBUG:  lifecheck child receives shutdown request signal 15, forwarding to
> all children
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19487:
> DEBUG:  watchdog heartbeat sender child receives shutdown request signal 15
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19482:
> DEBUG:  lifecheck child receives smart shutdown request
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19479:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19479:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19479:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19550:
> DEBUG:  PCP child receives shutdown request signal 15, Forwarding to all
> children
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19550:
> DEBUG:  PCP child receives smart shutdown request
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19566:
> LOG:  watchdog: de-escalation started
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19482:
> DEBUG:  lifecheck child receives shutdown request signal 15, forwarding to
> all children
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19482:
> DEBUG:  lifecheck child receives smart shutdown request
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19566:
> DEBUG:  watchdog exec interface up/down command: 'sudo ifconfig bond1.18:0
> down' succeeded
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19566:
> LOG:  successfully released the delegate IP:"172.18.255.40"
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19566:
> DETAIL:  'if_down_cmd' returned with success
>
> Apr 18 16:05:27 postgres_server1 pgpool: 2017-04-18 16:05:27: pid 19478:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 16:05:27 postgres_server1 systemd: Failed to start Pgpool-II.
>
> Apr 18 16:05:27 postgres_server1 systemd: Unit pgpool.service entered
> failed state.
>
> Apr 18 16:05:27 postgres_server1 systemd: pgpool.service failed.
>
> Apr 18 16:05:29 postgres_server1 ntpd[2889]: Deleting interface #6
> bond1.18:0, 172.18.255.40#123, interface stats: received=0, sent=0,
> dropped=0, active_time=598 secs
>
> Apr 18 16:09:29 postgres_server1 systemd: Starting Pgpool-II...
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> DEBUG:  loading hba configuration
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> DETAIL:  loading file :"/etc/pgpool-II/pool_hba.conf" for client
> authentication configuration file
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> LOG:  reading status file: 0 th backend is set to down status
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> DEBUG:  pool_coninfo_size: num_init_children (25) * max_pool (4) *
> MAX_NUM_BACKENDS (128) * sizeof(ConnectionInfo) (136) = 1740800 bytes
> requested for shared memory
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> DEBUG:  ProcessInfo: num_init_children (25) * sizeof(ProcessInfo) (32) =
> 800 bytes requested for shared memory
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> DEBUG:  Request info are: sizeof(POOL_REQUEST_INFO) 5304 bytes requested
> for shared memory
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> DEBUG:  Recovery management area: sizeof(int) 4 bytes requested for shared
> memory
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19630:
> LOG:  waiting for watchdog to initialize
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> LOG:  setting the local watchdog node name to "172.18.255.41:9999 Linux
> postgres_server1"
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> LOG:  watchdog cluster is configured with 1 remote nodes
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> LOG:  watchdog remote node:0 on 172.18.255.42:9000
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> LOG:  interface monitoring is disabled in watchdog
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> INFO:  IPC socket path: "/tmp/.s.PGPOOLWD_CMD.9000"
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> LOG:  watchdog node state changed from [DEAD] to [LOADING]
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> LOADING
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> DEBUG:  error in outbond connection to 172.18.255.42:9000
>
> Apr 18 16:09:29 postgres_server1 pgpool: 2017-04-18 16:09:29: pid 19631:
> DETAIL:  Connection refused
>
> Apr 18 16:09:34 postgres_server1 pgpool: 2017-04-18 16:09:34: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = LOADING
>
> Apr 18 16:09:34 postgres_server1 pgpool: 2017-04-18 16:09:34: pid 19631:
> LOG:  watchdog node state changed from [LOADING] to [JOINING]
>
> Apr 18 16:09:34 postgres_server1 pgpool: 2017-04-18 16:09:34: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> JOINING
>
> Apr 18 16:09:34 postgres_server1 pgpool: 2017-04-18 16:09:34: pid 19631:
> DEBUG:  error in outbond connection to 172.18.255.42:9000
>
> Apr 18 16:09:34 postgres_server1 pgpool: 2017-04-18 16:09:34: pid 19631:
> DETAIL:  Connection refused
>
> Apr 18 16:09:38 postgres_server1 pgpool: 2017-04-18 16:09:38: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> JOINING
>
> Apr 18 16:09:38 postgres_server1 pgpool: 2017-04-18 16:09:38: pid 19631:
> LOG:  watchdog node state changed from [JOINING] to [INITIALIZING]
>
> Apr 18 16:09:38 postgres_server1 pgpool: 2017-04-18 16:09:38: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> INITIALIZING
>
> Apr 18 16:09:39 postgres_server1 pgpool: 2017-04-18 16:09:39: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State =
> INITIALIZING
>
> Apr 18 16:09:39 postgres_server1 pgpool: 2017-04-18 16:09:39: pid 19631:
> LOG:  I am the only alive node in the watchdog cluster
>
> Apr 18 16:09:39 postgres_server1 pgpool: 2017-04-18 16:09:39: pid 19631:
> HINT:  skiping stand for coordinator state
>
> Apr 18 16:09:39 postgres_server1 pgpool: 2017-04-18 16:09:39: pid 19631:
> LOG:  watchdog node state changed from [INITIALIZING] to [MASTER]
>
> Apr 18 16:09:39 postgres_server1 pgpool: 2017-04-18 16:09:39: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State =
> MASTER
>
> Apr 18 16:09:39 postgres_server1 pgpool: 2017-04-18 16:09:39: pid 19631:
> LOG:  I am announcing my self as master/coordinator watchdog node
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  declare coordinator command finished with status:[COMMAND TIMEED
> OUT]
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  I am the cluster leader node
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DETAIL:  our declare coordinator message is accepted by all nodes
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  I am the cluster leader node. Starting escalation process
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  watchdog process is initialized
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  escalation process started with PID:19643
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  new IPC connection received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19643:
> LOG:  watchdog: escalation started
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  received the get data request from local pgpool-II on IPC interface
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> NOTICE:  error writing to IPC socket
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  1:0: Unexpected
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  I am the master watchdog node
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  using the local backend node status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> DEBUG:  I am watchdog lifecheck child with pid:19644
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  new IPC connection received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:549
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> LOG:  2 watchdog nodes are configured for lifecheck
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> LOG:  watchdog nodes ID:0 Name:"172.18.255.41:9999 Linux postgres_server1"
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> DETAIL:  Host:"172.18.255.41" WD Port:9000 pgpool-II port:9999
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> LOG:  watchdog nodes ID:1 Name:"Not_Set"
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> DETAIL:  Host:"172.18.255.42" WD Port:9000 pgpool-II port:9999
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> DEBUG:  watchdog checking life check is ready
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19644:
> DETAIL:  pgpool:1 at "172.18.255.42:9999" has not send the heartbeat
> signal yet
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  Setting up socket for 0.0.0.0:9999
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  Setting up socket for :::9999
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  perhaps failed to create INET domain socket
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  socket(::) failed: "Address family not supported by protocol"
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19652:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19653:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19654:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19655:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19656:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19657:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19658:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19659:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19660:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19661:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19662:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19663:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19664:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19665:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19666:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19667:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19668:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19670:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19671:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19672:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19673:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  network event received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DETAIL:  deleted = NO Link change event = NO
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = NETWORK IP IS ASSIGNED Current
> State = MASTER
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  network event received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DETAIL:  deleted = YES Link change event = NO
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = NETWORK IP IS REMOVED Current
> State = MASTER
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19674:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19675:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  network IP is removed but system still has a valid IP is assigned
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  IP = 172.19.255.41
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  IP = 172.18.255.41
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  IP = 172.18.255.40
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19676:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19677:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  network event received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DETAIL:  deleted = NO Link change event = NO
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = NETWORK IP IS ASSIGNED Current
> State = MASTER
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19643:
> DEBUG:  watchdog exec interface up/down command: 'sudo ifconfig bond1.18:0
> inet $_IP_$ netmask 255.255.255.0' succeeded
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19678:
> DEBUG:  I am PCP child with pid:19678
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  pgpool-II successfully started. version 3.6.2 (subaruboshi)
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  find_primary_node: checking backend no 0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  find_primary_node: checking backend no 1
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19680:
> DEBUG:  I am 19680
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19680:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  find_primary_node: no primary node found
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19680:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19680:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19680:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  starting health check
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  Backend DB node 1 status is 2
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  Trying to make persistent DB connection to backend node 1 having
> status 2
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  doing health check against database:template1 user:postgres
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  Backend DB node 1 status is 2
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  Trying to make persistent DB connection to backend node 1 having
> status 2
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  failed to connect to PostgreSQL server on "172.18.255.42:5432",
> getsockopt() detected error "Connection refused"
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> ERROR:  failed to make persistent db connection
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  connection to host:"172.18.255.42:5432" failed
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  setting backend node 1 status to NODE DOWN
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  received degenerate backend request for node_id: 1 from pid [19630]
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  new IPC connection received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  watchdog received the failover command from local pgpool-II on IPC
> interface
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  watchdog is processing the failover command
> [DEGENERATE_BACKEND_REQUEST] received from local pgpool-II on IPC interface
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  I am the only pgpool-II node in the watchdog cluster
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DETAIL:  no need to propagate the failover command
> [DEGENERATE_BACKEND_REQUEST]
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler called
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kind: 1 flags: 1 node_count: 1 index:0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  new IPC connection received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to become a lock holder for failover ID: 0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> the lock holder
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  starting to select new master node
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  starting degeneration. shutdown host 172.18.255.42(5432)
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  failover: no valid backends node found
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover/failback request details: STREAM: 1 reqkind: 1 detail: 1
> node_id: 1
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  Restart all children
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19652
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19653
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19654
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19655
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19656
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19657
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19658
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19659
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19660
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19661
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19662
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19663
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19664
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19665
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19666
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19667
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19668
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19670
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19671
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19672
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19673
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19674
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19675
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19676
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DEBUG:  failover handler
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> DETAIL:  kill process with PID:19677
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  execute command: /etc/pgpool/failover.sh 1 172.18.255.42 5432
> /postgres_db/data -1 1 "" 1 "" "" > AAA_failover_called.log
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  new IPC connection received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to release [FAILOVER] lock for failover ID 0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has released the [FAILOVER] lock for failover ID 0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  failover: no follow backends are degenerated
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  new IPC connection received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to release [FOLLOW MASTER] lock for failover ID 0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has released the [FOLLOW MASTER] lock for failover ID 0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19630:
> LOG:  failover: set new primary node: -1
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19687:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19688:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19689:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19690:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19691:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19692:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19693:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19694:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19695:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19696:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19697:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19698:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19699:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19700:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19701:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19702:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19703:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19704:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19705:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19706:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19707:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19708:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19709:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19710:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19680:
> LOG:  worker process received restart request
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  new IPC connection received
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  received the failover command lock request from local pgpool-II on
> IPC interface
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1" is
> requesting to resign from a lock holder for failover ID 0
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> LOG:  local pgpool-II node "172.18.255.41:9999 Linux postgres_server1"
> has resigned from the lock holder
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19631:
> DEBUG:  sending watchdog packet Socket:7, Type:[NULL], Command_ID:0, data
> Length:54
>
> Apr 18 16:09:43 postgres_server1 pgpool: failover done. shutdown host
> 172.18.255.42(5432)2017-04-18 16:09:43: pid 19630: LOG:  failover done.
> shutdown host 172.18.255.42(5432)
>
> Apr 18 16:09:43 postgres_server1 pgpool: 2017-04-18 16:09:43: pid 19711:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19648:
> LOG:  set SO_REUSEPORT option to the socket
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19648:
> LOG:  creating watchdog heartbeat receive socket.
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19648:
> DETAIL:  set SO_REUSEPORT
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19649:
> LOG:  set SO_REUSEPORT option to the socket
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19649:
> LOG:  creating socket for sending heartbeat
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19649:
> DETAIL:  set SO_REUSEPORT
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19678:
> LOG:  restart request received in pcp child process
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19678:
> DEBUG:  PCP child receives shutdown request signal 15, Forwarding to all
> children
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19678:
> DEBUG:  PCP child receives smart shutdown request
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  PCP child 19678 exits with status 0 in failover()
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  fork a new PCP child pid 19712 in failover()
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  starting health check
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  reaper handler
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19712:
> DEBUG:  I am PCP child with pid:19712
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19652 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19652 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19652 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19653 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19653 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19653 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19654 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19654 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19654 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19655 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19655 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19655 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19656 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19656 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19656 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19657 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19657 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19657 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19658 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19658 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19658 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19659 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19659 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19659 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19660 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19660 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19660 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19661 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19661 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19661 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19662 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19662 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19662 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19663 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19663 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19663 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19664 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19664 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19664 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19665 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19665 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19665 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19666 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19666 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19666 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19667 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19667 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19667 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19668 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19668 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19668 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19670 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19670 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19670 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19671 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19671 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19671 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19672 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19672 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19672 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19673 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19673 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19673 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19674 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19674 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19674 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19675 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19675 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19675 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19676 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19676 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19676 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  child process with pid: 19677 exit with SUCCESS. child will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19677 exits with status 0
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  child process with pid: 19677 exited with success and will not be
> restarted
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  worker child process with pid: 19680 exits with status 256
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> LOG:  fork a new worker child process with pid: 19713
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19630:
> DEBUG:  reaper handler: exiting normally
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19713:
> DEBUG:  I am 19713
>
> Apr 18 16:09:44 postgres_server1 pgpool: 2017-04-18 16:09:44: pid 19713:
> DEBUG:  initializing backend status
>
> Apr 18 16:09:44 postgres_server1 ntpd[2889]: Listen normally on 7
> bond1.18:0 172.18.255.40 UDP 123
>
> Apr 18 16:09:45 postgres_server1 pgpool: 2017-04-18 16:09:45: pid 19643:
> DEBUG:  watchdog exec interface up/down command: 'sudo arping -U $_IP_$ -w
> 1 -I bond1.18' succeeded
>
> Apr 18 16:09:46 postgres_server1 pgpool: 2017-04-18 16:09:46: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19643:
> DEBUG:  watchdog ping process for host "172.18.255.40" exited successfully
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19643:
> DEBUG:  watchdog ping
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19643:
> DETAIL:  ping data: PING 172.18.255.40 (172.18.255.40) 56(84) bytes of data.
>
> Apr 18 16:09:47 postgres_server1 pgpool: --- 172.18.255.40 ping statistics
> ---
>
> Apr 18 16:09:47 postgres_server1 pgpool: 3 packets transmitted, 3
> received, 0% packet loss, time 1999ms
>
> Apr 18 16:09:47 postgres_server1 pgpool: rtt min/avg/max/mdev =
> 0.026/0.027/0.029/0.004 ms
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19643:
> DEBUG:  watchdog succeeded to ping a host "172.18.255.40"
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19643:
> LOG:  successfully acquired the delegate IP:"172.18.255.40"
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19643:
> DETAIL:  'if_up_cmd' returned with success
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19631:
> DEBUG:  watchdog process signal handler
>
> Apr 18 16:09:47 postgres_server1 pgpool: 2017-04-18 16:09:47: pid 19631:
> LOG:  watchdog escalation process with pid: 19643 exit with SUCCESS.
>
> Apr 18 16:09:48 postgres_server1 pgpool: 2017-04-18 16:09:48: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:09:50 postgres_server1 pgpool: 2017-04-18 16:09:50: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:09:50 postgres_server1 pgpool: 2017-04-18 16:09:50: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:09:50 postgres_server1 pgpool: 2017-04-18 16:09:50: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:09:50 postgres_server1 pgpool: 2017-04-18 16:09:50: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:09:52 postgres_server1 pgpool: 2017-04-18 16:09:52: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:09:54 postgres_server1 pgpool: 2017-04-18 16:09:54: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:09:55 postgres_server1 pgpool: 2017-04-18 16:09:55: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:09:56 postgres_server1 pgpool: 2017-04-18 16:09:56: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:09:58 postgres_server1 pgpool: 2017-04-18 16:09:58: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:00 postgres_server1 pgpool: 2017-04-18 16:10:00: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:00 postgres_server1 pgpool: 2017-04-18 16:10:00: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:10:00 postgres_server1 pgpool: 2017-04-18 16:10:00: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:10:00 postgres_server1 pgpool: 2017-04-18 16:10:00: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:10:01 postgres_server1 systemd: Started Session 15 of user root.
>
> Apr 18 16:10:01 postgres_server1 systemd: Starting Session 15 of user root.
>
> Apr 18 16:10:02 postgres_server1 pgpool: 2017-04-18 16:10:02: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:04 postgres_server1 pgpool: 2017-04-18 16:10:04: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:05 postgres_server1 pgpool: 2017-04-18 16:10:05: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:10:06 postgres_server1 pgpool: 2017-04-18 16:10:06: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:08 postgres_server1 pgpool: 2017-04-18 16:10:08: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:10 postgres_server1 pgpool: 2017-04-18 16:10:10: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:10 postgres_server1 pgpool: 2017-04-18 16:10:10: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:10:10 postgres_server1 pgpool: 2017-04-18 16:10:10: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:10:10 postgres_server1 pgpool: 2017-04-18 16:10:10: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:10:12 postgres_server1 pgpool: 2017-04-18 16:10:12: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19630:
> DEBUG:  starting health check
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19630:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19630:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19630:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:10:14 postgres_server1 pgpool: 2017-04-18 16:10:14: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:10:15 postgres_server1 pgpool: 2017-04-18 16:10:15: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:10:16 postgres_server1 pgpool: 2017-04-18 16:10:16: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:18 postgres_server1 pgpool: 2017-04-18 16:10:18: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:20 postgres_server1 pgpool: 2017-04-18 16:10:20: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:20 postgres_server1 pgpool: 2017-04-18 16:10:20: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:10:20 postgres_server1 pgpool: 2017-04-18 16:10:20: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:10:20 postgres_server1 pgpool: 2017-04-18 16:10:20: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:10:22 postgres_server1 pgpool: 2017-04-18 16:10:22: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:24 postgres_server1 pgpool: 2017-04-18 16:10:24: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:25 postgres_server1 pgpool: 2017-04-18 16:10:25: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:10:26 postgres_server1 pgpool: 2017-04-18 16:10:26: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:28 postgres_server1 pgpool: 2017-04-18 16:10:28: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:30 postgres_server1 pgpool: 2017-04-18 16:10:30: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:30 postgres_server1 pgpool: 2017-04-18 16:10:30: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:10:30 postgres_server1 pgpool: 2017-04-18 16:10:30: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:10:30 postgres_server1 pgpool: 2017-04-18 16:10:30: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:10:32 postgres_server1 pgpool: 2017-04-18 16:10:32: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:34 postgres_server1 pgpool: 2017-04-18 16:10:34: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:35 postgres_server1 pgpool: 2017-04-18 16:10:35: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:10:36 postgres_server1 pgpool: 2017-04-18 16:10:36: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:38 postgres_server1 pgpool: 2017-04-18 16:10:38: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:40 postgres_server1 pgpool: 2017-04-18 16:10:40: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:40 postgres_server1 pgpool: 2017-04-18 16:10:40: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:10:40 postgres_server1 pgpool: 2017-04-18 16:10:40: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:10:40 postgres_server1 pgpool: 2017-04-18 16:10:40: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:10:42 postgres_server1 pgpool: 2017-04-18 16:10:42: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19630:
> DEBUG:  starting health check
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19630:
> DEBUG:  doing health check against database:postgres user:postgres
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19630:
> DEBUG:  Backend DB node 0 status is 3
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19630:
> DEBUG:  Backend DB node 1 status is 3
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:10:44 postgres_server1 pgpool: 2017-04-18 16:10:44: pid 19630:
> DEBUG:  health check: clearing alarm
>
> Apr 18 16:10:45 postgres_server1 pgpool: 2017-04-18 16:10:45: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:10:46 postgres_server1 pgpool: 2017-04-18 16:10:46: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:48 postgres_server1 pgpool: 2017-04-18 16:10:48: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:50 postgres_server1 pgpool: 2017-04-18 16:10:50: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:50 postgres_server1 pgpool: 2017-04-18 16:10:50: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:10:50 postgres_server1 pgpool: 2017-04-18 16:10:50: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:10:50 postgres_server1 pgpool: 2017-04-18 16:10:50: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:10:52 postgres_server1 pgpool: 2017-04-18 16:10:52: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:54 postgres_server1 pgpool: 2017-04-18 16:10:54: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:55 postgres_server1 pgpool: 2017-04-18 16:10:55: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = TIMEOUT Current State = MASTER
>
> Apr 18 16:10:56 postgres_server1 pgpool: 2017-04-18 16:10:56: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:58 postgres_server1 pgpool: 2017-04-18 16:10:58: pid 19649:
> DEBUG:  watchdog heartbeat: send heartbeat signal to 172.18.255.42:9694
>
> Apr 18 16:10:59 postgres_server1 systemd: pgpool.service start operation
> timed out. Terminating.
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19630:
> LOG:  received smart shutdown request
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19630:
> LOG:  shutdown request. closing listen socket
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19648:
> DEBUG:  watchdog heartbeat receiver child receives shutdown request signal
> 15
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19644:
> DEBUG:  lifecheck child receives shutdown request signal 15, forwarding to
> all children
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19649:
> DEBUG:  watchdog heartbeat sender child receives shutdown request signal 15
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19631:
> LOG:  Watchdog is shutting down
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19644:
> DEBUG:  lifecheck child receives smart shutdown request
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19631:
> DEBUG:  STATE MACHINE INVOKED WITH EVENT = COMMAND FINISHED Current State =
> MASTER
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19631:
> DEBUG:  I am the cluster leader node command finished with status:[COMMAND
> TIMEED OUT] which is success
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19631:
> DETAIL:  The command was sent to 0 nodes and 0 nodes replied to it
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19644:
> DEBUG:  lifecheck child receives shutdown request signal 15, forwarding to
> all children
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19644:
> DEBUG:  lifecheck child receives smart shutdown request
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19712:
> DEBUG:  PCP child receives shutdown request signal 15, Forwarding to all
> children
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19712:
> DEBUG:  PCP child receives smart shutdown request
>
> Apr 18 16:10:59 postgres_server1 pgpool: 2017-04-18 16:10:59: pid 19630:
> WARNING:  All the DB nodes are in down status and skip writing status file.
>
> Apr 18 16:10:59 postgres_server1 systemd: Failed to start Pgpool-II.
>
> Apr 18 16:10:59 postgres_server1 systemd: Unit pgpool.service entered
> failed state.
>
> Apr 18 16:10:59 postgres_server1 systemd: pgpool.service failed.
>
> Apr 18 16:12:44 postgres_server1 systemd-logind: New session 16 of user
> root.
>
> Apr 18 16:12:44 postgres_server1 systemd: Started Session 16 of user root.
>
> Apr 18 16:12:44 postgres_server1 systemd: Starting Session 16 of user root.
> IMPORTANT - This email and any attachments is intended for the above named
> addressee(s), and may contain information which is confidential or
> privileged. If you are not the intended recipient, please inform the sender
> immediately and delete this email: you should not copy or use this e-mail
> for any purpose nor disclose its contents to any person.
>
> _______________________________________________
> pgpool-general mailing list
> pgpool-general at pgpool.net
> http://www.pgpool.net/mailman/listinfo/pgpool-general
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20170419/0c3867cb/attachment-0001.html>


More information about the pgpool-general mailing list