Aug 22 11:49:49 dld220822093130 pgpool[44725]: [23-1] health_check0: LOG: pool_ssl: "SSL_read": "no SSL error reported" Aug 22 11:49:49 dld220822093130 pgpool[44725]: [24-1] health_check0: LOG: received degenerate backend request for node_id: 0 from pid [44725] Aug 22 11:49:49 dld220822093130 pgpool[44685]: [35-1] watchdog: LOG: new IPC connection received Aug 22 11:49:49 dld220822093130 pgpool[44726]: [23-1] health_check1: LOG: pool_ssl: "SSL_read": "no SSL error reported" Aug 22 11:49:49 dld220822093130 pgpool[44726]: [24-1] health_check1: LOG: received degenerate backend request for node_id: 1 from pid [44726] Aug 22 11:49:49 dld220822093130 pgpool[44685]: [36-1] watchdog: LOG: failover request from local pgpool-II node received on IPC interface is forwarded to leader watchdog node "dld220822093125:9999 Linux dld220822093125" Aug 22 11:49:49 dld220822093130 pgpool[44685]: [36-2] watchdog: DETAIL: waiting for the reply... Aug 22 11:49:49 dld220822093130 pgpool[44685]: [37-1] watchdog: LOG: new IPC connection received Aug 22 11:49:49 dld220822093130 pgpool[44685]: [38-1] watchdog: LOG: failover request from local pgpool-II node received on IPC interface is forwarded to leader watchdog node "dld220822093125:9999 Linux dld220822093125" Aug 22 11:49:49 dld220822093130 pgpool[44685]: [38-2] watchdog: DETAIL: waiting for the reply... Aug 22 11:49:49 dld220822093130 pgpool[44725]: [25-1] health_check0: LOG: degenerate backend request for 1 node(s) from pid [44725], will be handled by watchdog Aug 22 11:49:49 dld220822093130 pgpool[44726]: [25-1] health_check1: LOG: degenerate backend request for 1 node(s) from pid [44726], will be handled by watchdog Aug 22 11:49:49 dld220822093130 pgpool[44725]: [27-1] health_check0: LOG: health check retrying on DB node: 0 (round:1) Aug 22 11:49:49 dld220822093130 pgpool[44726]: [27-1] health_check1: LOG: health check retrying on DB node: 1 (round:1) Aug 22 11:49:50 dld220822093130 pgpool[44725]: [28-1] health_check0: LOG: health check retrying on DB node: 0 succeeded Aug 22 11:49:50 dld220822093130 pgpool[44726]: [28-1] health_check1: LOG: health check retrying on DB node: 1 succeeded Aug 22 11:49:50 dld220822093130 pgpool[44711]: [39-1] psql: LOG: connection closed. Aug 22 11:49:50 dld220822093130 pgpool[44711]: [39-2] psql: DETAIL: retry to create new connection pool Aug 22 11:49:50 dld220822093130 pgpool[44711]: [40-1] psql: LOG: pool_read_kind: error message from main backend:the database system is starting up Aug 22 11:49:50 dld220822093130 pgpool[44711]: [42-1] psql: LOG: pool_read_kind: error message from main backend:the database system is starting up Aug 22 11:49:50 dld220822093130 pgpool[44685]: [39-1] watchdog: LOG: signal_user1_to_parent_with_reason(2) Aug 22 11:49:50 dld220822093130 pgpool[44682]: [25-1] main: LOG: Pgpool-II parent process received SIGUSR1 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [26-1] main: LOG: Pgpool-II parent process received sync backend signal from watchdog Aug 22 11:49:50 dld220822093130 pgpool[44685]: [40-1] watchdog: LOG: new IPC connection received Aug 22 11:49:50 dld220822093130 pgpool[44682]: [27-1] main: LOG: leader watchdog has performed failover Aug 22 11:49:50 dld220822093130 pgpool[44682]: [27-2] main: DETAIL: syncing the backend states from the LEADER watchdog node Aug 22 11:49:50 dld220822093130 pgpool[44685]: [41-1] watchdog: LOG: new IPC connection received Aug 22 11:49:50 dld220822093130 pgpool[44685]: [42-1] watchdog: LOG: received the get data request from local pgpool-II on IPC interface Aug 22 11:49:50 dld220822093130 pgpool[44685]: [43-1] watchdog: LOG: get data request from local pgpool-II node received on IPC interface is forwarded to leader watchdog node "dld220822093125:9999 Linux dld220822093125" Aug 22 11:49:50 dld220822093130 pgpool[44685]: [43-2] watchdog: DETAIL: waiting for the reply... Aug 22 11:49:50 dld220822093130 pgpool[44682]: [28-1] main: LOG: leader watchdog node "dld220822093125:9999 Linux dld220822093125" returned status for 2 backend nodes Aug 22 11:49:50 dld220822093130 pgpool[44682]: [29-1] main: LOG: backend:0 is set to down status Aug 22 11:49:50 dld220822093130 pgpool[44682]: [29-2] main: DETAIL: backend:0 is DOWN on cluster leader "dld220822093125:9999 Linux dld220822093125" Aug 22 11:49:50 dld220822093130 pgpool[44682]: [30-1] main: LOG: backend:1 is set to down status Aug 22 11:49:50 dld220822093130 pgpool[44682]: [30-2] main: DETAIL: backend:1 is DOWN on cluster leader "dld220822093125:9999 Linux dld220822093125" Aug 22 11:49:50 dld220822093130 pgpool[44682]: [31-1] main: LOG: primary node:1 on leader watchdog node "dld220822093125:9999 Linux dld220822093125" is different from local primary node:0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [32-1] main: LOG: primary node was changed after the sync from "dld220822093125:9999 Linux dld220822093125" Aug 22 11:49:50 dld220822093130 pgpool[44682]: [32-2] main: DETAIL: all children needs to be restarted Aug 22 11:49:50 dld220822093130 pgpool[44682]: [33-1] main: LOG: child process with pid: 44693 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [34-1] main: LOG: child process with pid: 44694 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [35-1] main: LOG: child process with pid: 44695 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [36-1] main: LOG: child process with pid: 44699 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [37-1] main: LOG: child process with pid: 44700 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [38-1] main: LOG: child process with pid: 44701 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44724]: [23-1] sr_check_worker: LOG: worker process received restart request Aug 22 11:49:50 dld220822093130 pgpool[44682]: [39-1] main: LOG: child process with pid: 44691 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [41-1] main: LOG: child process with pid: 44696 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [42-1] main: LOG: child process with pid: 44697 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [43-1] main: LOG: child process with pid: 44698 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [44-1] main: LOG: child process with pid: 44703 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [49-1] main: LOG: child process with pid: 44711 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [50-1] main: LOG: child process with pid: 44712 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [51-1] main: LOG: child process with pid: 44713 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [52-1] main: LOG: child process with pid: 44714 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [53-1] main: LOG: child process with pid: 44715 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [55-1] main: LOG: child process with pid: 44717 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [59-1] main: LOG: child process with pid: 44721 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [60-1] main: LOG: child process with pid: 44722 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [61-1] main: LOG: worker child process with pid: 44724 exits with status 256 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [62-1] main: LOG: fork a new worker child process with pid: 47177 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [63-1] main: LOG: child process with pid: 44710 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [65-1] main: LOG: child process with pid: 44705 exits with status 0 Aug 22 11:49:50 dld220822093130 pgpool[47177]: [62-1] sr_check_worker: LOG: process started Aug 22 11:49:50 dld220822093130 pgpool[44685]: [44-1] watchdog: LOG: new IPC connection received Aug 22 11:49:50 dld220822093130 pgpool[44685]: [45-1] watchdog: LOG: processing follow primary looking[acquire] request from remote node "dld220822093125:9999 Linux dld220822093125" Aug 22 11:49:50 dld220822093130 pgpool[44682]: [67-1] main: LOG: child process with pid: 47162 exits with status 256 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [68-1] main: LOG: fork a new child process with pid: 47180 Aug 22 11:49:50 dld220822093130 pgpool[44685]: [46-1] watchdog: LOG: signal_user1_to_parent_with_reason(2) Aug 22 11:49:50 dld220822093130 pgpool[44682]: [69-1] main: LOG: Pgpool-II parent process received SIGUSR1 Aug 22 11:49:50 dld220822093130 pgpool[44682]: [70-1] main: LOG: Pgpool-II parent process received sync backend signal from watchdog Aug 22 11:49:50 dld220822093130 pgpool[44685]: [47-1] watchdog: LOG: new IPC connection received Aug 22 11:49:50 dld220822093130 pgpool[44682]: [71-1] main: LOG: leader watchdog has performed failover Aug 22 11:49:50 dld220822093130 pgpool[44682]: [71-2] main: DETAIL: syncing the backend states from the LEADER watchdog node