View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000823 | Pgpool-II | Bug | public | 2023-12-21 18:30 | 2024-06-11 20:22 |
| Reporter | openminded | Assigned To | Muhammad Usama | ||
| Priority | normal | Severity | major | Reproducibility | always |
| Status | assigned | Resolution | open | ||
| Product Version | 4.3.7 | ||||
| Summary | 0000823: Watchdog dies and kills pgpool2 when network gets shortly interrupted | ||||
| Description | The watchdog for pgpool instantly sets status to NETWORK TROUBLE if network gets restarted and then kills pgpool . Logs show this and even when a new process starts, pgpool gets killed. This seems to contradict with setting for retry. It happens in query and other mode. log shows : Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b systemd[1]: Started Network Configuration. Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b systemd[1]: Starting Wait for Network to be Configured... Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[16381]: [38-1] 2023-12-15 06:18:46: pid 16381: WARNING: network IP is removed and system has no IP is assigned Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[16381]: [38-2] 2023-12-15 06:18:46: pid 16381: DETAIL: changing the state to in network trouble Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[16381]: [39-1] 2023-12-15 06:18:46: pid 16381: LOG: watchdog node state changed from [STANDBY] to [IN NETWORK TROUBLE] Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[16381]: [40-1] 2023-12-15 06:18:46: pid 16381: FATAL: system has lost the network Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[16381]: [41-1] 2023-12-15 06:18:46: pid 16381: LOG: Watchdog is shutting down Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[16372]: [24-1] 2023-12-15 06:18:46: pid 16372: LOG: watchdog child process with pid: 16381 exits with status 768 Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[16372]: [25-1] 2023-12-15 06:18:46: pid 16372: FATAL: watchdog child process exit with fatal error. exiting pgpool-II Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[204092]: [25-1] 2023-12-15 06:18:46: pid 204092: LOG: setting the local watchdog node name to "pgsql01:9999 Linux stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b" Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[204092]: [26-1] 2023-12-15 06:18:46: pid 204092: LOG: watchdog cluster is configured with 2 remote nodes Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[204092]: [27-1] 2023-12-15 06:18:46: pid 204092: LOG: watchdog remote node:0 on pgsql02:9000 Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[204092]: [28-1] 2023-12-15 06:18:46: pid 204092: LOG: watchdog remote node:1 on pgsql03:9000 Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[204092]: [29-1] 2023-12-15 06:18:46: pid 204092: LOG: ensure availability on any interface Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[204092]: [30-1] 2023-12-15 06:18:46: pid 204092: INFO: IPC socket path: "/var/run/postgresql/.s.PGPOOLWD_CMD.9000" Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[204092]: [31-1] 2023-12-15 06:18:46: pid 204092: LOG: connect on socket failed Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[204092]: [31-2] 2023-12-15 06:18:46: pid 204092: DETAIL: connect failed with error: "Network is unreachable" Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[204092]: [32-1] 2023-12-15 06:18:46: pid 204092: LOG: connect on socket failed Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[204092]: [32-2] 2023-12-15 06:18:46: pid 204092: DETAIL: connect failed with error: "Network is unreachable" Dec 15 06:18:46 stack1-config-barssl-3-hostany-1-psql-1896c992-3e17-4fab-b084-b pgpool[204092]: [33-1] 2023-12-15 06:18:46: pid 204092: LOG: watchdog node state changed from [DEAD] to [LOADING] | ||||
| Steps To Reproduce | - setup pgpool - disrupt network, for ubuntu 22.04 : ip link set ens3 down && ip link set ens3 up ( where ens3 is the interface where it listens ) - see logs In my case this was caused by the standard apt daily job that is part of the cloud image, which seems to do things that result in a restart of the network. Workaround ( with Ansible ): the daily- name: disable systemd timers that run autoupgrade - prevent restart network ansible.builtin.shell: cmd: | systemctl mask apt-daily.service apt-daily-upgrade.service systemctl daemon-reload executable: /bin/bash become: true become_user: "root" ignore_errors: true This masks the daily job. | ||||
| Additional Information | It makes sense to kill pgpool if it would be isolated, if passing a certain time. Such short interrupted is wishful to be acceptible.- One option would be to have the systemd file make it restart, after all why would the watchdog need to be down ? This can be established by adding these lines to the systemd service for pgpool : Restart=always RestartPreventExitStatus=2 3 Exit status is 1 in this case so it will restart. other exit statuses could be omitted if needed. | ||||
| Tags | watchdog | ||||
|
|
hi, any updates on this? Is this behaviour supposed to be according to the settings for retry, timeout etc? Is anything else needed from my side ? |
|
|
Hi Vincent, I'm sorry for not getting back to you sooner. This report somehow fell through the cracks. Have you enabled interface monitoring by setting wd_monitoring_interfaces_list? Pgpool-II kills itself when it detects a network blackout to ensure that the backend health check does not proceed with a failover, as the backend health check would also see the same network failure. This is an extra precautionary measure to prevent false failovers. However, suppose you are confident that your backend health check configurations can tolerate temporary network issues or restarts. In that case, you can disable network monitoring, and Pgpool-II will not act on network restarts. To disable it, just set wd_monitoring_interfaces_list to '' or comment out this configuration in 'pgpool.conf' and restart the pgpool. |
|
|
hi Muhammad, thank you for getting back to me. Yes that options is set to any and I tried others too. I am not sure if I am following. The situation I am describing is a restart of the network interface, which takes a very short time. there are a variety of variables that seem to be present to make sure there is no direct failure notification at a small hiccup, like: wd_lifecheck_method = 'heartbeat' # Method of watchdog lifecheck ('heartbeat' or 'query' or 'external') wd_interval = 10 # lifecheck interval (sec) > 0 wd_heartbeat_keepalive = 2 # Interval time of sending heartbeat signal (sec) wd_heartbeat_deadtime = 30 First of all, it seems wrong to me that the process gets killed at network restart - there needs to be a way to prevent that because it can happen by automatic updates ( Ubuntu ). Second, the above settings seem to indicate that the way to prevent it is to have the values provide the tolerance. Did I misunderstand? To me query or heartbeat do not work, my observation is that something just instantly kills the process as you describe. What should be done to prevent that from happening ? Thank you and regards, Vincent |
|
|
This particular situation has nothing to do with wd_lifecheck settings. Basically, you can enable interface monitoring on Pgpool to check the status of network interfaces. If that monitoring is enabled, as soon as all network connections become unavailable, the Pgpool node commits suicide. Having said that this network interface monitoring can be safely disabled by un-setting wd_monitoring_interfaces_list parameter in pgpool.conf Can you please tell me what your current value is for the wd_monitoring_interfaces_list parameter? |
|
|
hi, wd_monitoring_interfaces_list is set to 'any' currently. I didn't expect this suicide - would you consider applying settings like the one to regulate the behavior? Since the above parameters are not related, what do they apply to? I do want a failover if actually network issues occur... thank you! |
|
|
Even if you disable network monitoring, Pgpool-II will still trigger a failover when the communication channel breaks, depending on your health check and life check configurations. Network monitoring is an additional feature to quickly shut down Pgpool-II on a node that loses all network interfaces. It is optional and not the only way Pgpool-II detects communication failures. By setting wd_monitoring_interfaces_list to '', you can disable network monitoring, and both backend and watchdog failovers will still function correctly. This parameter is recommended if you do not expect network blackouts during normal operations and want to quickly kill pgpool on a node with a failed network interface. We will also look into making the actions on network blackouts more configurable. |
|
|
hi, ok thank you for clarifying that, that wasn't clear from the information I found. Wouldn't you expect the mentioned settings to be applicable to the network too? It would be great if we can have the same or perhaps better similar options to handle a network error. after setting the wd_monitoring_interfaces_list to '', I still have the same problem : root@test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208:~# systemctl status pgpool2 × pgpool2.service - pgpool-II Loaded: loaded (/lib/systemd/system/pgpool2.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Fri 2024-05-31 09:24:47 UTC; 6s ago Docs: man:pgpool(8) Process: 628 ExecStart=/usr/sbin/pgpool -n (code=exited, status=1/FAILURE) Main PID: 628 (code=exited, status=1/FAILURE) CPU: 1.068s May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [36-1] 2024-05-31 09:24:46: pid 1482: LOG: connect on socket failed May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [36-2] 2024-05-31 09:24:46: pid 1482: DETAIL: connect failed with error: "Network is un> May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [37-1] 2024-05-31 09:24:46: pid 1482: LOG: connect on socket failed May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [37-2] 2024-05-31 09:24:46: pid 1482: DETAIL: connect failed with error: "Network is un> May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [38-1] 2024-05-31 09:24:46: pid 1482: LOG: watchdog node state changed from [DEAD] to [> May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [31-1] 2024-05-31 09:24:46: pid 628: LOG: shutting down May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [39-1] 2024-05-31 09:24:46: pid 1482: LOG: Watchdog is shutting down May 31 09:24:47 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Main process exited, code=exited, status=1/FAILURE May 31 09:24:47 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Failed with result 'exit-code'. May 31 09:24:47 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Consumed 1.068s CPU time. I think I actually tried this before. So it seems to me that the mentioned options do not work at all in this case ? After a simple restart it goes back to running: root@test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208:~# systemctl status pgpool2 ● pgpool2.service - pgpool-II Loaded: loaded (/lib/systemd/system/pgpool2.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2024-05-31 09:27:34 UTC; 1s ago The parameters: wd_monitoring_interfaces_list = '' # Comma separated list of interfaces names to monitor. wd_lifecheck_method = 'heartbeat' wd_interval = 10 wd_heartbeat_keepalive = 2 wd_heartbeat_deadtime = 30 heartbeat_hostname0 = 'pgsql01' heartbeat_port0 = '9694' heartbeat_device0 = '' heartbeat_hostname1 = 'pgsql02' heartbeat_port1 = '9694' heartbeat_device1 = '' heartbeat_hostname2 = 'pgsql03' Then : sudo ip link set ens3 down && sudo ip link set ens3 up root@test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208:~# systemctl status pgpool2 × pgpool2.service - pgpool-II Loaded: loaded (/lib/systemd/system/pgpool2.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Fri 2024-05-31 09:30:09 UTC; 2s ago Docs: man:pgpool(8) So the network interruption still kills the process and does not bring it back. Again, this is where I would expect the parameters to apply, so unless I am missing something this looks like a bug to me. To be sure, I removed 'any' from all 3 nodes in the config and restarted, the problem is the same. |
|
|
Can you share the complete pgpool log and pgpool.conf file? That will be helpful to identify the cause |
|
|
hi, Please find the files attached. Can you try on a local instance if you can reproduce this? I can't imagine this happens on all instances. Greetings Vincent pgpool.log.txt (214,481 bytes)
May 31 04:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1831927]: [15199-1] 2024-05-31 04:50:29: pid 1831927: LOG: statement: SHOW POOL_NODES; May 31 04:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1831927]: [15200-1] 2024-05-31 04:50:29: pid 1831927: LOG: DB node id: 0 backend pid: 2125689 statement: SELECT pg_catalog.version() May 31 04:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1831927]: [15201-1] 2024-05-31 04:50:29: pid 1831927: LOG: statement: DISCARD ALL May 31 04:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1831927]: [15202-1] 2024-05-31 04:50:29: pid 1831927: LOG: DB node id: 0 backend pid: 2125689 statement: DISCARD ALL May 31 04:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1831927]: [15203-1] 2024-05-31 04:50:29: pid 1831927: LOG: DB node id: 1 backend pid: 3994019 statement: DISCARD ALL May 31 04:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1831927]: [15204-1] 2024-05-31 04:50:29: pid 1831927: LOG: DB node id: 2 backend pid: 3962671 statement: DISCARD ALL May 31 04:50:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15666-1] 2024-05-31 04:50:30: pid 241871: LOG: child process with pid: 1952399 exits with status 256 May 31 04:50:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15667-1] 2024-05-31 04:50:30: pid 241871: LOG: fork a new child process with pid: 2125702 May 31 04:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1988771]: [15449-1] 2024-05-31 04:55:29: pid 1988771: LOG: statement: SHOW POOL_NODES; May 31 04:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1988771]: [15450-1] 2024-05-31 04:55:29: pid 1988771: LOG: DB node id: 0 backend pid: 2126940 statement: SELECT pg_catalog.version() May 31 04:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1988771]: [15451-1] 2024-05-31 04:55:29: pid 1988771: LOG: statement: DISCARD ALL May 31 04:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1988771]: [15452-1] 2024-05-31 04:55:29: pid 1988771: LOG: DB node id: 0 backend pid: 2126940 statement: DISCARD ALL May 31 04:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1988771]: [15453-1] 2024-05-31 04:55:29: pid 1988771: LOG: DB node id: 1 backend pid: 3994974 statement: DISCARD ALL May 31 04:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1988771]: [15454-1] 2024-05-31 04:55:29: pid 1988771: LOG: DB node id: 2 backend pid: 3963612 statement: DISCARD ALL May 31 04:55:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15668-1] 2024-05-31 04:55:30: pid 241871: LOG: child process with pid: 1457908 exits with status 256 May 31 04:55:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15669-1] 2024-05-31 04:55:30: pid 241871: LOG: fork a new child process with pid: 2126959 May 31 05:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2125702]: [15667-1] 2024-05-31 05:00:29: pid 2125702: LOG: statement: SHOW POOL_NODES; May 31 05:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2125702]: [15668-1] 2024-05-31 05:00:29: pid 2125702: LOG: DB node id: 0 backend pid: 2128191 statement: SELECT pg_catalog.version() May 31 05:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2125702]: [15669-1] 2024-05-31 05:00:29: pid 2125702: LOG: statement: DISCARD ALL May 31 05:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2125702]: [15670-1] 2024-05-31 05:00:29: pid 2125702: LOG: DB node id: 0 backend pid: 2128191 statement: DISCARD ALL May 31 05:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2125702]: [15671-1] 2024-05-31 05:00:29: pid 2125702: LOG: DB node id: 1 backend pid: 3995948 statement: DISCARD ALL May 31 05:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2125702]: [15672-1] 2024-05-31 05:00:29: pid 2125702: LOG: DB node id: 2 backend pid: 3964585 statement: DISCARD ALL May 31 05:00:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15670-1] 2024-05-31 05:00:30: pid 241871: LOG: child process with pid: 1831927 exits with status 256 May 31 05:00:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15671-1] 2024-05-31 05:00:30: pid 241871: LOG: fork a new child process with pid: 2128208 May 31 05:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1987517]: [15447-1] 2024-05-31 05:05:29: pid 1987517: LOG: statement: SHOW POOL_NODES; May 31 05:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1987517]: [15448-1] 2024-05-31 05:05:29: pid 1987517: LOG: DB node id: 0 backend pid: 2129446 statement: SELECT pg_catalog.version() May 31 05:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1987517]: [15449-1] 2024-05-31 05:05:29: pid 1987517: LOG: statement: DISCARD ALL May 31 05:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1987517]: [15450-1] 2024-05-31 05:05:29: pid 1987517: LOG: DB node id: 0 backend pid: 2129446 statement: DISCARD ALL May 31 05:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1987517]: [15451-1] 2024-05-31 05:05:29: pid 1987517: LOG: DB node id: 1 backend pid: 3996924 statement: DISCARD ALL May 31 05:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1987517]: [15452-1] 2024-05-31 05:05:29: pid 1987517: LOG: DB node id: 2 backend pid: 3965554 statement: DISCARD ALL May 31 05:05:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15672-1] 2024-05-31 05:05:30: pid 241871: LOG: child process with pid: 1988771 exits with status 256 May 31 05:05:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15673-1] 2024-05-31 05:05:30: pid 241871: LOG: fork a new child process with pid: 2129459 May 31 05:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2128208]: [15671-1] 2024-05-31 05:10:29: pid 2128208: LOG: statement: SHOW POOL_NODES; May 31 05:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2128208]: [15672-1] 2024-05-31 05:10:29: pid 2128208: LOG: DB node id: 0 backend pid: 2130699 statement: SELECT pg_catalog.version() May 31 05:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2128208]: [15673-1] 2024-05-31 05:10:29: pid 2128208: LOG: statement: DISCARD ALL May 31 05:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2128208]: [15674-1] 2024-05-31 05:10:29: pid 2128208: LOG: DB node id: 0 backend pid: 2130699 statement: DISCARD ALL May 31 05:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2128208]: [15675-1] 2024-05-31 05:10:29: pid 2128208: LOG: DB node id: 1 backend pid: 3997897 statement: DISCARD ALL May 31 05:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2128208]: [15676-1] 2024-05-31 05:10:29: pid 2128208: LOG: DB node id: 2 backend pid: 3966523 statement: DISCARD ALL May 31 05:10:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15674-1] 2024-05-31 05:10:30: pid 241871: LOG: child process with pid: 2125702 exits with status 256 May 31 05:10:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15675-1] 2024-05-31 05:10:30: pid 241871: LOG: fork a new child process with pid: 2130713 May 31 05:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1909815]: [15323-1] 2024-05-31 05:15:29: pid 1909815: LOG: statement: SHOW POOL_NODES; May 31 05:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1909815]: [15324-1] 2024-05-31 05:15:29: pid 1909815: LOG: DB node id: 0 backend pid: 2131952 statement: SELECT pg_catalog.version() May 31 05:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1909815]: [15325-1] 2024-05-31 05:15:29: pid 1909815: LOG: statement: DISCARD ALL May 31 05:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1909815]: [15326-1] 2024-05-31 05:15:29: pid 1909815: LOG: DB node id: 0 backend pid: 2131952 statement: DISCARD ALL May 31 05:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1909815]: [15327-1] 2024-05-31 05:15:29: pid 1909815: LOG: DB node id: 1 backend pid: 3998869 statement: DISCARD ALL May 31 05:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1909815]: [15328-1] 2024-05-31 05:15:29: pid 1909815: LOG: DB node id: 2 backend pid: 3967495 statement: DISCARD ALL May 31 05:15:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15676-1] 2024-05-31 05:15:30: pid 241871: LOG: child process with pid: 1987517 exits with status 256 May 31 05:15:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15677-1] 2024-05-31 05:15:30: pid 241871: LOG: fork a new child process with pid: 2131966 May 31 05:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2118188]: [15655-1] 2024-05-31 05:20:29: pid 2118188: LOG: statement: SHOW POOL_NODES; May 31 05:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2118188]: [15656-1] 2024-05-31 05:20:29: pid 2118188: LOG: DB node id: 0 backend pid: 2133207 statement: SELECT pg_catalog.version() May 31 05:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2118188]: [15657-1] 2024-05-31 05:20:29: pid 2118188: LOG: statement: DISCARD ALL May 31 05:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2118188]: [15658-1] 2024-05-31 05:20:29: pid 2118188: LOG: DB node id: 0 backend pid: 2133207 statement: DISCARD ALL May 31 05:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2118188]: [15659-1] 2024-05-31 05:20:29: pid 2118188: LOG: DB node id: 1 backend pid: 3999814 statement: DISCARD ALL May 31 05:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2118188]: [15660-1] 2024-05-31 05:20:29: pid 2118188: LOG: DB node id: 2 backend pid: 3968467 statement: DISCARD ALL May 31 05:20:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15678-1] 2024-05-31 05:20:30: pid 241871: LOG: child process with pid: 2128208 exits with status 256 May 31 05:20:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15679-1] 2024-05-31 05:20:30: pid 241871: LOG: fork a new child process with pid: 2133222 May 31 05:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2023889]: [15505-1] 2024-05-31 05:25:29: pid 2023889: LOG: statement: SHOW POOL_NODES; May 31 05:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2023889]: [15506-1] 2024-05-31 05:25:29: pid 2023889: LOG: DB node id: 0 backend pid: 2134459 statement: SELECT pg_catalog.version() May 31 05:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2023889]: [15507-1] 2024-05-31 05:25:29: pid 2023889: LOG: statement: DISCARD ALL May 31 05:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2023889]: [15508-1] 2024-05-31 05:25:29: pid 2023889: LOG: DB node id: 0 backend pid: 2134459 statement: DISCARD ALL May 31 05:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2023889]: [15509-1] 2024-05-31 05:25:29: pid 2023889: LOG: DB node id: 1 backend pid: 4000790 statement: DISCARD ALL May 31 05:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2023889]: [15510-1] 2024-05-31 05:25:29: pid 2023889: LOG: DB node id: 2 backend pid: 3969410 statement: DISCARD ALL May 31 05:25:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15680-1] 2024-05-31 05:25:30: pid 241871: LOG: child process with pid: 1909815 exits with status 256 May 31 05:25:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15681-1] 2024-05-31 05:25:30: pid 241871: LOG: fork a new child process with pid: 2134474 May 31 05:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1972477]: [15423-1] 2024-05-31 05:30:29: pid 1972477: LOG: statement: SHOW POOL_NODES; May 31 05:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1972477]: [15424-1] 2024-05-31 05:30:29: pid 1972477: LOG: DB node id: 0 backend pid: 2135709 statement: SELECT pg_catalog.version() May 31 05:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1972477]: [15425-1] 2024-05-31 05:30:29: pid 1972477: LOG: statement: DISCARD ALL May 31 05:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1972477]: [15426-1] 2024-05-31 05:30:29: pid 1972477: LOG: DB node id: 0 backend pid: 2135709 statement: DISCARD ALL May 31 05:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1972477]: [15427-1] 2024-05-31 05:30:29: pid 1972477: LOG: DB node id: 1 backend pid: 4001758 statement: DISCARD ALL May 31 05:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1972477]: [15428-1] 2024-05-31 05:30:29: pid 1972477: LOG: DB node id: 2 backend pid: 3970382 statement: DISCARD ALL May 31 05:30:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15682-1] 2024-05-31 05:30:30: pid 241871: LOG: child process with pid: 2118188 exits with status 256 May 31 05:30:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15683-1] 2024-05-31 05:30:30: pid 241871: LOG: fork a new child process with pid: 2135724 May 31 05:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1979993]: [15435-1] 2024-05-31 05:35:29: pid 1979993: LOG: statement: SHOW POOL_NODES; May 31 05:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1979993]: [15436-1] 2024-05-31 05:35:29: pid 1979993: LOG: DB node id: 0 backend pid: 2136961 statement: SELECT pg_catalog.version() May 31 05:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1979993]: [15437-1] 2024-05-31 05:35:29: pid 1979993: LOG: statement: DISCARD ALL May 31 05:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1979993]: [15438-1] 2024-05-31 05:35:29: pid 1979993: LOG: DB node id: 0 backend pid: 2136961 statement: DISCARD ALL May 31 05:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1979993]: [15439-1] 2024-05-31 05:35:29: pid 1979993: LOG: DB node id: 1 backend pid: 4002731 statement: DISCARD ALL May 31 05:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1979993]: [15440-1] 2024-05-31 05:35:29: pid 1979993: LOG: DB node id: 2 backend pid: 3971354 statement: DISCARD ALL May 31 05:35:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15684-1] 2024-05-31 05:35:30: pid 241871: LOG: child process with pid: 2023889 exits with status 256 May 31 05:35:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15685-1] 2024-05-31 05:35:30: pid 241871: LOG: fork a new child process with pid: 2136976 May 31 05:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2123196]: [15663-1] 2024-05-31 05:40:29: pid 2123196: LOG: statement: SHOW POOL_NODES; May 31 05:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2123196]: [15664-1] 2024-05-31 05:40:29: pid 2123196: LOG: DB node id: 0 backend pid: 2138216 statement: SELECT pg_catalog.version() May 31 05:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2123196]: [15665-1] 2024-05-31 05:40:29: pid 2123196: LOG: statement: DISCARD ALL May 31 05:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2123196]: [15666-1] 2024-05-31 05:40:29: pid 2123196: LOG: DB node id: 0 backend pid: 2138216 statement: DISCARD ALL May 31 05:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2123196]: [15667-1] 2024-05-31 05:40:29: pid 2123196: LOG: DB node id: 1 backend pid: 4003686 statement: DISCARD ALL May 31 05:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2123196]: [15668-1] 2024-05-31 05:40:29: pid 2123196: LOG: DB node id: 2 backend pid: 3972324 statement: DISCARD ALL May 31 05:40:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15686-1] 2024-05-31 05:40:30: pid 241871: LOG: child process with pid: 1972477 exits with status 256 May 31 05:40:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15687-1] 2024-05-31 05:40:30: pid 241871: LOG: fork a new child process with pid: 2138231 May 31 05:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2072879]: [15583-1] 2024-05-31 05:45:29: pid 2072879: LOG: statement: SHOW POOL_NODES; May 31 05:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2072879]: [15584-1] 2024-05-31 05:45:29: pid 2072879: LOG: DB node id: 0 backend pid: 2139471 statement: SELECT pg_catalog.version() May 31 05:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2072879]: [15585-1] 2024-05-31 05:45:29: pid 2072879: LOG: statement: DISCARD ALL May 31 05:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2072879]: [15586-1] 2024-05-31 05:45:29: pid 2072879: LOG: DB node id: 0 backend pid: 2139471 statement: DISCARD ALL May 31 05:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2072879]: [15587-1] 2024-05-31 05:45:29: pid 2072879: LOG: DB node id: 1 backend pid: 4004643 statement: DISCARD ALL May 31 05:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2072879]: [15588-1] 2024-05-31 05:45:29: pid 2072879: LOG: DB node id: 2 backend pid: 3973304 statement: DISCARD ALL May 31 05:45:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15688-1] 2024-05-31 05:45:30: pid 241871: LOG: child process with pid: 1979993 exits with status 256 May 31 05:45:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15689-1] 2024-05-31 05:45:30: pid 241871: LOG: fork a new child process with pid: 2139487 May 31 05:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1985012]: [15443-1] 2024-05-31 05:50:29: pid 1985012: LOG: statement: SHOW POOL_NODES; May 31 05:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1985012]: [15444-1] 2024-05-31 05:50:29: pid 1985012: LOG: DB node id: 0 backend pid: 2140723 statement: SELECT pg_catalog.version() May 31 05:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1985012]: [15445-1] 2024-05-31 05:50:29: pid 1985012: LOG: statement: DISCARD ALL May 31 05:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1985012]: [15446-1] 2024-05-31 05:50:29: pid 1985012: LOG: DB node id: 0 backend pid: 2140723 statement: DISCARD ALL May 31 05:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1985012]: [15447-1] 2024-05-31 05:50:29: pid 1985012: LOG: DB node id: 1 backend pid: 4005618 statement: DISCARD ALL May 31 05:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1985012]: [15448-1] 2024-05-31 05:50:29: pid 1985012: LOG: DB node id: 2 backend pid: 3974277 statement: DISCARD ALL May 31 05:50:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15690-1] 2024-05-31 05:50:30: pid 241871: LOG: child process with pid: 2123196 exits with status 256 May 31 05:50:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15691-1] 2024-05-31 05:50:30: pid 241871: LOG: fork a new child process with pid: 2140736 May 31 05:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2062793]: [15567-1] 2024-05-31 05:55:29: pid 2062793: LOG: statement: SHOW POOL_NODES; May 31 05:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2062793]: [15568-1] 2024-05-31 05:55:29: pid 2062793: LOG: DB node id: 0 backend pid: 2141972 statement: SELECT pg_catalog.version() May 31 05:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2062793]: [15569-1] 2024-05-31 05:55:29: pid 2062793: LOG: statement: DISCARD ALL May 31 05:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2062793]: [15570-1] 2024-05-31 05:55:29: pid 2062793: LOG: DB node id: 0 backend pid: 2141972 statement: DISCARD ALL May 31 05:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2062793]: [15571-1] 2024-05-31 05:55:29: pid 2062793: LOG: DB node id: 1 backend pid: 4006586 statement: DISCARD ALL May 31 05:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2062793]: [15572-1] 2024-05-31 05:55:29: pid 2062793: LOG: DB node id: 2 backend pid: 3975217 statement: DISCARD ALL May 31 05:55:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15692-1] 2024-05-31 05:55:30: pid 241871: LOG: child process with pid: 2072879 exits with status 256 May 31 05:55:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15693-1] 2024-05-31 05:55:30: pid 241871: LOG: fork a new child process with pid: 2141989 May 31 06:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1901017]: [15309-1] 2024-05-31 06:00:29: pid 1901017: LOG: statement: SHOW POOL_NODES; May 31 06:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1901017]: [15310-1] 2024-05-31 06:00:29: pid 1901017: LOG: DB node id: 0 backend pid: 2143224 statement: SELECT pg_catalog.version() May 31 06:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1901017]: [15311-1] 2024-05-31 06:00:29: pid 1901017: LOG: statement: DISCARD ALL May 31 06:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1901017]: [15312-1] 2024-05-31 06:00:29: pid 1901017: LOG: DB node id: 0 backend pid: 2143224 statement: DISCARD ALL May 31 06:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1901017]: [15313-1] 2024-05-31 06:00:29: pid 1901017: LOG: DB node id: 1 backend pid: 4007556 statement: DISCARD ALL May 31 06:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1901017]: [15314-1] 2024-05-31 06:00:29: pid 1901017: LOG: DB node id: 2 backend pid: 3976186 statement: DISCARD ALL May 31 06:00:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15694-1] 2024-05-31 06:00:30: pid 241871: LOG: child process with pid: 1985012 exits with status 256 May 31 06:00:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15695-1] 2024-05-31 06:00:30: pid 241871: LOG: fork a new child process with pid: 2143240 May 31 06:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2086827]: [15605-1] 2024-05-31 06:05:29: pid 2086827: LOG: statement: SHOW POOL_NODES; May 31 06:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2086827]: [15606-1] 2024-05-31 06:05:29: pid 2086827: LOG: DB node id: 0 backend pid: 2144483 statement: SELECT pg_catalog.version() May 31 06:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2086827]: [15607-1] 2024-05-31 06:05:29: pid 2086827: LOG: statement: DISCARD ALL May 31 06:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2086827]: [15608-1] 2024-05-31 06:05:29: pid 2086827: LOG: DB node id: 0 backend pid: 2144483 statement: DISCARD ALL May 31 06:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2086827]: [15609-1] 2024-05-31 06:05:29: pid 2086827: LOG: DB node id: 1 backend pid: 4008527 statement: DISCARD ALL May 31 06:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2086827]: [15610-1] 2024-05-31 06:05:29: pid 2086827: LOG: DB node id: 2 backend pid: 3977157 statement: DISCARD ALL May 31 06:05:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15696-1] 2024-05-31 06:05:30: pid 241871: LOG: child process with pid: 2062793 exits with status 256 May 31 06:05:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15697-1] 2024-05-31 06:05:30: pid 241871: LOG: fork a new child process with pid: 2144499 May 31 06:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2126959]: [15669-1] 2024-05-31 06:10:29: pid 2126959: LOG: statement: SHOW POOL_NODES; May 31 06:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2126959]: [15670-1] 2024-05-31 06:10:29: pid 2126959: LOG: DB node id: 0 backend pid: 2145736 statement: SELECT pg_catalog.version() May 31 06:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2126959]: [15671-1] 2024-05-31 06:10:29: pid 2126959: LOG: statement: DISCARD ALL May 31 06:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2126959]: [15672-1] 2024-05-31 06:10:29: pid 2126959: LOG: DB node id: 0 backend pid: 2145736 statement: DISCARD ALL May 31 06:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2126959]: [15673-1] 2024-05-31 06:10:29: pid 2126959: LOG: DB node id: 1 backend pid: 4009471 statement: DISCARD ALL May 31 06:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2126959]: [15674-1] 2024-05-31 06:10:29: pid 2126959: LOG: DB node id: 2 backend pid: 3978123 statement: DISCARD ALL May 31 06:10:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15698-1] 2024-05-31 06:10:30: pid 241871: LOG: child process with pid: 1901017 exits with status 256 May 31 06:10:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15699-1] 2024-05-31 06:10:30: pid 241871: LOG: fork a new child process with pid: 2145750 May 31 06:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1816885]: [15175-1] 2024-05-31 06:15:29: pid 1816885: LOG: statement: SHOW POOL_NODES; May 31 06:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1816885]: [15176-1] 2024-05-31 06:15:29: pid 1816885: LOG: DB node id: 0 backend pid: 2146992 statement: SELECT pg_catalog.version() May 31 06:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1816885]: [15177-1] 2024-05-31 06:15:29: pid 1816885: LOG: statement: DISCARD ALL May 31 06:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1816885]: [15178-1] 2024-05-31 06:15:29: pid 1816885: LOG: DB node id: 0 backend pid: 2146992 statement: DISCARD ALL May 31 06:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1816885]: [15179-1] 2024-05-31 06:15:29: pid 1816885: LOG: DB node id: 1 backend pid: 4010448 statement: DISCARD ALL May 31 06:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1816885]: [15180-1] 2024-05-31 06:15:29: pid 1816885: LOG: DB node id: 2 backend pid: 3979092 statement: DISCARD ALL May 31 06:15:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15700-1] 2024-05-31 06:15:30: pid 241871: LOG: child process with pid: 2086827 exits with status 256 May 31 06:15:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15701-1] 2024-05-31 06:15:30: pid 241871: LOG: fork a new child process with pid: 2147009 May 31 06:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2042732]: [15535-1] 2024-05-31 06:20:29: pid 2042732: LOG: statement: SHOW POOL_NODES; May 31 06:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2042732]: [15536-1] 2024-05-31 06:20:29: pid 2042732: LOG: DB node id: 0 backend pid: 2148250 statement: SELECT pg_catalog.version() May 31 06:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2042732]: [15537-1] 2024-05-31 06:20:29: pid 2042732: LOG: statement: DISCARD ALL May 31 06:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2042732]: [15538-1] 2024-05-31 06:20:29: pid 2042732: LOG: DB node id: 0 backend pid: 2148250 statement: DISCARD ALL May 31 06:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2042732]: [15539-1] 2024-05-31 06:20:29: pid 2042732: LOG: DB node id: 1 backend pid: 4011421 statement: DISCARD ALL May 31 06:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2042732]: [15540-1] 2024-05-31 06:20:29: pid 2042732: LOG: DB node id: 2 backend pid: 3980066 statement: DISCARD ALL May 31 06:20:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15702-1] 2024-05-31 06:20:30: pid 241871: LOG: child process with pid: 2126959 exits with status 256 May 31 06:20:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15703-1] 2024-05-31 06:20:30: pid 241871: LOG: fork a new child process with pid: 2148263 May 31 06:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1808115]: [15161-1] 2024-05-31 06:25:29: pid 1808115: LOG: statement: SHOW POOL_NODES; May 31 06:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1808115]: [15162-1] 2024-05-31 06:25:29: pid 1808115: LOG: DB node id: 0 backend pid: 2149512 statement: SELECT pg_catalog.version() May 31 06:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1808115]: [15163-1] 2024-05-31 06:25:29: pid 1808115: LOG: statement: DISCARD ALL May 31 06:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1808115]: [15164-1] 2024-05-31 06:25:29: pid 1808115: LOG: DB node id: 0 backend pid: 2149512 statement: DISCARD ALL May 31 06:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1808115]: [15165-1] 2024-05-31 06:25:29: pid 1808115: LOG: DB node id: 1 backend pid: 4012409 statement: DISCARD ALL May 31 06:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1808115]: [15166-1] 2024-05-31 06:25:29: pid 1808115: LOG: DB node id: 2 backend pid: 3981050 statement: DISCARD ALL May 31 06:25:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15704-1] 2024-05-31 06:25:30: pid 241871: LOG: child process with pid: 1816885 exits with status 256 May 31 06:25:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15705-1] 2024-05-31 06:25:30: pid 241871: LOG: fork a new child process with pid: 2149528 May 31 06:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2055279]: [15555-1] 2024-05-31 06:30:29: pid 2055279: LOG: statement: SHOW POOL_NODES; May 31 06:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2055279]: [15556-1] 2024-05-31 06:30:29: pid 2055279: LOG: DB node id: 0 backend pid: 2150783 statement: SELECT pg_catalog.version() May 31 06:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2055279]: [15557-1] 2024-05-31 06:30:29: pid 2055279: LOG: statement: DISCARD ALL May 31 06:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2055279]: [15558-1] 2024-05-31 06:30:29: pid 2055279: LOG: DB node id: 0 backend pid: 2150783 statement: DISCARD ALL May 31 06:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2055279]: [15559-1] 2024-05-31 06:30:29: pid 2055279: LOG: DB node id: 1 backend pid: 4013378 statement: DISCARD ALL May 31 06:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2055279]: [15560-1] 2024-05-31 06:30:29: pid 2055279: LOG: DB node id: 2 backend pid: 3981981 statement: DISCARD ALL May 31 06:30:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15706-1] 2024-05-31 06:30:30: pid 241871: LOG: child process with pid: 2042732 exits with status 256 May 31 06:30:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15707-1] 2024-05-31 06:30:30: pid 241871: LOG: fork a new child process with pid: 2150796 May 31 06:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2115682]: [15651-1] 2024-05-31 06:35:29: pid 2115682: LOG: statement: SHOW POOL_NODES; May 31 06:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2115682]: [15652-1] 2024-05-31 06:35:29: pid 2115682: LOG: DB node id: 0 backend pid: 2152038 statement: SELECT pg_catalog.version() May 31 06:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2115682]: [15653-1] 2024-05-31 06:35:29: pid 2115682: LOG: statement: DISCARD ALL May 31 06:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2115682]: [15654-1] 2024-05-31 06:35:29: pid 2115682: LOG: DB node id: 0 backend pid: 2152038 statement: DISCARD ALL May 31 06:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2115682]: [15655-1] 2024-05-31 06:35:29: pid 2115682: LOG: DB node id: 1 backend pid: 4014325 statement: DISCARD ALL May 31 06:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2115682]: [15656-1] 2024-05-31 06:35:29: pid 2115682: LOG: DB node id: 2 backend pid: 3982958 statement: DISCARD ALL May 31 06:35:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15708-1] 2024-05-31 06:35:30: pid 241871: LOG: child process with pid: 1808115 exits with status 256 May 31 06:35:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15709-1] 2024-05-31 06:35:30: pid 241871: LOG: fork a new child process with pid: 2152054 May 31 06:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2054029]: [15553-1] 2024-05-31 06:40:29: pid 2054029: LOG: statement: SHOW POOL_NODES; May 31 06:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2054029]: [15554-1] 2024-05-31 06:40:29: pid 2054029: LOG: DB node id: 0 backend pid: 2153293 statement: SELECT pg_catalog.version() May 31 06:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2054029]: [15555-1] 2024-05-31 06:40:29: pid 2054029: LOG: statement: DISCARD ALL May 31 06:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2054029]: [15556-1] 2024-05-31 06:40:29: pid 2054029: LOG: DB node id: 0 backend pid: 2153293 statement: DISCARD ALL May 31 06:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2054029]: [15557-1] 2024-05-31 06:40:29: pid 2054029: LOG: DB node id: 1 backend pid: 4015299 statement: DISCARD ALL May 31 06:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2054029]: [15558-1] 2024-05-31 06:40:29: pid 2054029: LOG: DB node id: 2 backend pid: 3983936 statement: DISCARD ALL May 31 06:40:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15710-1] 2024-05-31 06:40:30: pid 241871: LOG: child process with pid: 2055279 exits with status 256 May 31 06:40:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15711-1] 2024-05-31 06:40:30: pid 241871: LOG: fork a new child process with pid: 2153307 May 31 06:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2080524]: [15595-1] 2024-05-31 06:45:29: pid 2080524: LOG: statement: SHOW POOL_NODES; May 31 06:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2080524]: [15596-1] 2024-05-31 06:45:29: pid 2080524: LOG: DB node id: 0 backend pid: 2154547 statement: SELECT pg_catalog.version() May 31 06:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2080524]: [15597-1] 2024-05-31 06:45:29: pid 2080524: LOG: statement: DISCARD ALL May 31 06:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2080524]: [15598-1] 2024-05-31 06:45:29: pid 2080524: LOG: DB node id: 0 backend pid: 2154547 statement: DISCARD ALL May 31 06:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2080524]: [15599-1] 2024-05-31 06:45:29: pid 2080524: LOG: DB node id: 1 backend pid: 4016272 statement: DISCARD ALL May 31 06:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2080524]: [15600-1] 2024-05-31 06:45:29: pid 2080524: LOG: DB node id: 2 backend pid: 3984913 statement: DISCARD ALL May 31 06:45:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15712-1] 2024-05-31 06:45:30: pid 241871: LOG: child process with pid: 2115682 exits with status 256 May 31 06:45:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15713-1] 2024-05-31 06:45:30: pid 241871: LOG: fork a new child process with pid: 2154560 May 31 06:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2079256]: [15593-1] 2024-05-31 06:50:29: pid 2079256: LOG: statement: SHOW POOL_NODES; May 31 06:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2079256]: [15594-1] 2024-05-31 06:50:29: pid 2079256: LOG: DB node id: 0 backend pid: 2155794 statement: SELECT pg_catalog.version() May 31 06:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2079256]: [15595-1] 2024-05-31 06:50:29: pid 2079256: LOG: statement: DISCARD ALL May 31 06:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2079256]: [15596-1] 2024-05-31 06:50:29: pid 2079256: LOG: DB node id: 0 backend pid: 2155794 statement: DISCARD ALL May 31 06:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2079256]: [15597-1] 2024-05-31 06:50:29: pid 2079256: LOG: DB node id: 1 backend pid: 4017244 statement: DISCARD ALL May 31 06:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2079256]: [15598-1] 2024-05-31 06:50:29: pid 2079256: LOG: DB node id: 2 backend pid: 3985878 statement: DISCARD ALL May 31 06:50:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15714-1] 2024-05-31 06:50:30: pid 241871: LOG: child process with pid: 2054029 exits with status 256 May 31 06:50:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15715-1] 2024-05-31 06:50:30: pid 241871: LOG: fork a new child process with pid: 2155810 May 31 06:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1880824]: [15277-1] 2024-05-31 06:55:29: pid 1880824: LOG: statement: SHOW POOL_NODES; May 31 06:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1880824]: [15278-1] 2024-05-31 06:55:29: pid 1880824: LOG: DB node id: 0 backend pid: 2157050 statement: SELECT pg_catalog.version() May 31 06:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1880824]: [15279-1] 2024-05-31 06:55:29: pid 1880824: LOG: statement: DISCARD ALL May 31 06:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1880824]: [15280-1] 2024-05-31 06:55:29: pid 1880824: LOG: DB node id: 0 backend pid: 2157050 statement: DISCARD ALL May 31 06:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1880824]: [15281-1] 2024-05-31 06:55:29: pid 1880824: LOG: DB node id: 1 backend pid: 4018216 statement: DISCARD ALL May 31 06:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1880824]: [15282-1] 2024-05-31 06:55:29: pid 1880824: LOG: DB node id: 2 backend pid: 3986846 statement: DISCARD ALL May 31 06:55:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15716-1] 2024-05-31 06:55:30: pid 241871: LOG: child process with pid: 2080524 exits with status 256 May 31 06:55:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15717-1] 2024-05-31 06:55:30: pid 241871: LOG: fork a new child process with pid: 2157065 May 31 07:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2124452]: [15665-1] 2024-05-31 07:00:29: pid 2124452: LOG: statement: SHOW POOL_NODES; May 31 07:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2124452]: [15666-1] 2024-05-31 07:00:29: pid 2124452: LOG: DB node id: 0 backend pid: 2158303 statement: SELECT pg_catalog.version() May 31 07:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2124452]: [15667-1] 2024-05-31 07:00:29: pid 2124452: LOG: statement: DISCARD ALL May 31 07:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2124452]: [15668-1] 2024-05-31 07:00:29: pid 2124452: LOG: DB node id: 0 backend pid: 2158303 statement: DISCARD ALL May 31 07:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2124452]: [15669-1] 2024-05-31 07:00:29: pid 2124452: LOG: DB node id: 1 backend pid: 4019151 statement: DISCARD ALL May 31 07:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2124452]: [15670-1] 2024-05-31 07:00:29: pid 2124452: LOG: DB node id: 2 backend pid: 3987782 statement: DISCARD ALL May 31 07:00:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15718-1] 2024-05-31 07:00:30: pid 241871: LOG: child process with pid: 2079256 exits with status 256 May 31 07:00:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15719-1] 2024-05-31 07:00:30: pid 241871: LOG: fork a new child process with pid: 2158322 May 31 07:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2002597]: [15471-1] 2024-05-31 07:05:29: pid 2002597: LOG: statement: SHOW POOL_NODES; May 31 07:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2002597]: [15472-1] 2024-05-31 07:05:29: pid 2002597: LOG: DB node id: 0 backend pid: 2159555 statement: SELECT pg_catalog.version() May 31 07:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2002597]: [15473-1] 2024-05-31 07:05:29: pid 2002597: LOG: statement: DISCARD ALL May 31 07:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2002597]: [15474-1] 2024-05-31 07:05:29: pid 2002597: LOG: DB node id: 0 backend pid: 2159555 statement: DISCARD ALL May 31 07:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2002597]: [15475-1] 2024-05-31 07:05:29: pid 2002597: LOG: DB node id: 1 backend pid: 4020127 statement: DISCARD ALL May 31 07:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2002597]: [15476-1] 2024-05-31 07:05:29: pid 2002597: LOG: DB node id: 2 backend pid: 3988755 statement: DISCARD ALL May 31 07:05:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15720-1] 2024-05-31 07:05:30: pid 241871: LOG: child process with pid: 1880824 exits with status 256 May 31 07:05:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15721-1] 2024-05-31 07:05:30: pid 241871: LOG: fork a new child process with pid: 2159571 May 31 07:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1959916]: [15403-1] 2024-05-31 07:10:29: pid 1959916: LOG: statement: SHOW POOL_NODES; May 31 07:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1959916]: [15404-1] 2024-05-31 07:10:29: pid 1959916: LOG: DB node id: 0 backend pid: 2160809 statement: SELECT pg_catalog.version() May 31 07:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1959916]: [15405-1] 2024-05-31 07:10:29: pid 1959916: LOG: statement: DISCARD ALL May 31 07:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1959916]: [15406-1] 2024-05-31 07:10:29: pid 1959916: LOG: DB node id: 0 backend pid: 2160809 statement: DISCARD ALL May 31 07:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1959916]: [15407-1] 2024-05-31 07:10:29: pid 1959916: LOG: DB node id: 1 backend pid: 4021089 statement: DISCARD ALL May 31 07:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1959916]: [15408-1] 2024-05-31 07:10:29: pid 1959916: LOG: DB node id: 2 backend pid: 3989722 statement: DISCARD ALL May 31 07:10:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15722-1] 2024-05-31 07:10:30: pid 241871: LOG: child process with pid: 2124452 exits with status 256 May 31 07:10:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15723-1] 2024-05-31 07:10:30: pid 241871: LOG: fork a new child process with pid: 2160826 May 31 07:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2157065]: [15717-1] 2024-05-31 07:15:29: pid 2157065: LOG: statement: SHOW POOL_NODES; May 31 07:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2157065]: [15718-1] 2024-05-31 07:15:29: pid 2157065: LOG: DB node id: 0 backend pid: 2162064 statement: SELECT pg_catalog.version() May 31 07:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2157065]: [15719-1] 2024-05-31 07:15:29: pid 2157065: LOG: statement: DISCARD ALL May 31 07:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2157065]: [15720-1] 2024-05-31 07:15:29: pid 2157065: LOG: DB node id: 0 backend pid: 2162064 statement: DISCARD ALL May 31 07:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2157065]: [15721-1] 2024-05-31 07:15:29: pid 2157065: LOG: DB node id: 1 backend pid: 4022075 statement: DISCARD ALL May 31 07:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2157065]: [15722-1] 2024-05-31 07:15:29: pid 2157065: LOG: DB node id: 2 backend pid: 3990704 statement: DISCARD ALL May 31 07:15:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15724-1] 2024-05-31 07:15:30: pid 241871: LOG: child process with pid: 2002597 exits with status 256 May 31 07:15:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15725-1] 2024-05-31 07:15:30: pid 241871: LOG: fork a new child process with pid: 2162082 May 31 07:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1977489]: [15431-1] 2024-05-31 07:20:29: pid 1977489: LOG: statement: SHOW POOL_NODES; May 31 07:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1977489]: [15432-1] 2024-05-31 07:20:29: pid 1977489: LOG: DB node id: 0 backend pid: 2163318 statement: SELECT pg_catalog.version() May 31 07:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1977489]: [15433-1] 2024-05-31 07:20:29: pid 1977489: LOG: statement: DISCARD ALL May 31 07:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1977489]: [15434-1] 2024-05-31 07:20:29: pid 1977489: LOG: DB node id: 0 backend pid: 2163318 statement: DISCARD ALL May 31 07:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1977489]: [15435-1] 2024-05-31 07:20:29: pid 1977489: LOG: DB node id: 1 backend pid: 4023046 statement: DISCARD ALL May 31 07:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1977489]: [15436-1] 2024-05-31 07:20:29: pid 1977489: LOG: DB node id: 2 backend pid: 3991675 statement: DISCARD ALL May 31 07:20:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15726-1] 2024-05-31 07:20:30: pid 241871: LOG: child process with pid: 1959916 exits with status 256 May 31 07:20:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15727-1] 2024-05-31 07:20:30: pid 241871: LOG: fork a new child process with pid: 2163336 May 31 07:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2149528]: [15705-1] 2024-05-31 07:25:29: pid 2149528: LOG: statement: SHOW POOL_NODES; May 31 07:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2149528]: [15706-1] 2024-05-31 07:25:29: pid 2149528: LOG: DB node id: 0 backend pid: 2164569 statement: SELECT pg_catalog.version() May 31 07:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2149528]: [15707-1] 2024-05-31 07:25:29: pid 2149528: LOG: statement: DISCARD ALL May 31 07:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2149528]: [15708-1] 2024-05-31 07:25:29: pid 2149528: LOG: DB node id: 0 backend pid: 2164569 statement: DISCARD ALL May 31 07:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2149528]: [15709-1] 2024-05-31 07:25:29: pid 2149528: LOG: DB node id: 1 backend pid: 4023987 statement: DISCARD ALL May 31 07:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2149528]: [15710-1] 2024-05-31 07:25:29: pid 2149528: LOG: DB node id: 2 backend pid: 3992643 statement: DISCARD ALL May 31 07:25:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15728-1] 2024-05-31 07:25:30: pid 241871: LOG: child process with pid: 2157065 exits with status 256 May 31 07:25:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15729-1] 2024-05-31 07:25:30: pid 241871: LOG: fork a new child process with pid: 2164585 May 31 07:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2100644]: [15627-1] 2024-05-31 07:30:29: pid 2100644: LOG: statement: SHOW POOL_NODES; May 31 07:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2100644]: [15628-1] 2024-05-31 07:30:29: pid 2100644: LOG: DB node id: 0 backend pid: 2165817 statement: SELECT pg_catalog.version() May 31 07:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2100644]: [15629-1] 2024-05-31 07:30:29: pid 2100644: LOG: statement: DISCARD ALL May 31 07:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2100644]: [15630-1] 2024-05-31 07:30:29: pid 2100644: LOG: DB node id: 0 backend pid: 2165817 statement: DISCARD ALL May 31 07:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2100644]: [15631-1] 2024-05-31 07:30:29: pid 2100644: LOG: DB node id: 1 backend pid: 4024963 statement: DISCARD ALL May 31 07:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2100644]: [15632-1] 2024-05-31 07:30:29: pid 2100644: LOG: DB node id: 2 backend pid: 3993579 statement: DISCARD ALL May 31 07:30:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15730-1] 2024-05-31 07:30:30: pid 241871: LOG: child process with pid: 1977489 exits with status 256 May 31 07:30:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15731-1] 2024-05-31 07:30:30: pid 241871: LOG: fork a new child process with pid: 2165830 May 31 07:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1928599]: [15353-1] 2024-05-31 07:35:29: pid 1928599: LOG: statement: SHOW POOL_NODES; May 31 07:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1928599]: [15354-1] 2024-05-31 07:35:29: pid 1928599: LOG: DB node id: 0 backend pid: 2167070 statement: SELECT pg_catalog.version() May 31 07:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1928599]: [15355-1] 2024-05-31 07:35:29: pid 1928599: LOG: statement: DISCARD ALL May 31 07:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1928599]: [15356-1] 2024-05-31 07:35:29: pid 1928599: LOG: DB node id: 0 backend pid: 2167070 statement: DISCARD ALL May 31 07:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1928599]: [15357-1] 2024-05-31 07:35:29: pid 1928599: LOG: DB node id: 1 backend pid: 4025940 statement: DISCARD ALL May 31 07:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1928599]: [15358-1] 2024-05-31 07:35:29: pid 1928599: LOG: DB node id: 2 backend pid: 3994552 statement: DISCARD ALL May 31 07:35:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15732-1] 2024-05-31 07:35:30: pid 241871: LOG: child process with pid: 2149528 exits with status 256 May 31 07:35:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15733-1] 2024-05-31 07:35:30: pid 241871: LOG: fork a new child process with pid: 2167085 May 31 07:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1973727]: [15425-1] 2024-05-31 07:40:29: pid 1973727: LOG: statement: SHOW POOL_NODES; May 31 07:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1973727]: [15426-1] 2024-05-31 07:40:29: pid 1973727: LOG: DB node id: 0 backend pid: 2168319 statement: SELECT pg_catalog.version() May 31 07:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1973727]: [15427-1] 2024-05-31 07:40:29: pid 1973727: LOG: statement: DISCARD ALL May 31 07:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1973727]: [15428-1] 2024-05-31 07:40:29: pid 1973727: LOG: DB node id: 0 backend pid: 2168319 statement: DISCARD ALL May 31 07:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1973727]: [15429-1] 2024-05-31 07:40:29: pid 1973727: LOG: DB node id: 1 backend pid: 4026913 statement: DISCARD ALL May 31 07:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1973727]: [15430-1] 2024-05-31 07:40:29: pid 1973727: LOG: DB node id: 2 backend pid: 3995521 statement: DISCARD ALL May 31 07:40:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15734-1] 2024-05-31 07:40:30: pid 241871: LOG: child process with pid: 2100644 exits with status 256 May 31 07:40:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15735-1] 2024-05-31 07:40:30: pid 241871: LOG: fork a new child process with pid: 2168332 May 31 07:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2141989]: [15693-1] 2024-05-31 07:45:29: pid 2141989: LOG: statement: SHOW POOL_NODES; May 31 07:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2141989]: [15694-1] 2024-05-31 07:45:29: pid 2141989: LOG: DB node id: 0 backend pid: 2169574 statement: SELECT pg_catalog.version() May 31 07:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2141989]: [15695-1] 2024-05-31 07:45:29: pid 2141989: LOG: statement: DISCARD ALL May 31 07:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2141989]: [15696-1] 2024-05-31 07:45:29: pid 2141989: LOG: DB node id: 0 backend pid: 2169574 statement: DISCARD ALL May 31 07:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2141989]: [15697-1] 2024-05-31 07:45:29: pid 2141989: LOG: DB node id: 1 backend pid: 4027886 statement: DISCARD ALL May 31 07:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2141989]: [15698-1] 2024-05-31 07:45:29: pid 2141989: LOG: DB node id: 2 backend pid: 3996493 statement: DISCARD ALL May 31 07:45:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15736-1] 2024-05-31 07:45:30: pid 241871: LOG: child process with pid: 1928599 exits with status 256 May 31 07:45:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15737-1] 2024-05-31 07:45:30: pid 241871: LOG: fork a new child process with pid: 2169588 May 31 07:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2140736]: [15691-1] 2024-05-31 07:50:29: pid 2140736: LOG: statement: SHOW POOL_NODES; May 31 07:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2140736]: [15692-1] 2024-05-31 07:50:29: pid 2140736: LOG: DB node id: 0 backend pid: 2170824 statement: SELECT pg_catalog.version() May 31 07:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2140736]: [15693-1] 2024-05-31 07:50:29: pid 2140736: LOG: statement: DISCARD ALL May 31 07:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2140736]: [15694-1] 2024-05-31 07:50:29: pid 2140736: LOG: DB node id: 0 backend pid: 2170824 statement: DISCARD ALL May 31 07:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2140736]: [15695-1] 2024-05-31 07:50:29: pid 2140736: LOG: DB node id: 1 backend pid: 4028823 statement: DISCARD ALL May 31 07:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2140736]: [15696-1] 2024-05-31 07:50:29: pid 2140736: LOG: DB node id: 2 backend pid: 3997464 statement: DISCARD ALL May 31 07:50:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15738-1] 2024-05-31 07:50:30: pid 241871: LOG: child process with pid: 1973727 exits with status 256 May 31 07:50:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15739-1] 2024-05-31 07:50:30: pid 241871: LOG: fork a new child process with pid: 2170837 May 31 07:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2106915]: [15637-1] 2024-05-31 07:55:29: pid 2106915: LOG: statement: SHOW POOL_NODES; May 31 07:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2106915]: [15638-1] 2024-05-31 07:55:29: pid 2106915: LOG: DB node id: 0 backend pid: 2172077 statement: SELECT pg_catalog.version() May 31 07:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2106915]: [15639-1] 2024-05-31 07:55:29: pid 2106915: LOG: statement: DISCARD ALL May 31 07:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2106915]: [15640-1] 2024-05-31 07:55:29: pid 2106915: LOG: DB node id: 0 backend pid: 2172077 statement: DISCARD ALL May 31 07:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2106915]: [15641-1] 2024-05-31 07:55:29: pid 2106915: LOG: DB node id: 1 backend pid: 4029794 statement: DISCARD ALL May 31 07:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2106915]: [15642-1] 2024-05-31 07:55:29: pid 2106915: LOG: DB node id: 2 backend pid: 3998433 statement: DISCARD ALL May 31 08:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2001342]: [15469-1] 2024-05-31 08:00:29: pid 2001342: LOG: statement: SHOW POOL_NODES; May 31 08:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2001342]: [15470-1] 2024-05-31 08:00:29: pid 2001342: LOG: DB node id: 0 backend pid: 2173326 statement: SELECT pg_catalog.version() May 31 08:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2001342]: [15471-1] 2024-05-31 08:00:29: pid 2001342: LOG: statement: DISCARD ALL May 31 08:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2001342]: [15472-1] 2024-05-31 08:00:29: pid 2001342: LOG: DB node id: 0 backend pid: 2173326 statement: DISCARD ALL May 31 08:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2001342]: [15473-1] 2024-05-31 08:00:29: pid 2001342: LOG: DB node id: 1 backend pid: 4030765 statement: DISCARD ALL May 31 08:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2001342]: [15474-1] 2024-05-31 08:00:29: pid 2001342: LOG: DB node id: 2 backend pid: 3999364 statement: DISCARD ALL May 31 08:00:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15740-1] 2024-05-31 08:00:30: pid 241871: LOG: child process with pid: 2141989 exits with status 256 May 31 08:00:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15741-1] 2024-05-31 08:00:30: pid 241871: LOG: fork a new child process with pid: 2173341 May 31 08:00:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15742-1] 2024-05-31 08:00:30: pid 241871: LOG: child process with pid: 2140736 exits with status 256 May 31 08:00:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15743-1] 2024-05-31 08:00:30: pid 241871: LOG: fork a new child process with pid: 2173343 May 31 08:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2061542]: [15565-1] 2024-05-31 08:05:29: pid 2061542: LOG: statement: SHOW POOL_NODES; May 31 08:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2061542]: [15566-1] 2024-05-31 08:05:29: pid 2061542: LOG: DB node id: 0 backend pid: 2174577 statement: SELECT pg_catalog.version() May 31 08:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2061542]: [15567-1] 2024-05-31 08:05:29: pid 2061542: LOG: statement: DISCARD ALL May 31 08:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2061542]: [15568-1] 2024-05-31 08:05:29: pid 2061542: LOG: DB node id: 0 backend pid: 2174577 statement: DISCARD ALL May 31 08:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2061542]: [15569-1] 2024-05-31 08:05:29: pid 2061542: LOG: DB node id: 1 backend pid: 4031741 statement: DISCARD ALL May 31 08:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2061542]: [15570-1] 2024-05-31 08:05:29: pid 2061542: LOG: DB node id: 2 backend pid: 4000339 statement: DISCARD ALL May 31 08:05:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15744-1] 2024-05-31 08:05:30: pid 241871: LOG: child process with pid: 2106915 exits with status 256 May 31 08:05:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15745-1] 2024-05-31 08:05:30: pid 241871: LOG: fork a new child process with pid: 2174593 May 31 08:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2090608]: [15611-1] 2024-05-31 08:10:29: pid 2090608: LOG: statement: SHOW POOL_NODES; May 31 08:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2090608]: [15612-1] 2024-05-31 08:10:29: pid 2090608: LOG: DB node id: 0 backend pid: 2175830 statement: SELECT pg_catalog.version() May 31 08:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2090608]: [15613-1] 2024-05-31 08:10:29: pid 2090608: LOG: statement: DISCARD ALL May 31 08:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2090608]: [15614-1] 2024-05-31 08:10:29: pid 2090608: LOG: DB node id: 0 backend pid: 2175830 statement: DISCARD ALL May 31 08:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2090608]: [15615-1] 2024-05-31 08:10:29: pid 2090608: LOG: DB node id: 1 backend pid: 4032711 statement: DISCARD ALL May 31 08:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2090608]: [15616-1] 2024-05-31 08:10:29: pid 2090608: LOG: DB node id: 2 backend pid: 4001306 statement: DISCARD ALL May 31 08:10:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15746-1] 2024-05-31 08:10:30: pid 241871: LOG: child process with pid: 2001342 exits with status 256 May 31 08:10:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15747-1] 2024-05-31 08:10:30: pid 241871: LOG: fork a new child process with pid: 2175846 May 31 08:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2017635]: [15495-1] 2024-05-31 08:15:29: pid 2017635: LOG: statement: SHOW POOL_NODES; May 31 08:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2017635]: [15496-1] 2024-05-31 08:15:29: pid 2017635: LOG: DB node id: 0 backend pid: 2177087 statement: SELECT pg_catalog.version() May 31 08:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2017635]: [15497-1] 2024-05-31 08:15:29: pid 2017635: LOG: statement: DISCARD ALL May 31 08:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2017635]: [15498-1] 2024-05-31 08:15:29: pid 2017635: LOG: DB node id: 0 backend pid: 2177087 statement: DISCARD ALL May 31 08:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2017635]: [15499-1] 2024-05-31 08:15:29: pid 2017635: LOG: DB node id: 1 backend pid: 4033648 statement: DISCARD ALL May 31 08:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2017635]: [15500-1] 2024-05-31 08:15:29: pid 2017635: LOG: DB node id: 2 backend pid: 4002274 statement: DISCARD ALL May 31 08:15:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15748-1] 2024-05-31 08:15:30: pid 241871: LOG: child process with pid: 2061542 exits with status 256 May 31 08:15:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15749-1] 2024-05-31 08:15:30: pid 241871: LOG: fork a new child process with pid: 2177100 May 31 08:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2116935]: [15653-1] 2024-05-31 08:20:29: pid 2116935: LOG: statement: SHOW POOL_NODES; May 31 08:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2116935]: [15654-1] 2024-05-31 08:20:29: pid 2116935: LOG: DB node id: 0 backend pid: 2178336 statement: SELECT pg_catalog.version() May 31 08:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2116935]: [15655-1] 2024-05-31 08:20:29: pid 2116935: LOG: statement: DISCARD ALL May 31 08:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2116935]: [15656-1] 2024-05-31 08:20:29: pid 2116935: LOG: DB node id: 0 backend pid: 2178336 statement: DISCARD ALL May 31 08:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2116935]: [15657-1] 2024-05-31 08:20:29: pid 2116935: LOG: DB node id: 1 backend pid: 4034617 statement: DISCARD ALL May 31 08:20:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2116935]: [15658-1] 2024-05-31 08:20:29: pid 2116935: LOG: DB node id: 2 backend pid: 4003248 statement: DISCARD ALL May 31 08:20:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15750-1] 2024-05-31 08:20:30: pid 241871: LOG: child process with pid: 2090608 exits with status 256 May 31 08:20:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15751-1] 2024-05-31 08:20:30: pid 241871: LOG: fork a new child process with pid: 2178354 May 31 08:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2059046]: [15561-1] 2024-05-31 08:25:29: pid 2059046: LOG: statement: SHOW POOL_NODES; May 31 08:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2059046]: [15562-1] 2024-05-31 08:25:29: pid 2059046: LOG: DB node id: 0 backend pid: 2179597 statement: SELECT pg_catalog.version() May 31 08:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2059046]: [15563-1] 2024-05-31 08:25:29: pid 2059046: LOG: statement: DISCARD ALL May 31 08:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2059046]: [15564-1] 2024-05-31 08:25:29: pid 2059046: LOG: DB node id: 0 backend pid: 2179597 statement: DISCARD ALL May 31 08:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2059046]: [15565-1] 2024-05-31 08:25:29: pid 2059046: LOG: DB node id: 1 backend pid: 4035598 statement: DISCARD ALL May 31 08:25:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2059046]: [15566-1] 2024-05-31 08:25:29: pid 2059046: LOG: DB node id: 2 backend pid: 4004223 statement: DISCARD ALL May 31 08:25:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15752-1] 2024-05-31 08:25:30: pid 241871: LOG: child process with pid: 2017635 exits with status 256 May 31 08:25:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15753-1] 2024-05-31 08:25:30: pid 241871: LOG: fork a new child process with pid: 2179610 May 31 08:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2094376]: [15617-1] 2024-05-31 08:30:29: pid 2094376: LOG: statement: SHOW POOL_NODES; May 31 08:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2094376]: [15618-1] 2024-05-31 08:30:29: pid 2094376: LOG: DB node id: 0 backend pid: 2180846 statement: SELECT pg_catalog.version() May 31 08:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2094376]: [15619-1] 2024-05-31 08:30:29: pid 2094376: LOG: statement: DISCARD ALL May 31 08:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2094376]: [15620-1] 2024-05-31 08:30:29: pid 2094376: LOG: DB node id: 0 backend pid: 2180846 statement: DISCARD ALL May 31 08:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2094376]: [15621-1] 2024-05-31 08:30:29: pid 2094376: LOG: DB node id: 1 backend pid: 4036568 statement: DISCARD ALL May 31 08:30:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2094376]: [15622-1] 2024-05-31 08:30:29: pid 2094376: LOG: DB node id: 2 backend pid: 4005160 statement: DISCARD ALL May 31 08:30:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15754-1] 2024-05-31 08:30:30: pid 241871: LOG: child process with pid: 2116935 exits with status 256 May 31 08:30:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15755-1] 2024-05-31 08:30:30: pid 241871: LOG: fork a new child process with pid: 2180863 May 31 08:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1978740]: [15433-1] 2024-05-31 08:35:29: pid 1978740: LOG: statement: SHOW POOL_NODES; May 31 08:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1978740]: [15434-1] 2024-05-31 08:35:29: pid 1978740: LOG: DB node id: 0 backend pid: 2182102 statement: SELECT pg_catalog.version() May 31 08:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1978740]: [15435-1] 2024-05-31 08:35:29: pid 1978740: LOG: statement: DISCARD ALL May 31 08:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1978740]: [15436-1] 2024-05-31 08:35:29: pid 1978740: LOG: DB node id: 0 backend pid: 2182102 statement: DISCARD ALL May 31 08:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1978740]: [15437-1] 2024-05-31 08:35:29: pid 1978740: LOG: DB node id: 1 backend pid: 4037548 statement: DISCARD ALL May 31 08:35:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1978740]: [15438-1] 2024-05-31 08:35:29: pid 1978740: LOG: DB node id: 2 backend pid: 4006132 statement: DISCARD ALL May 31 08:35:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15756-1] 2024-05-31 08:35:30: pid 241871: LOG: child process with pid: 2059046 exits with status 256 May 31 08:35:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15757-1] 2024-05-31 08:35:30: pid 241871: LOG: fork a new child process with pid: 2182114 May 31 08:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2047746]: [15543-1] 2024-05-31 08:40:29: pid 2047746: LOG: statement: SHOW POOL_NODES; May 31 08:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2047746]: [15544-1] 2024-05-31 08:40:29: pid 2047746: LOG: DB node id: 0 backend pid: 2183350 statement: SELECT pg_catalog.version() May 31 08:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2047746]: [15545-1] 2024-05-31 08:40:29: pid 2047746: LOG: statement: DISCARD ALL May 31 08:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2047746]: [15546-1] 2024-05-31 08:40:29: pid 2047746: LOG: DB node id: 0 backend pid: 2183350 statement: DISCARD ALL May 31 08:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2047746]: [15547-1] 2024-05-31 08:40:29: pid 2047746: LOG: DB node id: 1 backend pid: 4038488 statement: DISCARD ALL May 31 08:40:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2047746]: [15548-1] 2024-05-31 08:40:29: pid 2047746: LOG: DB node id: 2 backend pid: 4007109 statement: DISCARD ALL May 31 08:40:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15758-1] 2024-05-31 08:40:30: pid 241871: LOG: child process with pid: 2094376 exits with status 256 May 31 08:40:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15759-1] 2024-05-31 08:40:30: pid 241871: LOG: fork a new child process with pid: 2183367 May 31 08:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2105662]: [15635-1] 2024-05-31 08:45:29: pid 2105662: LOG: statement: SHOW POOL_NODES; May 31 08:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2105662]: [15636-1] 2024-05-31 08:45:29: pid 2105662: LOG: DB node id: 0 backend pid: 2184605 statement: SELECT pg_catalog.version() May 31 08:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2105662]: [15637-1] 2024-05-31 08:45:29: pid 2105662: LOG: statement: DISCARD ALL May 31 08:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2105662]: [15638-1] 2024-05-31 08:45:29: pid 2105662: LOG: DB node id: 0 backend pid: 2184605 statement: DISCARD ALL May 31 08:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2105662]: [15639-1] 2024-05-31 08:45:29: pid 2105662: LOG: DB node id: 1 backend pid: 4039463 statement: DISCARD ALL May 31 08:45:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2105662]: [15640-1] 2024-05-31 08:45:29: pid 2105662: LOG: DB node id: 2 backend pid: 4008078 statement: DISCARD ALL May 31 08:45:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15760-1] 2024-05-31 08:45:30: pid 241871: LOG: child process with pid: 1978740 exits with status 256 May 31 08:45:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15761-1] 2024-05-31 08:45:30: pid 241871: LOG: fork a new child process with pid: 2184621 May 31 08:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1954907]: [15395-1] 2024-05-31 08:50:29: pid 1954907: LOG: statement: SHOW POOL_NODES; May 31 08:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1954907]: [15396-1] 2024-05-31 08:50:29: pid 1954907: LOG: DB node id: 0 backend pid: 2185857 statement: SELECT pg_catalog.version() May 31 08:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1954907]: [15397-1] 2024-05-31 08:50:29: pid 1954907: LOG: statement: DISCARD ALL May 31 08:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1954907]: [15398-1] 2024-05-31 08:50:29: pid 1954907: LOG: DB node id: 0 backend pid: 2185857 statement: DISCARD ALL May 31 08:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1954907]: [15399-1] 2024-05-31 08:50:29: pid 1954907: LOG: DB node id: 1 backend pid: 4040439 statement: DISCARD ALL May 31 08:50:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1954907]: [15400-1] 2024-05-31 08:50:29: pid 1954907: LOG: DB node id: 2 backend pid: 4009045 statement: DISCARD ALL May 31 08:50:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15762-1] 2024-05-31 08:50:30: pid 241871: LOG: child process with pid: 2047746 exits with status 256 May 31 08:50:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15763-1] 2024-05-31 08:50:30: pid 241871: LOG: fork a new child process with pid: 2185873 May 31 08:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2021390]: [15501-1] 2024-05-31 08:55:29: pid 2021390: LOG: statement: SHOW POOL_NODES; May 31 08:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2021390]: [15502-1] 2024-05-31 08:55:29: pid 2021390: LOG: DB node id: 0 backend pid: 2187113 statement: SELECT pg_catalog.version() May 31 08:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2021390]: [15503-1] 2024-05-31 08:55:29: pid 2021390: LOG: statement: DISCARD ALL May 31 08:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2021390]: [15504-1] 2024-05-31 08:55:29: pid 2021390: LOG: DB node id: 0 backend pid: 2187113 statement: DISCARD ALL May 31 08:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2021390]: [15505-1] 2024-05-31 08:55:29: pid 2021390: LOG: DB node id: 1 backend pid: 4041408 statement: DISCARD ALL May 31 08:55:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2021390]: [15506-1] 2024-05-31 08:55:29: pid 2021390: LOG: DB node id: 2 backend pid: 4010016 statement: DISCARD ALL May 31 08:55:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15764-1] 2024-05-31 08:55:30: pid 241871: LOG: child process with pid: 2105662 exits with status 256 May 31 08:55:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15765-1] 2024-05-31 08:55:30: pid 241871: LOG: fork a new child process with pid: 2187130 May 31 09:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2038962]: [15529-1] 2024-05-31 09:00:29: pid 2038962: LOG: statement: SHOW POOL_NODES; May 31 09:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2038962]: [15530-1] 2024-05-31 09:00:29: pid 2038962: LOG: DB node id: 0 backend pid: 2188362 statement: SELECT pg_catalog.version() May 31 09:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2038962]: [15531-1] 2024-05-31 09:00:29: pid 2038962: LOG: statement: DISCARD ALL May 31 09:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2038962]: [15532-1] 2024-05-31 09:00:29: pid 2038962: LOG: DB node id: 0 backend pid: 2188362 statement: DISCARD ALL May 31 09:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2038962]: [15533-1] 2024-05-31 09:00:29: pid 2038962: LOG: DB node id: 1 backend pid: 4042384 statement: DISCARD ALL May 31 09:00:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2038962]: [15534-1] 2024-05-31 09:00:29: pid 2038962: LOG: DB node id: 2 backend pid: 4010964 statement: DISCARD ALL May 31 09:00:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15766-1] 2024-05-31 09:00:30: pid 241871: LOG: child process with pid: 1954907 exits with status 256 May 31 09:00:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15767-1] 2024-05-31 09:00:30: pid 241871: LOG: fork a new child process with pid: 2188376 May 31 09:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2005105]: [15475-1] 2024-05-31 09:05:29: pid 2005105: LOG: statement: SHOW POOL_NODES; May 31 09:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2005105]: [15476-1] 2024-05-31 09:05:29: pid 2005105: LOG: DB node id: 0 backend pid: 2189611 statement: SELECT pg_catalog.version() May 31 09:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2005105]: [15477-1] 2024-05-31 09:05:29: pid 2005105: LOG: statement: DISCARD ALL May 31 09:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2005105]: [15478-1] 2024-05-31 09:05:29: pid 2005105: LOG: DB node id: 0 backend pid: 2189611 statement: DISCARD ALL May 31 09:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2005105]: [15479-1] 2024-05-31 09:05:29: pid 2005105: LOG: DB node id: 1 backend pid: 4043358 statement: DISCARD ALL May 31 09:05:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2005105]: [15480-1] 2024-05-31 09:05:29: pid 2005105: LOG: DB node id: 2 backend pid: 4011923 statement: DISCARD ALL May 31 09:05:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15768-1] 2024-05-31 09:05:30: pid 241871: LOG: child process with pid: 2021390 exits with status 256 May 31 09:05:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15769-1] 2024-05-31 09:05:30: pid 241871: LOG: fork a new child process with pid: 2189629 May 31 09:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2177100]: [15749-1] 2024-05-31 09:10:29: pid 2177100: LOG: statement: SHOW POOL_NODES; May 31 09:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2177100]: [15750-1] 2024-05-31 09:10:29: pid 2177100: LOG: DB node id: 0 backend pid: 2190861 statement: SELECT pg_catalog.version() May 31 09:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2177100]: [15751-1] 2024-05-31 09:10:29: pid 2177100: LOG: statement: DISCARD ALL May 31 09:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2177100]: [15752-1] 2024-05-31 09:10:29: pid 2177100: LOG: DB node id: 0 backend pid: 2190861 statement: DISCARD ALL May 31 09:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2177100]: [15753-1] 2024-05-31 09:10:29: pid 2177100: LOG: DB node id: 1 backend pid: 4044330 statement: DISCARD ALL May 31 09:10:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2177100]: [15754-1] 2024-05-31 09:10:29: pid 2177100: LOG: DB node id: 2 backend pid: 4012890 statement: DISCARD ALL May 31 09:10:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15770-1] 2024-05-31 09:10:30: pid 241871: LOG: child process with pid: 2038962 exits with status 256 May 31 09:10:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15771-1] 2024-05-31 09:10:30: pid 241871: LOG: fork a new child process with pid: 2190877 May 31 09:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2030180]: [15515-1] 2024-05-31 09:15:29: pid 2030180: LOG: statement: SHOW POOL_NODES; May 31 09:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2030180]: [15516-1] 2024-05-31 09:15:29: pid 2030180: LOG: DB node id: 0 backend pid: 2192258 statement: SELECT pg_catalog.version() May 31 09:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2030180]: [15517-1] 2024-05-31 09:15:29: pid 2030180: LOG: statement: DISCARD ALL May 31 09:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2030180]: [15518-1] 2024-05-31 09:15:29: pid 2030180: LOG: DB node id: 0 backend pid: 2192258 statement: DISCARD ALL May 31 09:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2030180]: [15519-1] 2024-05-31 09:15:29: pid 2030180: LOG: DB node id: 1 backend pid: 4045283 statement: DISCARD ALL May 31 09:15:29 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2030180]: [15520-1] 2024-05-31 09:15:29: pid 2030180: LOG: DB node id: 2 backend pid: 4013864 statement: DISCARD ALL May 31 09:15:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15772-1] 2024-05-31 09:15:30: pid 241871: LOG: child process with pid: 2005105 exits with status 256 May 31 09:15:30 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15773-1] 2024-05-31 09:15:30: pid 241871: LOG: fork a new child process with pid: 2192275 May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: Stopping pgpool-II... May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15774-1] 2024-05-31 09:16:12: pid 241871: LOG: shutting down by signal 2 May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241878]: [34-1] 2024-05-31 09:16:12: pid 241878: LOG: Watchdog is shutting down May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15775-1] 2024-05-31 09:16:12: pid 241871: LOG: terminating all child processes May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[241871]: [15776-1] 2024-05-31 09:16:12: pid 241871: LOG: Pgpool-II system is shutdown May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Deactivated successfully. May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: Stopped pgpool-II. May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Consumed 4h 2min 26.713s CPU time. May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: Started pgpool-II. May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [1-1] 2024-05-31 09:16:12: pid 2192461: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [2-1] 2024-05-31 09:16:12: pid 2192461: LOG: memory cache initialized May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [2-2] 2024-05-31 09:16:12: pid 2192461: DETAIL: memcache blocks :64 May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [3-1] 2024-05-31 09:16:12: pid 2192461: LOG: allocating (138460264) bytes of shared memory segment May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [4-1] 2024-05-31 09:16:12: pid 2192461: LOG: allocating shared memory segment of size: 138460264 May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [5-1] 2024-05-31 09:16:12: pid 2192461: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [6-1] 2024-05-31 09:16:12: pid 2192461: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [7-1] 2024-05-31 09:16:12: pid 2192461: LOG: memory cache initialized May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [7-2] 2024-05-31 09:16:12: pid 2192461: DETAIL: memcache blocks :64 May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [8-1] 2024-05-31 09:16:12: pid 2192461: LOG: pool_discard_oid_maps: discarded memqcache oid maps May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [9-1] 2024-05-31 09:16:12: pid 2192461: LOG: waiting for watchdog to initialize May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [9-1] 2024-05-31 09:16:12: pid 2192468: LOG: setting the local watchdog node name to "pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [10-1] 2024-05-31 09:16:12: pid 2192468: LOG: watchdog cluster is configured with 2 remote nodes May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [11-1] 2024-05-31 09:16:12: pid 2192468: LOG: watchdog remote node:0 on pgsql02:9000 May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [12-1] 2024-05-31 09:16:12: pid 2192468: LOG: watchdog remote node:1 on pgsql03:9000 May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [13-1] 2024-05-31 09:16:12: pid 2192468: LOG: interface monitoring is disabled in watchdog May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [14-1] 2024-05-31 09:16:12: pid 2192468: INFO: IPC socket path: "/var/run/postgresql/.s.PGPOOLWD_CMD.9000" May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [15-1] 2024-05-31 09:16:12: pid 2192468: LOG: watchdog node state changed from [DEAD] to [LOADING] May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [16-1] 2024-05-31 09:16:12: pid 2192468: LOG: new outbound connection to pgsql03:9000 May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [17-1] 2024-05-31 09:16:12: pid 2192468: LOG: new outbound connection to pgsql02:9000 May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [18-1] 2024-05-31 09:16:12: pid 2192468: LOG: watchdog node state changed from [LOADING] to [INITIALIZING] May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [19-1] 2024-05-31 09:16:12: pid 2192468: LOG: new watchdog node connection is received from "192.168.56.12:1719" May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [20-1] 2024-05-31 09:16:12: pid 2192468: LOG: new node joined the cluster hostname:"pgsql03" port:9000 pgpool_port:9999 May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [20-2] 2024-05-31 09:16:12: pid 2192468: DETAIL: Pgpool-II version:"4.3.7" watchdog messaging version: 1.2 May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [21-1] 2024-05-31 09:16:12: pid 2192468: LOG: setting the remote node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" as watchdog cluster leader May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [22-1] 2024-05-31 09:16:12: pid 2192468: LOG: new watchdog node connection is received from "192.168.56.11:47752" May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [23-1] 2024-05-31 09:16:12: pid 2192468: LOG: new node joined the cluster hostname:"pgsql02" port:9000 pgpool_port:9999 May 31 09:16:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [23-2] 2024-05-31 09:16:12: pid 2192468: DETAIL: Pgpool-II version:"4.3.7" watchdog messaging version: 1.2 May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [24-1] 2024-05-31 09:16:13: pid 2192468: LOG: watchdog node state changed from [INITIALIZING] to [STANDBY] May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [25-1] 2024-05-31 09:16:13: pid 2192468: LOG: signal_user1_to_parent_with_reason(1) May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [26-1] 2024-05-31 09:16:13: pid 2192468: LOG: successfully joined the watchdog cluster as standby node May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [26-2] 2024-05-31 09:16:13: pid 2192468: DETAIL: our join coordinator request is accepted by cluster leader node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [10-1] 2024-05-31 09:16:13: pid 2192461: LOG: watchdog process is initialized May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [10-2] 2024-05-31 09:16:13: pid 2192461: DETAIL: watchdog messaging data version: 1.2 May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [11-1] 2024-05-31 09:16:13: pid 2192461: LOG: Pgpool-II parent process received SIGUSR1 May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [12-1] 2024-05-31 09:16:13: pid 2192461: LOG: Pgpool-II parent process received watchdog state change signal from watchdog May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [13-1] 2024-05-31 09:16:13: pid 2192461: LOG: we have joined the watchdog cluster as STANDBY node May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [13-2] 2024-05-31 09:16:13: pid 2192461: DETAIL: syncing the backend states from the LEADER watchdog node May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [27-1] 2024-05-31 09:16:13: pid 2192468: LOG: received the get data request from local pgpool-II on IPC interface May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [28-1] 2024-05-31 09:16:13: pid 2192468: LOG: get data request from local pgpool-II node received on IPC interface is forwarded to leader watchdog node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [28-2] 2024-05-31 09:16:13: pid 2192468: DETAIL: waiting for the reply... May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192471]: [11-1] 2024-05-31 09:16:13: pid 2192471: LOG: 3 watchdog nodes are configured for lifecheck May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192471]: [12-1] 2024-05-31 09:16:13: pid 2192471: LOG: watchdog nodes ID:0 Name:"pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192471]: [12-2] 2024-05-31 09:16:13: pid 2192471: DETAIL: Host:"pgsql01" WD Port:9000 pgpool-II port:9999 May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192471]: [13-1] 2024-05-31 09:16:13: pid 2192471: LOG: watchdog nodes ID:1 Name:"pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192471]: [13-2] 2024-05-31 09:16:13: pid 2192471: DETAIL: Host:"pgsql02" WD Port:9000 pgpool-II port:9999 May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192471]: [14-1] 2024-05-31 09:16:13: pid 2192471: LOG: watchdog nodes ID:2 Name:"pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192471]: [14-2] 2024-05-31 09:16:13: pid 2192471: DETAIL: Host:"pgsql03" WD Port:9000 pgpool-II port:9999 May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [14-1] 2024-05-31 09:16:13: pid 2192461: LOG: leader watchdog node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" returned status for 3 backend nodes May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [15-1] 2024-05-31 09:16:13: pid 2192461: LOG: backend:0 is set to UP status May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [15-2] 2024-05-31 09:16:13: pid 2192461: DETAIL: backend:0 is UP on cluster leader "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [16-1] 2024-05-31 09:16:13: pid 2192461: LOG: backend:1 is set to UP status May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [16-2] 2024-05-31 09:16:13: pid 2192461: DETAIL: backend:1 is UP on cluster leader "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [17-1] 2024-05-31 09:16:13: pid 2192461: LOG: backend:2 is set to UP status May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [17-2] 2024-05-31 09:16:13: pid 2192461: DETAIL: backend:2 is UP on cluster leader "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [18-1] 2024-05-31 09:16:13: pid 2192461: LOG: Setting up socket for 0.0.0.0:9999 May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [19-1] 2024-05-31 09:16:13: pid 2192461: LOG: Setting up socket for :::9999 May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192471]: [15-1] 2024-05-31 09:16:13: pid 2192471: LOG: watchdog lifecheck trusted server "pgsql01" added for the availability check May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192471]: [16-1] 2024-05-31 09:16:13: pid 2192471: LOG: watchdog lifecheck trusted server "pgsql02" added for the availability check May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192471]: [17-1] 2024-05-31 09:16:13: pid 2192471: LOG: watchdog lifecheck trusted server "pgsql03" added for the availability check May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [20-1] 2024-05-31 09:16:13: pid 2192461: LOG: pgpool-II successfully started. version 4.3.7 (tamahomeboshi) May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [21-1] 2024-05-31 09:16:13: pid 2192461: LOG: node status[0]: 0 May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [22-1] 2024-05-31 09:16:13: pid 2192461: LOG: node status[1]: 0 May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [23-1] 2024-05-31 09:16:13: pid 2192461: LOG: node status[2]: 0 May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192576]: [20-1] 2024-05-31 09:16:13: pid 2192576: LOG: PCP process: 2192576 started May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192578]: [20-1] 2024-05-31 09:16:13: pid 2192578: LOG: process started May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192577]: [20-1] 2024-05-31 09:16:13: pid 2192577: LOG: process started May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192580]: [20-1] 2024-05-31 09:16:13: pid 2192580: LOG: process started May 31 09:16:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192579]: [20-1] 2024-05-31 09:16:13: pid 2192579: LOG: process started May 31 09:16:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192472]: [15-1] 2024-05-31 09:16:14: pid 2192472: LOG: set SO_REUSEPORT option to the socket May 31 09:16:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192472]: [16-1] 2024-05-31 09:16:14: pid 2192472: LOG: creating watchdog heartbeat receive socket. May 31 09:16:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192472]: [16-2] 2024-05-31 09:16:14: pid 2192472: DETAIL: set SO_REUSEPORT May 31 09:16:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192474]: [15-1] 2024-05-31 09:16:14: pid 2192474: LOG: set SO_REUSEPORT option to the socket May 31 09:16:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192474]: [16-1] 2024-05-31 09:16:14: pid 2192474: LOG: creating watchdog heartbeat receive socket. May 31 09:16:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192474]: [16-2] 2024-05-31 09:16:14: pid 2192474: DETAIL: set SO_REUSEPORT May 31 09:16:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192475]: [15-1] 2024-05-31 09:16:14: pid 2192475: LOG: set SO_REUSEPORT option to the socket May 31 09:16:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192475]: [16-1] 2024-05-31 09:16:14: pid 2192475: LOG: creating socket for sending heartbeat May 31 09:16:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192475]: [16-2] 2024-05-31 09:16:14: pid 2192475: DETAIL: set SO_REUSEPORT May 31 09:16:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192473]: [15-1] 2024-05-31 09:16:14: pid 2192473: LOG: set SO_REUSEPORT option to the socket May 31 09:16:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192473]: [16-1] 2024-05-31 09:16:14: pid 2192473: LOG: creating socket for sending heartbeat May 31 09:16:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192473]: [16-2] 2024-05-31 09:16:14: pid 2192473: DETAIL: set SO_REUSEPORT May 31 09:17:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192471]: [18-1] 2024-05-31 09:17:53: pid 2192471: LOG: watchdog: lifecheck started May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [29-1] 2024-05-31 09:19:14: pid 2192468: WARNING: network IP is removed and system has no IP is assigned May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [29-2] 2024-05-31 09:19:14: pid 2192468: DETAIL: changing the state to in network trouble May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [30-1] 2024-05-31 09:19:14: pid 2192468: LOG: watchdog node state changed from [STANDBY] to [IN NETWORK TROUBLE] May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [31-1] 2024-05-31 09:19:14: pid 2192468: FATAL: system has lost the network May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192468]: [32-1] 2024-05-31 09:19:14: pid 2192468: LOG: Watchdog is shutting down May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [24-1] 2024-05-31 09:19:14: pid 2192461: LOG: watchdog child process with pid: 2192468 exits with status 768 May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [25-1] 2024-05-31 09:19:14: pid 2192461: FATAL: watchdog child process exit with fatal error. exiting pgpool-II May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2193260]: [25-1] 2024-05-31 09:19:14: pid 2193260: LOG: setting the local watchdog node name to "pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2193260]: [26-1] 2024-05-31 09:19:14: pid 2193260: LOG: watchdog cluster is configured with 2 remote nodes May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2193260]: [27-1] 2024-05-31 09:19:14: pid 2193260: LOG: watchdog remote node:0 on pgsql02:9000 May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2193260]: [28-1] 2024-05-31 09:19:14: pid 2193260: LOG: watchdog remote node:1 on pgsql03:9000 May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2193260]: [29-1] 2024-05-31 09:19:14: pid 2193260: LOG: interface monitoring is disabled in watchdog May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2193260]: [30-1] 2024-05-31 09:19:14: pid 2193260: INFO: IPC socket path: "/var/run/postgresql/.s.PGPOOLWD_CMD.9000" May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2193260]: [31-1] 2024-05-31 09:19:14: pid 2193260: LOG: connect on socket failed May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2193260]: [31-2] 2024-05-31 09:19:14: pid 2193260: DETAIL: connect failed with error: "Network is unreachable" May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2193260]: [32-1] 2024-05-31 09:19:14: pid 2193260: LOG: connect on socket failed May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2193260]: [32-2] 2024-05-31 09:19:14: pid 2193260: DETAIL: connect failed with error: "Network is unreachable" May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2193260]: [33-1] 2024-05-31 09:19:14: pid 2193260: LOG: watchdog node state changed from [DEAD] to [LOADING] May 31 09:19:14 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2192461]: [26-1] 2024-05-31 09:19:14: pid 2192461: LOG: shutting down May 31 09:19:15 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2193260]: [34-1] 2024-05-31 09:19:15: pid 2193260: LOG: Watchdog is shutting down May 31 09:19:16 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Main process exited, code=exited, status=1/FAILURE May 31 09:19:16 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Failed with result 'exit-code'. May 31 09:19:16 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Consumed 1.206s CPU time. -- Boot 192d986ab3754cf3923502b4701dfdb5 -- May 31 09:22:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: Started pgpool-II. May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [1-1] 2024-05-31 09:22:08: pid 628: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [2-1] 2024-05-31 09:22:08: pid 628: LOG: memory cache initialized May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [2-2] 2024-05-31 09:22:08: pid 628: DETAIL: memcache blocks :64 May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [3-1] 2024-05-31 09:22:08: pid 628: LOG: allocating (138460264) bytes of shared memory segment May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [4-1] 2024-05-31 09:22:08: pid 628: LOG: allocating shared memory segment of size: 138460264 May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [5-1] 2024-05-31 09:22:08: pid 628: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [6-1] 2024-05-31 09:22:08: pid 628: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [7-1] 2024-05-31 09:22:08: pid 628: LOG: memory cache initialized May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [7-2] 2024-05-31 09:22:08: pid 628: DETAIL: memcache blocks :64 May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [8-1] 2024-05-31 09:22:08: pid 628: LOG: pool_discard_oid_maps: discarded memqcache oid maps May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [9-1] 2024-05-31 09:22:08: pid 628: LOG: waiting for watchdog to initialize May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [9-1] 2024-05-31 09:22:08: pid 691: LOG: setting the local watchdog node name to "pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [10-1] 2024-05-31 09:22:08: pid 691: LOG: watchdog cluster is configured with 2 remote nodes May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [11-1] 2024-05-31 09:22:08: pid 691: LOG: watchdog remote node:0 on pgsql02:9000 May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [12-1] 2024-05-31 09:22:08: pid 691: LOG: watchdog remote node:1 on pgsql03:9000 May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [13-1] 2024-05-31 09:22:08: pid 691: LOG: interface monitoring is disabled in watchdog May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [14-1] 2024-05-31 09:22:08: pid 691: INFO: IPC socket path: "/var/run/postgresql/.s.PGPOOLWD_CMD.9000" May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [15-1] 2024-05-31 09:22:08: pid 691: LOG: watchdog node state changed from [DEAD] to [LOADING] May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [16-1] 2024-05-31 09:22:08: pid 691: LOG: new outbound connection to pgsql02:9000 May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [17-1] 2024-05-31 09:22:08: pid 691: LOG: new outbound connection to pgsql03:9000 May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [18-1] 2024-05-31 09:22:08: pid 691: LOG: watchdog node state changed from [LOADING] to [INITIALIZING] May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [19-1] 2024-05-31 09:22:08: pid 691: LOG: remote node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" is reporting that it has found us again May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [20-1] 2024-05-31 09:22:08: pid 691: LOG: setting the remote node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" as watchdog cluster leader May 31 09:22:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [21-1] 2024-05-31 09:22:08: pid 691: LOG: remote node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" is reporting that it has found us again May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [22-1] 2024-05-31 09:22:09: pid 691: LOG: watchdog node state changed from [INITIALIZING] to [STANDBY] May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [23-1] 2024-05-31 09:22:09: pid 691: LOG: signal_user1_to_parent_with_reason(1) May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [24-1] 2024-05-31 09:22:09: pid 691: LOG: successfully joined the watchdog cluster as standby node May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [24-2] 2024-05-31 09:22:09: pid 691: DETAIL: our join coordinator request is accepted by cluster leader node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [10-1] 2024-05-31 09:22:09: pid 628: LOG: watchdog process is initialized May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [10-2] 2024-05-31 09:22:09: pid 628: DETAIL: watchdog messaging data version: 1.2 May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [11-1] 2024-05-31 09:22:09: pid 628: LOG: Pgpool-II parent process received SIGUSR1 May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [12-1] 2024-05-31 09:22:09: pid 628: LOG: Pgpool-II parent process received watchdog state change signal from watchdog May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [13-1] 2024-05-31 09:22:09: pid 628: LOG: we have joined the watchdog cluster as STANDBY node May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [13-2] 2024-05-31 09:22:09: pid 628: DETAIL: syncing the backend states from the LEADER watchdog node May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [25-1] 2024-05-31 09:22:09: pid 691: LOG: received the get data request from local pgpool-II on IPC interface May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [26-1] 2024-05-31 09:22:09: pid 691: LOG: get data request from local pgpool-II node received on IPC interface is forwarded to leader watchdog node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [26-2] 2024-05-31 09:22:09: pid 691: DETAIL: waiting for the reply... May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [14-1] 2024-05-31 09:22:09: pid 628: LOG: leader watchdog node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" returned status for 3 backend nodes May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [15-1] 2024-05-31 09:22:09: pid 628: LOG: backend:0 is set to down status May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [15-2] 2024-05-31 09:22:09: pid 628: DETAIL: backend:0 is DOWN on cluster leader "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [16-1] 2024-05-31 09:22:09: pid 628: LOG: backend:1 is set to UP status May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [16-2] 2024-05-31 09:22:09: pid 628: DETAIL: backend:1 is UP on cluster leader "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [17-1] 2024-05-31 09:22:09: pid 628: LOG: backend:2 is set to UP status May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [17-2] 2024-05-31 09:22:09: pid 628: DETAIL: backend:2 is UP on cluster leader "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [18-1] 2024-05-31 09:22:09: pid 628: LOG: Setting up socket for 0.0.0.0:9999 May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [19-1] 2024-05-31 09:22:09: pid 628: LOG: Setting up socket for :::9999 May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[745]: [11-1] 2024-05-31 09:22:09: pid 745: LOG: 3 watchdog nodes are configured for lifecheck May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[745]: [12-1] 2024-05-31 09:22:09: pid 745: LOG: watchdog nodes ID:0 Name:"pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[745]: [12-2] 2024-05-31 09:22:09: pid 745: DETAIL: Host:"pgsql01" WD Port:9000 pgpool-II port:9999 May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[745]: [13-1] 2024-05-31 09:22:09: pid 745: LOG: watchdog nodes ID:1 Name:"pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[745]: [13-2] 2024-05-31 09:22:09: pid 745: DETAIL: Host:"pgsql02" WD Port:9000 pgpool-II port:9999 May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[745]: [14-1] 2024-05-31 09:22:09: pid 745: LOG: watchdog nodes ID:2 Name:"pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[745]: [14-2] 2024-05-31 09:22:09: pid 745: DETAIL: Host:"pgsql03" WD Port:9000 pgpool-II port:9999 May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[745]: [15-1] 2024-05-31 09:22:09: pid 745: LOG: watchdog lifecheck trusted server "pgsql01" added for the availability check May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[745]: [16-1] 2024-05-31 09:22:09: pid 745: LOG: watchdog lifecheck trusted server "pgsql02" added for the availability check May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[745]: [17-1] 2024-05-31 09:22:09: pid 745: LOG: watchdog lifecheck trusted server "pgsql03" added for the availability check May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[851]: [20-1] 2024-05-31 09:22:09: pid 851: LOG: process started May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[852]: [20-1] 2024-05-31 09:22:09: pid 852: LOG: process started May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[850]: [20-1] 2024-05-31 09:22:09: pid 850: LOG: PCP process: 850 started May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[853]: [20-1] 2024-05-31 09:22:09: pid 853: LOG: process started May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [20-1] 2024-05-31 09:22:09: pid 628: LOG: pgpool-II successfully started. version 4.3.7 (tamahomeboshi) May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [21-1] 2024-05-31 09:22:09: pid 628: LOG: node status[0]: 0 May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [22-1] 2024-05-31 09:22:09: pid 628: LOG: node status[1]: 0 May 31 09:22:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [23-1] 2024-05-31 09:22:09: pid 628: LOG: node status[2]: 0 May 31 09:22:10 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[775]: [15-1] 2024-05-31 09:22:10: pid 775: LOG: set SO_REUSEPORT option to the socket May 31 09:22:10 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[776]: [15-1] 2024-05-31 09:22:10: pid 776: LOG: set SO_REUSEPORT option to the socket May 31 09:22:10 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[776]: [16-1] 2024-05-31 09:22:10: pid 776: LOG: creating socket for sending heartbeat May 31 09:22:10 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[776]: [16-2] 2024-05-31 09:22:10: pid 776: DETAIL: set SO_REUSEPORT May 31 09:22:10 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[775]: [16-1] 2024-05-31 09:22:10: pid 775: LOG: creating watchdog heartbeat receive socket. May 31 09:22:10 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[775]: [16-2] 2024-05-31 09:22:10: pid 775: DETAIL: set SO_REUSEPORT May 31 09:22:10 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[777]: [15-1] 2024-05-31 09:22:10: pid 777: LOG: set SO_REUSEPORT option to the socket May 31 09:22:10 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[777]: [16-1] 2024-05-31 09:22:10: pid 777: LOG: creating watchdog heartbeat receive socket. May 31 09:22:10 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[777]: [16-2] 2024-05-31 09:22:10: pid 777: DETAIL: set SO_REUSEPORT May 31 09:22:10 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[778]: [15-1] 2024-05-31 09:22:10: pid 778: LOG: set SO_REUSEPORT option to the socket May 31 09:22:10 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[778]: [16-1] 2024-05-31 09:22:10: pid 778: LOG: creating socket for sending heartbeat May 31 09:22:10 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[778]: [16-2] 2024-05-31 09:22:10: pid 778: DETAIL: set SO_REUSEPORT May 31 09:22:10 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [27-1] 2024-05-31 09:22:10: pid 691: LOG: remote node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" is reporting that it has found us again May 31 09:22:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [28-1] 2024-05-31 09:22:13: pid 691: LOG: remote node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" is reporting that it has found us again May 31 09:22:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [29-1] 2024-05-31 09:22:13: pid 691: LOG: re-sending join coordinator message to leader node: "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:22:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [30-1] 2024-05-31 09:22:13: pid 691: LOG: signal_user1_to_parent_with_reason(1) May 31 09:22:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [31-1] 2024-05-31 09:22:13: pid 691: LOG: successfully joined the watchdog cluster as standby node May 31 09:22:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [31-2] 2024-05-31 09:22:13: pid 691: DETAIL: our join coordinator request is accepted by cluster leader node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:22:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [24-1] 2024-05-31 09:22:13: pid 628: LOG: Pgpool-II parent process received SIGUSR1 May 31 09:22:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [25-1] 2024-05-31 09:22:13: pid 628: LOG: Pgpool-II parent process received watchdog state change signal from watchdog May 31 09:22:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [26-1] 2024-05-31 09:22:13: pid 628: LOG: we have joined the watchdog cluster as STANDBY node May 31 09:22:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [26-2] 2024-05-31 09:22:13: pid 628: DETAIL: syncing the backend states from the LEADER watchdog node May 31 09:22:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [32-1] 2024-05-31 09:22:13: pid 691: LOG: received the get data request from local pgpool-II on IPC interface May 31 09:22:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [33-1] 2024-05-31 09:22:13: pid 691: LOG: get data request from local pgpool-II node received on IPC interface is forwarded to leader watchdog node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:22:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [33-2] 2024-05-31 09:22:13: pid 691: DETAIL: waiting for the reply... May 31 09:22:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [27-1] 2024-05-31 09:22:13: pid 628: LOG: leader watchdog node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" returned status for 3 backend nodes May 31 09:22:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [28-1] 2024-05-31 09:22:13: pid 628: LOG: backend nodes status remains same after the sync from "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:22:17 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [34-1] 2024-05-31 09:22:17: pid 691: LOG: new watchdog node connection is received from "192.168.56.12:8375" May 31 09:22:17 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [35-1] 2024-05-31 09:22:17: pid 691: LOG: new node joined the cluster hostname:"pgsql03" port:9000 pgpool_port:9999 May 31 09:22:17 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [35-2] 2024-05-31 09:22:17: pid 691: DETAIL: Pgpool-II version:"4.3.7" watchdog messaging version: 1.2 May 31 09:22:19 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [36-1] 2024-05-31 09:22:19: pid 691: LOG: new watchdog node connection is received from "192.168.56.11:55432" May 31 09:22:19 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [37-1] 2024-05-31 09:22:19: pid 691: LOG: new node joined the cluster hostname:"pgsql02" port:9000 pgpool_port:9999 May 31 09:22:19 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [37-2] 2024-05-31 09:22:19: pid 691: DETAIL: Pgpool-II version:"4.3.7" watchdog messaging version: 1.2 May 31 09:22:28 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[834]: [20-1] 2024-05-31 09:22:28: pid 834: LOG: statement: SHOW POOL_NODES; May 31 09:22:28 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[834]: [21-1] 2024-05-31 09:22:28: pid 834: LOG: DB node id: 1 backend pid: 4046995 statement: SELECT pg_catalog.version() May 31 09:22:28 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[834]: [22-1] 2024-05-31 09:22:28: pid 834: LOG: statement: DISCARD ALL May 31 09:22:28 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[834]: [23-1] 2024-05-31 09:22:28: pid 834: LOG: DB node id: 1 backend pid: 4046995 statement: DISCARD ALL May 31 09:22:28 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[834]: [24-1] 2024-05-31 09:22:28: pid 834: LOG: DB node id: 2 backend pid: 4015505 statement: DISCARD ALL May 31 09:23:50 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[745]: [18-1] 2024-05-31 09:23:50: pid 745: LOG: watchdog: lifecheck started May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [38-1] 2024-05-31 09:24:46: pid 691: WARNING: network IP is removed and system has no IP is assigned May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [38-2] 2024-05-31 09:24:46: pid 691: DETAIL: changing the state to in network trouble May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [39-1] 2024-05-31 09:24:46: pid 691: LOG: watchdog node state changed from [STANDBY] to [IN NETWORK TROUBLE] May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [40-1] 2024-05-31 09:24:46: pid 691: FATAL: system has lost the network May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[691]: [41-1] 2024-05-31 09:24:46: pid 691: LOG: Watchdog is shutting down May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [29-1] 2024-05-31 09:24:46: pid 628: LOG: watchdog child process with pid: 691 exits with status 768 May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [30-1] 2024-05-31 09:24:46: pid 628: FATAL: watchdog child process exit with fatal error. exiting pgpool-II May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [30-1] 2024-05-31 09:24:46: pid 1482: LOG: setting the local watchdog node name to "pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [31-1] 2024-05-31 09:24:46: pid 1482: LOG: watchdog cluster is configured with 2 remote nodes May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [32-1] 2024-05-31 09:24:46: pid 1482: LOG: watchdog remote node:0 on pgsql02:9000 May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [33-1] 2024-05-31 09:24:46: pid 1482: LOG: watchdog remote node:1 on pgsql03:9000 May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [34-1] 2024-05-31 09:24:46: pid 1482: LOG: interface monitoring is disabled in watchdog May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [35-1] 2024-05-31 09:24:46: pid 1482: INFO: IPC socket path: "/var/run/postgresql/.s.PGPOOLWD_CMD.9000" May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [36-1] 2024-05-31 09:24:46: pid 1482: LOG: connect on socket failed May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [36-2] 2024-05-31 09:24:46: pid 1482: DETAIL: connect failed with error: "Network is unreachable" May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [37-1] 2024-05-31 09:24:46: pid 1482: LOG: connect on socket failed May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [37-2] 2024-05-31 09:24:46: pid 1482: DETAIL: connect failed with error: "Network is unreachable" May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [38-1] 2024-05-31 09:24:46: pid 1482: LOG: watchdog node state changed from [DEAD] to [LOADING] May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[628]: [31-1] 2024-05-31 09:24:46: pid 628: LOG: shutting down May 31 09:24:46 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1482]: [39-1] 2024-05-31 09:24:46: pid 1482: LOG: Watchdog is shutting down May 31 09:24:47 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Main process exited, code=exited, status=1/FAILURE May 31 09:24:47 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Failed with result 'exit-code'. May 31 09:24:47 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Consumed 1.068s CPU time. May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: Started pgpool-II. May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [1-1] 2024-05-31 09:27:34: pid 1710: LOG: reading status file: 0 th backend is set to down status May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [2-1] 2024-05-31 09:27:34: pid 1710: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [3-1] 2024-05-31 09:27:34: pid 1710: LOG: memory cache initialized May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [3-2] 2024-05-31 09:27:34: pid 1710: DETAIL: memcache blocks :64 May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [4-1] 2024-05-31 09:27:34: pid 1710: LOG: allocating (138460264) bytes of shared memory segment May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [5-1] 2024-05-31 09:27:34: pid 1710: LOG: allocating shared memory segment of size: 138460264 May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [6-1] 2024-05-31 09:27:34: pid 1710: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [7-1] 2024-05-31 09:27:34: pid 1710: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [8-1] 2024-05-31 09:27:34: pid 1710: LOG: memory cache initialized May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [8-2] 2024-05-31 09:27:34: pid 1710: DETAIL: memcache blocks :64 May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [9-1] 2024-05-31 09:27:34: pid 1710: LOG: pool_discard_oid_maps: discarded memqcache oid maps May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [10-1] 2024-05-31 09:27:34: pid 1710: LOG: waiting for watchdog to initialize May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [10-1] 2024-05-31 09:27:34: pid 1713: LOG: setting the local watchdog node name to "pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [11-1] 2024-05-31 09:27:34: pid 1713: LOG: watchdog cluster is configured with 2 remote nodes May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [12-1] 2024-05-31 09:27:34: pid 1713: LOG: watchdog remote node:0 on pgsql02:9000 May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [13-1] 2024-05-31 09:27:34: pid 1713: LOG: watchdog remote node:1 on pgsql03:9000 May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [14-1] 2024-05-31 09:27:34: pid 1713: LOG: interface monitoring is disabled in watchdog May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [15-1] 2024-05-31 09:27:34: pid 1713: INFO: IPC socket path: "/var/run/postgresql/.s.PGPOOLWD_CMD.9000" May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [16-1] 2024-05-31 09:27:34: pid 1713: LOG: watchdog node state changed from [DEAD] to [LOADING] May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [17-1] 2024-05-31 09:27:34: pid 1713: LOG: new outbound connection to pgsql03:9000 May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [18-1] 2024-05-31 09:27:34: pid 1713: LOG: new outbound connection to pgsql02:9000 May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [19-1] 2024-05-31 09:27:34: pid 1713: LOG: watchdog node state changed from [LOADING] to [INITIALIZING] May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [20-1] 2024-05-31 09:27:34: pid 1713: LOG: setting the remote node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" as watchdog cluster leader May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [21-1] 2024-05-31 09:27:34: pid 1713: LOG: new watchdog node connection is received from "192.168.56.12:8887" May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [22-1] 2024-05-31 09:27:34: pid 1713: LOG: new node joined the cluster hostname:"pgsql03" port:9000 pgpool_port:9999 May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [22-2] 2024-05-31 09:27:34: pid 1713: DETAIL: Pgpool-II version:"4.3.7" watchdog messaging version: 1.2 May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [23-1] 2024-05-31 09:27:34: pid 1713: LOG: new watchdog node connection is received from "192.168.56.11:55944" May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [24-1] 2024-05-31 09:27:34: pid 1713: LOG: new node joined the cluster hostname:"pgsql02" port:9000 pgpool_port:9999 May 31 09:27:34 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [24-2] 2024-05-31 09:27:34: pid 1713: DETAIL: Pgpool-II version:"4.3.7" watchdog messaging version: 1.2 May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [25-1] 2024-05-31 09:27:35: pid 1713: LOG: watchdog node state changed from [INITIALIZING] to [STANDBY] May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [26-1] 2024-05-31 09:27:35: pid 1713: LOG: signal_user1_to_parent_with_reason(1) May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [27-1] 2024-05-31 09:27:35: pid 1713: LOG: successfully joined the watchdog cluster as standby node May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [27-2] 2024-05-31 09:27:35: pid 1713: DETAIL: our join coordinator request is accepted by cluster leader node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [11-1] 2024-05-31 09:27:35: pid 1710: LOG: watchdog process is initialized May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [11-2] 2024-05-31 09:27:35: pid 1710: DETAIL: watchdog messaging data version: 1.2 May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [12-1] 2024-05-31 09:27:35: pid 1710: LOG: Pgpool-II parent process received SIGUSR1 May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [13-1] 2024-05-31 09:27:35: pid 1710: LOG: Pgpool-II parent process received watchdog state change signal from watchdog May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [14-1] 2024-05-31 09:27:35: pid 1710: LOG: we have joined the watchdog cluster as STANDBY node May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [14-2] 2024-05-31 09:27:35: pid 1710: DETAIL: syncing the backend states from the LEADER watchdog node May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [28-1] 2024-05-31 09:27:35: pid 1713: LOG: received the get data request from local pgpool-II on IPC interface May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [29-1] 2024-05-31 09:27:35: pid 1713: LOG: get data request from local pgpool-II node received on IPC interface is forwarded to leader watchdog node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [29-2] 2024-05-31 09:27:35: pid 1713: DETAIL: waiting for the reply... May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [15-1] 2024-05-31 09:27:35: pid 1710: LOG: leader watchdog node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" returned status for 3 backend nodes May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [16-1] 2024-05-31 09:27:35: pid 1710: LOG: backend:1 is set to UP status May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [16-2] 2024-05-31 09:27:35: pid 1710: DETAIL: backend:1 is UP on cluster leader "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [17-1] 2024-05-31 09:27:35: pid 1710: LOG: backend:2 is set to UP status May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [17-2] 2024-05-31 09:27:35: pid 1710: DETAIL: backend:2 is UP on cluster leader "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [18-1] 2024-05-31 09:27:35: pid 1710: LOG: Setting up socket for 0.0.0.0:9999 May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [19-1] 2024-05-31 09:27:35: pid 1710: LOG: Setting up socket for :::9999 May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1714]: [12-1] 2024-05-31 09:27:35: pid 1714: LOG: 3 watchdog nodes are configured for lifecheck May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1714]: [13-1] 2024-05-31 09:27:35: pid 1714: LOG: watchdog nodes ID:0 Name:"pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1714]: [13-2] 2024-05-31 09:27:35: pid 1714: DETAIL: Host:"pgsql01" WD Port:9000 pgpool-II port:9999 May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1714]: [14-1] 2024-05-31 09:27:35: pid 1714: LOG: watchdog nodes ID:1 Name:"pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1714]: [14-2] 2024-05-31 09:27:35: pid 1714: DETAIL: Host:"pgsql02" WD Port:9000 pgpool-II port:9999 May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1714]: [15-1] 2024-05-31 09:27:35: pid 1714: LOG: watchdog nodes ID:2 Name:"pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1714]: [15-2] 2024-05-31 09:27:35: pid 1714: DETAIL: Host:"pgsql03" WD Port:9000 pgpool-II port:9999 May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1714]: [16-1] 2024-05-31 09:27:35: pid 1714: LOG: watchdog lifecheck trusted server "pgsql01" added for the availability check May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1714]: [17-1] 2024-05-31 09:27:35: pid 1714: LOG: watchdog lifecheck trusted server "pgsql02" added for the availability check May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1714]: [18-1] 2024-05-31 09:27:35: pid 1714: LOG: watchdog lifecheck trusted server "pgsql03" added for the availability check May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1820]: [20-1] 2024-05-31 09:27:35: pid 1820: LOG: process started May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1819]: [20-1] 2024-05-31 09:27:35: pid 1819: LOG: PCP process: 1819 started May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1822]: [20-1] 2024-05-31 09:27:35: pid 1822: LOG: process started May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1821]: [20-1] 2024-05-31 09:27:35: pid 1821: LOG: process started May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [20-1] 2024-05-31 09:27:35: pid 1710: LOG: pgpool-II successfully started. version 4.3.7 (tamahomeboshi) May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [21-1] 2024-05-31 09:27:35: pid 1710: LOG: node status[0]: 0 May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [22-1] 2024-05-31 09:27:35: pid 1710: LOG: node status[1]: 0 May 31 09:27:35 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [23-1] 2024-05-31 09:27:35: pid 1710: LOG: node status[2]: 0 May 31 09:27:36 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1725]: [16-1] 2024-05-31 09:27:36: pid 1725: LOG: set SO_REUSEPORT option to the socket May 31 09:27:36 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1725]: [17-1] 2024-05-31 09:27:36: pid 1725: LOG: creating socket for sending heartbeat May 31 09:27:36 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1725]: [17-2] 2024-05-31 09:27:36: pid 1725: DETAIL: set SO_REUSEPORT May 31 09:27:36 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1721]: [16-1] 2024-05-31 09:27:36: pid 1721: LOG: set SO_REUSEPORT option to the socket May 31 09:27:36 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1719]: [16-1] 2024-05-31 09:27:36: pid 1719: LOG: set SO_REUSEPORT option to the socket May 31 09:27:36 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1719]: [17-1] 2024-05-31 09:27:36: pid 1719: LOG: creating watchdog heartbeat receive socket. May 31 09:27:36 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1719]: [17-2] 2024-05-31 09:27:36: pid 1719: DETAIL: set SO_REUSEPORT May 31 09:27:36 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1723]: [16-1] 2024-05-31 09:27:36: pid 1723: LOG: set SO_REUSEPORT option to the socket May 31 09:27:36 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1723]: [17-1] 2024-05-31 09:27:36: pid 1723: LOG: creating watchdog heartbeat receive socket. May 31 09:27:36 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1723]: [17-2] 2024-05-31 09:27:36: pid 1723: DETAIL: set SO_REUSEPORT May 31 09:27:36 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1721]: [17-1] 2024-05-31 09:27:36: pid 1721: LOG: creating socket for sending heartbeat May 31 09:27:36 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1721]: [17-2] 2024-05-31 09:27:36: pid 1721: DETAIL: set SO_REUSEPORT May 31 09:27:37 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [30-1] 2024-05-31 09:27:37: pid 1713: LOG: remote node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" is reporting that it has found us again May 31 09:27:38 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [31-1] 2024-05-31 09:27:38: pid 1713: LOG: remote node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" is reporting that it has found us again May 31 09:27:38 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [32-1] 2024-05-31 09:27:38: pid 1713: LOG: re-sending join coordinator message to leader node: "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:27:38 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [33-1] 2024-05-31 09:27:38: pid 1713: LOG: signal_user1_to_parent_with_reason(1) May 31 09:27:38 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [34-1] 2024-05-31 09:27:38: pid 1713: LOG: successfully joined the watchdog cluster as standby node May 31 09:27:38 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [34-2] 2024-05-31 09:27:38: pid 1713: DETAIL: our join coordinator request is accepted by cluster leader node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:27:38 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [24-1] 2024-05-31 09:27:38: pid 1710: LOG: Pgpool-II parent process received SIGUSR1 May 31 09:27:38 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [25-1] 2024-05-31 09:27:38: pid 1710: LOG: Pgpool-II parent process received watchdog state change signal from watchdog May 31 09:27:38 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [26-1] 2024-05-31 09:27:38: pid 1710: LOG: we have joined the watchdog cluster as STANDBY node May 31 09:27:38 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [26-2] 2024-05-31 09:27:38: pid 1710: DETAIL: syncing the backend states from the LEADER watchdog node May 31 09:27:38 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [35-1] 2024-05-31 09:27:38: pid 1713: LOG: received the get data request from local pgpool-II on IPC interface May 31 09:27:38 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [36-1] 2024-05-31 09:27:38: pid 1713: LOG: get data request from local pgpool-II node received on IPC interface is forwarded to leader watchdog node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:27:38 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [36-2] 2024-05-31 09:27:38: pid 1713: DETAIL: waiting for the reply... May 31 09:27:38 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [27-1] 2024-05-31 09:27:38: pid 1710: LOG: leader watchdog node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" returned status for 3 backend nodes May 31 09:27:38 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [28-1] 2024-05-31 09:27:38: pid 1710: LOG: backend nodes status remains same after the sync from "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:29:15 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1714]: [19-1] 2024-05-31 09:29:15: pid 1714: LOG: watchdog: lifecheck started May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [37-1] 2024-05-31 09:30:09: pid 1713: WARNING: network IP is removed and system has no IP is assigned May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [37-2] 2024-05-31 09:30:09: pid 1713: DETAIL: changing the state to in network trouble May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [38-1] 2024-05-31 09:30:09: pid 1713: LOG: watchdog node state changed from [STANDBY] to [IN NETWORK TROUBLE] May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [39-1] 2024-05-31 09:30:09: pid 1713: FATAL: system has lost the network May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1713]: [40-1] 2024-05-31 09:30:09: pid 1713: LOG: Watchdog is shutting down May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [29-1] 2024-05-31 09:30:09: pid 1710: LOG: watchdog child process with pid: 1713 exits with status 768 May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [30-1] 2024-05-31 09:30:09: pid 1710: FATAL: watchdog child process exit with fatal error. exiting pgpool-II May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2066]: [30-1] 2024-05-31 09:30:09: pid 2066: LOG: setting the local watchdog node name to "pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2066]: [31-1] 2024-05-31 09:30:09: pid 2066: LOG: watchdog cluster is configured with 2 remote nodes May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2066]: [32-1] 2024-05-31 09:30:09: pid 2066: LOG: watchdog remote node:0 on pgsql02:9000 May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2066]: [33-1] 2024-05-31 09:30:09: pid 2066: LOG: watchdog remote node:1 on pgsql03:9000 May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2066]: [34-1] 2024-05-31 09:30:09: pid 2066: LOG: interface monitoring is disabled in watchdog May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2066]: [35-1] 2024-05-31 09:30:09: pid 2066: INFO: IPC socket path: "/var/run/postgresql/.s.PGPOOLWD_CMD.9000" May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2066]: [36-1] 2024-05-31 09:30:09: pid 2066: LOG: connect on socket failed May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2066]: [36-2] 2024-05-31 09:30:09: pid 2066: DETAIL: connect failed with error: "Network is unreachable" May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2066]: [37-1] 2024-05-31 09:30:09: pid 2066: LOG: connect on socket failed May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2066]: [37-2] 2024-05-31 09:30:09: pid 2066: DETAIL: connect failed with error: "Network is unreachable" May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2066]: [38-1] 2024-05-31 09:30:09: pid 2066: LOG: watchdog node state changed from [DEAD] to [LOADING] May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[1710]: [31-1] 2024-05-31 09:30:09: pid 1710: LOG: shutting down May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2066]: [39-1] 2024-05-31 09:30:09: pid 2066: LOG: Watchdog is shutting down May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Main process exited, code=exited, status=1/FAILURE May 31 09:30:09 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Failed with result 'exit-code'. May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: Started pgpool-II. May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [1-1] 2024-05-31 09:33:48: pid 2470: LOG: reading status file: 0 th backend is set to down status May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [2-1] 2024-05-31 09:33:48: pid 2470: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [3-1] 2024-05-31 09:33:48: pid 2470: LOG: memory cache initialized May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [3-2] 2024-05-31 09:33:48: pid 2470: DETAIL: memcache blocks :64 May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [4-1] 2024-05-31 09:33:48: pid 2470: LOG: allocating (138460264) bytes of shared memory segment May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [5-1] 2024-05-31 09:33:48: pid 2470: LOG: allocating shared memory segment of size: 138460264 May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [6-1] 2024-05-31 09:33:48: pid 2470: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [7-1] 2024-05-31 09:33:48: pid 2470: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [8-1] 2024-05-31 09:33:48: pid 2470: LOG: memory cache initialized May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [8-2] 2024-05-31 09:33:48: pid 2470: DETAIL: memcache blocks :64 May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [9-1] 2024-05-31 09:33:48: pid 2470: LOG: pool_discard_oid_maps: discarded memqcache oid maps May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [10-1] 2024-05-31 09:33:48: pid 2470: LOG: waiting for watchdog to initialize May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [10-1] 2024-05-31 09:33:48: pid 2473: LOG: setting the local watchdog node name to "pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [11-1] 2024-05-31 09:33:48: pid 2473: LOG: watchdog cluster is configured with 2 remote nodes May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [12-1] 2024-05-31 09:33:48: pid 2473: LOG: watchdog remote node:0 on pgsql02:9000 May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [13-1] 2024-05-31 09:33:48: pid 2473: LOG: watchdog remote node:1 on pgsql03:9000 May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [14-1] 2024-05-31 09:33:48: pid 2473: LOG: interface monitoring is disabled in watchdog May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [15-1] 2024-05-31 09:33:48: pid 2473: INFO: IPC socket path: "/var/run/postgresql/.s.PGPOOLWD_CMD.9000" May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [16-1] 2024-05-31 09:33:48: pid 2473: LOG: watchdog node state changed from [DEAD] to [LOADING] May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [17-1] 2024-05-31 09:33:48: pid 2473: LOG: new outbound connection to pgsql02:9000 May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [18-1] 2024-05-31 09:33:48: pid 2473: LOG: new outbound connection to pgsql03:9000 May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [19-1] 2024-05-31 09:33:48: pid 2473: LOG: new watchdog node connection is received from "192.168.56.12:10423" May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [20-1] 2024-05-31 09:33:48: pid 2473: LOG: new watchdog node connection is received from "192.168.56.11:57992" May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [21-1] 2024-05-31 09:33:48: pid 2473: LOG: new node joined the cluster hostname:"pgsql03" port:9000 pgpool_port:9999 May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [21-2] 2024-05-31 09:33:48: pid 2473: DETAIL: Pgpool-II version:"4.3.7" watchdog messaging version: 1.2 May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [22-1] 2024-05-31 09:33:48: pid 2473: LOG: new node joined the cluster hostname:"pgsql02" port:9000 pgpool_port:9999 May 31 09:33:48 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [22-2] 2024-05-31 09:33:48: pid 2473: DETAIL: Pgpool-II version:"4.3.7" watchdog messaging version: 1.2 May 31 09:33:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [23-1] 2024-05-31 09:33:53: pid 2473: LOG: watchdog node state changed from [LOADING] to [JOINING] May 31 09:33:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [24-1] 2024-05-31 09:33:53: pid 2473: LOG: setting the remote node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" as watchdog cluster leader May 31 09:33:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [25-1] 2024-05-31 09:33:53: pid 2473: LOG: watchdog node state changed from [JOINING] to [INITIALIZING] May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [26-1] 2024-05-31 09:33:54: pid 2473: LOG: watchdog node state changed from [INITIALIZING] to [STANDBY] May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [27-1] 2024-05-31 09:33:54: pid 2473: LOG: signal_user1_to_parent_with_reason(1) May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [28-1] 2024-05-31 09:33:54: pid 2473: LOG: successfully joined the watchdog cluster as standby node May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [28-2] 2024-05-31 09:33:54: pid 2473: DETAIL: our join coordinator request is accepted by cluster leader node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [11-1] 2024-05-31 09:33:54: pid 2470: LOG: watchdog process is initialized May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [11-2] 2024-05-31 09:33:54: pid 2470: DETAIL: watchdog messaging data version: 1.2 May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [12-1] 2024-05-31 09:33:54: pid 2470: LOG: Pgpool-II parent process received SIGUSR1 May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [13-1] 2024-05-31 09:33:54: pid 2470: LOG: Pgpool-II parent process received watchdog state change signal from watchdog May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [14-1] 2024-05-31 09:33:54: pid 2470: LOG: we have joined the watchdog cluster as STANDBY node May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [14-2] 2024-05-31 09:33:54: pid 2470: DETAIL: syncing the backend states from the LEADER watchdog node May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [29-1] 2024-05-31 09:33:54: pid 2473: LOG: received the get data request from local pgpool-II on IPC interface May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [30-1] 2024-05-31 09:33:54: pid 2473: LOG: get data request from local pgpool-II node received on IPC interface is forwarded to leader watchdog node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [30-2] 2024-05-31 09:33:54: pid 2473: DETAIL: waiting for the reply... May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [15-1] 2024-05-31 09:33:54: pid 2470: LOG: leader watchdog node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" returned status for 3 backend nodes May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [16-1] 2024-05-31 09:33:54: pid 2470: LOG: backend:1 is set to UP status May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [16-2] 2024-05-31 09:33:54: pid 2470: DETAIL: backend:1 is UP on cluster leader "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [17-1] 2024-05-31 09:33:54: pid 2470: LOG: backend:2 is set to UP status May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [17-2] 2024-05-31 09:33:54: pid 2470: DETAIL: backend:2 is UP on cluster leader "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [18-1] 2024-05-31 09:33:54: pid 2470: LOG: Setting up socket for 0.0.0.0:9999 May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [19-1] 2024-05-31 09:33:54: pid 2470: LOG: Setting up socket for :::9999 May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2480]: [12-1] 2024-05-31 09:33:54: pid 2480: LOG: 3 watchdog nodes are configured for lifecheck May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2480]: [13-1] 2024-05-31 09:33:54: pid 2480: LOG: watchdog nodes ID:0 Name:"pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2480]: [13-2] 2024-05-31 09:33:54: pid 2480: DETAIL: Host:"pgsql01" WD Port:9000 pgpool-II port:9999 May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2480]: [14-1] 2024-05-31 09:33:54: pid 2480: LOG: watchdog nodes ID:1 Name:"pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2480]: [14-2] 2024-05-31 09:33:54: pid 2480: DETAIL: Host:"pgsql02" WD Port:9000 pgpool-II port:9999 May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2480]: [15-1] 2024-05-31 09:33:54: pid 2480: LOG: watchdog nodes ID:2 Name:"pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2480]: [15-2] 2024-05-31 09:33:54: pid 2480: DETAIL: Host:"pgsql03" WD Port:9000 pgpool-II port:9999 May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2480]: [16-1] 2024-05-31 09:33:54: pid 2480: LOG: watchdog lifecheck trusted server "pgsql01" added for the availability check May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2480]: [17-1] 2024-05-31 09:33:54: pid 2480: LOG: watchdog lifecheck trusted server "pgsql02" added for the availability check May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2480]: [18-1] 2024-05-31 09:33:54: pid 2480: LOG: watchdog lifecheck trusted server "pgsql03" added for the availability check May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2586]: [20-1] 2024-05-31 09:33:54: pid 2586: LOG: process started May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2585]: [20-1] 2024-05-31 09:33:54: pid 2585: LOG: PCP process: 2585 started May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2587]: [20-1] 2024-05-31 09:33:54: pid 2587: LOG: process started May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2588]: [20-1] 2024-05-31 09:33:54: pid 2588: LOG: process started May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [20-1] 2024-05-31 09:33:54: pid 2470: LOG: pgpool-II successfully started. version 4.3.7 (tamahomeboshi) May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [21-1] 2024-05-31 09:33:54: pid 2470: LOG: node status[0]: 0 May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [22-1] 2024-05-31 09:33:54: pid 2470: LOG: node status[1]: 0 May 31 09:33:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [23-1] 2024-05-31 09:33:54: pid 2470: LOG: node status[2]: 0 May 31 09:33:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2488]: [16-1] 2024-05-31 09:33:55: pid 2488: LOG: set SO_REUSEPORT option to the socket May 31 09:33:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2492]: [16-1] 2024-05-31 09:33:55: pid 2492: LOG: set SO_REUSEPORT option to the socket May 31 09:33:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2492]: [17-1] 2024-05-31 09:33:55: pid 2492: LOG: creating watchdog heartbeat receive socket. May 31 09:33:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2492]: [17-2] 2024-05-31 09:33:55: pid 2492: DETAIL: set SO_REUSEPORT May 31 09:33:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2490]: [16-1] 2024-05-31 09:33:55: pid 2490: LOG: set SO_REUSEPORT option to the socket May 31 09:33:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2490]: [17-1] 2024-05-31 09:33:55: pid 2490: LOG: creating socket for sending heartbeat May 31 09:33:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2490]: [17-2] 2024-05-31 09:33:55: pid 2490: DETAIL: set SO_REUSEPORT May 31 09:33:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2494]: [16-1] 2024-05-31 09:33:55: pid 2494: LOG: set SO_REUSEPORT option to the socket May 31 09:33:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2488]: [17-1] 2024-05-31 09:33:55: pid 2488: LOG: creating watchdog heartbeat receive socket. May 31 09:33:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2488]: [17-2] 2024-05-31 09:33:55: pid 2488: DETAIL: set SO_REUSEPORT May 31 09:33:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2494]: [17-1] 2024-05-31 09:33:55: pid 2494: LOG: creating socket for sending heartbeat May 31 09:33:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2494]: [17-2] 2024-05-31 09:33:55: pid 2494: DETAIL: set SO_REUSEPORT May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [31-1] 2024-05-31 09:34:07: pid 2473: WARNING: network IP is removed and system has no IP is assigned May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [31-2] 2024-05-31 09:34:07: pid 2473: DETAIL: changing the state to in network trouble May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [32-1] 2024-05-31 09:34:07: pid 2473: LOG: watchdog node state changed from [STANDBY] to [IN NETWORK TROUBLE] May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [33-1] 2024-05-31 09:34:07: pid 2473: FATAL: system has lost the network May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2473]: [34-1] 2024-05-31 09:34:07: pid 2473: LOG: Watchdog is shutting down May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [24-1] 2024-05-31 09:34:07: pid 2470: LOG: watchdog child process with pid: 2473 exits with status 768 May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [25-1] 2024-05-31 09:34:07: pid 2470: FATAL: watchdog child process exit with fatal error. exiting pgpool-II May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2613]: [25-1] 2024-05-31 09:34:07: pid 2613: LOG: setting the local watchdog node name to "pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2613]: [26-1] 2024-05-31 09:34:07: pid 2613: LOG: watchdog cluster is configured with 2 remote nodes May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2613]: [27-1] 2024-05-31 09:34:07: pid 2613: LOG: watchdog remote node:0 on pgsql02:9000 May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2613]: [28-1] 2024-05-31 09:34:07: pid 2613: LOG: watchdog remote node:1 on pgsql03:9000 May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2613]: [29-1] 2024-05-31 09:34:07: pid 2613: LOG: interface monitoring is disabled in watchdog May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2613]: [30-1] 2024-05-31 09:34:07: pid 2613: INFO: IPC socket path: "/var/run/postgresql/.s.PGPOOLWD_CMD.9000" May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2613]: [31-1] 2024-05-31 09:34:07: pid 2613: LOG: connect on socket failed May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2613]: [31-2] 2024-05-31 09:34:07: pid 2613: DETAIL: connect failed with error: "Network is unreachable" May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2613]: [32-1] 2024-05-31 09:34:07: pid 2613: LOG: connect on socket failed May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2613]: [32-2] 2024-05-31 09:34:07: pid 2613: DETAIL: connect failed with error: "Network is unreachable" May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2613]: [33-1] 2024-05-31 09:34:07: pid 2613: LOG: watchdog node state changed from [DEAD] to [LOADING] May 31 09:34:07 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2470]: [26-1] 2024-05-31 09:34:07: pid 2470: LOG: shutting down May 31 09:34:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2613]: [34-1] 2024-05-31 09:34:08: pid 2613: LOG: Watchdog is shutting down May 31 09:34:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Main process exited, code=exited, status=1/FAILURE May 31 09:34:08 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Failed with result 'exit-code'. May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: Started pgpool-II. May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [1-1] 2024-05-31 09:34:53: pid 2675: LOG: reading status file: 0 th backend is set to down status May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [2-1] 2024-05-31 09:34:53: pid 2675: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [3-1] 2024-05-31 09:34:53: pid 2675: LOG: memory cache initialized May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [3-2] 2024-05-31 09:34:53: pid 2675: DETAIL: memcache blocks :64 May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [4-1] 2024-05-31 09:34:53: pid 2675: LOG: allocating (138460264) bytes of shared memory segment May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [5-1] 2024-05-31 09:34:53: pid 2675: LOG: allocating shared memory segment of size: 138460264 May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [6-1] 2024-05-31 09:34:53: pid 2675: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [7-1] 2024-05-31 09:34:53: pid 2675: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [8-1] 2024-05-31 09:34:53: pid 2675: LOG: memory cache initialized May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [8-2] 2024-05-31 09:34:53: pid 2675: DETAIL: memcache blocks :64 May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [9-1] 2024-05-31 09:34:53: pid 2675: LOG: pool_discard_oid_maps: discarded memqcache oid maps May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [10-1] 2024-05-31 09:34:53: pid 2675: LOG: waiting for watchdog to initialize May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [10-1] 2024-05-31 09:34:53: pid 2678: LOG: setting the local watchdog node name to "pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [11-1] 2024-05-31 09:34:53: pid 2678: LOG: watchdog cluster is configured with 2 remote nodes May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [12-1] 2024-05-31 09:34:53: pid 2678: LOG: watchdog remote node:0 on pgsql02:9000 May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [13-1] 2024-05-31 09:34:53: pid 2678: LOG: watchdog remote node:1 on pgsql03:9000 May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [14-1] 2024-05-31 09:34:53: pid 2678: LOG: interface monitoring is disabled in watchdog May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [15-1] 2024-05-31 09:34:53: pid 2678: INFO: IPC socket path: "/var/run/postgresql/.s.PGPOOLWD_CMD.9000" May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [16-1] 2024-05-31 09:34:53: pid 2678: LOG: watchdog node state changed from [DEAD] to [LOADING] May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [17-1] 2024-05-31 09:34:53: pid 2678: LOG: new outbound connection to pgsql03:9000 May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [18-1] 2024-05-31 09:34:53: pid 2678: LOG: new outbound connection to pgsql02:9000 May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [19-1] 2024-05-31 09:34:53: pid 2678: LOG: setting the remote node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" as watchdog cluster leader May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [20-1] 2024-05-31 09:34:53: pid 2678: LOG: watchdog node state changed from [LOADING] to [INITIALIZING] May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [21-1] 2024-05-31 09:34:53: pid 2678: LOG: new watchdog node connection is received from "192.168.56.12:11447" May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [22-1] 2024-05-31 09:34:53: pid 2678: LOG: new watchdog node connection is received from "192.168.56.11:59016" May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [23-1] 2024-05-31 09:34:53: pid 2678: LOG: new node joined the cluster hostname:"pgsql03" port:9000 pgpool_port:9999 May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [23-2] 2024-05-31 09:34:53: pid 2678: DETAIL: Pgpool-II version:"4.3.7" watchdog messaging version: 1.2 May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [24-1] 2024-05-31 09:34:53: pid 2678: LOG: new node joined the cluster hostname:"pgsql02" port:9000 pgpool_port:9999 May 31 09:34:53 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [24-2] 2024-05-31 09:34:53: pid 2678: DETAIL: Pgpool-II version:"4.3.7" watchdog messaging version: 1.2 May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [25-1] 2024-05-31 09:34:54: pid 2678: LOG: watchdog node state changed from [INITIALIZING] to [STANDBY] May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [26-1] 2024-05-31 09:34:54: pid 2678: LOG: signal_user1_to_parent_with_reason(1) May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [27-1] 2024-05-31 09:34:54: pid 2678: LOG: successfully joined the watchdog cluster as standby node May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [27-2] 2024-05-31 09:34:54: pid 2678: DETAIL: our join coordinator request is accepted by cluster leader node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [11-1] 2024-05-31 09:34:54: pid 2675: LOG: watchdog process is initialized May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [11-2] 2024-05-31 09:34:54: pid 2675: DETAIL: watchdog messaging data version: 1.2 May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [12-1] 2024-05-31 09:34:54: pid 2675: LOG: Pgpool-II parent process received SIGUSR1 May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [13-1] 2024-05-31 09:34:54: pid 2675: LOG: Pgpool-II parent process received watchdog state change signal from watchdog May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [14-1] 2024-05-31 09:34:54: pid 2675: LOG: we have joined the watchdog cluster as STANDBY node May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [14-2] 2024-05-31 09:34:54: pid 2675: DETAIL: syncing the backend states from the LEADER watchdog node May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [28-1] 2024-05-31 09:34:54: pid 2678: LOG: received the get data request from local pgpool-II on IPC interface May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [29-1] 2024-05-31 09:34:54: pid 2678: LOG: get data request from local pgpool-II node received on IPC interface is forwarded to leader watchdog node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [29-2] 2024-05-31 09:34:54: pid 2678: DETAIL: waiting for the reply... May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [15-1] 2024-05-31 09:34:54: pid 2675: LOG: leader watchdog node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" returned status for 3 backend nodes May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [16-1] 2024-05-31 09:34:54: pid 2675: LOG: backend:1 is set to UP status May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [16-2] 2024-05-31 09:34:54: pid 2675: DETAIL: backend:1 is UP on cluster leader "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [17-1] 2024-05-31 09:34:54: pid 2675: LOG: backend:2 is set to UP status May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [17-2] 2024-05-31 09:34:54: pid 2675: DETAIL: backend:2 is UP on cluster leader "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [18-1] 2024-05-31 09:34:54: pid 2675: LOG: Setting up socket for 0.0.0.0:9999 May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [19-1] 2024-05-31 09:34:54: pid 2675: LOG: Setting up socket for :::9999 May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2681]: [12-1] 2024-05-31 09:34:54: pid 2681: LOG: 3 watchdog nodes are configured for lifecheck May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2681]: [13-1] 2024-05-31 09:34:54: pid 2681: LOG: watchdog nodes ID:0 Name:"pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2681]: [13-2] 2024-05-31 09:34:54: pid 2681: DETAIL: Host:"pgsql01" WD Port:9000 pgpool-II port:9999 May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2681]: [14-1] 2024-05-31 09:34:54: pid 2681: LOG: watchdog nodes ID:1 Name:"pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2681]: [14-2] 2024-05-31 09:34:54: pid 2681: DETAIL: Host:"pgsql02" WD Port:9000 pgpool-II port:9999 May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2681]: [15-1] 2024-05-31 09:34:54: pid 2681: LOG: watchdog nodes ID:2 Name:"pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2681]: [15-2] 2024-05-31 09:34:54: pid 2681: DETAIL: Host:"pgsql03" WD Port:9000 pgpool-II port:9999 May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2681]: [16-1] 2024-05-31 09:34:54: pid 2681: LOG: watchdog lifecheck trusted server "pgsql01" added for the availability check May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2681]: [17-1] 2024-05-31 09:34:54: pid 2681: LOG: watchdog lifecheck trusted server "pgsql02" added for the availability check May 31 09:34:54 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2681]: [18-1] 2024-05-31 09:34:54: pid 2681: LOG: watchdog lifecheck trusted server "pgsql03" added for the availability check May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2786]: [20-1] 2024-05-31 09:34:55: pid 2786: LOG: PCP process: 2786 started May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2787]: [20-1] 2024-05-31 09:34:55: pid 2787: LOG: process started May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2788]: [20-1] 2024-05-31 09:34:55: pid 2788: LOG: process started May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2789]: [20-1] 2024-05-31 09:34:55: pid 2789: LOG: process started May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [20-1] 2024-05-31 09:34:55: pid 2675: LOG: pgpool-II successfully started. version 4.3.7 (tamahomeboshi) May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [21-1] 2024-05-31 09:34:55: pid 2675: LOG: node status[0]: 0 May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [22-1] 2024-05-31 09:34:55: pid 2675: LOG: node status[1]: 0 May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [23-1] 2024-05-31 09:34:55: pid 2675: LOG: node status[2]: 0 May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2691]: [16-1] 2024-05-31 09:34:55: pid 2691: LOG: set SO_REUSEPORT option to the socket May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2689]: [16-1] 2024-05-31 09:34:55: pid 2689: LOG: set SO_REUSEPORT option to the socket May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2693]: [16-1] 2024-05-31 09:34:55: pid 2693: LOG: set SO_REUSEPORT option to the socket May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2693]: [17-1] 2024-05-31 09:34:55: pid 2693: LOG: creating socket for sending heartbeat May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2693]: [17-2] 2024-05-31 09:34:55: pid 2693: DETAIL: set SO_REUSEPORT May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2687]: [16-1] 2024-05-31 09:34:55: pid 2687: LOG: set SO_REUSEPORT option to the socket May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2687]: [17-1] 2024-05-31 09:34:55: pid 2687: LOG: creating watchdog heartbeat receive socket. May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2687]: [17-2] 2024-05-31 09:34:55: pid 2687: DETAIL: set SO_REUSEPORT May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2691]: [17-1] 2024-05-31 09:34:55: pid 2691: LOG: creating watchdog heartbeat receive socket. May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2691]: [17-2] 2024-05-31 09:34:55: pid 2691: DETAIL: set SO_REUSEPORT May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2689]: [17-1] 2024-05-31 09:34:55: pid 2689: LOG: creating socket for sending heartbeat May 31 09:34:55 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2689]: [17-2] 2024-05-31 09:34:55: pid 2689: DETAIL: set SO_REUSEPORT May 31 09:34:56 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [30-1] 2024-05-31 09:34:56: pid 2678: LOG: remote node "pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" is reporting that it has found us again May 31 09:34:57 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [31-1] 2024-05-31 09:34:57: pid 2678: LOG: remote node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" is reporting that it has found us again May 31 09:34:57 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [32-1] 2024-05-31 09:34:57: pid 2678: LOG: re-sending join coordinator message to leader node: "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:34:57 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [33-1] 2024-05-31 09:34:57: pid 2678: LOG: signal_user1_to_parent_with_reason(1) May 31 09:34:57 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [34-1] 2024-05-31 09:34:57: pid 2678: LOG: successfully joined the watchdog cluster as standby node May 31 09:34:57 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [34-2] 2024-05-31 09:34:57: pid 2678: DETAIL: our join coordinator request is accepted by cluster leader node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:34:57 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [24-1] 2024-05-31 09:34:57: pid 2675: LOG: Pgpool-II parent process received SIGUSR1 May 31 09:34:57 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [25-1] 2024-05-31 09:34:57: pid 2675: LOG: Pgpool-II parent process received watchdog state change signal from watchdog May 31 09:34:57 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [26-1] 2024-05-31 09:34:57: pid 2675: LOG: we have joined the watchdog cluster as STANDBY node May 31 09:34:57 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [26-2] 2024-05-31 09:34:57: pid 2675: DETAIL: syncing the backend states from the LEADER watchdog node May 31 09:34:57 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [35-1] 2024-05-31 09:34:57: pid 2678: LOG: received the get data request from local pgpool-II on IPC interface May 31 09:34:57 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [36-1] 2024-05-31 09:34:57: pid 2678: LOG: get data request from local pgpool-II node received on IPC interface is forwarded to leader watchdog node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:34:57 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [36-2] 2024-05-31 09:34:57: pid 2678: DETAIL: waiting for the reply... May 31 09:34:57 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [27-1] 2024-05-31 09:34:57: pid 2675: LOG: leader watchdog node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" returned status for 3 backend nodes May 31 09:34:57 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [28-1] 2024-05-31 09:34:57: pid 2675: LOG: backend nodes status remains same after the sync from "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [37-1] 2024-05-31 09:35:01: pid 2678: WARNING: network IP is removed and system has no IP is assigned May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [37-2] 2024-05-31 09:35:01: pid 2678: DETAIL: changing the state to in network trouble May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [38-1] 2024-05-31 09:35:01: pid 2678: LOG: watchdog node state changed from [STANDBY] to [IN NETWORK TROUBLE] May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [39-1] 2024-05-31 09:35:01: pid 2678: FATAL: system has lost the network May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2678]: [40-1] 2024-05-31 09:35:01: pid 2678: LOG: Watchdog is shutting down May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [29-1] 2024-05-31 09:35:01: pid 2675: LOG: watchdog child process with pid: 2678 exits with status 768 May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [30-1] 2024-05-31 09:35:01: pid 2675: FATAL: watchdog child process exit with fatal error. exiting pgpool-II May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2806]: [30-1] 2024-05-31 09:35:01: pid 2806: LOG: setting the local watchdog node name to "pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2806]: [31-1] 2024-05-31 09:35:01: pid 2806: LOG: watchdog cluster is configured with 2 remote nodes May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2806]: [32-1] 2024-05-31 09:35:01: pid 2806: LOG: watchdog remote node:0 on pgsql02:9000 May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2806]: [33-1] 2024-05-31 09:35:01: pid 2806: LOG: watchdog remote node:1 on pgsql03:9000 May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2806]: [34-1] 2024-05-31 09:35:01: pid 2806: LOG: interface monitoring is disabled in watchdog May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2806]: [35-1] 2024-05-31 09:35:01: pid 2806: INFO: IPC socket path: "/var/run/postgresql/.s.PGPOOLWD_CMD.9000" May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2806]: [36-1] 2024-05-31 09:35:01: pid 2806: LOG: connect on socket failed May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2806]: [36-2] 2024-05-31 09:35:01: pid 2806: DETAIL: connect failed with error: "Network is unreachable" May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2806]: [37-1] 2024-05-31 09:35:01: pid 2806: LOG: connect on socket failed May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2806]: [37-2] 2024-05-31 09:35:01: pid 2806: DETAIL: connect failed with error: "Network is unreachable" May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2806]: [38-1] 2024-05-31 09:35:01: pid 2806: LOG: watchdog node state changed from [DEAD] to [LOADING] May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2675]: [31-1] 2024-05-31 09:35:01: pid 2675: LOG: shutting down May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2806]: [39-1] 2024-05-31 09:35:01: pid 2806: LOG: Watchdog is shutting down May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Main process exited, code=exited, status=1/FAILURE May 31 09:35:01 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: pgpool2.service: Failed with result 'exit-code'. May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 systemd[1]: Started pgpool-II. May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [1-1] 2024-05-31 09:35:11: pid 2815: LOG: reading status file: 0 th backend is set to down status May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [2-1] 2024-05-31 09:35:11: pid 2815: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [3-1] 2024-05-31 09:35:11: pid 2815: LOG: memory cache initialized May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [3-2] 2024-05-31 09:35:11: pid 2815: DETAIL: memcache blocks :64 May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [4-1] 2024-05-31 09:35:11: pid 2815: LOG: allocating (138460264) bytes of shared memory segment May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [5-1] 2024-05-31 09:35:11: pid 2815: LOG: allocating shared memory segment of size: 138460264 May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [6-1] 2024-05-31 09:35:11: pid 2815: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [7-1] 2024-05-31 09:35:11: pid 2815: LOG: health_check_stats_shared_memory_size: requested size: 12288 May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [8-1] 2024-05-31 09:35:11: pid 2815: LOG: memory cache initialized May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [8-2] 2024-05-31 09:35:11: pid 2815: DETAIL: memcache blocks :64 May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [9-1] 2024-05-31 09:35:11: pid 2815: LOG: pool_discard_oid_maps: discarded memqcache oid maps May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [10-1] 2024-05-31 09:35:11: pid 2815: LOG: waiting for watchdog to initialize May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [10-1] 2024-05-31 09:35:11: pid 2818: LOG: setting the local watchdog node name to "pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [11-1] 2024-05-31 09:35:11: pid 2818: LOG: watchdog cluster is configured with 2 remote nodes May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [12-1] 2024-05-31 09:35:11: pid 2818: LOG: watchdog remote node:0 on pgsql02:9000 May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [13-1] 2024-05-31 09:35:11: pid 2818: LOG: watchdog remote node:1 on pgsql03:9000 May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [14-1] 2024-05-31 09:35:11: pid 2818: LOG: interface monitoring is disabled in watchdog May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [15-1] 2024-05-31 09:35:11: pid 2818: INFO: IPC socket path: "/var/run/postgresql/.s.PGPOOLWD_CMD.9000" May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [16-1] 2024-05-31 09:35:11: pid 2818: LOG: watchdog node state changed from [DEAD] to [LOADING] May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [17-1] 2024-05-31 09:35:11: pid 2818: LOG: new outbound connection to pgsql02:9000 May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [18-1] 2024-05-31 09:35:11: pid 2818: LOG: new outbound connection to pgsql03:9000 May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [19-1] 2024-05-31 09:35:11: pid 2818: LOG: watchdog node state changed from [LOADING] to [INITIALIZING] May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [20-1] 2024-05-31 09:35:11: pid 2818: LOG: new watchdog node connection is received from "192.168.56.11:59528" May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [21-1] 2024-05-31 09:35:11: pid 2818: LOG: setting the remote node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" as watchdog cluster leader May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [22-1] 2024-05-31 09:35:11: pid 2818: LOG: new node joined the cluster hostname:"pgsql02" port:9000 pgpool_port:9999 May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [22-2] 2024-05-31 09:35:11: pid 2818: DETAIL: Pgpool-II version:"4.3.7" watchdog messaging version: 1.2 May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [23-1] 2024-05-31 09:35:11: pid 2818: LOG: new watchdog node connection is received from "192.168.56.12:11959" May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [24-1] 2024-05-31 09:35:11: pid 2818: LOG: new node joined the cluster hostname:"pgsql03" port:9000 pgpool_port:9999 May 31 09:35:11 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [24-2] 2024-05-31 09:35:11: pid 2818: DETAIL: Pgpool-II version:"4.3.7" watchdog messaging version: 1.2 May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [25-1] 2024-05-31 09:35:12: pid 2818: LOG: watchdog node state changed from [INITIALIZING] to [STANDBY] May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [26-1] 2024-05-31 09:35:12: pid 2818: LOG: signal_user1_to_parent_with_reason(1) May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [27-1] 2024-05-31 09:35:12: pid 2818: LOG: successfully joined the watchdog cluster as standby node May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [27-2] 2024-05-31 09:35:12: pid 2818: DETAIL: our join coordinator request is accepted by cluster leader node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [11-1] 2024-05-31 09:35:12: pid 2815: LOG: watchdog process is initialized May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [11-2] 2024-05-31 09:35:12: pid 2815: DETAIL: watchdog messaging data version: 1.2 May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [12-1] 2024-05-31 09:35:12: pid 2815: LOG: Pgpool-II parent process received SIGUSR1 May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [13-1] 2024-05-31 09:35:12: pid 2815: LOG: Pgpool-II parent process received watchdog state change signal from watchdog May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [14-1] 2024-05-31 09:35:12: pid 2815: LOG: we have joined the watchdog cluster as STANDBY node May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [14-2] 2024-05-31 09:35:12: pid 2815: DETAIL: syncing the backend states from the LEADER watchdog node May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [28-1] 2024-05-31 09:35:12: pid 2818: LOG: received the get data request from local pgpool-II on IPC interface May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [29-1] 2024-05-31 09:35:12: pid 2818: LOG: get data request from local pgpool-II node received on IPC interface is forwarded to leader watchdog node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2818]: [29-2] 2024-05-31 09:35:12: pid 2818: DETAIL: waiting for the reply... May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2821]: [12-1] 2024-05-31 09:35:12: pid 2821: LOG: 3 watchdog nodes are configured for lifecheck May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2821]: [13-1] 2024-05-31 09:35:12: pid 2821: LOG: watchdog nodes ID:0 Name:"pgsql01:9999 Linux test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2821]: [13-2] 2024-05-31 09:35:12: pid 2821: DETAIL: Host:"pgsql01" WD Port:9000 pgpool-II port:9999 May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2821]: [14-1] 2024-05-31 09:35:12: pid 2821: LOG: watchdog nodes ID:1 Name:"pgsql02:9999 Linux test-no-zone-2-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2821]: [14-2] 2024-05-31 09:35:12: pid 2821: DETAIL: Host:"pgsql02" WD Port:9000 pgpool-II port:9999 May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2821]: [15-1] 2024-05-31 09:35:12: pid 2821: LOG: watchdog nodes ID:2 Name:"pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2821]: [15-2] 2024-05-31 09:35:12: pid 2821: DETAIL: Host:"pgsql03" WD Port:9000 pgpool-II port:9999 May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [15-1] 2024-05-31 09:35:12: pid 2815: LOG: leader watchdog node "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" returned status for 3 backend nodes May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [16-1] 2024-05-31 09:35:12: pid 2815: LOG: backend:1 is set to UP status May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [16-2] 2024-05-31 09:35:12: pid 2815: DETAIL: backend:1 is UP on cluster leader "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [17-1] 2024-05-31 09:35:12: pid 2815: LOG: backend:2 is set to UP status May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [17-2] 2024-05-31 09:35:12: pid 2815: DETAIL: backend:2 is UP on cluster leader "pgsql03:9999 Linux test-no-zone-3-psql-17ab45bb-7c81-46b5-a617-3af6450d7208" May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [18-1] 2024-05-31 09:35:12: pid 2815: LOG: Setting up socket for 0.0.0.0:9999 May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [19-1] 2024-05-31 09:35:12: pid 2815: LOG: Setting up socket for :::9999 May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2821]: [16-1] 2024-05-31 09:35:12: pid 2821: LOG: watchdog lifecheck trusted server "pgsql01" added for the availability check May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2821]: [17-1] 2024-05-31 09:35:12: pid 2821: LOG: watchdog lifecheck trusted server "pgsql02" added for the availability check May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2821]: [18-1] 2024-05-31 09:35:12: pid 2821: LOG: watchdog lifecheck trusted server "pgsql03" added for the availability check May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2928]: [20-1] 2024-05-31 09:35:12: pid 2928: LOG: process started May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2929]: [20-1] 2024-05-31 09:35:12: pid 2929: LOG: process started May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2926]: [20-1] 2024-05-31 09:35:12: pid 2926: LOG: PCP process: 2926 started May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2927]: [20-1] 2024-05-31 09:35:12: pid 2927: LOG: process started May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [20-1] 2024-05-31 09:35:12: pid 2815: LOG: pgpool-II successfully started. version 4.3.7 (tamahomeboshi) May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [21-1] 2024-05-31 09:35:12: pid 2815: LOG: node status[0]: 0 May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [22-1] 2024-05-31 09:35:12: pid 2815: LOG: node status[1]: 0 May 31 09:35:12 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [23-1] 2024-05-31 09:35:12: pid 2815: LOG: node status[2]: 0 May 31 09:35:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2869]: [16-1] 2024-05-31 09:35:13: pid 2869: LOG: set SO_REUSEPORT option to the socket May 31 09:35:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2869]: [17-1] 2024-05-31 09:35:13: pid 2869: LOG: creating watchdog heartbeat receive socket. May 31 09:35:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2869]: [17-2] 2024-05-31 09:35:13: pid 2869: DETAIL: set SO_REUSEPORT May 31 09:35:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2870]: [16-1] 2024-05-31 09:35:13: pid 2870: LOG: set SO_REUSEPORT option to the socket May 31 09:35:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2870]: [17-1] 2024-05-31 09:35:13: pid 2870: LOG: creating socket for sending heartbeat May 31 09:35:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2870]: [17-2] 2024-05-31 09:35:13: pid 2870: DETAIL: set SO_REUSEPORT May 31 09:35:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2871]: [16-1] 2024-05-31 09:35:13: pid 2871: LOG: set SO_REUSEPORT option to the socket May 31 09:35:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2871]: [17-1] 2024-05-31 09:35:13: pid 2871: LOG: creating watchdog heartbeat receive socket. May 31 09:35:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2871]: [17-2] 2024-05-31 09:35:13: pid 2871: DETAIL: set SO_REUSEPORT May 31 09:35:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2872]: [16-1] 2024-05-31 09:35:13: pid 2872: LOG: set SO_REUSEPORT option to the socket May 31 09:35:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2872]: [17-1] 2024-05-31 09:35:13: pid 2872: LOG: creating socket for sending heartbeat May 31 09:35:13 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2872]: [17-2] 2024-05-31 09:35:13: pid 2872: DETAIL: set SO_REUSEPORT May 31 09:36:52 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2821]: [19-1] 2024-05-31 09:36:52: pid 2821: LOG: watchdog: lifecheck started May 31 09:37:27 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2924]: [20-1] 2024-05-31 09:37:27: pid 2924: LOG: statement: SHOW POOL_NODES; May 31 09:37:27 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2924]: [21-1] 2024-05-31 09:37:27: pid 2924: LOG: DB node id: 1 backend pid: 4050610 statement: SELECT pg_catalog.version() May 31 09:37:27 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2924]: [22-1] 2024-05-31 09:37:27: pid 2924: LOG: statement: DISCARD ALL May 31 09:37:27 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2924]: [23-1] 2024-05-31 09:37:27: pid 2924: LOG: DB node id: 1 backend pid: 4050610 statement: DISCARD ALL May 31 09:37:27 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2924]: [24-1] 2024-05-31 09:37:27: pid 2924: LOG: DB node id: 2 backend pid: 4018580 statement: DISCARD ALL May 31 09:42:26 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2925]: [20-1] 2024-05-31 09:42:26: pid 2925: LOG: statement: SHOW POOL_NODES; May 31 09:42:26 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2925]: [21-1] 2024-05-31 09:42:26: pid 2925: LOG: DB node id: 1 backend pid: 4051753 statement: SELECT pg_catalog.version() May 31 09:42:26 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2925]: [22-1] 2024-05-31 09:42:26: pid 2925: LOG: statement: DISCARD ALL May 31 09:42:26 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2925]: [23-1] 2024-05-31 09:42:26: pid 2925: LOG: DB node id: 1 backend pid: 4051753 statement: DISCARD ALL May 31 09:42:26 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2925]: [24-1] 2024-05-31 09:42:26: pid 2925: LOG: DB node id: 2 backend pid: 4019540 statement: DISCARD ALL May 31 09:47:25 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2876]: [20-1] 2024-05-31 09:47:25: pid 2876: LOG: statement: SHOW POOL_NODES; May 31 09:47:25 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2876]: [21-1] 2024-05-31 09:47:25: pid 2876: LOG: DB node id: 1 backend pid: 4052855 statement: SELECT pg_catalog.version() May 31 09:47:25 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2876]: [22-1] 2024-05-31 09:47:25: pid 2876: LOG: statement: DISCARD ALL May 31 09:47:25 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2876]: [23-1] 2024-05-31 09:47:25: pid 2876: LOG: DB node id: 1 backend pid: 4052855 statement: DISCARD ALL May 31 09:47:25 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2876]: [24-1] 2024-05-31 09:47:25: pid 2876: LOG: DB node id: 2 backend pid: 4020513 statement: DISCARD ALL May 31 09:47:28 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [24-1] 2024-05-31 09:47:28: pid 2815: LOG: child process with pid: 2924 exits with status 256 May 31 09:47:28 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [25-1] 2024-05-31 09:47:28: pid 2815: LOG: fork a new child process with pid: 4472 May 31 09:52:24 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2874]: [20-1] 2024-05-31 09:52:24: pid 2874: LOG: statement: SHOW POOL_NODES; May 31 09:52:24 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2874]: [21-1] 2024-05-31 09:52:24: pid 2874: LOG: DB node id: 1 backend pid: 4054029 statement: SELECT pg_catalog.version() May 31 09:52:24 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2874]: [22-1] 2024-05-31 09:52:24: pid 2874: LOG: statement: DISCARD ALL May 31 09:52:24 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2874]: [23-1] 2024-05-31 09:52:24: pid 2874: LOG: DB node id: 1 backend pid: 4054029 statement: DISCARD ALL May 31 09:52:24 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2874]: [24-1] 2024-05-31 09:52:24: pid 2874: LOG: DB node id: 2 backend pid: 4021484 statement: DISCARD ALL May 31 09:52:27 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [26-1] 2024-05-31 09:52:27: pid 2815: LOG: child process with pid: 2925 exits with status 256 May 31 09:52:27 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [27-1] 2024-05-31 09:52:27: pid 2815: LOG: fork a new child process with pid: 5133 May 31 09:57:23 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[4472]: [25-1] 2024-05-31 09:57:23: pid 4472: LOG: statement: SHOW POOL_NODES; May 31 09:57:23 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[4472]: [26-1] 2024-05-31 09:57:23: pid 4472: LOG: DB node id: 1 backend pid: 4055175 statement: SELECT pg_catalog.version() May 31 09:57:23 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[4472]: [27-1] 2024-05-31 09:57:23: pid 4472: LOG: statement: DISCARD ALL May 31 09:57:23 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[4472]: [28-1] 2024-05-31 09:57:23: pid 4472: LOG: DB node id: 1 backend pid: 4055175 statement: DISCARD ALL May 31 09:57:23 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[4472]: [29-1] 2024-05-31 09:57:23: pid 4472: LOG: DB node id: 2 backend pid: 4022414 statement: DISCARD ALL May 31 09:57:26 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [28-1] 2024-05-31 09:57:26: pid 2815: LOG: child process with pid: 2876 exits with status 256 May 31 09:57:26 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [29-1] 2024-05-31 09:57:26: pid 2815: LOG: fork a new child process with pid: 5785 May 31 10:02:22 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[4472]: [30-1] 2024-05-31 10:02:22: pid 4472: LOG: statement: SHOW POOL_NODES; May 31 10:02:22 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[4472]: [31-1] 2024-05-31 10:02:22: pid 4472: LOG: statement: DISCARD ALL May 31 10:02:22 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[4472]: [32-1] 2024-05-31 10:02:22: pid 4472: LOG: DB node id: 1 backend pid: 4055175 statement: DISCARD ALL May 31 10:02:22 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[4472]: [33-1] 2024-05-31 10:02:22: pid 4472: LOG: DB node id: 2 backend pid: 4022414 statement: DISCARD ALL May 31 10:02:25 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [30-1] 2024-05-31 10:02:25: pid 2815: LOG: child process with pid: 2874 exits with status 256 May 31 10:02:25 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [31-1] 2024-05-31 10:02:25: pid 2815: LOG: fork a new child process with pid: 6433 May 31 10:07:21 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[5785]: [29-1] 2024-05-31 10:07:21: pid 5785: LOG: statement: SHOW POOL_NODES; May 31 10:07:21 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[5785]: [30-1] 2024-05-31 10:07:21: pid 5785: LOG: DB node id: 1 backend pid: 4057457 statement: SELECT pg_catalog.version() May 31 10:07:21 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[5785]: [31-1] 2024-05-31 10:07:21: pid 5785: LOG: statement: DISCARD ALL May 31 10:07:21 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[5785]: [32-1] 2024-05-31 10:07:21: pid 5785: LOG: DB node id: 1 backend pid: 4057457 statement: DISCARD ALL May 31 10:07:21 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[5785]: [33-1] 2024-05-31 10:07:21: pid 5785: LOG: DB node id: 2 backend pid: 4024354 statement: DISCARD ALL May 31 10:12:21 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2861]: [20-1] 2024-05-31 10:12:21: pid 2861: LOG: statement: SHOW POOL_NODES; May 31 10:12:21 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2861]: [21-1] 2024-05-31 10:12:21: pid 2861: LOG: DB node id: 1 backend pid: 4058566 statement: SELECT pg_catalog.version() May 31 10:12:21 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2861]: [22-1] 2024-05-31 10:12:21: pid 2861: LOG: statement: DISCARD ALL May 31 10:12:21 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2861]: [23-1] 2024-05-31 10:12:21: pid 2861: LOG: DB node id: 1 backend pid: 4058566 statement: DISCARD ALL May 31 10:12:21 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2861]: [24-1] 2024-05-31 10:12:21: pid 2861: LOG: DB node id: 2 backend pid: 4025324 statement: DISCARD ALL May 31 10:12:24 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [32-1] 2024-05-31 10:12:24: pid 2815: LOG: child process with pid: 4472 exits with status 256 May 31 10:12:24 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [33-1] 2024-05-31 10:12:24: pid 2815: LOG: fork a new child process with pid: 7728 May 31 10:17:20 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2904]: [20-1] 2024-05-31 10:17:20: pid 2904: LOG: statement: SHOW POOL_NODES; May 31 10:17:20 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2904]: [21-1] 2024-05-31 10:17:20: pid 2904: LOG: DB node id: 1 backend pid: 4059714 statement: SELECT pg_catalog.version() May 31 10:17:20 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2904]: [22-1] 2024-05-31 10:17:20: pid 2904: LOG: statement: DISCARD ALL May 31 10:17:20 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2904]: [23-1] 2024-05-31 10:17:20: pid 2904: LOG: DB node id: 1 backend pid: 4059714 statement: DISCARD ALL May 31 10:17:20 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2904]: [24-1] 2024-05-31 10:17:20: pid 2904: LOG: DB node id: 2 backend pid: 4026306 statement: DISCARD ALL May 31 10:17:23 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [34-1] 2024-05-31 10:17:23: pid 2815: LOG: child process with pid: 5785 exits with status 256 May 31 10:17:23 test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208 pgpool[2815]: [35-1] 2024-05-31 10:17:23: pid 2815: LOG: fork a new child process with pid: 8522 pgpool.conf.txt (44,378 bytes)
root@test-no-zone-1-psql-17ab45bb-7c81-46b5-a617-3af6450d7208:/var/log/pgpool# cat /etc/pgpool2/pgpool.conf
# ----------------------------
# pgPool-II configuration file
# ----------------------------
#
# This file consists of lines of the form:
#
# name = value
#
# Whitespace may be used. Comments are introduced with "#" anywhere on a line.
# The complete list of parameter names and allowed values can be found in the
# pgPool-II documentation.
#
# This file is read on server startup and when the server receives a SIGHUP
# signal. If you edit the file on a running system, you have to SIGHUP the
# server for the changes to take effect, or use "pgpool reload". Some
# parameters, which are marked below, require a server shutdown and restart to
# take effect.
#
#------------------------------------------------------------------------------
# CONNECTIONS
#------------------------------------------------------------------------------
# - pgpool Connection Settings -
listen_addresses = '*'
# Host name or IP address to listen on:
# '*' for all, '' for no TCP/IP connections
# (change requires restart)
port = 9999
# Port number
# (change requires restart)
socket_dir = '/var/run/postgresql'
# Unix domain socket path
# The Debian package defaults to
# /var/run/postgresql
# (change requires restart)
listen_backlog_multiplier = 2
# Set the backlog parameter of listen(2) to
# num_init_children * listen_backlog_multiplier.
# (change requires restart)
serialize_accept = off
# whether to serialize accept() call to avoid thundering herd problem
# (change requires restart)
reserved_connections = 4
# Number of reserved connections.
# Pgpool-II does not accept connections if over
# num_init_chidlren - reserved_connections.
# - pgpool Communication Manager Connection Settings -
pcp_listen_addresses = '*'
# Host name or IP address for pcp process to listen on:
# '*' for all, '' for no TCP/IP connections
# (change requires restart)
pcp_port = 9898
# Port number for pcp
# (change requires restart)
pcp_socket_dir = '/var/run/postgresql'
# Unix domain socket path for pcp
# The Debian package defaults to
# /var/run/postgresql
# (change requires restart)
# - Backend Connection Settings -
backend_hostname0 = 'pgsql01'
backend_port0 = 5432
backend_weight0 = 1
backend_data_directory0 = '/var/lib/postgresql/15/main'
backend_flag0 = 'ALLOW_TO_FAILOVER'
backend_application_name0 = 'pgsql01'
backend_hostname1 = 'pgsql02'
backend_port1 = 5432
backend_weight1 = 1
backend_data_directory1 = '/var/lib/postgresql/15/main'
backend_flag1 = 'ALLOW_TO_FAILOVER'
backend_application_name1 = 'pgsql02'
backend_hostname2 = 'pgsql03'
backend_port2 = 5432
backend_weight2 = 1
backend_data_directory2 = '/var/lib/postgresql/15/main'
backend_flag2 = 'ALLOW_TO_FAILOVER'
backend_application_name2 = 'pgsql03'
#backend_hostname0 = 'localhost'
# Host name or IP address to connect to for backend 0
#backend_port0 = 5432
# Port number for backend 0
#backend_weight0 = 1
# Weight for backend 0 (only in load balancing mode)
#backend_data_directory0 = '/var/lib/pgsql/data'
# Data directory for backend 0
#backend_flag0 = 'ALLOW_TO_FAILOVER'
# Controls various backend behavior
# ALLOW_TO_FAILOVER, DISALLOW_TO_FAILOVER
# or ALWAYS_MASTER
#backend_application_name0 = 'server0'
# walsender's application_name, used for "show pool_nodes" command
# - Authentication -
enable_pool_hba = on
# Use pool_hba.conf for client authentication
pool_passwd = 'pool_passwd'
# File name of pool_passwd for md5 authentication.
# "" disables pool_passwd.
# (change requires restart)
authentication_timeout = 60
# Delay in seconds to complete client authentication
# 0 means no timeout.
allow_clear_text_frontend_auth = on
# Allow Pgpool-II to use clear text password authentication
# with clients, when pool_passwd does not
# contain the user password
# - SSL Connections -
ssl = on
# Enable SSL support
# (change requires restart)
ssl_key = './server.key'
# Path to the SSL private key file
# (change requires restart)
ssl_cert = './server.cert'
# Path to the SSL public certificate file
# (change requires restart)
ssl_ca_cert = ''
# Path to a single PEM format file
# containing CA root certificate(s)
# (change requires restart)
ssl_ca_cert_dir = ''
# Directory containing CA root certificate(s)
# (change requires restart)
ssl_ciphers = 'HIGH:MEDIUM:+3DES:!aNULL'
# Allowed SSL ciphers
# (change requires restart)
ssl_prefer_server_ciphers = off
# Use server's SSL cipher preferences,
# rather than the client's
# (change requires restart)
ssl_ecdh_curve = 'prime256v1'
# Name of the curve to use in ECDH key exchange
ssl_dh_params_file = ''
# Name of the file containing Diffie-Hellman parameters used
# for so-called ephemeral DH family of SSL cipher.
#------------------------------------------------------------------------------
# POOLS
#------------------------------------------------------------------------------
# - Concurrent session and pool size -
num_init_children = 100
# Number of concurrent sessions allowed
# (change requires restart)
max_pool = 2
# Number of connection pool caches per connection
# (change requires restart)
# - Life time -
child_life_time = 300
# Pool exits after being idle for this many seconds
child_max_connections = 10000
# Pool exits after receiving that many connections
# 0 means no exit
connection_life_time = 300
# Connection to backend closes after being idle for this many seconds
# 0 means no close
client_idle_limit = 120
# Client is disconnected after being idle for that many seconds
# (even inside an explicit transactions!)
# 0 means no disconnection
#------------------------------------------------------------------------------
# LOGS
#------------------------------------------------------------------------------
# - Where to log -
log_destination = 'syslog'
# Where to log
# Valid values are combinations of stderr,
# and syslog. Default to stderr.
# - What to log -
log_line_prefix = '%t: pid %p: ' # printf-style string to output at beginning of each log line.
log_connections = off
# Log connections
log_hostname = on
# Hostname will be shown in ps status
# and in logs if connections are logged
log_statement = on
# Log all statements
log_per_node_statement = on
# Log all statements
# with node and backend informations
log_client_messages = off
# Log any client messages
log_standby_delay = 'none'
# Log standby delay
# Valid values are combinations of always,
# if_over_threshold, none
# - Syslog specific -
syslog_facility = 'LOCAL1'
# Syslog local facility. Default to LOCAL0
syslog_ident = 'pgpool'
# Syslog program identification string
# Default to 'pgpool'
# - Debug -
log_error_verbosity = default # terse, default, or verbose messages
client_min_messages = notice
# values in order of decreasing detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# log
# notice
# warning
# error
log_min_messages = info
# values in order of decreasing detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# info
# notice
# warning
# error
# log
# fatal
# panic
#------------------------------------------------------------------------------
# FILE LOCATIONS
#------------------------------------------------------------------------------
pid_file_name = '/var/run/postgresql/pgpool.pid'
# PID file name
# Can be specified as relative to the"
# location of pgpool.conf file or
# as an absolute path
# (change requires restart)
logdir = '/var/log/pgpool'
# Directory of pgPool status file
# (change requires restart)
#------------------------------------------------------------------------------
# CONNECTION POOLING
#------------------------------------------------------------------------------
connection_cache = on
# Activate connection pools
# (change requires restart)
# Semicolon separated list of queries
# to be issued at the end of a session
# The default is for 8.3 and later
reset_query_list = 'ABORT; DISCARD ALL'
# The following one is for 8.2 and before
#reset_query_list = 'ABORT; RESET ALL; SET SESSION AUTHORIZATION DEFAULT'
#------------------------------------------------------------------------------
# REPLICATION MODE
#------------------------------------------------------------------------------
replication_mode = off
# Activate replication mode
# (change requires restart)
replicate_select = off
# Replicate SELECT statements
# when in replication mode
# replicate_select is higher priority than
# load_balance_mode.
insert_lock = on
# Automatically locks a dummy row or a table
# with INSERT statements to keep SERIAL data
# consistency
# Without SERIAL, no lock will be issued
lobj_lock_table =
# When rewriting lo_creat command in
# replication mode, specify table name to
# lock
# - Degenerate handling -
replication_stop_on_mismatch = off
# On disagreement with the packet kind
# sent from backend, degenerate the node
# which is most likely "minority"
# If off, just force to exit this session
failover_if_affected_tuples_mismatch = off
# On disagreement with the number of affected
# tuples in UPDATE/DELETE queries, then
# degenerate the node which is most likely
# "minority".
# If off, just abort the transaction to
# keep the consistency
#------------------------------------------------------------------------------
# LOAD BALANCING MODE
#------------------------------------------------------------------------------
load_balance_mode = on
# Activate load balancing mode
# (change requires restart)
ignore_leading_white_space = on
# Ignore leading white spaces of each query
read_only_function_list = ''
# Comma separated list of function names
# that don't write to database
# Regexp are accepted
write_function_list = 'currval,lastval,nextval,setval'
# Comma separated list of function names
# that write to database
# Regexp are accepted
primary_routing_query_pattern_list = ''
# Semicolon separated list of query patterns
# that should be sent to primary node
# Regexp are accepted
# valid for streaming replicaton mode only.
database_redirect_preference_list = ''
# comma separated list of pairs of database and node id.
# example: postgres:primary,mydb[0-4]:1,mydb[5-9]:2'
# valid for streaming replicaton mode only.
app_name_redirect_preference_list = ''
# comma separated list of pairs of app name and node id.
# example: 'psql:primary,myapp[0-4]:1,myapp[5-9]:standby'
# valid for streaming replicaton mode only.
allow_sql_comments = off
# if on, ignore SQL comments when judging if load balance or
# query cache is possible.
# If off, SQL comments effectively prevent the judgment
# (pre 3.4 behavior).
disable_load_balance_on_write = 'transaction'
# Load balance behavior when write query is issued
# in an explicit transaction.
# Note that any query not in an explicit transaction
# is not affected by the parameter.
# 'transaction' (the default): if a write query is issued,
# subsequent read queries will not be load balanced
# until the transaction ends.
# 'trans_transaction': if a write query is issued,
# subsequent read queries in an explicit transaction
# will not be load balanced until the session ends.
# 'always': if a write query is issued, read queries will
# not be load balanced until the session ends.
statement_level_load_balance = on
# Enables statement level load balancing
#------------------------------------------------------------------------------
# MASTER/SLAVE MODE
#------------------------------------------------------------------------------
# https://www.pgpool.net/docs/latest/en/html/runtime-streaming-replication-check.html
backend_clustering_mode = 'streaming_replication'
# - Streaming -
sr_check_period = 10
# Streaming replication check period
# Disabled (0) by default
sr_check_user = 'repmgr'
# Streaming replication check user
# This is necessary even if you disable
# streaming replication delay check with
# sr_check_period = 0
sr_check_password = 'repmgr'
# Password for streaming replication check user.
# Leaving it empty will make Pgpool-II to first look for the
# Password in pool_passwd file before using the empty password
sr_check_database = 'repmgr'
# Database name for streaming replication check
delay_threshold = 10000000
# Threshold before not dispatching query to standby node
# Unit is in bytes
# Disabled (0) by default
# - Special commands -
follow_primary_command = '/etc/pgpool2/follow_master.sh %d %h %p %D %m %H %M %P %r %R %N %S'
# Executes this command after master failover
# Special values:
# %d = failed node id
# %h = failed node host name
# %p = failed node port number
# %D = failed node database cluster path
# %m = new master node id
# %H = new master node hostname
# %M = old master node id
# %P = old primary node id
# %r = new master port number
# %R = new master database cluster path
# %N = old primary node hostname
# %S = old primary node port number
# %% = '%' character
#------------------------------------------------------------------------------
# HEALTH CHECK GLOBAL PARAMETERS
#------------------------------------------------------------------------------
health_check_period = 10
# Health check period
# Disabled (0) by default
health_check_timeout = 40
# Health check timeout
# 0 means no timeout
health_check_user = 'repmgr'
# Health check user
health_check_password = 'repmgr'
# Password for health check user
# Leaving it empty will make Pgpool-II to first look for the
# Password in pool_passwd file before using the empty password
health_check_database = 'repmgr'
# Database name for health check. If '', tries 'postgres' frist, then 'template1'
health_check_max_retries = 3
# Maximum number of times to retry a failed health check before giving up.
health_check_retry_delay = 2
# Amount of time to wait (in seconds) between retries.
connect_timeout = 10000
# Timeout value in milliseconds before giving up to connect to backend.
# Default is 10000 ms (10 second). Flaky network user may want to increase
# the value. 0 means no timeout.
# Note that this value is not only used for health check,
# but also for ordinary conection to backend.
#------------------------------------------------------------------------------
# HEALTH CHECK PER NODE PARAMETERS (OPTIONAL)
#------------------------------------------------------------------------------
#health_check_period0 = 0
#health_check_timeout0 = 20
#health_check_user0 = 'nobody'
#health_check_password0 = ''
#health_check_database0 = ''
#health_check_max_retries0 = 0
#health_check_retry_delay0 = 1
#connect_timeout0 = 10000
#------------------------------------------------------------------------------
# FAILOVER AND FAILBACK
#------------------------------------------------------------------------------
failover_command = '/etc/pgpool2/failover.sh %d %h %p %D %m %H %M %P %r %R %N %S'
# Executes this command at failover
# Special values:
# %d = failed node id
# %h = failed node host name
# %p = failed node port number
# %D = failed node database cluster path
# %m = new master node id
# %H = new master node hostname
# %M = old master node id
# %P = old primary node id
# %r = new master port number
# %R = new master database cluster path
# %N = old primary node hostname
# %S = old primary node port number
# %% = '%' character
failback_command = ''
# Executes this command at failback.
# Special values:
# %d = failed node id
# %h = failed node host name
# %p = failed node port number
# %D = failed node database cluster path
# %m = new master node id
# %H = new master node hostname
# %M = old master node id
# %P = old primary node id
# %r = new master port number
# %R = new master database cluster path
# %N = old primary node hostname
# %S = old primary node port number
# %% = '%' character
failover_on_backend_error = off
# Initiates failover when reading/writing to the
# backend communication socket fails
# If set to off, pgpool will report an
# error and disconnect the session.
detach_false_primary = off
# Detach false primary if on. Only
# valid in streaming replicaton
# mode and with PostgreSQL 9.6 or
# after.
search_primary_node_timeout = 300
# Timeout in seconds to search for the
# primary node when a failover occurs.
# 0 means no timeout, keep searching
# for a primary node forever.
auto_failback = on
# Dettached backend node reattach automatically
# if replication_state is 'streaming'.
auto_failback_interval = 60
# Min interval of executing auto_failback in
# seconds.
#------------------------------------------------------------------------------
# ONLINE RECOVERY
#------------------------------------------------------------------------------
recovery_user = 'postgres'
# Online recovery user
recovery_password = ''
# Online recovery password
# Leaving it empty will make Pgpool-II to first look for the
# Password in pool_passwd file before using the empty password
recovery_1st_stage_command = 'recovery_1st_stage.sh'
# Executes a command in first stage
recovery_2nd_stage_command = ''
# Executes a command in second stage
recovery_timeout = 90
# Timeout in seconds to wait for the
# recovering node's postmaster to start up
# 0 means no wait
client_idle_limit_in_recovery = 0
# Client is disconnected after being idle
# for that many seconds in the second stage
# of online recovery
# 0 means no disconnection
# -1 means immediate disconnection
#------------------------------------------------------------------------------
# WATCHDOG
#------------------------------------------------------------------------------
# - Enabling -
use_watchdog = on
# Activates watchdog
# (change requires restart)
# - Connection to up stream servers -
trusted_servers = 'pgsql01,pgsql02,pgsql03'
# trusted server list which are used
# to confirm network connection
# (hostA,hostB,hostC,...)
# (change requires restart)
ping_path = '/usr/bin'
# ping command path
# (change requires restart)
# - Watchdog communication Settings -
#hostname0 = 'pgsql01'
# Host name or IP address of this watchdog
# (change requires restart)
#wd_port0 = 9000
# port number for watchdog service
# (change requires restart)
wd_priority = 1
# priority of this watchdog in leader election
# (change requires restart)
wd_authkey = ''
# Authentication key for watchdog communication
# (change requires restart)
wd_ipc_socket_dir = '/var/run/postgresql'
# Unix domain socket path for watchdog IPC socket
# The Debian package defaults to
# /var/run/postgresql
# (change requires restart)
# - Virtual IP control Setting -
delegate_IP = '192.168.56.30'
# delegate IP address
# If this is empty, virtual IP never bring up.
# (change requires restart)
if_cmd_path = '/usr/sbin'
# path to the directory where if_up/down_cmd exists
# If if_up/down_cmd starts with "/", if_cmd_path will be ignored.
# (change requires restart)
if_up_cmd = '/usr/bin/sudo /usr/sbin/ip addr add $_IP_$/24 broadcast 192.168.56.255 dev ens3'
# startup delegate IP command
# (change requires restart)
if_down_cmd = '/usr/bin/sudo /usr/sbin/ip addr del $_IP_$/24 broadcast 192.168.56.255 dev ens3'
# shutdown delegate IP command
# (change requires restart)
arping_path = '/usr/sbin'
# arping command path
# If arping_cmd starts with "/", if_cmd_path will be ignored.
# (change requires restart)
arping_cmd = '/usr/bin/sudo /usr/sbin/arping -U $_IP_$ -w 1 -I ens3'
# arping command
# (change requires restart)
# - Behavior on escalation Setting -
clear_memqcache_on_escalation = on
# Clear all the query cache on shared memory
# when standby pgpool escalate to active pgpool
# (= virtual IP holder).
# This should be off if client connects to pgpool
# not using virtual IP.
# (change requires restart)
wd_escalation_command = ''
# Executes this command at escalation on new active pgpool.
# (change requires restart)
wd_de_escalation_command = ''
# Executes this command when master pgpool resigns from being master.
# (change requires restart)
# - Watchdog consensus settings for failover -
failover_when_quorum_exists = on
# Only perform backend node failover
# when the watchdog cluster holds the quorum
# (change requires restart)
failover_require_consensus = on
# Perform failover when majority of Pgpool-II nodes
# aggrees on the backend node status change
# (change requires restart)
allow_multiple_failover_requests_from_node = off
# A Pgpool-II node can cast multiple votes
# for building the consensus on failover
# (change requires restart)
enable_consensus_with_half_votes = off
# apply majority rule for consensus and quorum computation
# at 50% of votes in a cluster with even number of nodes.
# when enabled the existence of quorum and consensus
# on failover is resolved after receiving half of the
# total votes in the cluster, otherwise both these
# decisions require at least one more vote than
# half of the total votes.
# (change requires restart)
# - Lifecheck Setting -
# -- common --
wd_monitoring_interfaces_list = '' # Comma separated list of interfaces names to monitor.
# if any interface from the list is active the watchdog will
# consider the network is fine
# 'any' to enable monitoring on all interfaces except loopback
# '' to disable monitoring
# (change requires restart)
wd_lifecheck_method = 'heartbeat'
# Method of watchdog lifecheck ('heartbeat' or 'query' or 'external')
# (change requires restart)
wd_interval = 10
# lifecheck interval (sec) > 0
# (change requires restart)
# -- heartbeat mode --
# heartbeat_hostname0 = pgsql01
# heartbeat_port0 = 9694
# # Port number for receiving heartbeat signal
# # (change requires restart)
# heartbeat_device0 = ''
wd_heartbeat_keepalive = 2
# Interval time of sending heartbeat signal (sec)
# (change requires restart)
wd_heartbeat_deadtime = 30
# Deadtime interval for heartbeat signal (sec)
# (change requires restart)
heartbeat_hostname0 = 'pgsql01'
heartbeat_port0 = '9694'
heartbeat_device0 = ''
heartbeat_hostname1 = 'pgsql02'
heartbeat_port1 = '9694'
heartbeat_device1 = ''
heartbeat_hostname2 = 'pgsql03'
heartbeat_port2 = '9694'
heartbeat_device2 = ''
#heartbeat_destination0 = 'host0_ip1'
# Host name or IP address of destination 0
# for sending heartbeat signal.
# (change requires restart)
#heartbeat_destination_port0 = 9694
# Port number of destination 0 for sending
# heartbeat signal. Usually this is the
# same as wd_heartbeat_port.
# (change requires restart)
#heartbeat_device0 = ''
# Name of NIC device (such like 'eth0')
# used for sending/receiving heartbeat
# signal to/from destination 0.
# This works only when this is not empty
# and pgpool has root privilege.
# (change requires restart)
# -- query mode --
wd_life_point = 3
# lifecheck retry times
# (change requires restart)
wd_lifecheck_query = 'SELECT 1'
# lifecheck query to pgpool from watchdog
# (change requires restart)
wd_lifecheck_dbname = 'template1'
# Database name connected for lifecheck
# (change requires restart)
wd_lifecheck_user = 'nobody'
# watchdog user monitoring pgpools in lifecheck
# (change requires restart)
wd_lifecheck_password = ''
# Password for watchdog user in lifecheck
# Leaving it empty will make Pgpool-II to first look for the
# Password in pool_passwd file before using the empty password
# (change requires restart)
# - Other pgpool Connection Settings -
hostname0 = pgsql01
pgpool_port0 = 9999
wd_port0 = 9000
hostname1 = pgsql02
pgpool_port1 = 9999
wd_port1 = 9000
hostname2 = pgsql03
pgpool_port2 = 9999
wd_port2 = 9000
#other_pgpool_hostname0 = 'host0'
# Host name or IP address to connect to for other pgpool 0
# (change requires restart)
#other_pgpool_port0 = 5432
# Port number for other pgpool 0
# (change requires restart)
#other_wd_port0 = 9000
# Port number for other watchdog 0
# (change requires restart)
#------------------------------------------------------------------------------
# OTHERS
#------------------------------------------------------------------------------
relcache_expire = 0
# Life time of relation cache in seconds.
# 0 means no cache expiration(the default).
# The relation cache is used for cache the
# query result against PostgreSQL system
# catalog to obtain various information
# including table structures or if it's a
# temporary table or not. The cache is
# maintained in a pgpool child local memory
# and being kept as long as it survives.
# If someone modify the table by using
# ALTER TABLE or some such, the relcache is
# not consistent anymore.
# For this purpose, cache_expiration
# controls the life time of the cache.
relcache_size = 256
# Number of relation cache
# entry. If you see frequently:
# "pool_search_relcache: cache replacement happend"
# in the pgpool log, you might want to increate this number.
check_temp_table = catalog
# Temporary table check method. catalog, trace or none.
# Default is catalog.
check_unlogged_table = on
# If on, enable unlogged table check in SELECT statements.
# This initiates queries against system catalog of primary/master
# thus increases load of master.
# If you are absolutely sure that your system never uses unlogged tables
# and you want to save access to primary/master, you could turn this off.
# Default is on.
enable_shared_relcache = on
# If on, relation cache stored in memory cache,
# the cache is shared among child process.
# Default is on.
# (change requires restart)
relcache_query_target = primary
# Target node to send relcache queries. Default is master (primary) node.
# If load_balance_node is specified, queries will be sent to load balance node.
#------------------------------------------------------------------------------
# IN MEMORY QUERY MEMORY CACHE
#------------------------------------------------------------------------------
memory_cache_enabled = off
# If on, use the memory cache functionality, off by default
# (change requires restart)
memqcache_method = 'shmem'
# Cache storage method. either 'shmem'(shared memory) or
# 'memcached'. 'shmem' by default
# (change requires restart)
memqcache_memcached_host = 'localhost'
# Memcached host name or IP address. Mandatory if
# memqcache_method = 'memcached'.
# Defaults to localhost.
# (change requires restart)
memqcache_memcached_port = 11211
# Memcached port number. Mondatory if memqcache_method = 'memcached'.
# Defaults to 11211.
# (change requires restart)
memqcache_total_size = 67108864
# Total memory size in bytes for storing memory cache.
# Mandatory if memqcache_method = 'shmem'.
# Defaults to 64MB.
# (change requires restart)
memqcache_max_num_cache = 1000000
# Total number of cache entries. Mandatory
# if memqcache_method = 'shmem'.
# Each cache entry consumes 48 bytes on shared memory.
# Defaults to 1,000,000(45.8MB).
# (change requires restart)
memqcache_expire = 0
# Memory cache entry life time specified in seconds.
# 0 means infinite life time. 0 by default.
# (change requires restart)
memqcache_auto_cache_invalidation = on
# If on, invalidation of query cache is triggered by corresponding
# DDL/DML/DCL(and memqcache_expire). If off, it is only triggered
# by memqcache_expire. on by default.
# (change requires restart)
memqcache_maxcache = 409600
# Maximum SELECT result size in bytes.
# Must be smaller than memqcache_cache_block_size. Defaults to 400KB.
# (change requires restart)
memqcache_cache_block_size = 1048576
# Cache block size in bytes. Mandatory if memqcache_method = 'shmem'.
# Defaults to 1MB.
# (change requires restart)
memqcache_oiddir = '/var/log/pgpool/oiddir'
# Temporary work directory to record table oids
# (change requires restart)
cache_safe_memqcache_table_list = ''
# Comma separated list of table names to memcache
# that don't write to database
# Regexp are accepted
cache_unsafe_memqcache_table_list = ''
# Comma separated list of table names not to memcache
# that don't write to database
# Regexp are accepted
|
|
|
I meant i can't imagine it doesn't happen. I install this with ansible based on two roles and I have done that many times, the result is always the same. The machines are VMs in OpenStack |
|
|
Hi Vincent, Thank you for providing all the information and log files. I believe I have identified the issue. The problem was that network monitoring kept polling for events even when monitoring was disabled. I have created a patch to address this and will commit it after additional testing. In the meantime, if you would like to test your scenario with the patch, I am attaching it here. The patch (fix_823.diff) is generated against the 4.5 branch, but it should also work with the master branch. Thank you for your patience. Best regards, Muhammad Usama fix_823.diff (1,750 bytes)
diff --git a/src/watchdog/watchdog.c b/src/watchdog/watchdog.c
index f929623cb..a1a06aa68 100644
--- a/src/watchdog/watchdog.c
+++ b/src/watchdog/watchdog.c
@@ -812,6 +812,7 @@ wd_cluster_initialize(void)
g_cluster.de_escalation_pid = 0;
g_cluster.unidentified_socks = NULL;
g_cluster.command_server_sock = 0;
+ g_cluster.network_monitor_sock = 0;
g_cluster.notify_clients = NULL;
g_cluster.ipc_command_socks = NULL;
g_cluster.wd_timer_commands = NULL;
@@ -1195,8 +1196,8 @@ watchdog_main(void)
/* open the command server */
g_cluster.command_server_sock = wd_create_command_server_socket();
- /* try connecting to all watchdog nodes */
- g_cluster.network_monitor_sock = create_monitoring_socket();
+ if (g_cluster.wdInterfaceToMonitor)
+ g_cluster.network_monitor_sock = create_monitoring_socket();
if (any_interface_available() == false)
{
@@ -1207,6 +1208,7 @@ watchdog_main(void)
errhint("you can disable interface checking by setting wd_monitoring_interfaces_list = '' in pgpool config")));
}
+ /* try connecting to all watchdog nodes */
connect_with_all_configured_nodes();
/* set the initial state of local node */
@@ -1414,9 +1416,12 @@ prepare_fds(fd_set *rmask, fd_set *wmask, fd_set *emask)
if (fd_max < g_cluster.command_server_sock)
fd_max = g_cluster.command_server_sock;
- FD_SET(g_cluster.network_monitor_sock, rmask);
- if (fd_max < g_cluster.network_monitor_sock)
- fd_max = g_cluster.network_monitor_sock;
+ if (g_cluster.network_monitor_sock > 0)
+ {
+ FD_SET(g_cluster.network_monitor_sock, rmask);
+ if (fd_max < g_cluster.network_monitor_sock)
+ fd_max = g_cluster.network_monitor_sock;
+ }
/*
* set write fdset for all waiting for connection sockets, while already
|
|
|
hi Muhammad, thank you for the response and the patch, that's great! I am running pgpool from the default ubuntu 22.04 repositories. do you happen to build those somewhere so I can test it with patch? I see there are if statements added that check for the network monitoring setting, does it mean if that is disabled that the other parameters apply ? If you cannot provide apt based deb packages for me to try, is there a description somewhere on how to build ? thank you and regards, Vincent |
|
|
Hi Vincent, Yes, that is true. To try out the scenario, you must disable the interface monitoring with this patch. Unfortunately Pgpool-II community does not build the deb package, but I guess you can use the spec files from https://salsa.debian.org/postgresql/pgpool2 repository if you want to build it yourself. Thanks |
|
|
hi, ok, is there a binary you compiled for x86_64 Linux that may run on Ubuntu 22.04 ? I am happy to test that, I have little time to build a deb at the moment, unless there is a clear manual somewhere. Did you test the changes ? I mean, shouldn't there be some tests in place that try to catch these kind of errors? I'll do my best to try and test this, but I can't make any promises if I need to build it :). If you need help setting up some integration tests, let me know, I've done that before. cheers |
|
|
Hi, I've committed the patch to all supported branches, and it should fix your issue. Unfortunately, I don't have binaries for x86_64 as I use Mac for development and testing. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2023-12-21 18:30 | openminded | New Issue | |
| 2023-12-21 18:30 | openminded | Tag Attached: watchdog | |
| 2023-12-26 14:33 | pengbo | Assigned To | => Muhammad Usama |
| 2023-12-26 14:33 | pengbo | Status | new => assigned |
| 2024-02-02 07:39 | openminded | Note Added: 0004480 | |
| 2024-05-30 21:38 | Muhammad Usama | Note Added: 0004506 | |
| 2024-05-30 23:14 | openminded | Note Added: 0004507 | |
| 2024-05-31 00:24 | Muhammad Usama | Note Added: 0004508 | |
| 2024-05-31 01:30 | openminded | Note Added: 0004509 | |
| 2024-05-31 16:47 | Muhammad Usama | Note Added: 0004510 | |
| 2024-05-31 18:34 | openminded | Note Added: 0004511 | |
| 2024-05-31 18:52 | Muhammad Usama | Note Added: 0004512 | |
| 2024-05-31 19:22 | openminded | Note Added: 0004513 | |
| 2024-05-31 19:22 | openminded | File Added: pgpool.log.txt | |
| 2024-05-31 19:22 | openminded | File Added: pgpool.conf.txt | |
| 2024-05-31 23:28 | openminded | Note Added: 0004514 | |
| 2024-06-03 20:11 | Muhammad Usama | Note Added: 0004515 | |
| 2024-06-03 20:11 | Muhammad Usama | File Added: fix_823.diff | |
| 2024-06-03 20:19 | openminded | Note Added: 0004516 | |
| 2024-06-03 23:58 | Muhammad Usama | Note Added: 0004517 | |
| 2024-06-05 05:42 | openminded | Note Added: 0004518 | |
| 2024-06-11 20:22 | Muhammad Usama | Note Added: 0004519 |
