[pgpool-general: 7823] Re: Pgpool2 example with watchdog: Unable to test

[Bo Peng]

Hello,

Is Pgpool-II started properly? 
You may need to check the logs after starting pgpool.

On Sun, 24 Oct 2021 20:18:28 +0300
Nirav Bhatt <niravbhatt.cpp at gmail.com> wrote:

> After doing all the config (config files, passwordless ssh, firewall-cmd
> etc) as given in the following link, I am unable to test my setup. My OS is
> Ubuntu 20.04.3 LTS.
> 
> https://www.pgpool.net/docs/41/en/html/example-cluster.html#EXAMPLE-CLUSTER-STRUCTURE
> 
> All the port numbers are as listed in the example.
> 
> The part in example section: 8.3.7.1 and onwards is not working.
> 
> When I run on server1 (my primary):
> psql -h 20.20.20.20 -p 9999 -U pgpool postgres -c "show pool_nodes"
> 
> I got error:
> 
> psql: error: connection to server at "20.20.20.20", port 9999 failed:
> Connection refused
> Is the server running on that host and accepting TCP/IP connections?
> 
> I restarted postgresql and pgpool2 services time and again.
> 
> pgpool2 service most recent logs:
> 
> server1 (current primary):
> Oct 24 17:01:41 ip-10-192-10-10 pgpool[104691]: 2021-10-24 17:01:41: pid
> 104691: LOG:  new watchdog node connection is received from "<server1
> host>:16553"
> Oct 24 17:01:41 ip-10-192-10-10 pgpool[104691]: 2021-10-24 17:01:41: pid
> 104691: LOG:  new watchdog node connection is received from "<server1
> host>:17577"
> Oct 24 17:04:02 ip-10-192-10-10 pgpool[104691]: 2021-10-24 17:04:02: pid
> 104691: LOG:  new watchdog node connection is received from "<server2
> host>:49304"
> Oct 24 17:04:02 ip-10-192-10-10 pgpool[104691]: 2021-10-24 17:04:02: pid
> 104691: LOG:  new watchdog node connection is received from "<server2
> host>:50328"
> Oct 24 17:11:12 ip-10-192-10-10 pgpool[104691]: 2021-10-24 17:11:12: pid
> 104691: LOG:  new watchdog node connection is received from "<server3
> host>:51864"
> Oct 24 17:11:12 ip-10-192-10-10 pgpool[104691]: 2021-10-24 17:11:12: pid
> 104691: LOG:  new watchdog node connection is received from "<server3
> host>:52888"
> 
> server2 and server3 (looks like pgpool2 service is failing):
> 
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24 17:11:12: pid
> 125018: LOG:  setting the local watchdog node name to "server3:5433 Linux
> ip-<server 3 host>"
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24 17:11:12: pid
> 125018: LOG:  watchdog cluster is configured with 2 remote nodes
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24 17:11:12: pid
> 125018: LOG:  watchdog remote node:0 on server1:9000
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24 17:11:12: pid
> 125018: LOG:  watchdog remote node:1 on server2:9000
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24 17:11:12: pid
> 125018: LOG:  interface monitoring is disabled in watchdog
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24 17:11:12: pid
> 125018: LOG:  watchdog node state changed from [DEAD] to [LOADING]
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24 17:11:12: pid
> 125018: LOG:  new outbound connection to server1:9000
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24 17:11:12: pid
> 125018: FATAL:  Add to watchdog cluster request is rejected by node
> "server1:9000"
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24 17:11:12: pid
> 125018: HINT:  check the watchdog configurations.
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24 17:11:12: pid
> 125018: LOG:  Watchdog is shutting down
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125014]: 2021-10-24 17:11:12: pid
> 125014: LOG:  watchdog child process with pid: 125018 exits with status 768
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125014]: 2021-10-24 17:11:12: pid
> 125014: FATAL:  watchdog child process exit with fatal error. exiting
> pgpool-II
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24 17:11:12: pid
> 125019: LOG:  setting the local watchdog node name to "server3:5433 Linux
> ip-<server 3 host>"
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24 17:11:12: pid
> 125019: LOG:  watchdog cluster is configured with 2 remote nodes
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24 17:11:12: pid
> 125019: LOG:  watchdog remote node:0 on server1:9000
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24 17:11:12: pid
> 125019: LOG:  watchdog remote node:1 on server2:9000
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24 17:11:12: pid
> 125019: LOG:  interface monitoring is disabled in watchdog
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24 17:11:12: pid
> 125019: LOG:  watchdog node state changed from [DEAD] to [LOADING]
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24 17:11:12: pid
> 125019: LOG:  new outbound connection to server1:9000
> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24 17:11:12: pid
> 125019: LOG:  Watchdog is shutting down
> Oct 24 17:11:12 ip-10-192-12-30 systemd[1]: pgpool2.service: Main process
> exited, code=exited, status=3/NOTIMPLEMENTED
> Oct 24 17:11:12 ip-10-192-12-30 systemd[1]: pgpool2.service: Failed with
> result 'exit-code'.
> 
> What steps am I missing?
> 
> Thank you for the assistance,
> -Nirav


-- 
Bo Peng <pengbo at sraoss.co.jp>
SRA OSS, Inc. Japan
http://www.sraoss.co.jp/