[pgpool-general: 7382] Re: Watchdog New Primary & Standby shutdown when Node 0 Fails
Joe.Madden at mottmac.com
Mon Dec 21 22:49:37 JST 2020
No problem - Thanks for the update and fixing it so quickly - We do appreciate it!
Senior Systems Engineer
joe.madden at mottmac.com
From: Tatsuo Ishii <ishii at sraoss.co.jp>
Sent: 21 December 2020 12:42
To: Joe Madden <Joe.Madden at mottmac.com>
Cc: m.usama at gmail.com; pgpool-general at pgpool.net
Subject: Re: [pgpool-general: 7372] Re: Watchdog New Primary & Standby shutdown when Node 0 Fails
I was able to reproduce your problem by using watchdog_setup. I just made a mistake in testing. Sorry for noise.
Steps to reproduce the problem:
$ watchdog_setup -wn 3 -n 2 # create 3 watchdog node and 2 postgres node.
$ cd pgpool0
EDIT shutdownall script to not shutdown postgres
$PGPOOL_INSTALL_DIR/bin/pgpool -f $dir/etc/pgpool.conf -m f stop while [ -f $dir/run/pgpool.pid ];do sleep 1;done #/usr/local/pgsql/bin/pg_ctl -D /home/t-ishii/work/Pgpool-II/current/a/pgpool0/data0 -m f stop <-- comment out #/usr/local/pgsql/bin/pg_ctl -D /home/t-ishii/work/Pgpool-II/current/a/pgpool0/data1 -m f stop <-- comment out
# shutdown pgpool0
With 4.2.0 watchdog, port for pgpool1 and pgpool2 cannot be connected (your problem).
$ pcp_watchdog_info -w -p 50005
ERROR: connection to socket "/tmp/.s.PGSQL.50005" failed with error "No such file or directory"
$ pcp_watchdog_info -w -p 50009
ERROR: connection to socket "/tmp/.s.PGSQL.50009" failed with error "No such file or directory"
With 4.2 stable head:
$ pcp_watchdog_info -w -p 50005
localhost:50004 Linux tishii-CFSV7-1 localhost 50004 50006 4 LEADER <-- pgpool1 is properly promoted
localhost:50000 Linux tishii-CFSV7-1 localhost 50000 50002 10 SHUTDOWN <-- pgpool0 shutdown as expected
localhost:50008 Linux tishii-CFSV7-1 localhost 50008 50010 7 STANDBY
So I can confirm that the problem is solved in 4.2 stable head (= supposed to be 4.2.1).
BTW, Pengbo said that she is going to release 4.2.1 on this Wednesday, December 23. I believe she is going to release RPMs as well.
> Hi Tatsuo,
> I haven't tried it with the watchdog_setup, I just configured it myself following the documentation.
> I will try before the holidays and let you know.
> Joe Madden
> Senior Systems Engineer
> D 01412224666
> joe.madden at mottmac.com
> -----Original Message-----
> From: Tatsuo Ishii <ishii at sraoss.co.jp>
> Sent: 20 December 2020 10:58
> To: m.usama at gmail.com
> Cc: Joe Madden <Joe.Madden at mottmac.com>; pgpool-general at pgpool.net
> Subject: Re: [pgpool-general: 7372] Re: Watchdog New Primary & Standby
> shutdown when Node 0 Fails
> Hi Usama,
>> Both wd_cli and the lifecheck mechanism uses the same path and commit
>> messages only mentions the wd_cli.
>> Looking at the email I think it's a very critical issue and we should
>> do a point release for 4.2 The issue was caused by an oversight by
>> the "simplifying watchdog configuration" feature which was introduced
>> in 4.2, so the older versions should not have the same problem.
>> Best regards
>> Muhammad Usama
> BTW, I wonder why watchdog cluster created by watchdog_setup does not show the problem.
>> If I configure with node 0 always being dead:
> Do you have any idea?
> Tatsuo Ishii
> SRA OSS, Inc. Japan
More information about the pgpool-general