[pgpool-general: 7840] Re: pgpool-general Digest, Vol 120, Issue 45

Luca Maranzano liuk001 at gmail.com
Sun Oct 31 05:31:13 JST 2021 

What about the netmask of the network interface on 3 nodes?

Il sab 30 ott 2021, 22:26 Nirav Bhatt <niravbhatt.cpp at gmail.com> ha scritto:

> Hi Luca
>
> Thanks for the response.
>
> Those logs belonged to the case when I put same pgpool.conf on all 3
> servers, based on your suggestion.
> As I saw later, this should not be done for 4.1.
> So we should probably discard that case.
>
> With separate pgpool.conf file, it is logging the things I sent in earlier
> mail. (connection refused by server1:9000)
>
> Plus:
>
> server2:
> LOG:  setting the local watchdog node name to "server2:5433 Linux
> ip-10-192-11-20"
> LOG:  watchdog remote node:0 on server1:9000
> LOG:  watchdog remote node:1 on server3:9000
>
> server3:
> LOG:  setting the local watchdog node name to "server3:5433 Linux
> ip-10-192-12-30"
> LOG:  watchdog remote node:0 on server1:9000
> LOG:  watchdog remote node:1 on server2:9000
>
> Thanks,
> -Nirav
>
> On Sat, Oct 30, 2021 at 11:01 PM Luca Maranzano <liuk001 at gmail.com> wrote:
>
>> Hi Nirav,
>> you're right, the node id is only from 4.2, sorry for my oversight, I
>> thought it was valid for versione 4.X. It's really strange that there are
>> still no 4.2 debian packages available for Debian and Ubuntu, 4.2 was
>> released in 2020.
>>
>> I took a look at the logs, I found strange this:
>>
>> server1: LOG:  setting the local node "server1:5433 Linux
>> ip-10-192-10-10" as watchdog cluster master    (it seems ok)
>> server2: LOG:  setting the local watchdog node name to "*server1:5433 *Linux
>> ip-10-192-11-20"
>> server3: LOG:  setting the local watchdog node name to "*server1:5433*
>> Linux ip-10-192-12-30"
>>
>> why server2 and server3 are referencing themselves as server1?
>>
>> Besides that you should set the VIP on the same subnet of the servers,
>> for example 10.192.11.100, not with an IP from a different subnet
>> (20.20.20.20)
>> I still have to understand how parameters must be differentiated in 4.1
>> if you cannot use the node id file.
>>
>> I'll try to investigate later.
>> Cheers,
>> Luca
>>
>>
>>
>>
>>
>>
>>
>> On Sat, Oct 30, 2021 at 7:43 PM Nirav Bhatt <niravbhatt.cpp at gmail.com>
>> wrote:
>>
>>> To reiterate, this pgpool2 4.1 setup.
>>>
>>> 1 - I have now restored my different pgpool.conf files on each server,
>>> as described in the link for 4.1:
>>> https://www.pgpool.net/docs/41/en/html/example-cluster.html
>>>
>>> They are now no longer identical.
>>>
>>> They only differ in
>>> - wd_hostname (this server),
>>> - heartbeat_destination0 & heartbeat_destination1 (other servers eg for
>>> server1, this is server2 and server3, and vice versa),
>>> - other_pgpool_hostname0 & other_pgpool_hostname1 (again, other servers)
>>>
>>> I also diffed other files under /etc/pgpool2 and postgres home,
>>> including password files, they are also same across the 3 servers.
>>>
>>> 2 - I restarted the servers just in case port change was not in the
>>> effect.
>>> I also tried telnet and I could see connection going from server2 &
>>> server3 to server1 on 9000.
>>> So port or connectivity isn't issue.
>>>
>>> 3 - The issue is: WD on server1 is refusing connection to server2 and
>>> server3 watchdogs on 9000.
>>>
>>> On server2 and server3 log:
>>>
>>> LOG:  watchdog node state changed from [DEAD] to [LOADING]
>>> LOG:  new outbound connection to server1:9000
>>> FATAL:  Add to watchdog cluster request is rejected by node
>>> "server1:9000"
>>> HINT:  check the watchdog configurations.
>>> LOG:  Watchdog is shutting down
>>>
>>> while server1 log:
>>>
>>> LOG:  new watchdog node connection is received from "10.192.11.20:23277"
>>> <-- server2
>>> LOG:  new watchdog node connection is received from "10.192.12.30:16596"
>>> <-- server3
>>>
>>> (nothing else about why it refused connection - just this thing above)
>>>
>>> I believe that's enough debug info, if you could spot any problems,
>>> please let me know.
>>>
>>> Thanks for the assistance so far,
>>> -Nirav
>>>
>>>
>>>
>>> On Sat, Oct 30, 2021 at 7:24 PM Nirav Bhatt <niravbhatt.cpp at gmail.com>
>>> wrote:
>>>
>>>> I also suspect if following line is true for 4.1 setup:
>>>>
>>>>
>>>> *From Pgpool-II 4.2, now all configuration parameters are identical on
>>>> all hosts*
>>>>
>>>> -Nirav
>>>>
>>>> On Sat, Oct 30, 2021 at 7:22 PM Nirav Bhatt <niravbhatt.cpp at gmail.com>
>>>> wrote:
>>>>
>>>>> Hello
>>>>>
>>>>> I am following this link, which is pgpool2 v4.1.
>>>>>
>>>>> https://www.pgpool.net/docs/41/en/html/example-cluster.html
>>>>>
>>>>> On my Ubuntu setup also that is the latest I can get:
>>>>> *4.1.4-3.pgdg20.04+2*
>>>>>
>>>>> If you still insist, I will try to set it up, but I guess you are
>>>>> quoting from this link:
>>>>>
>>>>>
>>>>> https://www.pgpool.net/docs/latest/en/html/example-cluster.html#EXAMPLE-CLUSTER-PGPOOL-NODE-ID
>>>>>
>>>>> It is a more recent version of pgpool2 than what I have.
>>>>>
>>>>> -Nirav
>>>>>
>>>>> On Sat, Oct 30, 2021 at 7:03 PM Luca Maranzano <liuk001 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello,
>>>>>> I think that the pgpool_node_id file is mandatory since it is used by
>>>>>> every server to identify itself in the cluster with its node id.
>>>>>> Where is it documented that it is not necessary?
>>>>>>
>>>>>> Try to configure it
>>>>>>
>>>>>> HTH
>>>>>> Regards
>>>>>> Luca
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sat, Oct 30, 2021 at 5:42 PM Nirav Bhatt <niravbhatt.cpp at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hello
>>>>>>> Sorry for the late answer.
>>>>>>>
>>>>>>> I now made pgpool.conf identical on all 3 servers, then restarted
>>>>>>> postgresql and pgpool2 services.
>>>>>>>
>>>>>>> I opened ports on each server: 22, 9999/tcp, 9898/tcp, 9000/tcp,
>>>>>>> 9694/udp, 5433, 5432
>>>>>>>
>>>>>>> Yet the problem persists on server2 and server3 - it can not connect
>>>>>>> to 9000 port (on themselves)
>>>>>>> Server1 pgpool seems to be running without problem.
>>>>>>>
>>>>>>> I am attaching my pgpool.conf file. Also, I am attaching pgpool2
>>>>>>> service logs from all 3 servers.
>>>>>>>
>>>>>>> 10.192.10.10 server1
>>>>>>> 10.192.11.20 server2
>>>>>>> 10.192.12.30 server3
>>>>>>>
>>>>>>> Virtual IP is 20.20.20.20
>>>>>>>
>>>>>>> I am running the following pgpool2 version so I do not think I need
>>>>>>> pgpool_node_id file.
>>>>>>>
>>>>>>> 4.1.4-3.pgdg20.04+2
>>>>>>>
>>>>>>> Thanks for the support,
>>>>>>> -Nirav
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Oct 27, 2021 at 8:48 AM Luca Maranzano <liuk001 at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi, still not clear to me, from the doc:
>>>>>>>>
>>>>>>>> 8.2.5. Create pgpool_node_id
>>>>>>>>
>>>>>>>> From Pgpool-II 4.2, now all configuration parameters are identical
>>>>>>>> on all hosts. If watchdog feature is enabled, to distingusish
>>>>>>>> which host is which, a pgpool_node_id file is required. You need
>>>>>>>> to create a pgpool_node_id file and specify the pgpool (watchdog)
>>>>>>>> node number (e.g. 0, 1, 2 ...) to identify pgpool (watchdog) host.
>>>>>>>>
>>>>>>>> In our systems the pgpool.conf are *identical *
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Oct 26, 2021 at 8:09 PM Nirav Bhatt <
>>>>>>>> niravbhatt.cpp at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> The difference lies in “other” server host names.
>>>>>>>>>
>>>>>>>>> -Nirav
>>>>>>>>>
>>>>>>>>> On Tue, 26 Oct 2021 at 21.07, Luca Maranzano <liuk001 at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> AFAIK the pgpool.conf file must be identical on all cluster nodes.
>>>>>>>>>> Which are the differences in detail?
>>>>>>>>>>
>>>>>>>>>> Cheers
>>>>>>>>>> Luca
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Oct 25, 2021 at 9:46 PM Nirav Bhatt <
>>>>>>>>>> niravbhatt.cpp at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I added all ports via ufw on server1:
>>>>>>>>>>> 22,9999,9898,9000,9694,5433,5432
>>>>>>>>>>>
>>>>>>>>>>> I enabled firewall past that point.
>>>>>>>>>>>
>>>>>>>>>>> I diffed all conf files across servers - all files are same,
>>>>>>>>>>> including hba and .pgpass and .pcppass.
>>>>>>>>>>> Only pgpool.conf files differ, that too in parameters like
>>>>>>>>>>> server hosts for pgpool and watchdog, no other differences.
>>>>>>>>>>>
>>>>>>>>>>> No error is observed on primary server1. The last log lines on
>>>>>>>>>>> server1 says: new watchdog node connection is received from <server2> or
>>>>>>>>>>> <server3>
>>>>>>>>>>>
>>>>>>>>>>> But secondary server pgpool2 keeps exiting with:
>>>>>>>>>>>
>>>>>>>>>>> Add to watchdog cluster request is rejected by node
>>>>>>>>>>> "server1:9000"
>>>>>>>>>>>
>>>>>>>>>>> I somehow suspect it has to do with wd_authkey which I leave
>>>>>>>>>>> empty:
>>>>>>>>>>> https://www.sraoss.jp/pipermail/pgpool-hackers/2017-August/002497.html
>>>>>>>>>>>
>>>>>>>>>>> But it is an old issue, so I can't say for sure.
>>>>>>>>>>>
>>>>>>>>>>> -Nirav
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Oct 25, 2021 at 8:07 PM Luca Maranzano <
>>>>>>>>>>> liuk001 at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Firewall-cmd has to be used on Redhat derived flavours, on
>>>>>>>>>>>> Ubuntu standard is ufw, so "ufw disable" should open all ports
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Oct 25, 2021 at 6:59 PM Nirav Bhatt <
>>>>>>>>>>>> niravbhatt.cpp at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I ran sudo ufw status and firewall is already inactive.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Not sure what firewall-cmd did for all the pgpool ports. (they
>>>>>>>>>>>>> are showing as list of open ports)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Should add them all via ufw also?
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Nirav
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Oct 25, 2021 at 7:46 PM Luca Maranzano <
>>>>>>>>>>>>> liuk001 at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Try to disable firewall to see it it's related, all the
>>>>>>>>>>>>>> pgpool configuration files are identical on the 3 nodes?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Oct 25, 2021 at 6:41 PM Nirav Bhatt <
>>>>>>>>>>>>>> niravbhatt.cpp at gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks for the response.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I restarted pgpool2 service on all 3 nodes (with some delay,
>>>>>>>>>>>>>>> and in the order: primary, secondary1, secondary2).
>>>>>>>>>>>>>>> My primary pgpool2 service logs do show
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> LOG:  new watchdog node connection is received from
>>>>>>>>>>>>>>> <secondary host>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> But my secondary pgpool2 service logs (among other normal
>>>>>>>>>>>>>>> messages) show:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> FATAL:  Add to watchdog cluster request is rejected by node
>>>>>>>>>>>>>>> "server1:9000"
>>>>>>>>>>>>>>> FATAL:  watchdog child process exit with fatal error.
>>>>>>>>>>>>>>> exiting pgpool-II
>>>>>>>>>>>>>>> LOG:  watchdog node state changed from [DEAD] to [LOADING]
>>>>>>>>>>>>>>> LOG:  new outbound connection to server1:9000
>>>>>>>>>>>>>>> FATAL:  Add to watchdog cluster request is rejected by node
>>>>>>>>>>>>>>> "server1:9000"
>>>>>>>>>>>>>>> HINT:  check the watchdog configurations.
>>>>>>>>>>>>>>> LOG:  Watchdog is shutting down
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I checked that server1 has 9000 port open, along with 9999,
>>>>>>>>>>>>>>> 9898 and 9694:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > firewall-cmd --list-ports
>>>>>>>>>>>>>>> 9999/tcp 9898/tcp 9000/tcp 9694/udp
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What could be going on?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -Nirav
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Oct 25, 2021 at 7:13 PM Luca Maranzano <
>>>>>>>>>>>>>>> liuk001 at gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> if you cannot connect to 20.20.20.20 -p 9999 it means that
>>>>>>>>>>>>>>>> hte PGPool cluster is not well formed and that the VIP has not been enabled.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You can check that with pcp_watchdog_info.
>>>>>>>>>>>>>>>> The logs from pgpool should also help to diagnose
>>>>>>>>>>>>>>>> the problem.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> HTH
>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>>> Luca
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ---------- Forwarded message ---------
>>>>>>>>>>>>>>>> From: <pgpool-general-request at pgpool.net>
>>>>>>>>>>>>>>>> Date: Mon, Oct 25, 2021 at 5:00 AM
>>>>>>>>>>>>>>>> Subject: pgpool-general Digest, Vol 120, Issue 45
>>>>>>>>>>>>>>>> To: <pgpool-general at pgpool.net>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Send pgpool-general mailing list submissions to
>>>>>>>>>>>>>>>>         pgpool-general at pgpool.net
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> https://www.pgpool.net/mailman/listinfo/pgpool-general
>>>>>>>>>>>>>>>> or, via email, send a message with subject or body 'help' to
>>>>>>>>>>>>>>>>         pgpool-general-request at pgpool.net
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You can reach the person managing the list at
>>>>>>>>>>>>>>>>         pgpool-general-owner at pgpool.net
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> When replying, please edit your Subject line so it is more
>>>>>>>>>>>>>>>> specific
>>>>>>>>>>>>>>>> than "Re: Contents of pgpool-general digest..."
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Today's Topics:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>    1. [pgpool-general: 7822] Pgpool2 example with watchdog:
>>>>>>>>>>>>>>>> Unable
>>>>>>>>>>>>>>>>       to test (Nirav Bhatt)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ----------------------------------------------------------------------
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Message: 1
>>>>>>>>>>>>>>>> Date: Sun, 24 Oct 2021 20:18:28 +0300
>>>>>>>>>>>>>>>> From: Nirav Bhatt <niravbhatt.cpp at gmail.com>
>>>>>>>>>>>>>>>> To: pgpool-general at pgpool.net
>>>>>>>>>>>>>>>> Subject: [pgpool-general: 7822] Pgpool2 example with
>>>>>>>>>>>>>>>> watchdog: Unable
>>>>>>>>>>>>>>>>         to test
>>>>>>>>>>>>>>>> Message-ID:
>>>>>>>>>>>>>>>>         <CABz_77PMu9ksuYM2QGBaAuf=m_edGoKcidmWqC=K7Rpr=
>>>>>>>>>>>>>>>> J_ZGw at mail.gmail.com>
>>>>>>>>>>>>>>>> Content-Type: text/plain; charset="utf-8"
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> After doing all the config (config files, passwordless ssh,
>>>>>>>>>>>>>>>> firewall-cmd
>>>>>>>>>>>>>>>> etc) as given in the following link, I am unable to test my
>>>>>>>>>>>>>>>> setup. My OS is
>>>>>>>>>>>>>>>> Ubuntu 20.04.3 LTS.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> https://www.pgpool.net/docs/41/en/html/example-cluster.html#EXAMPLE-CLUSTER-STRUCTURE
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> All the port numbers are as listed in the example.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The part in example section: 8.3.7.1 and onwards is not
>>>>>>>>>>>>>>>> working.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> When I run on server1 (my primary):
>>>>>>>>>>>>>>>> psql -h 20.20.20.20 -p 9999 -U pgpool postgres -c "show
>>>>>>>>>>>>>>>> pool_nodes"
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I got error:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> psql: error: connection to server at "20.20.20.20", port
>>>>>>>>>>>>>>>> 9999 failed:
>>>>>>>>>>>>>>>> Connection refused
>>>>>>>>>>>>>>>> Is the server running on that host and accepting TCP/IP
>>>>>>>>>>>>>>>> connections?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I restarted postgresql and pgpool2 services time and again.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> pgpool2 service most recent logs:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> server1 (current primary):
>>>>>>>>>>>>>>>> Oct 24 17:01:41 ip-10-192-10-10 pgpool[104691]: 2021-10-24
>>>>>>>>>>>>>>>> 17:01:41: pid
>>>>>>>>>>>>>>>> 104691: LOG:  new watchdog node connection is received from
>>>>>>>>>>>>>>>> "<server1
>>>>>>>>>>>>>>>> host>:16553"
>>>>>>>>>>>>>>>> Oct 24 17:01:41 ip-10-192-10-10 pgpool[104691]: 2021-10-24
>>>>>>>>>>>>>>>> 17:01:41: pid
>>>>>>>>>>>>>>>> 104691: LOG:  new watchdog node connection is received from
>>>>>>>>>>>>>>>> "<server1
>>>>>>>>>>>>>>>> host>:17577"
>>>>>>>>>>>>>>>> Oct 24 17:04:02 ip-10-192-10-10 pgpool[104691]: 2021-10-24
>>>>>>>>>>>>>>>> 17:04:02: pid
>>>>>>>>>>>>>>>> 104691: LOG:  new watchdog node connection is received from
>>>>>>>>>>>>>>>> "<server2
>>>>>>>>>>>>>>>> host>:49304"
>>>>>>>>>>>>>>>> Oct 24 17:04:02 ip-10-192-10-10 pgpool[104691]: 2021-10-24
>>>>>>>>>>>>>>>> 17:04:02: pid
>>>>>>>>>>>>>>>> 104691: LOG:  new watchdog node connection is received from
>>>>>>>>>>>>>>>> "<server2
>>>>>>>>>>>>>>>> host>:50328"
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-10-10 pgpool[104691]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 104691: LOG:  new watchdog node connection is received from
>>>>>>>>>>>>>>>> "<server3
>>>>>>>>>>>>>>>> host>:51864"
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-10-10 pgpool[104691]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 104691: LOG:  new watchdog node connection is received from
>>>>>>>>>>>>>>>> "<server3
>>>>>>>>>>>>>>>> host>:52888"
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> server2 and server3 (looks like pgpool2 service is failing):
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125018: LOG:  setting the local watchdog node name to
>>>>>>>>>>>>>>>> "server3:5433 Linux
>>>>>>>>>>>>>>>> ip-<server 3 host>"
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125018: LOG:  watchdog cluster is configured with 2 remote
>>>>>>>>>>>>>>>> nodes
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125018: LOG:  watchdog remote node:0 on server1:9000
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125018: LOG:  watchdog remote node:1 on server2:9000
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125018: LOG:  interface monitoring is disabled in watchdog
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125018: LOG:  watchdog node state changed from [DEAD] to
>>>>>>>>>>>>>>>> [LOADING]
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125018: LOG:  new outbound connection to server1:9000
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125018: FATAL:  Add to watchdog cluster request is rejected
>>>>>>>>>>>>>>>> by node
>>>>>>>>>>>>>>>> "server1:9000"
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125018: HINT:  check the watchdog configurations.
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125018: LOG:  Watchdog is shutting down
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125014]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125014: LOG:  watchdog child process with pid: 125018 exits
>>>>>>>>>>>>>>>> with status 768
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125014]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125014: FATAL:  watchdog child process exit with fatal
>>>>>>>>>>>>>>>> error. exiting
>>>>>>>>>>>>>>>> pgpool-II
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125019: LOG:  setting the local watchdog node name to
>>>>>>>>>>>>>>>> "server3:5433 Linux
>>>>>>>>>>>>>>>> ip-<server 3 host>"
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125019: LOG:  watchdog cluster is configured with 2 remote
>>>>>>>>>>>>>>>> nodes
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125019: LOG:  watchdog remote node:0 on server1:9000
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125019: LOG:  watchdog remote node:1 on server2:9000
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125019: LOG:  interface monitoring is disabled in watchdog
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125019: LOG:  watchdog node state changed from [DEAD] to
>>>>>>>>>>>>>>>> [LOADING]
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125019: LOG:  new outbound connection to server1:9000
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]: 2021-10-24
>>>>>>>>>>>>>>>> 17:11:12: pid
>>>>>>>>>>>>>>>> 125019: LOG:  Watchdog is shutting down
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 systemd[1]:
>>>>>>>>>>>>>>>> pgpool2.service: Main process
>>>>>>>>>>>>>>>> exited, code=exited, status=3/NOTIMPLEMENTED
>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 systemd[1]:
>>>>>>>>>>>>>>>> pgpool2.service: Failed with
>>>>>>>>>>>>>>>> result 'exit-code'.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What steps am I missing?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thank you for the assistance,
>>>>>>>>>>>>>>>> -Nirav
>>>>>>>>>>>>>>>> -------------- next part --------------
>>>>>>>>>>>>>>>> An HTML attachment was scrubbed...
>>>>>>>>>>>>>>>> URL: <
>>>>>>>>>>>>>>>> http://www.pgpool.net/pipermail/pgpool-general/attachments/20211024/98ab6d68/attachment-0001.htm
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Subject: Digest Footer
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>> pgpool-general mailing list
>>>>>>>>>>>>>>>> pgpool-general at pgpool.net
>>>>>>>>>>>>>>>> http://www.pgpool.net/mailman/listinfo/pgpool-general
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> End of pgpool-general Digest, Vol 120, Issue 45
>>>>>>>>>>>>>>>> ***********************************************
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.pgpool.net/pipermail/pgpool-general/attachments/20211030/3033185d/attachment-0001.htm>

More information about the pgpool-general mailing list