[pgpool-general: 7845] Re: pgpool-general Digest, Vol 120, Issue 45

Nirav Bhatt niravbhatt.cpp at gmail.com
Mon Nov 1 02:17:56 JST 2021 

The example has 9999 value.

https://www.pgpool.net/docs/41/en/html/example-cluster.html

The serverN is alias to servers in /etc/hosts file.

10.192.10.10 server1 <== I started this first, for it to be master
(master.log - not full, only part)
10.192.11.20 server2 <== Started 2nd, failed soon
10.192.12.30 server3 <== Started last, its log is attached as other.log

I tried debug5 (thanks for that suggestion), and here are my logs on master
(server1) and server3.

I saw that server3 tries connection attempt to server2 as well (which had
pgpool2 failed already) - so that connection refuse message should be
ignored.

-Nirav



On Sun, Oct 31, 2021 at 6:39 PM Luca Maranzano <liuk001 at gmail.com> wrote:

> I put the 3 conf file into kdiff, and I noticed this:
>
> pgpool port is set to 5433 (from default of 9999) but then here
>
> other_pgpool_port0 = 9999
>
> you are using a wrong port, it should be 5433
>
> Besides: you need to put real hostname in place of "serverN" or use dns/hosts alias.
>
> What are the hostname of the nodes and how is configured /etc/hosts?
>
> Cheers
>
>
>
> On Sun, Oct 31, 2021 at 5:24 PM Luca Maranzano <liuk001 at gmail.com> wrote:
>
>> Hi Nirav,
>> from server2.log:
>> 2021-10-31 14:15:54: pid 7785: WARNING:  pool key file
>> "/var/lib/postgresql/.pgpoolkey" has group or world acc>
>> you must set "chmod 600 /var/lib/postgresql/.pgpoolkey" on all servers
>>
>> Oct 31 14:15:54 ip-10-192-11-20 pgpool[7789]: 2021-10-31 14:15:54: pid
>> 7789: FATAL:  Add to watchdog cluster request is rejected by node
>> "server1:9000"
>> Oct 31 14:15:54 ip-10-192-11-20 pgpool[7789]: 2021-10-31 14:15:54: pid
>> 7789: HINT:  check the watchdog configurations.
>>
>> Log says that the watchdog request is rejected, so connection is OK but
>> it's surely a misconfiguration problem.
>>
>> Try to increase debug level and check logs again
>>
>> Cheers
>>
>>
>> On Sun, Oct 31, 2021 at 3:30 PM Nirav Bhatt <niravbhatt.cpp at gmail.com>
>> wrote:
>>
>>> Btw - I forgot to attach the 3 pgpool.conf files (4.1, so all 3 are
>>> different) I use on each server.
>>>
>>> I am also attaching pgpool2 service log on server1 (primary, for which
>>> pgpool-primary.conf is there) and that on server2 (secondary, for which
>>> pgpool-secondary.conf is there)
>>>
>>> Thanks
>>> -Nirav
>>>
>>> On Sun, Oct 31, 2021 at 12:39 AM Nirav Bhatt <niravbhatt.cpp at gmail.com>
>>> wrote:
>>>
>>>> pool_key problem was only observed in detached mode i.e. pgpool -n
>>>> option.
>>>> pgpool was unable to decrypt password key somehow.
>>>>
>>>> It is not happening with systemctl start pgpool2.
>>>> (I do not know what's the difference though, and whether process
>>>> invocation could provide any debug info)
>>>>
>>>> I can't see any error in service logs when I start using systemctl.
>>>> So I assume things are good from password standpoint.
>>>>
>>>> You are right - Routes in AWS enable routing to specific instance using
>>>> virtual ip (attached to interface via runtime script - up cmd in this case)
>>>>
>>>>
>>>> https://aws.amazon.com/blogs/apn/making-application-failover-seamless-by-failing-over-your-private-virtual-ip-across-availability-zones/
>>>>
>>>> Regarding pgpool2, it's a pity though such a well documented, highly
>>>> customized & fine grained tool can't operate after all the config, and it
>>>> is not obvious to find the reason from the service logs.
>>>> (I have been on this now since more than 1.5 months, but cannot get it
>>>> working - my limited knowledge of linux is also playing its part)
>>>>
>>>> To see if any of my server config was wrong, lastly I even switched
>>>> servers i.e started server2 process first (which became the master) and
>>>> started server1 process later - which failed just like the earlier case
>>>> while connecting to server2:9000
>>>>
>>>> Will continue tomorrow about it.
>>>>
>>>> Thanks all along,
>>>> -Nirav
>>>>
>>>> On Sat, Oct 30, 2021 at 11:51 PM Nirav Bhatt <niravbhatt.cpp at gmail.com>
>>>> wrote:
>>>>
>>>>> 3 subnets in 3 AWS Availability zones for redundancy. They still share
>>>>> the same network.
>>>>> This is a perfectly valid use case. (I have read about it earlier).
>>>>>
>>>>> There seems some pool_key problem on server1, which I am investigating
>>>>> now.
>>>>>
>>>>> -Nirav
>>>>>
>>>>>
>>>>> On Sat, Oct 30, 2021 at 11:43 PM Luca Maranzano <liuk001 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> How different? Are the 3 nodes on 3 separate networks? If yes why?
>>>>>>
>>>>>> Il sab 30 ott 2021, 22:40 Nirav Bhatt <niravbhatt.cpp at gmail.com> ha
>>>>>> scritto:
>>>>>>
>>>>>>> They are different, but everything including ssh copy and ping is
>>>>>>> working between them.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Sat, Oct 30, 2021 at 11:31 PM Luca Maranzano <liuk001 at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> What about the netmask of the network interface on 3 nodes?
>>>>>>>>
>>>>>>>> Il sab 30 ott 2021, 22:26 Nirav Bhatt <niravbhatt.cpp at gmail.com>
>>>>>>>> ha scritto:
>>>>>>>>
>>>>>>>>> Hi Luca
>>>>>>>>>
>>>>>>>>> Thanks for the response.
>>>>>>>>>
>>>>>>>>> Those logs belonged to the case when I put same pgpool.conf on all
>>>>>>>>> 3 servers, based on your suggestion.
>>>>>>>>> As I saw later, this should not be done for 4.1.
>>>>>>>>> So we should probably discard that case.
>>>>>>>>>
>>>>>>>>> With separate pgpool.conf file, it is logging the things I sent in
>>>>>>>>> earlier mail. (connection refused by server1:9000)
>>>>>>>>>
>>>>>>>>> Plus:
>>>>>>>>>
>>>>>>>>> server2:
>>>>>>>>> LOG:  setting the local watchdog node name to "server2:5433 Linux
>>>>>>>>> ip-10-192-11-20"
>>>>>>>>> LOG:  watchdog remote node:0 on server1:9000
>>>>>>>>> LOG:  watchdog remote node:1 on server3:9000
>>>>>>>>>
>>>>>>>>> server3:
>>>>>>>>> LOG:  setting the local watchdog node name to "server3:5433 Linux
>>>>>>>>> ip-10-192-12-30"
>>>>>>>>> LOG:  watchdog remote node:0 on server1:9000
>>>>>>>>> LOG:  watchdog remote node:1 on server2:9000
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -Nirav
>>>>>>>>>
>>>>>>>>> On Sat, Oct 30, 2021 at 11:01 PM Luca Maranzano <liuk001 at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Nirav,
>>>>>>>>>> you're right, the node id is only from 4.2, sorry for my
>>>>>>>>>> oversight, I thought it was valid for versione 4.X. It's really strange
>>>>>>>>>> that there are still no 4.2 debian packages available for Debian and
>>>>>>>>>> Ubuntu, 4.2 was released in 2020.
>>>>>>>>>>
>>>>>>>>>> I took a look at the logs, I found strange this:
>>>>>>>>>>
>>>>>>>>>> server1: LOG:  setting the local node "server1:5433 Linux
>>>>>>>>>> ip-10-192-10-10" as watchdog cluster master    (it seems ok)
>>>>>>>>>> server2: LOG:  setting the local watchdog node name to "*server1:5433
>>>>>>>>>> *Linux ip-10-192-11-20"
>>>>>>>>>> server3: LOG:  setting the local watchdog node name to "
>>>>>>>>>> *server1:5433* Linux ip-10-192-12-30"
>>>>>>>>>>
>>>>>>>>>> why server2 and server3 are referencing themselves as server1?
>>>>>>>>>>
>>>>>>>>>> Besides that you should set the VIP on the same subnet of the
>>>>>>>>>> servers, for example 10.192.11.100, not with an IP from a different subnet
>>>>>>>>>> (20.20.20.20)
>>>>>>>>>> I still have to understand how parameters must be differentiated
>>>>>>>>>> in 4.1 if you cannot use the node id file.
>>>>>>>>>>
>>>>>>>>>> I'll try to investigate later.
>>>>>>>>>> Cheers,
>>>>>>>>>> Luca
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sat, Oct 30, 2021 at 7:43 PM Nirav Bhatt <
>>>>>>>>>> niravbhatt.cpp at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> To reiterate, this pgpool2 4.1 setup.
>>>>>>>>>>>
>>>>>>>>>>> 1 - I have now restored my different pgpool.conf files on each
>>>>>>>>>>> server, as described in the link for 4.1:
>>>>>>>>>>> https://www.pgpool.net/docs/41/en/html/example-cluster.html
>>>>>>>>>>>
>>>>>>>>>>> They are now no longer identical.
>>>>>>>>>>>
>>>>>>>>>>> They only differ in
>>>>>>>>>>> - wd_hostname (this server),
>>>>>>>>>>> - heartbeat_destination0 & heartbeat_destination1 (other servers
>>>>>>>>>>> eg for server1, this is server2 and server3, and vice versa),
>>>>>>>>>>> - other_pgpool_hostname0 & other_pgpool_hostname1 (again, other
>>>>>>>>>>> servers)
>>>>>>>>>>>
>>>>>>>>>>> I also diffed other files under /etc/pgpool2 and postgres home,
>>>>>>>>>>> including password files, they are also same across the 3 servers.
>>>>>>>>>>>
>>>>>>>>>>> 2 - I restarted the servers just in case port change was not in
>>>>>>>>>>> the effect.
>>>>>>>>>>> I also tried telnet and I could see connection going from
>>>>>>>>>>> server2 & server3 to server1 on 9000.
>>>>>>>>>>> So port or connectivity isn't issue.
>>>>>>>>>>>
>>>>>>>>>>> 3 - The issue is: WD on server1 is refusing connection to
>>>>>>>>>>> server2 and server3 watchdogs on 9000.
>>>>>>>>>>>
>>>>>>>>>>> On server2 and server3 log:
>>>>>>>>>>>
>>>>>>>>>>> LOG:  watchdog node state changed from [DEAD] to [LOADING]
>>>>>>>>>>> LOG:  new outbound connection to server1:9000
>>>>>>>>>>> FATAL:  Add to watchdog cluster request is rejected by node
>>>>>>>>>>> "server1:9000"
>>>>>>>>>>> HINT:  check the watchdog configurations.
>>>>>>>>>>> LOG:  Watchdog is shutting down
>>>>>>>>>>>
>>>>>>>>>>> while server1 log:
>>>>>>>>>>>
>>>>>>>>>>> LOG:  new watchdog node connection is received from "
>>>>>>>>>>> 10.192.11.20:23277" <-- server2
>>>>>>>>>>> LOG:  new watchdog node connection is received from "
>>>>>>>>>>> 10.192.12.30:16596" <-- server3
>>>>>>>>>>>
>>>>>>>>>>> (nothing else about why it refused connection - just this thing
>>>>>>>>>>> above)
>>>>>>>>>>>
>>>>>>>>>>> I believe that's enough debug info, if you could spot any
>>>>>>>>>>> problems, please let me know.
>>>>>>>>>>>
>>>>>>>>>>> Thanks for the assistance so far,
>>>>>>>>>>> -Nirav
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Sat, Oct 30, 2021 at 7:24 PM Nirav Bhatt <
>>>>>>>>>>> niravbhatt.cpp at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I also suspect if following line is true for 4.1 setup:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> *From Pgpool-II 4.2, now all configuration parameters are
>>>>>>>>>>>> identical on all hosts*
>>>>>>>>>>>>
>>>>>>>>>>>> -Nirav
>>>>>>>>>>>>
>>>>>>>>>>>> On Sat, Oct 30, 2021 at 7:22 PM Nirav Bhatt <
>>>>>>>>>>>> niravbhatt.cpp at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am following this link, which is pgpool2 v4.1.
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://www.pgpool.net/docs/41/en/html/example-cluster.html
>>>>>>>>>>>>>
>>>>>>>>>>>>> On my Ubuntu setup also that is the latest I can get:
>>>>>>>>>>>>> *4.1.4-3.pgdg20.04+2*
>>>>>>>>>>>>>
>>>>>>>>>>>>> If you still insist, I will try to set it up, but I guess you
>>>>>>>>>>>>> are quoting from this link:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://www.pgpool.net/docs/latest/en/html/example-cluster.html#EXAMPLE-CLUSTER-PGPOOL-NODE-ID
>>>>>>>>>>>>>
>>>>>>>>>>>>> It is a more recent version of pgpool2 than what I have.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Nirav
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sat, Oct 30, 2021 at 7:03 PM Luca Maranzano <
>>>>>>>>>>>>> liuk001 at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>> I think that the pgpool_node_id file is mandatory since it is
>>>>>>>>>>>>>> used by every server to identify itself in the cluster with its node id.
>>>>>>>>>>>>>> Where is it documented that it is not necessary?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Try to configure it
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> HTH
>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>> Luca
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sat, Oct 30, 2021 at 5:42 PM Nirav Bhatt <
>>>>>>>>>>>>>> niravbhatt.cpp at gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello
>>>>>>>>>>>>>>> Sorry for the late answer.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I now made pgpool.conf identical on all 3 servers, then
>>>>>>>>>>>>>>> restarted postgresql and pgpool2 services.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I opened ports on each server: 22, 9999/tcp, 9898/tcp,
>>>>>>>>>>>>>>> 9000/tcp, 9694/udp, 5433, 5432
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yet the problem persists on server2 and server3 - it can not
>>>>>>>>>>>>>>> connect to 9000 port (on themselves)
>>>>>>>>>>>>>>> Server1 pgpool seems to be running without problem.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I am attaching my pgpool.conf file. Also, I am attaching
>>>>>>>>>>>>>>> pgpool2 service logs from all 3 servers.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 10.192.10.10 server1
>>>>>>>>>>>>>>> 10.192.11.20 server2
>>>>>>>>>>>>>>> 10.192.12.30 server3
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Virtual IP is 20.20.20.20
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I am running the following pgpool2 version so I do not think
>>>>>>>>>>>>>>> I need pgpool_node_id file.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 4.1.4-3.pgdg20.04+2
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks for the support,
>>>>>>>>>>>>>>> -Nirav
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Oct 27, 2021 at 8:48 AM Luca Maranzano <
>>>>>>>>>>>>>>> liuk001 at gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi, still not clear to me, from the doc:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 8.2.5. Create pgpool_node_id
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> From Pgpool-II 4.2, now all configuration parameters are
>>>>>>>>>>>>>>>> identical on all hosts. If watchdog feature is enabled, to
>>>>>>>>>>>>>>>> distingusish which host is which, a pgpool_node_id file is
>>>>>>>>>>>>>>>> required. You need to create a pgpool_node_id file and
>>>>>>>>>>>>>>>> specify the pgpool (watchdog) node number (e.g. 0, 1, 2 ...) to identify
>>>>>>>>>>>>>>>> pgpool (watchdog) host.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> In our systems the pgpool.conf are *identical *
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Oct 26, 2021 at 8:09 PM Nirav Bhatt <
>>>>>>>>>>>>>>>> niravbhatt.cpp at gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The difference lies in “other” server host names.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> -Nirav
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, 26 Oct 2021 at 21.07, Luca Maranzano <
>>>>>>>>>>>>>>>>> liuk001 at gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> AFAIK the pgpool.conf file must be identical on all
>>>>>>>>>>>>>>>>>> cluster nodes.
>>>>>>>>>>>>>>>>>> Which are the differences in detail?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>> Luca
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Oct 25, 2021 at 9:46 PM Nirav Bhatt <
>>>>>>>>>>>>>>>>>> niravbhatt.cpp at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I added all ports via ufw on server1:
>>>>>>>>>>>>>>>>>>> 22,9999,9898,9000,9694,5433,5432
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I enabled firewall past that point.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I diffed all conf files across servers - all files are
>>>>>>>>>>>>>>>>>>> same, including hba and .pgpass and .pcppass.
>>>>>>>>>>>>>>>>>>> Only pgpool.conf files differ, that too in parameters
>>>>>>>>>>>>>>>>>>> like server hosts for pgpool and watchdog, no other differences.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> No error is observed on primary server1. The last log
>>>>>>>>>>>>>>>>>>> lines on server1 says: new watchdog node connection is received from
>>>>>>>>>>>>>>>>>>> <server2> or <server3>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> But secondary server pgpool2 keeps exiting with:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Add to watchdog cluster request is rejected by node
>>>>>>>>>>>>>>>>>>> "server1:9000"
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I somehow suspect it has to do with wd_authkey which I
>>>>>>>>>>>>>>>>>>> leave empty:
>>>>>>>>>>>>>>>>>>> https://www.sraoss.jp/pipermail/pgpool-hackers/2017-August/002497.html
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> But it is an old issue, so I can't say for sure.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> -Nirav
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Oct 25, 2021 at 8:07 PM Luca Maranzano <
>>>>>>>>>>>>>>>>>>> liuk001 at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Firewall-cmd has to be used on Redhat derived flavours,
>>>>>>>>>>>>>>>>>>>> on Ubuntu standard is ufw, so "ufw disable" should open all ports
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Mon, Oct 25, 2021 at 6:59 PM Nirav Bhatt <
>>>>>>>>>>>>>>>>>>>> niravbhatt.cpp at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I ran sudo ufw status and firewall is already
>>>>>>>>>>>>>>>>>>>>> inactive.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Not sure what firewall-cmd did for all the pgpool
>>>>>>>>>>>>>>>>>>>>> ports. (they are showing as list of open ports)
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Should add them all via ufw also?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> -Nirav
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Mon, Oct 25, 2021 at 7:46 PM Luca Maranzano <
>>>>>>>>>>>>>>>>>>>>> liuk001 at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Try to disable firewall to see it it's related, all
>>>>>>>>>>>>>>>>>>>>>> the pgpool configuration files are identical on the 3 nodes?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Mon, Oct 25, 2021 at 6:41 PM Nirav Bhatt <
>>>>>>>>>>>>>>>>>>>>>> niravbhatt.cpp at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks for the response.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I restarted pgpool2 service on all 3 nodes (with
>>>>>>>>>>>>>>>>>>>>>>> some delay, and in the order: primary, secondary1, secondary2).
>>>>>>>>>>>>>>>>>>>>>>> My primary pgpool2 service logs do show
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> LOG:  new watchdog node connection is received from
>>>>>>>>>>>>>>>>>>>>>>> <secondary host>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> But my secondary pgpool2 service logs (among other
>>>>>>>>>>>>>>>>>>>>>>> normal messages) show:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> FATAL:  Add to watchdog cluster request is rejected
>>>>>>>>>>>>>>>>>>>>>>> by node "server1:9000"
>>>>>>>>>>>>>>>>>>>>>>> FATAL:  watchdog child process exit with fatal
>>>>>>>>>>>>>>>>>>>>>>> error. exiting pgpool-II
>>>>>>>>>>>>>>>>>>>>>>> LOG:  watchdog node state changed from [DEAD] to
>>>>>>>>>>>>>>>>>>>>>>> [LOADING]
>>>>>>>>>>>>>>>>>>>>>>> LOG:  new outbound connection to server1:9000
>>>>>>>>>>>>>>>>>>>>>>> FATAL:  Add to watchdog cluster request is rejected
>>>>>>>>>>>>>>>>>>>>>>> by node "server1:9000"
>>>>>>>>>>>>>>>>>>>>>>> HINT:  check the watchdog configurations.
>>>>>>>>>>>>>>>>>>>>>>> LOG:  Watchdog is shutting down
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I checked that server1 has 9000 port open, along
>>>>>>>>>>>>>>>>>>>>>>> with 9999, 9898 and 9694:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> > firewall-cmd --list-ports
>>>>>>>>>>>>>>>>>>>>>>> 9999/tcp 9898/tcp 9000/tcp 9694/udp
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> What could be going on?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> -Nirav
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Mon, Oct 25, 2021 at 7:13 PM Luca Maranzano <
>>>>>>>>>>>>>>>>>>>>>>> liuk001 at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> if you cannot connect to 20.20.20.20 -p 9999 it
>>>>>>>>>>>>>>>>>>>>>>>> means that hte PGPool cluster is not well formed and that the VIP has not
>>>>>>>>>>>>>>>>>>>>>>>> been enabled.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> You can check that with pcp_watchdog_info.
>>>>>>>>>>>>>>>>>>>>>>>> The logs from pgpool should also help to diagnose
>>>>>>>>>>>>>>>>>>>>>>>> the problem.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> HTH
>>>>>>>>>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>>>>>>>>>>> Luca
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> ---------- Forwarded message ---------
>>>>>>>>>>>>>>>>>>>>>>>> From: <pgpool-general-request at pgpool.net>
>>>>>>>>>>>>>>>>>>>>>>>> Date: Mon, Oct 25, 2021 at 5:00 AM
>>>>>>>>>>>>>>>>>>>>>>>> Subject: pgpool-general Digest, Vol 120, Issue 45
>>>>>>>>>>>>>>>>>>>>>>>> To: <pgpool-general at pgpool.net>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Send pgpool-general mailing list submissions to
>>>>>>>>>>>>>>>>>>>>>>>>         pgpool-general at pgpool.net
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> To subscribe or unsubscribe via the World Wide Web,
>>>>>>>>>>>>>>>>>>>>>>>> visit
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> https://www.pgpool.net/mailman/listinfo/pgpool-general
>>>>>>>>>>>>>>>>>>>>>>>> or, via email, send a message with subject or body
>>>>>>>>>>>>>>>>>>>>>>>> 'help' to
>>>>>>>>>>>>>>>>>>>>>>>>         pgpool-general-request at pgpool.net
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> You can reach the person managing the list at
>>>>>>>>>>>>>>>>>>>>>>>>         pgpool-general-owner at pgpool.net
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> When replying, please edit your Subject line so it
>>>>>>>>>>>>>>>>>>>>>>>> is more specific
>>>>>>>>>>>>>>>>>>>>>>>> than "Re: Contents of pgpool-general digest..."
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Today's Topics:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>    1. [pgpool-general: 7822] Pgpool2 example with
>>>>>>>>>>>>>>>>>>>>>>>> watchdog: Unable
>>>>>>>>>>>>>>>>>>>>>>>>       to test (Nirav Bhatt)
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> ----------------------------------------------------------------------
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Message: 1
>>>>>>>>>>>>>>>>>>>>>>>> Date: Sun, 24 Oct 2021 20:18:28 +0300
>>>>>>>>>>>>>>>>>>>>>>>> From: Nirav Bhatt <niravbhatt.cpp at gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>> To: pgpool-general at pgpool.net
>>>>>>>>>>>>>>>>>>>>>>>> Subject: [pgpool-general: 7822] Pgpool2 example
>>>>>>>>>>>>>>>>>>>>>>>> with watchdog: Unable
>>>>>>>>>>>>>>>>>>>>>>>>         to test
>>>>>>>>>>>>>>>>>>>>>>>> Message-ID:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> <CABz_77PMu9ksuYM2QGBaAuf=m_edGoKcidmWqC=K7Rpr=
>>>>>>>>>>>>>>>>>>>>>>>> J_ZGw at mail.gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>> Content-Type: text/plain; charset="utf-8"
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> After doing all the config (config files,
>>>>>>>>>>>>>>>>>>>>>>>> passwordless ssh, firewall-cmd
>>>>>>>>>>>>>>>>>>>>>>>> etc) as given in the following link, I am unable to
>>>>>>>>>>>>>>>>>>>>>>>> test my setup. My OS is
>>>>>>>>>>>>>>>>>>>>>>>> Ubuntu 20.04.3 LTS.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> https://www.pgpool.net/docs/41/en/html/example-cluster.html#EXAMPLE-CLUSTER-STRUCTURE
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> All the port numbers are as listed in the example.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The part in example section: 8.3.7.1 and onwards is
>>>>>>>>>>>>>>>>>>>>>>>> not working.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> When I run on server1 (my primary):
>>>>>>>>>>>>>>>>>>>>>>>> psql -h 20.20.20.20 -p 9999 -U pgpool postgres -c
>>>>>>>>>>>>>>>>>>>>>>>> "show pool_nodes"
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I got error:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> psql: error: connection to server at "20.20.20.20",
>>>>>>>>>>>>>>>>>>>>>>>> port 9999 failed:
>>>>>>>>>>>>>>>>>>>>>>>> Connection refused
>>>>>>>>>>>>>>>>>>>>>>>> Is the server running on that host and accepting
>>>>>>>>>>>>>>>>>>>>>>>> TCP/IP connections?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I restarted postgresql and pgpool2 services time
>>>>>>>>>>>>>>>>>>>>>>>> and again.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> pgpool2 service most recent logs:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> server1 (current primary):
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:01:41 ip-10-192-10-10 pgpool[104691]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:01:41: pid
>>>>>>>>>>>>>>>>>>>>>>>> 104691: LOG:  new watchdog node connection is
>>>>>>>>>>>>>>>>>>>>>>>> received from "<server1
>>>>>>>>>>>>>>>>>>>>>>>> host>:16553"
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:01:41 ip-10-192-10-10 pgpool[104691]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:01:41: pid
>>>>>>>>>>>>>>>>>>>>>>>> 104691: LOG:  new watchdog node connection is
>>>>>>>>>>>>>>>>>>>>>>>> received from "<server1
>>>>>>>>>>>>>>>>>>>>>>>> host>:17577"
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:04:02 ip-10-192-10-10 pgpool[104691]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:04:02: pid
>>>>>>>>>>>>>>>>>>>>>>>> 104691: LOG:  new watchdog node connection is
>>>>>>>>>>>>>>>>>>>>>>>> received from "<server2
>>>>>>>>>>>>>>>>>>>>>>>> host>:49304"
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:04:02 ip-10-192-10-10 pgpool[104691]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:04:02: pid
>>>>>>>>>>>>>>>>>>>>>>>> 104691: LOG:  new watchdog node connection is
>>>>>>>>>>>>>>>>>>>>>>>> received from "<server2
>>>>>>>>>>>>>>>>>>>>>>>> host>:50328"
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-10-10 pgpool[104691]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 104691: LOG:  new watchdog node connection is
>>>>>>>>>>>>>>>>>>>>>>>> received from "<server3
>>>>>>>>>>>>>>>>>>>>>>>> host>:51864"
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-10-10 pgpool[104691]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 104691: LOG:  new watchdog node connection is
>>>>>>>>>>>>>>>>>>>>>>>> received from "<server3
>>>>>>>>>>>>>>>>>>>>>>>> host>:52888"
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> server2 and server3 (looks like pgpool2 service is
>>>>>>>>>>>>>>>>>>>>>>>> failing):
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125018: LOG:  setting the local watchdog node name
>>>>>>>>>>>>>>>>>>>>>>>> to "server3:5433 Linux
>>>>>>>>>>>>>>>>>>>>>>>> ip-<server 3 host>"
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125018: LOG:  watchdog cluster is configured with 2
>>>>>>>>>>>>>>>>>>>>>>>> remote nodes
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125018: LOG:  watchdog remote node:0 on server1:9000
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125018: LOG:  watchdog remote node:1 on server2:9000
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125018: LOG:  interface monitoring is disabled in
>>>>>>>>>>>>>>>>>>>>>>>> watchdog
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125018: LOG:  watchdog node state changed from
>>>>>>>>>>>>>>>>>>>>>>>> [DEAD] to [LOADING]
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125018: LOG:  new outbound connection to
>>>>>>>>>>>>>>>>>>>>>>>> server1:9000
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125018: FATAL:  Add to watchdog cluster request is
>>>>>>>>>>>>>>>>>>>>>>>> rejected by node
>>>>>>>>>>>>>>>>>>>>>>>> "server1:9000"
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125018: HINT:  check the watchdog configurations.
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125018]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125018: LOG:  Watchdog is shutting down
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125014]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125014: LOG:  watchdog child process with pid:
>>>>>>>>>>>>>>>>>>>>>>>> 125018 exits with status 768
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125014]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125014: FATAL:  watchdog child process exit with
>>>>>>>>>>>>>>>>>>>>>>>> fatal error. exiting
>>>>>>>>>>>>>>>>>>>>>>>> pgpool-II
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125019: LOG:  setting the local watchdog node name
>>>>>>>>>>>>>>>>>>>>>>>> to "server3:5433 Linux
>>>>>>>>>>>>>>>>>>>>>>>> ip-<server 3 host>"
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125019: LOG:  watchdog cluster is configured with 2
>>>>>>>>>>>>>>>>>>>>>>>> remote nodes
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125019: LOG:  watchdog remote node:0 on server1:9000
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125019: LOG:  watchdog remote node:1 on server2:9000
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125019: LOG:  interface monitoring is disabled in
>>>>>>>>>>>>>>>>>>>>>>>> watchdog
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125019: LOG:  watchdog node state changed from
>>>>>>>>>>>>>>>>>>>>>>>> [DEAD] to [LOADING]
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125019: LOG:  new outbound connection to
>>>>>>>>>>>>>>>>>>>>>>>> server1:9000
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 pgpool[125019]:
>>>>>>>>>>>>>>>>>>>>>>>> 2021-10-24 17:11:12: pid
>>>>>>>>>>>>>>>>>>>>>>>> 125019: LOG:  Watchdog is shutting down
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 systemd[1]:
>>>>>>>>>>>>>>>>>>>>>>>> pgpool2.service: Main process
>>>>>>>>>>>>>>>>>>>>>>>> exited, code=exited, status=3/NOTIMPLEMENTED
>>>>>>>>>>>>>>>>>>>>>>>> Oct 24 17:11:12 ip-10-192-12-30 systemd[1]:
>>>>>>>>>>>>>>>>>>>>>>>> pgpool2.service: Failed with
>>>>>>>>>>>>>>>>>>>>>>>> result 'exit-code'.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> What steps am I missing?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thank you for the assistance,
>>>>>>>>>>>>>>>>>>>>>>>> -Nirav
>>>>>>>>>>>>>>>>>>>>>>>> -------------- next part --------------
>>>>>>>>>>>>>>>>>>>>>>>> An HTML attachment was scrubbed...
>>>>>>>>>>>>>>>>>>>>>>>> URL: <
>>>>>>>>>>>>>>>>>>>>>>>> http://www.pgpool.net/pipermail/pgpool-general/attachments/20211024/98ab6d68/attachment-0001.htm
>>>>>>>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Subject: Digest Footer
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>>>>>>>> pgpool-general mailing list
>>>>>>>>>>>>>>>>>>>>>>>> pgpool-general at pgpool.net
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> http://www.pgpool.net/mailman/listinfo/pgpool-general
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> End of pgpool-general Digest, Vol 120, Issue 45
>>>>>>>>>>>>>>>>>>>>>>>> ***********************************************
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.pgpool.net/pipermail/pgpool-general/attachments/20211031/e4e1980e/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Increased_log_level.zip
Type: application/zip
Size: 2991 bytes
Desc: not available
URL: <http://www.pgpool.net/pipermail/pgpool-general/attachments/20211031/e4e1980e/attachment-0001.zip>

More information about the pgpool-general mailing list