[pgpool-general: 5331] Re: pg_terminate_backend -> postmaster on DB node 1 was shutdown by administrative command

Muhammad Usama m.usama at gmail.com
Tue Feb 14 04:55:01 JST 2017


On Sun, Feb 12, 2017 at 2:56 PM, Avi Weinberg <AviW at gilat.com> wrote:

> Thank for the quick reply.
>
>
>
> Is there a way to bypass this unwanted failover problem or to minimize it
> in the current version?
>

This was basically a random behavior and unfortunately I can't think of a
simple way to reliably bypass the issue. But I have pushed the fix for and
you can try by building the Pgpool-II from source if you don't want to wait
until the next minor release which is expected around the end of current
month.

Thanks
Best regards
Muhammad Usama


>
> Thanks
>
> Avi
>
>
>
>
>
>
>
> *From:* Muhammad Usama [mailto:m.usama at gmail.com]
> *Sent:* Thursday, February 09, 2017 8:39 PM
> *To:* Avi Weinberg <AviW at gilat.com>
> *Cc:* pgpool-general at pgpool.net
> *Subject:* Re: [pgpool-general: 5315] pg_terminate_backend -> postmaster
> on DB node 1 was shutdown by administrative command
>
>
>
> Apparently their is an issue in Pgpool that is causing this problem. I am
> looking into this and will update you with the findings.
>
>
>
> Thanks
>
> Best regards
>
> Muhammad Usama
>
>
> On 09-Feb-2017, at 2:34 PM, Avi Weinberg <AviW at gilat.com> wrote:
>
> Hi all,
>
>
>
> With pgpool v 3.6.1 when we use the VIP and port 9999 we see that pgpool
> is setting the flag, but still performing failover.  What are we doing
> wrong?
>
>
>
> Thanks
>
> Avi
>
>
>
>
>
> The command :
>
> psql -h 172.18.255.40 -U postgres -p 9999 -t -c "select
> pg_terminate_backend(22755)"
>
>
>
>
>
> The log:
>
> : pid 25368: LOG:  child process with pid: 26073 exits with status 256
>
> 2017-02-09 05:31:01: pid 25368: LOG:  fork a new child process with pid:
> 27342
>
> 2017-02-09 05:31:11: pid 25368: LOG:  child process with pid: 26076 exits
> with status 256
>
> 2017-02-09 05:31:11: pid 25368: LOG:  fork a new child process with pid:
> 27357
>
> 2017-02-09 05:32:21: pid 25368: LOG:  child process with pid: 26077 exits
> with status 256
>
> 2017-02-09 05:32:21: pid 25368: LOG:  fork a new child process with pid:
> 27492
>
> 2017-02-09 05:32:42: pid 25368: LOG:  child process with pid: 26074 exits
> with status 256
>
> 2017-02-09 05:32:42: pid 25368: LOG:  fork a new child process with pid:
> 27522
>
> 2017-02-09 05:33:39: pid 27522: LOG:  found the pg_terminate_backend
> request for backend pid:22755 on backend node:1
>
> 2017-02-09 05:33:39: pid 27522: DETAIL:  setting the connection flag
>
> 2017-02-09 05:33:39: pid 26064: LOG:  reading and processing packets
>
> 2017-02-09 05:33:39: pid 26064: DETAIL:  postmaster on DB node 1 was
> shutdown by administrative command
>
> 2017-02-09 05:33:39: pid 26064: LOG:  received degenerate backend request
> for node_id: 1 from pid [26064]
>
> 2017-02-09 05:33:39: pid 26064: LOG:  received degenerate backend request
> for node_id: 1 from pid [26064]
>
> 2017-02-09 05:33:39: pid 25370: LOG:  new IPC connection received
>
> 2017-02-09 05:33:39: pid 25370: LOG:  watchdog received the failover
> command from local pgpool-II on IPC interface
>
> 2017-02-09 05:33:39: pid 25370: LOG:  watchdog is processing the failover
> command [DEGENERATE_BACKEND_REQUEST] received from local pgpool-II on IPC
> interface
>
> 2017-02-09 05:33:39: pid 25370: LOG:  forwarding the failover request
> [DEGENERATE_BACKEND_REQUEST] to all alive nodes
>
> 2017-02-09 05:33:39: pid 25370: DETAIL:  watchdog cluster currently has 1
> connected remote nodes
>
> 2017-02-09 05:33:39: pid 25370: LOG:  new IPC connection received
>
> 2017-02-09 05:33:39: pid 25370: LOG:  received the failover command lock
> request from local pgpool-II on IPC interface
>
> 2017-02-09 05:33:39: pid 25370: LOG:  local pgpool-II node "
> 172.18.255.41:9999 Linux h1-nms" is requesting to become a lock holder
> for failover ID: 88
>
> 2017-02-09 05:33:39: pid 25370: LOG:  local pgpool-II node "
> 172.18.255.41:9999 Linux h1-nms" is the lock holder
>
> 2017-02-09 05:33:39: pid 25368: LOG:  starting degeneration. shutdown
> host 172.18.255.42(5432)
>
> 2017-02-09 05:33:39: pid 25370: LOG:  received the failover command lock
> request from remote pgpool-II node "172.18.255.42:9999 Linux h2-nms"
>
> 2017-02-09 05:33:39: pid 25370: LOG:  remote pgpool-II node "
> 172.18.255.42:9999 Linux h2-nms" is requesting to become a lock holder
> for failover ID: 88
>
> 2017-02-09 05:33:39: pid 25370: LOG:  lock holder request denied to
> remote pgpool-II node "172.18.255.42:9999 Linux h2-nms"
>
> 2017-02-09 05:33:39: pid 25370: DETAIL:  local pgpool-II node "
> 172.18.255.41:9999 Linux h1-nms" is already holding the locks
>
> 2017-02-09 05:33:39: pid 25370: LOG:  received the failover command lock
> request from remote pgpool-II node "172.18.255.42:9999 Linux h2-nms"
>
> 2017-02-09 05:33:39: pid 25370: LOG:  remote pgpool-II node "
> 172.18.255.42:9999 Linux h2-nms" is requesting to become a lock holder
> for failover ID: 88
>
> 2017-02-09 05:33:39: pid 25370: LOG:  lock holder request denied to
> remote pgpool-II node "172.18.255.42:9999 Linux h2-nms"
>
> 2017-02-09 05:33:39: pid 25370: DETAIL:  local pgpool-II node "
> 172.18.255.41:9999 Linux h1-nms" is already holding the locks
>
> 2017-02-09 05:33:39: pid 25368: LOG:  Restart all children
>
> 2017-02-09 05:33:39: pid 26008: LOG:  child process received shutdown
> request signal 3
>
> 2017-02-09 05:33:39: pid 25368: LOG:  execute command:
> /etc/pgpool/failover.sh 1 172.18.255.42 5432 /postgres_db/data 0 0
> 172.18.255.41 1 5432 /postgres_db/data > AAA_failover_called.log
>
> 2017-02-09 05:33:39: pid 26011: LOG:  child process received shutdown
> request signal 3
>
> 2017-02-09 05:33:39: pid 26009: LOG:  child process received shutdown
> request signal 3
>
>
>
> IMPORTANT - This email and any attachments is intended for the above named
> addressee(s), and may contain information which is confidential or
> privileged. If you are not the intended recipient, please inform the sender
> immediately and delete this email: you should not copy or use this e-mail
> for any purpose nor disclose its contents to any person.
>
> _______________________________________________
> pgpool-general mailing list
> pgpool-general at pgpool.net
> http://www.pgpool.net/mailman/listinfo/pgpool-general
>
> IMPORTANT - This email and any attachments is intended for the above named
> addressee(s), and may contain information which is confidential or
> privileged. If you are not the intended recipient, please inform the sender
> immediately and delete this email: you should not copy or use this e-mail
> for any purpose nor disclose its contents to any person.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20170214/c9e4edeb/attachment-0001.html>


More information about the pgpool-general mailing list