[pgpool-general: 7055] Re: Rebooting the cluster?

[Anssi Kanninen]

Hi again,

Thank you for a quick answer.

Booting the services on one node is not a problem. But booting the whole 
cluster is. The booting sequence for all the nodes one at a time goes like 
this:

1st node starts up
==================

PgPool:
centos8-1v2-int 5432 9000 4 MASTER
centos8-2v2-int 5432 9000 0 DEAD
centos8-3v2-int 5432 9000 0 DEAD

Database:
  node_id |    hostname     | port |   status   |  role   | replication_state
---------+-----------------+------+------------+---------+-------------------
  0       | centos8-1v2-int | 5433 | up         | primary |
  1       | centos8-2v2-int | 5433 | quarantine | standby |
  2       | centos8-3v2-int | 5433 | quarantine | standby |

=> Delegate IP is not up yet because there is no quorum. This is ok.

2nd node starts up
==================

PgPool:
centos8-1v2-int 5432 9000 4 MASTER
centos8-2v2-int 5432 9000 7 STANDBY
centos8-3v2-int 5432 9000 0 DEAD

At first we are waiting...

  node_id |    hostname     | port | status  |  role   | replication_state
---------+-----------------+------+---------+---------+-------------------
  0       | centos8-1v2-int | 5433 | up      | primary |
  1       | centos8-2v2-int | 5433 | up      | standby |
  2       | centos8-3v2-int | 5433 | waiting | standby |

And then PgPool detaches the last node:

  node_id |    hostname     | port | status |  role   | replication_state
---------+-----------------+------+--------+---------+-------------------
  0       | centos8-1v2-int | 5433 | up     | primary |
  1       | centos8-2v2-int | 5433 | up     | standby | streaming
  2       | centos8-3v2-int | 5433 | down   | standby |

=> The PgPool service is up and delegate IP is also up.

3rd node starts up
==================

centos8-1v2-int 5432 9000 4 MASTER
centos8-2v2-int 5432 9000 7 STANDBY
centos8-3v2-int 5432 9000 7 STANDBY

  node_id |    hostname     | port | status |  role   | replication_state
---------+-----------------+------+--------+---------+-------------------
  0       | centos8-1v2-int | 5433 | up     | primary |
  1       | centos8-2v2-int | 5433 | up     | standby | streaming
  2       | centos8-3v2-int | 5433 | down   | standby |

So the 3rd node stays down. I've configured failback.sh so that it 
recreates the replication slot on primary DB on failback event, so 
pcp_attach_node is enough to bring the last node up:

  node_id |    hostname     | port | status |  role   | replication_state
---------+-----------------+------+--------+---------+-------------------
  0       | centos8-1v2-int | 5433 | up     | primary |
  1       | centos8-2v2-int | 5433 | up     | standby | streaming
  2       | centos8-3v2-int | 5433 | up     | standby | streaming

Otherwise my configuration is pretty much the same as the PgPool+Watchdog 
example in the PgPool web pages.

So, would it be possible to configure this so that just starting up all 
the nodes is enough to bring the whole cluster online? It may be so if I 
start them all at the same time but in production this is not always 
possible.

Cheers,
   - Anssi

On Fri, 29 May 2020, Bo Peng wrote:

> Hello,
>
> On Thu, 28 May 2020 13:18:29 +0300 (FLE Daylight Time)
> Anssi Kanninen <anssi at iki.fi> wrote:
>
>> Hi everyone and thanks for a great software!
>>
>> How do you guys boot your clusters?
>> It would be nice to have the whole 3-node cluster up cleanly (if shutdown
>> was clean) just be rebooting without any other user intervention. If I
>> just enable (with systemctl) all pgpool services, one node will always
>> left as "down" after rebooting.
>
> Pgpool should be started after starting PostgreSQL.
> Otherwise pgpool can not find that node.
>
> If you configure "systemctl anable ...", Pgpool may be started before PostgreSQL.
>
> If PostgreSQL and Pgpool are starting on the same node and PostgreSQL is started by systemd,
> to avoid this problem, you can configure "After=postgres.service" in pgpool.service.
>
>
>> Cheers!
>> - Anssi Kanninen, Helsinki Finland
>>
>> --
>> anssi at iki.fi
>> _______________________________________________
>> pgpool-general mailing list
>> pgpool-general at pgpool.net
>> http://www.pgpool.net/mailman/listinfo/pgpool-general
>
>
> -- 
> Bo Peng <pengbo at sraoss.co.jp>
> SRA OSS, Inc. Japan
> _______________________________________________
> pgpool-general mailing list
> pgpool-general at pgpool.net
> http://www.pgpool.net/mailman/listinfo/pgpool-general
>

-- 
anssi at iki.fi