View Issue Details

IDProjectCategoryView StatusLast Update
0000566Pgpool-II[All Projects] Generalpublic2020-02-03 10:54
Reportersureshreddy21Assigned Topengbo 
PrioritylowSeverityminorReproducibilityalways
Status feedbackResolutionopen 
PlatformLinuxOSOracle Linux Server release 8.1OS Version4.18.0-147.0.3
Product Version4.1.0 
Target VersionFixed in Version 
Summary0000566: Pgpool is not failing over compeltely to First node when 2nd and 3rd nodes were down
DescriptionWe have 3 node PGPOOL-II with watchdog and repmgr configured.
3rd node became the primrary during failovers.
after we bring up First node , it is alwys giving below error.

Dec 10 16:19:17 rac01-prod pgpool[27256]: 2019-12-10 16:19:17: pid 27256: ERROR: failed to make persistent db connection
Dec 10 16:19:17 rac01-prod pgpool[27256]: 2019-12-10 16:19:17: pid 27256: DETAIL: connection to host:"rac03-prod:5432" failed
Dec 10 16:19:17 rac01-prod pgpool[27256]: 2019-12-10 16:19:17: pid 27256: LOG: find_primary_node: make_persistent_db_connection_noerror failed on node 2
Dec 10 16:19:20 rac01-prod pgpool[27256]: 2019-12-10 16:19:20: pid 27256: LOG: failed to connect to PostgreSQL server on "rac03-prod:5432", getsockopt() detected error "No route to host"
TagsNo tags attached.

Activities

sureshreddy21

2019-12-11 06:21

reporter  

pgpool.conf (4,640 bytes)

pengbo

2019-12-12 18:05

developer   ~0003021

> after we bring up First node , it is alwys giving below error.

After you start the First node, did you attach this node to pgpool using "pcp_attach_node" command?

Could you show the result of "show pool_nodes" command?

$ psql -h localhost -p 9999 -U postgres -c "show pool_nodes"

sureshreddy21

2019-12-18 09:30

reporter   ~0003025

I am not attaching other standby nodes explicitly, I am thinking those should take care automatically by failover and follow_master script in /etc/pgpool-II folder and anoterh 2 scripts in $PGDATA like pgpool_remote_start and recovery_st_stage.

[postgres@rac01-prod ~]$ psql -h localhost -p 9999 -U postgres -c "show pool_nodes"
 node_id | hostname | port | status | lb_weight | role | select_cnt | load_balance_node | replication_delay | replication_state | replication_sync_state | last_stat
us_change
---------+------------+------+--------+-----------+---------+------------+-------------------+-------------------+-------------------+------------------------+----------
-----------
 0 | rac01-prod | 5432 | up | 0.500000 | primary | 0 | true | 0 | | | 2019-12-1
7 19:29:57
 1 | rac02-prod | 5432 | down | 0.500000 | standby | 0 | false | 0 | | | 2019-12-1
7 19:26:54
(2 rows)

pengbo

2019-12-24 14:19

developer   ~0003029

Last edited: 2019-12-24 14:19

View 2 revisions

If you run "pcp_recovery_node" to recover and start a DOWN node, the DOWN node will be attached to cluster automatically.
The "pgpool_remote_start" and "recovery_1st_stage" scripts are used by "pcp_recovery_node" command.

If you recover the DOWN node using the pg_basebackup command, etc., it is necessary to execute "pcp_attach_node"
to manually attach this node to the cluster.
https://www.pgpool.net/docs/latest/en/html/pcp-attach-node.html

pengbo

2019-12-25 10:41

developer   ~0003033

There are 3 nodes set in pgpool.conf.
But in the reset of "show pool_nodes", there are only 2 nodes.

Before failover, is the node3 (rac03-prod) started correctly?

pengbo

2020-02-03 10:54

developer   ~0003118

If you have resolved this issue, may I close this one?

Issue History

Date Modified Username Field Change
2019-12-11 06:21 sureshreddy21 New Issue
2019-12-11 06:21 sureshreddy21 File Added: pgpool.conf
2019-12-12 17:48 pengbo Assigned To => pengbo
2019-12-12 17:48 pengbo Status new => assigned
2019-12-12 17:48 pengbo Description Updated View Revisions
2019-12-12 18:05 pengbo Note Added: 0003021
2019-12-18 09:30 sureshreddy21 Note Added: 0003025
2019-12-24 14:19 pengbo Note Added: 0003029
2019-12-24 14:19 pengbo Note Edited: 0003029 View Revisions
2019-12-25 10:41 pengbo Note Added: 0003033
2020-01-20 12:30 pengbo Status assigned => feedback
2020-02-03 10:54 pengbo Note Added: 0003118