View Issue Details

IDProjectCategoryView StatusLast Update
0000454Pgpool-IIBugpublic2019-02-18 14:31
Reporterjprieto Assigned Topengbo  
PriorityurgentSeverityfeatureReproducibilityalways
Status closedResolutionopen 
Product Version4.0.2 
Summary0000454: When multiple failover happens the pgpool pool node status is going into standby -stanby state
DescriptionI have 2 cluster nodes installed with pgpool-II(pgpool-II-4.0.2) and postgresql(postgresql PostgreSQL 9.4.20). The two nodes are configured as streaming replication mode. Lets name the two clusetr node as master-server(primary) and slave-server(secondary ).

Use case:

1. Failover from primary(master-server) to secondary(slave-server)
2. Failover from secondary(slave-server) to primary(master-server)

Thsi scenario leads the pgpool node status to standby-standby state
Steps To Reproduce1. Started postgers DB on primary node (master-server)
2. Started the pgpool on primary node (slave-server)
3. Once the delegate IP is up on the primary node (master-server), started the pgpool on the secondary node (slave-serve).
4. pgpool node status is follows

postgres=# show pool_nodes;
 node_id | hostname | port | status | lb_weight | role | select_cnt | load_balance_node | replication_delay | last_status_change
---------+---------------+------+--------+-----------+---------+------------+-------------------+-------------------+---------------------
 0 | master-server | 5432 | up | 0.500000 | primary | 0 | false | 0 | 2019-01-04 10:03:24
 1 | slave-server | 5432 | up | 0.500000 | standby | 0 | true | 0 | 2019-01-04 10:57:08
(2 filas)

5. I want to do the failover from master-server to slave-server. SO i brought down the postgres DB on the primary(vm1).Output follows

postgres=# show pool_nodes;
 node_id | hostname | port | status | lb_weight | role | select_cnt | load_balance_node | replication_delay | last_status_change
---------+---------------+------+--------+-----------+---------+------------+-------------------+-------------------+---------------------
 0 | master-server | 5432 | down | 0.500000 | standby | 1 | false | 0 | 2019-01-04 11:13:50
 1 | slave-server | 5432 | up | 0.500000 | primary | 1 | true | 0 | 2019-01-04 11:13:50
(2 filas)

The secondary(slave-server) node became the primary after the failover.

6. Then started the pcp recovery for primary node(master-server). Output follows

[root@slave-server data]# pcp_recovery_node -h 172.20.10.11 -p 9898 -U postgres -n 0 -d

postgres=# show pool_nodes;
 node_id | hostname | port | status | lb_weight | role | select_cnt | load_balance_node | replication_delay | last_status_change
---------+---------------+------+--------+-----------+---------+------------+-------------------+-------------------+---------------------
 0 | master-server | 5432 | up | 0.500000 | standby | 1 | true | 0 | 2019-01-04 11:23:35
 1 | slave-server | 5432 | up | 0.500000 | primary | 1 | false | 0 | 2019-01-04 11:13:50
(2 filas)

Prmary(master-server) node db is up now.

7. Again i want to do the failover from slave-server to master-server. SO i brought down the DB from slave-server. Output as follows

postgres=# show pool_nodes;
 node_id | hostname | port | status | lb_weight | role | select_cnt | load_balance_node | replication_delay | last_status_change
---------+---------------+------+--------+-----------+---------+------------+-------------------+-------------------+---------------------
 0 | master-server | 5432 | up | 0.500000 | standby | 2 | true | 0 | 2019-01-04 11:23:35
 1 | slave-server | 5432 | down | 0.500000 | standby | 1 | false | 0 | 2019-01-04 11:30:45
(2 filas)

 DOing this leads the pgpool node into standby-standby state.
TagsNo tags attached.

Activities

jprieto

2019-01-05 00:05

reporter  

pgpool.conf.slave-server (38,326 bytes)
pgpool.log.master-server (2,015,409 bytes)
pgpool.log.slave-server (1,385,347 bytes)

pengbo

2019-01-07 09:43

developer   ~0002313

It seems that Pgpool-II didn't find new primary node.
Could you share the failover.sh?

=====
2019-01-04 11:30:42: pid 6456: DEBUG: do_query: extended:0 query:"SELECT pg_is_in_recovery()"
2019-01-04 11:30:42: pid 6456: DEBUG: verify_backend_node_status: there's no primary node
2019-01-04 11:30:42: pid 6456: LOG: find_primary_node: standby node is 0
...
2019-01-04 11:30:45: pid 6456: LOG: failover: set new primary node: -1
2019-01-04 11:30:45: pid 6456: LOG: failover: set new master node: 0
======

administrator

2019-02-18 14:31

administrator   ~0002391

No response more than one month.
Close issue.

Issue History

Date Modified Username Field Change
2019-01-05 00:05 jprieto New Issue
2019-01-05 00:05 jprieto File Added: pgpool.conf.master-server
2019-01-05 00:05 jprieto File Added: pgpool.conf.slave-server
2019-01-05 00:05 jprieto File Added: pgpool.log.master-server
2019-01-05 00:05 jprieto File Added: pgpool.log.slave-server
2019-01-05 00:05 jprieto Tag Attached: logs
2019-01-05 00:07 jprieto Tag Detached: logs
2019-01-07 09:43 pengbo Note Added: 0002313
2019-01-09 10:04 pengbo Assigned To => pengbo
2019-01-09 10:04 pengbo Status new => feedback
2019-02-18 14:31 administrator Note Added: 0002391
2019-02-18 14:31 administrator Status feedback => closed