View Issue Details

IDProjectCategoryView StatusLast Update
0000649Pgpool-IIBugpublic2020-12-09 17:05
Reporterwhitebbai Assigned Topengbo  
PriorityimmediateSeveritymajorReproducibilityalways
Status feedbackResolutionopen 
Product Version4.1.2 
Summary0000649: Add to watchdog cluster request is rejected by node "master:9000"
Description2020-09-12 11:48:33: pid 4407: DEBUG: initializing pool configuration
2020-09-12 11:48:33: pid 4407: DETAIL: num_backends: 2 total_weight: 2.000000
2020-09-12 11:48:33: pid 4407: DEBUG: initializing pool configuration
2020-09-12 11:48:33: pid 4407: DETAIL: backend 0 weight: 1073741823.500000 flag: 0000
2020-09-12 11:48:33: pid 4407: DEBUG: initializing pool configuration
2020-09-12 11:48:33: pid 4407: DETAIL: backend 1 weight: 1073741823.500000 flag: 0000
2020-09-12 11:48:33: pid 4407: LOG: Backend status file /var/log/pgpool/pgpool_status discarded
2020-09-12 11:48:33: pid 4407: DEBUG: pool_coninfo_size: num_init_children (50) * max_pool (4) * MAX_NUM_BACKENDS (128) * sizeof(ConnectionInfo) (136) = 3481600 bytes requested for shared memory
2020-09-12 11:48:33: pid 4407: DEBUG: ProcessInfo: num_init_children (50) * sizeof(ProcessInfo) (32) = 1600 bytes requested for shared memory
2020-09-12 11:48:33: pid 4407: DEBUG: Request info are: sizeof(POOL_REQUEST_INFO) 5264 bytes requested for shared memory
2020-09-12 11:48:33: pid 4407: DEBUG: Recovery management area: sizeof(int) 4 bytes requested for shared memory
2020-09-12 11:48:33: pid 4407: LOG: memory cache initialized
2020-09-12 11:48:33: pid 4407: DETAIL: memcache blocks :64
2020-09-12 11:48:33: pid 4407: DEBUG: memory cache request size : 67108864
2020-09-12 11:48:33: pid 4407: LOG: pool_discard_oid_maps: discarded memqcache oid maps
2020-09-12 11:48:33: pid 4407: LOG: waiting for watchdog to initialize
2020-09-12 11:48:33: pid 4409: LOG: setting the local watchdog node name to "salve:9999 Linux bogon"
2020-09-12 11:48:33: pid 4409: LOG: watchdog cluster is configured with 1 remote nodes
2020-09-12 11:48:33: pid 4409: LOG: watchdog remote node:0 on master:9000
2020-09-12 11:48:33: pid 4409: LOG: interface monitoring is disabled in watchdog
2020-09-12 11:48:33: pid 4409: INFO: IPC socket path: "/var/run/postgresql/.s.PGPOOLWD_CMD.9000"
2020-09-12 11:48:33: pid 4409: LOG: watchdog node state changed from [DEAD] to [LOADING]
2020-09-12 11:48:33: pid 4409: DEBUG: STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State = LOADING
2020-09-12 11:48:33: pid 4409: LOG: new outbound connection to master:9000
2020-09-12 11:48:33: pid 4409: DEBUG: STATE MACHINE INVOKED WITH EVENT = NEW OUTBOUND_CONNECTION Current State = LOADING
2020-09-12 11:48:33: pid 4409: DEBUG: sending packet, watchdog node:[] command id:[4] type:[ADD NODE] state:[LOADING]
2020-09-12 11:48:33: pid 4409: DEBUG: sending watchdog packet to socket:7, type:[A], command ID:4, data Length:311
2020-09-12 11:48:33: pid 4409: DEBUG: received watchdog packet type:R
2020-09-12 11:48:33: pid 4409: DEBUG: reading packet type R of length 0
2020-09-12 11:48:33: pid 4409: DEBUG: STATE MACHINE INVOKED WITH EVENT = PACKET RECEIVED Current State = LOADING
2020-09-12 11:48:33: pid 4409: DEBUG: received packet, watchdog node:[] command id:[4] type:[REJECT] state:[LOADING]
2020-09-12 11:48:33: pid 4409: FATAL: Add to watchdog cluster request is rejected by node "master:9000"
2020-09-12 11:48:33: pid 4409: HINT: check the watchdog configurations.
2020-09-12 11:48:33: pid 4409: LOG: Watchdog is shutting down
2020-09-12 11:48:33: pid 4407: DEBUG: reaper handler
2020-09-12 11:48:33: pid 4407: DEBUG: watchdog child process with pid: 4409 exit with FATAL ERROR. pgpool-II will be shutdown
2020-09-12 11:48:33: pid 4407: LOG: watchdog child process with pid: 4409 exits with status 768
2020-09-12 11:48:33: pid 4407: FATAL: watchdog child process exit with fatal error. exiting pgpool-II
2020-09-12 11:48:33: pid 4410: LOG: setting the local watchdog node name to "salve:9999 Linux bogon"
2020-09-12 11:48:33: pid 4410: LOG: watchdog cluster is configured with 1 remote nodes
2020-09-12 11:48:33: pid 4410: LOG: watchdog remote node:0 on master:9000
2020-09-12 11:48:33: pid 4410: LOG: interface monitoring is disabled in watchdog
2020-09-12 11:48:33: pid 4410: INFO: IPC socket path: "/var/run/postgresql/.s.PGPOOLWD_CMD.9000"
2020-09-12 11:48:33: pid 4410: LOG: watchdog node state changed from [DEAD] to [LOADING]
2020-09-12 11:48:33: pid 4410: DEBUG: STATE MACHINE INVOKED WITH EVENT = STATE CHANGED Current State = LOADING
2020-09-12 11:48:33: pid 4410: LOG: new outbound connection to master:9000
2020-09-12 11:48:33: pid 4410: DEBUG: STATE MACHINE INVOKED WITH EVENT = NEW OUTBOUND_CONNECTION Current State = LOADING
2020-09-12 11:48:33: pid 4410: DEBUG: sending packet, watchdog node:[] command id:[4] type:[ADD NODE] state:[LOADING]
2020-09-12 11:48:33: pid 4410: DEBUG: sending watchdog packet to socket:7, type:[A], command ID:4, data Length:311
2020-09-12 11:48:33: pid 4410: DEBUG: received watchdog packet type:R
2020-09-12 11:48:33: pid 4410: DEBUG: reading packet type R of length 0
2020-09-12 11:48:33: pid 4410: DEBUG: STATE MACHINE INVOKED WITH EVENT = PACKET RECEIVED Current State = LOADING
2020-09-12 11:48:33: pid 4410: DEBUG: received packet, watchdog node:[] command id:[4] type:[REJECT] state:[LOADING]
2020-09-12 11:48:33: pid 4410: FATAL: Add to watchdog cluster request is rejected by node "master:9000"
2020-09-12 11:48:33: pid 4410: HINT: check the watchdog configurations.
2020-09-12 11:48:33: pid 4410: LOG: Watchdog is shutting down
Steps To Reproducestart pgpool on host 1
start pgpool on host2

pgpool on host2 is shutdown
Additional InformationI am using postgres 12.4
repmgr 5.0
pgpool 4.1

master and slave in the host configuration, and they can be connected, the master-slave backup is already implemented
TagsNo tags attached.

Activities

whitebbai

2020-09-12 17:05

reporter  

pgpool-slave.conf (43,558 bytes)
pgpool- -Master.conf (43,557 bytes)

pengbo

2020-09-14 14:07

developer   ~0003562

I found there is a typo in "pgpool-slave.conf".

     wd_hostname = 'salve'

I think it should be wd_hostname = 'slave'.

whitebbai

2020-09-15 12:58

reporter   ~0003568

thank you,I apologize for my carelessness. Now the guard dog can be activated,However, when I stopped the primary database, the backup database would not switch automatically. I'm still connecting to master's database.
failover_stream.sh (266 bytes)   
failover_stream.sh (266 bytes)   
pgpool-master.log (151,850 bytes)
pgpool-slave.log (95,108 bytes)

pengbo

2020-12-09 17:05

developer   ~0003649

Sorry for late response.
Have you resolved this issue?

Issue History

Date Modified Username Field Change
2020-09-12 17:05 whitebbai New Issue
2020-09-12 17:05 whitebbai File Added: pgpool-slave.conf
2020-09-12 17:05 whitebbai File Added: pgpool- -Master.conf
2020-09-14 09:04 pengbo Assigned To => pengbo
2020-09-14 09:04 pengbo Status new => assigned
2020-09-14 09:04 pengbo Description Updated
2020-09-14 09:04 pengbo Steps to Reproduce Updated
2020-09-14 09:04 pengbo Additional Information Updated
2020-09-14 14:07 pengbo Note Added: 0003562
2020-09-14 14:07 pengbo Status assigned => feedback
2020-09-15 12:58 whitebbai File Added: failover_stream.sh
2020-09-15 12:58 whitebbai File Added: pgpool-master.log
2020-09-15 12:58 whitebbai File Added: pgpool-slave.log
2020-09-15 12:58 whitebbai Note Added: 0003568
2020-09-15 12:58 whitebbai Status feedback => assigned
2020-12-09 17:05 pengbo Note Added: 0003649
2020-12-09 17:05 pengbo Status assigned => feedback