View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000709 | Pgpool-II | Bug | public | 2021-05-07 00:49 | 2021-05-07 09:44 |
| Reporter | rblst | Assigned To | pengbo | ||
| Priority | normal | Severity | crash | Reproducibility | always |
| Status | assigned | Resolution | open | ||
| Platform | x86-64 | OS | Red Hat Enterprise Linux | OS Version | 8.3 (Ootpa) |
| Product Version | 4.1.4 | ||||
| Summary | 0000709: segfault while triggering failover command | ||||
| Description | After startup of the PgPool cluster, if the primary backend is not reachable, a failover is triggered, but never completed, because the PgPool-II instance crashes with segmentation fault while triggering failover command: kernel: pgpool[31415]: segfault at 0 ip 00007f75e9ab9845 sp 00007fff15904128 error 4 in libc-2.28.so[7f75e995c000+1b9000] kernel: Code: 00 00 00 00 00 31 c0 c5 f8 77 c3 66 2e 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 89 f9 48 89 fa c5 f9 ef c0 83 e1 3f 83 f9 20 77 2b <c5> fd 74 0f c5 fd d7 c1 85 c0 0f 85 eb 00 00 00 48 83 c7 20 83 e1 systemd-coredump[32417]: Process 31415 (pgpool) of user 1002 dumped core. Stack trace of thread 31415: #0 0x00007f75e9ab9845 __strlen_avx2 (libc.so.6) 0000001 0x0000000000490666 string_append_char (pgpool) 0000002 0x0000000000408041 trigger_failover_command.part.6 (pgpool) 0000003 0x000000000040bb7e failover (pgpool) 0000004 0x000000000040d6fd sigusr1_interupt_processor (pgpool) 0000005 0x000000000040e9c7 PgpoolMain (pgpool) 0000006 0x00000000004058cd main (pgpool) 0000007 0x00007f75e997f7b3 __libc_start_main (libc.so.6) 0000008 0x0000000000405e3e _start (pgpool) | ||||
| Steps To Reproduce | * Configure master/slave mode. * Configure failover command. * Configure health check, but make it fail, ie. by using a wrong password. * Start up PgPool cluster. * Wait for timeout. | ||||
| Tags | failover | ||||
|
|
systemd_journal.txt (26,075 bytes)
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31422: DEBUG: received heartbeat signal from "10.100.230.74(psql-node3):5000" node:Not_Set
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31422: DEBUG: received heartbeat signal from "10.100.230.73(psql-node2):5000" node:Not_Set
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31421: DEBUG: watchdog life checking by heartbeat
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31421: DETAIL: checking pgpool 1 (psql-node2:5000)
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31421: DEBUG: watchdog checking if pgpool is alive using heartbeat
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31421: DETAIL: the last heartbeat from "psql-node2:5000" received 0 seconds ago
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31421: DEBUG: checking pgpool status by heartbeat
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31421: DETAIL: OK; status OK
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31421: DEBUG: watchdog life checking by heartbeat
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31421: DETAIL: checking pgpool 2 (psql-node3:5000)
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31421: DEBUG: watchdog checking if pgpool is alive using heartbeat
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31421: DETAIL: the last heartbeat from "psql-node3:5000" received 0 seconds ago
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31421: DEBUG: checking pgpool status by heartbeat
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31421: DETAIL: OK; status OK
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: received watchdog packet type:f
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: reading packet type f of length 178
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: STATE MACHINE INVOKED WITH EVENT = PACKET RECEIVED Current State = MASTER
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: received packet, watchdog node:[psql-node2:5000 Linux psql-node2] command id:[12] type:[FAILOVER FUNCTION COMMAND] state:[MASTER]
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: LOG: watchdog received the failover command from remote pgpool-II node "psql-node2:5000 Linux psql-node2"
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: LOG: watchdog is processing the failover command [DEGENERATE_BACKEND_REQUEST] received from psql-node2:5000 Linux psql-node2
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: LOG: we have got the consensus to perform the failover
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DETAIL: 2 node(s) voted in the favor
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: removing failover request from 2 nodes with ID:158
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: LOG: received degenerate backend request for node_id: 2 from pid [31417]
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: sending signal:10 to the parent process with PID:31415
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: sending packet, watchdog node:[psql-node2:5000 Linux psql-node2] command id:[12] type:[COMMAND REPLY IN DATA] state:[MASTER]
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: sending watchdog packet to socket:9, type:[-], command ID:12, data Length:54
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31415: DEBUG: Pgpool-II parent process received SIGUSR1
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31415: LOG: Pgpool-II parent process has received failover request
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31415: DEBUG: failover handler called
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31415: DEBUG: failover handler
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31415: DETAIL: kind: 1 flags: 3 node_count: 1 index:0
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: LOG: new IPC connection received
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: LOG: received the failover indication from Pgpool-II on IPC interface
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: LOG: watchdog is informed of failover start by the main process
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: sending packet, watchdog node:[psql-node2:5000 Linux psql-node2] command id:[165] type:[UNKNOWN] state:[MASTER]
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: sending watchdog packet to socket:9, type:[F], command ID:165, data Length:0
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: sending packet, watchdog node:[psql-node3:5000 Linux psql-node3] command id:[165] type:[UNKNOWN] state:[MASTER]
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: sending watchdog packet to socket:11, type:[F], command ID:165, data Length:0
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: sending watchdog packet to socket:12, type:[7], command ID:0, data Length:54
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31415: DEBUG: failover handler
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31415: DETAIL: starting to select new master node
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31415: LOG: starting degeneration. shutdown host psql-node3(5432)
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: received watchdog packet type:f
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: reading packet type f of length 178
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: STATE MACHINE INVOKED WITH EVENT = PACKET RECEIVED Current State = MASTER
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: received packet, watchdog node:[psql-node2:5000 Linux psql-node2] command id:[13] type:[FAILOVER FUNCTION COMMAND] state:[MASTER]
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: LOG: watchdog received the failover command from remote pgpool-II node "psql-node2:5000 Linux psql-node2"
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: LOG: watchdog is processing the failover command [DEGENERATE_BACKEND_REQUEST] received from psql-node2:5000 Linux psql-node2
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: LOG: we have got the consensus to perform the failover
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DETAIL: 2 node(s) voted in the favor
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: removing failover request from 2 nodes with ID:162
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: LOG: received degenerate backend request for node_id: 1 from pid [31417]
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: sending packet, watchdog node:[psql-node2:5000 Linux psql-node2] command id:[13] type:[COMMAND REPLY IN DATA] state:[MASTER]
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: sending watchdog packet to socket:9, type:[-], command ID:13, data Length:54
Apr 28 10:33:46 psql-node1 kernel: pgpool[31415]: segfault at 0 ip 00007f75e9ab9845 sp 00007fff15904128 error 4 in libc-2.28.so[7f75e995c000+1b9000]
Apr 28 10:33:46 psql-node1 kernel: Code: 00 00 00 00 00 31 c0 c5 f8 77 c3 66 2e 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 89 f9 48 89 fa c5 f9 ef c0 83 e1 3f 83 f9 20 77 2b <c5> fd 74 0f c5 fd d7 c1 85 c0 0f 85 eb 00 00 00 48 83 c7 20 83 e1
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31415: DEBUG: failover/failback request details: STREAM: 1 reqkind: 1 detail: 1 node_id: 2
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31415: LOG: Do not restart children because we are switching over node id 2 host: psql-node3 port: 5432 and we are in streaming replication mode
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: received watchdog packet type:f
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: reading packet type f of length 178
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: STATE MACHINE INVOKED WITH EVENT = PACKET RECEIVED Current State = MASTER
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: received packet, watchdog node:[psql-node2:5000 Linux psql-node2] command id:[14] type:[FAILOVER FUNCTION COMMAND] state:[MASTER]
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: LOG: watchdog received the failover command from remote pgpool-II node "psql-node2:5000 Linux psql-node2"
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: LOG: watchdog is processing the failover command [DEGENERATE_BACKEND_REQUEST] received from psql-node2:5000 Linux psql-node2
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: LOG: we have got the consensus to perform the failover
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DETAIL: 2 node(s) voted in the favor
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: removing failover request from 2 nodes with ID:160
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: LOG: received degenerate backend request for node_id: 0 from pid [31417]
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: sending packet, watchdog node:[psql-node2:5000 Linux psql-node2] command id:[14] type:[COMMAND REPLY IN DATA] state:[MASTER]
Apr 28 10:33:46 psql-node1 pgpool[31415]: 2021-04-28 10:33:46: pid 31417: DEBUG: sending watchdog packet to socket:9, type:[-], command ID:14, data Length:54
Apr 28 10:33:46 psql-node1 systemd[1]: Started Process Core Dump (PID 32415/UID 0).
-- Subject: Unit systemd-coredump@1-32415-0.service has finished start-up
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit systemd-coredump@1-32415-0.service has finished starting up.
--
-- The start-up result is done.
Apr 28 10:33:47 psql-node1 pgpool[31415]: 2021-04-28 10:33:47: pid 31423: DEBUG: watchdog heartbeat: send heartbeat signal to psql-node2:9694
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Main process exited, code=dumped, status=11/SEGV
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31417 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31421 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31422 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31423 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31424 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31425 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31426 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31427 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31428 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31429 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31430 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31431 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31432 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31433 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31434 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31435 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31436 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31437 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31438 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31439 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31440 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31441 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31442 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31443 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31444 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31445 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31446 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31447 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31448 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31449 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31450 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31451 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31452 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31453 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31454 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31455 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31456 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31457 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31458 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31459 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31460 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31461 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31462 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31463 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31464 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31465 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31466 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31467 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31468 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31469 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31470 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31471 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31472 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31473 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31474 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31475 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31476 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31477 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31478 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31479 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31480 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31481 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31482 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31483 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31484 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31485 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31486 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31487 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31488 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31489 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31490 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31491 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31492 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31493 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31494 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31495 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31496 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31497 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31498 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31499 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31500 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31501 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31502 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31503 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31504 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31505 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31506 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31507 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31508 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31509 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31510 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31511 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31512 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31513 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31514 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31515 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31516 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31517 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31518 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31519 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31520 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31521 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31522 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31523 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31524 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31525 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31526 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31527 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31528 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31963 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31964 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31965 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31966 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Killing process 31967 (pgpool) with signal SIGKILL.
Apr 28 10:33:48 psql-node1 systemd[1]: pgpool-II-13.service: Failed with result 'core-dump'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The unit pgpool-II-13.service has entered the 'failed' state with result 'core-dump'.
Apr 28 10:33:48 psql-node1 systemd-coredump[32417]: Process 31415 (pgpool) of user 1002 dumped core.
Stack trace of thread 31415:
#0 0x00007f75e9ab9845 __strlen_avx2 (libc.so.6)
#1 0x0000000000490666 string_append_char (pgpool)
#2 0x0000000000408041 trigger_failover_command.part.6 (pgpool)
#3 0x000000000040bb7e failover (pgpool)
#4 0x000000000040d6fd sigusr1_interupt_processor (pgpool)
#5 0x000000000040e9c7 PgpoolMain (pgpool)
#6 0x00000000004058cd main (pgpool)
#7 0x00007f75e997f7b3 __libc_start_main (libc.so.6)
#8 0x0000000000405e3e _start (pgpool)
-- Subject: Process 31415 (pgpool) dumped core
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
-- Documentation: man:core(5)
--
-- Process 31415 (pgpool) crashed and dumped core.
--
-- This usually indicates a programming error in the crashing program and
-- should be reported to its vendor as a bug.
Apr 28 10:33:48 psql-node1 systemd[1]: systemd-coredump@1-32415-0.service: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The unit systemd-coredump@1-32415-0.service has successfully entered the 'dead' state.
|
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2021-05-07 00:49 | rblst | New Issue | |
| 2021-05-07 00:49 | rblst | Tag Attached: failover | |
| 2021-05-07 00:49 | rblst | File Added: systemd_journal.txt | |
| 2021-05-07 09:44 | administrator | Assigned To | => pengbo |
| 2021-05-07 09:44 | administrator | Status | new => assigned |
| 2021-05-07 09:44 | administrator | Description Updated | |
| 2021-05-07 09:44 | administrator | Steps to Reproduce Updated |
