View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000504 | Pgpool-II | General | public | 2019-04-30 14:26 | 2019-05-21 15:54 |
| Reporter | bhuwan | Assigned To | Muhammad Usama | ||
| Priority | urgent | Severity | major | Reproducibility | always |
| Status | resolved | Resolution | open | ||
| Product Version | 3.6.16 | ||||
| Summary | 0000504: Getting error during failover. Failover script is not running properly. | ||||
| Description | Hi, We have configured two Pgpool and two Postgresql server and trying to setup High availability. On database side we replication is working fine. But when master database is goesdown its unable to run failover script properly. Due to this whole system get down. | ||||
| Additional Information | Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool[76354]: [40-1] 2019-04-28 14:40:26: pid 76354: LOG: processing failover command lock request from IPC socket Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76400: LOG: reading and processing packets Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76400: DETAIL: postmaster on DB node 0 was shutdown by administrative command Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76400: LOG: received degenerate backend request for node_id: 0 from pid [76400] Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool[76354]: [41-1] 2019-04-28 14:40:26: pid 76354: LOG: local pgpool-II node "Linux_kca-vas-stg-pgp-1_5432" is requesting to become a lock holder for failover ID: 63 Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: new IPC connection received Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: watchdog received the failover command on IPC socket Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: watchdog received failover command [DEGENERATE_BACKEND_REQUEST] Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: ignoring the failover command [DEGENERATE_BACKEND_REQUEST] request, similar failover request is already in progress Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool[76354]: [42-1] 2019-04-28 14:40:26: pid 76354: LOG: local pgpool-II node "Linux_kca-vas-stg-pgp-1_5432" is the lock holder Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76378: LOG: reading and processing packets Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76378: DETAIL: postmaster on DB node 0 was shutdown by administrative command Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76378: LOG: received degenerate backend request for node_id: 0 from pid [76378] Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool[76354]: [43-1] 2019-04-28 14:40:26: pid 76354: LOG: new IPC connection received Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: new IPC connection received Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: watchdog received the failover command on IPC socket Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: watchdog received failover command [DEGENERATE_BACKEND_REQUEST] Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool[76354]: [44-1] 2019-04-28 14:40:26: pid 76354: LOG: watchdog received the failover command on IPC socket Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76400: LOG: degenerate backend request for 1 node(s) from pid [76400] is canceled by other pgpool Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool[76347]: [14-1] 2019-04-28 14:40:26: pid 76347: LOG: starting degeneration. shutdown host 10.183.126.166(5432) Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: ignoring the failover command [DEGENERATE_BACKEND_REQUEST] request, similar failover request is already in progress Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool[76354]: [45-1] 2019-04-28 14:40:26: pid 76354: LOG: watchdog received failover command [DEGENERATE_BACKEND_REQUEST] Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76378: LOG: degenerate backend request for 1 node(s) from pid [76378] is canceled by other pgpool Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool[76406]: [976-1] 2019-04-28 14:40:26: pid 76406: LOG: reading and processing packets Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76382: LOG: reading and processing packets Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76382: DETAIL: postmaster on DB node 0 was shutdown by administrative command Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76382: LOG: received degenerate backend request for node_id: 0 from pid [76382] Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool[76354]: [46-1] 2019-04-28 14:40:26: pid 76354: LOG: ignoring the failover command [DEGENERATE_BACKEND_REQUEST] request, similar failover request is already in progress Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: new IPC connection received Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: watchdog received the failover command on IPC socket Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: watchdog received failover command [DEGENERATE_BACKEND_REQUEST] Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: ignoring the failover command [DEGENERATE_BACKEND_REQUEST] request, similar failover request is already in progress Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool[76406]: [976-2] 2019-04-28 14:40:26: pid 76406: DETAIL: postmaster on DB node 0 was shutdown by administrative command Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76382: LOG: degenerate backend request for 1 node(s) from pid [76382] is canceled by other pgpool Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool[76406]: [977-1] 2019-04-28 14:40:26: pid 76406: LOG: received degenerate backend request for node_id: 0 from pid [76406] Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76379: LOG: reading and processing packets Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76379: DETAIL: postmaster on DB node 0 was shutdown by administrative command Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76379: LOG: received degenerate backend request for node_id: 0 from pid [76379] Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool[76354]: [47-1] 2019-04-28 14:40:26: pid 76354: LOG: new IPC connection received Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: new IPC connection received Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: watchdog received the failover command on IPC socket Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: watchdog received failover command [DEGENERATE_BACKEND_REQUEST] Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76354: LOG: ignoring the failover command [DEGENERATE_BACKEND_REQUEST] request, similar failover request is already in progress Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool[76380]: [550-1] 2019-04-28 14:40:26: pid 76380: LOG: degenerate backend request for 1 node(s) from pid [76380] is canceled by other pgpool Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76379: LOG: degenerate backend request for 1 node(s) from pid [76379] is canceled by other pgpool Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool[76354]: [48-1] 2019-04-28 14:40:26: pid 76354: LOG: remote pgpool-II node "Linux_kca-vas-stg-pgp-2_5432" is requesting to become a lock holder for failover ID: 63 Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76347: LOG: failover: no valid backends node found Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76347: LOG: Restart all children Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool: 2019-04-28 14:40:26: pid 76347: LOG: execute command: /etc/pgpool-II-94/Scripts/failover.sh 0 10.183.126.166 /var/lib/pgsql/9.4/data -1 "" "" 0 0 Apr 28 14:40:26 kca-vas-stg-pgp-1 pgpool[76354]: [49-1] 2019-04-28 14:40:26: pid 76354: LOG: lock holder request denied to remote pgpool-II node "Linux_kca-vas-stg-pgp-2_5432" | ||||
| Tags | failover | ||||
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2019-04-30 14:26 | bhuwan | New Issue | |
| 2019-04-30 14:26 | bhuwan | File Added: messages | |
| 2019-04-30 14:26 | bhuwan | Tag Attached: failover | |
| 2019-05-07 08:44 | administrator | Assigned To | => Muhammad Usama |
| 2019-05-07 08:44 | administrator | Status | new => assigned |
| 2019-05-07 08:44 | administrator | Description Updated | |
| 2019-05-07 08:44 | administrator | Additional Information Updated | |
| 2019-05-07 15:34 | bhuwan | Note Added: 0002586 | |
| 2019-05-21 15:54 | administrator | Status | assigned => resolved |
