View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000055 | Pgpool-II | Bug | public | 2013-04-11 00:35 | 2015-09-17 15:32 |
| Reporter | ckoelbl | Assigned To | t-ishii | ||
| Priority | normal | Severity | major | Reproducibility | sometimes |
| Status | closed | Resolution | open | ||
| Platform | Linux | OS | CentOS | OS Version | 5.4 |
| Summary | 0000055: Child process ##### was terminated by segmentation fault | ||||
| Description | Pgpool 3.2.3 is running in front of two Postgres 9.2 databases with replication and load balancing enabled. On some queries, we're getting segmentatnion faults. The same query that caused the segmentation fault has completed successfully before the fault and will complete again successfully after the fault. Updating pgpool from 3.2.1 to 3.2.3 did not resolve the issue. | ||||
| Additional Information | Log entries from /var/log/messages: 2013-04-07T09:21:28-04:00 voldb04 kernel: pgpool[3463] general protection rip:46cb4f rsp:7fff33045a00 error:0 2013-04-07T09:26:42-04:00 voldb04 kernel: pgpool[2254] general protection rip:46cb4f rsp:7fff33045a50 error:0 2013-04-08T15:28:58-04:00 voldb04 kernel: pgpool[14096] general protection rip:46cb4f rsp:7fff33045a00 error:0 2013-04-08T19:25:05-04:00 voldb04 kernel: pgpool[25753]: segfault at 0000000000000020 rip 000000000046cb4f rsp 00007fff33045a00 error 4 2013-04-09T08:49:32-04:00 voldb04 kernel: pgpool[1548] general protection rip:46d2af rsp:7fff75521f80 error:0 2013-04-09T08:53:52-04:00 voldb04 kernel: pgpool[1635]: segfault at 0000000000000008 rip 0000003620672b7c rsp 00007fff75521f70 error 4 2013-04-09T09:13:23-04:00 voldb04 kernel: pgpool[1632] general protection rip:46d2af rsp:7fff75522140 error:0 2013-04-09T09:14:15-04:00 voldb04 kernel: pgpool[1547]: segfault at 0000003600000218 rip 0000003620672faf rsp 00007fff75521d30 error 4 2013-04-09T09:18:25-04:00 voldb04 kernel: pgpool[308] general protection rip:3620672b7c rsp:7fff75522190 error:0 2013-04-09T09:18:34-04:00 voldb04 kernel: pgpool[1636] general protection rip:46d2af rsp:7fff75522140 error:0 2013-04-09T09:19:13-04:00 voldb04 kernel: pgpool[1549]: segfault at 0000000000000108 rip 0000003620673084 rsp 00007fff75521e00 error 4 2013-04-09T09:19:33-04:00 voldb04 kernel: pgpool[488]: segfault at 0000003600000100 rip 000000000046d2af rsp 00007fff75522140 error 4 2013-04-09T09:19:46-04:00 voldb04 kernel: pgpool[30316] general protection rip:3620673084 rsp:7fff75521dc0 error:0 2013-04-09T09:22:57-04:00 voldb04 kernel: pgpool[1639] general protection rip:46d2af rsp:7fff75521f80 error:0 2013-04-09T09:24:31-04:00 voldb04 kernel: pgpool[1847]: segfault at 0000003600000100 rip 000000000046d2af rsp 00007fff75521f80 error 4 2013-04-09T09:33:15-04:00 voldb04 kernel: pgpool[1842]: segfault at 0000003600000218 rip 0000003620672faf rsp 00007fff75521d30 error 4 2013-04-09T09:33:21-04:00 voldb04 kernel: pgpool[1844]: segfault at 0000003600000040 rip 000000000046d2af rsp 00007fff755221a0 error 4 2013-04-09T09:33:24-04:00 voldb04 kernel: pgpool[1840]: segfault at 0000003600000200 rip 000000000046d2af rsp 00007fff755221a0 error 4 2013-04-09T09:37:11-04:00 voldb04 kernel: pgpool[1913]: segfault at 0000003600000218 rip 0000003620671d1d rsp 00007fff755220f0 error 4 2013-04-09T09:40:53-04:00 voldb04 kernel: pgpool[32623] general protection rip:46d2af rsp:7fff75521f80 error:0 2013-04-09T09:41:11-04:00 voldb04 kernel: pgpool[1906]: segfault at 0000000000000020 rip 000000000046d2af rsp 00007fff75521f80 error 4 2013-04-09T10:24:48-04:00 voldb04 kernel: pgpool[2294] general protection rip:46d2af rsp:7fff75521f80 error:0 2013-04-09T10:28:51-04:00 voldb04 kernel: pgpool[702]: segfault at 000000000000015f rip 000000000046d2af rsp 00007fff75521f20 error 4 2013-04-09T10:29:06-04:00 voldb04 kernel: pgpool[2296] general protection rip:46d2af rsp:7fff75521f80 error:0 2013-04-09T10:40:15-04:00 voldb04 kernel: pgpool[32652] general protection rip:46d2af rsp:7fff75521f80 error:0 2013-04-09T12:24:04-04:00 voldb04 kernel: pgpool[3227]: segfault at 0000000000000020 rip 000000000046d2af rsp 00007fff75521f80 error 4 2013-04-09T12:45:00-04:00 voldb04 kernel: pgpool[3461] general protection rip:46d2af rsp:7fff75521f80 error:0 2013-04-09T12:45:52-04:00 voldb04 kernel: pgpool[3412]: segfault at 0000000000000020 rip 000000000046d2af rsp 00007fff75522140 error 4 2013-04-09T13:19:31-04:00 voldb04 kernel: pgpool[3736]: segfault at 0000000000000020 rip 000000000046d2af rsp 00007fff75522140 error 4 2013-04-09T13:46:41-04:00 voldb04 kernel: pgpool[4074] general protection rip:46d2af rsp:7fff755221a0 error:0 2013-04-09T15:30:54-04:00 voldb04 kernel: pgpool[3890] general protection rip:46d2af rsp:7fff75521fd0 error:0 2013-04-09T16:16:31-04:00 voldb04 kernel: pgpool[5135] general protection rip:46d2af rsp:7fff75521fd0 error:0 2013-04-10T00:29:50-04:00 voldb04 kernel: pgpool[6961]: segfault at 0000000000000000 rip 000000362067be82 rsp 00007fff755221a8 error 4 2013-04-10T08:54:29-04:00 voldb04 kernel: pgpool[11441] general protection rip:46d2af rsp:7fff755221a0 error:0 | ||||
| Tags | No tags attached. | ||||
|
|
This issue appears similar if not directly related to issue 0000028. However no resolution to that ticket was indicated in the notes that highlighted either the underlying cause or a fix. |
|
|
|
|
|
Not sure how to interpret the message. Do you know how to convert it to gdb style backtrace? In the meantime because you did not show how to reporduce the problem, I cannot think of exact cause of the problem. However this: http://git.postgresql.org/gitweb/?p=pgpool2.git;a=commit;h=27e55bf17baa83c77afc20d27e856b1c1b6a4bd0 may or may not be related to your problem. |
|
|
Running same setup as ckoelbl, with CentOS 5.4 as well. We have a newer kernel (2.6.34.14), and glibc (2.5-107). We see the same problem on both 3.2.3 and 3.2.4. From pgpool.log 2013-06-23 09:29:45 ERROR: pid 28465: Child process 30289 was terminated by segmentation fault 2013-06-23 09:30:45 ERROR: pid 28465: Child process 29220 was terminated by segmentation fault 2013-06-23 09:31:16 ERROR: pid 28465: Child process 28533 was terminated by segmentation fault 2013-06-23 09:32:17 ERROR: pid 28465: Child process 28544 was terminated by segmentation fault 2013-06-23 09:34:58 ERROR: pid 28465: Child process 11448 was terminated by segmentation fault 2013-06-23 09:36:00 ERROR: pid 28465: Child process 11523 was terminated by segmentation fault 2013-06-23 09:46:29 ERROR: pid 28465: Child process 28496 was terminated by segmentation fault 2013-06-23 09:51:11 ERROR: pid 28465: Child process 13503 was terminated by segmentation fault 2013-06-23 09:54:38 ERROR: pid 28465: Child process 28487 was terminated by segmentation fault 2013-06-23 10:08:41 ERROR: pid 28465: Child process 21977 was terminated by segmentation fault 2013-06-23 10:18:16 ERROR: pid 28465: Child process 11534 was terminated by segmentation fault 2013-06-23 10:22:12 ERROR: pid 28465: Child process 10915 was terminated by segmentation fault 2013-06-23 10:24:01 ERROR: pid 28465: Child process 28539 was terminated by segmentation fault Same error in messages: Jun 23 09:29:13 tlf-ts200-tk-1 kernel: pgpool[28536]: segfault at ffffffffdf7ade88 ip 000000000044b24b sp 00007fff14557280 error 4 in pgpool[400000+ee000] Jun 23 09:29:41 tlf-ts200-tk-1 kernel: pgpool[7273]: segfault at ffffffffdf774058 ip 000000000044b24b sp 00007fff14557240 error 4 in pgpool[400000+ee000] Jun 23 09:29:45 tlf-ts200-tk-1 kernel: pgpool[30289]: segfault at ffffffffdf783c18 ip 000000000044b24b sp 00007fff14557240 error 4 in pgpool[400000+ee000] Jun 23 09:30:45 tlf-ts200-tk-1 kernel: pgpool[29220]: segfault at ffffffffffad6ffd ip 000000000044b24b sp 00007fff14557240 error 4 in pgpool[400000+ee000] |
|
|
Also tried with a build from master (last commit: a90a76), with the same result; 2013-06-23 16:40:21 ERROR: pid 1702: Child process 1710 was terminated by segmentation fault 2013-06-23 16:42:47 ERROR: pid 1702: Child process 1761 was terminated by segmentation fault 2013-06-23 16:43:05 ERROR: pid 1702: Child process 1742 was terminated by segmentation fault 2013-06-23 16:43:51 ERROR: pid 1702: Child process 1744 was terminated by segmentation fault 2013-06-23 16:45:23 ERROR: pid 1702: Child process 8604 was terminated by segmentation fault 2013-06-23 16:45:25 ERROR: pid 1702: Child process 30373 was terminated by segmentation fault |
|
|
Without a backtrace or a reproducable test case, It's very hard to fix. |
|
|
No feedback over 1 year. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2013-04-11 00:35 | ckoelbl | New Issue | |
| 2013-04-11 00:37 | ckoelbl | Note Added: 0000262 | |
| 2013-04-11 00:42 | ckoelbl | File Added: pgpool.conf | |
| 2013-04-11 09:42 | t-ishii | Note Added: 0000263 | |
| 2013-04-29 16:18 | t-ishii | Assigned To | => t-ishii |
| 2013-04-29 16:18 | t-ishii | Status | new => feedback |
| 2013-06-23 23:24 | goddva | Note Added: 0000291 | |
| 2013-06-23 23:46 | goddva | Note Added: 0000292 | |
| 2013-06-24 09:28 | t-ishii | Note Added: 0000293 | |
| 2015-09-17 15:32 | t-ishii | Note Added: 0000566 | |
| 2015-09-17 15:32 | t-ishii | Status | feedback => closed |
