View Issue Details

IDProjectCategoryView StatusLast Update
0000418Pgpool-IIBugpublic2019-01-30 10:04
ReporterSeemaAssigned Topengbo 
PriorityurgentSeveritymajorReproducibilityalways
Status assignedResolutionopen 
PlatformPGPOOL-IIOSFREEBSD 11.1-RELEASEOS Version11.1
Product Version3.6.11 
Target VersionFixed in Version 
Summary0000418: pg_dumpall is exiting with coredump
DescriptionHi,

I have configured PGPOOL-II with two DB Nodes running postgres. Versions are:

PGPOOL version --- pgpool-II version 3.6.11 (subaruboshi)
OS: Freebsd 11

Pgpool and DB is working fine with our applications.

Problem:
When I try t export the DB using pg_dump from the pgpool server, I am getting the following error:

root@host1:/ # pg_dump -h <host IP> -p 5432 -d <db name> -U pgsql > /cl/BACKUP-DB/dbname_pgp.sql
Password:
Segmentation fault (core dumped)
root@host1:/ #


My Observations:
1. pg_dumpall is working fine if executed directly on DB Nodes
2. pg_dump of single db - works sometimes, at other times it gives core dump (for the same DB)

The following is appearing in the pgpool.log :
Jul 30 11:48:42 pgp42 pgpool[89936]: [2-1] 2018-07-30 11:48:42: pid 89936: ERROR: unable to read data from frontend
Jul 30 11:48:42 pgp42 pgpool[89936]: [2-2] 2018-07-30 11:48:42: pid 89936: DETAIL: EOF encountered with frontend
Jul 30 11:48:42 pgp42 pgpool[89936]: [2-3] 2018-07-30 11:48:42: pid 89936: LOCATION: pool_stream.c:265
Jul 30 11:48:42 pgp42 pgpool[90187]: [2-1] 2018-07-30 11:48:42: pid 90187: ERROR: unable to read data from frontend
Jul 30 11:48:42 pgp42 pgpool[90187]: [2-2] 2018-07-30 11:48:42: pid 90187: DETAIL: EOF encountered with frontend
Jul 30 11:48:42 pgp42 pgpool[90187]: [2-3] 2018-07-30 11:48:42: pid 90187: LOCATION: pool_stream.c:265

The following parameters are given in /boot/loader.conf.local
kern.maxusers="1024"
kern.ipc.semmns="2048"
kern.ipc.semmni="128"
kern.ipc.shmall="33554432"
kern.ipc.shmseg="1024"
kern.ipc.shmmax=137438953472
kern.ipc.maxsockets="256000"
kern.ipc.maxsockbuf="2621440"
kern.maxswzone="335544320"
accfhttp_load="YES"
hw.igb.enable_msix=0
aesni_load="YES"

I am attaching the coredump file for your reference.

Expecting a quick solution, since we are not able to backup our production DB.

Regards,
Seema
TagsNo tags attached.

Activities

Seema

2018-07-30 15:44

reporter  

postgresql.conf (21,233 bytes)
pgpool.conf.orig (35,335 bytes)
pg_dump.core (7,151,616 bytes)

pengbo

2018-08-01 13:24

developer   ~0002130

Last edited: 2018-08-01 13:25

View 2 revisions

I think it is the same issue like 0000415.
https://www.pgpool.net/mantisbt/view.php?id=415

It seems that some PostgreSQL package was lacked for FreeBSD.

Seema

2018-08-01 13:59

reporter   ~0002131

Hi,
The issue is the same, but, this is a newly installed server with a more recent version of PGPOOL... Is the reason still the same?

Request you to analyse the coredump and suggest what could be the cause..

Regards,
Seema

pengbo

2018-08-01 14:09

developer   ~0002132

I think it is the same problem.

Could you try the following command like last time and show me the output?

# gdb pg_dump pg_dump.core
bt full

Seema

2018-08-01 14:47

reporter   ~0002133

The following is the output

 gdb pg_dump pg_dump.core
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...(no debugging symbols found)...
Core was generated by `/usr/local/bin/pg_dump -Fp host=139.59.65.160 port=5432 user=pgsql dbname=bsnlfi'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /usr/local/lib/libpq.so.5...(no debugging symbols found)...done.
Loaded symbols for /usr/local/lib/libpq.so.5
Reading symbols from /lib/libthr.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib/libthr.so.3
Reading symbols from /usr/local/lib/libintl.so.8...(no debugging symbols found)...done.
Loaded symbols for /usr/local/lib/libintl.so.8
Reading symbols from /lib/libz.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libz.so.6
Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.7
Reading symbols from /usr/lib/libssl.so.8...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libssl.so.8
Reading symbols from /lib/libcrypto.so.8...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypto.so.8
Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done.
Loaded symbols for /libexec/ld-elf.so.1
#0 0x000000000042b91c in ?? ()
(gdb)
(gdb)
(gdb) bt full
#0 0x000000000042b91c in ?? ()
No symbol table info available.
0000001 0x00000000004169a6 in ?? ()
No symbol table info available.
0000002 0x0000000000406116 in ?? ()
No symbol table info available.
0000003 0x00000000004041cf in ?? ()
No symbol table info available.
0000004 0x0000000800673000 in ?? ()
No symbol table info available.
0000005 0x0000000000000000 in ?? ()
No symbol table info available.
(gdb)

pengbo

2018-08-02 17:48

developer   ~0002134

It shows the same error about symbol table.
It seems that some PostgreSQL package was lacked for FreeBSD.

Seema

2018-08-06 14:01

reporter   ~0002137

Hi,
If I run the pg_dump directly from the DB nodes it works perfectly fine.
I also tried running the pg_dump from another freebsd server and that also works perfectly fine.

I am seeing this issue only when I run pg_dump from the PGPOOL server.

I may be wrong., but I am trying to understand how POSTGRES table is causing the issue. If the symbol table was the issue then, pg_dump shouldn't work from DB nodes and the standalone freebsd server also right.

I still feel there is some problem with the new environment. Could you help us fine the solution , since this is our Production environment.

Regards,
Seema

administrator

2018-12-04 10:46

administrator   ~0002301

If you have resolved this issue, could you close this item?

Seema

2018-12-04 13:49

reporter   ~0002303

Issue is still persisting. Right now we are taking pg_dump from a different server, while connecting to same DB.

Issue is seen when we try to take pg_dump from PGPOOL Server.

Issue History

Date Modified Username Field Change
2018-07-30 15:44 Seema New Issue
2018-07-30 15:44 Seema File Added: pg_dump.core
2018-07-30 15:44 Seema File Added: pgpool.conf.orig
2018-07-30 15:44 Seema File Added: postgresql.conf
2018-08-01 13:24 pengbo Note Added: 0002130
2018-08-01 13:25 pengbo Note Edited: 0002130 View Revisions
2018-08-01 13:59 Seema Note Added: 0002131
2018-08-01 14:09 pengbo Note Added: 0002132
2018-08-01 14:47 Seema Note Added: 0002133
2018-08-02 17:48 pengbo Note Added: 0002134
2018-08-06 14:01 Seema Note Added: 0002137
2018-12-04 10:46 administrator Note Added: 0002301
2018-12-04 13:49 Seema Note Added: 0002303
2019-01-30 10:04 pengbo Assigned To => pengbo
2019-01-30 10:04 pengbo Status new => assigned