[Pgpool-general] pgpool-II-2.2.3 coredumps repeatedly

[Bob Lunney]

Tatsuo,

Thanks for taking a look at this.  It has stumped me, too.

The problem goes away when child_life_time is set to zero.  Also, it always happens when my_proc_id = 8, never some other number.

Bob

--- On Sat, 8/15/09, Tatsuo Ishii <ishii at sraoss.co.jp> wrote:

> From: Tatsuo Ishii <ishii at sraoss.co.jp>
> Subject: Re: [Pgpool-general] pgpool-II-2.2.3 coredumps repeatedly
> To: bob_lunney at yahoo.com
> Cc: pgpool-general at pgfoundry.org
> Date: Saturday, August 15, 2009, 8:59 PM
> I'm looking into this but with very
> little success.
> 
> Does any one experience the same problem with replication
> mode?
> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> 
> > I'm running pgpool-II-2.2.3 against PostgreSQL 8.4 on
> SLES 10.1.  After running for awhile pgpool dumps
> core.  It continues to run, but it repeatedly forks
> children and segfaults, filling the PG system db logs with
> messages like:
> > 
> > :4a82d4e0.77ef::30703:1:2009-08-12 09:42:40.235
> CDTLOG:  connection received: host=[local]
> > [local]:4a82d4e0.77ef::30703:2:2009-08-12 09:42:40.235
> CDTLOG:  connection authorized: user=pgpool
> database=pgpool
> > [local]:4a82d4e0.77ef::30703:3:2009-08-12 09:42:40.296
> CDTLOG:  unexpected EOF on client connection
> > [local]:4a82d4e0.77ef::30703:4:2009-08-12 09:42:40.296
> CDTLOG:  disconnection: session time: 0:00:00.061
> user=pgpool database=pgpool host
> > 
> > pgpool generates sets of these messages every few
> milliseconds.
> > 
> > It looks to me like the connection_info pointer has
> gotten corrupted:
> > 
> > GNU gdb 6.4
> > Copyright 2005 Free Software Foundation, Inc.
> > GDB is free software, covered by the GNU General
> Public License, and you are
> > welcome to change it and/or distribute copies of it
> under certain conditions.
> > Type "show copying" to see the conditions.
> > There is absolutely no warranty for GDB.  Type
> "show warranty" for details.
> > This GDB was configured as "x86_64-suse-linux"...Using
> host libthread_db library "/lib64/libthread_db.so.1".
> > 
> > Core was generated by `pgpool:     
>                
>                
>                
>              
>    '.
> > Program terminated with signal 11, Segmentation
> fault.
> > Reading symbols from
> /opt/pg-8.4.0/lib/libpq.so.5...done.
> > Loaded symbols for /opt/pg-8.4.0/lib/libpq.so.5
> > Reading symbols from
> /opt/pgpool/lib/libpcp.so.0...done.
> > Loaded symbols for /opt/pgpool/lib/libpcp.so.0
> > Reading symbols from /lib64/libresolv.so.2...done.
> > Loaded symbols for /lib64/libresolv.so.2
> > Reading symbols from /lib64/libnsl.so.1...done.
> > Loaded symbols for /lib64/libnsl.so.1
> > Reading symbols from /lib64/libm.so.6...done.
> > Loaded symbols for /lib64/libm.so.6
> > Reading symbols from /lib64/libc.so.6...done.
> > Loaded symbols for /lib64/libc.so.6
> > Reading symbols from /lib64/libcrypt.so.1...done.
> > Loaded symbols for /lib64/libcrypt.so.1
> > Reading symbols from /lib64/libpthread.so.0...done.
> > Loaded symbols for /lib64/libpthread.so.0
> > Reading symbols from
> /lib64/ld-linux-x86-64.so.2...done.
> > Loaded symbols for /lib64/ld-linux-x86-64.so.2
> > #0  0x00002b3505bd9b52 in memset () from
> /lib64/libc.so.6
> > (gdb) where
> > #0  0x00002b3505bd9b52 in memset () from
> /lib64/libc.so.6
> > #1  0x0000000000423291 in pool_init_cp () at
> pool_connection_pool.c:72
> > #2  0x00000000004076d9 in do_child (unix_fd=3,
> inet_fd=4) at child.c:154
> > #3  0x0000000000404c39 in fork_a_child
> (unix_fd=3, inet_fd=4, id=8) at main.c:828
> > #4  0x0000000000406b96 in reaper () at
> main.c:1617
> > #5  0x0000000000404021 in main (argc=6,
> argv=0x7fffa561c2a8) at main.c:474
> > (gdb) up
> > #1  0x0000000000423291 in pool_init_cp () at
> pool_connection_pool.c:72
> > 72             
>        
> memset(pool_connection_pool[i].info, 0,
> sizeof(ConnectionInfo));
> > (gdb) l
> > 67             
> memset(pool_connection_pool, 0,
> sizeof(POOL_CONNECTION_POOL)*pool_config->max_pool);
> > 68
> > 69              for
> (i = 0; i < pool_config->max_pool; i++)
> > 70              {
> > 71             
>         pool_connection_pool[i].info =
> &(MY_PROCESS_INFO.connection_info[i]);
> > 72             
>        
> memset(pool_connection_pool[i].info, 0,
> sizeof(ConnectionInfo));
> > 73              }
> > 74             
> return 0;
> > 75      }
> > 76
> > (gdb) p my_proc_id
> > $1 = 8
> > (gdb) p pids[7].connection_info[0]
> > $2 = {database = '\0' <repeats 63 times>, user =
> '\0' <repeats 31 times>, major = 0, minor = 0, pid =
> -565575680, key = -1413945014, counter = 0, create_time =
> 0,
> >   load_balancing_node = 0}
> > (gdb) p pids[8].connection_info[0]
> > Cannot access memory at address 0x2e77741b8c5c0000
> > 
> > Running in parallel mode, no replication, no
> loadbalance:
> > 
> > num_init_children = 10
> > max_pool = 4
> > child_life_time = 300
> > connection_life_time = 0
> > child_max_connections = 0
> > client_idle_limit = 0
> > 
> > Any suggestions on how to fix this?
> > 
> > 
> > 
> >       
> > _______________________________________________
> > Pgpool-general mailing list
> > Pgpool-general at pgfoundry.org
> > http://pgfoundry.org/mailman/listinfo/pgpool-general
>