[Pgpool-general] health_check

[Xu Qian]

Hi All,

I am using pgpool 3.1 to replicate database.
I noticed that everyday 2-3 pgpool child processes will be dead. After 3
weeks, all pgpool child processes (max = 50) will be dead.
Consequentially, my application cannot access to database any more.

I typed "netstat -anp|grep CLOSE_WAIT" and debugged one of the dead
process.
I got the following callstack information:
---------------------------------------------------------------------------
#0  0xffffe410 in ?? ()
#1  0xbfffa76c in ?? ()
#2  0x00000002 in ?? ()
#3  0x00000000 in ?? ()
#4  0x401351be in __lll_mutex_lock_wait () from /lib/tls/libc.so.6
#5  0x4013bc99 in _L_mutex_lock_25 () from /lib/tls/libc.so.6
#6  0x0805b4ef in _IO_stdin_used ()
#7  0x00000000 in ?? ()
#8  0xbfffa7fc in ?? ()
#9  0x08051ba4 in pool_get_cp (user=0x80702c9 "mvv32", database=0x6
<Address 0x6 out of bounds>, protoMajor=1) at pool_connection_pool.c:117
#10 0x080512f4 in connect_inet_domain_socket (secondary_backend=0) at
pool_connection_pool.c:358
#11 0x0805184c in create_cp (cp=0x8073d20, secondary_backend=Variable
"secondary_backend" is not available.
) at pool_connection_pool.c:443
#12 0x08051883 in new_connection (p=0x8070cf0) at
pool_connection_pool.c:468
#13 0x0804d205 in do_child (unix_fd=4, inet_fd=5) at child.c:953
#14 0x08049ea9 in fork_a_child (unix_fd=4, inet_fd=5) at main.c:918
#15 0x0804b764 in reap_handler (sig=17) at main.c:1611
#16 <signal handler called>
#17 0xffffe410 in ?? ()
#18 0xbfffabf0 in ?? ()
#19 0x00001000 in ?? ()
#20 0x40189000 in ?? ()
#21 0x4011b4b3 in __read_nocancel () from /lib/tls/libc.so.6
#22 0x400cdc28 in _IO_file_read_internal () from /lib/tls/libc.so.6
#23 0x400cdd7e in _IO_new_file_underflow () from /lib/tls/libc.so.6
#24 0x400cf4eb in _IO_default_uflow_internal () from /lib/tls/libc.so.6
#25 0x400d0896 in __uflow () from /lib/tls/libc.so.6
#26 0x400c4e00 in _IO_getline_info_internal () from /lib/tls/libc.so.6
#27 0x400c4f2f in _IO_getline_internal () from /lib/tls/libc.so.6
#28 0x400cc97d in fgets_unlocked () from /lib/tls/libc.so.6
#29 0x40198fd2 in internal_getent () from /lib/libnss_files.so.2
#30 0x401998a0 in _nss_files_gethostbyname_r ()
from /lib/libnss_files.so.2
#31 0x4013c276 in gethostbyname_r@@GLIBC_2.1.2 ()
from /lib/tls/libc.so.6
#32 0x4013bbe4 in gethostbyname () from /lib/tls/libc.so.6
#33 0x080512f4 in connect_inet_domain_socket (secondary_backend=0) at
pool_connection_pool.c:358
#34 0x0804bb5c in health_check () 
---------------------------------------------------------------------------

Can somebody explain the reason or give my some hints?

Best regards
--Qian


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://pgfoundry.org/pipermail/pgpool-general/attachments/20090129/941f19ff/attachment.html