[pgpool-general: 4274] Re: Pgpool - connection hangs in DISCARD ALL
Gerhard Wiesinger
lists at wiesinger.com
Thu Dec 31 16:08:14 JST 2015
On 31.12.2015 07:51, Gerhard Wiesinger wrote:
> On 28.12.2015 08:11, Tatsuo Ishii wrote:
>>
>> Would be great if you could release a stable 3.4.4 with this major
>> issue fixed,
>> Yes, we are going to release 3.4.4 in coming January.
>
> Hello Tatsuo,
>
> Great to hear. BTW: Would be great if you could tag ALL releases (e.g.
> 3.4.3 is missing) and make also release branches in git. I think that
> will make development easier.
>
> It worked now for 3 days well but today I still had the same issue.
> Looks like the probability has been reduced, but not fixed in all cases.
>
> ps wwwaux --sort=user,command,pid | grep -v grep | grep -E
> '^USER|pgpool|postgres:'
> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> postgres 864 0.0 0.0 197428 352 ? Ss Dec27 0:00
> postgres: logger process
> postgres 876 0.0 13.7 342424 68980 ? Ss Dec27 0:34
> postgres: checkpointer process
> postgres 877 0.0 19.4 342424 97400 ? Ss Dec27 0:16
> postgres: writer process
> postgres 878 0.0 0.7 342424 3996 ? Ss Dec27 0:09
> postgres: wal writer process
> postgres 879 0.0 0.7 342916 3896 ? Ss Dec27 1:09
> postgres: autovacuum launcher process
> postgres 880 0.0 0.4 198444 2380 ? Ss Dec27 2:13
> postgres: stats collector process
> postgres 2680 0.0 1.6 343548 8104 ? Ss 00:25 0:00
> postgres: hei hei 127.0.0.1(46596) idle
> postgres 2690 0.0 1.7 345652 8920 ? Ss 00:25 0:01
> postgres: nag hei 127.0.0.1(46606) BIND
> postgres 2691 0.0 1.9 345656 9928 ? Ss 00:25 0:01
> postgres: nag hei 127.0.0.1(46608) BIND
> postgres 2693 0.0 0.7 345564 3808 ? Ss 00:25 0:01
> postgres: nag tem 127.0.0.1(46610) SELECT
> postgres 2694 0.0 1.4 344964 7420 ? Ss 00:25 0:01
> postgres: nag hei 127.0.0.1(46612) idle
> postgres 2695 0.0 1.8 345640 9304 ? Ss 00:25 0:01
> postgres: nag hei 127.0.0.1(46614) BIND
> postgres 2696 0.0 1.5 345640 8004 ? Ss 00:25 0:01
> postgres: nag hei 127.0.0.1(46616) BIND
> postgres 2698 0.0 1.1 345708 5832 ? Ss 00:25 0:01
> postgres: nag tem 127.0.0.1(46620) SELECT
> postgres 2700 0.0 2.0 345640 10332 ? Ss 00:25 0:01
> postgres: nag hei 127.0.0.1(46624) BIND
> postgres 2701 0.0 1.6 345640 8236 ? Ss 00:25 0:01
> postgres: nag hei 127.0.0.1(46626) BIND
> postgres 2703 0.0 2.0 345640 10164 ? Ss 00:25 0:01
> postgres: nag hei 127.0.0.1(46630) BIND
> postgres 2704 0.0 2.2 347936 11480 ? Ss 00:25 0:02
> postgres: nag hei 127.0.0.1(46632) BIND
> postgres 2706 0.0 1.6 344348 8380 ? Ss 00:25 0:01
> postgres: nag hei 127.0.0.1(46636) PARSE
> postgres 2707 0.0 1.8 345360 9364 ? Ss 00:25 0:01
> postgres: nag hei 127.0.0.1(46638) BIND
> postgres 2756 0.0 2.2 345680 11436 ? Ss 00:27 0:01
> postgres: hei hei 127.0.0.1(46702) BIND
> postgres 2766 0.0 2.4 345640 12296 ? Ss 00:27 0:01
> postgres: nag hei 127.0.0.1(46716) BIND
> postgres 2767 0.0 1.3 346176 6668 ? Ss 00:27 0:01
> postgres: nag tem 127.0.0.1(46718) SELECT
> postgres 2768 0.0 2.4 345656 12132 ? Ss 00:27 0:01
> postgres: nag hei 127.0.0.1(46720) BIND
> postgres 2769 0.0 2.4 345640 12340 ? Ss 00:27 0:01
> postgres: nag hei 127.0.0.1(46722) BIND
> postgres 2771 0.0 2.4 345656 12136 ? Ss 00:27 0:01
> postgres: nag hei 127.0.0.1(46724) BIND
> postgres 2772 0.0 2.0 345360 10068 ? Ss 00:27 0:01
> postgres: nag hei 127.0.0.1(46726) BIND
> postgres 2773 0.0 2.7 347700 13972 ? Ss 00:27 0:02
> postgres: nag hei 127.0.0.1(46728) BIND
> postgres 2774 0.0 2.7 347488 13780 ? Ss 00:27 0:01
> postgres: nag hei 127.0.0.1(46730) idle
> postgres 2775 0.0 5.4 350560 27272 ? Ss 00:27 0:03
> postgres: nag hei 127.0.0.1(46732) idle
> postgres 2776 0.0 1.8 344348 9328 ? Ss 00:27 0:01
> postgres: nag hei 127.0.0.1(46734) PARSE
> postgres 2777 0.0 2.1 345360 10864 ? Ss 00:27 0:01
> postgres: nag hei 127.0.0.1(46736) BIND
> postgres 2778 0.0 1.8 344332 9120 ? Ss 00:27 0:01
> postgres: nag hei 127.0.0.1(46738) PARSE
> postgres 2779 0.0 1.1 345712 5772 ? Ss 00:27 0:01
> postgres: nag tem 127.0.0.1(46740) SELECT
> postgres 2780 0.0 1.6 345788 8360 ? Ss 00:27 0:01
> postgres: nag tem 127.0.0.1(46742) SELECT
> postgres 2783 0.0 1.5 344316 7916 ? Ss 00:27 0:01
> postgres: nag hei 127.0.0.1(46744) PARSE
> postgres 2784 0.0 2.2 345656 11316 ? Ss 00:27 0:00
> postgres: nag hei 127.0.0.1(46746) BIND
> postgres 2810 0.0 2.5 347168 13016 ? Ss 00:29 0:01
> postgres: nag hei 127.0.0.1(46784) BIND
> postgres 2812 0.0 3.2 352784 16136 ? Ss 00:29 0:03
> postgres: nag hei 127.0.0.1(46786) SELECT
> postgres 2840 0.0 1.7 344380 8784 ? Ss 00:31 0:01
> postgres: hei hei 127.0.0.1(46806) PARSE
> postgres 2861 0.0 1.6 343548 8480 ? Ss 00:32 0:00
> postgres: hei hei 127.0.0.1(46820) idle
> postgres 2888 0.0 0.6 343220 3224 ? Ss 00:34 0:00
> postgres: nag oen 127.0.0.1(46840) idle
> postgres 2891 0.0 1.7 346000 8828 ? Ss 00:34 0:01
> postgres: nag tem 127.0.0.1(46844) SELECT
> postgres 30552 0.2 11.8 347804 59316 ? Ss 07:01 0:06
> postgres: hei hei 127.0.0.1(47240) idle in transaction
> root 1349 0.0 0.1 94248 864 ? S 00:13 0:01
> pgpool: nag hei 192.168.0.9(39886) DISCARD
> root 1350 0.0 0.0 94252 432 ? S 00:13 0:00
> pgpool: nag hei 192.168.0.9(39636) DISCARD
> root 1862 0.0 0.1 94252 720 ? S 00:17 0:00
> pgpool: nag hei 192.168.0.9(39606) DISCARD
> root 1906 0.0 0.0 94248 468 ? S 00:18 0:00
> pgpool: nag hei 192.168.0.9(39864) DISCARD
> root 1959 0.0 0.0 94248 476 ? S 00:19 0:00
> pgpool: nag hei 192.168.0.9(39884) DISCARD
> root 1966 0.0 0.0 94252 448 ? S 00:19 0:00
> pgpool: nag tem 192.168.0.9(39592) DISCARD
> root 1967 0.0 0.0 94252 440 ? S 00:19 0:00
> pgpool: nag tem 192.168.0.9(39596) DISCARD
> root 1970 0.0 0.0 94248 484 ? S 00:19 0:00
> pgpool: nag hei 192.168.0.9(39888) DISCARD
> root 2310 0.0 0.0 94248 480 ? S 00:20 0:00
> pgpool: nag tem 192.168.0.9(39862) DISCARD
> root 2428 0.0 0.0 94252 440 ? S 00:22 0:00
> pgpool: nag hei 192.168.0.9(39594) DISCARD
> root 2433 0.0 0.0 94252 440 ? S 00:22 0:00
> pgpool: nag hei 192.168.0.9(39632) DISCARD
> root 2474 0.0 0.0 94252 436 ? S 00:23 0:00
> pgpool: nag hei 192.168.0.9(39634) DISCARD
> root 2475 0.0 0.0 94248 496 ? S 00:23 0:00
> pgpool: nag hei 192.168.0.9(40096) DISCARD
> root 2481 0.0 0.0 94248 488 ? S 00:23 0:00
> pgpool: nag hei 192.168.0.9(39856) DISCARD
> root 2528 0.0 0.1 94248 524 ? S 00:24 0:01
> pgpool: nag tem 192.168.0.9(40578) DISCARD
> root 2529 0.0 0.0 94252 436 ? S 00:24 0:00
> pgpool: nag hei 192.168.0.9(39600) DISCARD
> root 2530 0.0 0.0 94252 432 ? S 00:24 0:00
> pgpool: nag hei 192.168.0.9(39628) DISCARD
> root 2531 0.0 0.0 94252 496 ? S 00:24 0:00
> pgpool: nag hei 192.168.0.9(39872) DISCARD
> root 2532 0.0 0.0 94248 484 ? S 00:24 0:00
> pgpool: nag tem 192.168.0.9(39880) DISCARD
> root 2534 0.0 0.0 94252 448 ? S 00:24 0:00
> pgpool: nag hei 192.168.0.9(39608) DISCARD
> root 2537 0.0 0.1 94248 508 ? S 00:24 0:01
> pgpool: hei hei 192.168.0.9(40196) ABORT
> root 2539 0.0 0.0 94248 488 ? S 00:24 0:00
> pgpool: nag hei 192.168.0.9(39866) DISCARD
> root 2540 0.0 0.0 94248 488 ? S 00:24 0:00
> pgpool: nag hei 192.168.0.9(39878) DISCARD
> root 2543 0.0 0.0 94248 476 ? S 00:24 0:00
> pgpool: nag tem 192.168.0.9(39882) DISCARD
> root 2546 0.0 0.0 94248 464 ? S 00:24 0:00
> pgpool: nag hei 192.168.0.9(39590) DISCARD
> root 2717 0.0 0.1 94252 584 ? S 00:25 0:00
> pgpool: hei hei 192.168.0.9(39790) ABORT
> root 2718 0.0 0.0 94248 492 ? S 00:26 0:00
> pgpool: nag hei 192.168.0.9(39860) DISCARD
> root 2719 0.0 0.1 94248 512 ? S 00:26 0:00
> pgpool: nag hei 192.168.0.9(39876) DISCARD
> root 2721 0.0 0.0 94252 496 ? S 00:26 0:00
> pgpool: nag hei 192.168.0.9(39858) DISCARD
> root 2787 0.0 0.1 94248 544 ? S 00:28 0:02
> pgpool: nag hei 192.168.0.9(40580) DISCARD
> root 2788 0.0 0.1 94248 512 ? S 00:28 0:01
> pgpool: nag hei 192.168.0.9(40200) DISCARD
> root 24235 0.0 0.4 92000 2312 ? Ss Dec28 1:52
> /usr/bin/pgpool -f /etc/pgpool-II/pgpool.conf -n
> root 24270 0.0 0.0 92000 280 ? S Dec28 0:00
> pgpool: PCP: wait for connection request
> root 24271 0.0 0.0 92000 308 ? S Dec28 0:00
> pgpool: worker process
> root 30551 0.0 0.9 94252 4624 ? S 07:01 0:02
> pgpool: hei hei 192.168.0.113(42516) idle in transaction
>
> Any further ideas/fixes?
>
Some further information below:
# Hangs in DISCARD (SAME FOR OTHER WHICH HANG IN ABORT OR OTHER DISCARDS)
gdb -p 2718
backtrace
#0 0x00007efc97b54d63 in __select_nocancel () from /lib64/libc.so.6
#1 0x0000560ec8fb9347 in read_packets_and_process
(frontend=frontend at entry=0x560ec9bf0270,
backend=backend at entry=0x560ec9bef210, reset_request=reset_request at entry=1,
state=state at entry=0x7ffc0a53fe2c,
num_fields=num_fields at entry=0x7ffc0a53fe2a,
cont=cont at entry=0x7ffc0a53fe34 "\001") at protocol/pool_process_query.c:4654
#2 0x0000560ec8fba233 in pool_process_query (frontend=0x560ec9bf0270,
backend=0x560ec9bef210, reset_request=reset_request at entry=1) at
protocol/pool_process_query.c:273
#3 0x0000560ec8fb19e1 in backend_cleanup (backend=<optimized out>,
frontend=0x560ec92cb4e0 <child_frontend>) at protocol/child.c:440
#4 0x0000560ec8fb3c65 in do_child (fds=fds at entry=0x560ec9bea440) at
protocol/child.c:236
#5 0x0000560ec8f9168e in fork_a_child (fds=0x560ec9bea440, id=1) at
main/pgpool_main.c:678
#6 0x0000560ec8f92785 in reaper () at main/pgpool_main.c:2148
#7 0x0000560ec8f961db in PgpoolMain (discard_status=<optimized out>,
clear_memcache_oidmaps=<optimized out>) at main/pgpool_main.c:411
#8 0x0000560ec8f8fd5d in main (argc=<optimized out>,
argv=0x7ffc0a545378) at main/main.c:319
frame 1
#1 0x0000560ec8fb9347 in read_packets_and_process
(frontend=frontend at entry=0x560ec9bf0270,
backend=backend at entry=0x560ec9bef210, reset_request=reset_request at entry=1,
state=state at entry=0x7ffc0a53fe2c,
num_fields=num_fields at entry=0x7ffc0a53fe2a,
cont=cont at entry=0x7ffc0a53fe34 "\001") at protocol/pool_process_query.c:4654
4654 fds = select(num_fds, &readmask, &writemask,
&exceptmask, timeout);
info locals
readmask = {fds_bits = {256, 0 <repeats 15 times>}}
writemask = {fds_bits = {0 <repeats 16 times>}}
exceptmask = {fds_bits = {256, 0 <repeats 15 times>}}
fds = <optimized out>
timeoutdata = {tv_sec = 94621514089552, tv_usec = 94621501747623}
timeout = <optimized out>
num_fds = 9
was_error = <optimized out>
status = <optimized out>
i = <optimized out>
idle_count = 0
idle_count_in_recovery = 0
Thnx.
Ciao,
Gerhard
More information about the pgpool-general
mailing list