[Pgpool-general] Failover shuts down incoming connections
Tatsuo Ishii
ishii at sraoss.co.jp
Wed Jan 21 01:46:51 UTC 2009
Marcelo,
Sorry for confusion. The patch I posted was wrong one. Yes, the patch
is intended for debugging purpose since Daniel's problem cannot be
reproduced here and I need to gather as much info. I have just
comitted new verion of pool_process_query.c which includes the new
debugging code.
If you could reproduce the problem Daniel mentioned, please let me
know.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
> What patch command do you use to apply these patches ? I have tried
> with the -c flag for context diffs but no luck there
>
> I basically added the patch code to the "pool_process_query.c" file
> and recompiled but I think I missed something cause any query I
> execute now hangs.
>
>
> patch -p 0 -c < pool_process_query.c.patch
> patching file pool_process_query.c
> Hunk #1 FAILED at 251.
> 1 out of 1 hunk FAILED -- saving rejects to file
> pool_process_query.c.rej
>
> thanks,
> Marcelo
> PostgreSQL DBA
> Linux/Solaris System Administrator
>
> On Jan 18, 2009, at 7:01 AM, Tatsuo Ishii wrote:
>
> > Thanks for testing. This error is very strange. It seems only *once* I
> > was able to reproduce the error. But since that was not reproducable,
> > it is very hard to track the case of the problem.
> >
> > If you easily reproduce the error, could you apply included patches
> > and show me the log? I have just added a debug log.
> > --
> > Tatsuo Ishii
> > SRA OSS, Inc. Japan
> >
> >> Hello,
> >>
> >> I finally have a log for what I noticed in the CVS head that I
> >> downloaded one week ago. (Right now I don't have permissions to do
> >> a new
> >> download).
> >>
> >> I started pgpool-II with -n and -d flags. In addition to this, I
> >> started
> >> it with the primary backend stopped. So as soon as it starts, it
> >> detects
> >> a failure in backend0, supposing to keep backend1 alive, but it's not
> >> the case: I can connect to pgpool but can't do anything. It says that
> >> the connection is closed, although I'm still in psql. Please, look at
> >> the below log and comments between lines.
> >>
> >> -bash-3.1$ pgpool -n -d
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: key: listen_addresses
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: value: '*' kind: 4
> >> [... loading up pgpool.conf info ...]
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: value: '/var/lib/pgsql/data'
> >> kind:
> >> 4
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: num_backends: 2 num_backends: 2
> >> total_weight: 2.000000
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: backend 0 weight:
> >> 1073741823.500000
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: backend 1 weight:
> >> 1073741823.500000
> >> 2009-01-15 12:56:38 DEBUG: pid 6617: I am 6617
> >> 2009-01-15 12:56:38 DEBUG: pid 6618: I am 6618
> >> 2009-01-15 12:56:38 DEBUG: pid 6619: I am 6619
> >> 2009-01-15 12:56:38 DEBUG: pid 6620: I am 6620
> >> 2009-01-15 12:56:38 DEBUG: pid 6621: I am 6621
> >> 2009-01-15 12:56:38 DEBUG: pid 6622: I am 6622
> >> 2009-01-15 12:56:38 DEBUG: pid 6623: I am 6623
> >> 2009-01-15 12:56:38 DEBUG: pid 6624: I am 6624
> >> 2009-01-15 12:56:38 DEBUG: pid 6625: I am 6625
> >> 2009-01-15 12:56:38 DEBUG: pid 6626: I am 6626
> >> 2009-01-15 12:56:38 DEBUG: pid 6627: I am 6627
> >> 2009-01-15 12:56:38 DEBUG: pid 6628: I am 6628
> >> 2009-01-15 12:56:38 DEBUG: pid 6629: I am 6629
> >> 2009-01-15 12:56:38 DEBUG: pid 6630: I am 6630
> >> 2009-01-15 12:56:38 DEBUG: pid 6631: I am 6631
> >> 2009-01-15 12:56:38 DEBUG: pid 6632: I am 6632
> >> 2009-01-15 12:56:38 DEBUG: pid 6633: I am 6633
> >> 2009-01-15 12:56:38 DEBUG: pid 6634: I am 6634
> >> 2009-01-15 12:56:38 DEBUG: pid 6635: I am 6635
> >> 2009-01-15 12:56:38 DEBUG: pid 6636: I am 6636
> >> 2009-01-15 12:56:38 DEBUG: pid 6637: I am 6637
> >> 2009-01-15 12:56:38 DEBUG: pid 6638: I am 6638
> >> 2009-01-15 12:56:38 DEBUG: pid 6639: I am 6639
> >> 2009-01-15 12:56:38 DEBUG: pid 6640: I am 6640
> >> 2009-01-15 12:56:38 DEBUG: pid 6641: I am 6641
> >> 2009-01-15 12:56:38 DEBUG: pid 6642: I am 6642
> >> 2009-01-15 12:56:38 DEBUG: pid 6643: I am 6643
> >> 2009-01-15 12:56:38 DEBUG: pid 6644: I am 6644
> >> 2009-01-15 12:56:38 DEBUG: pid 6645: I am 6645
> >> 2009-01-15 12:56:38 DEBUG: pid 6646: I am 6646
> >> 2009-01-15 12:56:38 DEBUG: pid 6647: I am 6647
> >> 2009-01-15 12:56:38 DEBUG: pid 6648: I am 6648
> >> 2009-01-15 12:56:38 DEBUG: pid 6649: I am 6649
> >> 2009-01-15 12:56:38 DEBUG: pid 6650: I am 6650
> >> 2009-01-15 12:56:38 DEBUG: pid 6651: I am 6651
> >> 2009-01-15 12:56:38 DEBUG: pid 6652: I am 6652
> >> 2009-01-15 12:56:38 DEBUG: pid 6653: I am 6653
> >> 2009-01-15 12:56:38 DEBUG: pid 6654: I am 6654
> >> 2009-01-15 12:56:38 DEBUG: pid 6655: I am 6655
> >> 2009-01-15 12:56:38 DEBUG: pid 6656: I am 6656
> >> 2009-01-15 12:56:38 DEBUG: pid 6657: I am 6657
> >> 2009-01-15 12:56:38 DEBUG: pid 6658: I am 6658
> >> 2009-01-15 12:56:38 DEBUG: pid 6659: I am 6659
> >> 2009-01-15 12:56:38 DEBUG: pid 6660: I am 6660
> >> 2009-01-15 12:56:38 DEBUG: pid 6661: I am 6661
> >> 2009-01-15 12:56:38 DEBUG: pid 6662: I am 6662
> >> 2009-01-15 12:56:38 DEBUG: pid 6663: I am 6663
> >> 2009-01-15 12:56:38 DEBUG: pid 6664: I am 6664
> >> 2009-01-15 12:56:38 DEBUG: pid 6665: I am 6665
> >> 2009-01-15 12:56:38 DEBUG: pid 6666: I am 6666
> >> 2009-01-15 12:56:38 DEBUG: pid 6667: I am 6667
> >> 2009-01-15 12:56:38 DEBUG: pid 6668: I am 6668
> >> 2009-01-15 12:56:38 DEBUG: pid 6669: I am 6669
> >> 2009-01-15 12:56:38 DEBUG: pid 6670: I am 6670
> >> 2009-01-15 12:56:38 DEBUG: pid 6671: I am 6671
> >> 2009-01-15 12:56:38 DEBUG: pid 6672: I am 6672
> >> 2009-01-15 12:56:38 DEBUG: pid 6673: I am 6673
> >> 2009-01-15 12:56:38 DEBUG: pid 6674: I am 6674
> >> 2009-01-15 12:56:38 DEBUG: pid 6675: I am 6675
> >> 2009-01-15 12:56:38 DEBUG: pid 6676: I am 6676
> >> 2009-01-15 12:56:38 DEBUG: pid 6677: I am 6677
> >> 2009-01-15 12:56:38 DEBUG: pid 6678: I am 6678
> >> 2009-01-15 12:56:38 DEBUG: pid 6679: I am 6679
> >> 2009-01-15 12:56:38 DEBUG: pid 6680: I am 6680
> >> 2009-01-15 12:56:38 DEBUG: pid 6681: I am 6681
> >> 2009-01-15 12:56:38 DEBUG: pid 6682: I am 6682
> >> 2009-01-15 12:56:38 DEBUG: pid 6683: I am 6683
> >> 2009-01-15 12:56:38 DEBUG: pid 6684: I am 6684
> >> 2009-01-15 12:56:38 DEBUG: pid 6685: I am 6685
> >> 2009-01-15 12:56:38 DEBUG: pid 6686: I am 6686
> >> 2009-01-15 12:56:38 DEBUG: pid 6687: I am 6687
> >> 2009-01-15 12:56:38 DEBUG: pid 6688: I am 6688
> >> 2009-01-15 12:56:38 DEBUG: pid 6689: I am 6689
> >> 2009-01-15 12:56:38 DEBUG: pid 6690: I am 6690
> >> 2009-01-15 12:56:38 DEBUG: pid 6691: I am 6691
> >> 2009-01-15 12:56:38 DEBUG: pid 6692: I am 6692
> >> 2009-01-15 12:56:38 DEBUG: pid 6693: I am 6693
> >> 2009-01-15 12:56:38 DEBUG: pid 6694: I am 6694
> >> 2009-01-15 12:56:38 DEBUG: pid 6695: I am 6695
> >> 2009-01-15 12:56:38 DEBUG: pid 6696: I am 6696
> >> 2009-01-15 12:56:38 DEBUG: pid 6697: I am 6697
> >> 2009-01-15 12:56:38 DEBUG: pid 6698: I am 6698
> >> 2009-01-15 12:56:38 DEBUG: pid 6699: I am 6699
> >> 2009-01-15 12:56:38 DEBUG: pid 6700: I am 6700
> >> 2009-01-15 12:56:38 DEBUG: pid 6701: I am 6701
> >> 2009-01-15 12:56:38 DEBUG: pid 6702: I am 6702
> >> 2009-01-15 12:56:38 DEBUG: pid 6703: I am 6703
> >> 2009-01-15 12:56:38 DEBUG: pid 6704: I am 6704
> >> 2009-01-15 12:56:38 DEBUG: pid 6705: I am 6705
> >> 2009-01-15 12:56:38 DEBUG: pid 6706: I am 6706
> >> 2009-01-15 12:56:38 DEBUG: pid 6707: I am 6707
> >> 2009-01-15 12:56:38 DEBUG: pid 6708: I am 6708
> >> 2009-01-15 12:56:38 DEBUG: pid 6709: I am 6709
> >> 2009-01-15 12:56:38 DEBUG: pid 6710: I am 6710
> >> 2009-01-15 12:56:38 DEBUG: pid 6711: I am 6711
> >> 2009-01-15 12:56:38 DEBUG: pid 6712: I am 6712
> >> 2009-01-15 12:56:38 DEBUG: pid 6713: I am 6713
> >> 2009-01-15 12:56:38 DEBUG: pid 6714: I am 6714
> >> 2009-01-15 12:56:38 DEBUG: pid 6715: I am 6715
> >> 2009-01-15 12:56:38 DEBUG: pid 6716: I am 6716
> >> 2009-01-15 12:56:38 LOG: pid 6616: pgpool successfully started
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: starting health checking
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: health_check: 0 the DB node
> >> status:
> >> 1
> >> 2009-01-15 12:56:38 DEBUG: pid 6717: I am PCP 6717
> >> 2009-01-15 12:56:38 ERROR: pid 6616: connect_inet_domain_socket:
> >> connect() failed: Connection refused
> >> 2009-01-15 12:56:38 ERROR: pid 6616: health check failed. 0 th host
> >> 172.20.79.163 at port 5432 is down
> >> 2009-01-15 12:56:38 LOG: pid 6616: set 0 th backend down status
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler called
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: starting to
> >> select new master node
> >> 2009-01-15 12:56:38 LOG: pid 6616: starting degeneration. shutdown
> >> host 172.20.79.163(5432)
> >> 2009-01-15 12:56:38 DEBUG: pid 6617: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6617
> >> 2009-01-15 12:56:38 DEBUG: pid 6618: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6618
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6619
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6620
> >> 2009-01-15 12:56:38 DEBUG: pid 6619: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6621
> >> 2009-01-15 12:56:38 DEBUG: pid 6622: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6620: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6621: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6622
> >> 2009-01-15 12:56:38 DEBUG: pid 6623: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6623
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6624
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6625
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6626
> >> 2009-01-15 12:56:38 DEBUG: pid 6625: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6624: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6627: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6627
> >> 2009-01-15 12:56:38 DEBUG: pid 6626: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6628
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6629
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6630
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6631
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6632
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6633
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6634
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6635
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6636
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6637
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6638
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6639
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6640
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6641
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6642
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6643
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6644
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6645
> >> 2009-01-15 12:56:38 DEBUG: pid 6628: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6646: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6629: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6646
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6647
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6648
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6649
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6650
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6651
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6652
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6653
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6654
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6655
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6656
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6657
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6658
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6659
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6660
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6661
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6662
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6663
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6664
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6665
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6666
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6667
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6668
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6669
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6670
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6671
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6672
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6673
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6674
> >> 2009-01-15 12:56:38 DEBUG: pid 6631: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6675
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6676
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6677
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6678
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6679
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6680
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6681
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6682
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6683
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6684
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6685
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6686
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6687
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6688
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6689
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6690
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6691
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6692
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6693
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6694
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6695
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6696
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6697
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6698
> >> 2009-01-15 12:56:38 DEBUG: pid 6630: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6699
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6700
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6701
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6702
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6703
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6704
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6705
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6706
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6707
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6708
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6709
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6710
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6711
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6712
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6713
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6714
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6715
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: failover_handler: kill 6716
> >> 2009-01-15 12:56:38 LOG: pid 6616: failover_handler: set new master
> >> node: 1
> >> 2009-01-15 12:56:38 DEBUG: pid 6699: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6718: I am 6718
> >> 2009-01-15 12:56:38 DEBUG: pid 6714: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6703: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6719: I am 6719
> >> 2009-01-15 12:56:38 DEBUG: pid 6702: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6704: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6720: I am 6720
> >> 2009-01-15 12:56:38 DEBUG: pid 6707: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6713: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6705: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6721: I am 6721
> >> 2009-01-15 12:56:38 DEBUG: pid 6708: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6709: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6712: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6722: I am 6722
> >> 2009-01-15 12:56:38 DEBUG: pid 6710: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6701: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6723: I am 6723
> >> 2009-01-15 12:56:38 DEBUG: pid 6711: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6716: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6647: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6700: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6632: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6715: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6724: I am 6724
> >> 2009-01-15 12:56:38 DEBUG: pid 6648: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6633: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6725: I am 6725
> >> 2009-01-15 12:56:38 DEBUG: pid 6726: I am 6726
> >> 2009-01-15 12:56:38 DEBUG: pid 6727: I am 6727
> >> 2009-01-15 12:56:38 DEBUG: pid 6652: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6634: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6728: I am 6728
> >> 2009-01-15 12:56:38 DEBUG: pid 6653: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6635: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6729: I am 6729
> >> 2009-01-15 12:56:38 DEBUG: pid 6654: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6636: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6730: I am 6730
> >> 2009-01-15 12:56:38 DEBUG: pid 6655: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6637: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6731: I am 6731
> >> 2009-01-15 12:56:38 DEBUG: pid 6732: I am 6732
> >> 2009-01-15 12:56:38 DEBUG: pid 6656: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6638: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6657: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6733: I am 6733
> >> 2009-01-15 12:56:38 DEBUG: pid 6639: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6658: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6734: I am 6734
> >> 2009-01-15 12:56:38 DEBUG: pid 6640: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6735: I am 6735
> >> 2009-01-15 12:56:38 DEBUG: pid 6659: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6641: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6736: I am 6736
> >> 2009-01-15 12:56:38 DEBUG: pid 6660: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6642: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6737: I am 6737
> >> 2009-01-15 12:56:38 DEBUG: pid 6661: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6643: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6662: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6738: I am 6738
> >> 2009-01-15 12:56:38 DEBUG: pid 6644: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6739: I am 6739
> >> 2009-01-15 12:56:38 DEBUG: pid 6663: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6645: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6740: I am 6740
> >> 2009-01-15 12:56:38 DEBUG: pid 6664: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6741: I am 6741
> >> 2009-01-15 12:56:38 DEBUG: pid 6742: I am 6742
> >> 2009-01-15 12:56:38 DEBUG: pid 6649: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6743: I am 6743
> >> 2009-01-15 12:56:38 DEBUG: pid 6665: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6650: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6744: I am 6744
> >> 2009-01-15 12:56:38 DEBUG: pid 6666: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6651: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6667: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6673: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6745: I am 6745
> >> 2009-01-15 12:56:38 DEBUG: pid 6668: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6674: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6746: I am 6746
> >> 2009-01-15 12:56:38 DEBUG: pid 6669: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6675: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6670: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6747: I am 6747
> >> 2009-01-15 12:56:38 DEBUG: pid 6679: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6748: I am 6748
> >> 2009-01-15 12:56:38 DEBUG: pid 6671: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6749: I am 6749
> >> 2009-01-15 12:56:38 DEBUG: pid 6680: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6750: I am 6750
> >> 2009-01-15 12:56:38 DEBUG: pid 6672: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6681: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6751: I am 6751
> >> 2009-01-15 12:56:38 DEBUG: pid 6676: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6685: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6677: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6686: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6752: I am 6752
> >> 2009-01-15 12:56:38 DEBUG: pid 6678: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6687: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6753: I am 6753
> >> 2009-01-15 12:56:38 DEBUG: pid 6682: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6691: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6754: I am 6754
> >> 2009-01-15 12:56:38 DEBUG: pid 6683: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6755: I am 6755
> >> 2009-01-15 12:56:38 DEBUG: pid 6692: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6684: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6756: I am 6756
> >> 2009-01-15 12:56:38 DEBUG: pid 6693: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6757: I am 6757
> >> 2009-01-15 12:56:38 DEBUG: pid 6688: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6758: I am 6758
> >> 2009-01-15 12:56:38 DEBUG: pid 6697: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6689: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6698: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6760: I am 6760
> >> 2009-01-15 12:56:38 DEBUG: pid 6761: I am 6761
> >> 2009-01-15 12:56:38 DEBUG: pid 6762: I am 6762
> >> 2009-01-15 12:56:38 DEBUG: pid 6690: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6694: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6763: I am 6763
> >> 2009-01-15 12:56:38 DEBUG: pid 6695: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6764: I am 6764
> >> 2009-01-15 12:56:38 DEBUG: pid 6696: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6706: child receives shutdown request
> >> signal 3
> >> 2009-01-15 12:56:38 DEBUG: pid 6765: I am 6765
> >> 2009-01-15 12:56:38 DEBUG: pid 6759: I am 6759
> >> 2009-01-15 12:56:38 DEBUG: pid 6766: I am 6766
> >> 2009-01-15 12:56:38 DEBUG: pid 6767: I am 6767
> >> 2009-01-15 12:56:38 DEBUG: pid 6768: I am 6768
> >> 2009-01-15 12:56:38 DEBUG: pid 6769: I am 6769
> >> 2009-01-15 12:56:38 DEBUG: pid 6770: I am 6770
> >> 2009-01-15 12:56:38 DEBUG: pid 6771: I am 6771
> >> 2009-01-15 12:56:38 DEBUG: pid 6772: I am 6772
> >> 2009-01-15 12:56:38 DEBUG: pid 6773: I am 6773
> >> 2009-01-15 12:56:38 DEBUG: pid 6774: I am 6774
> >> 2009-01-15 12:56:38 DEBUG: pid 6775: I am 6775
> >> 2009-01-15 12:56:38 DEBUG: pid 6776: I am 6776
> >> 2009-01-15 12:56:38 DEBUG: pid 6777: I am 6777
> >> 2009-01-15 12:56:38 DEBUG: pid 6778: I am 6778
> >> 2009-01-15 12:56:38 DEBUG: pid 6779: I am 6779
> >> 2009-01-15 12:56:38 DEBUG: pid 6781: I am 6781
> >> 2009-01-15 12:56:38 DEBUG: pid 6782: I am 6782
> >> 2009-01-15 12:56:38 DEBUG: pid 6783: I am 6783
> >> 2009-01-15 12:56:38 DEBUG: pid 6780: I am 6780
> >> 2009-01-15 12:56:38 DEBUG: pid 6784: I am 6784
> >> 2009-01-15 12:56:38 DEBUG: pid 6785: I am 6785
> >> 2009-01-15 12:56:38 DEBUG: pid 6786: I am 6786
> >> 2009-01-15 12:56:38 DEBUG: pid 6787: I am 6787
> >> 2009-01-15 12:56:38 DEBUG: pid 6788: I am 6788
> >> 2009-01-15 12:56:38 DEBUG: pid 6789: I am 6789
> >> 2009-01-15 12:56:38 DEBUG: pid 6790: I am 6790
> >> 2009-01-15 12:56:38 DEBUG: pid 6791: I am 6791
> >> 2009-01-15 12:56:38 DEBUG: pid 6792: I am 6792
> >> 2009-01-15 12:56:38 DEBUG: pid 6793: I am 6793
> >> 2009-01-15 12:56:38 DEBUG: pid 6794: I am 6794
> >> 2009-01-15 12:56:38 DEBUG: pid 6795: I am 6795
> >> 2009-01-15 12:56:38 DEBUG: pid 6796: I am 6796
> >> 2009-01-15 12:56:38 DEBUG: pid 6797: I am 6797
> >> 2009-01-15 12:56:38 DEBUG: pid 6798: I am 6798
> >> 2009-01-15 12:56:38 DEBUG: pid 6799: I am 6799
> >> 2009-01-15 12:56:38 DEBUG: pid 6800: I am 6800
> >> 2009-01-15 12:56:38 DEBUG: pid 6801: I am 6801
> >> 2009-01-15 12:56:38 DEBUG: pid 6802: I am 6802
> >> 2009-01-15 12:56:38 DEBUG: pid 6803: I am 6803
> >> 2009-01-15 12:56:38 DEBUG: pid 6804: I am 6804
> >> 2009-01-15 12:56:38 DEBUG: pid 6805: I am 6805
> >> 2009-01-15 12:56:38 DEBUG: pid 6806: I am 6806
> >> 2009-01-15 12:56:38 DEBUG: pid 6807: I am 6807
> >> 2009-01-15 12:56:38 DEBUG: pid 6808: I am 6808
> >> 2009-01-15 12:56:38 DEBUG: pid 6809: I am 6809
> >> 2009-01-15 12:56:38 DEBUG: pid 6810: I am 6810
> >> 2009-01-15 12:56:38 DEBUG: pid 6811: I am 6811
> >> 2009-01-15 12:56:38 DEBUG: pid 6812: I am 6812
> >> 2009-01-15 12:56:38 DEBUG: pid 6813: I am 6813
> >> 2009-01-15 12:56:38 DEBUG: pid 6814: I am 6814
> >> 2009-01-15 12:56:38 DEBUG: pid 6815: I am 6815
> >> 2009-01-15 12:56:38 DEBUG: pid 6816: I am 6816
> >> 2009-01-15 12:56:38 DEBUG: pid 6817: I am 6817
> >> 2009-01-15 12:56:38 LOG: pid 6616: failover done. shutdown host
> >> 172.20.79.163(5432)
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: reap_handler called
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: reap_handler: call wait3
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6617 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6618 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6619 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6620 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6621 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6622 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6623 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6624 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6625 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6626 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6627 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6628 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6629 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6630 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6631 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6632 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6633 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6634 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6635 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6636 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6637 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6638 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6639 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6640 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6641 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6642 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6643 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6644 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6645 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6646 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6647 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6648 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6649 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6650 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6651 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6652 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6653 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6654 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6655 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6656 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6657 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6658 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6659 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6660 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6661 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6662 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6663 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6664 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6665 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6666 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6667 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6668 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6669 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6670 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6671 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6672 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6673 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6674 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6675 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6676 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6677 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6678 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6679 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6680 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6681 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6682 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6683 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6684 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6685 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6686 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6687 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6688 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6689 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6690 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6691 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6692 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6693 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6694 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6695 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6696 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6697 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6698 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6699 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6700 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6701 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6702 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6703 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6704 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6705 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6706 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6707 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6708 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6709 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6710 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6711 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6712 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6713 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6714 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6715 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: child 6716 exits with status 0
> >> by
> >> signal 0
> >> 2009-01-15 12:56:38 DEBUG: pid 6616: reap_handler: normally exited
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> [this stops here until I do something with my already opened
> >> connection:
> >> I had an open database connection, in which I did:
> >> eds_db=# select 1;
> >> server closed the connection unexpectedly
> >> This probably means the server terminated abnormally
> >> before or while processing the request.
> >> The connection to the server was lost. Attempting reset: Succeeded.
> >>
> >> The log continued below]
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: I am 6816 accept fd 5
> >> 2009-01-15 12:56:42 LOG: pid 6816: connection received:
> >> host=[local]
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: Protocol Major: 3 Minor: 0
> >> database: eds_db user: eds_db
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: new_connection: connecting 0
> >> backend
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: new_connection: skipping slot 0
> >> because backend_status = 3
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: new_connection: connecting 1
> >> backend
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: pool_read_message_length:
> >> slot: 1
> >> length: 8
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: pool_read_message_length2:
> >> master
> >> slot: 1 length: 25
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: 1 th backend: name:
> >> client_encoding
> >> value: UTF8
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: pool_read_message_length2:
> >> master
> >> slot: 1 length: 23
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: 1 th backend: name: DateStyle
> >> value: ISO, MDY
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: pool_read_message_length2:
> >> master
> >> slot: 1 length: 26
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: 1 th backend: name:
> >> integer_datetimes value: off
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: pool_read_message_length2:
> >> master
> >> slot: 1 length: 20
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: 1 th backend: name: is_superuser
> >> value: on
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: pool_read_message_length2:
> >> master
> >> slot: 1 length: 25
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: 1 th backend: name:
> >> server_encoding
> >> value: UTF8
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: pool_read_message_length2:
> >> master
> >> slot: 1 length: 25
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: 1 th backend: name:
> >> server_version
> >> value: 8.2.1
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: pool_read_message_length2:
> >> master
> >> slot: 1 length: 33
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: 1 th backend: name:
> >> session_authorization value: eds_db
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: pool_read_message_length2:
> >> master
> >> slot: 1 length: 36
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: 1 th backend: name:
> >> standard_conforming_strings value: off
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: pool_read_message_length2:
> >> master
> >> slot: 1 length: 24
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: 1 th backend: name: TimeZone
> >> value:
> >> US/Eastern
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: pool_read_message_length:
> >> slot: 1
> >> length: 12
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: pool_send_auth_ok: send pid
> >> 14410
> >> to frontend
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: read_kind_from_backend: read
> >> kind
> >> from 1 th backend Z NUM_BACKENDS: 2
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: pool_process_query: kind from
> >> backend: Z
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: pool_read_message_length:
> >> slot: 1
> >> length: 5
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: ReadyForQuery: message length: 5
> >> 2009-01-15 12:56:42 DEBUG: pid 6816: ReadyForQuery: transaction
> >> state: I
> >> 2009-01-15 12:56:43 DEBUG: pid 6616: starting health checking
> >> 2009-01-15 12:56:43 DEBUG: pid 6616: health_check: 0 the DB node
> >> status:
> >> 3
> >> 2009-01-15 12:56:43 DEBUG: pid 6616: health_check: 1 the DB node
> >> status:
> >> 2
> >> [...]
> >>
> >>
> >> Before this CVS downloaded version, I had pgpool-II-2.1. This was not
> >> happening. Instead of keep connections closed, it was blocking
> >> existent
> >> connections, which could be solved by disconnecting and connecting
> >> again.
> >>
> >> I'm using FC6 and PostgreSQL 8.2.1.
> >>
> >> Thanks,
> >> Daniel
> >> _______________________________________________
> >> Pgpool-general mailing list
> >> Pgpool-general at pgfoundry.org
> >> http://pgfoundry.org/mailman/listinfo/pgpool-general
> > Index: pool_process_query.c
> > ===================================================================
> > RCS file: /cvsroot/pgpool/pgpool-II/pool_process_query.c,v
> > retrieving revision 1.132
> > diff -c -r1.132 pool_process_query.c
> > *** pool_process_query.c 14 Jan 2009 14:54:08 -0000 1.132
> > --- pool_process_query.c 18 Jan 2009 12:53:23 -0000
> > ***************
> > *** 251,259 ****
> >
> > for (i = 0; i < NUM_BACKENDS; i++)
> > {
> > ! if (VALID_BACKEND(i) && FD_ISSET(CONNECTION(backend, i)->fd,
> > &readmask))
> > {
> > /*
> > * admin shutdown postmaster or postmaster goes down
> > */
> > if (detect_postmaster_down_error(CONNECTION(backend, i),
> > MAJOR(backend)) == SPECIFIED_ERROR)
> > --- 251,271 ----
> >
> > for (i = 0; i < NUM_BACKENDS; i++)
> > {
> > ! if (VALID_BACKEND(i))
> > {
> > /*
> > + * make sure that connection slot exists
> > + */
> > + if (CONNECTION_SLOT(backend, i) == 0)
> > + {
> > + pool_log("FATAL ERROR: VALID_BACKEND returns non 0 but
> > connection slot is empty. backend id:%d RAW_MODE:%d in_load_balance:
> > %d LOAD_BALANCE_STATUS:%d status:%d",
> > + i, RAW_MODE, in_load_balance, LOAD_BALANCE_STATUS(i),
> > BACKEND_INFO(i).backend_status);
> > + was_error = 1;
> > + break;
> > + }
> > +
> > + if (FD_ISSET(CONNECTION(backend, i)->fd, &readmask))
> > + /*
> > * admin shutdown postmaster or postmaster goes down
> > */
> > if (detect_postmaster_down_error(CONNECTION(backend, i),
> > MAJOR(backend)) == SPECIFIED_ERROR)
> > _______________________________________________
> > Pgpool-general mailing list
> > Pgpool-general at pgfoundry.org
> > http://pgfoundry.org/mailman/listinfo/pgpool-general
>
More information about the Pgpool-general
mailing list