[pgpool-general: 136] Re: Healthcheck timeout not always respected

[Tatsuo Ishii]

I did following:

Do following on the host where pgpool is running on:

# iptables -A FORWARD -j REJECT --reject-with icmp-port-unreachable -d 133.137.177.124
(133.137.177.124 is the host where backend is running on)

Pull network cable from backend0 host network interface. Pgpool
detected the host being down as expected. Am I missing something?
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

> Hello Tatsuo,
> 
> With backend0 on one host just configure following rule on other host where
> pgpool is:
> 
> iptables -A FORWARD -j REJECT --reject-with icmp-port-unreachable
> 
> and then have pgpool startup with health checking and retrying configured,
> and then pull network cable from backend0 host network interface.
> 
> Regards,
> Stevo.
> 
> On Wed, Jan 11, 2012 at 6:27 AM, Tatsuo Ishii <ishii at postgresql.org> wrote:
> 
>> I want to try to test the situation you descrived:
>>
>> >> > When system is configured for security reasons not to return
>> destination
>> >> > host unreachable messages, even though health_check_timeout is
>>
>> But I don't know how to do it. I pulled out the network cable and
>> pgpool detected it as expected. Also I configured the server which
>> PostgreSQL is running on to disable the 5432 port. In this case
>> connect(2) returned EHOSTUNREACH (No route to host) so pgpool detected
>> the error as expected.
>>
>> Could you please instruct me?
>> --
>> Tatsuo Ishii
>> SRA OSS, Inc. Japan
>> English: http://www.sraoss.co.jp/index_en.php
>> Japanese: http://www.sraoss.co.jp
>>
>> > Hello Tatsuo,
>> >
>> > Thank you for replying!
>> >
>> > I'm not sure what exactly is blocking, just by pgpool code analysis I
>> > suspect it is the part where a connection is made to the db and it
>> doesn't
>> > seem to get interrupted by alarm. Tested thoroughly health check
>> behaviour,
>> > it works really well when host/ip is there and just backend/postgres is
>> > down, but not when backend host/ip is down. I could see in log that
>> initial
>> > health check and each retry got delayed when host/ip is not reachable,
>> > while when just backend is not listening (is down) on the reachable
>> host/ip
>> > then initial health check and all retries are exact to the settings in
>> > pgpool.conf.
>> >
>> > PGCONNECT_TIMEOUT is listed as one of the libpq environment variables in
>> > the docs (see
>> http://www.postgresql.org/docs/9.1/static/libpq-envars.html )
>> > There is equivalent parameter in libpq PGconnectdbParams ( see
>> >
>> http://www.postgresql.org/docs/9.1/static/libpq-connect.html#LIBPQ-CONNECT-CONNECT-TIMEOUT
>> )
>> > At the beginning of that same page there are some important infos on
>> using
>> > these functions.
>> >
>> > psql respects PGCONNECT_TIMEOUT.
>> >
>> > Regards,
>> > Stevo.
>> >
>> > On Wed, Jan 11, 2012 at 12:13 AM, Tatsuo Ishii <ishii at postgresql.org>
>> wrote:
>> >
>> >> > Hello pgpool community,
>> >> >
>> >> > When system is configured for security reasons not to return
>> destination
>> >> > host unreachable messages, even though health_check_timeout is
>> >> configured,
>> >> > socket call will block and alarm will not get raised until TCP timeout
>> >> > occurs.
>> >>
>> >> Interesting. So are you saying that read(2) cannot be interrupted by
>> >> alarm signal if the system is configured not to return destination
>> >> host unreachable message? Could you please guide me where I can get
>> >> such that info? (I'm not a network expert).
>> >>
>> >> > Not a C programmer, found some info that select call could be replace
>> >> with
>> >> > select/pselect calls. Maybe it would be best if PGCONNECT_TIMEOUT
>> value
>> >> > could be used here for connection timeout. pgpool has libpq as
>> >> dependency,
>> >> > why isn't it using libpq for the healthcheck db connect calls, then
>> >> > PGCONNECT_TIMEOUT would be applied?
>> >>
>> >> I don't think libpq uses select/pselect for establishing connection,
>> >> but using libpq instead of homebrew code seems to be an idea. Let me
>> >> think about it.
>> >>
>> >> One question. Are you sure that libpq can deal with the case (not to
>> >> return destination host unreachable messages) by using
>> >> PGCONNECT_TIMEOUT?
>> >> --
>> >> Tatsuo Ishii
>> >> SRA OSS, Inc. Japan
>> >> English: http://www.sraoss.co.jp/index_en.php
>> >> Japanese: http://www.sraoss.co.jp
>> >>
>>