[pgpool-general: 7504] Re: pgpool2 in kubernetes dies

[Nikolay Mastilo (HM)]

Hello. The approach with tuning tcp timeouts is workable but in some 
circumstance it is not applicable. Our customer needs solution with 
configuration only in software components, such as: pgpool, postgresql, 
etc. The networks where such solution will be deployed can be 
unpredictably different in different network components. So, our 
solution should be configurable itself for different network environments.

We found out that in the event of shortterm loss of connection between 
pgpool and database, pgpool  detaches node from the pool and doesn't 
reconnect its own without restarting. Besides it, clients that were 
connected to pgpool have disconnected and cannot reconnect until pgpool 
is restarted.

The point is that in our solution we use pgpool as connection pooler and 
load balancer only, without controling cluster. Failoving management is 
carried out by patroni.

Could anyone advise any pgpool parameters set configuration when pgpool 
never detach nodes from its pool but try reconnect with specified period 
and continue interact with the client.

Regards,
   Nikolay

12.04.2021 11:05, Nathan Ward пишет:
>
>
>> On 8/04/2021, at 5:13 PM, Nikolay Mastilo (HM) <nikm71 at hotmail.com 
>> <mailto:nikm71 at hotmail.com>> wrote:
>>
>> Hi all! Does anyone have experience in launching pgpool2 in 
>> kubernetes? Our pgpool2 image running in k8s freezes after 90min and 
>> does not answer on requests.
>>
>> It answers after restart only. Database was neither stopped nor 
>> restarted. Please help!
>>
>
> Hi
>
> This sounds to me a lot like your kernel is configured with a 
> conntrack timeout of 90m somewhere. By default the linux tcp keepalive 
> doesn’t kick in until 7200s (120m). This is very common when using 
> hardware stateful firewalls with low tcp timeouts.
>
> Check the sysctl net.netfilter.nf_conntrack_tcp_timeout_established in 
> your host, and/or adjust the kernel tcp keepalive start timer 
> (net.ipv4.tcp_keepalive_time).
>
> Also check other similar keepalive / timeout things which apply. 
> iptables can set the timeout per flow, as well.
>
> Your environment might also have some proxy stuff going on with 
> different timeouts and/or keepalive timers, so you’ll need to look in 
> to that.
>
> Almost certainly though, this is a tcp timeout in a firewall (probably 
> iptables).
>
> --
> Nathan Ward
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.pgpool.net/pipermail/pgpool-general/attachments/20210413/f8dd6e2d/attachment.htm>