[pgpool-general: 2162] Re: pgpool latency, hesitation under load

[Tatsuo Ishii]

> I have some questions.
> 
>> Hello,
>> Thanks for your message.
>> Yes, we were cascading pgpool-II.
>> At first, we tried only one, but when connections exceeded around 10% the
>> available pool, it increased latency for new connections to up 30 seconds.
>> By cascading pooling the maximum latency reduced to around 3 seconds.
>> But even 3 seconds were too much trouble at peak times. 
>> The web application impose around 2 thousand transactions per second, and even
>> more at peak times.
>> By my limited knowledge about pgpool2, it seems that each process forked has its
>> own queue. Therefore, one given connection could be assigned by kernel to a long
>> queue composed of long running queries, while at the same moment a side process
>> containing short queries has its queue flushed.
> 
> I think there's only one queue which is managed by the kernel.
> 
> Pgpool-II's process architecture is similar to Apache's model (if its
> worker is configured as process). The parent process issues listen()
> system call. The child process is forked off from the parent, thus
> inherits the same file descriptor of the parent's listen fd and issues
> accept() system call if it is not processing queries. When a client
> sends a connection request to the pgpool-II listen port, the kernel
> wakes up all the child process which have issued accept() system
> call. The kernel selects one of the child process then let the child
> process it. Because Apache's model is very successful as far as I
> know, I really don't have an idea which part illustrated above takes
> up to 30 seconds.

I have add a locking before calling accept() system call in child
process, which effectively limit the number of process to 1 which is
calling accept() at a time. I hope this mitigate the problem you
have. Can you please try the patch?
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp
-------------- next part --------------
A non-text attachment was scrubbed...
Name: accept.patch
Type: text/x-patch
Size: 918 bytes
Desc: not available
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20130926/21a7c492/attachment.bin>