[pgpool-general: 3937] Re: accept() scalability issues

Michael Renner michael.renner at amd.co.at
Fri Aug 7 18:16:43 JST 2015

> On 07 Aug 2015, at 02:10, Tatsuo Ishii <ishii at postgresql.org> wrote:
>> Do you have any plans to implement an event- or lock-based approach in pgpool?
> Long time ago I played with accept+semaphores approach to see if
> there's any performance improvement. The result was not positive and I
> gave up the approach. Unfortunately nor test code or test data remain
> though.

I’d be surprised if you don’t see performance problems on todays hardware with medium load. Here’s numbers from our environment.

We’ve got three pgpool nodes, next to pgpool there’s also a mongodb config server running there, doing very little work.

Each server has two Intel Xeon E5-2430 v2, 6 cores @ 2.50GHz and 16GB of 1333MHz DDR3 RAM - fairly recent hardware and not the slowest to boot with.

Here’s a vmstat -w 1 output from an idle node: 

http://nopaste.narf.at/show/25530/ <http://nopaste.narf.at/show/25530/>

About 5000 context switches/sec, not nice but not bad either.

Here’s the same command from the node running the active pgpool instance:

http://nopaste.narf.at/show/25531/ <http://nopaste.narf.at/show/25531/>

About ~400.000 context switches/second - about 60% of _ALL_ available CPU time is spent in the kernel.

tcpdump shows a connection rate of about 350 connections/sec: http://nopaste.narf.at/show/25533/ <http://nopaste.narf.at/show/25533/>

stracing a single pgpool process shows the wakeups due to the unfenced select/accept loop: http://nopaste.narf.at/show/25527/ <http://nopaste.narf.at/show/25527/>

http://nopaste.narf.at/show/jPt7GaDbsRLCxZKhKu9y/ <http://nopaste.narf.at/show/jPt7GaDbsRLCxZKhKu9y/> is the configuration we run with, relevant settings:

connection_cache = on
num_init_children = 1500
max_pool = 1
child_life_time = 300
child_max_connections = 2000
connection_life_time = 0
client_idle_limit = 600

So if there’s nothing horribly wrong/out of the ordinary with our setup, I’d pretty much say that with a high number of children fencing around select/accept is dearly needed.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20150807/0559a6f1/attachment.html>

More information about the pgpool-general mailing list