[pgpool-general: 5724] Re: PGpool Configuration
Rishi
rishikeshms at gmail.com
Tue Sep 12 18:25:30 JST 2017
Hi Tatsuo,
Am not able to close connection jmeter until the process completed. Load
and memory goes high with 2 hours and request slows down :(
top - 09:22:34 up 3 days, 23:43, 2 users, load average: 38.14, 35.55,
27.58
Tasks: 2142 total, 12 running, 2130 sleeping, 0 stopped, 0 zombie
%Cpu(s): 89.7 us, 4.3 sy, 0.0 ni, 2.2 id, 0.0 wa, 0.0 hi, 3.8 si,
0.0 st
KiB Mem : 28811504 total, 13259276 free, 14191604 used, 1360624 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 13808564 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
COMMAND
8500 root 20 0 375112 147420 1552 S 6.1 0.5 3:24.52
pgpool
8543 root 20 0 375384 147584 1552 S 6.1 0.5 3:27.50
pgpool
6992 root 20 0 380484 152872 1552 S 5.8 0.5 3:37.96
pgpool
7001 root 20 0 374860 147028 1552 S 5.8 0.5 3:21.55
pgpool
8520 root 20 0 374340 147424 1552 S 5.8 0.5 3:22.88
pgpool
6705 root 20 0 375856 147892 1552 S 5.5 0.5 3:26.17
pgpool
6746 root 20 0 377352 149632 1552 S 5.5 0.5 3:32.25
pgpool
6747 root 20 0 377864 150276 1552 R 5.5 0.5 3:32.64
pgpool
6966 root 20 0 375784 147948 1552 S 5.5 0.5 3:26.64
pgpool
7489 root 20 0 378320 150740 1552 R 5.5 0.5 3:34.04
pgpool
8170 root 20 0 371288 143188 1552 S 5.5 0.5 3:13.26
pgpool
8209 root 20 0 376976 149300 1552 S 5.5 0.5 3:30.02
pgpool
8306 root 20 0 371124 143308 1552 R 5.2 0.5 3:16.86
pgpool
8409 root 20 0 373364 145544 1552 S 5.2 0.5 3:17.43
pgpool
8490 root 20 0 372132 144200 1552 R 5.2 0.5 3:18.36
pgpool
8576 root 20 0 374488 146660 1552 S 5.2 0.5 3:21.82
pgpool
7669 root 20 0 373664 145804 1552 S 4.9 0.5 3:22.21
pgpool
8165 root 20 0 377304 149512 1552 S 4.9 0.5 3:31.29
pgpool
8252 root 20 0 377180 149448 1552 S 4.9 0.5 3:32.48
pgpool
8301 root 20 0 374192 146504 1552 S 4.9 0.5 3:21.87
pgpool
8305 root 20 0 378708 151176 1552 S 4.9 0.5 3:36.54
pgpool
8494 root 20 0 373588 145716 1552 S 4.9 0.5 3:16.83
pgpool
8510 root 20 0 374984 147192 1552 S 4.9 0.5 3:19.79
pgpool
8512 root 20 0 375156 147312 1552 S 4.9 0.5 3:26.39
pgpool
8535 root 20 0 378300 150712 1552 S 4.9 0.5 3:29.06
pgpool
8542 root 20 0 377912 150176 1552 S 4.9 0.5 3:27.46
pgpool
7347 root 20 0 378188 151536 1552 S 4.6 0.5 3:33.59
pgpool
7794 root 20 0 373264 145372 1552 R 4.6 0.5 3:17.37
pgpool
8130 root 20 0 371908 144100 1552 R 4.6 0.5 3:14.56
pgpool
8215 root 20 0 375508 147592 1552 S 4.6 0.5 3:22.25
pgpool
8243 root 20 0 376120 148292 1552 S 4.6 0.5 3:30.88
pgpool
Here are the conf :
#------------------------------------------------------------------------------
# POOLS
#------------------------------------------------------------------------------
# - Concurrent session and pool size -
num_init_children = 2000
# Number of concurrent sessions allowed
# (change requires restart)
max_pool = 4
# Number of connection pool caches per
connection
# (change requires restart)
# - Life time -
#child_life_time = 300
child_life_time = 300
# Pool exits after being idle for this
many seconds
child_max_connections = 100
# Pool exits after receiving that many
connections
# 0 means no exit
connection_life_time =
# Connection to backend closes after
being idle for this many seconds
# 0 means no close
client_idle_limit = 0
# Client is disconnected after being
idle for that many seconds
# (even inside an explicit transactions!)
# 0 means no disconnection
We have 2 servers one is master and other one is slave with replication. (
both servers has 16core and 28G Memory with 2k connection open in each
server)
Let me know if you need any other details.
On Tue, Sep 12, 2017 at 12:03 PM, Tatsuo Ishii <ishii at sraoss.co.jp> wrote:
> Occasionally close connections from jemeter to Pgpool-II. I am not
> sure if you can do it with jmeter though.
>
> And set this:
>
> child_max_connections = 100
>
> will restart Pgpool-II child process every 100 times connections to
> Pgpool-II and it will free memory used by Pgpool-II child process.
>
> Best regards,
> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> English: http://www.sraoss.co.jp/index_en.php
> Japanese:http://www.sraoss.co.jp
>
> > Team,
> >
> > We have upgraded the latest version. but still the memory leak issue not
> > resolved. We are doing the performance testing using jmeter with 4k
> > concurrent users for more than a day where the memory is getting filled
> and
> > request has been slow down with errors.
> >
> > Any possible solutions?
> >
> >
> > On Thu, Sep 7, 2017 at 9:24 AM, Tatsuo Ishii <ishii at sraoss.co.jp> wrote:
> >
> >> >>From the above setup we are trying hit the concurrent users of 4k...
> But
> >> at
> >> > some point of time PGpool load got increased and memory is fully
> occupied
> >> > even if i increase the Memory full memory got occupied. This resulting
> >> > requests slow down and request per second throughput and failures.
> >>
> >> Sounds like an old problem in Pgpool-II 3.6.2. You should upgrade to
> >> the latest version: 3.6.6.
> >> >
> >> > But without PGpool directly connecting to Master works fine without
> >> errors
> >> > and much better throughput.
> >> >
> >> > We would like to scale 20k concurrent users with PGPool setup. We
> would
> >> > like to know whether PGpool is enterprise standard to work on.. Else
> we
> >> > have to look for other solutions.
> >>
> >> But that requires PostgreSQL to accept 20k concurrent connections or
> >> 40k connections (num_init_children * max_pool), which I have never
> >> seen in the field. Probably you should test out with PostgreSQL
> >> whether it works with 20k concurrent connections first.
> >>
> >> Best regards,
> >> --
> >> Tatsuo Ishii
> >> SRA OSS, Inc. Japan
> >> English: http://www.sraoss.co.jp/index_en.php
> >> Japanese:http://www.sraoss.co.jp
> >>
> >
> >
> >
> > --
> >
> > Regards,
> > Rishi
>
--
Regards,
Rishi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20170912/3ffb4220/attachment-0001.html>
More information about the pgpool-general
mailing list