[Pgpool-general] pgpool-II-3.0.3and4 memory leaks

Thu Jul 7 02:06:16 UTC 2011

This problem occurs on pgpool-II-3.0.X and pgpool-II-3.1.0-alpha2 but
does NOT occur on pgpool-II-2.5.X.

2011/7/7 takehiro wada <takehiro.wada at gmail.com>

> This problem occurs via some programming languages which are Python, Perl
> and Java as long as I checked.
> I checked the mount of used memory by ps auxww every 5 mins while my script
> runs.
>
>
> Result
> ----------------
> Wed Jul 6 17:16:05 JST 2011
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> postgres 15270  0.1  0.0   4692  1568 pts/7    S    17:09   0:00 pgpool:
> postgres postgres 127.0.0.1(39315) idle in transaction
>
> Wed Jul 6 17:21:05 JST 2011
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> postgres 15270  0.2  0.5  20948 17936 pts/7    S    17:09   0:00 pgpool:
> postgres postgres 127.0.0.1(39315) idle in transaction
>
> Wed Jul 6 17:26:05 JST 2011
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> postgres 15270  0.2  0.8  28940 25856 pts/7    S    17:09   0:00 pgpool:
> postgres postgres 127.0.0.1(39315) idle in transaction
>
>
> Wed Jul 6 17:31:05 JST 2011
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> postgres 15270  0.2  1.0  36932 33776 pts/7    S    17:09   0:03 pgpool:
> postgres postgres 127.0.0.1(39315) idle in transaction
>
>
> Wed Jul 6 17:36:05 JST 2011
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> postgres 15270  0.2  1.3  44928 41960 pts/7    S    17:09   0:04 pgpool:
> postgres postgres 127.0.0.1(39315) idle in transaction
>
>
> Wed Jul 6 17:41:05 JST 2011
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> postgres 15270  0.3  1.6  52784 49880 pts/7    S    17:09   0:05 pgpool:
> postgres postgres 127.0.0.1(39315) idle in transaction
>
>
> Wed Jul 6 17:46:05 JST 2011
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> postgres 15270  0.3  1.8  60776 57800 pts/7    S    17:09   0:06 pgpool:
> postgres postgres 127.0.0.1(39315) idle in transaction
>
>
> Wed Jul 6 17:51:05 JST 2011
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> postgres 15270  0.3  2.1  68768 65720 pts/7    S    17:09   0:07 pgpool:
> postgres postgres 127.0.0.1(39315) idle in transaction
>
>
> Wed Jul 6 17:56:05 JST 2011
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> postgres 15270  0.3  2.3  76764 73640 pts/7    S    17:09   0:08 pgpool:
> postgres postgres 127.0.0.1(39315) idle in transaction
>
>
> Wed Jul 6 18:01:05 JST 2011
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> postgres 15270  0.3  2.6  84756 81560 pts/7    S    17:09   0:09 pgpool:
> postgres postgres 127.0.0.1(39315) idle in transaction
>
>
> Wed Jul 6 18:06:05 JST 2011
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> postgres 15270  0.3  2.9  92612 89744 pts/7    S    17:09   0:10 pgpool:
> postgres postgres 127.0.0.1(39315) idle in transaction
> ----------------
>
> Additionally, I found another testcase with shell and psql.
>
> testcase
> -----------
> while(true)
> do
>  echo 'select count(*) from pg_tables;'
>  sleep 0.1
> done | psql -h localhost -p 10000 -U postgres postgres
> -----------
>
> 2011/7/6 Tatsuo Ishii <ishii at sraoss.co.jp>
>
>> I am not familiar with Python and am not sure what kind of queries are
>> actually sent. How long did it take before observing the memory leak?
>> minutes? hours? days?
>> --
>> Tatsuo Ishii
>> SRA OSS, Inc. Japan
>> English: http://www.sraoss.co.jp/index_en.php
>> Japanese: http://www.sraoss.co.jp
>>
>> > Hi,
>> >
>> > I'm using pgpool-II-3.0.3 and I have a memory problem of
>> pgpool-II-3.0.3.
>> > -------
>> > $ ps uaxww | grep i18n_generic_search
>> >  postgres 11188  0.0 40.0 3722712 2452996 ?     S    Jun20   7:54
>> pgpool:
>> > postgres postgres 127.0.0.1(49676) idle
>> > * physical memory is used 2GB the above process.
>> > -------
>> >
>> > I have been looking into the problem by creating a testcase.
>> >>From the testcase, what I found is that the mount of used memory keeps
>> > growing while an application executes SQLs in the period that the
>> > application is connecting to pgpool.
>> > Have you had any similar case so far?
>> >
>> > testcase
>> > --------------------
>> > import psycopg2
>> > import time
>> > import sys
>> >
>> > try:
>> >     connection = psycopg2.connect( database='postgres',
>> >                                    user='postgres'    ,
>> >                                    host='127.0.0.1'   , << pgpool
>> server's
>> > host
>> >                                    port='10000'       , << pgpool
>> server's
>> > port
>> >                                    password='postgres')
>> > except:
>> >     print ("Could Not connect to PostgreSQL or pgpool Server")
>> >
>> > ### main ###
>> > cursor = connection.cursor()
>> >
>> > while 1:
>> >   cursor.execute("select datname from pg_database")
>> >   rows = cursor.fetchall()
>> >   time.sleep(0.1)
>> >
>> > cursor.close()
>> > connection.close()
>> >
>> > exit()
>> > --------------------
>>
>
>
>
> --
> *************************
> Name:Takehiro Wada
> mail:takehiro.wada at gmail.com
> *************************
>

-- 
*************************
Name:Takehiro Wada
mail:takehiro.wada at gmail.com
*************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://pgfoundry.org/pipermail/pgpool-general/attachments/20110707/b34aedad/attachment-0001.html>