View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000438 | Pgpool-II | Bug | public | 2018-10-19 05:18 | 2018-11-20 11:39 |
| Reporter | rafaelthca | Assigned To | pengbo | ||
| Priority | normal | Severity | minor | Reproducibility | always |
| Status | assigned | Resolution | open | ||
| Summary | 0000438: Memory consumption | ||||
| Description | Hi, I'm using 3.7.5. My scenario is that I have applications using pgpool and they never disconnect. Pgpool worker processes with high CPU time are consuming a lot of memory. Here is one example: postgres 15329 4.9 13.9 1885164 1730852 ? S Oct11 419:56 pgpool: XXX XXX X.X.X.X(54197) idle That machine has 12GB of ram and the process's memory keeps increasing. Is that a known behavior in pgpool and we MUST disconnect from time to time to allow pgpool to recycle the workers or can this be a memory leak? Thanks, Rafael. | ||||
| Tags | No tags attached. | ||||
|
|
It seems that we met the same problem(Bug 436). After investigation, we found that when too many stmt objects created(if you use php with pdo, each query statment will create a stmt), when a long session(that is what you talk about a connection will not disconnect) used, the pgpool child process(which handle the connection) will use too much memory and cpu resource. |
|
|
Hanyugang01, Thanks for that. Did you take any action to prevent that? The only thing I can think of is changing applications to disconnect from time to time and then configure pgpool to recycle processes after N connections, this can be done with the parameter child_max_connections. |
|
|
In our environment, we need a daemon process to fetch data from the message queue, and then import data to postgresql. The dataset is huge, so connection which disconnected from time to time is not acceptable, we just using the pgpool to get the master postgresql instance, and then connect to the master postgresql server directly. |
|
|
Could you provide a test program to reproduce this problem? |
|
|
hello, U can use this test tool to reproduce this problem. Usage of this tool: -d string database name (default "test") -h string host (default "localhost") -n int insert number (default 10000) -p string port (default "9999") -t string test type (default "good") -u string user (default "postgres") -w string passwd (default "postgres") specially, -t parameter includes "good" and "bad" , U can switch this parameter to compare this two situation. By the way, I use the flame tool to inspect the bad situation, I found that in bad situation these three functions consume too much CPU time("can_query_context_destroy" && "pool_remove_sent_message"&&"pool_get_sent_message" ) Thanks very much! |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2018-10-19 05:18 | rafaelthca | New Issue | |
| 2018-10-19 10:38 | hanyugang01 | Note Added: 0002217 | |
| 2018-10-19 12:15 | rafaelthca | Note Added: 0002218 | |
| 2018-10-19 17:38 | hanyugang01 | Note Added: 0002221 | |
| 2018-10-22 09:54 | pengbo | Note Added: 0002225 | |
| 2018-10-24 19:30 | hanyugang01 | File Added: pgpool_darwin | |
| 2018-10-24 19:30 | hanyugang01 | File Added: pgpool_linux | |
| 2018-10-24 19:30 | hanyugang01 | File Added: pgpool_windows.exe | |
| 2018-10-24 19:30 | hanyugang01 | File Added: pgpool.go | |
| 2018-10-24 19:30 | hanyugang01 | File Added: pgpool_in_bad_situation_flame.svg | |
| 2018-10-24 19:30 | hanyugang01 | Note Added: 0002228 | |
| 2018-11-20 11:39 | pengbo | Assigned To | => pengbo |
| 2018-11-20 11:39 | pengbo | Status | new => assigned |