View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000721 | Pgpool-II | Bug | public | 2021-07-15 10:52 | 2022-06-28 11:23 |
| Reporter | shutao917 | Assigned To | pengbo | ||
| Priority | immediate | Severity | major | Reproducibility | always |
| Status | closed | Resolution | open | ||
| OS | CentOS | OS Version | 7.6.1810 | ||
| Product Version | 4.2.2 | ||||
| Summary | 0000721: cpu load high frequently | ||||
| Description | we recently installed 4.2.2 in production env with parameter like below,about ervery 5 minutes cpu load get very high。we also use 4.0.5 for about 1 years with the same parameter but had no promble. num_init_children = 1800 max_pool = 1 # - Life time - child_life_time = 300 child_max_connections = 0 connection_life_time = 0 client_idle_limit = 0 | ||||
| Tags | No tags attached. | ||||
|
|
|
|
|
Could you share pgpool.conf? I want to find which Process is causing the high CPU usage. if it is possible, could you run the following command to check the CPU usage of each pgpool process? timeout 1800 sh -c "while :; do echo '' >> res.txt; date >> res.txt; ps auxwww | grep -e %CPU -e pgpool | grep -v grep | head -n 20 >> res.txt;sleep 60; done" If every 5 minutes CPU load gets very high, are you running particular queries every 5 minutes? Could you share the scenarios or queries issued by client to reproduce this issue? If you enable "log_per_node_statement" parameter, the client queries will be output to the log file. |
|
|
from the pgpool log file,i find two things: 1.a java promgram connection to pgpool and exec some pretty simple query,but not use connection pool。change it direct to postgres,and the promble sees to be impoved(see 1.jpg) 2.another promgram connection to an not exist database(see 2.jpg),we had not find which promgram it is yet.i think stop that,cpu load promble would be resolve Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348979-1] 2021-07-15 14:47:29: pid 22205: LOG: new connection received Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348979-2] 2021-07-15 14:47:29: pid 22205: DETAIL: connecting host=xxxxxxxxx port=57168 Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348980-1] 2021-07-15 14:47:29: pid 22205: LOG: backend throws an error message Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348980-2] 2021-07-15 14:47:29: pid 22205: DETAIL: database "pre_bizcenter_hmc" does not exist Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348981-1] 2021-07-15 14:47:29: pid 22205: ERROR: unable to forward message to frontend Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348981-2] 2021-07-15 14:47:29: pid 22205: DETAIL: FATAL error occured on backend so,i doubt is the child_life_time=300 leader this promble |
|
|
I am not sure if it is caused by Pgpool-II. > 2.another promgram connection to an not exist database(see 2.jpg),we had not find which promgram it is yet.i think stop that,cpu load promble would be resolve > Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348979-1] 2021-07-15 14:47:29: pid 22205: LOG: new connection received > Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348979-2] 2021-07-15 14:47:29: pid 22205: DETAIL: connecting host=xxxxxxxxx port=57168 > Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348980-1] 2021-07-15 14:47:29: pid 22205: LOG: backend throws an error message > Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348980-2] 2021-07-15 14:47:29: pid 22205: DETAIL: database "pre_bizcenter_hmc" does not exist > Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348981-1] 2021-07-15 14:47:29: pid 22205: ERROR: unable to forward message to frontend > Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348981-2] 2021-07-15 14:47:29: pid 22205: DETAIL: FATAL error occured on backend Does the ERROR above occur frequently? |
|
|
> 2.another promgram connection to an not exist database(see 2.jpg),we had not find which promgram it is yet.i think stop that,cpu load promble would be resolve > Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348979-1] 2021-07-15 14:47:29: pid 22205: LOG: new connection received > Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348979-2] 2021-07-15 14:47:29: pid 22205: DETAIL: connecting host=xxxxxxxxx port=57168 > Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348980-1] 2021-07-15 14:47:29: pid 22205: LOG: backend throws an error message > Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348980-2] 2021-07-15 14:47:29: pid 22205: DETAIL: database "pre_bizcenter_hmc" does not exist > Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348981-1] 2021-07-15 14:47:29: pid 22205: ERROR: unable to forward message to frontend > Jul 15 14:47:29 pg62 pgpool-prod-biz[22205]: [3348981-2] 2021-07-15 14:47:29: pid 22205: DETAIL: FATAL error occured on backend This is a flink job。when occur,up to 200/s |
|
|
Sorry. I am not sure if it's the cause of pgpool because I can't reproduce this issue. I think if you fix the program which is executing queries to database "pre_bizcenter_hmc", this issue might be resolved. |
|
|
No response over two weeks. Close issue. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2021-07-15 10:52 | shutao917 | New Issue | |
| 2021-07-15 10:52 | shutao917 | File Added: image.png | |
| 2021-07-15 10:52 | shutao917 | File Added: cpu load.jpg | |
| 2021-07-15 12:15 | pengbo | Assigned To | => pengbo |
| 2021-07-15 12:15 | pengbo | Status | new => assigned |
| 2021-07-15 12:15 | pengbo | Description Updated | |
| 2021-07-15 14:23 | pengbo | Note Added: 0003898 | |
| 2021-07-15 14:23 | pengbo | Status | assigned => feedback |
| 2021-07-15 16:49 | shutao917 | Note Added: 0003899 | |
| 2021-07-15 16:49 | shutao917 | File Added: 1.jpg | |
| 2021-07-15 16:49 | shutao917 | File Added: 2.jpg | |
| 2021-07-15 16:49 | shutao917 | Status | feedback => assigned |
| 2021-07-21 12:56 | pengbo | Note Added: 0003903 | |
| 2021-07-21 12:56 | pengbo | Status | assigned => feedback |
| 2021-07-21 20:17 | shutao917 | Note Added: 0003904 | |
| 2021-07-21 20:17 | shutao917 | Status | feedback => assigned |
| 2021-07-27 15:37 | pengbo | Note Added: 0003905 | |
| 2021-07-30 15:01 | pengbo | Status | assigned => feedback |
| 2022-06-28 11:23 | pengbo | Note Added: 0004075 | |
| 2022-06-28 11:23 | pengbo | Status | feedback => closed |