[pgpool-general: 8514] Re: Segfaults on PgPool-II 4.3.2

Tatsuo Ishii ishii at sraoss.co.jp
Fri Dec 9 10:23:20 JST 2022


> Hello PgPool Team,
> 
> We've been experiencing frequent segfaults from PgPool-II 4.3.2, sometimes
> as frequently as every 30min, when the cluster is under significant (though
> normal for us) load. We've been using pgpool without issue for a while, and
> these issues began occurring after we launched a cluster of all new
> machines with no changes on the pgpool machines, and updated shared_buffers
> (from "8000MB" to "14000MB" on machines with 32GB RAM) and decreased
> swappiness on the PSQL nodes (vm.swappiness=1 to vm.swappiness=0 in
> sysctl.conf). Following the segfaults the cluster appears to continue
> behaving normally, though it's still a bit early for us to know for sure.
> 
> Details follow including core dump backtraces from gdb for two segfaults
> we've observed recently (one in memcpy, another in
> pool_find_item_on_shmem_cache), including `bt` and `bt full`. Please let us
> know if there is any more information we can provide.
> 
> Thanks,
> Jordan
> 
> Environment details (ubuntu-focal-20.04-amd64):
> 
>    - Cluster contains 3 PgPool nodes and 3 PSQL nodes, all on AWS EC2
>    instances, pgpool.conf attached. Segfaults occurred on the leader at the
>    time.
>    - $ uname -a
>    Linux ip-172-30-64-83 5.4.0-1038-aws #40-Ubuntu SMP Fri Feb 5 23:50:40
>    UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
>    - # SHOW POOL_VERSION;
>         pool_version
>    -----------------------
>     4.3.2 (tamahomeboshi)
>    (1 row)
>    - In both cases the log line for segfault is like:
>    pgpool[45529]: segfault at 7ff9cfa1094e ip 000056215c2d53fe sp
>    00007ffe3e7db160 error 6 in pgpool[56215c28c000+d9000]
>    - Backtraces attached:
>    - segfault_memcpy_202212208_1500
>       - segfault_pool_find_item_on_shmem_cache_20221208_1500

The pgpool.conf is not for 4.3.x, though it's not directly related to
the issue, I think.

It seems all the segfaults are related to shared relcache, which was
introduced in 4.3. Can you disable the feature and see if the segfaults still occur?
You can do it by adding:

enable_shared_relcache = off

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp


More information about the pgpool-general mailing list