View Issue Details

IDProjectCategoryView StatusLast Update
0000633Pgpool-IIBugpublic2020-08-24 10:08
Reporterpgdude Assigned Tohoshiai  
PrioritynormalSeverityminorReproducibilityalways
Status feedbackResolutionopen 
PlatformlinuxOSubuntuOS Version20.04
Product Version4.1.2 
Summary0000633: Getting random, but frequent "failed to unlock semaphore" errors
DescriptionI am using postgresql 12. 3. I am using pgpool2 only in connection pooling and load balancing mode with one primary and one standby. Watchdog is turned off. pgpool2 and postgresql are on the same host.

I get these messages frequently in pgpool.log file on both primary and standby hosts.

Aug 17 12:06:56 ltpgsql11 pgpool[19549]: 2020-08-17 12:06:56 pid: 19558 db: postgres usr: postgres LOCATION: pool_proto_modules.c:3348
Aug 17 12:06:56 ltpgsql11 pgpool[19549]: 2020-08-17 12:06:56 pid: 19558 db: postgres usr: postgres WARNING: failed to lock semaphore error:"Invalid argument"
Aug 17 12:06:56 ltpgsql11 pgpool[19549]: 2020-08-17 12:06:56 pid: 19558 db: postgres usr: postgres CONTEXT: while searching system catalog, When relcache is missed
Aug 17 12:06:56 ltpgsql11 pgpool[19549]: 2020-08-17 12:06:56 pid: 19558 db: postgres usr: postgres LOCATION: pool_sema.c:129
Aug 17 12:06:56 ltpgsql11 pgpool[19549]: 2020-08-17 12:06:56 pid: 19558 db: postgres usr: postgres WARNING: failed to unlock semaphore error:"Invalid argument"
Aug 17 12:06:56 ltpgsql11 pgpool[19549]: 2020-08-17 12:06:56 pid: 19558 db: postgres usr: postgres CONTEXT: while searching system catalog, When relcache is missed
Aug 17 12:06:56 ltpgsql11 pgpool[19549]: 2020-08-17 12:06:56 pid: 19558 db: postgres usr: postgres LOCATION: pool_sema.c:158
Aug 17 12:06:56 ltpgsql11 pgpool[19549]: 2020-08-17 12:06:56 pid: 19558 db: postgres usr: postgres LOG: DB node id: 0 backend pid: 3587 statement: create database testing;
Aug 17 12:06:56 ltpgsql11 pgpool[19549]: 2020-08-17 12:06:56 pid: 19558 db: postgres usr: postgres LOCATION: pool_proto_modules.c:3348


Steps To ReproduceJust issue normal sql statements and you will see these error messages spitting out.
TagsNo tags attached.

Activities

pgdude

2020-08-18 08:08

reporter   ~0003474

psql -U postgres -p 5433 -h myprimaryhost -c "SHOW pool_cache"

WARNING: failed to lock semaphore error:"Invalid argument"
WARNING: failed to unlock semaphore error:"Invalid argument"
WARNING: failed to lock semaphore error:"Invalid argument"
WARNING: failed to unlock semaphore error:"Invalid argument"
 num_cache_hits | num_selects | cache_hit_ratio | num_hash_entries | used_hash_entries | num_cache_entries | used_cache_entries_size | free_cache_entries_size | fragment_cache_entries_size
----------------+-------------+-----------------+------------------+-------------------+-------------------+-------------------------+-------------------------+-----------------------------
 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0 | 0

hoshiai

2020-08-18 17:49

developer   ~0003476

I think that we need more details.

Do this problem reappear with after pgpool restart?
And, I would like to need more information.
Could you share logs of pgpool which is started with '--debug' option (debug mode) and your pgpool.conf file.

pgdude

2020-08-18 22:36

reporter   ~0003477

Yes it reappears after a pgpool restart.

when you say start with --debug option, is that the same as setting client_min_message and log_min_message to one of the debug levels?

pgdude

2020-08-18 23:19

reporter   ~0003482

I will get the debug logs to you in a few hours...

pgdude

2020-08-19 05:47

reporter   ~0003483

The source code for this error is in src/utils/pool_sema.c in function, pool_semaphore_lock(). No interesting comments here either.

It does not adversely affect anything that I am aware of, and when i turn on more debugging levels, debug1, debug2, it goes away.

So if you want to close this, be my guest.

hoshiai

2020-08-21 16:37

developer   ~0003494

Ok, I recognized that this problem is not happened currently.

If this problem is not reproduced by you, I will close this ticket. Please let me know If there is any new update.

pgdude

2020-08-22 03:12

reporter   ~0003497

Actually I still get these warnings sporadically, sometimes they come out of a psql session in stdout/stderr and not just in the pgpool.log.

hoshiai

2020-08-24 10:08

developer   ~0003506

If you can get this warning sporadically, we may be able to know this cause by comparing WARNINIG happend log with not WARNING happend log, with debug mode. For example, it may only occur with a particular semaphore lock.

Issue History

Date Modified Username Field Change
2020-08-18 07:48 pgdude New Issue
2020-08-18 08:08 pgdude Note Added: 0003474
2020-08-18 17:13 hoshiai Assigned To => hoshiai
2020-08-18 17:13 hoshiai Status new => assigned
2020-08-18 17:49 hoshiai Status assigned => feedback
2020-08-18 17:49 hoshiai Note Added: 0003476
2020-08-18 22:36 pgdude Note Added: 0003477
2020-08-18 22:36 pgdude Status feedback => assigned
2020-08-18 23:19 pgdude Note Added: 0003482
2020-08-19 05:47 pgdude Note Added: 0003483
2020-08-21 16:37 hoshiai Status assigned => feedback
2020-08-21 16:37 hoshiai Note Added: 0003494
2020-08-22 03:12 pgdude Note Added: 0003497
2020-08-22 03:12 pgdude Status feedback => assigned
2020-08-24 10:08 hoshiai Status assigned => feedback
2020-08-24 10:08 hoshiai Note Added: 0003506