View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0000108Pgpool-IIBugpublic2014-07-30 17:572015-07-15 20:05
Reporterekass Assigned ToMuhammad Usama  
PrioritynormalSeveritycrashReproducibilityalways
Status closedResolutionunable to reproduce 
Platformamd64OSDebianOS VersionWheezy
Summary0000108: PGPool crashes on pcp_attach_node after all backends have been detached
Descriptionpgpool2 v.3.3.3 with postgresql-9.3
pgpool2 configured in master slave mode with streaming replication

It seems that pgpool crashes when I detach all backends and then try to reattach them. Maybe similar to bug 0000032.
Steps To Reproduce$ pcp_detach_node 10 localhost 9898 pgpool pgpool 2
$ pcp_detach_node 10 localhost 9898 pgpool pgpool 1
$ pcp_detach_node 10 localhost 9898 pgpool pgpool 0
$ pcp_node_info 10 localhost 9898 pgpool pgpool 2
192.168.0.11 5432 3 0.333333
$ pcp_node_info 10 localhost 9898 pgpool pgpool 1
192.168.0.6 5432 3 0.333333
$ pcp_node_info 10 localhost 9898 pgpool pgpool 0
192.168.0.3 5432 3 0.333333

$ pcp_attach_node 10 localhost 9898 pgpool pgpool 0
BackendError
$ pgpool stop
2014-07-30 11:24:16 DEBUG: pid 2738: key: pid_file_name
2014-07-30 11:24:16 DEBUG: pid 2738: value: '/var/run/pgpool/pgpool.pid' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: logdir
2014-07-30 11:24:16 DEBUG: pid 2738: value: '/tmp' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: connection_cache
2014-07-30 11:24:16 DEBUG: pid 2738: value: on kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: key: reset_query_list
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'ABORT; DISCARD ALL' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: extract_string_tokens: token: ABORT
2014-07-30 11:24:16 DEBUG: pid 2738: extract_string_tokens: token: DISCARD ALL
2014-07-30 11:24:16 DEBUG: pid 2738: key: replication_mode
2014-07-30 11:24:16 DEBUG: pid 2738: value: off kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: key: replicate_select
2014-07-30 11:24:16 DEBUG: pid 2738: value: off kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: replicate_select: 0
2014-07-30 11:24:16 DEBUG: pid 2738: key: insert_lock
2014-07-30 11:24:16 DEBUG: pid 2738: value: on kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: key: lobj_lock_table
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: replication_stop_on_mismatch
2014-07-30 11:24:16 DEBUG: pid 2738: value: off kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: replication_stop_on_mismatch: 0
2014-07-30 11:24:16 DEBUG: pid 2738: key: failover_if_affected_tuples_mismatch
2014-07-30 11:24:16 DEBUG: pid 2738: value: off kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: failover_if_affected_tuples_mismatch: 0
2014-07-30 11:24:16 DEBUG: pid 2738: key: load_balance_mode
2014-07-30 11:24:16 DEBUG: pid 2738: value: on kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: key: ignore_leading_white_space
2014-07-30 11:24:16 DEBUG: pid 2738: value: on kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: key: white_function_list
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: black_function_list
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'nextval,setval' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: extract_string_tokens: token: nextval
2014-07-30 11:24:16 DEBUG: pid 2738: extract_string_tokens: token: setval
2014-07-30 11:24:16 DEBUG: pid 2738: add_to_patterns: regex pattern: ^nextval$
2014-07-30 11:24:16 DEBUG: pid 2738: add_to_patterns: regex pattern: ^setval$
2014-07-30 11:24:16 DEBUG: pid 2738: key: master_slave_mode
2014-07-30 11:24:16 DEBUG: pid 2738: value: on kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: key: master_slave_sub_mode
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'stream' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: sr_check_period
2014-07-30 11:24:16 DEBUG: pid 2738: value: 0 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: sr_check_user
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'rep' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: sr_check_password
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'rep' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: delay_threshold
2014-07-30 11:24:16 DEBUG: pid 2738: value: 0 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: follow_master_command
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: parallel_mode
2014-07-30 11:24:16 DEBUG: pid 2738: value: off kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: key: pgpool2_hostname
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: system_db_hostname
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'localhost' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: system_db_port
2014-07-30 11:24:16 DEBUG: pid 2738: value: 5432 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: system_db_dbname
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'pgpool' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: system_db_schema
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'pgpool_catalog' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: system_db_user
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'pgpool' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: system_db_password
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: health_check_period
2014-07-30 11:24:16 DEBUG: pid 2738: value: 30 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: health_check_timeout
2014-07-30 11:24:16 DEBUG: pid 2738: value: 20 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: health_check_user
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'reposhare' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: health_check_password
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'rep0sh@re' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: health_check_max_retries
2014-07-30 11:24:16 DEBUG: pid 2738: value: 2 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: health_check_retry_delay
2014-07-30 11:24:16 DEBUG: pid 2738: value: 1 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: failover_command
2014-07-30 11:24:16 DEBUG: pid 2738: value: '/var/lib/postgresql/bin/failover.sh %d %H %m %M >> /var/log/postgresql/failover.log' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: failback_command
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: fail_over_on_backend_error
2014-07-30 11:24:16 DEBUG: pid 2738: value: off kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: key: search_primary_node_timeout
2014-07-30 11:24:16 DEBUG: pid 2738: value: 10 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: recovery_user
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'nobody' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: recovery_password
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: recovery_1st_stage_command
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: recovery_2nd_stage_command
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: recovery_timeout
2014-07-30 11:24:16 DEBUG: pid 2738: value: 90 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: client_idle_limit_in_recovery
2014-07-30 11:24:16 DEBUG: pid 2738: value: 0 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: use_watchdog
2014-07-30 11:24:16 DEBUG: pid 2738: value: off kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: key: trusted_servers
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: ping_path
2014-07-30 11:24:16 DEBUG: pid 2738: value: '/bin' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: wd_hostname
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: wd_port
2014-07-30 11:24:16 DEBUG: pid 2738: value: 9000 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: wd_authkey
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: delegate_IP
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: ifconfig_path
2014-07-30 11:24:16 DEBUG: pid 2738: value: '/sbin' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: if_up_cmd
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'ifconfig eth0:0 inet $_IP_$ netmask 255.255.255.0' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: if_down_cmd
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'ifconfig eth0:0 down' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: arping_path
2014-07-30 11:24:16 DEBUG: pid 2738: value: '/usr/sbin' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: arping_cmd
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'arping -U $_IP_$ -w 1' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: clear_memqcache_on_escalation
2014-07-30 11:24:16 DEBUG: pid 2738: value: on kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: key: wd_escalation_command
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: wd_lifecheck_method
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'heartbeat' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: wd_interval
2014-07-30 11:24:16 DEBUG: pid 2738: value: 10 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: wd_heartbeat_port
2014-07-30 11:24:16 DEBUG: pid 2738: value: 9694 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: wd_heartbeat_keepalive
2014-07-30 11:24:16 DEBUG: pid 2738: value: 2 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: wd_heartbeat_deadtime
2014-07-30 11:24:16 DEBUG: pid 2738: value: 30 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: heartbeat_destination0
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'host0_ip1' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: heartbeat_destination_port0
2014-07-30 11:24:16 DEBUG: pid 2738: value: 9694 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: heartbeat_device0
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: wd_life_point
2014-07-30 11:24:16 DEBUG: pid 2738: value: 3 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: wd_lifecheck_query
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'SELECT 1' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: wd_lifecheck_dbname
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'template1' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: wd_lifecheck_user
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'nobody' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: wd_lifecheck_password
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: relcache_expire
2014-07-30 11:24:16 DEBUG: pid 2738: value: 0 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: relcache_size
2014-07-30 11:24:16 DEBUG: pid 2738: value: 256 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: check_temp_table
2014-07-30 11:24:16 DEBUG: pid 2738: value: on kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: key: memory_cache_enabled
2014-07-30 11:24:16 DEBUG: pid 2738: value: off kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: key: memqcache_method
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'shmem' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: memqcache_memcached_host
2014-07-30 11:24:16 DEBUG: pid 2738: value: 'localhost' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: memqcache_memcached_port
2014-07-30 11:24:16 DEBUG: pid 2738: value: 11211 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: memqcache_total_size
2014-07-30 11:24:16 DEBUG: pid 2738: value: 67108864 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: memqcache_max_num_cache
2014-07-30 11:24:16 DEBUG: pid 2738: value: 1000000 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: memqcache_expire
2014-07-30 11:24:16 DEBUG: pid 2738: value: 0 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: memqcache_auto_cache_invalidation
2014-07-30 11:24:16 DEBUG: pid 2738: value: on kind: 1
2014-07-30 11:24:16 DEBUG: pid 2738: key: memqcache_maxcache
2014-07-30 11:24:16 DEBUG: pid 2738: value: 409600 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: memqcache_cache_block_size
2014-07-30 11:24:16 DEBUG: pid 2738: value: 1048576 kind: 2
2014-07-30 11:24:16 DEBUG: pid 2738: key: memqcache_oiddir
2014-07-30 11:24:16 DEBUG: pid 2738: value: '/var/log/pgpool/oiddir' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: white_memqcache_table_list
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: key: black_memqcache_table_list
2014-07-30 11:24:16 DEBUG: pid 2738: value: '' kind: 4
2014-07-30 11:24:16 DEBUG: pid 2738: num_backends: 3 total_weight: 3.000000
2014-07-30 11:24:16 DEBUG: pid 2738: backend 0 weight: 715827882.333333
2014-07-30 11:24:16 DEBUG: pid 2738: backend 0 flag: 0000
2014-07-30 11:24:16 DEBUG: pid 2738: backend 1 weight: 715827882.333333
2014-07-30 11:24:16 DEBUG: pid 2738: backend 1 flag: 0000
2014-07-30 11:24:16 DEBUG: pid 2738: backend 2 weight: 715827882.333333
2014-07-30 11:24:16 DEBUG: pid 2738: backend 2 flag: 0000
stop request sent to pgpool. waiting for termination.........................................................................................^C
$ ps aux | grep pgpool
root 8674 0.0 0.0 7836 828 pts/0 S+ 11:24 0:00 grep pgpool
root 22189 0.1 0.1 80256 5972 ? Ss Jul29 0:59 pgpool
root 27018 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27019 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27020 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27021 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27022 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27023 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27024 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27025 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27026 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27027 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27028 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27029 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27030 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27031 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27032 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27033 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27034 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27035 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27036 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27037 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27038 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27039 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27040 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27041 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27042 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27043 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27044 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27045 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27046 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27047 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27048 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27049 0.0 0.0 0 0 ? Z 10:41 0:00 [pgpool] <defunct>
root 27050 1.5 0.0 80472 2428 ? S 10:41 0:40 pgpool: PCP: wait for connection request
root 27051 0.0 0.0 80472 2248 ? S 10:41 0:00 pgpool: worker process
TagsNo tags attached.

Activities

Muhammad Usama

2015-05-01 22:04

developer   ~0000533

I have tried to reproduce this issue with the current head of pgpool-II 3.3 and the it is not reproducible. Can you please confirm if you are still getting the crash?

ekass

2015-07-02 22:26

reporter   ~0000546

Sorry for the delay, no I can no longer reproduce this. I have made various changes to the failover script that I used since then, so this may have been related to a failover script not exiting that caused pgpool to freeze. Also I noticed that while the failover script was executed I could not connect to pgpool admin or do anything else. Is this behavior normal?

Muhammad Usama

2015-07-15 20:05

developer   ~0000547

Thanks for the confirmation. Yes, it is the expected behaviour of pgpool-II, It does not allow the connections while the failover is in progress.

Issue History

Date Modified Username Field Change
2014-07-30 17:57 ekass New Issue
2014-07-31 17:27 t-ishii Assigned To => Muhammad Usama
2014-07-31 17:27 t-ishii Status new => assigned
2015-05-01 22:04 Muhammad Usama Note Added: 0000533
2015-06-29 18:27 Muhammad Usama Status assigned => feedback
2015-07-02 22:26 ekass Note Added: 0000546
2015-07-02 22:26 ekass Status feedback => assigned
2015-07-15 20:05 Muhammad Usama Note Added: 0000547
2015-07-15 20:05 Muhammad Usama Status assigned => closed
2015-07-15 20:05 Muhammad Usama Resolution open => unable to reproduce