[Pgpool-general] Cannot add node after failure
Fernando Morgenstern
fernando at consultorpc.com
Sun Dec 13 17:38:13 UTC 2009
Hello,
Thanks for your answer. I am sending the full log attached and some
additional info. I intentionally changed the postgres ports on all
nodes to 4000. Just to make things simplier.
Clear log and start pgpool
[root at im-pp2 ~]# cat /dev/null > /tmp/pgpool.log
[root at im-pp2 ~]# pgpool -n -d > /tmp/pgpool.log 2>&1 &
[1] 13214
Connect to pp3 and pp4
[root at im-pp2 ~]# su - postgres
[postgres at im-pp2 ~]$ psql -U postgres -h im-pp3 -p 4000 template1
psql (8.4.1)
Type "help" for help.
template1=# \q
[postgres at im-pp2 ~]$ psql -U postgres -h im-pp4 -p 4000 template1
psql (8.4.1)
Type "help" for help.
template1=# \q
Show pgpool conf
[postgres at im-pp2 ~]$ cat /usr/local/etc/pgpool.conf
#
# pgpool-II configuration file sample
# $Header: /cvsroot/pgpool/pgpool-II/pgpool.conf.sample,v 1.29
2009/12/06 08:46:34 t-ishii Exp $
# Host name or IP address to listen on: '*' for all, '' for no TCP/IP
# connections
listen_addresses = '*'
# Port number for pgpool
port = 5432
# Port number for pgpool communication manager
pcp_port = 9898
# Unix domain socket path. (The Debian package defaults to
# /var/run/postgresql.)
socket_dir = '/tmp'
# Unix domain socket path for pgpool communication manager.
# (Debian package defaults to /var/run/postgresql)
pcp_socket_dir = '/tmp'
# Unix domain socket path for the backend. Debian package defaults to /
var/run/postgresql!
backend_socket_dir = '/tmp'
# pgpool communication manager timeout. 0 means no timeout, but
strongly not recommended!
pcp_timeout = 10
# number of pre-forked child process
num_init_children = 300
# Number of connection pools allowed for a child process
max_pool = 2
# If idle for this many seconds, child exits. 0 means no timeout.
child_life_time = 300
# If idle for this many seconds, connection to PostgreSQL closes.
# 0 means no timeout.
connection_life_time = 0
# If child_max_connections connections were received, child exits.
# 0 means no exit.
child_max_connections = 0
# If client_idle_limit is n (n > 0), the client is forced to be
# disconnected whenever after n seconds idle (even inside an explicit
# transactions!)
# 0 means no disconnect.
client_idle_limit = 0
# Maximum time in seconds to complete client authentication.
# 0 means no timeout.
authentication_timeout = 60
# Logging directory
logdir = '/tmp'
# pid file name
pid_file_name = '/var/run/pgpool/pgpool.pid'
# Replication mode
replication_mode = true
# Load balancing mode, i.e., all SELECTs are load balanced.
# This is ignored if replication_mode is false.
load_balance_mode = true
# if there's a data mismatch between master and secondary
# start degeneration to stop replication mode
replication_stop_on_mismatch = false
# If true, replicate SELECT statement when load balancing is disabled.
# If false, it is only sent to the master node.
replicate_select = false
# Semicolon separated list of queries to be issued at the end of a
# session
reset_query_list = 'ABORT; DISCARD ALL'
# for 8.2 or older this should be as follows.
#reset_query_list = 'ABORT; RESET ALL; SET SESSION AUTHORIZATION
DEFAULT'
# If true print timestamp on each log line.
print_timestamp = true
# If true, operate in master/slave mode.
master_slave_mode = false
# If true, cache connection pool.
connection_cache = true
# Health check timeout. 0 means no timeout.
health_check_timeout = 20
# Health check period. 0 means no health check.
health_check_period = 30
# Health check user
health_check_user = 'postgres'
# Execute command by failover.
# special values: %d = node id
# %h = host name
# %p = port number
# %D = database cluster path
# %m = new master node id
# %M = old master node id
# %% = '%' character
#
failover_command = 'echo host:%h, new master id:%m, old master id:%M
> /tmp/failover.log'
# Execute command by failback.
# special values: %d = node id
# %h = host name
# %p = port number
# %D = database cluster path
# %m = new master node id
# %M = old master node id
# %% = '%' character
#
failback_command = 'echo host:%h, new master id:%m, old master id:%M
> /tmp/failback.log'
# If true, trigger fail over when writing to the backend communication
# socket fails. This is the same behavior of pgpool-II 2.2.x or
# earlier. If set to false, pgpool will report an error and disconnect
# the session.
fail_over_on_backend_error = true
# If true, automatically locks a table with INSERT statements to keep
# SERIAL data consistency. If the data does not have SERIAL data
# type, no lock will be issued. An /*INSERT LOCK*/ comment has the
# same effect. A /NO INSERT LOCK*/ comment disables the effect.
insert_lock = true
# If true, ignore leading white spaces of each query while pgpool judges
# whether the query is a SELECT so that it can be load balanced. This
# is useful for certain APIs such as DBI/DBD which is known to adding an
# extra leading white space.
ignore_leading_white_space = true
# If true, print all statements to the log. Like the log_statement
option
# to PostgreSQL, this allows for observing queries without engaging in
full
# debugging.
log_statement = false
# If true, print all statements to the log. Similar to log_statement
except
# that prints DB node id and backend process id info.
log_per_node_statement = false
# If true, incoming connections will be printed to the log.
log_connections = false
# If true, hostname will be shown in ps status. Also shown in
# connection log if log_connections = true.
# Be warned that this feature will add overhead to look up hostname.
log_hostname = false
# if non 0, run in parallel query mode
parallel_mode = false
# if non 0, use query cache
enable_query_cache = false
#set pgpool2 hostname
pgpool2_hostname = ''
# system DB info
system_db_hostname = 'localhost'
system_db_port = 4002
system_db_dbname = 'pgpool'
system_db_schema = 'public'
system_db_user = 'postgres'
system_db_password = ''
# backend_hostname, backend_port, backend_weight
# - HBA -
# If true, use pool_hba.conf for client authentication. In pgpool-II
# 1.1, the default value is false. The default value will be true in
# 1.2.
enable_pool_hba = true
# - online recovery -
# online recovery user
#recovery_user = 'postgres'
# online recovery password
#recovery_password = ''
# execute a command in first stage.
#recovery_1st_stage_command = 'copy_base_backup'
# execute a command in second stage.
#recovery_2nd_stage_command = 'pgpool_recovery_pitr'
# maximum time in seconds to wait for the recovering node's postmaster
# start-up. 0 means no wait.
# this is also used as a timer waiting for clients disconnected before
# starting 2nd stage
#recovery_timeout = 90
# If client_idle_limit_in_recovery is n (n > 0), the client is forced
# to be disconnected whenever after n seconds idle (even inside an
# explicit transactions!) 0 means no disconnect. This parameter only
# takes effect in recovery 2nd stage.
client_idle_limit_in_recovery = 0
replication_timeout = 5000
backend_hostname0 = 'localhost'
backend_port0 = 4000
backend_weight0 = 10
backend_data_directory0 = '/usr/local/pgpool/data-pp2'
backend_hostname1 = 'im-pp3'
backend_port1 = 4000
backend_weight1 = 10
backend_data_directory1 = '/usr/local/pgpool/data-pp3'
backend_hostname2 = 'im-pp1'
backend_port2 = 4000
backend_weight2 = 10
backend_data_directory2 = '/usr/local/pgpool/data-pp1'
backend_hostname3 = 'im-pp4'
backend_port3 = 4000
backend_weight3 = 10
backend_data_directory3 = '/usr/local/pgpool/data-pp4'
Full pgpool log
[postgres at im-pp2 ~]$ cat /tmp/pgpool.log
2009-12-13 10:31:27 DEBUG: pid 13214: key: listen_addresses
2009-12-13 10:31:27 DEBUG: pid 13214: value: '*' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: port
2009-12-13 10:31:27 DEBUG: pid 13214: value: 5432 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: pcp_port
2009-12-13 10:31:27 DEBUG: pid 13214: value: 9898 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: socket_dir
2009-12-13 10:31:27 DEBUG: pid 13214: value: '/tmp' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: pcp_socket_dir
2009-12-13 10:31:27 DEBUG: pid 13214: value: '/tmp' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_socket_dir
2009-12-13 10:31:27 DEBUG: pid 13214: value: '/tmp' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: pcp_timeout
2009-12-13 10:31:27 DEBUG: pid 13214: value: 10 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: num_init_children
2009-12-13 10:31:27 DEBUG: pid 13214: value: 300 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: max_pool
2009-12-13 10:31:27 DEBUG: pid 13214: value: 2 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: child_life_time
2009-12-13 10:31:27 DEBUG: pid 13214: value: 300 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: connection_life_time
2009-12-13 10:31:27 DEBUG: pid 13214: value: 0 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: child_max_connections
2009-12-13 10:31:27 DEBUG: pid 13214: value: 0 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: client_idle_limit
2009-12-13 10:31:27 DEBUG: pid 13214: value: 0 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: authentication_timeout
2009-12-13 10:31:27 DEBUG: pid 13214: value: 60 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: logdir
2009-12-13 10:31:27 DEBUG: pid 13214: value: '/tmp' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: pid_file_name
2009-12-13 10:31:27 DEBUG: pid 13214: value: '/var/run/pgpool/
pgpool.pid' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: replication_mode
2009-12-13 10:31:27 DEBUG: pid 13214: value: true kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: load_balance_mode
2009-12-13 10:31:27 DEBUG: pid 13214: value: true kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: replication_stop_on_mismatch
2009-12-13 10:31:27 DEBUG: pid 13214: value: false kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: replication_stop_on_mismatch: 0
2009-12-13 10:31:27 DEBUG: pid 13214: key: replicate_select
2009-12-13 10:31:27 DEBUG: pid 13214: value: false kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: replicate_select: 0
2009-12-13 10:31:27 DEBUG: pid 13214: key: reset_query_list
2009-12-13 10:31:27 DEBUG: pid 13214: value: 'ABORT; DISCARD ALL'
kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: extract_string_tokens: token:
ABORT
2009-12-13 10:31:27 DEBUG: pid 13214: extract_string_tokens: token:
DISCARD ALL
2009-12-13 10:31:27 DEBUG: pid 13214: key: print_timestamp
2009-12-13 10:31:27 DEBUG: pid 13214: value: true kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: master_slave_mode
2009-12-13 10:31:27 DEBUG: pid 13214: value: false kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: connection_cache
2009-12-13 10:31:27 DEBUG: pid 13214: value: true kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: health_check_timeout
2009-12-13 10:31:27 DEBUG: pid 13214: value: 20 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: health_check_period
2009-12-13 10:31:27 DEBUG: pid 13214: value: 30 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: health_check_user
2009-12-13 10:31:27 DEBUG: pid 13214: value: 'postgres' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: failover_command
2009-12-13 10:31:27 DEBUG: pid 13214: value: 'echo host:%h, new master
id:%m, old master id:%M > /tmp/failover.log' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: failback_command
2009-12-13 10:31:27 DEBUG: pid 13214: value: 'echo host:%h, new master
id:%m, old master id:%M > /tmp/failback.log' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: fail_over_on_backend_error
2009-12-13 10:31:27 DEBUG: pid 13214: value: true kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: insert_lock
2009-12-13 10:31:27 DEBUG: pid 13214: value: true kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: ignore_leading_white_space
2009-12-13 10:31:27 DEBUG: pid 13214: value: true kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: log_statement
2009-12-13 10:31:27 DEBUG: pid 13214: value: false kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: log_per_node_statement
2009-12-13 10:31:27 DEBUG: pid 13214: value: false kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: log_connections
2009-12-13 10:31:27 DEBUG: pid 13214: value: false kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: log_hostname
2009-12-13 10:31:27 DEBUG: pid 13214: value: false kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: parallel_mode
2009-12-13 10:31:27 DEBUG: pid 13214: value: false kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: enable_query_cache
2009-12-13 10:31:27 DEBUG: pid 13214: value: false kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: pgpool2_hostname
2009-12-13 10:31:27 DEBUG: pid 13214: value: '' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: system_db_hostname
2009-12-13 10:31:27 DEBUG: pid 13214: value: 'localhost' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: system_db_port
2009-12-13 10:31:27 DEBUG: pid 13214: value: 4002 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: system_db_dbname
2009-12-13 10:31:27 DEBUG: pid 13214: value: 'pgpool' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: system_db_schema
2009-12-13 10:31:27 DEBUG: pid 13214: value: 'public' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: system_db_user
2009-12-13 10:31:27 DEBUG: pid 13214: value: 'postgres' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: system_db_password
2009-12-13 10:31:27 DEBUG: pid 13214: value: '' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: enable_pool_hba
2009-12-13 10:31:27 DEBUG: pid 13214: value: true kind: 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: client_idle_limit_in_recovery
2009-12-13 10:31:27 DEBUG: pid 13214: value: 0 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: replication_timeout
2009-12-13 10:31:27 DEBUG: pid 13214: value: 5000 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_hostname0
2009-12-13 10:31:27 DEBUG: pid 13214: value: 'localhost' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_port0
2009-12-13 10:31:27 DEBUG: pid 13214: value: 4000 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: pool_config: port slot number 0
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_weight0
2009-12-13 10:31:27 DEBUG: pid 13214: value: 10 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: pool_config: weight slot number
0 weight: 10.000000
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_data_directory0
2009-12-13 10:31:27 DEBUG: pid 13214: value: '/usr/local/pgpool/data-
pp2' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_hostname1
2009-12-13 10:31:27 DEBUG: pid 13214: value: 'im-pp3' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_port1
2009-12-13 10:31:27 DEBUG: pid 13214: value: 4000 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: pool_config: port slot number 1
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_weight1
2009-12-13 10:31:27 DEBUG: pid 13214: value: 10 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: pool_config: weight slot number
1 weight: 10.000000
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_data_directory1
2009-12-13 10:31:27 DEBUG: pid 13214: value: '/usr/local/pgpool/data-
pp3' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_hostname2
2009-12-13 10:31:27 DEBUG: pid 13214: value: 'im-pp1' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_port2
2009-12-13 10:31:27 DEBUG: pid 13214: value: 4000 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: pool_config: port slot number 2
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_weight2
2009-12-13 10:31:27 DEBUG: pid 13214: value: 10 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: pool_config: weight slot number
2 weight: 10.000000
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_data_directory2
2009-12-13 10:31:27 DEBUG: pid 13214: value: '/usr/local/pgpool/data-
pp1' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_hostname3
2009-12-13 10:31:27 DEBUG: pid 13214: value: 'im-pp4' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_port3
2009-12-13 10:31:27 DEBUG: pid 13214: value: 4000 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: pool_config: port slot number 3
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_weight3
2009-12-13 10:31:27 DEBUG: pid 13214: value: 10 kind: 2
2009-12-13 10:31:27 DEBUG: pid 13214: pool_config: weight slot number
3 weight: 10.000000
2009-12-13 10:31:27 DEBUG: pid 13214: key: backend_data_directory3
2009-12-13 10:31:27 DEBUG: pid 13214: value: '/usr/local/pgpool/data-
pp4' kind: 4
2009-12-13 10:31:27 DEBUG: pid 13214: num_backends: 4 num_backends: 4
total_weight: 40.000000
2009-12-13 10:31:27 DEBUG: pid 13214: backend 0 weight: 536870911.750000
2009-12-13 10:31:27 DEBUG: pid 13214: backend 1 weight: 536870911.750000
2009-12-13 10:31:27 DEBUG: pid 13214: backend 2 weight: 536870911.750000
2009-12-13 10:31:27 DEBUG: pid 13214: backend 3 weight: 536870911.750000
2009-12-13 10:31:27 DEBUG: pid 13214: loading "/usr/local/etc/
pool_hba.conf" for client authentication configuration file
2009-12-13 10:31:27 DEBUG: pid 13215: I am 13215
2009-12-13 10:31:27 DEBUG: pid 13216: I am 13216
2009-12-13 10:31:27 DEBUG: pid 13217: I am 13217
2009-12-13 10:31:27 DEBUG: pid 13218: I am 13218
2009-12-13 10:31:27 DEBUG: pid 13219: I am 13219
2009-12-13 10:31:27 DEBUG: pid 13220: I am 13220
2009-12-13 10:31:27 DEBUG: pid 13221: I am 13221
2009-12-13 10:31:27 DEBUG: pid 13222: I am 13222
2009-12-13 10:31:27 DEBUG: pid 13223: I am 13223
2009-12-13 10:31:27 DEBUG: pid 13224: I am 13224
2009-12-13 10:31:27 DEBUG: pid 13225: I am 13225
2009-12-13 10:31:27 DEBUG: pid 13226: I am 13226
2009-12-13 10:31:27 DEBUG: pid 13227: I am 13227
2009-12-13 10:31:27 DEBUG: pid 13228: I am 13228
2009-12-13 10:31:27 DEBUG: pid 13229: I am 13229
2009-12-13 10:31:27 DEBUG: pid 13230: I am 13230
2009-12-13 10:31:27 DEBUG: pid 13231: I am 13231
2009-12-13 10:31:27 DEBUG: pid 13232: I am 13232
2009-12-13 10:31:27 DEBUG: pid 13233: I am 13233
2009-12-13 10:31:27 DEBUG: pid 13234: I am 13234
2009-12-13 10:31:27 DEBUG: pid 13235: I am 13235
2009-12-13 10:31:27 DEBUG: pid 13236: I am 13236
2009-12-13 10:31:27 DEBUG: pid 13237: I am 13237
2009-12-13 10:31:27 DEBUG: pid 13238: I am 13238
2009-12-13 10:31:27 DEBUG: pid 13239: I am 13239
2009-12-13 10:31:27 DEBUG: pid 13240: I am 13240
2009-12-13 10:31:27 DEBUG: pid 13241: I am 13241
2009-12-13 10:31:27 DEBUG: pid 13242: I am 13242
2009-12-13 10:31:27 DEBUG: pid 13243: I am 13243
2009-12-13 10:31:27 DEBUG: pid 13244: I am 13244
2009-12-13 10:31:27 DEBUG: pid 13245: I am 13245
2009-12-13 10:31:27 DEBUG: pid 13246: I am 13246
2009-12-13 10:31:27 DEBUG: pid 13247: I am 13247
2009-12-13 10:31:27 DEBUG: pid 13248: I am 13248
2009-12-13 10:31:27 DEBUG: pid 13249: I am 13249
2009-12-13 10:31:27 DEBUG: pid 13250: I am 13250
2009-12-13 10:31:27 DEBUG: pid 13251: I am 13251
2009-12-13 10:31:27 DEBUG: pid 13252: I am 13252
2009-12-13 10:31:27 DEBUG: pid 13253: I am 13253
2009-12-13 10:31:27 DEBUG: pid 13254: I am 13254
2009-12-13 10:31:27 DEBUG: pid 13255: I am 13255
2009-12-13 10:31:27 DEBUG: pid 13256: I am 13256
2009-12-13 10:31:27 DEBUG: pid 13257: I am 13257
2009-12-13 10:31:27 DEBUG: pid 13258: I am 13258
2009-12-13 10:31:27 DEBUG: pid 13259: I am 13259
2009-12-13 10:31:27 DEBUG: pid 13260: I am 13260
2009-12-13 10:31:27 DEBUG: pid 13261: I am 13261
2009-12-13 10:31:27 DEBUG: pid 13262: I am 13262
2009-12-13 10:31:27 DEBUG: pid 13263: I am 13263
2009-12-13 10:31:27 DEBUG: pid 13264: I am 13264
2009-12-13 10:31:27 DEBUG: pid 13265: I am 13265
2009-12-13 10:31:27 DEBUG: pid 13266: I am 13266
2009-12-13 10:31:27 DEBUG: pid 13267: I am 13267
2009-12-13 10:31:27 DEBUG: pid 13268: I am 13268
2009-12-13 10:31:27 DEBUG: pid 13269: I am 13269
2009-12-13 10:31:27 DEBUG: pid 13270: I am 13270
2009-12-13 10:31:27 DEBUG: pid 13271: I am 13271
2009-12-13 10:31:27 DEBUG: pid 13272: I am 13272
2009-12-13 10:31:27 DEBUG: pid 13273: I am 13273
2009-12-13 10:31:27 DEBUG: pid 13274: I am 13274
2009-12-13 10:31:27 DEBUG: pid 13275: I am 13275
2009-12-13 10:31:27 DEBUG: pid 13276: I am 13276
2009-12-13 10:31:27 DEBUG: pid 13277: I am 13277
2009-12-13 10:31:27 DEBUG: pid 13278: I am 13278
2009-12-13 10:31:27 DEBUG: pid 13279: I am 13279
2009-12-13 10:31:27 DEBUG: pid 13280: I am 13280
2009-12-13 10:31:27 DEBUG: pid 13281: I am 13281
2009-12-13 10:31:27 DEBUG: pid 13282: I am 13282
2009-12-13 10:31:27 DEBUG: pid 13283: I am 13283
2009-12-13 10:31:27 DEBUG: pid 13284: I am 13284
2009-12-13 10:31:27 DEBUG: pid 13285: I am 13285
2009-12-13 10:31:27 DEBUG: pid 13286: I am 13286
2009-12-13 10:31:27 DEBUG: pid 13287: I am 13287
2009-12-13 10:31:27 DEBUG: pid 13288: I am 13288
2009-12-13 10:31:27 DEBUG: pid 13289: I am 13289
2009-12-13 10:31:27 DEBUG: pid 13290: I am 13290
2009-12-13 10:31:27 DEBUG: pid 13291: I am 13291
2009-12-13 10:31:27 DEBUG: pid 13292: I am 13292
2009-12-13 10:31:27 DEBUG: pid 13293: I am 13293
2009-12-13 10:31:27 DEBUG: pid 13294: I am 13294
2009-12-13 10:31:27 DEBUG: pid 13295: I am 13295
2009-12-13 10:31:27 DEBUG: pid 13296: I am 13296
2009-12-13 10:31:27 DEBUG: pid 13297: I am 13297
2009-12-13 10:31:27 DEBUG: pid 13298: I am 13298
2009-12-13 10:31:27 DEBUG: pid 13299: I am 13299
2009-12-13 10:31:27 DEBUG: pid 13300: I am 13300
2009-12-13 10:31:27 DEBUG: pid 13301: I am 13301
2009-12-13 10:31:27 DEBUG: pid 13302: I am 13302
2009-12-13 10:31:27 DEBUG: pid 13303: I am 13303
2009-12-13 10:31:27 DEBUG: pid 13304: I am 13304
2009-12-13 10:31:27 DEBUG: pid 13305: I am 13305
2009-12-13 10:31:27 DEBUG: pid 13306: I am 13306
2009-12-13 10:31:27 DEBUG: pid 13307: I am 13307
2009-12-13 10:31:27 DEBUG: pid 13308: I am 13308
2009-12-13 10:31:27 DEBUG: pid 13309: I am 13309
2009-12-13 10:31:27 DEBUG: pid 13310: I am 13310
2009-12-13 10:31:27 DEBUG: pid 13311: I am 13311
2009-12-13 10:31:27 DEBUG: pid 13312: I am 13312
2009-12-13 10:31:27 DEBUG: pid 13313: I am 13313
2009-12-13 10:31:27 DEBUG: pid 13314: I am 13314
2009-12-13 10:31:27 DEBUG: pid 13315: I am 13315
2009-12-13 10:31:27 DEBUG: pid 13316: I am 13316
2009-12-13 10:31:27 DEBUG: pid 13317: I am 13317
2009-12-13 10:31:27 DEBUG: pid 13318: I am 13318
2009-12-13 10:31:27 DEBUG: pid 13319: I am 13319
2009-12-13 10:31:27 DEBUG: pid 13320: I am 13320
2009-12-13 10:31:27 DEBUG: pid 13321: I am 13321
2009-12-13 10:31:27 DEBUG: pid 13322: I am 13322
2009-12-13 10:31:27 DEBUG: pid 13323: I am 13323
2009-12-13 10:31:27 DEBUG: pid 13324: I am 13324
2009-12-13 10:31:27 DEBUG: pid 13325: I am 13325
2009-12-13 10:31:27 DEBUG: pid 13326: I am 13326
2009-12-13 10:31:27 DEBUG: pid 13327: I am 13327
2009-12-13 10:31:27 DEBUG: pid 13328: I am 13328
2009-12-13 10:31:27 DEBUG: pid 13329: I am 13329
2009-12-13 10:31:27 DEBUG: pid 13330: I am 13330
2009-12-13 10:31:27 DEBUG: pid 13331: I am 13331
2009-12-13 10:31:27 DEBUG: pid 13332: I am 13332
2009-12-13 10:31:27 DEBUG: pid 13333: I am 13333
2009-12-13 10:31:27 DEBUG: pid 13334: I am 13334
2009-12-13 10:31:27 DEBUG: pid 13335: I am 13335
2009-12-13 10:31:27 DEBUG: pid 13336: I am 13336
2009-12-13 10:31:27 DEBUG: pid 13337: I am 13337
2009-12-13 10:31:27 DEBUG: pid 13338: I am 13338
2009-12-13 10:31:27 DEBUG: pid 13339: I am 13339
2009-12-13 10:31:27 DEBUG: pid 13340: I am 13340
2009-12-13 10:31:27 DEBUG: pid 13341: I am 13341
2009-12-13 10:31:27 DEBUG: pid 13342: I am 13342
2009-12-13 10:31:27 DEBUG: pid 13343: I am 13343
2009-12-13 10:31:27 DEBUG: pid 13344: I am 13344
2009-12-13 10:31:27 DEBUG: pid 13345: I am 13345
2009-12-13 10:31:27 DEBUG: pid 13346: I am 13346
2009-12-13 10:31:27 DEBUG: pid 13347: I am 13347
2009-12-13 10:31:27 DEBUG: pid 13348: I am 13348
2009-12-13 10:31:27 DEBUG: pid 13349: I am 13349
2009-12-13 10:31:27 DEBUG: pid 13350: I am 13350
2009-12-13 10:31:27 DEBUG: pid 13351: I am 13351
2009-12-13 10:31:27 DEBUG: pid 13352: I am 13352
2009-12-13 10:31:27 DEBUG: pid 13353: I am 13353
2009-12-13 10:31:27 DEBUG: pid 13354: I am 13354
2009-12-13 10:31:27 DEBUG: pid 13355: I am 13355
2009-12-13 10:31:27 DEBUG: pid 13356: I am 13356
2009-12-13 10:31:27 DEBUG: pid 13357: I am 13357
2009-12-13 10:31:27 DEBUG: pid 13358: I am 13358
2009-12-13 10:31:27 DEBUG: pid 13359: I am 13359
2009-12-13 10:31:27 DEBUG: pid 13360: I am 13360
2009-12-13 10:31:27 DEBUG: pid 13361: I am 13361
2009-12-13 10:31:27 DEBUG: pid 13362: I am 13362
2009-12-13 10:31:27 DEBUG: pid 13363: I am 13363
2009-12-13 10:31:27 DEBUG: pid 13364: I am 13364
2009-12-13 10:31:27 DEBUG: pid 13365: I am 13365
2009-12-13 10:31:27 DEBUG: pid 13366: I am 13366
2009-12-13 10:31:27 DEBUG: pid 13367: I am 13367
2009-12-13 10:31:27 DEBUG: pid 13368: I am 13368
2009-12-13 10:31:27 DEBUG: pid 13369: I am 13369
2009-12-13 10:31:27 DEBUG: pid 13370: I am 13370
2009-12-13 10:31:27 DEBUG: pid 13371: I am 13371
2009-12-13 10:31:27 DEBUG: pid 13372: I am 13372
2009-12-13 10:31:27 DEBUG: pid 13373: I am 13373
2009-12-13 10:31:27 DEBUG: pid 13374: I am 13374
2009-12-13 10:31:27 DEBUG: pid 13375: I am 13375
2009-12-13 10:31:27 DEBUG: pid 13376: I am 13376
2009-12-13 10:31:27 DEBUG: pid 13377: I am 13377
2009-12-13 10:31:27 DEBUG: pid 13378: I am 13378
2009-12-13 10:31:27 DEBUG: pid 13379: I am 13379
2009-12-13 10:31:27 DEBUG: pid 13380: I am 13380
2009-12-13 10:31:27 DEBUG: pid 13381: I am 13381
2009-12-13 10:31:27 DEBUG: pid 13382: I am 13382
2009-12-13 10:31:27 DEBUG: pid 13383: I am 13383
2009-12-13 10:31:27 DEBUG: pid 13384: I am 13384
2009-12-13 10:31:27 DEBUG: pid 13385: I am 13385
2009-12-13 10:31:27 DEBUG: pid 13386: I am 13386
2009-12-13 10:31:27 DEBUG: pid 13387: I am 13387
2009-12-13 10:31:27 DEBUG: pid 13388: I am 13388
2009-12-13 10:31:27 DEBUG: pid 13389: I am 13389
2009-12-13 10:31:27 DEBUG: pid 13390: I am 13390
2009-12-13 10:31:27 DEBUG: pid 13391: I am 13391
2009-12-13 10:31:27 DEBUG: pid 13392: I am 13392
2009-12-13 10:31:27 DEBUG: pid 13393: I am 13393
2009-12-13 10:31:27 DEBUG: pid 13394: I am 13394
2009-12-13 10:31:27 DEBUG: pid 13395: I am 13395
2009-12-13 10:31:27 DEBUG: pid 13396: I am 13396
2009-12-13 10:31:27 DEBUG: pid 13397: I am 13397
2009-12-13 10:31:27 DEBUG: pid 13398: I am 13398
2009-12-13 10:31:27 DEBUG: pid 13399: I am 13399
2009-12-13 10:31:27 DEBUG: pid 13400: I am 13400
2009-12-13 10:31:27 DEBUG: pid 13401: I am 13401
2009-12-13 10:31:27 DEBUG: pid 13402: I am 13402
2009-12-13 10:31:27 DEBUG: pid 13403: I am 13403
2009-12-13 10:31:27 DEBUG: pid 13404: I am 13404
2009-12-13 10:31:27 DEBUG: pid 13405: I am 13405
2009-12-13 10:31:27 DEBUG: pid 13406: I am 13406
2009-12-13 10:31:27 DEBUG: pid 13407: I am 13407
2009-12-13 10:31:27 DEBUG: pid 13408: I am 13408
2009-12-13 10:31:27 DEBUG: pid 13409: I am 13409
2009-12-13 10:31:27 DEBUG: pid 13410: I am 13410
2009-12-13 10:31:27 DEBUG: pid 13411: I am 13411
2009-12-13 10:31:27 DEBUG: pid 13412: I am 13412
2009-12-13 10:31:27 DEBUG: pid 13413: I am 13413
2009-12-13 10:31:27 DEBUG: pid 13414: I am 13414
2009-12-13 10:31:27 DEBUG: pid 13415: I am 13415
2009-12-13 10:31:27 DEBUG: pid 13416: I am 13416
2009-12-13 10:31:27 DEBUG: pid 13417: I am 13417
2009-12-13 10:31:27 DEBUG: pid 13418: I am 13418
2009-12-13 10:31:27 DEBUG: pid 13419: I am 13419
2009-12-13 10:31:27 DEBUG: pid 13420: I am 13420
2009-12-13 10:31:27 DEBUG: pid 13421: I am 13421
2009-12-13 10:31:27 DEBUG: pid 13422: I am 13422
2009-12-13 10:31:27 DEBUG: pid 13423: I am 13423
2009-12-13 10:31:27 DEBUG: pid 13424: I am 13424
2009-12-13 10:31:27 DEBUG: pid 13425: I am 13425
2009-12-13 10:31:27 DEBUG: pid 13426: I am 13426
2009-12-13 10:31:27 DEBUG: pid 13427: I am 13427
2009-12-13 10:31:27 DEBUG: pid 13428: I am 13428
2009-12-13 10:31:27 DEBUG: pid 13429: I am 13429
2009-12-13 10:31:27 DEBUG: pid 13430: I am 13430
2009-12-13 10:31:27 DEBUG: pid 13431: I am 13431
2009-12-13 10:31:27 DEBUG: pid 13432: I am 13432
2009-12-13 10:31:27 DEBUG: pid 13433: I am 13433
2009-12-13 10:31:27 DEBUG: pid 13434: I am 13434
2009-12-13 10:31:27 DEBUG: pid 13435: I am 13435
2009-12-13 10:31:27 DEBUG: pid 13436: I am 13436
2009-12-13 10:31:27 DEBUG: pid 13437: I am 13437
2009-12-13 10:31:27 DEBUG: pid 13438: I am 13438
2009-12-13 10:31:27 DEBUG: pid 13439: I am 13439
2009-12-13 10:31:27 DEBUG: pid 13440: I am 13440
2009-12-13 10:31:27 DEBUG: pid 13441: I am 13441
2009-12-13 10:31:27 DEBUG: pid 13442: I am 13442
2009-12-13 10:31:27 DEBUG: pid 13443: I am 13443
2009-12-13 10:31:27 DEBUG: pid 13444: I am 13444
2009-12-13 10:31:27 DEBUG: pid 13445: I am 13445
2009-12-13 10:31:27 DEBUG: pid 13446: I am 13446
2009-12-13 10:31:27 DEBUG: pid 13447: I am 13447
2009-12-13 10:31:27 DEBUG: pid 13448: I am 13448
2009-12-13 10:31:27 DEBUG: pid 13449: I am 13449
2009-12-13 10:31:27 DEBUG: pid 13450: I am 13450
2009-12-13 10:31:27 DEBUG: pid 13451: I am 13451
2009-12-13 10:31:27 DEBUG: pid 13452: I am 13452
2009-12-13 10:31:27 DEBUG: pid 13453: I am 13453
2009-12-13 10:31:27 DEBUG: pid 13454: I am 13454
2009-12-13 10:31:27 DEBUG: pid 13455: I am 13455
2009-12-13 10:31:27 DEBUG: pid 13456: I am 13456
2009-12-13 10:31:27 DEBUG: pid 13457: I am 13457
2009-12-13 10:31:27 DEBUG: pid 13458: I am 13458
2009-12-13 10:31:27 DEBUG: pid 13459: I am 13459
2009-12-13 10:31:27 DEBUG: pid 13460: I am 13460
2009-12-13 10:31:27 DEBUG: pid 13461: I am 13461
2009-12-13 10:31:27 DEBUG: pid 13462: I am 13462
2009-12-13 10:31:27 DEBUG: pid 13463: I am 13463
2009-12-13 10:31:27 DEBUG: pid 13464: I am 13464
2009-12-13 10:31:27 DEBUG: pid 13465: I am 13465
2009-12-13 10:31:27 DEBUG: pid 13466: I am 13466
2009-12-13 10:31:27 DEBUG: pid 13467: I am 13467
2009-12-13 10:31:27 DEBUG: pid 13468: I am 13468
2009-12-13 10:31:27 DEBUG: pid 13469: I am 13469
2009-12-13 10:31:27 DEBUG: pid 13470: I am 13470
2009-12-13 10:31:27 DEBUG: pid 13471: I am 13471
2009-12-13 10:31:27 DEBUG: pid 13472: I am 13472
2009-12-13 10:31:27 DEBUG: pid 13473: I am 13473
2009-12-13 10:31:27 DEBUG: pid 13474: I am 13474
2009-12-13 10:31:27 DEBUG: pid 13475: I am 13475
2009-12-13 10:31:27 DEBUG: pid 13476: I am 13476
2009-12-13 10:31:27 DEBUG: pid 13477: I am 13477
2009-12-13 10:31:27 DEBUG: pid 13478: I am 13478
2009-12-13 10:31:27 DEBUG: pid 13479: I am 13479
2009-12-13 10:31:27 DEBUG: pid 13480: I am 13480
2009-12-13 10:31:27 DEBUG: pid 13481: I am 13481
2009-12-13 10:31:27 DEBUG: pid 13482: I am 13482
2009-12-13 10:31:27 DEBUG: pid 13483: I am 13483
2009-12-13 10:31:27 DEBUG: pid 13484: I am 13484
2009-12-13 10:31:27 DEBUG: pid 13485: I am 13485
2009-12-13 10:31:27 DEBUG: pid 13486: I am 13486
2009-12-13 10:31:27 DEBUG: pid 13487: I am 13487
2009-12-13 10:31:27 DEBUG: pid 13488: I am 13488
2009-12-13 10:31:27 DEBUG: pid 13489: I am 13489
2009-12-13 10:31:27 DEBUG: pid 13490: I am 13490
2009-12-13 10:31:27 DEBUG: pid 13491: I am 13491
2009-12-13 10:31:27 DEBUG: pid 13492: I am 13492
2009-12-13 10:31:27 DEBUG: pid 13493: I am 13493
2009-12-13 10:31:27 DEBUG: pid 13494: I am 13494
2009-12-13 10:31:27 DEBUG: pid 13495: I am 13495
2009-12-13 10:31:27 DEBUG: pid 13496: I am 13496
2009-12-13 10:31:27 DEBUG: pid 13497: I am 13497
2009-12-13 10:31:27 DEBUG: pid 13498: I am 13498
2009-12-13 10:31:27 DEBUG: pid 13499: I am 13499
2009-12-13 10:31:27 DEBUG: pid 13500: I am 13500
2009-12-13 10:31:27 DEBUG: pid 13501: I am 13501
2009-12-13 10:31:27 DEBUG: pid 13502: I am 13502
2009-12-13 10:31:27 DEBUG: pid 13503: I am 13503
2009-12-13 10:31:27 DEBUG: pid 13504: I am 13504
2009-12-13 10:31:27 DEBUG: pid 13505: I am 13505
2009-12-13 10:31:27 DEBUG: pid 13506: I am 13506
2009-12-13 10:31:27 DEBUG: pid 13507: I am 13507
2009-12-13 10:31:27 DEBUG: pid 13508: I am 13508
2009-12-13 10:31:27 DEBUG: pid 13509: I am 13509
2009-12-13 10:31:27 DEBUG: pid 13510: I am 13510
2009-12-13 10:31:27 DEBUG: pid 13511: I am 13511
2009-12-13 10:31:27 DEBUG: pid 13512: I am 13512
2009-12-13 10:31:27 DEBUG: pid 13513: I am 13513
2009-12-13 10:31:27 DEBUG: pid 13514: I am 13514
2009-12-13 10:31:27 LOG: pid 13214: pgpool successfully started
2009-12-13 10:31:27 DEBUG: pid 13515: I am PCP 13515
2009-12-13 10:31:27 DEBUG: pid 13214: starting health checking
2009-12-13 10:31:27 DEBUG: pid 13214: health_check: 0 th DB node
status: 3
2009-12-13 10:31:27 DEBUG: pid 13214: health_check: 1 th DB node
status: 3
2009-12-13 10:31:27 DEBUG: pid 13214: health_check: 2 th DB node
status: 3
2009-12-13 10:31:27 DEBUG: pid 13214: health_check: 3 th DB node
status: 3
2009-12-13 10:31:57 DEBUG: pid 13214: starting health checking
2009-12-13 10:31:57 DEBUG: pid 13214: health_check: 0 th DB node
status: 3
2009-12-13 10:31:57 DEBUG: pid 13214: health_check: 1 th DB node
status: 3
2009-12-13 10:31:57 DEBUG: pid 13214: health_check: 2 th DB node
status: 3
2009-12-13 10:31:57 DEBUG: pid 13214: health_check: 3 th DB node
status: 3
2009-12-13 10:32:27 DEBUG: pid 13214: starting health checking
2009-12-13 10:32:27 DEBUG: pid 13214: health_check: 0 th DB node
status: 3
2009-12-13 10:32:27 DEBUG: pid 13214: health_check: 1 th DB node
status: 3
2009-12-13 10:32:27 DEBUG: pid 13214: health_check: 2 th DB node
status: 3
2009-12-13 10:32:27 DEBUG: pid 13214: health_check: 3 th DB node
status: 3
2009-12-13 10:32:57 DEBUG: pid 13214: starting health checking
2009-12-13 10:32:57 DEBUG: pid 13214: health_check: 0 th DB node
status: 3
2009-12-13 10:32:57 DEBUG: pid 13214: health_check: 1 th DB node
status: 3
2009-12-13 10:32:57 DEBUG: pid 13214: health_check: 2 th DB node
status: 3
2009-12-13 10:32:57 DEBUG: pid 13214: health_check: 3 th DB node
status: 3
2009-12-13 10:33:27 DEBUG: pid 13214: starting health checking
2009-12-13 10:33:27 DEBUG: pid 13214: health_check: 0 th DB node
status: 3
2009-12-13 10:33:27 DEBUG: pid 13214: health_check: 1 th DB node
status: 3
2009-12-13 10:33:27 DEBUG: pid 13214: health_check: 2 th DB node
status: 3
2009-12-13 10:33:27 DEBUG: pid 13214: health_check: 3 th DB node
status: 3
2009-12-13 10:33:57 DEBUG: pid 13214: starting health checking
2009-12-13 10:33:57 DEBUG: pid 13214: health_check: 0 th DB node
status: 3
2009-12-13 10:33:57 DEBUG: pid 13214: health_check: 1 th DB node
status: 3
2009-12-13 10:33:57 DEBUG: pid 13214: health_check: 2 th DB node
status: 3
2009-12-13 10:33:57 DEBUG: pid 13214: health_check: 3 th DB node
status: 3
2009-12-13 10:34:27 DEBUG: pid 13214: starting health checking
2009-12-13 10:34:27 DEBUG: pid 13214: health_check: 0 th DB node
status: 3
2009-12-13 10:34:27 DEBUG: pid 13214: health_check: 1 th DB node
status: 3
2009-12-13 10:34:27 DEBUG: pid 13214: health_check: 2 th DB node
status: 3
2009-12-13 10:34:27 DEBUG: pid 13214: health_check: 3 th DB node
status: 3
2009-12-13 10:34:57 DEBUG: pid 13214: starting health checking
2009-12-13 10:34:57 DEBUG: pid 13214: health_check: 0 th DB node
status: 3
2009-12-13 10:34:57 DEBUG: pid 13214: health_check: 1 th DB node
status: 3
2009-12-13 10:34:57 DEBUG: pid 13214: health_check: 2 th DB node
status: 3
2009-12-13 10:34:57 DEBUG: pid 13214: health_check: 3 th DB node
status: 3
---
Fernando Marcelo
www.consultorpc.com
fernando at consultorpc.com
Em 13/12/2009, às 06:38, Tatsuo Ishii escreveu:
>> Hello,
>>
>> I started to use pgpool a few days ago and, while testing it, i am
>> having some issues to bring nodes back.
>>
>> I have 4 nodes running, so i decided to stop one of them. I saw on
>> logs that its status changed to 3. Ok, perfect.
>>
>> Them, i started the failed node again, but its status was still 3.
>
> This is an expected behavior. pgpool does not re-connect recovered
> node. If pgpool does this, pgpool will repeat connect/disconnect node
> forever if it's connected through a flakey network.
>
>> I decided to stop and start pgpool again, now all nodes have status
>> 3:
>>
>> 2009-12-11 11:29:28 DEBUG: pid 1107: health_check: 0 th DB node
>> status: 3
>> 2009-12-11 11:29:28 DEBUG: pid 1107: health_check: 1 th DB node
>> status: 3
>> 2009-12-11 11:29:28 DEBUG: pid 1107: health_check: 2 th DB node
>> status: 3
>> 2009-12-11 11:29:28 DEBUG: pid 1107: health_check: 3 th DB node
>> status: 3
>
> This is not what I'm expecting. Can you double check you can connect
> to the DB nodes by:
>
> psql -U postgres -h your_db_host_name -p its_port_number template1:
>
> For example,
>
> psql -U postgres -h im-pp3 -p 4003 template1
>
> Also can you show me the all the log after restarting pgool?
> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
More information about the Pgpool-general
mailing list