[pgpool-general: 1225] Re: pgpool 3.2.1 - Health check failing to connect

Wed Dec 5 08:17:07 JST 2012

Great. We alreay have similar bug report from BSD user
(http://www.pgpool.net/mantisbt/view.php?id=29). Since OS X is based
on BSD, it is not surprising that you have same problem. I asked the
reporter this question:

		 "Socket is already connected": what is the error no and error
		 symbol(macro)? I couldn't find it on Linux.

and got no answer so far. I need this info to fix the problem. Do you have any idea?
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

> OSX Snow Leopard Server
> 
> 
> 
> -----Original Message-----
> From: Tatsuo Ishii [mailto:ishii at postgresql.org]
> Sent: Tuesday, December 04, 2012 2:59 PM
> To: Mikola Rose
> Cc: pgpool-general at pgpool.net
> Subject: Re: [pgpool-general: 1218] pgpool 3.2.1 - Health check failing to connect
> 
>> this is what i have which just repeats
>>
>> ec  4 14:36:41 pgpool1 pgpool[70606]: connect_inet_domain_socket:
>> connect() failed: Socket is already connected
> 
> This is it. Since pgpool-II 3.2, it uses a non blocking socket to connect to backend.  I suspect the implementation has difficulty with some platforms. What OS/version are you using?
> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> English: http://www.sraoss.co.jp/index_en.php
> Japanese: http://www.sraoss.co.jp
> 
>> Dec  4 14:36:41 pgpool1 pgpool[70606]: make_persistent_db_connection:
>> connection to prod1.db.power-soft.net(5433) failed Dec  4 14:36:41
>> pgpool1 pgpool[70606]: health check failed. 0 th host
>> prod1.db.power-soft.net at port 5433 is down Dec  4 14:36:41 pgpool1
>> pgpool[70606]: health_check: 0 failover is canceld because failover is
>> disallowed Dec  4 14:36:44 pgpool1 pgpool[70606]:
>> connect_inet_domain_socket: connect() failed: Socket is already
>> connected Dec  4 14:36:44 pgpool1 pgpool[70606]:
>> make_persistent_db_connection: connection to
>> prod1.db.power-soft.net(5433) failed Dec  4 14:36:44 pgpool1
>> pgpool[70606]: health check failed. 0 th host prod1.db.power-soft.net
>> at port 5433 is down Dec  4 14:36:44 pgpool1 pgpool[70606]:
>> health_check: 0 failover is canceld because failover is disallowed Dec
>> 4 14:36:45 pgpool1 pgpool[70619]: connect_inet_domain_socket:
>> connect() failed: Socket is already connected Dec  4 14:36:45 pgpool1
>> pgpool[70619]: make_persistent_db_connection: connection to
>> prod1.db.power-soft.net(5433) failed Dec  4 14:36:45 pgpool1
>> pgpool[70619]: connect_inet_domain_socket: connect() failed: Socket is
>> already connected Dec  4 14:36:45 pgpool1 pgpool[70619]:
>> make_persistent_db_connection: connection to
>> prod2.db.power-soft.net(5434) failed Dec  4 14:36:45 pgpool1
>> pgpool[70619]: check_replication_time_lag: could not connect to DB
>> node 0, check sr_check_user and sr_check_password
>>
>>
>>
>> On 2012-12-04, at 2:39 PM, Tatsuo Ishii <ishii at postgresql.org>
>>  wrote:
>>
>>> You should have more error messages before this:
>>>
>>>> Dec  4 11:49:47 pgpool1 pgpool[67018]:
>>>> make_persistent_db_connection: connection to
>>>> prod1.db.power-soft.net<http://prod1.db.power-soft.net>(5433) failed
>>>
>>> Can you show it?
>>> --
>>> Tatsuo Ishii
>>> SRA OSS, Inc. Japan
>>> English: http://www.sraoss.co.jp/index_en.php
>>> Japanese: http://www.sraoss.co.jp
>>>
>>>> Hello List,
>>>>
>>>> I am having some difficulty with heath check in pgpool. It appears that it can not make a persistent connection so it fails health check.
>>>>
>>>> The servers are listed as up
>>>> prod1.db.power-soft.net5433Up[Detail]
>>>> prod2.db.power-soft.net5434Up[Detail]
>>>>
>>>> IP Address
>>>>
>>>> Port
>>>>
>>>> Status
>>>>
>>>> Weight
>>>>
>>>> prod1.db.power-soft.net<http://prod1.db.power-soft.net>
>>>>
>>>> 5433
>>>>
>>>> Up. Disconnect. Running as primary server
>>>>
>>>> 0.500
>>>>
>>>> [Disconnect]
>>>>
>>>> prod2.db.power-soft.net<http://prod2.db.power-soft.net>
>>>>
>>>> 5434
>>>>
>>>> Up. Disconnect. Running as standby server
>>>>
>>>> 0.500
>>>>
>>>> [Disconnect] [Promote]
>>>>
>>>>
>>>>
>>>> Dec  4 11:49:47 pgpool1 pgpool[67018]:
>>>> make_persistent_db_connection: connection to
>>>> prod1.db.power-soft.net<http://prod1.db.power-soft.net>(5433) failed
>>>> Dec  4 11:49:47 pgpool1 pgpool[67018]: health check failed. 0 th
>>>> host prod1.db.power-soft.net<http://prod1.db.power-soft.net> at port
>>>> 5433 is down Dec  4 11:49:56 pgpool1 pgpool[67030]:
>>>> check_replication_time_lag: could not connect to DB node 0, check
>>>> sr_check_user and sr_check_password
>>>>
>>>> I am using postgres/postgres for health_check_user/password which I can connect with manually to both the databases.
>>>> sr_check_user/password are using postgres/postgres as well.
>>>>
>>>>
>>>> Any suggestions out there?  The only thing I can find in the FAQ that is related to this is if the user does not exist that its trying to connect with and the return error is completely different.
>>>>
>>>> Both databases are postgresql 9.0.10
>>>>
>>>> pgpool is on a separate machine pgpool1.db.power-soft.net<http://pgpool1.db.power-soft.net>  there is no firewall blocking connections...
>>>>
>>>> If I enable failover it will make both prod1 and 2 down.
>>>>
>>>> here is the debug from a restart, if this helps any
>>>>
>>>>
>>>> Any help would be greatly appreciated!
>>>>
>>>>
>>>>
>>>> pgpool restart succeed
>>>>
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: listen_addresses
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '*' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: port
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 5432 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: socket_dir
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '/tmp' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: pcp_port
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 9898 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: pcp_socket_dir
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '/tmp' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: enable_pool_hba
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: on kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: authentication_timeout
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 60 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: ssl
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: off kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: num_init_children
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 10 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: max_pool
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 4 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: child_life_time
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 300 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: child_max_connections
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 0 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: connection_life_time
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 0 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: client_idle_limit
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 0 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: log_destination
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'syslog' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: print_timestamp
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: on kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: log_connections
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: on kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: log_hostname
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: off kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: log_statement
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: off kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: log_per_node_statement
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: off kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: log_standby_delay
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'if_over_threshold'
>>>> kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: syslog_facility
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'LOCAL0' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: syslog_ident
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'pgpool' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: debug_level
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 10 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: pid_file_name
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value:
>>>> '/var/run/pgpool/pgpool.pid' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: logdir
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '/tmp' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: connection_cache
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: on kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: reset_query_list
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'ABORT; DISCARD ALL'
>>>> kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: extract_string_tokens: token:
>>>> ABORT
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: extract_string_tokens: token:
>>>> DISCARD ALL
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: replication_mode
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: off kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: replicate_select
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: off kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: replicate_select: 0
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: insert_lock
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: off kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: lobj_lock_table
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key:
>>>> replication_stop_on_mismatch
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: off kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: replication_stop_on_mismatch:
>>>> 0
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key:
>>>> failover_if_affected_tuples_mismatch
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: off kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605:
>>>> failover_if_affected_tuples_mismatch: 0
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: load_balance_mode
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: on kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key:
>>>> ignore_leading_white_space
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: on kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: white_function_list
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: black_function_list
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value:
>>>> 'currval,lastval,nextval,setval' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: extract_string_tokens: token:
>>>> currval
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: extract_string_tokens: token:
>>>> lastval
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: extract_string_tokens: token:
>>>> nextval
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: extract_string_tokens: token:
>>>> setval
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: add_to_patterns: regex
>>>> pattern: ^currval$
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: add_to_patterns: regex
>>>> pattern: ^lastval$
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: add_to_patterns: regex
>>>> pattern: ^nextval$
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: add_to_patterns: regex
>>>> pattern: ^setval$
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: master_slave_mode
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: on kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: master_slave_sub_mode
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'stream' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: sr_check_period
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 5 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: sr_check_user
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'postgres' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: sr_check_password
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'postgres' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: delay_threshold
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 0 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: follow_master_command
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: parallel_mode
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: off kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: pgpool2_hostname
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: health_check_period
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 1 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: health_check_timeout
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 1 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: health_check_user
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'postgres' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: health_check_password
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'postgres' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: health_check_max_retries
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 3 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: health_check_retry_delay
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 5 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: failover_command
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value:
>>>> '/usr/local/pgsql/bin/failover_stream.sh %d %H
>>>> /db-array/db_repo/tmp/trigger_file0' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: failback_command
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key:
>>>> fail_over_on_backend_error
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: off kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: recovery_user
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'postgres' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: recovery_password
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'postgres' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key:
>>>> recovery_1st_stage_command
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key:
>>>> recovery_2nd_stage_command
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: recovery_timeout
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 90 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key:
>>>> client_idle_limit_in_recovery
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 0 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: use_watchdog
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: off kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: trusted_servers
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: delegate_IP
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: wd_hostname
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: wd_port
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 9000 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: wd_interval
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 10 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: ping_path
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '/bin' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: ifconfig_path
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '/sbin' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: if_up_cmd
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'ifconfig eth0:0 inet
>>>> $_IP_$ netmask 255.255.255.0' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: if_down_cmd
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'ifconfig eth0:0 down'
>>>> kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: arping_path
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '/usr/sbin # arping
>>>> command path' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: arping_cmd
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'arping -U $_IP_$ -w 1'
>>>> kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: wd_life_point
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 3 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: wd_lifecheck_query
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'SELECT 1' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: relcache_expire
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 0 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: relcache_size
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 256 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: check_temp_table
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: on kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: memory_cache_enabled
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: off kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: memqcache_method
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'shmem' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: memqcache_memcached_host
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'localhost' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: memqcache_memcached_port
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 11211 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: memqcache_total_size
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 67108864 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: memqcache_max_num_cache
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 1000000 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: memqcache_expire
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 0 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key:
>>>> memqcache_auto_cache_invalidation
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: on kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: memqcache_maxcache
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 409600 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key:
>>>> memqcache_cache_block_size
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 1048576 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: memqcache_oiddir
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value:
>>>> '/var/log/pgpool/oiddir' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key:
>>>> white_memqcache_table_list
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key:
>>>> black_memqcache_table_list
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: ssl_key
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: ssl_cert
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: ssl_ca_cert
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: ssl_ca_cert_dir
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: enable_query_cache
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: off kind: 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: system_db_hostname
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'system_db_hostname'
>>>> kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: system_db_port
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 5432 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: system_db_dbname
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'pgpool' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: system_db_schema
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'pgpool_catalog' kind:
>>>> 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: system_db_user
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'pgpool' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: system_db_password
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: backend_socket_dir
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: '/tmp' kind: 4
>>>> 2012-12-04 12:08:24 LOG: pid 70605: pool_config: backend_socket_dir
>>>> is deprecated, please use backend_hostname
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: backend_hostname0
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value:
>>>> 'prod1.db.power-soft.net<http://prod1.db.power-soft.net>' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: backend_port0
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 5433 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: pool_config: port slot number
>>>> 0
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: backend_weight0
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 1 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: pool_config: weight slot
>>>> number 0 weight: 1.000000
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: backend_data_directory0
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value:
>>>> '/db-array/db_repo/prod1/data' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: backend_flag0
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'DISALLOW_TO_FAILOVER'
>>>> kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: extract_string_tokens: token:
>>>> DISALLOW_TO_FAILOVER
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: pool_config:
>>>> disallow_to_failover on
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: pool_config: slot number 0
>>>> flag: 0001
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: backend_hostname1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value:
>>>> 'prod2.db.power-soft.net<http://prod2.db.power-soft.net>' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: backend_port1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 5434 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: pool_config: port slot number
>>>> 1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: backend_weight1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 1 kind: 2
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: pool_config: weight slot
>>>> number 1 weight: 1.000000
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: backend_data_directory1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value:
>>>> '/db-array/db_repo/prod2/data' kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: key: backend_flag1
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: value: 'DISALLOW_TO_FAILOVER'
>>>> kind: 4
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: extract_string_tokens: token:
>>>> DISALLOW_TO_FAILOVER
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: pool_config:
>>>> disallow_to_failover on
>>>> 2012-12-04 12:08:24 DEBUG: pid 70605: pool_config: slot number 1
>>>> flag: 0001
>>>>
>>>>
>>