View Issue Details

IDProjectCategoryView StatusLast Update
0000612Pgpool-II[All Projects] Generalpublic2020-05-12 13:16
ReporterRaushanAssigned Topengbo 
PriorityurgentSeveritymajorReproducibilityalways
Status assignedResolutionopen 
Product Version3.4.26 
Target VersionFixed in Version 
Summary0000612: Getting connection error when I tried to connect with tcp port
DescriptionWhenever I tried to connect with tcp port , we always getting a connection . We doesn't understand why it is get.
Steps To Reproduce[promotion@WRCHDHS1 ~]$ pgpool –n >> /tmp/pgpool.log 2>&1 &
[1] 24091
[promotion@WRCHDHS1 ~]$ pcp_node_info 2 localhost 9898 postgres P0stgres 0
ConnectionError
[1]+ Exit 1 pgpool –n >> /tmp/pgpool.log 2>&1
[promotion@WRCHDHS1 ~]$ pcp_node_info 2 localhost 9898 postgres P0stgres 1
ConnectionError
[promotion@WRCHDHS1 ~]$ pcp_attach_node -d 2 localhost 9898 postgres P0stgres 0
DEBUG: could not connect to "localhost"
ConnectionError
[promotion@WRCHDHS1 ~]$ pcp_attach_node -d 2 localhost 9898 postgres P0stgres 1
DEBUG: could not connect to "localhost"
ConnectionError
[promotion@WRCHDHS1 ~]$ pgpool -m f stop
2020-05-07 17:35:29: pid 24139: FATAL: could not read pid file

Additional InformationHere I attached the pgpool.conf as well as pg'log

pgpool.conf

# ----------------------------
# pgPool-II configuration file
# ----------------------------
#
# This file consists of lines of the form:
#
# name = value
#
# Whitespace may be used. Comments are introduced with "#" anywhere on a line.
# The complete list of parameter names and allowed values can be found in the
# pgPool-II documentation.
#
# This file is read on server startup and when the server receives a SIGHUP
# signal. If you edit the file on a running system, you have to SIGHUP the
# server for the changes to take effect, or use "pgpool reload". Some
# parameters, which are marked below, require a server shutdown and restart to
# take effect.
#


#------------------------------------------------------------------------------
# CONNECTIONS
#------------------------------------------------------------------------------

# - pgpool Connection Settings -

listen_addresses = '172.16.222.131'
                                   # Host name or IP address to listen on:
                                   # '*' for all, '' for no TCP/IP connections
                                   # (change requires restart)
port = 9999
                                   # Port number
                                   # (change requires restart)
socket_dir = '/tmp'
                                   # Unix domain socket path
                                   # The Debian package defaults to
                                   # /var/run/postgresql
                                   # (change requires restart)


# - pgpool Communication Manager Connection Settings -

pcp_listen_addresses = '*'
                                   # Host name or IP address for pcp process to listen on:
                                   # '*' for all, '' for no TCP/IP connections
                                   # (change requires restart)
pcp_port = 9898
                                   # Port number for pcp
                                   # (change requires restart)
pcp_socket_dir = '/tmp'
                                   # Unix domain socket path for pcp
                                   # The Debian package defaults to
                                   # /var/run/postgresql
                                   # (change requires restart)
listen_backlog_multiplier = 2
                                   # Set the backlog parameter of listen(2) to
                                   # num_init_children * listen_backlog_multiplier.
                                   # (change requires restart)

# - Backend Connection Settings -

backend_hostname0 = '172.16.222.131'
                                   # Host name or IP address to connect to for backend 0
backend_port0 = 5432
                                   # Port number for backend 0
backend_weight0 = 1
                                   # Weight for backend 0 (only in load balancing mode)
backend_data_directory0 = '/var/lib/pgsql/9.4/data'
                                   # Data directory for backend 0
backend_flag0 = 'ALLOW_TO_FAILOVER'
                                   # Controls various backend behavior
                                   # ALLOW_TO_FAILOVER or DISALLOW_TO_FAILOVER
backend_hostname1 = '172.16.222.132'
backend_port1 = 5432
#backend_weight1 = 1
backend_data_directory1 = '/var/lib/pgsql/9.4/data'
backend_flag1 = 'ALLOW_TO_FAILOVER'

# - Authentication -

enable_pool_hba = off
                                   # Use pool_hba.conf for client authentication
pool_passwd = ''
                                   # File name of pool_passwd for md5 authentication.
                                   # "" disables pool_passwd.
                                   # (change requires restart)
authentication_timeout = 60
                                   # Delay in seconds to complete client authentication
                                   # 0 means no timeout.

# - SSL Connections -

ssl = off
                                   # Enable SSL support
                                   # (change requires restart)
#ssl_key = './server.key'
                                   # Path to the SSL private key file
                                   # (change requires restart)
#ssl_cert = './server.cert'
                                   # Path to the SSL public certificate file
                                   # (change requires restart)
#ssl_ca_cert = ''
                                   # Path to a single PEM format file
                                   # containing CA root certificate(s)
                                   # (change requires restart)
#ssl_ca_cert_dir = ''
                                   # Directory containing CA root certificate(s)
                                   # (change requires restart)


#------------------------------------------------------------------------------
# POOLS
#------------------------------------------------------------------------------

# - Pool size -

num_init_children = 3
                                   # Number of pools
                                   # (change requires restart)
max_pool = 1
                                   # Number of connections per pool
                                   # (change requires restart)

# - Life time -

child_life_time = 300
                                   # Pool exits after being idle for this many seconds
child_max_connections = 0
                                   # Pool exits after receiving that many connections
                                   # 0 means no exit
connection_life_time = 0
                                   # Connection to backend closes after being idle for this many seconds
                                   # 0 means no close
client_idle_limit = 0
                                   # Client is disconnected after being idle for that many seconds
                                   # (even inside an explicit transactions!)
                                   # 0 means no disconnection


#------------------------------------------------------------------------------
# LOGS
#------------------------------------------------------------------------------

# - Where to log -

log_destination = 'stderr'
                                   # Where to log
                                   # Valid values are combinations of stderr,
                                   # and syslog. Default to stderr.

# - What to log -

log_line_prefix = '%t: pid %p: ' # printf-style string to output at beginning of each log line.

log_connections = off
                                   # Log connections
log_hostname = off
                                   # Hostname will be shown in ps status
                                   # and in logs if connections are logged
log_statement = off
                                   # Log all statements
log_per_node_statement = off
                                   # Log all statements
                                   # with node and backend informations
log_standby_delay = 'if_over_threshold'
                                   # Log standby delay
                                   # Valid values are combinations of always,
                                   # if_over_threshold, none

# - Syslog specific -

syslog_facility = 'LOCAL0'
                                   # Syslog local facility. Default to LOCAL0
syslog_ident = 'pgpool'
                                   # Syslog program identification string
                                   # Default to 'pgpool'

# - Debug -

debug_level = 0
                                   # Debug message verbosity level
                                   # 0 means no message, 1 or more mean verbose

#log_error_verbosity = default # terse, default, or verbose messages

#client_min_messages = notice # values in order of decreasing detail:
                                        # debug5
                                        # debug4
                                        # debug3
                                        # debug2
                                        # debug1
                                        # log
                                        # notice
                                        # warning
                                        # error

#log_min_messages = warning # values in order of decreasing detail:
                                        # debug5
                                        # debug4
                                        # debug3
                                        # debug2
                                        # debug1
                                        # info
                                        # notice
                                        # warning
                                        # error
                                        # log
                                        # fatal
                                        # panic

#------------------------------------------------------------------------------
# FILE LOCATIONS
#------------------------------------------------------------------------------

pid_file_name = '/var/run/pgpool/pgpool.pid'
                                   # PID file name
                                   # (change requires restart)
logdir = '/tmp'
                                   # Directory of pgPool status file
                                   # (change requires restart)


#------------------------------------------------------------------------------
# CONNECTION POOLING
#------------------------------------------------------------------------------

connection_cache = off
                                   # Activate connection pools
                                   # (change requires restart)

                                   # Semicolon separated list of queries
                                   # to be issued at the end of a session
                                   # The default is for 8.3 and later
reset_query_list = 'ABORT; DISCARD ALL'
                                   # The following one is for 8.2 and before
#reset_query_list = 'ABORT; RESET ALL; SET SESSION AUTHORIZATION DEFAULT'


#------------------------------------------------------------------------------
# REPLICATION MODE
#------------------------------------------------------------------------------

replication_mode = off
                                   # Activate replication mode
                                   # (change requires restart)
replicate_select = off
                                   # Replicate SELECT statements
                                   # when in replication mode
                                   # replicate_select is higher priority than
                                   # load_balance_mode.

insert_lock = off
                                   # Automatically locks a dummy row or a table
                                   # with INSERT statements to keep SERIAL data
                                   # consistency
                                   # Without SERIAL, no lock will be issued
lobj_lock_table = ''
                                   # When rewriting lo_creat command in
                                   # replication mode, specify table name to
                                   # lock

# - Degenerate handling -

replication_stop_on_mismatch = off
                                   # On disagreement with the packet kind
                                   # sent from backend, degenerate the node
                                   # which is most likely "minority"
                                   # If off, just force to exit this session

failover_if_affected_tuples_mismatch = off
                                   # On disagreement with the number of affected
                                   # tuples in UPDATE/DELETE queries, then
                                   # degenerate the node which is most likely
                                   # "minority".
                                   # If off, just abort the transaction to
                                   # keep the consistency


#------------------------------------------------------------------------------
# LOAD BALANCING MODE
#------------------------------------------------------------------------------

load_balance_mode = off
                                   # Activate load balancing mode
                                   # (change requires restart)
ignore_leading_white_space = on
                                   # Ignore leading white spaces of each query
white_function_list = ''
                                   # Comma separated list of function names
                                   # that don't write to database
                                   # Regexp are accepted
black_function_list = 'currval,lastval,nextval,setval'
                                   # Comma separated list of function names
                                   # that write to database
                                   # Regexp are accepted

database_redirect_preference_list = ''
                                   # comma separated list of pairs of database and node id.
                                   # example: postgres:primary,mydb[0-4]:1,mydb[5-9]:2'
                                   # valid for streaming replicaton mode only.

app_name_redirect_preference_list = ''
                                   # comma separated list of pairs of app name and node id.
                                   # example: 'psql:primary,myapp[0-4]:1,myapp[5-9]:standby'
                                   # valid for streaming replicaton mode only.
allow_sql_comments = off
                                   # if on, ignore SQL comments when judging if load balance or
                                   # query cache is possible.
                                   # If off, SQL comments effectively prevent the judgment
                                   # (pre 3.4 behavior).

#------------------------------------------------------------------------------
# MASTER/SLAVE MODE
#------------------------------------------------------------------------------

master_slave_mode = on
                                   # Activate master/slave mode
                                   # (change requires restart)
master_slave_sub_mode = 'stream'
                                   # Master/slave sub mode
                                   # Valid values are combinations slony or
                                   # stream. Default is slony.
                                   # (change requires restart)

# - Streaming -

sr_check_period = 0
                                   # Streaming replication check period
                                   # Disabled (0) by default
sr_check_user = 'postgres'
                                   # Streaming replication check user
                                   # This is neccessary even if you disable streaming
                                   # replication delay check by sr_check_period = 0
sr_check_password = 'P0stgres'
                                   # Password for streaming replication check user
delay_threshold = 10000000
                                   # Threshold before not dispatching query to standby node
                                   # Unit is in bytes
                                   # Disabled (0) by default

# - Special commands -

follow_master_command = ''
                                   # Executes this command after master failover
                                   # Special values:
                                   # %d = node id
                                   # %h = host name
                                   # %p = port number
                                   # %D = database cluster path
                                   # %m = new master node id
                                   # %H = hostname of the new master node
                                   # %M = old master node id
                                   # %P = old primary node id
                                   # %r = new master port number
                                   # %R = new master database cluster path
                                   # %% = '%' character

#------------------------------------------------------------------------------
# HEALTH CHECK
#------------------------------------------------------------------------------

health_check_period = 2
                                   # Health check period
                                   # Disabled (0) by default
health_check_timeout = 10
                                   # Health check timeout
                                   # 0 means no timeout
health_check_user = 'postgres'
                                   # Health check user
health_check_password = 'P0stgres'
                                   # Password for health check user
health_check_max_retries = 0
                                   # Maximum number of times to retry a failed health check before giving up.
health_check_retry_delay = 1
                                   # Amount of time to wait (in seconds) between retries.
connect_timeout = 10000
                                   # Timeout value in milliseconds before giving up to connect to backend.
                                   # Default is 10000 ms (10 second). Flaky network user may want to increase
                                   # the value. 0 means no timeout.
                                   # Note that this value is not only used for health check,
                                   # but also for ordinary conection to backend.

#------------------------------------------------------------------------------
# FAILOVER AND FAILBACK
#------------------------------------------------------------------------------

failover_command = '/opt/phasorpoint-historian/bin/postgres_failover_script.sh %d %P %m %H %R'
                                   # Executes this command at failover
                                   # Special values:
                                   # %d = node id
                                   # %h = host name
                                   # %p = port number
                                   # %D = database cluster path
                                   # %m = new master node id
                                   # %H = hostname of the new master node
                                   # %M = old master node id
                                   # %P = old primary node id
                                   # %r = new master port number
                                   # %R = new master database cluster path
                                   # %% = '%' character
failback_command = ''
                                   # Executes this command at failback.
                                   # Special values:
                                   # %d = node id
                                   # %h = host name
                                   # %p = port number
                                   # %D = database cluster path
                                   # %m = new master node id
                                   # %H = hostname of the new master node
                                   # %M = old master node id
                                   # %P = old primary node id
                                   # %r = new master port number
                                   # %R = new master database cluster path
                                   # %% = '%' character

fail_over_on_backend_error = on
                                   # Initiates failover when reading/writing to the
                                   # backend communication socket fails
                                   # If set to off, pgpool will report an
                                   # error and disconnect the session.

search_primary_node_timeout = 10
                                   # Timeout in seconds to search for the
                                   # primary node when a failover occurs.
                                   # 0 means no timeout, keep searching
                                   # for a primary node forever.

#------------------------------------------------------------------------------
# ONLINE RECOVERY
#------------------------------------------------------------------------------

recovery_user = 'nobody'
                                   # Online recovery user
recovery_password = ''
                                   # Online recovery password
recovery_1st_stage_command = ''
                                   # Executes a command in first stage
recovery_2nd_stage_command = ''
                                   # Executes a command in second stage
recovery_timeout = 90
                                   # Timeout in seconds to wait for the
                                   # recovering node's postmaster to start up
                                   # 0 means no wait
client_idle_limit_in_recovery = 0
                                   # Client is disconnected after being idle
                                   # for that many seconds in the second stage
                                   # of online recovery
                                   # 0 means no disconnection
                                   # -1 means immediate disconnection


#------------------------------------------------------------------------------
# WATCHDOG
#------------------------------------------------------------------------------

# - Enabling -

use_watchdog = off
                                    # Activates watchdog
                                    # (change requires restart)

# -Connection to up stream servers -

trusted_servers = ''
                                    # trusted server list which are used
                                    # to confirm network connection
                                    # (hostA,hostB,hostC,...)
                                    # (change requires restart)
ping_path = '/bin'
                                    # ping command path
                                    # (change requires restart)

# - Watchdog communication Settings -

wd_hostname = ''
                                    # Host name or IP address of this watchdog
                                    # (change requires restart)
wd_port = 9000
                                    # port number for watchdog service
                                    # (change requires restart)
wd_authkey = ''
                                    # Authentication key for watchdog communication
                                    # (change requires restart)

# - Virtual IP control Setting -

delegate_IP = ''
                                    # delegate IP address
                                    # If this is empty, virtual IP never bring up.
                                    # (change requires restart)
ifconfig_path = '/sbin'
                                    # ifconfig command path
                                    # (change requires restart)
if_up_cmd = 'ifconfig eth0:0 inet $_IP_$ netmask 255.255.255.0'
                                    # startup delegate IP command
                                    # (change requires restart)
if_down_cmd = 'ifconfig eth0:0 down'
                                    # shutdown delegate IP command
                                    # (change requires restart)

arping_path = '/usr/sbin' # arping command path
                                    # (change requires restart)

arping_cmd = 'arping -U $_IP_$ -w 1'
                                    # arping command
                                    # (change requires restart)

# - Behaivor on escalation Setting -

clear_memqcache_on_escalation = on
                                    # Clear all the query cache on shared memory
                                    # when standby pgpool escalate to active pgpool
                                    # (= virtual IP holder).
                                    # This should be off if client connects to pgpool
                                    # not using virtual IP.
                                    # (change requires restart)
wd_escalation_command = ''
                                    # Executes this command at escalation on new active pgpool.
                                    # (change requires restart)

# - Lifecheck Setting -

# -- common --

wd_lifecheck_method = 'heartbeat'
                                    # Method of watchdog lifecheck ('heartbeat' or 'query')
                                    # (change requires restart)
wd_interval = 10
                                    # lifecheck interval (sec) > 0
                                    # (change requires restart)

# -- heartbeat mode --

wd_heartbeat_port = 9694
                                    # Port number for receiving heartbeat signal
                                    # (change requires restart)
wd_heartbeat_keepalive = 2
                                    # Interval time of sending heartbeat signal (sec)
                                    # (change requires restart)
wd_heartbeat_deadtime = 30
                                    # Deadtime interval for heartbeat signal (sec)
                                    # (change requires restart)
heartbeat_destination0 = 'host0_ip1'
                                    # Host name or IP address of destination 0
                                    # for sending heartbeat signal.
                                    # (change requires restart)
heartbeat_destination_port0 = 9694
                                    # Port number of destination 0 for sending
                                    # heartbeat signal. Usually this is the
                                    # same as wd_heartbeat_port.
                                    # (change requires restart)
heartbeat_device0 = ''
                                    # Name of NIC device (such like 'eth0')
                                    # used for sending/receiving heartbeat
                                    # signal to/from destination 0.
                                    # This works only when this is not empty
                                    # and pgpool has root privilege.
                                    # (change requires restart)

#heartbeat_destination1 = 'host0_ip2'
#heartbeat_destination_port1 = 9694
#heartbeat_device1 = ''

# -- query mode --

wd_life_point = 3
                                    # lifecheck retry times
                                    # (change requires restart)
wd_lifecheck_query = 'SELECT 1'
                                    # lifecheck query to pgpool from watchdog
                                    # (change requires restart)
wd_lifecheck_dbname = 'template1'
                                    # Database name connected for lifecheck
                                    # (change requires restart)
wd_lifecheck_user = 'nobody'
                                    # watchdog user monitoring pgpools in lifecheck
                                    # (change requires restart)
wd_lifecheck_password = ''
                                    # Password for watchdog user in lifecheck
                                    # (change requires restart)

# - Other pgpool Connection Settings -

#other_pgpool_hostname0 = 'host0'
                                    # Host name or IP address to connect to for other pgpool 0
                                    # (change requires restart)
#other_pgpool_port0 = 5432
                                    # Port number for othet pgpool 0
                                    # (change requires restart)
#other_wd_port0 = 9000
                                    # Port number for othet watchdog 0
                                    # (change requires restart)
#other_pgpool_hostname1 = 'host1'
#other_pgpool_port1 = 5432
#other_wd_port1 = 9000


#------------------------------------------------------------------------------
# OTHERS
#------------------------------------------------------------------------------
relcache_expire = 0
                                   # Life time of relation cache in seconds.
                                   # 0 means no cache expiration(the default).
                                   # The relation cache is used for cache the
                                   # query result against PostgreSQL system
                                   # catalog to obtain various information
                                   # including table structures or if it's a
                                   # temporary table or not. The cache is
                                   # maintained in a pgpool child local memory
                                   # and being kept as long as it survives.
                                   # If someone modify the table by using
                                   # ALTER TABLE or some such, the relcache is
                                   # not consistent anymore.
                                   # For this purpose, cache_expiration
                                   # controls the life time of the cache.
relcache_size = 256
                                   # Number of relation cache
                                   # entry. If you see frequently:
                                   # "pool_search_relcache: cache replacement happend"
                                   # in the pgpool log, you might want to increate this number.

check_temp_table = on
                                   # If on, enable temporary table check in SELECT statements.
                                   # This initiates queries against system catalog of primary/master
                                   # thus increases load of master.
                                   # If you are absolutely sure that your system never uses temporary tables
                                   # and you want to save access to primary/master, you could turn this off.
                                   # Default is on.

check_unlogged_table = on
                                   # If on, enable unlogged table check in SELECT statements.
                                   # This initiates queries against system catalog of primary/master
                                   # thus increases load of master.
                                   # If you are absolutely sure that your system never uses unlogged tables
                                   # and you want to save access to primary/master, you could turn this off.
                                   # Default is on.

#------------------------------------------------------------------------------
# IN MEMORY QUERY MEMORY CACHE
#------------------------------------------------------------------------------
memory_cache_enabled = off
                                   # If on, use the memory cache functionality, off by default
memqcache_method = 'shmem'
                                   # Cache storage method. either 'shmem'(shared memory) or
                                   # 'memcached'. 'shmem' by default
                                   # (change requires restart)
memqcache_memcached_host = 'localhost'
                                   # Memcached host name or IP address. Mandatory if
                                   # memqcache_method = 'memcached'.
                                   # Defaults to localhost.
                                   # (change requires restart)
memqcache_memcached_port = 11211
                                   # Memcached port number. Mondatory if memqcache_method = 'memcached'.
                                   # Defaults to 11211.
                                   # (change requires restart)
memqcache_total_size = 67108864
                                   # Total memory size in bytes for storing memory cache.
                                   # Mandatory if memqcache_method = 'shmem'.
                                   # Defaults to 64MB.
                                   # (change requires restart)
memqcache_max_num_cache = 1000000
                                   # Total number of cache entries. Mandatory
                                   # if memqcache_method = 'shmem'.
                                   # Each cache entry consumes 48 bytes on shared memory.
                                   # Defaults to 1,000,000(45.8MB).
                                   # (change requires restart)
memqcache_expire = 0
                                   # Memory cache entry life time specified in seconds.
                                   # 0 means infinite life time. 0 by default.
                                   # (change requires restart)
memqcache_auto_cache_invalidation = on
                                   # If on, invalidation of query cache is triggered by corresponding
                                   # DDL/DML/DCL(and memqcache_expire). If off, it is only triggered
                                   # by memqcache_expire. on by default.
                                   # (change requires restart)
memqcache_maxcache = 409600
                                   # Maximum SELECT result size in bytes.
                                   # Must be smaller than memqcache_cache_block_size. Defaults to 400KB.
                                   # (change requires restart)
memqcache_cache_block_size = 1048576
                                   # Cache block size in bytes. Mandatory if memqcache_method = 'shmem'.
                                   # Defaults to 1MB.
                                   # (change requires restart)
memqcache_oiddir = '/var/log/pgpool/oiddir'
                                      # Temporary work directory to record table oids
                                   # (change requires restart)
white_memqcache_table_list = ''
                                   # Comma separated list of table names to memcache
                                   # that don't write to database
                                   # Regexp are accepted
black_memqcache_table_list = ''
                                   # Comma separated list of table names not to memcache
                                   # that don't write to database
                                   # Regexp are accepted


PG_log

2020-05-06 18:11:27: pid 6112: LOG: Setting up socket for 172.16.222.131:9999
2020-05-06 18:11:27: pid 6112: LOG: pgpool-II successfully started. version 3.4.6 (tataraboshi)
2020-05-06 18:11:27: pid 6112: LOG: find_primary_node: checking backend no 0
    
2020-05-06 18:11:27: pid 6112: LOG: failed to connect to PostgreSQL server on "172.16.222.131:5432", getsockopt() detected error "Connection refused"
2020-05-06 18:11:27: pid 6112: ERROR: failed to make persistent db connection
2020-05-06 18:11:27: pid 6112: DETAIL: connection to host:"172.16.222.131:5432" failed
2020-05-06 18:11:27: pid 6112: LOG: find_primary_node: checking backend no 1
    
2020-05-06 18:11:27: pid 6112: LOG: failed to connect to PostgreSQL server on "172.16.222.132:5432", getsockopt() detected error "Connection refused"
2020-05-06 18:11:27: pid 6112: ERROR: failed to make persistent db connection
2020-05-06 18:11:27: pid 6112: DETAIL: connection to host:"172.16.222.132:5432" failed
2020-05-06 18:11:27: pid 6112: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:11:27: pid 6112: DETAIL: "Permission denied"
2020-05-06 18:11:27: pid 6112: LOG: failed to connect to PostgreSQL server on "172.16.222.131:5432", getsockopt() detected error "Connection refused"
2020-05-06 18:11:27: pid 6112: ERROR: failed to make persistent db connection
2020-05-06 18:11:27: pid 6112: DETAIL: connection to host:"172.16.222.131:5432" failed
2020-05-06 18:11:27: pid 6112: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:11:27: pid 6112: DETAIL: "Permission denied"
2020-05-06 18:11:27: pid 6112: LOG: failed to connect to PostgreSQL server on "172.16.222.131:5432", getsockopt() detected error "Connection refused"
2020-05-06 18:11:27: pid 6112: ERROR: failed to make persistent db connection
2020-05-06 18:11:27: pid 6112: DETAIL: connection to host:"172.16.222.131:5432" failed
2020-05-06 18:11:27: pid 6112: LOG: setting backend node 0 status to NODE DOWN
2020-05-06 18:11:27: pid 6112: LOG: starting degeneration. shutdown host 172.16.222.131(5432)
2020-05-06 18:11:27: pid 6112: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:11:27: pid 6112: DETAIL: "Permission denied"
2020-05-06 18:11:27: pid 6112: LOG: Restart all children
2020-05-06 18:11:27: pid 6112: LOG: execute command: /opt/phasorpoint-historian/bin/postgres_failover_script.sh 0 0 1 172.16.222.132 /var/lib/pgsql/9.4/data
2020-05-06 18:11:27: pid 6115: LOG: child process received shutdown request signal 3
2020-05-06 18:11:27: pid 6114: LOG: child process received shutdown request signal 3
2020-05-06 18:11:27: pid 6116: LOG: child process received shutdown request signal 3
----------------------------------------------------------------------
The primary node (node 0) is down.
Node 1 takes over as the primary.
Pseudo-terminal will not be allocated because stdin is not a terminal.
sudo: sorry, you must have a tty to run sudo
2020-05-06 18:11:27: pid 6112: LOG: failover: set new primary node: -1
2020-05-06 18:11:27: pid 6112: LOG: failover: set new master node: 1
failover done. shutdown host 172.16.222.131(5432)2020-05-06 18:11:27: pid 6112: LOG: failover done. shutdown host 172.16.222.131(5432)
2020-05-06 18:11:27: pid 6118: LOG: worker process received restart request
2020-05-06 18:11:28: pid 6117: LOG: restart request received in pcp child process
2020-05-06 18:11:28: pid 6112: LOG: PCP child 6117 exits with status 256 in failover()
2020-05-06 18:11:28: pid 6112: LOG: fork a new PCP child pid 6126 in failover()
2020-05-06 18:11:28: pid 6112: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:11:28: pid 6112: DETAIL: "Permission denied"
2020-05-06 18:11:28: pid 6112: LOG: failed to connect to PostgreSQL server on "172.16.222.132:5432", getsockopt() detected error "Connection refused"
2020-05-06 18:11:28: pid 6112: ERROR: failed to make persistent db connection
2020-05-06 18:11:28: pid 6112: DETAIL: connection to host:"172.16.222.132:5432" failed
2020-05-06 18:11:28: pid 6112: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:11:28: pid 6112: DETAIL: "Permission denied"
2020-05-06 18:11:28: pid 6112: LOG: child process with pid: 6114 exits with status 0
2020-05-06 18:11:28: pid 6112: LOG: child process with pid: 6114 exited with success and will not be restarted
2020-05-06 18:11:28: pid 6112: LOG: child process with pid: 6115 exits with status 0
2020-05-06 18:11:28: pid 6112: LOG: child process with pid: 6115 exited with success and will not be restarted
2020-05-06 18:11:28: pid 6112: LOG: child process with pid: 6116 exits with status 0
2020-05-06 18:11:28: pid 6112: LOG: child process with pid: 6116 exited with success and will not be restarted
2020-05-06 18:11:28: pid 6112: LOG: worker child process with pid: 6118 exits with status 256
2020-05-06 18:11:28: pid 6112: LOG: fork a new worker child process with pid: 6127
2020-05-06 18:11:28: pid 6112: LOG: failed to connect to PostgreSQL server on "172.16.222.132:5432", getsockopt() detected error "Connection refused"
2020-05-06 18:11:28: pid 6112: ERROR: failed to make persistent db connection
2020-05-06 18:11:28: pid 6112: DETAIL: connection to host:"172.16.222.132:5432" failed
2020-05-06 18:11:28: pid 6112: LOG: setting backend node 1 status to NODE DOWN
2020-05-06 18:11:28: pid 6112: LOG: starting degeneration. shutdown host 172.16.222.132(5432)
2020-05-06 18:11:28: pid 6112: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:11:28: pid 6112: DETAIL: "Permission denied"
2020-05-06 18:11:28: pid 6112: LOG: failover: no valid backends node found
2020-05-06 18:11:28: pid 6112: LOG: Restart all children
2020-05-06 18:11:28: pid 6112: LOG: execute command: /opt/phasorpoint-historian/bin/postgres_failover_script.sh 1 1 -1 "" ""
2020-05-06 18:11:28: pid 6121: LOG: child process received shutdown request signal 3
2020-05-06 18:11:28: pid 6122: LOG: child process received shutdown request signal 3
2020-05-06 18:11:28: pid 6123: LOG: child process received shutdown request signal 3
----------------------------------------------------------------------
The primary node (node 1) is down.
Node -1 takes over as the primary.
Pseudo-terminal will not be allocated because stdin is not a terminal.
ssh: Could not resolve hostname : Name or service not known
2020-05-06 18:11:28: pid 6112: LOG: failover: set new primary node: -1
failover done. shutdown host 172.16.222.132(5432)2020-05-06 18:11:28: pid 6112: LOG: failover done. shutdown host 172.16.222.132(5432)
2020-05-06 18:11:28: pid 6127: LOG: worker process received restart request
2020-05-06 18:11:29: pid 6126: LOG: received failback request for node_id: 0 from pid [6126]
2020-05-06 18:11:29: pid 6126: LOG: received failback request for node_id: 1 from pid [6126]
2020-05-06 18:11:29: pid 6126: LOG: restart request received in pcp child process
2020-05-06 18:11:29: pid 6112: LOG: PCP child 6126 exits with status 256 in failover()
2020-05-06 18:11:29: pid 6112: LOG: fork a new PCP child pid 6191 in failover()
2020-05-06 18:11:29: pid 6112: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:11:29: pid 6112: DETAIL: "Permission denied"
2020-05-06 18:11:29: pid 6112: LOG: received fast shutdown request
2020-05-06 18:11:29: pid 6112: LOG: shutdown request. closing listen socket
2020-05-06 18:11:29: pid 6130: LOG: child process received shutdown request signal 2
2020-05-06 18:11:29: pid 6131: LOG: child process received shutdown request signal 2
2020-05-06 18:11:29: pid 6132: LOG: child process received shutdown request signal 2
2020-05-06 18:11:29: pid 6112: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:11:29: pid 6112: DETAIL: "Permission denied"
2020-05-06 18:11:31: pid 6210: LOG: Backend status file /var/log/pgpool_status does not exist
2020-05-06 18:11:31: pid 6210: LOG: Setting up socket for 172.16.222.131:9999
2020-05-06 18:11:31: pid 6210: LOG: pgpool-II successfully started. version 3.4.6 (tataraboshi)
2020-05-06 18:11:31: pid 6210: LOG: find_primary_node: checking backend no 0
    
2020-05-06 18:11:31: pid 6210: LOG: failed to connect to PostgreSQL server on "172.16.222.131:5432", getsockopt() detected error "Connection refused"
2020-05-06 18:11:31: pid 6210: ERROR: failed to make persistent db connection
2020-05-06 18:11:31: pid 6210: DETAIL: connection to host:"172.16.222.131:5432" failed
2020-05-06 18:11:31: pid 6210: LOG: find_primary_node: checking backend no 1
    
2020-05-06 18:11:31: pid 6210: LOG: failed to connect to PostgreSQL server on "172.16.222.132:5432", getsockopt() detected error "Connection refused"
2020-05-06 18:11:31: pid 6210: ERROR: failed to make persistent db connection
2020-05-06 18:11:31: pid 6210: DETAIL: connection to host:"172.16.222.132:5432" failed
2020-05-06 18:11:31: pid 6210: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:11:31: pid 6210: DETAIL: "Permission denied"
2020-05-06 18:11:31: pid 6210: LOG: failed to connect to PostgreSQL server on "172.16.222.131:5432", getsockopt() detected error "Connection refused"
2020-05-06 18:11:31: pid 6210: ERROR: failed to make persistent db connection
2020-05-06 18:11:31: pid 6210: DETAIL: connection to host:"172.16.222.131:5432" failed
2020-05-06 18:11:31: pid 6210: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:11:31: pid 6210: DETAIL: "Permission denied"
2020-05-06 18:11:31: pid 6210: LOG: failed to connect to PostgreSQL server on "172.16.222.131:5432", getsockopt() detected error "Connection refused"
2020-05-06 18:11:31: pid 6210: ERROR: failed to make persistent db connection
2020-05-06 18:11:31: pid 6210: DETAIL: connection to host:"172.16.222.131:5432" failed
2020-05-06 18:11:31: pid 6210: LOG: setting backend node 0 status to NODE DOWN
2020-05-06 18:11:31: pid 6210: LOG: starting degeneration. shutdown host 172.16.222.131(5432)
2020-05-06 18:11:31: pid 6210: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:11:31: pid 6210: DETAIL: "Permission denied"
2020-05-06 18:11:31: pid 6210: LOG: Restart all children
2020-05-06 18:11:31: pid 6210: LOG: execute command: /opt/phasorpoint-historian/bin/postgres_failover_script.sh 0 0 1 172.16.222.132 /var/lib/pgsql/9.4/data
2020-05-06 18:11:31: pid 6212: LOG: child process received shutdown request signal 3
2020-05-06 18:11:31: pid 6213: LOG: child process received shutdown request signal 3
2020-05-06 18:11:31: pid 6214: LOG: child process received shutdown request signal 3
----------------------------------------------------------------------
The primary node (node 0) is down.
Node 1 takes over as the primary.
Pseudo-terminal will not be allocated because stdin is not a terminal.
sudo: sorry, you must have a tty to run sudo
2020-05-06 18:11:31: pid 6210: LOG: failover: set new primary node: -1
2020-05-06 18:11:31: pid 6210: LOG: failover: set new master node: 1
failover done. shutdown host 172.16.222.131(5432)2020-05-06 18:11:31: pid 6210: LOG: failover done. shutdown host 172.16.222.131(5432)
2020-05-06 18:11:31: pid 6216: LOG: worker process received restart request
2020-05-06 18:11:32: pid 6215: LOG: restart request received in pcp child process
2020-05-06 18:11:32: pid 6210: LOG: PCP child 6215 exits with status 256 in failover()
2020-05-06 18:11:32: pid 6210: LOG: fork a new PCP child pid 6222 in failover()
2020-05-06 18:11:32: pid 6210: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:11:32: pid 6210: DETAIL: "Permission denied"
2020-05-06 18:11:32: pid 6210: LOG: failed to connect to PostgreSQL server on "172.16.222.132:5432", getsockopt() detected error "Connection refused"
2020-05-06 18:11:32: pid 6210: ERROR: failed to make persistent db connection
2020-05-06 18:11:32: pid 6210: DETAIL: connection to host:"172.16.222.132:5432" failed
2020-05-06 18:11:32: pid 6210: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:11:32: pid 6210: DETAIL: "Permission denied"
2020-05-06 18:11:32: pid 6210: LOG: child process with pid: 6212 exits with status 0
2020-05-06 18:11:32: pid 6210: LOG: child process with pid: 6212 exited with success and will not be restarted
2020-05-06 18:11:32: pid 6210: LOG: child process with pid: 6213 exits with status 0
2020-05-06 18:11:32: pid 6210: LOG: child process with pid: 6213 exited with success and will not be restarted
2020-05-06 18:11:32: pid 6210: LOG: child process with pid: 6214 exits with status 0
2020-05-06 18:11:32: pid 6210: LOG: child process with pid: 6214 exited with success and will not be restarted
2020-05-06 18:11:32: pid 6210: LOG: worker child process with pid: 6216 exits with status 256
2020-05-06 18:11:32: pid 6210: LOG: fork a new worker child process with pid: 6223
2020-05-06 18:11:32: pid 6210: LOG: failed to connect to PostgreSQL server on "172.16.222.132:5432", getsockopt() detected error "Connection refused"
2020-05-06 18:11:32: pid 6210: ERROR: failed to make persistent db connection
2020-05-06 18:11:32: pid 6210: DETAIL: connection to host:"172.16.222.132:5432" failed
2020-05-06 18:11:32: pid 6210: LOG: setting backend node 1 status to NODE DOWN
2020-05-06 18:11:32: pid 6210: LOG: starting degeneration. shutdown host 172.16.222.132(5432)
2020-05-06 18:11:32: pid 6210: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:11:32: pid 6210: DETAIL: "Permission denied"
2020-05-06 18:11:32: pid 6210: LOG: failover: no valid backends node found
2020-05-06 18:11:32: pid 6210: LOG: Restart all children
2020-05-06 18:11:32: pid 6210: LOG: execute command: /opt/phasorpoint-historian/bin/postgres_failover_script.sh 1 1 -1 "" ""
2020-05-06 18:11:32: pid 6220: LOG: child process received shutdown request signal 3
2020-05-06 18:11:32: pid 6221: LOG: child process received shutdown request signal 3
2020-05-06 18:11:32: pid 6219: LOG: child process received shutdown request signal 3
----------------------------------------------------------------------
The primary node (node 1) is down.
Node -1 takes over as the primary.
Pseudo-terminal will not be allocated because stdin is not a terminal.
ssh: Could not resolve hostname : Name or service not known
2020-05-06 18:11:32: pid 6210: LOG: failover: set new primary node: -1
failover done. shutdown host 172.16.222.132(5432)2020-05-06 18:11:32: pid 6210: LOG: failover done. shutdown host 172.16.222.132(5432)
2020-05-06 18:11:32: pid 6223: LOG: worker process received restart request
2020-05-06 18:11:33: pid 6222: LOG: restart request received in pcp child process
2020-05-06 18:11:33: pid 6210: LOG: PCP child 6222 exits with status 256 in failover()
2020-05-06 18:11:33: pid 6210: LOG: fork a new PCP child pid 6289 in failover()
2020-05-06 18:11:33: pid 6210: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:11:33: pid 6210: DETAIL: "Permission denied"
2020-05-06 18:11:33: pid 6210: LOG: child process with pid: 6219 exits with status 0
2020-05-06 18:11:33: pid 6210: LOG: child process with pid: 6219 exited with success and will not be restarted
2020-05-06 18:11:33: pid 6210: LOG: child process with pid: 6220 exits with status 0
2020-05-06 18:11:33: pid 6210: LOG: child process with pid: 6220 exited with success and will not be restarted
2020-05-06 18:11:33: pid 6210: LOG: child process with pid: 6221 exits with status 0
2020-05-06 18:11:33: pid 6210: LOG: child process with pid: 6221 exited with success and will not be restarted
2020-05-06 18:11:33: pid 6210: LOG: worker child process with pid: 6223 exits with status 256
2020-05-06 18:11:33: pid 6210: LOG: fork a new worker child process with pid: 6290
2020-05-06 18:12:38: pid 6289: LOG: received failback request for node_id: 1 from pid [6289]
2020-05-06 18:12:38: pid 6210: LOG: starting fail back. reconnect host 172.16.222.132(5432)
2020-05-06 18:12:38: pid 6210: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:12:38: pid 6210: DETAIL: "Permission denied"
2020-05-06 18:12:38: pid 6210: LOG: Do not restart children because we are failbacking node id 1 host172.16.222.132 port:5432 and we are in streaming replication mode
2020-05-06 18:12:38: pid 6210: LOG: find_primary_node_repeatedly: waiting for finding a primary node
2020-05-06 18:12:38: pid 6210: LOG: find_primary_node: checking backend no 0
    
2020-05-06 18:12:38: pid 6210: LOG: find_primary_node: checking backend no 1
    
2020-05-06 18:12:43: pid 6427: FATAL: pid file found. is another pgpool(6210) is running?
    
2020-05-06 18:12:48: pid 6210: LOG: trying connecting to PostgreSQL server on "172.16.222.132:5432" by INET socket
2020-05-06 18:12:48: pid 6210: DETAIL: timed out. retrying...
2020-05-06 18:12:58: pid 6210: LOG: trying connecting to PostgreSQL server on "172.16.222.132:5432" by INET socket
2020-05-06 18:12:58: pid 6210: DETAIL: timed out. retrying...
2020-05-06 18:13:08: pid 6210: LOG: trying connecting to PostgreSQL server on "172.16.222.132:5432" by INET socket
2020-05-06 18:13:08: pid 6210: DETAIL: timed out. retrying...
2020-05-06 18:13:12: pid 6210: LOG: failed to connect to PostgreSQL server on "172.16.222.132:5432", getsockopt() detected error "No route to host"
2020-05-06 18:13:12: pid 6210: ERROR: failed to make persistent db connection
2020-05-06 18:13:12: pid 6210: DETAIL: connection to host:"172.16.222.132:5432" failed
2020-05-06 18:13:12: pid 6210: LOG: received fast shutdown request
2020-05-06 18:13:12: pid 6210: LOG: shutdown request. closing listen socket
2020-05-06 18:13:12: pid 6227: LOG: child process received shutdown request signal 2
2020-05-06 18:13:12: pid 6226: LOG: child process received shutdown request signal 2
2020-05-06 18:13:12: pid 6228: LOG: child process received shutdown request signal 2
2020-05-06 18:13:12: pid 6210: WARNING: failed to open status file at: "/var/log/pgpool_status"
2020-05-06 18:13:12: pid 6210: DETAIL: "Permission denied"
TagsNo tags attached.

Activities

pengbo

2020-05-08 14:55

developer   ~0003361

It seems that pgpool could not connect to PostgreSQL backend servers.

Please check the backend_* specified in pgpool.conf.

Raushan

2020-05-12 13:16

reporter   ~0003364

Thanks for getting response fast. We have specified ip address of both database server in backend_. But it doesn't work.

Issue History

Date Modified Username Field Change
2020-05-07 21:10 Raushan New Issue
2020-05-08 14:32 pengbo Assigned To => pengbo
2020-05-08 14:32 pengbo Status new => assigned
2020-05-08 14:55 pengbo Note Added: 0003361
2020-05-12 13:16 Raushan Note Added: 0003364