[pgpool-general: 8516] Re: PgPool showing node as down but it's up

[Bo Peng]

Hi,

> Hi,
> I have a 3-nodes PgPool 4.2.10 cluster connected to a 3-nodes PostgreSQL
> 12.13 cluster.
> I am in streaming replication mode, the streaming replication is actually
> working well without any errors.
> 
> Suddenly PgPool began to recognise the node_id 2 as down, but it is up and
> running and reachable from the PgPool node itself:
> =================================================================
> bash-4.4$ psql -h 127.0.0.1 -p 5432 -U pgpool -c "show pool_nodes;" postgres
>  node_id |       hostname        | port | status | lb_weight |  role   |
> select_cnt | load_balance_node | replication_delay | replication_state |
> repl
> ication_sync_state | last_status_change
> ---------+-----------------------+------+--------+-----------+---------+------------+-------------------+-------------------+-------------------+-----
> -------------------+---------------------
>  0       | postgres1.my.domain.com | 5432 | up     | 0.333333  | primary |
> 2801705    | true              | 0                 |                   |
> 
>                    | 2022-12-06 20:28:45
>  1       | postgres2.my.domain.com | 5432 | up     | 0.333333  | standby |
> 0          | false             | 0                 |                   |
> 
>                    | 2022-12-06 20:28:45
>  2       | postgres3.my.domain.com | 5432 | down   | 0.333333  | standby |
> 0          | false             | 0                 |                   |
> 
>                    | 2022-12-06 20:26:32
> (3 rows)
> =================================================================

To figure out why node2 is marked "down",
could you share your pgpool logs?

> The PostgreSQL node is up and running and reachable from the PgPool node
> itself:
> =================================================================
> bash-4.4$ psql -hpostgres3.my.domain.com -Upgpool -W -dpostgres -c'SELECT
> pg_is_in_recovery();'
> Password for user pgpool:
>  pg_is_in_recovery
> -------------------
>  t
> (1 row)
> =================================================================
> 
> I tried to attach it using pcp_attach_node, but nothing happens (no errors
> and no logs into the PgPool log file even if it is configured as debug1):
> =================================================================
> bash-4.4$ pcp_attach_node 2 -h localhost -p 9898 -w -v
> pcp_attach_node -- Command Successful
> =================================================================
> 
> Could you help me to understand what's wrong, please?

Normally if you run pcp command, the related log will
be output into pgpool logs.
Could you share the full logs to figure out the cause?

-- 
Bo Peng <pengbo at sraoss.co.jp>
SRA OSS LLC
https://www.sraoss.co.jp/