[pgpool-general: 7717] Re: PGPOOL-II Cluster Example | ERROR: executing recovery, execution of command failed at "1st stage"

[Bo Peng]

Hello,

>   rm -rf /data
> ...
> rm: cannot remove ‘/data’: Permission denied

Postgre user has no permission to romove /data directory.
Are you running PostgreSQL uing postgres user?
if so, please make sure that only postgres user has peemission of $PGDATA.


> Hi,
> 
> I'm using pgpool-II 4.2.5 and following the cluster example from the
> documentation and I'm seeking your support as when I reach 8.2.9.1 the
> command fails with error and the $PGDATA directory from the standby node is
> cleared out.
> 
> this is the command printout:
> [postgres at svr7 ~]$ pcp_recovery_node -v -d -h vip.vip.vip.vip -p 9898 -U
> pgpool -n 1
> Password:
> DEBUG: recv: tos="m", len=8
> DEBUG: recv: tos="r", len=21
> DEBUG: send: tos="D", len=6
> DEBUG: recv: tos="E", len=130
> ERROR:  executing recovery, execution of command failed at "1st stage"
> DETAIL:  command:"recovery_1st_stage"
> 
> DEBUG: send: tos="X", len=4
> [postgres at svr7 ~]$
> 
> and below is the postgres logs from primary node:
> 
> + PRIMARY_NODE_PGDATA=/home/data
> + DEST_NODE_HOST=svr5
> + DEST_NODE_PGDATA=/data
> + PRIMARY_NODE_PORT=5432
> + DEST_NODE_ID=1
> + DEST_NODE_PORT=5432
> ++ hostname
> + PRIMARY_NODE_HOST=svr7
> + PGHOME=/usr/pgsql-13
> + ARCHIVEDIR=/var/lib/pgsql/archivedir
> + REPLUSER=repl
> + REPL_SLOT_NAME=svr5
> + echo recovery_1st_stage: start: pg_basebackup for Standby node 1
> recovery_1st_stage: start: pg_basebackup for Standby node 1
> + ssh -T -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
> postgres at svr5 -i /home/postgres/.ssh/id_rsa_pgpool ls /tmp
> Warning: Permanently added 'svr5,ip.ip.ip.ip' (ECDSA) to the list of known
> hosts.
> + '[' 0 -ne 0 ']'
> ++ /usr/pgsql-13/bin/initdb -V
> ++ awk '{print $3}'
> ++ sed 's/\..*//'
> ++ sed 's/\([0-9]*\)[a-zA-Z].*/\1/'
> + PGVERSION=13
> + '[' 13 -ge 12 ']'
> + RECOVERYCONF=/data/myrecovery.conf
> + /usr/pgsql-13/bin/psql -h svr7 -p 5432 -c 'SELECT
> pg_create_physical_replication_slot('\''svr5'\'');'
> + '[' 0 -ne 0 ']'
> ++ echo /data/myrecovery.conf
> ++ sed -e 's/\//\\\//g'
> ++ echo /data/myrecovery.conf
> ++ sed -e 's/\//\\\//g'
> + ssh -T -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
> postgres at svr5 -i /home/postgres/.ssh/id_rsa_pgpool '
> 
>     set -o errexit
> 
>     rm -rf /data
>     rm -rf /var/lib/pgsql/archivedir/*
> 
>     /usr/pgsql-13/bin/pg_basebackup -h svr7 -U repl -p 5432 -D /data -X
> stream
> 
>     cat > /data/myrecovery.conf << EOT
> primary_conninfo = '\''host=svr7 port=5432 user=repl application_name=svr5
> passfile='\'''\''/var/lib/pgsql/.pgpass'\'''\'''\''
> recovery_target_timeline = '\''latest'\''
> restore_command = '\''scp svr7:/var/lib/pgsql/archivedir/%f %p'\''
> primary_slot_name = '\''svr5'\''
> EOT
> 
>     if [ 13 -ge 12 ]; then
>         sed -i -e "\$ainclude_if_exists = '\''\/data\/myrecovery.conf'\''"
>                -e "/^include_if_exists = '\''\/data\/myrecovery.conf'\''/d"
> /data/postgresql.conf
>         touch /data/standby.signal
>     else
>         echo "standby_mode = '\''on'\''" >> /data/myrecovery.conf
>     fi
> 
>     sed -i "s/#*port = .*/port = 5432/" /data/postgresql.conf
> '
> Warning: Permanently added 'svr5,ip.ip.ip.ip' (ECDSA) to the list of known
> hosts.
> rm: cannot remove ‘/data’: Permission denied
> + '[' 1 -ne 0 ']'
> + /usr/pgsql-13/bin/psql -h svr7 -p 5432 -c 'SELECT
> pg_drop_replication_slot('\''svr5'\'');'
> + '[' 0 -ne 0 ']'
> + echo ERROR: recovery_1st_stage: end: pg_basebackup failed. online
> recovery failed
> ERROR: recovery_1st_stage: end: pg_basebackup failed. online recovery failed
> + exit 1
> 2021-09-21 04:46:43.704 EET [41572] ERROR:  pgpool_recovery failed
> 2021-09-21 04:46:43.704 EET [41572] STATEMENT:  SELECT
> pgpool_recovery('recovery_1st_stage', 'svr5', '/data', '5432', 1, '5432')
> 
> 
> -- 
> Regards,
> Amr


-- 
Bo Peng <pengbo at sraoss.co.jp>
SRA OSS, Inc. Japan
http://www.sraoss.co.jp/