[pgpool-general: 1782] Re: [PGPOOL] online recovery fails due to "WAL ends before end of online backup"

[Tatsuo Ishii]

Dear Tina,

Thanks for testing. It seems there must be a problem with
watchdog. Nagata is in charge of watchdog and he will make some
investigation.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

> Dear Ishii,
> 
> Thank you for reply so quickly.
> The problem does not happen if we disable watchdog. Online recovery can be done successfully.
> We are wondering why online recovery can be done successfully if watchdog is off. Would you please let us know the cause?
> In our environment, we still need to set watchdog on. Do you plan to solve this issue? If yes, when the patch/version will be available?
> Thank you.
> 
> Regards,
> Tina
> 
> -----Original Message-----
> From: Tatsuo Ishii [mailto:ishii at postgresql.org] 
> Sent: Tuesday, May 28, 2013 7:22 AM
> To: Tina Wang (王冠婷)
> Cc: pgpool-general at pgpool.net; Eric Fang (方俊雄); Kuo Huang (黃種國)
> Subject: Re: [PGPOOL] online recovery fails due to "WAL ends before end of online backup"
> 
> Does the problem happen even if you disable watchdog?
> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> English: http://www.sraoss.co.jp/index_en.php
> Japanese: http://www.sraoss.co.jp
> 
>> Dear pgpool developers,
>> 
>> I encounter an issue while doing online recovery. I try to solve the problem but still have no idea about it. Recovery usually fails in the step: remote restart database server. It displays “FATAL: WAL ends before end of online backup. HINT: Online backup started with pg_start_backup() must be ended with pg_stop_backup(), and all WAL up to that point must be available at recovery.”
>> 
>> Software:
>> OS: Ubuntu 12.04.2 LTS/Server/precise
>> pgpool version: pgpool-II-3.2.3
>> postgresql version: postgresql_9.2.2
>> 
>> Environment:
>> 2 pgpools: PG0 and PG1 (with watchdog on)
>> 2 databases: DB0 and DB1
>> 
>> Reproduce steps:
>> 
>> 1.      Update data via VIP. (23.7kB size JPEG pictures were kept being updated to the database via VIP. And there are 10 requests in one second.)
>> 
>> 2.      Stop DB1 service.
>> 
>> 3.      Use pcp_recovery_node to do recovery on PG1.
>> 
>> 4.      Recovery done.
>> 
>> Note that recovery usually fails at remote restart. It is interesting that the success rate of online recovery is higher when updating DB with only text than with pictures.
>> 
>> Regards,
>> Tina
>