[Pgpool-general] pgpool "freeze" when pcp_recovery_node fails

[Tatsuo Ishii]

> Sometimes, recovery fails with the second node saying i.e.:
> 
> Mar 12 05:38:28 db20 postgres[32140]: [5-1] 2010-03-12 05:38:28 SGT LOG:  invalid primary checkpoint record
> Mar 12 05:38:28 db20 postgres[32140]: [6-1] 2010-03-12 05:38:28 SGT LOG:  could not open file "pg_xlog/000000030000002D00000024" (log file 45, segment 36): No such file or
> Mar 12 05:38:28 db20 postgres[32140]: [6-2]  directory
> Mar 12 05:38:28 db20 postgres[32140]: [7-1] 2010-03-12 05:38:28 SGT LOG:  invalid secondary checkpoint record
> Mar 12 05:38:28 db20 postgres[32140]: [8-1] 2010-03-12 05:38:28 SGT PANIC:  could not locate a valid checkpoint record
> Mar 12 05:38:28 db20 postgres[32139]: [1-1] 2010-03-12 05:38:28 SGT LOG:  startup process (PID 32140) was terminated by signal 6: Aborted
> Mar 12 05:38:28 db20 postgres[32139]: [2-1] 2010-03-12 05:38:28 SGT LOG:  aborting startup due to startup process failure

These errors are from PostgreSQL, not pgpool. You should check your
recovery script is properly configured.

> When this happens, such command never exits (it should take up to 240 seconds):

How long you need to wait is depending on your configurations. I don't
know your configuration, but in my case my pgpool_remote_start calls
pg_ctl -w which should wait up to 60 secs according to PostgreSQL 8.4
doc plus pgpool wait for postmaster coming up to recovery_timeout/3 *
3 secs (= 90 secs in my case).

> # pcp_recovery_node -d 240 127.0.0.1 9898 user password 1
> DEBUG: send: tos="R", len=45
> DEBUG: recv: tos="r", len=21, data=AuthenticationOK
> DEBUG: send: tos="D", len=6
> 
> The only way to kill pgpool in this state is to use kill -9.
> 
> 
> Is it a known issue?
> 
> I use pgpool-II 2.3.2.2.
> 
> -- 
> Tomasz Chmielewski
> http://wpkg.org
> _______________________________________________
> Pgpool-general mailing list
> Pgpool-general at pgfoundry.org
> http://pgfoundry.org/mailman/listinfo/pgpool-general