[pgpool-general: 1824] Re: pgpool_switch_xlog on Amazon S3

Tatsuo Ishii ishii at postgresql.org
Wed Jun 5 16:31:28 JST 2013


> Hi all
> 
> I'm just setting up pgpool replication on Amazon AWS.
> 
> I'm sending WAL archives to an S3 bucket, which doesn't appear as a
> directory on the server.
> PgSQL allows this by simply requiring a program to be called to
> perform the archiving, similarly for recovery.
> 
> However, pgpool_switch_xlog expects a standard folder in which the WAL
> files are written and clearly won't work in this case.

Too bad.

> I see two possibilities:
> * change my archiving script to create a dummy file in a local
> * directory, suitable for pgpool_switch_xlog to check, or
> * change my archiving script to call pg_switch_xlog, and then check
> * $PGDATA/pg_xlog/archive_status/NNNNN.done in a shell script loop.
> 
> The first is probably easier to implement, and won't cost much
> storage.
> The second is more complex but doesn't require any extra files.
> 
> Does anyone care to comment?

I think we should modify pgpool_switch_xlog to check
$PGDATA/pg_xlog/archive_status/NNNNN.done. Moreover, we should do that
way from the beginning. pgpool_switch_xlog currently waits for:
- the archive file is created
- the size is not 0
- the size is multiply of 1024

These all checks are to make sure that the archive file is actually
created. But these checking can be replaced by just checking creation
of NNNNN.done, and it is more efficient and accurate.

Attached patch changes pgpool_switch_xlog to check NNNN.done.

Can you please try it out? It should work without changing your
archive program.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pgpool-recovery.c.patch
Type: text/x-patch
Size: 739 bytes
Desc: not available
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20130605/93833a14/attachment.bin>


More information about the pgpool-general mailing list