[Pgpool-general] inconsistency when using two pgpool instances in warm standby?

[Duco Fijma]

Hello,

I'm designing a warm-standby Postgres/pgpool system accepting 
connections from a number of application servers.

Of course, running a single instance of pgpool introduces a single point 
of failure. However, I'm having difficulties seeing how multiple 
instances of pgpool, in general, can have a consistent picture of the 
(failure) state of the "master" database.

For example, consider two appservers "A" and "B", both running an 
instance of pgpool. Additionally, we have a master postgres database 
server and a standby postgres database server.

In that situation, the network connection between "B" and the master 
database server is going down. The pgpool instance on B detect a failure 
and triggers a failover to the standby server, which therefore exits 
recovery mode and starts accepting database connections.

Especially in a design without single points of failure, the network 
connection between appserver "A" and the master database is likely not 
to be failing at the same time. Therefore, from the perspective of 
appserver "A" the master database is not failing at all. Its pgpool 
instance therefore happily continues to proxy database connections to 
the master database.

The net effect is that our system is broken into two independent halves. 
It just takes one database transaction to make these two halves into two 
inconsistent systems :)

I find it difficult to think of a solutions to this. Whatever schema I 
use, the very same failure that caused the database failover in the 
first place, can or will also hinder (direct or indirect) communication 
between the two pgpool instances to prevent inconsistent failover states.

--
Duco