[pgpool-general: 3608] Re: PGPool sending r/w queries to wrong DB node

[Yugo Nagata]

Hi,

First, pgpool-II detached backend node #0 which was primary.

 LOG:  received degenerate backend request for node_id

I don't know why node #0 is detached, but this might be shutdowned.

 LOG:  pool_send_and_wait: Error or notice message from backend: : DB node id: 0 backend pid: 31461 statement: "delete from t_case_file_tag_association where case_file_id=$1" message: "terminating connection due to administrator command"

Second, pgpool-II started searching a new primary.

 LOG:  find_primary_node_repeatedly: waiting for finding a primary node

However, unfortunately, it is seemed that pgpool-II failed to find
a new primary. Then, queries were to send the new master, node #1,
which is standby.

 LOG:  failover: set new primary node: -1
 LOG:  failover: set new master node: 1


I can't say why pgpool-II failed to find a new primary node, but
maybe, failover_command might be wrong. Could you please check your
failover_command and PostgreSQL's log around 17:24?

On Wed, 08 Apr 2015 17:05:58 -0400
Pablo Sanchez <pablo at blueoakdb.com> wrote:

> Hi,
> 
> I'm currently running pgpool-II, version 3.4.1 in a master/slave set up. 
>   We have two PG databases in streaming replication mode, version 9.3.
> 
> The Master pgpool-II was down (my error) and we were on the Slave 
> pgpool-II for many days.  All was working well.
> 
> Unexpectedly, according to the pgpool logs, it seems it thought the 
> database was down.
> 
> When it righted itself, pgpool started sending r/w transactions to the 
> Slave (read-only) db which of course failed.  Ouch.
> 
> I'm attaching a gzip'd pgpool.log file with I believe sufficient data 
> showing pgpool in a good state, then in an not so good state.  :)
> 
> I've checked /var/log/messages, PG logs at the time when things went 
> wrong and nothing appears out of the ordinary.
> 
> Any advice would be appreciated.
> 
> Cheers,
> -- 
> Pablo Sanchez - Blueoak Database Engineering, Inc
> Ph:    819.459.1926         Blog:  http://pablo-blog.blueoakdb.com
> iNum:  883.5100.0990.1054


-- 
Yugo Nagata <nagata at sraoss.co.jp>