[pgpool-general: 4849] Re: Problem shutting down interfaces
Guptesh Kumar
guptesh.cg4 at gmail.com
Wed Aug 3 01:25:59 JST 2016
Hi Usama,
Please let me know if I can provide any more input which can help you
debug/reproduce this issue.
We are considering pgpool(with watchdog) as a high availability solution
for DB. And this issue is kind of show stopper because gracefully handling
server crash/network problems is key in a Highly Available solution.
I think this issue is same as Bug
http://www.pgpool.net/mantisbt/view.php?id=227 .
Looking forward for a solution of this issue.
Thanks & Regards,
Guptesh
On Sat, Jul 30, 2016 at 12:17 AM, Guptesh Kumar <guptesh.cg4 at gmail.com>
wrote:
> Hi Usama,
>
>
> I am still facing issues here-
>
> I shutdown primary(Node0) and standby(Node1) became primary successfully.
> When Node0 came up, i recovered postgres0(postgres on Node0). Now Node0 is
> standby and Node1 is primary.
> Again if I shutdown Node1(now primary), failover is not happening on
> Node0. Below are the logs after health_check retry count is expired
>
>
>
> Jul 29 23:50:02 ]: 2016-07-29 23:50:02: pid 4211: DETAIL: timed out.
> retrying...
> Jul 29 23:50:02 ]: 2016-07-29 23:50:02: pid 4185: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:02 ]: 2016-07-29 23:50:02: pid 4185: DETAIL: timed out.
> retrying...
> Jul 29 23:50:04 ]: 2016-07-29 23:50:04: pid 4312: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:04 ]: 2016-07-29 23:50:04: pid 4312: DETAIL: timed out.
> retrying...
> Jul 29 23:50:04 ]: 2016-07-29 23:50:04: pid 4279: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:04 ]: 2016-07-29 23:50:04: pid 4279: DETAIL: timed out.
> retrying...
> Jul 29 23:50:04 ]: 2016-07-29 23:50:04: pid 4257: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:04 ]: 2016-07-29 23:50:04: pid 4257: DETAIL: timed out.
> retrying...
> Jul 29 23:50:04 ]: 2016-07-29 23:50:04: pid 4230: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:04 ]: 2016-07-29 23:50:04: pid 4230: DETAIL: timed out.
> retrying...
> Jul 29 23:50:04 ]: 2016-07-29 23:50:04: pid 4210: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:04 ]: 2016-07-29 23:50:04: pid 4210: DETAIL: timed out.
> retrying...
> Jul 29 23:50:04 ]: 2016-07-29 23:50:04: pid 4278: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:04 ]: 2016-07-29 23:50:04: pid 4278: DETAIL: timed out.
> retrying...
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4304: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4304: DETAIL: timed out.
> retrying...
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4227: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4227: DETAIL: timed out.
> retrying...
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4276: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4276: DETAIL: timed out.
> retrying...
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4261: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4261: DETAIL: timed out.
> retrying...
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4297: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4297: DETAIL: timed out.
> retrying...
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4204: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4204: DETAIL: timed out.
> retrying...
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4295: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4295: DETAIL: timed out.
> retrying...
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4291: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4291: DETAIL: timed out.
> retrying...
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4228: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:06 ]: 2016-07-29 23:50:06: pid 4228: DETAIL: timed out.
> retrying...
> Jul 29 23:50:07 ]: 2016-07-29 23:50:07: pid 4211: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:07 ]: 2016-07-29 23:50:07: pid 4211: DETAIL: timed out.
> retrying...
> Jul 29 23:50:07 ]: 2016-07-29 23:50:07: pid 4185: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:07 ]: 2016-07-29 23:50:07: pid 4185: DETAIL: timed out.
> retrying...
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4228: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4291: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4261: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4297: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4295: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4227: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4276: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4228: LOG: received
> degenerate backend request for node_id: 1 from pid [4228]
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4261: LOG: received
> degenerate backend request for node_id: 1 from pid [4261]
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4297: LOG: received
> degenerate backend request for node_id: 1 from pid [4297]
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4291: LOG: received
> degenerate backend request for node_id: 1 from pid [4291]
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4295: LOG: received
> degenerate backend request for node_id: 1 from pid [4295]
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4227: LOG: received
> degenerate backend request for node_id: 1 from pid [4227]
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4276: LOG: received
> degenerate backend request for node_id: 1 from pid [4276]
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4204: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4204: LOG: received
> degenerate backend request for node_id: 1 from pid [4204]
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4304: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4304: LOG: received
> degenerate backend request for node_id: 1 from pid [4304]
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:08 ]: 2016-07-29 23:50:08: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4257: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4257: DETAIL: timed out.
> retrying...
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4279: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4279: DETAIL: timed out.
> retrying...
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4230: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4230: DETAIL: timed out.
> retrying...
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4210: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4210: DETAIL: timed out.
> retrying...
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4312: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4312: DETAIL: timed out.
> retrying...
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4278: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4278: DETAIL: timed out.
> retrying...
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4250: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4250: DETAIL: ipc command
> timeout
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4329: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4329: DETAIL: ipc command
> timeout
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4269: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4269: DETAIL: ipc command
> timeout
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4289: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4289: DETAIL: ipc command
> timeout
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4217: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4217: DETAIL: ipc command
> timeout
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4266: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4266: DETAIL: ipc command
> timeout
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4329: FATAL: failed to create
> a backend connection
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4329: DETAIL: executing
> failover on backend
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4217: FATAL: failed to create
> a backend connection
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4217: DETAIL: executing
> failover on backend
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4266: FATAL: failed to create
> a backend connection
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4266: DETAIL: executing
> failover on backend
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4269: FATAL: failed to create
> a backend connection
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4269: DETAIL: executing
> failover on backend
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4289: FATAL: failed to create
> a backend connection
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4289: DETAIL: executing
> failover on backend
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4250: FATAL: failed to create
> a backend connection
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4250: DETAIL: executing
> failover on backend
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:09 ]: 2016-07-29 23:50:09: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:12 ]: 2016-07-29 23:50:12: pid 4185: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:12 ]: 2016-07-29 23:50:12: pid 4185: DETAIL: timed out.
> retrying...
> Jul 29 23:50:12 ]: 2016-07-29 23:50:12: pid 4211: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:12 ]: 2016-07-29 23:50:12: pid 4211: DETAIL: timed out.
> retrying...
> Jul 29 23:50:14 ]: 2016-07-29 23:50:14: pid 4230: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:14 ]: 2016-07-29 23:50:14: pid 4230: DETAIL: timed out.
> retrying...
> Jul 29 23:50:14 ]: 2016-07-29 23:50:14: pid 4312: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:14 ]: 2016-07-29 23:50:14: pid 4312: DETAIL: timed out.
> retrying...
> Jul 29 23:50:14 ]: 2016-07-29 23:50:14: pid 4279: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:14 ]: 2016-07-29 23:50:14: pid 4279: DETAIL: timed out.
> retrying...
> Jul 29 23:50:14 ]: 2016-07-29 23:50:14: pid 4257: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:14 ]: 2016-07-29 23:50:14: pid 4257: DETAIL: timed out.
> retrying...
> Jul 29 23:50:14 ]: 2016-07-29 23:50:14: pid 4210: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:14 ]: 2016-07-29 23:50:14: pid 4210: DETAIL: timed out.
> retrying...
> Jul 29 23:50:14 ]: 2016-07-29 23:50:14: pid 4278: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:14 ]: 2016-07-29 23:50:14: pid 4278: DETAIL: timed out.
> retrying...
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4210: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4230: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4312: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4279: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4278: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4312: LOG: received
> degenerate backend request for node_id: 1 from pid [4312]
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4230: LOG: received
> degenerate backend request for node_id: 1 from pid [4230]
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4210: LOG: received
> degenerate backend request for node_id: 1 from pid [4210]
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4279: LOG: received
> degenerate backend request for node_id: 1 from pid [4279]
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4278: LOG: received
> degenerate backend request for node_id: 1 from pid [4278]
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4257: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4257: LOG: received
> degenerate backend request for node_id: 1 from pid [4257]
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:16 ]: 2016-07-29 23:50:16: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:17 ]: 2016-07-29 23:50:17: pid 4185: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:17 ]: 2016-07-29 23:50:17: pid 4185: DETAIL: timed out.
> retrying...
> Jul 29 23:50:17 ]: 2016-07-29 23:50:17: pid 4211: LOG: trying connecting
> to PostgreSQL server on "192.168.160.5:5432" by INET socket
> Jul 29 23:50:17 ]: 2016-07-29 23:50:17: pid 4211: DETAIL: timed out.
> retrying...
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4295: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4295: DETAIL: ipc command
> timeout
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4295: FATAL: failed to create
> a backend connection
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4295: DETAIL: executing
> failover on backend
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4227: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4227: DETAIL: ipc command
> timeout
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4227: FATAL: failed to create
> a backend connection
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4227: DETAIL: executing
> failover on backend
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4304: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4304: DETAIL: ipc command
> timeout
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4304: FATAL: failed to create
> a backend connection
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4304: DETAIL: executing
> failover on backend
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4228: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4228: DETAIL: ipc command
> timeout
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4228: FATAL: failed to create
> a backend connection
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4228: DETAIL: executing
> failover on backend
> Jul 29 23:50:18 ]: 2016-07-29 23:50:18: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:19 ]: 2016-07-29 23:50:19: pid 4185: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:19 ]: 2016-07-29 23:50:19: pid 4211: LOG: failed to connect
> to PostgreSQL server on "192.168.160.5:5432", getsockopt() detected error
> "Connection timed out"
> Jul 29 23:50:19 ]: 2016-07-29 23:50:19: pid 4185: LOG: received
> degenerate backend request for node_id: 1 from pid [4185]
> Jul 29 23:50:19 ]: 2016-07-29 23:50:19: pid 4211: LOG: received
> degenerate backend request for node_id: 1 from pid [4211]
> Jul 29 23:50:19 ]: 2016-07-29 23:50:19: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:19 ]: 2016-07-29 23:50:19: pid 4179: LOG: new IPC connection
> received
> Jul 29 23:50:25 ]: 2016-07-29 23:50:25: pid 4312: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:25 ]: 2016-07-29 23:50:25: pid 4312: DETAIL: ipc command
> timeout
> Jul 29 23:50:25 ]: 2016-07-29 23:50:25: pid 4279: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:25 ]: 2016-07-29 23:50:25: pid 4279: DETAIL: ipc command
> timeout
> Jul 29 23:50:25 ]: 2016-07-29 23:50:25: pid 4278: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:25 ]: 2016-07-29 23:50:25: pid 4278: DETAIL: ipc command
> timeout
> Jul 29 23:50:25 ]: 2016-07-29 23:50:25: pid 4312: FATAL: failed to create
> a backend connection
> Jul 29 23:50:25 ]: 2016-07-29 23:50:25: pid 4312: DETAIL: executing
> failover on backend
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4257: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4257: DETAIL: ipc command
> timeout
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4210: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4210: DETAIL: ipc command
> timeout
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4278: FATAL: failed to create
> a backend connection
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4278: DETAIL: executing
> failover on backend
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4279: FATAL: failed to create
> a backend connection
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4279: DETAIL: executing
> failover on backend
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4210: FATAL: failed to create
> a backend connection
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4210: DETAIL: executing
> failover on backend
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4257: FATAL: failed to create
> a backend connection
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4257: DETAIL: executing
> failover on backend
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4230: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4230: DETAIL: ipc command
> timeout
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4230: FATAL: failed to create
> a backend connection
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4230: DETAIL: executing
> failover on backend
> Jul 29 23:50:26 ]: 2016-07-29 23:50:25: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:29 ]: 2016-07-29 23:50:29: pid 4211: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:29 ]: 2016-07-29 23:50:29: pid 4211: DETAIL: ipc command
> timeout
> Jul 29 23:50:29 ]: 2016-07-29 23:50:29: pid 4185: WARNING: degenerate
> backend set command failed
> Jul 29 23:50:29 ]: 2016-07-29 23:50:29: pid 4185: DETAIL: ipc command
> timeout
> Jul 29 23:50:29 ]: 2016-07-29 23:50:29: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:50:29 ]: 2016-07-29 23:50:29: pid 4185: FATAL: failed to create
> a backend connection
> Jul 29 23:50:29 ]: 2016-07-29 23:50:29: pid 4185: DETAIL: executing
> failover on backend
> Jul 29 23:50:29 ]: 2016-07-29 23:50:29: pid 4211: FATAL: failed to create
> a backend connection
> Jul 29 23:50:29 ]: 2016-07-29 23:50:29: pid 4211: DETAIL: executing
> failover on backend
> Jul 29 23:50:29 ]: 2016-07-29 23:50:29: pid 4179: LOG: read from socket
> failed, remote end closed the connection
> Jul 29 23:51:13 ]: 2016-07-29 23:51:13: pid 4179: LOG: read from socket
> failed with error :"Connection reset by peer"
> Jul 29 23:51:19 ]: 2016-07-29 23:51:19: pid 4179: LOG: read from socket
> failed with error :"Connection reset by peer"
>
>
> Thanks & Regards,
> Guptesh
>
> On Thu, Jun 16, 2016 at 6:57 PM, Guptesh Kumar <guptesh.cg4 at gmail.com>
> wrote:
>
>> Hi Usama,
>>
>> I built from V3_5_STABLE head and tested. The problem is fixed :)
>>
>>
>> Thanks & Regards,
>> Guptesh
>>
>> On Thu, Jun 16, 2016 at 1:14 AM, Muhammad Usama <m.usama at gmail.com>
>> wrote:
>>
>>> Hi
>>>
>>> I have fixed the similar problem in pgpool-II 3.5 a few days back and it
>>> will be part of next point release, which is expected in a few days time
>>>
>>>
>>> http://git.postgresql.org/gitweb?p=pgpool2.git;a=commitdiff;h=6ae42e74d48391b8cfaa97927e00bd65aec68fe8
>>>
>>> You can try if building from the V3_5_STABLE head fixes the problem.
>>>
>>> Regards
>>> Muhammad Usama
>>>
>>>
>>> On Wed, Jun 15, 2016 at 11:24 PM, Guptesh Kumar <guptesh.cg4 at gmail.com>
>>> wrote:
>>>
>>>> Hi Ricardo,
>>>>
>>>> I am facing a similar issue -
>>>>
>>>> Node 1 : Master pgpool, Master Postgresql
>>>> Node 2 : Slave Pgpool, Slave Postgresql
>>>>
>>>> When I pull out the power cable of master node(Node 1), the Slave
>>>> watchdog promotes itself to Master correctly. But the pgpool in NOW master
>>>> node(Node 2) neither disconnects the Node 1 Postgresql from the pool nor
>>>> it promotes the Node 2 postgresql as Master(failover does not happen). All
>>>> my DB connection hangs.
>>>>
>>>> I tried various values for health_check parameters but none of them
>>>> helped.
>>>>
>>>> Softwares Used are -
>>>> pgpool-II-pg94-3.5.2-1pgdg.rhel7.x86_64
>>>> postgresql94-server-9.4.7-1PGDG.rhel7.x86_64
>>>>
>>>> Thanks & Regards,
>>>> Guptesh
>>>>
>>>>
>>>> On Tue, May 10, 2016 at 11:36 PM, Ricardo Larrañaga <
>>>> ricardo.larranaga at gmail.com> wrote:
>>>>
>>>>> Hi guys:
>>>>> I have been working on this a little bit, and i encounter this problem
>>>>> (running pgpool 3.5.2 on a 2 node cluster with streaming replication, if i
>>>>> shut down the interface of the slave, pgpool on the master hangs the
>>>>> connections). And i realize that the problem only happens when i shut down
>>>>> the interface. If i either shut down the processes or use the firewall to
>>>>> block backend communication, pgpool reacts properly.
>>>>>
>>>>> Does anyone have experience with this or have a 2 node cluster with
>>>>> streaming replication to test it. I am trying to identify what makes the
>>>>> setup hang, since shutting down the interface is the best way i have to
>>>>> simulate a node crash. When i shut down the interface, both postgres and
>>>>> pgpool loose connection, but this is a valid scenario that could happen in
>>>>> real life (when the whole server crashes).
>>>>> Thanks a lot.
>>>>> Regards
>>>>>
>>>>>
>>>>> On Thu, May 5, 2016 at 4:55 PM, Ricardo Larrañaga <
>>>>> ricardo.larranaga at gmail.com> wrote:
>>>>>
>>>>>> Hello guys:
>>>>>> I am running pgpool 3.5.2 in a 2 node cluster with postgresql-9.5 in
>>>>>> master-slave and streaming replication mode.
>>>>>>
>>>>>> I have been testing failover and failback for a while. While doing
>>>>>> failover by shutting down the processes, everything looks fine.
>>>>>>
>>>>>>
>>>>>> The one test that is failing is when i shutdown the interface of one
>>>>>> node.
>>>>>> Right now i am shutting down the interface of the slave node (both
>>>>>> pgool slave and postgresql slave).
>>>>>>
>>>>>> The problem i am running into is that after doing this, all
>>>>>> connections to my database (through pgpool) hang. i am testing it with
>>>>>> psql, and psql just hangs and does not give me an output. When i bring the
>>>>>> interface of the slave back up, and try to connect to psql again, it looks
>>>>>> like pgpool never marked the postgresql node as disconnected.
>>>>>>
>>>>>> I tried both with and without health check, and also tried with
>>>>>> different values of health_check_timeout. my connect timeout value is
>>>>>> default (10 seconds).
>>>>>>
>>>>>> Any one has encountered this issue? I just dont see pgpool attempting
>>>>>> to do any failover. Pgpool is still running though, i can see the logs
>>>>>> still comming. I just never see an error.
>>>>>>
>>>>>> I am NOT using interface monitoring, and i would prefer not to use it.
>>>>>>
>>>>>> Any pointers on how could i troubleshoot this?
>>>>>> Thanks.
>>>>>> Regards
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> pgpool-general mailing list
>>>>> pgpool-general at pgpool.net
>>>>> http://www.pgpool.net/mailman/listinfo/pgpool-general
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> pgpool-general mailing list
>>>> pgpool-general at pgpool.net
>>>> http://www.pgpool.net/mailman/listinfo/pgpool-general
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20160802/d90b61b3/attachment-0001.html>
More information about the pgpool-general
mailing list