[pgpool-committers: 3660] pgpool: Fixing the design of failover command propagation on watchdog c

[Muhammad Usama]

Fixing the design of failover command propagation on watchdog cluster

Overhauling the design of how failover, failback and promote node commands are
propagated to the watchdog nodes. Previously the watchdog on pgpool-II node that
needs to perform the node command (failover, failback or promote node) used to
broadcast the failover command to all attached pgpool-II nodes. And this
sometimes makes the synchronization issues, especially when the watchdog cluster
contains a large number of nodes and consequently the failover command sometimes
gets executed by more than one pgpool-II.

Now with this commit all the node commands are forwarded to the
master/coordinator watchdog, which in turn propagates to all standby nodes.
Apart from above the commit also changes the failover command interlocking
mechanism and now only the master/coordinator node can become the lock holder
so the failover commands will only get executed on the master/coordinator node.

Branch
------
master

Details
-------
http://git.postgresql.org/gitweb?p=pgpool2.git;a=commitdiff;h=ef6c0c8fb0899c65d37c2d83150683a952bad756

Modified Files
--------------
src/include/pool.h                     |   13 +-
src/include/watchdog/watchdog.h        |    3 +
src/include/watchdog/wd_ipc_commands.h |   26 +-
src/include/watchdog/wd_ipc_defines.h  |   23 +-
src/include/watchdog/wd_json_data.h    |    2 +-
src/main/pgpool_main.c                 |  115 +-
src/pcp_con/pcp_worker.c               |   12 +-
src/pcp_con/recovery.c                 |    2 +-
src/protocol/pool_process_query.c      |    2 +-
src/protocol/pool_proto_modules.c      |    4 +-
src/utils/pool_signal.c                |    2 +-
src/utils/pool_stream.c                |   23 +
src/watchdog/watchdog.c                | 2199 ++++++++++++++++++++------------
src/watchdog/wd_commands.c             |  373 ++----
src/watchdog/wd_json_data.c            |    6 +-
15 files changed, 1663 insertions(+), 1142 deletions(-)