5.14. Watchdog

Watchdog configuration parameters are described in pgpool.conf. There is sample configuration in the WATCHDOG section of pgpool.conf.sample file. All following options are required to be specified in watchdog process.

5.14.1. Enable watchdog

use_watchdog (boolean)

If on, activates the watchdog. Default is off

This parameter can only be set at server start.

5.14.2. Watchdog communication

wd_hostname (string)

Specifies the hostname or IP address of Pgpool-II server. This is used for sending/receiving queries and packets, and also as an identifier of the watchdog node.

This parameter can only be set at server start.

wd_port (integer)

Specifies the port number to be used by watchdog process to listen for connections. Default is 9000.

This parameter can only be set at server start.

wd_authkey (string)

Specifies the authentication key used for all watchdog communications. All Pgpool-II must have the same key. Packets from watchdog having different key will get rejected. This authentication is also applied to the heartbeat signals when the heartbeat mode is used as a lifecheck method.

Since in Pgpool-IIV3.5 or beyond wd_authkey is also used to authenticate the watchdog IPC clients, all clients communicating with Pgpool-II watchdog process needs to provide this wd_authkey value for "IPCAuthKey" key in the JSON data of the command.

Default is '' (empty) which means disables the watchdog authentication.

This parameter can only be set at server start.

5.14.3. Upstream server connection

trusted_servers (string)

Specifies the list of trusted servers to check the up stream connections. Each server in the list is required to respond to ping. Specify a comma separated list of servers such as "hostA,hostB,hostC". If none of the server are reachable, watchdog will regard it as failure of the Pgpool-II. Therefore, it is recommended to specify multiple servers. Please note that you should not assign PostgreSQL servers to this parameter.

This parameter can only be set at server start.

ping_path (string)

Specifies the path of a ping command for monitoring connection to the upper servers. Set the only path of the directory containing the ping utility, such as "/bin" or such directory.

This parameter can only be set at server start.

5.14.4. Virtual IP control

delegate_IP (string)

Specifies the virtual IP address (VIP) of Pgpool-II that is connected from client servers (application servers etc.). When a Pgpool-II is switched from standby to active, the Pgpool-II takes over this VIP. Default is ''(empty): which means virtual IP will never be brought up.

This parameter can only be set at server start.

if_cmd_path (string)

Specifies the path to the command that Pgpool-II will use to switch the virtual IP on the system. Set only the path of the directory containing the binary, such as "/sbin" or such directory.

This parameter can only be set at server start.

if_up_cmd (string)

Specifies the command to bring up the virtual IP. Set the command and parameters such as "ip addr add $_IP_$/24 dev eth0 label eth0:0" $_IP_$ will get replaced by the IP address specified in the delegate_IP.

This parameter can only be set at server start.

if_down_cmd (string)

Specifies the command to bring down the virtual IP. Set the command and parameters such as "ip addr del $_IP_$/24 dev eth0".

This parameter can only be set at server start.

arping_path (string)

Specifies the path to the command that Pgpool-II will use to send the ARP requests after the virtual IP switch. Set only the path of the directory containing the binary, such as "/usr/sbin" or such directory.

This parameter can only be set at server start.

arping_cmd (string)

Specifies the command to use for sending the ARP requests after the virtual IP switch. Set the command and parameters such as "arping -U $_IP_$ -w 1". $_IP_$ will get replaced by the IP address specified in the delegate_IP.

This parameter can only be set at server start.

5.14.5. Behaviour on escalation and de-escalation

Configuration about behavior when Pgpool-II escalates to active (virtual IP holder)

clear_memqcache_on_escalation (boolean)

When set to on, watchdog clears all the query cache in the shared memory when pgpool-II escalates to active. This prevents the new active Pgpool-II from using old query caches inconsistent to the old active.

Default is on.

This works only if memqcache_method is 'shmem'.

This parameter can only be set at server start.

wd_escalation_command (string)

Watchdog executes this command on the node that is escalated to the master watchdog.

This command is executed just before bringing up the virtual IP if that is configured on the node.

This parameter can only be set at server start.

wd_de_escalation_command (string)

Watchdog executes this command on the master Pgpool-II watchdog node when that node resigns from the master node responsibilities. A master watchdog node can resign from being a master node, when the master node Pgpool-II shuts down, detects a network blackout or detects the lost of quorum.

This command is executed before bringing down the virtual/floating IP address if it is configured on the watchdog node.

wd_de_escalation_command is not available prior to Pgpool-IIV3.5.

This parameter can only be set at server start.

5.14.6. Life checking Pgpool-II

Watchdog checks pgpool-II status periodically. This is called "life check".

wd_lifecheck_method (string)

Specifies the method of life check. This can be either of 'heartbeat' (default), 'query' or 'external'.

heartbeat: In this mode, watchdog sends the heartbeat signals (UDP packets) periodically to other Pgpool-II. Similarly watchdog also receives the signals from other Pgpool-II . If there are no signal for a certain period, watchdog regards is as failure of the Pgpool-II .

query: In this mode, watchdog sends the monitoring queries to other Pgpool-II and checks the response

Caution

In query mode, you need to set num_init_children large enough if you plan to use watchdog. This is because the watchdog process connects to Pgpool-II as a client.

external: This mode disables the built in lifecheck of Pgpool-II watchdog and relies on external system to provide node health checking of local and remote watchdog nodes.

external mode is not available in versions prior to Pgpool-II V3.5.

This parameter can only be set at server start.

wd_monitoring_interfaces_list (string)

Specify a comma separated list of network device names, to be monitored by the watchdog process for the network link state. If all network interfaces in the list becomes inactive (disabled or cable unplugged), the watchdog will consider it as a complete network failure and the Pgpool-II node will commit the suicide. Specifying an ''(empty) list disables the network interface monitoring. Setting it to 'any' enables the monitoring on all available network interfaces except the loopback. Default is '' empty list (monitoring disabled).

wd_monitoring_interfaces_list is not available in versions prior to Pgpool-II V3.5.

This parameter can only be set at server start.

wd_interval (integer)

Specifies the interval between life checks of Pgpool-II in seconds. (A number greater than or equal to 1) Default is 10.

This parameter can only be set at server start.

wd_priority (integer)

This parameter can be used to elevate the local watchdog node priority in the elections to select master watchdog node. The node with the higher wd_priority value will get selected as master watchdog node when cluster will be electing its new master node in the event of old master watchdog node failure. wd_priority is also valid at the time of cluster startup. When some watchdog nodes start up at same time,a node with the higher wd_priority value is selected as a master node. So we should start watchdog nodes in order of wd_priority priority to prevent unintended nodes from being selected as masters.

wd_priority is not available in versions prior to Pgpool-II V3.5.

This parameter can only be set at server start.

wd_ipc_socket_dir (string)

The directory where the UNIX domain socket accepting Pgpool-II watchdog IPC connections will be created. Default is '/tmp'. Be aware that this socket might be deleted by a cron job. We recommend to set this value to '/var/run' or such directory.

wd_ipc_socket_dir is not available in versions prior to Pgpool-II V3.5.

This parameter can only be set at server start.

5.14.7. Lifecheck Heartbeat mode configuration

wd_heartbeat_port (integer)

Specifies the UDP port number to receive heartbeat signals. Default is 9694. wd_heartbeat_port is only applicable if the wd_lifecheck_method is set to 'heartbeat'

This parameter can only be set at server start.

wd_heartbeat_keepalive (integer)

Specifies the interval time in seconds between sending the heartbeat signals. Default is 2. wd_heartbeat_keepalive is only applicable if the wd_lifecheck_method is set to 'heartbeat'

This parameter can only be set at server start.

wd_heartbeat_deadtime (integer)

Specifies the time in seconds before marking the remote watchdog node as failed/dead node, if no heartbeat signal is received within that time. Default is 30 wd_heartbeat_deadtime is only applicable if the wd_lifecheck_method is set to 'heartbeat'

This parameter can only be set at server start.

heartbeat_destination0 (string)

Specifies the IP address or hostname of destination the remote Pgpool-II for sending the heartbeat signals. Multiple destinations can be configured for the heartbeat signals, the number at the end of the parameter name is referred as the "destination number", that starts from 0.

heartbeat_destination is only applicable if the wd_lifecheck_method is set to 'heartbeat'

This parameter can only be set at server start.

heartbeat_destination_port0 (integer)

Specifies the destination port number of the remote Pgpool-II for sending the heartbeat signals. Multiple destinations can be configured for the heartbeat signals, the number at the end of the parameter name is referred as the "destination number", that starts from 0.

heartbeat_destination_port is only applicable if the wd_lifecheck_method is set to 'heartbeat'

This parameter can only be set at server start.

heartbeat_device0 (string)

Specifies the network device name for sending the heartbeat signals to the destination specified by heartbeat_destinationX:heartbeat_destination_portX Different heartbeat devices can be configured for each heartbeat destination by changing the value of X(destination number). at the end of parameter name. The destination index number starts from 0.

heartbeat_device is only applicable if Pgpool-II is started with root privilege. If not, leave it as an empty string ('').

heartbeat_device is only applicable if the wd_lifecheck_method is set to 'heartbeat'

This parameter can only be set at server start.

5.14.8. Lifecheck Query mode configuration

wd_life_point (integer)

Specifies the number of times to retry a failed life check of pgpool-II. Valid value could be a number greater than or equal to 1. Default is 3.

wd_life_point is only applicable if the wd_lifecheck_method is set to 'query'

This parameter can only be set at server start.

wd_lifecheck_query (string)

Specifies the query to use for the life check of remote Pgpool-II. Default is "SELECT 1".

wd_lifecheck_query is only applicable if the wd_lifecheck_method is set to 'query'

This parameter can only be set at server start.

wd_lifecheck_dbname (string)

Specifies the database name for the connection used for the life check of remote Pgpool-II. Default is "template1".

wd_lifecheck_dbname is only applicable if the wd_lifecheck_method is set to 'query'

This parameter can only be set at server start.

wd_lifecheck_user (string)

Specifies the user name for the connection used for the life check of remote Pgpool-II. Default is "nobody".

wd_lifecheck_user is only applicable if the wd_lifecheck_method is set to 'query'

This parameter can only be set at server start.

wd_lifecheck_password (string)

Specifies the password for the user used for the life check of remote Pgpool-II. Default is ''(empty).

wd_lifecheck_password is only applicable if the wd_lifecheck_method is set to 'query'

This parameter can only be set at server start.

5.14.9. Watchdog servers configurations

other_pgpool_hostname0 (string)

Specifies the hostname of remote Pgpool-II server for watchdog node. The number at the end of the parameter name is referred as "server id", and it starts from 0.

This parameter can only be set at server start.

other_pgpool_port0 (integer)

Specifies the port number of the remote Pgpool-II server for watchdog node. The number at the end of the parameter name is referred as "server id", and it starts from 0.

This parameter can only be set at server start.

other_wd_port0 (integer)

Specifies the watchdog port number of the remote Pgpool-II server for watchdog node. The number at the end of the parameter name is referred as "server id", and it starts from 0.

This parameter can only be set at server start.