8.4. AWS Configuration Example

This tutorial explains the simple way to try "Watchdog" on AWS and using the Elastic IP Address as the Virtual IP for the high availability solution.

Note: You can use watchdog with Pgpool-II in any mode: replication mode, master/slave mode and raw mode.

8.4.1. AWS Setup

For this example, we will use two node Pgpool-II watchdog cluster. So we will set up two Linux Amazon EC2 instances and one Elastic IP address. So for this example, do the following steps:

8.4.2. Pgpool-II configurations

Mostly the Pgpool-II configurations for this example will be same as in the Section 8.2, except the delegate_IP which we will not set in this example instead we will use wd_escalation_command and wd_de_escalation_command to switch the Elastic IP address to the master/Active Pgpool-II node.

8.4.2.1. Pgpool-II configurations on Instance-1

     use_watchdog = on
     delegate_IP = ''
     wd_hostname = 'instance-1-private-ip'
     other_pgpool_hostname0 = 'instance-2-private-ip'
     other_pgpool_port0 = 9999
     other_wd_port0 = 9000
     wd_escalation_command = '$path_to_script/aws-escalation.sh'
     wd_de_escalation_command = '$path_to_script/aws-de-escalation.sh'
    

8.4.2.2. Pgpool-II configurations on Instance-2

     use_watchdog = on
     delegate_IP = ''
     wd_hostname = 'instance-2-private-ip'
     other_pgpool_hostname0 = 'instance-1-private-ip'
     other_pgpool_port0 = 9999
     other_wd_port0 = 9000
     wd_escalation_command = '$path_to_script/aws-escalation.sh'
     wd_de_escalation_command = '$path_to_script/aws-de-escalation.sh'
    

8.4.3. escalation and de-escalation Scripts

Create the aws-escalation.sh and aws-de-escalation.sh scripts on both instances and point the wd_escalation_command and wd_de_escalation_command to the respective scripts.

Note: You may need to configure the AWS CLI first on all AWS instances to enable the execution of commands used by wd-escalation.sh and wd-de-escalation.sh. See configure AWS CLI

8.4.3.1. escalation script

This script will be executed by the watchdog to assign the Elastic IP on the instance when the watchdog becomes the active/master node. Change the INSTANCE_ID and ELASTIC_IP values as per your AWS setup values.

aws-escalation.sh:

     #! /bin/sh

     ELASTIC_IP=35.163.178.3
     # replace it with the Elastic IP address you
     # allocated from the aws console
     INSTANCE_ID=i-0a9b64e449b17ed4b
     # replace it with the instance id of the Instance
     # this script is installed on

     echo "Assigning Elastic IP $ELASTIC_IP to the instance $INSTANCE_ID"
     # bring up the Elastic IP
     aws ec2 associate-address --instance-id $INSTANCE_ID --public-ip $ELASTIC_IP

     exit 0
    

8.4.3.2. de-escalation script

This script will be executed by watchdog to remove the Elastic IP from the instance when the watchdog resign from the active/master node.

aws-de-escalation.sh:

     #! /bin/sh

     ELASTIC_IP=35.163.178.3
     # replace it with the Elastic IP address you
     # allocated from the aws console

     echo "disassociating the Elastic IP $ELASTIC_IP from the instance"
     # bring down the Elastic IP
     aws ec2 disassociate-address --public-ip $ELASTIC_IP
     exit 0
    

AWS Command References

"Configure AWS CLI", AWS Documentation: Configuring the AWS Command Line Interface.

"associate-address", AWS Documentation: associate-address reference.

"disassociate-address", AWS Documentation: disassociate-address reference.

8.4.4. Try it out

Start Pgpool-II on each server with "-n" switch and redirect log messages to the pgpool.log file. The log message of master/active Pgpool-II node will show the message of Elastic IP assignment.

    LOG:  I am the cluster leader node. Starting escalation process
    LOG:  escalation process started with PID:23543
    LOG:  watchdog: escalation started
         Assigning Elastic IP 35.163.178.3 to the instance i-0a9b64e449b17ed4b
     {
     "AssociationId": "eipassoc-39853c42"
     }
    
    LOG:  watchdog escalation successful
    LOG:  watchdog escalation process with pid: 23543 exit with SUCCESS.
   

Confirm to ping to the Elastic IP address.

    [user@someserver]$ ping 35.163.178.3
    PING 35.163.178.3 (35.163.178.3) 56(84) bytes of data.
    64 bytes from 35.163.178.3: icmp_seq=1 ttl=64 time=0.328 ms
    64 bytes from 35.163.178.3: icmp_seq=2 ttl=64 time=0.264 ms
    64 bytes from 35.163.178.3: icmp_seq=3 ttl=64 time=0.412 ms
   

Try to connect PostgreSQL by "psql -h ELASTIC_IP -p port".

    [user@someserver]$ psql -h 35.163.178.3 -p 9999 -l
   

8.4.5. Switching Elastic IP

To confirm if the Standby server acquires the Elastic IP when the Active server becomes unavailable, Stop the Pgpool-II on the Active server. Then, the Standby server should start using the Elastic IP address, And the Pgpool-II log will show the below messages.

         LOG:  remote node "172.31.2.94:9999 [Linux ip-172-31-2-94]" is shutting down
     LOG:  watchdog cluster has lost the coordinator node
    
    LOG:  watchdog node state changed from [STANDBY] to [JOINING]
    LOG:  watchdog node state changed from [JOINING] to [INITIALIZING]
    LOG:  I am the only alive node in the watchdog cluster
    HINT:  skipping stand for coordinator state
    LOG:  watchdog node state changed from [INITIALIZING] to [MASTER]
    LOG:  I am announcing my self as master/coordinator watchdog node
    LOG:  I am the cluster leader node
    DETAIL:  our declare coordinator message is accepted by all nodes
    LOG:  I am the cluster leader node. Starting escalation process
    LOG:  escalation process started with PID:23543
    LOG:  watchdog: escalation started
         Assigning Elastic IP 35.163.178.3 to the instance i-0dd3e60734a6ebe14
     {
     "AssociationId": "eipassoc-39853c42"
     }
    
    LOG:  watchdog escalation successful
    LOG:  watchdog escalation process with pid: 61581 exit with SUCCESS.
   

Confirm to ping to the Elastic IP address again.

    [user@someserver]$ ping 35.163.178.3
    PING 35.163.178.3 (35.163.178.3) 56(84) bytes of data.
    64 bytes from 35.163.178.3: icmp_seq=1 ttl=64 time=0.328 ms
    64 bytes from 35.163.178.3: icmp_seq=2 ttl=64 time=0.264 ms
    64 bytes from 35.163.178.3: icmp_seq=3 ttl=64 time=0.412 ms
   

Try to connect PostgreSQL by "psql -h ELASTIC_IP -p port".

    [user@someserver]$ psql -h 35.163.178.3 -p 9999 -l