[pgpool-general: 1955] pgpool-II 3.3.0 and pgpoolAdmin 3.3.0 officially released

Yugo Nagata nagata at sraoss.co.jp
Tue Jul 30 18:39:56 JST 2013

Pgpool Global Development Group is pleased to announce the availability of
pgpool-II 3.3.0 and pgpoolAdmin 3.3.0.

You can download them from here:

(pgpool-II)    http://www.pgpool.net/download.php?f=pgpool-II-3.3.0.tar.gz
(pgpoolAdmin)  http://www.pgpool.net/download.php?f=pgpoolAdmin-3.3.0.tar.gz  

V3.3's new features are:

    Enhancements for watchdog
        New monitoring method of watchdog lifecheck using heartbeat signal
        Interlocking of failover/failback script
        Secure watchdog communication
        PCP command for retrieving the watchdog status
        Import PostgreSQL 9.2 raw parser
        New pgpool_setup tool
        Support for using CREATE EXTENSION to install pgpool specific extensions
        regression test suit

Release Note:

pgpool-II 3.3.0

      This is the first version of pgpool-II 3.3 series.
      That is, a "major version up" from 3.2 series.


* Incompatible changes

    - All the follwing are about watchdog.
      See "New features" section below for details of these changes.

      - Default monitoring method was changed from query mode to heartbeat mode.

      - Failover/failback commands are executed in only one pgpool-II.

      - In default, all the query caches on shared memory are cleared when
        standby pgpool-II escalates to active.

      - Database name, user name, and password used for monitring other
        pgpool-II by query are specified by dedicated parameters.
        Previously, template1, recovery_user, and recovery_password are used.


* New features

  ** Watchdog

    - Add a new monitring method using heartbeat signal of UDP packet in
      lifecheck. (Yugo Nagata)

      You can select a method from either of two methods, "heartbeat" mode or
      "query" mode.

      The heartbeat mode is the new method. In this mode, watchdog monitors
      other pgpool-II process by using heartbeat signal. Watchdog receives
      heartbeat signals sent by other pgpool-II periodically. If there are no
      signal for a certain period, watchdog regards it as failure of the
      pgpool-II. For redundancy you can use multiple networks for heartbeat
      exchange between pgpool-IIs. This mode is default and recommended one.

      The query mode is the conventional method. In this mode, watchdog monitors
      pgpool-II's service rather than process. Watchdog sends queries to other
      pgpool-II and checks the response. Note that this method requires connections
      from other pgpool-IIs, so it would fail motoring if num_init_children isn't
      large enough. This mode is deprecated and left for backward compatibility.

      Add these new parameters:
          - wd_lifecheck_method
          - wd_heartbeat_port
          - wd_heartbeat_keepalive
          - wd_heartbeat_deadtime
          - heartbeat_destinationX
          - heartbeat_destination_portX
          - heartbeat_deviceX

    - Add interlocking mechanism of exclusive failover/failback command
      execution. (Yugo Nagata)

      When using multiple pgpool-IIs with watchdog enabled, failover commands
      (failover_command, failback_command, and follow_master_command) get
      executed only at one pgpool-II.

      Previously, these command got executed at all pgpool-IIs.

    - Add authentication mechanism for watchdog packet communication.
      (Yugo Nagata)

      Watchdog packets (include heartbeat signal) from pgpool-II with wrong
      authentication key are rejected. All pgpool-IIs must have the same key,
      which is specified wd_authkey parameter in pgpool.conf. pgpool-II with
      wrong authkey can't even start watchdog, because the startup packet is
      rejected by other pgpool-IIs.

    - Add clear_memqcache_on_escalation parameter. (Yugo Nagata)

      If this is on, all the query caches on shared memory are cleared when
      standby pgpool-II escalates to active.

      This is aimed to prevent the new active pgpool-II from using inconsistent
      query caches with the previous active.

    - Add wd_escalation_command parameter. (Yugo Nagata)

      This specifies command which is executed at escalation on the new active
      pgpool-II server. The timing is just before virtual IP is brought up.

    - Add parameters wd_lifecheck_dbname, wd_lifecheck_user, and
      wd_lifecheck_password. (Yugo Nagata)

      These parameters specify the database name, the user name, and password
      used in query mode lifecheck of watchdog . Previously, these are hard
      coded to use template1, recovery_user, and recovery_password.

    - When delegate_IP parameter is emply, viertual IP is neither brought up
      nor switched. (Yugo Nagata)

      This allows multi-master like configuration without virtual IP.

    - Add pcp_watchdog_info command (Yugo Nagata)

      This is PCP command for retrieving the watchdog status.

  ** Others

    - Import PostgreSQL 9.2 raw parser. (Nozomi Anzai, Tatsuo Ishii)

    - Add a tool called pgpool_setup to set up pgpool-II and PostgreSQL
      temporary installation in current directory for *testing* purpose.
      (Tatsuo Ishii)

      usage: pgpool_setup [-m r|s][-n num_clusters][--no-stop]
         -m s: create an installation as streaming replication mode.
         (the default)
         -m r: create an installation as native replication mode.
         -n num_clusters: create num_clusters PostgreSQL database cluster nodes
         -p base_port: specify base port. The first PostgreSQL node's port is
                       base_port. Second PostgreSQL node's port is base_port+1
                       ... nth PostgreSQL node's port is base_port+n-1, pgpool
                       port is base_port+n, pcp port is base_port+n+1.
         --no-stop: do not stop pgpool and PostgreSQL after the work

    - Support installation method using CREATE EXTENSION for pgpool-recovery
      and pgpool-regclass. (Tatsuo Ishii)

      Older installtion method is still preserved.

      Note: extension names are "pgpool_recovery" and "pgpool_regclass", not
      "pgpool-recovery" and "pgpool-regclass" because latters are not
      convenient in using CREATE EXTENSION command (requires double quotes).

    - Add a function "pgpool_pgctl()" which enebles to execute
      pg_ctl stop/restart/reload (except for start) by SQL. (Nozomi Anzai)

      $ psql sales -c "select pgpool_pgctl('reload', 'fast')";
      (1 row)

      This function always ignores the actual result and returns 't', so the
      user can't know if pg_ctl succeeded or failed. To use this we have to set
      the custom variable for security which limits the users to execute pg_ctl
      who has the permission of data directory.

    - Add shell scripts to install pgpool-II and pgpoolAdmin by RPM.
      (Nozomi Anzai, Yugo Nagata)

      To make the installer package execute getsources.sh, and the directory
      named "work" will be created. And you rpmbuild each spec files in work/,
      put RPMs into work/installer and make tar ball of work/installer.
      The installer does not only install RPMs but also edit postgresql.conf,
      pgpool.conf, pg_hba.conf, recovery.conf and scripts for failover and
      online recovery.

      This assumes two-nodes configuration and the install script have to be
      executed in both nodes.

    - Add new parameter "search_primary_node_timeout".
      (Muhammad Usama, Tatsuo Ishii)

      The parameter specifies the maximum amount of time in seconds to
      search for a primary node when a failover scenario occurs. Patch
      contributed by Muhammad Usama.  Japanese doc and slight editing of
      English doc by Tatsuo Ishii.

    - Chinese tutorials for memqcache and watchdog. (Bambo Huang)

    - Add regression test suit. (Tatsuo Ishii)


* Bug fixes

    - Consider timeout waiting for compeletion of failback request in on line
      recovery. (Tatsuo Ishii)

      This will prevent the situation that recovery operation continues forever
      and we cannot even shutdown pgpool-II main process. This could happen
      especially while executing follow master command.

    - Fix a bug that %H of follow_master_command is not assigned correctly the
      new primary node in stream replication mode.
      (Tatsuo Ishii)

    - Fix not to execute escalation when the pgpool-II which is already active
      receives down notification from other pgpool-II. (Yugo Nagata)

    - Fix wd_create_send_socket() not to execute select() before connect().
      (Yugo Nagata)

      How select() works on an unconnected socket is undefined, and differs
      between platform. On Linux, this returns 2 and it is eventually harmless.
      However, on Soraris, this returns 0 and it is indistinguishable from time
      timeout, so watchdog wouldn't work correctly.

    - Fix error when pgpool_regclass is not installed. (Tatsuo Ishii)

      The query used in pool_has_pgpool_regclass() fails if pgpool_regclass
      does not exist. The bug was introduced in 3.2.4.  See [pgpool-general:
      1722] for more details.

      [pgpool-general: 1722] [PgPool-II 3.2.4] pgpool_regclass now mandatory?

    - Fix do_query() not to hang when PostgreSQL returns an error.
      (Tatsuo Ishii)

      The typical symptom is "I see SELECT is keep on running according to
      pg_stat_activity". To fix this pgpool-II just exits the process and
      kill the existig connection.  This is not gentle but at this point I
      believe this is the best solution.

    - Fix possible deadlock during failover with watchdog enabled.
      (Yugo Nagata)

      This is reported in Bug track #54 by arshu arora

    - Fix unnecessary degeneration caused by error on commit. (Tatsuo Ishii)

      In master slave mode, if master gets an error at commit, while other
      slaves are normal at commit, we don't need to degenrate any backend
      because it is likely that the "kind mismatch error" was caused by a
      deferred trigger.

    - Fix bug with do_query which causes hung in extended protocol.
      (Tatsuo Ishii)

      This problem could occur when insert lock is enabled and
      pgpool_catalog.insert_lock exists, See [pgpool-general: 1684] for more

      [pgpool-general: 1684] insert_lock hangs

    - Fix possible failure of query cache invalidation for DML in transaction.
      (Tatsuo Ishii)

      SELECT * FROM t1;
      DELETE FROM t2 WHERE i = 0;
      INSERT INTO t1(i) VALUES(1);

      SELECT * FROM t1;

      At commit pgpool tries to delete cache for t2 but failes because
      there's no oid table entry for t2. Problem is, it fails to check oid
      table for t1. So cahce for t1 remains and the last SELECT incorrectly
      returns cached data. Fix is, continuing to check oid table entries.

      This is reported in Bug track #58 by wms

    - Fix to register pgpool_regclass in pg_catalog schema (Tatsuo Ishii)

      This is necessary to deal with clients which restricts schema search path
      to pg_catalog only. Postgres_fdw is such a client.

    - Fix a hang of "pgpool -m f stop" (Tatsuo Ishii)

      This is caused by unmanaged pgpool children remaining. This could happen
      when multiple PostgreSQLs are going down or even when pgpool is started
      before PostgreSQL startup.

    - Fix a potential crash in pg_md5 command (Muhammad Usama)

    - Fix a segmentation fault that occurs when on memory query cache enabled
      and the query is issued in extended query mode and the result is too large
      (Tatsuo Ishii)

      This is reported in Bug track #63 by harukat.
      Analysis and a test case are also provided.

      #63 Child process was terminated by segmentation fault with memcached

    - Fix a segmentation fault of a child process that occurs when a startup
      packet has no PostgreSQL user information (Yugo Nagata)

      You can reproduce it by

          $ psql -p 9999 -U ''

      If enable_pool_hba is on, a child process terminates by segmentation
      fault. Otherwise if enable_pool_hba is off, the error message is

          ERROR: pool_discard_cp: cannot get connection pool for user (null)
                 database (null)

      In both cases, psql terminates with no message on frontend.

      In the fixed version, if PostgreSQL user is not specified in startup packet,
      the message as following is output to both log and frontend. This is
      the same behavior as PostgreSQL.

          FATAL: no PostgreSQL user name specified in startup packet

    - Fix memory allocation logic in extended query processing with on-memory
      query cache enabled (Tatsuo Ishii)

      When very long query string (> 1024 bytes) supplied in extended query
      with bind parameters, it fails to allocate enough memory.

    - Fix to verify the backend node number in pcp_recovery_node (Yugo Nagata)

      When an invalid number is used, null value is passed as an arguments
      of recovery script, and this causes a malfunction. In especially,
      rsync may delete unrelated files in basebackup scripts.

    - Fix a segmentation fault on main process that could occures after backend
      error detection (Tatsuo Ishii)

      This is reported in Bug track #62 by tuomas.

      #62 Slave network outage causes a segmentation fault on main process

    - Fix bug with health check when used with child_life_time (Tatsuo Ishii)

      Failover could happen even if the backend was running fine.
      This problem is reported in [pgpool-general: 1892] by larisa sabban.

      [pgpool-general: 1892] Pgpool is unable to connect backend PostgreSQL

    - Fix bug in parsing prepared statements with transaction handling in
      replication mode (Tatsuo Ishii)

      Parse() automatically starts a transaction for non SELECT query to keep
      consistency among nodes in replication mode. But this wasn't closed. If
      wrong query comes in, the transaction goes into an abort state but pgpool
      does not close the transaction. Thus next query causes error because the
      transaction is still in abort status.
      This problem was reported in [pgpool-general: 1877] by Sean Hogan.

      [pgpool-general: 1877] current transaction is aborted, commands ignored


* Enhancements

    - Add mention about "-D" option to the man page.
      (Tatsuo Ishii)

    - Fix to restart watchdog processes automatically when these exit abnormally.
      (Yugo Nagata)

    - Add more error checks and reportings to functions executing ping command
      with watchdog enabled. (Tatsuo Ishii)

    - Replace some unsafe functions, sprintf and strncpy, with more safe ones,
      snprintf and strlcpy respectively. (Yugo Nagata)

    - Replace "sticky bit" to "setuid bit" in log message, comments and
      funcation names. (Yugo Nagata)

      These words were used mistakenly and caused confusion.

    - Fix description on SSL in pool_hba.conf.sample. (Tatsuo Ishii)

    - Allow to load balancing in an explicit transaction in replication mode.
      (Tatsuo Ishii)

        The condition to allow the load balancing is as follows:
        1) replicate_select is off
        2) no writing functions are used
        3) transaction isolation level is not serializable
        4) no DML/DDL are issued in the transaction

    - Chinese manual is updated to the latest especially about watchdog. (Bambo Huang)

    - Add mention about "-D" option to the man page. (Tatsuo Ishii)

    - Move ssl_ca_cert and ssl_ca_cert_dir descriptions to the SSL section
      in the document (Yugo Nagata)

    - Add ssl_ca_cert and ssl_ca_cert_dir descriptions to the japanese document
      (Yugo Nagata)

pgpoolAdmin 3.3

pgpoolAdmin 3.3 adopts pgpool-II 3.3.

** New features

    - Add new parameters for pgpool-II 3.3 (Nozomi Anzai)
      wd_authkey, wd_escalation_command, clear_memqcache_on_escalation,
      wd_lifecheck_method, wd_heartbeat_port, wd_heartbeat_keepalive,
      wd_heartbeat_deadtime wd_lifecheck_dbname, wd_lifecheck_user,
      wd_lifecheck_password, heartbeat_destination, heartbeat_destination_port,
    - Add a watchdog statuse view in status page  (Nozomi Anzai, Yugo Nagata)
      This show watchdog statuses of pgpool-IIs and the virtual IP etc. when
      watchdog is enabled.

    - Add functionality to stop/reload/restart PostgreSQL from GUI
      (Nozomi Anzai)

      pgpool_pgctl function need to be installed in the backends. This function
      is a new feature of pgpool-II 3.3 provided by pgpool_recovery extension.
      In addition, a superuser whose name is same as the pgpoolAdmin's login user
      also must exists in the backend.

    - Add functionality to add/remove a backend from GUI (Nozomi Anzai)

    - Show the hostname on which pgpoolAdmin is working in status page
      (Andrew Bruce, Nozomi Anzai)

    - Add fuctionality to show limited lines in log view  (Yugo Nagata)

** Bug fixes

    - Fix to not write deprecated parmeters to when updating pgpool.conf 
       (Yugo Nagata)

Yugo Nagata <nagata at sraoss.co.jp>

More information about the pgpool-general mailing list