[pgpool-general: 4747] Re: 【Not execute failover with kamailio】

Noriyuki Hayashi nhayashi at wats.gr.jp
Wed Jun 29 20:50:48 JST 2016


Hi,


Solved this problem that has kamailio config.

#!define DBURL
"postgres://asterisk:password@Float.ing.IP.pgpool:9999/kamailio"
#!ifdef WITH_ASTERISK
#!define DBASTURL
"postgres://asterisk:password@Float.ing.IP.pgpool:9999/asterisk"
#!endif
#!endif

I have changed below as follow.
#!define DBURL
"postgres://asterisk:password@Float.ing.IP.pgpool/kamailio"
#!ifdef WITH_ASTERISK
#!define DBASTURL
"postgres://asterisk:password@Float.ing.IP.pgpool/asterisk"
#!endif
#!endif

I defined the port as 9999 for pgpool2postgres that was not correct.
I removed the definition port as 9999 for pgpool2postgres that is
working all and fine.

I do not know the reason.
By the way I will report to the ALL.

Thank you so much.

Noriyuki Hayashi in Japan


On Mon, 27 Jun 2016 22:27:37 +0900
Noriyuki Hayashi <nhayashi at wats.gr.jp> wrote:

> Hi
> 
> Environment are as follows.
> 1. CentOS-6.6 x86_64 E5-2670*2 384GB Memory.
> 2. postgresql-server-9.4.6
> 3. pgpool-II-pg94-3.5.2-1
> 4. kamailio-4.3.5-2.1
> 5. asterisk-11.21.2-1
> 
> I am testing failover by 1 server shutdown.
> pgpool could not execute failover.sh with kamailio.
> child process received shutdown request signal 3 could execute failover.
> But child process received shutdown request signal 2 could not execute failover.
> If I setup FloatingIP:9999 to kamailio.cfg, 
> I could not connect to 
> "psql-h FloatingIP -p 9999 -U postgres DB.
> 
> pgpool with asterisk has no problem.
> 
> If you give me any advise or hint with much appreciated.
> 
> I am looking forward from any hint and advise.
> 
> Kind regards,
> 
> Noriyuki Hayashi
> 
> Please see below logs.(There is no any error on the kamailio.)
> failover without kamailio
> -----------------------------------------------------------------
> 2016-06-27 21:40:54: pid 22119: LOG:  read from socket failed, remote end closed the connection
> 2016-06-27 21:40:54: pid 22119: LOG:  remote node "Linux_sp0.voiplus.net_9999" is shutting down
> 2016-06-27 21:40:54: pid 22119: LOG:  read from socket failed, remote end closed the connection
> 2016-06-27 21:40:55: pid 22155: LOG:  reading and processing packets
> 2016-06-27 21:40:55: pid 22155: DETAIL:  postmaster on DB node 0 was shutdown by administrative command
> 2016-06-27 21:40:55: pid 22130: LOG:  reading and processing packets
> 2016-06-27 21:40:55: pid 22130: DETAIL:  postmaster on DB node 0 was shutdown by administrative command
> 2016-06-27 21:40:55: pid 22130: LOG:  received degenerate backend request for node_id: 0 from pid [22130]
> 2016-06-27 21:40:55: pid 22119: LOG:  new IPC connection received
> 2016-06-27 21:40:55: pid 22117: LOG:  starting degeneration. shutdown host 192.168.192.136(5432)
> 2016-06-27 21:40:55: pid 22155: LOG:  received degenerate backend request for node_id: 0 from pid [22155]
> 2016-06-27 21:40:55: pid 22119: LOG:  new IPC connection received
> 2016-06-27 21:40:55: pid 22117: LOG:  Restart all children
> 2016-06-27 21:40:55: pid 22130: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22126: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22133: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22124: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22128: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22137: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22135: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22150: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22141: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22144: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22148: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 26646: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22153: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22156: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 27242: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22152: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 26644: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22119: LOG:  new IPC connection received
> 2016-06-27 21:40:55: pid 22147: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22149: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22139: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22119: LOG:  processing sync request from IPC socket
> 2016-06-27 21:40:55: pid 22151: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22132: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22119: LOG:  watchdog node "Linux_sp9.voiplus.net_9999" is requesting to aquire lock for failover command start
> 2016-06-27 21:40:55: pid 22119: LOG:  watchdog node "Linux_sp9.voiplus.net_9999" is requesting to aquire lock for failover command start
> 2016-06-27 21:40:55: pid 22117: LOG:  execute command: /etc/pgpool-II/sbin/failover.sh 0 0 192.168.192.137 /var/lib/pgsql/9.4/data 192.168.192.136
> 2016-06-27 21:40:55: pid 22145: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22154: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22129: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22143: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22140: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 22146: LOG:  child process received shutdown request signal 3
> 2016-06-27 21:40:55: pid 26645: LOG:  child process received shutdown request signal 3
> + FALLING_NODE=0
> + OLDPRIMARY_NODE=0
> + NEW_PRIMARY=192.168.192.137
> + PGDATA=/var/lib/pgsql/9.4/data
> + FALLING_HOST=192.168.192.136
> + LOGFILE=/var/log/failover.log
> + echo '###---------------------------------- starting failover --------------------------###'
> + date
> 
> failover with kamailio
> -------------------------------------------------------------------------------------------
> 2016-06-27 20:48:37: pid 8011: ERROR:  unable to read data from frontend
> 2016-06-27 20:48:37: pid 8011: DETAIL:  EOF encountered with frontend
> 2016-06-27 20:48:49: pid 7976: LOG:  received fast shutdown request
> 2016-06-27 20:48:49: pid 21370: LOG:  stop request sent to pgpool. waiting for termination...
> 2016-06-27 20:48:49: pid 7976: LOG:  shutdown request. closing listen socket
> .2016-06-27 20:48:49: pid 17768: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 18079: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 21245: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 18981: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 19507: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 20720: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 20895: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 13788: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 15988: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 8000: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 15695: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 13085: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 7983: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 19095: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 17949: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 19564: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 21316: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 20088: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 20543: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 20519: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 17473: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 19270: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 19623: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 20603: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 18076: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 18570: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 17005: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 20309: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 7978: LOG:  Watchdog is shutting down
> 2016-06-27 20:48:49: pid 14487: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 18453: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 21385: LOG:  watchdog: de-escalation started
> 2016-06-27 20:48:49: pid 8011: LOG:  child process received shutdown request signal 2
> 2016-06-27 20:48:49: pid 19332: LOG:  child process received shutdown request signal 2
> ............2016-06-27 20:49:01: pid 21385: WARNING:  watchdog failed to ping host"192.168.192.142"
> 2016-06-27 20:49:01: pid 21385: DETAIL:  ping process exits with code: 1
> 2016-06-27 20:49:01: pid 21385: LOG:  watchdog bringing down delegate IP
> 2016-06-27 20:49:01: pid 21385: DETAIL:  if_down_cmd succeeded
> done.
> 2016-06-27 21:23:30: pid 22117: LOG:  Backend status file /var/log/pgpool/pgpool_status does not exist
> 2016-06-27 21:23:30: pid 22117: LOG:  waiting for watchdog to initialize
> 2016-06-27 21:23:30: pid 22119: LOG:  setting the local watchdog node name to "Linux_sp9.voiplus.net_9999"
> 2016-06-27 21:23:30: pid 22119: LOG:  watchdog cluster configured with 1 remote nodes
> 2016-06-27 21:23:30: pid 22119: LOG:  watchdog remote node:0 on 192.168.192.136:9000
> 2016-06-27 21:23:30: pid 22119: LOG:  interface monitoring is disabled in watchdog
> 2016-06-27 21:23:30: pid 22119: LOG:  IPC socket path: "/tmp/.s.PGPOOLWD_CMD.9000"
> 2016-06-27 21:23:30: pid 22119: LOG:  new outbond connection to 192.168.192.136:9000
> 2016-06-27 21:23:30: pid 22119: LOG:  watchdog node state changed from [LOADING] to [INITIALIZING]
> 2016-06-27 21:23:31: pid 22119: LOG:  watchdog node state changed from [INITIALIZING] to [STANDING FOR MASTER]
> 2016-06-27 21:23:31: pid 22119: LOG:  watchdog node state changed from [STANDING FOR MASTER] to [MASTER]
> 2016-06-27 21:23:31: pid 22119: LOG:  I am announcing my self as master/coordinator watchdog node
> 2016-06-27 21:23:31: pid 22119: LOG:  I am the cluster leader node
> 2016-06-27 21:23:31: pid 22119: DETAIL:  our declare coordinator message is accepted by all nodes
> 2016-06-27 21:23:31: pid 22119: LOG:  I am the cluster leader node. Starting escalation process
> 2016-06-27 21:23:31: pid 22117: LOG:  watchdog process is initialized
> 2016-06-27 21:23:31: pid 22119: LOG:  escalation process started with PID:22120
> 2016-06-27 21:23:31: pid 22120: LOG:  watchdog: escalation started
> 2016-06-27 21:23:31: pid 22117: LOG:  Setting up socket for 0.0.0.0:9999
> 2016-06-27 21:23:31: pid 22117: LOG:  Setting up socket for :::9999
> 2016-06-27 21:23:31: pid 22119: LOG:  new IPC connection received
> 2016-06-27 21:23:31: pid 22121: LOG:  2 watchdog nodes are configured for lifecheck
> 2016-06-27 21:23:31: pid 22121: LOG:  watchdog nodes ID:0 Name:"Linux_sp9.voiplus.net_9999"
> 2016-06-27 21:23:31: pid 22121: DETAIL:  Host:"192.168.192.137" WD Port:9000 pgpool-II port:9999
> 2016-06-27 21:23:31: pid 22121: LOG:  watchdog nodes ID:1 Name:"Linux_sp0.voiplus.net_9999"
> 2016-06-27 21:23:31: pid 22121: DETAIL:  Host:"192.168.192.136" WD Port:9000 pgpool-II port:9999
> 2016-06-27 21:23:31: pid 22117: LOG:  pgpool-II successfully started. version 3.5.2 (ekieboshi)
> 2016-06-27 21:23:31: pid 22117: LOG:  find_primary_node: checking backend no 0
> WARNING: interface is ignored: Operation not permitted
> 2016-06-27 21:23:31: pid 22117: LOG:  find_primary_node: primary node id is 0
> 2016-06-27 21:23:31: pid 22134: LOG:  statement: select min(rates_cntry_area_num) as rates_cntry_area_num,min(rates_jpy_voiplus_rate) as rates_jpy_voiplus_rate,rates_unit_time, rates_destination_jp,rates_destination_en from rates where (rates_destination_en like '%Proper%' or rates_destination_en like '%Domestic%') and (rates_jpy_voiplus_rate > 0 and rates_jpy_voiplus_rate < 23) and rates_intl_num='002' group by rates_destination_en,rates_destination_jp,rates_unit_time order by random() limit 6
> 2016-06-27 21:23:32: pid 22125: LOG:  set SO_REUSEPORT option to the socket
> 2016-06-27 21:23:32: pid 22125: LOG:  creating watchdog heartbeat receive socket.
> 2016-06-27 21:23:32: pid 22125: DETAIL:  set SO_REUSEPORT
> 2016-06-27 21:23:32: pid 22127: LOG:  set SO_REUSEPORT option to the socket
> 2016-06-27 21:23:32: pid 22127: LOG:  creating socket for sending heartbeat
> 2016-06-27 21:23:32: pid 22127: DETAIL:  set SO_REUSEPORT
> 2016-06-27 21:23:34: pid 22119: LOG:  new watchdog node connection is received from "192.168.192.136:4535"
> 2016-06-27 21:23:45: pid 22120: WARNING:  watchdog failed to ping
> host"192.168.192.142"
> 2016-06-27 21:23:45: pid 22120: DETAIL:  ping process exits with code: 1
> -------------------------------------------------------------------------------------------
> 
> 
> _______________________________________________
> pgpool-general mailing list
> pgpool-general at pgpool.net
> http://www.pgpool.net/mailman/listinfo/pgpool-general

/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/

Telecommunications Association License No. A-18-9191
Government Resell  License No. 301039703002
              WATS CO.,LTD.   
  Kawana Bldg, 5F  Kamata
Ota-ku Tokyo, 144-0052 JAPAN
       Phone 81-50-5830-5940 
       Ext&Mobile:201 VoiceMailDirect:201*1
       FAX   81-50-5830-5941
            http://wats.gr.jp
          Mail: wats @ wats.gr.jp
Please remove the space between @ as double side

Key fingerprint = B53D FF2F BFEA FDA8 1439  38AA 8281 9A3E C9B6 2FC9

/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/



More information about the pgpool-general mailing list