[pgpool-general: 1454] Re: Fwd: Pgpool 3.2.2 issue on AIX
Daniele Di Vito
adivitog at gmail.com
Tue Mar 5 02:34:42 JST 2013
Hi Tatsuo, as promised I've tested pgpool 3.2.3 on AIX 5.2
it behaves in a really strange way, at the beginning when I was starting
pgpool it was seeming to be working pretty fine,
after some stop and start it became to behave really strangely.
it seems to mess with some cached status does exists some cache_status
file? i removed backend files .s.PGSQL.9898 % .s.PGSQL.9999 butit doesn't
change anything.
No changes in pgpool.conf nor in postgresql.conf
it returned to loose communication with the main process as described for
3.2.2 version, the main process abnormally exit
I've tried to change some settings in pgpool.conf but it doesn't take
effets on pool behaviour
For example I tried to increase child life time cause i was suspecting some
issue on child process destruction
child_life_time = 100000000000
# Pool exits after being idle for this
many seconds
child_max_connections = 0
# Pool exits after receiving that many
connections
# 0 means no exit
connection_life_time = 0
# Connection to backend closes after
being idle for this many seconds
# 0 means no close
client_idle_limit = 0
I attach debug output truss output and pgpool.conf
Hoping that someone can help me to solve this trouble on AIX.
Thanks
Great!
>
> Yes, I've just install the 3.2.3 et it seems to be working great!
>
> Obviouslly I will test it properly and I will let you know how this
> version works on AIX.
>
> Thanks again
>
>
>
> 2013/2/22 Tatsuo Ishii <ishii at postgresql.org>
>
>> I'm going to check the data you posted.
>>
>> In the meatime, I think it is posiible your problem is caused by the
>> bug fixed in pgpool-II 3.2.3, especiall if the problem goes away by
>> disabling ealth checking. Can you try 3.2.3?
>> --
>> Tatsuo Ishii
>> SRA OSS, Inc. Japan
>> English: http://www.sraoss.co.jp/index_en.php
>> Japanese: http://www.sraoss.co.jp
>>
>> > Thanks for your quick reply Tatsuo,
>> >
>> > Before to get your reply i tried to understand where the main process
>> stops
>> > adding some print in the line right above every exit call in the main.c
>> > class.
>> > I found out that it never stops calling the exit command in an explicit
>> > way.
>> >
>> > After your reply I tracked it's behaviour using truss.
>> > Truss output is in the attched file.
>> >
>> >
>> > Commenting the dup2 file in the demonize function, I'd been able to get
>> > which error happens just before the process "death".
>> >
>> > This is the error:
>> >
>> > pool_flush_it: write failed to backend (0). reason: Socket is not
>> connected
>> > offset: 0 wlen: 41
>> >
>> > it happens after the first health_check call.
>> >
>> > it seems that the socket is not connected to the local backend (backend
>> 0
>> > is on the same host where pgpool is running) but stragely the
>> replication
>> > on the backend 0 it work normally so I think that it's connected.
>> >
>> > The real trouble is that pgppol will never check for failover or
>> failback
>> > loosing the main process.
>> >
>> > The pgsql version of every backend is 8.3. I attach the pgpool config
>> file
>> > too.
>> >
>> >
>> > Thanks again for your help
>> >
>> >
>> > --
>> >
>> > Daniele Di vito
>> >
>> >
>> > 2013/2/21 Tatsuo Ishii <ishii at postgresql.org>
>> >
>> >> > HI everybody, I've compiled pgpool 3.2.2 on AIX 5.2.
>> >> >
>> >> > I configured the pool for using replication mode. The configuration
>> is
>> >> > working really fine on some linux virtual machine, but when I try to
>> use
>> >> > pgpool with the same configuration on AIX I have a big trouble.
>> >> >
>> >> > Starting with "pgpool -d" the server seems to be starting normally.
>> it
>> >> > create pcp process and it create the pool connections waiting for
>> >> > connection requests.
>> >> >
>> >> > When I lunch a "ps -fu postgres | grep pgpool" i get this output:
>> >> >
>> >> >
>> >> >
>> >> > postgres 62164 1 0 10:05:48 - 0:00 pgpool: wait for
>> >> > connection request
>> >> > postgres 75470 1 0 10:05:48 - 0:00 pgpool: PCP: wait
>> for
>> >> > connection request
>> >> > postgres 84072 1 0 10:05:48 - 0:00 pgpool: wait for
>> >> > connection request
>> >> > postgres 96828 1 0 10:05:48 - 0:00 pgpool: wait for
>> >> > connection request
>> >> > postgres 100026 1 0 10:05:48 - 0:00 pgpool: wait for
>> >> > connection request
>> >> > postgres 106670 1 0 10:05:47 - 0:00 pgpool: wait for
>> >> > connection request
>> >> > postgres 109864 1 0 10:05:48 - 0:00 pgpool: worker
>> process
>> >> > postgres 116412 1 0 10:05:48 - 0:00 pgpool: wait for
>> >> > connection request
>> >> >
>> >> > but, as you can see looking at the output listed above,no pgpool
>> daemon
>> >> is
>> >> > running and every subprocess created by it now have as ppid 1.
>> >> >
>> >> > if I look into the pgpool.pid i get a pid that is not running on the
>> AIX
>> >> > machine.
>> >> > Obviously if i try to stop pgpool it says that the process is not
>> >> running
>> >> > so i have to kill every process and to remove every temporary file
>> >> manually.
>> >> >
>> >> > If i run it without a daemon using "pgppool -n"
>> >> >
>> >> > the pgpool -n process is listed for some minutes in the "ps -fu
>> >> postgres |
>> >> > grep pgpool" and every subprocess have the right ppid.
>> >> > Some minutes later i get the same output I listed for the "pgpool -d"
>> >> > command start.
>> >> >
>> >> > Any idea on how to solve this trouble?
>> >> >
>> >> > I've already tried to find some error while in debug mode, but no
>> error
>> >> > listed.
>> >>
>> >> Does AIX have something like "strace" or "truss"? If so, taking a
>> >> system call trace by using it, may provide valuable information. You
>> >> take system call trace until pgpool-II parent process disappears.
>> >> --
>> >> Tatsuo Ishii
>> >> SRA OSS, Inc. Japan
>> >> English: http://www.sraoss.co.jp/index_en.php
>> >> Japanese: http://www.sraoss.co.jp
>> >>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20130304/9fb758c2/attachment-0001.html>
-------------- next part --------------
[1] 100852
[postgres at bldlam:/usr/local/pgpool-II]# 2013-03-04 17:44:18 DEBUG: pid 72000: key: listen_addresses
2013-03-04 17:44:18 DEBUG: pid 72000: value: '*' kind: 4
2013-03-04 17:44:18 DEBUG: pid 72000: key: port
2013-03-04 17:44:18 DEBUG: pid 72000: value: 9999 kind: 2
2013-03-04 17:44:18 DEBUG: pid 72000: key: socket_dir
2013-03-04 17:44:18 DEBUG: pid 72000: value: '/tmp' kind: 4
2013-03-04 17:44:18 DEBUG: pid 72000: key: pcp_port
2013-03-04 17:44:18 DEBUG: pid 72000: value: 9898 kind: 2
2013-03-04 17:44:18 DEBUG: pid 72000: key: pcp_socket_dir
2013-03-04 17:44:18 DEBUG: pid 72000: value: '/tmp' kind: 4
2013-03-04 17:44:18 DEBUG: pid 72000: key: backend_hostname0
2013-03-04 17:44:18 DEBUG: pid 72000: value: 'bldlam' kind: 4
2013-03-04 17:44:18 DEBUG: pid 72000: key: backend_port0
2013-03-04 17:44:18 DEBUG: pid 72000: value: 5445 kind: 2
2013-03-04 17:44:18 DEBUG: pid 72000: pool_config: port slot number 0
2013-03-04 17:44:18 DEBUG: pid 72000: key: backend_weight0
2013-03-04 17:44:18 DEBUG: pid 72000: value: 1 kind: 2
2013-03-04 17:44:18 DEBUG: pid 72000: pool_config: weight slot number 0 weight: 1.000000
2013-03-04 17:44:18 DEBUG: pid 72000: key: backend_data_directory0
2013-03-04 17:44:18 DEBUG: pid 72000: value: '/usr/local/pgsql-test/data' kind: 4
2013-03-04 17:44:18 DEBUG: pid 72000: key: backend_flag0
2013-03-04 17:44:18 DEBUG: pid 72000: value: 'ALLOW_TO_FAILOVER' kind: 4
2013-03-04 17:44:18 DEBUG: pid 72000: extract_string_tokens: token: ALLOW_TO_FAILOVER
2013-03-04 17:44:18 DEBUG: pid 72000: pool_config: allow_to_failover on
2013-03-04 17:44:18 DEBUG: pid 72000: pool_config: slot number 0 flag: 0000
2013-03-04 17:44:18 DEBUG: pid 72000: key: backend_hostname1
2013-03-04 17:44:18 DEBUG: pid 72000: value: 'devlam0' kind: 4
2013-03-04 17:44:18 DEBUG: pid 72000: key: backend_port1
2013-03-04 17:44:18 DEBUG: pid 72000: value: 5431 kind: 2
2013-03-04 17:44:18 DEBUG: pid 72000: pool_config: port slot number 1
2013-03-04 17:44:18 DEBUG: pid 72000: key: backend_weight1
2013-03-04 17:44:18 DEBUG: pid 72000: value: 1 kind: 2
2013-03-04 17:44:18 DEBUG: pid 72000: pool_config: weight slot number 1 weight: 1.000000
2013-03-04 17:44:18 DEBUG: pid 72000: key: backend_data_directory1
2013-03-04 17:44:18 DEBUG: pid 72000: value: '/usr/local/bldlam/pgsql/data' kind: 4
2013-03-04 17:44:18 DEBUG: pid 72000: key: backend_flag1
2013-03-04 17:44:18 DEBUG: pid 72000: value: 'ALLOW_TO_FAILOVER' kind: 4
2013-03-04 17:44:18 DEBUG: pid 72000: extract_string_tokens: token: ALLOW_TO_FAILOVER
2013-03-04 17:44:18 DEBUG: pid 72000: pool_config: allow_to_failover on
2013-03-04 17:44:18 DEBUG: pid 72000: pool_config: slot number 1 flag: 0000
2013-03-04 17:44:18 DEBUG: pid 72000: key: enable_pool_hba
2013-03-04 17:44:18 DEBUG: pid 72000: value: off kind: 1
2013-03-04 17:44:18 DEBUG: pid 72000: key: authentication_timeout
2013-03-04 17:44:18 DEBUG: pid 72000: value: 0 kind: 2
2013-03-04 17:44:18 DEBUG: pid 72000: key: ssl
2013-03-04 17:44:18 DEBUG: pid 72000: value: off kind: 1
2013-03-04 17:44:18 DEBUG: pid 72000: key: num_init_children
2013-03-04 17:44:18 DEBUG: pid 72000: value: 6 kind: 2
2013-03-04 17:44:18 DEBUG: pid 72000: key: max_pool
2013-03-04 17:44:18 DEBUG: pid 72000: value: 4 kind: 2
2013-03-04 17:44:18 DEBUG: pid 72000: key: child_life_time
2013-03-04 17:44:18 DEBUG: pid 72000: value: 100000000000 kind: 2
2013-03-04 17:44:18 DEBUG: pid 72000: key: child_max_connections
2013-03-04 17:44:18 DEBUG: pid 72000: value: 0 kind: 2
2013-03-04 17:44:18 DEBUG: pid 72000: key: connection_life_time
2013-03-04 17:44:18 DEBUG: pid 72000: value: 0 kind: 2
2013-03-04 17:44:18 DEBUG: pid 72000: key: client_idle_limit
2013-03-04 17:44:18 DEBUG: pid 72000: value: 0 kind: 2
2013-03-04 17:44:18 DEBUG: pid 72000: key: log_destination
2013-03-04 17:44:18 DEBUG: pid 72000: value: 'stderr' kind: 4
2013-03-04 17:44:18 DEBUG: pid 72000: key: print_timestamp
2013-03-04 17:44:18 DEBUG: pid 72000: value: on kind: 1
2013-03-04 17:44:18 DEBUG: pid 72000: key: log_connections
2013-03-04 17:44:18 DEBUG: pid 72000: value: off kind: 1
2013-03-04 17:44:18 DEBUG: pid 72000: key: log_hostname
2013-03-04 17:44:18 DEBUG: pid 72000: value: off kind: 1
2013-03-04 17:44:18 DEBUG: pid 72000: key: log_statement
2013-03-04 17:44:18 DEBUG: pid 72000: value: off kind: 1
2013-03-04 17:44:18 DEBUG: pid 72000: key: log_per_node_statement
2013-03-04 17:44:18 DEBUG: pid 72000: value: off kind: 1
2013-03-04 17:44:18 DEBUG: pid 72000: key: log_standby_delay
2013-03-04 17:44:18 DEBUG: pid 72000: value: 'none' kind: 4
2013-03-04 17:44:18 DEBUG: pid 72000: key: syslog_facility
2013-03-04 17:44:18 DEBUG: pid 72000: value: 'LOCAL0' kind: 4
2013-03-04 17:44:18 DEBUG: pid 72000: key: syslog_ident
2013-03-04 17:44:18 DEBUG: pid 72000: value: 'pgpool' kind: 4
2013-03-04 17:44:18 DEBUG: pid 72000: key: debug_level
2013-03-04 17:44:18 DEBUG: pid 72000: value: 0 kind: 2
2013-03-04 17:44:18 ERROR: pid 72000: Could not read backend status file as /tmp/pgpool_status. reason: No such file or directory
2013-03-04 17:44:18 LOG: pid 72000: pgpool-II successfully started. version 3.2.3 (namameboshi)2013-03-04 17:44:18 DEBUG: pid 53642: 2013-03-04 17:44:18 DEBUG: pid
1028042013-03-04 17:44:18: 2013-03-04 17:44:18 DEBUG: pid 2013-03-04 17:44:182013-03-04 17:44:18 DEBUG: pid 69624 DEBUG: pid DEBUG: pid 76750: 13161679302: : : I am I am I am I am I am I am 13161679302102804767505364269624
2013-03-04 17:44:18 DEBUG: pid 118622: I am 118622
2013-03-04 17:44:18 DEBUG: pid 72000: find_primary_node: not in streaming replication mode
2013-03-04 17:44:18 DEBUG: pid 72000: starting health checking
2013-03-04 17:44:19 DEBUG: pid 72000: health_check: 0 th DB node status: 1
2013-03-04 17:44:19 DEBUG: pid 118622: pool_initialize_private_backend_status: initialize backend status
2013-03-04 17:44:192013-03-04 17:44:192013-03-04 17:44:192013-03-04 17:44:19 DEBUG: pid 2013-03-04 17:44:192013-03-04 17:44:19 DEBUG: pid DEBUG: pid DEBUG: pid 102804 DEBUG: pid DEBUG: pid 7675013161679302: 6962453642: : : pool_initialize_private_backend_status: initialize backend status: : pool_initialize_private_backend_status: initialize backend statuspool_initialize_private_backend_status: initialize backend statuspool_initialize_private_backend_status: initialize backend status
pool_initialize_private_backend_status: initialize backend statuspool_initialize_private_backend_status: initialize backend status
2013-03-04 17:44:19 DEBUG: pid 72000: pool_ssl: SSL requested but SSL support is not available
2013-03-04 17:44:19 DEBUG: pid 72000: s_do_auth: auth kind: 0
2013-03-04 17:44:19 DEBUG: pid 72000: s_do_auth: backend key data received
2013-03-04 17:44:19 DEBUG: pid 72000: s_do_auth: transaction state: I
2013-03-04 17:44:19 DEBUG: pid 72000: health_check: 1 th DB node status: 1
2013-03-04 17:44:19 DEBUG: pid 72000: pool_ssl: SSL requested but SSL support is not available
2013-03-04 17:44:19 ERROR: pid 72000: pool_flush_it: write failed to backend (0). reason: Broken pipe offset: 0 wlen: 41
2013-03-04 17:44:19 LOG: pid 72000: notice_backend_error: called from pgpool main. ignored.
2013-03-04 17:44:19 LOG: pid 72000: child_exit: called from pgpool main. ignored.
2013-03-04 17:44:19 ERROR: pid 72000: pool_read: read failed (Connection refused)
2013-03-04 17:44:19 LOG: pid 72000: notice_backend_error: called from pgpool main. ignored.
2013-03-04 17:44:19 LOG: pid 72000: child_exit: called from pgpool main. ignored.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pgpool_truss.out
Type: application/octet-stream
Size: 488 bytes
Desc: not available
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20130304/9fb758c2/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pgpool.conf
Type: application/octet-stream
Size: 25105 bytes
Desc: not available
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20130304/9fb758c2/attachment-0003.obj>
More information about the pgpool-general
mailing list