[pgpool-general: 2819] Re: bash script can't see env variables when run by pgpool

Mon May 12 20:14:11 JST 2014

Hi,

On Fri, 25 Apr 2014 18:21:56 +0100
Gintautas Sulskus <gingaz at gmail.com> wrote:

> Hi Yugo,
> 
> 
> > I can't confirm this. Where did you get the pgpoolAdmin, tar ball or rpm?
> > And could you please tell me the actual steps to reproduce this?
> 
> 
> I will try to reinstall it cleanly.
> On my machine I can reproduce this by simply modifying any value under
> pgpoolAdmin->"pgpool.conf Setting" and clicking "update" button.

Please confirm pgpoolAdmin Setting->"pgpool-II version".
This should be 3.3, otherwise wd_lifecheck_method is set empty since
there is no such parameter in 3.2 or earlier.

> 
>  "Disconnect" button executes pcp_detach_node command, and "return" button
> > executes pcp_attach_node command.
> 
> 
> Consider such scenario: I have 2 nodes with synchronous replication setting
> = enabled.
> I can then disconnect one node from the pgpool cluster and write data to
> the remaining node via pgpool.
> Data inconsistency occurs.
> After that I can reconnect the disconnected node and the cluster behaves as
> if nothing happened even though there is data mismatch.
> 
> Is there any protection planned to be implemented from such cases?

In this case, you should use "Recover" button instead of "Return".
This executes pcp_recovery_node and disconnected node is recovered
by basebackup from master DB.

> 
> Did you add pgpool.pg_ctl parameter into postgresql.conf?
> > http://www.pgpool.net/docs/latest/pgpool-en.html#install_functions
> > In addition, backends must have a superuser whose name is the same
> > as pgpoolAdmin's login user.
> 
> 
> I have added it, but it didn't help. Buttons are disabled and status shows
> "superuser: unknown".
> The connection string is formed "psql -p X -U X -password X -database
> template1". For some reason my 9.3 psql can't connect to the pgpool unless
> I explicitly specify "-h localhost". However, after correcting this issue
> buttons still remain disabled.
> 
> I removed button attribute "disabled" manually and tried to use the
> buttons. They work, postgres DB can be manipulated.
> For some reason button conditional $reason is_superuser='no', even though i
> set "isSuperUser()" to return 'yes'.

There is a recent fix of pgpoolAdmin, which might have resolved this problem.
http://git.postgresql.org/gitweb/?p=pgpooladmin.git;a=commitdiff;h=5ec3c83cdcb909ac0dc71ca93ae7a8e8e4217858

> 
> ------------
> Additional problems that I am still experiencing:
> 
> 1. If I execute the pcp_recovery_node and interrupt it (ctrl+C), all pcp_*
> operations become unresponsive. Connections to pgpool:9999 still work
> though. The only way to make pcp_* work again is to reboot pgpool.

While recovery, pcp_* command is unresponsive and would response after 
the recovery on the backend side is finished.

> 
> 2. pgpoolAdmin freezes while pgpool is executing pcp_* operation (e.g. very
> long pcp_recovery)

We recognize this as a problem to be resolve in future.

> 3. minor: sr_check_password, health_check_password are not starred.

You metion stars [*] at pgpool.conf Setting view that means a 
parameter requires restart? As you say, I confirm that there are 
some wrong notations. We'll review and revise this.

> 
> Cheers,
> Gintas
> 
> 
> On Mon, Apr 21, 2014 at 6:39 AM, Yugo Nagata <nagata at sraoss.co.jp> wrote:
> 
> > Hi,
> >
> > On Fri, 11 Apr 2014 11:40:52 +0100
> > Gintautas Sulskus <gingaz at gmail.com> wrote:
> >
> > > Hello Yugo,
> > >
> > > no worries, your replies are much appreciated :)
> > >
> > > The problem was in the recovery process on a remotely failed node. If I
> > can
> > > remember clearly, scp operation asked for
> > > host authorisation ("yes/no") and hung the script. A more verbose message
> > > providing clues to pinpoint the problem quickly
> > > would be really great.
> > >
> > > I updated to pgpoolAdmin 3.3.1 and pgpool 3.3.3. and have noticed an
> > issue:
> > > 1. If I update pgpool.conf via pgpoolAdmin, it erases
> > wd_lifecheck_method,
> > > wd_interval, wd_heartbeat_port, wd_heartbeat_keepalive entries.
> > > 2014-04-11 10:38:59 ERROR: pid 32410: pool_config: wd_lifecheck_method
> > must
> > > be either "heartbeat" or "query"
> > > 2014-04-11 10:38:59 ERROR: pid 32410: Unable to get configuration.
> > > Exiting...
> > > I have to go to edit them in file. Does it occur to anyone else?
> >
> > I can't confirm this. Where did you get the pgpoolAdmin, tar ball or rpm?
> > And could you please tell me the accual steps to reproduce this?
> >
> > > pgpoolAdmin questions:
> > > 1. Could you please explain me, what disconnect/return button should do?
> > I
> > > assumed, that it would invoke failover/failback functionality.
> > > E.g. I could "disconnect" server and then on "return" it would
> > > resynchronise with the cluster automatically (cia pcp_recovery_node).
> >
> > "Disconnect" button executes pcp_detach_node command, and "return" button
> > executes pcp_attach_node command.
> >
> > >
> > > If my assumptions are wrong, is it possible to call pcp_recovery_node via
> > > interface?
> >
> > Yes. "Recovery" button which appears when the backend goes down executes
> > pcp_recovery_node command.
> >
> > >
> > > 2. I could not find any documentation how to enable stop/start/restart
> > > buttons in pgpoolAdmin. Currently they are disabled.
> > > I am not certain what settings should enable this.
> >
> > Did you add pgpool.pg_ctl parameter into postgresql.conf?
> > http://www.pgpool.net/docs/latest/pgpool-en.html#install_functions
> > In addition, backends must have a superuser whose name is the same
> > as pgpoolAdmin's login user.
> >
> > >
> > > Thanks for your patience! :)
> > >
> > > Cheers,
> > > Gintautas
> > >
> > >
> > >
> > > On Thu, Apr 10, 2014 at 8:38 AM, Yugo Nagata <nagata at sraoss.co.jp>
> > wrote:
> > >
> > > > Hi Gintautas,
> > > >
> > > > I'm sorry for replying late.
> > > >
> > > > Recovery scripts are executed by PostgreSQL, so clues would be in log
> > > > outputs
> > > > of backend server. Does the scripts have the permission to be executed
> > by
> > > > postgres user?
> > > >
> > > > On Sat, 29 Mar 2014 19:35:32 +0000
> > > > Gintautas Sulskus <gingaz at gmail.com> wrote:
> > > >
> > > > > Hi Yugo,
> > > > >
> > > > > thanks for clarifying! Set up is working.
> > > > >
> > > > > I guess that a environment variable defiend in /etc/enviromment is
> > > > reffered
> > > > > > in pg_ni_up.sh but this doesn't work well, right?
> > > > >
> > > > >
> > > > > > How about to try 'echo $PATH > /tmp/test ' in pg_ni_up.sh?
> > > > > > Is the $PATH (or other value) defined in /etc/environment output to
> > > > > > /tmp/test or not?
> > > > >
> > > > >
> > > > >  Even $PATH was not displayed correctly. I presume it was issue with
> > > > > permissions, although it is not entirely clear to me what has caused
> > this
> > > > > issue.
> > > > >
> > > > >
> > > > > I am still struggling with pgpool configuration though. Hopefully it
> > is
> > > > the
> > > > > last question. Could please anyone give any clues on this problem:
> > > > >
> > > > > I have tested my online recovery steps (1st and 2nd recovery) by
> > manually
> > > > > running scripts. They work just fine. Everything gets logged
> > properly.
> > > > >
> > > > > However, when I try to run pcp_recovery_node -d I get:
> > > > > DEBUG: send: tos="R", len=46
> > > > > DEBUG: recv: tos="r", len=21, data=AuthenticationOK
> > > > > DEBUG: send: tos="D", len=6
> > > > > DEBUG: recv: tos="e", len=20, data=recovery failed
> > > > > DEBUG: command failed. reason=recovery failed
> > > > > BackendError
> > > > > DEBUG: send: tos="X", len=4
> > > > >
> > > > > *pgpool log output:*
> > > > > (pgpool started in debug mode with debug_level=10)
> > > > > CHECKPOINT in the 1st stage done
> > > > > starting recovery command: "SELECT pgpool_recovery('pg_1st_recovery',
> > > > > 'failed_node_ip_address', '/data/postgres/main/')"
> > > > > exec_recovery: pg_1st_recovery command failed at 1st stage
> > > > >
> > > > > *pg_1st_recovery logs:*
> > > > > none
> > > > >
> > > > > *my pgpool configuration:*
> > > > > recovery_1st_stage_command=pg_1st_recovery
> > > > > (I expect $1 $2 $3 parameters from pgpool as in examples)
> > > > >
> > > > > Could you please give me any hints what could be wrong?
> > > > > Even a rough direction instead of "BackendError" would be extremely
> > > > > valuable.
> > > > >
> > > > > Thanks,
> > > > > Gintas
> > > > >
> > > > >
> > > > > On Wed, Mar 26, 2014 at 7:02 AM, Yugo Nagata <nagata at sraoss.co.jp>
> > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > On Fri, 21 Mar 2014 00:55:51 +0000
> > > > > > Gintautas Sulskus <gingaz at gmail.com> wrote:
> > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > more problems regarding watchdog:
> > > > > > > On one of the servers I see a log entry:
> > "wd_create_hb_send_socket:
> > > > > > > setsockopt(SO_BINDTODEVICE) requies root privilege".
> > > > > > > Any clues what this may be related to? I assume it's permission
> > > > problem.
> > > > > >
> > > > > > You can ignore this message because SO_BINDTODEVICE is not
> > necessary.
> > > > > >
> > > > > > >
> > > > > > > Much appreciated!
> > > > > > >
> > > > > > > Gintautas
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Mar 21, 2014 at 12:48 AM, Gintautas Sulskus <
> > > > gingaz at gmail.com
> > > > > > >wrote:
> > > > > > >
> > > > > > > > Hello,
> > > > > > > >
> > > > > > > > ifconfig_path = '/home/ubuntu/apps/scripts'
> > > > > > > > PgpoolAdmin description of ifconfig_path is: The path of a
> > command
> > > > to
> > > > > > > > switch the IP address. I understand it as the path for
> > if_up_cmd
> > > > > > > > and if_down_cmd commands.
> > > > > > > >
> > > > > > > > if_up_cmd = 'pg_ni_up.sh up eth0:1 10.0.1.244 255.255.255.0'
> > > > > > > > if_down_cmd = 'pg_ni_up.sh down eth0:1'
> > > > > >
> > > > > > I guess that a environment variable defiend in /etc/enviromment is
> > > > reffered
> > > > > > in pg_ni_up.sh but this doesn't work well, right?
> > > > > >
> > > > > > How about to try 'echo $PATH > /tmp/test ' in pg_ni_up.sh?
> > > > > > Is the $PATH (or other value) defined in /etc/enviroment output to
> > > > > > /tmp/test or not?
> > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > PS. Is this mailing list the right place to discuss about
> > > > PgpoolAdmin?
> > > > > > > > In the latest PgpoolAdmin version "if_*up*_cmd " is described
> > as
> > > > "The
> > > > > > > > command to bring *down* the virtual IP" and "if_*down*_cmd" as
> > "The
> > > > > > > > command to bring *up* the virtual IP". Clearly descriptions are
> > > > mixed
> > > > > > up.
> > > > > > > >
> > > > > > > > Gintautas
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Mar 10, 2014 at 2:46 AM, Yugo Nagata <
> > nagata at sraoss.co.jp>
> > > > > > wrote:
> > > > > > > >
> > > > > > > >> Hi,
> > > > > > > >>
> > > > > > > >> On Sun, 9 Mar 2014 03:00:52 +0000
> > > > > > > >> Gintautas Sulskus <gingaz at gmail.com> wrote:
> > > > > > > >>
> > > > > > > >> > Hello,
> > > > > > > >> >
> > > > > > > >> > I am trying to set up pgpool watchdog. For virtual IP
> > control my
> > > > > > plan
> > > > > > > >> is to
> > > > > > > >> > use bash scripts (if_up_cmd/if_down_cmd). In my script I use
> > > > some
> > > > > > > >> > environment variables.
> > > > > > > >> >
> > > > > > > >> > A strange thing occurs here. No matter under what user I run
> > > > pgpool,
> > > > > > > >> script
> > > > > > > >> > can't pick up my custom environment variables from
> > > > /etc/environment
> > > > > > > >> > (including customised PATH). It still sees standard binaries
> > > > like
> > > > > > > >> ifconfig
> > > > > > > >> > though.
> > > > > > > >>
> > > > > > > >> How do you configure pgpool.conf about if_up_cmd, if_down_cmd,
> > > > > > > >> ifconfig_path?
> > > > > > > >> pgpool see ifconfig commands on the path specified by these
> > > > options.
> > > > > > > >>
> > > > > > > >> >
> > > > > > > >> > Same script, when run manually by me, works under all
> > users. Any
> > > > > > ideas
> > > > > > > >> what
> > > > > > > >> > can be wrong?
> > > > > > > >> >
> > > > > > > >> > The only solution I have come up is to redefine env
> > variables
> > > > in the
> > > > > > > >> script.
> > > > > > > >> >
> > > > > > > >> > Thanks.
> > > > > > > >> >
> > > > > > > >> > Best Regards,
> > > > > > > >> > Gintas
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> --
> > > > > > > >> Yugo Nagata <nagata at sraoss.co.jp>
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best Regards,
> > > > > > > > Gintautas Sulskus
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best Regards,
> > > > > > > Gintautas Sulskus
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Yugo Nagata <nagata at sraoss.co.jp>
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best Regards,
> > > > > Gintautas Sulskus
> > > >
> > > >
> > > > --
> > > > Yugo Nagata <nagata at sraoss.co.jp>
> > > >
> > >
> > >
> > >
> > > --
> > > Best Regards,
> > > Gintautas Sulskus
> >
> >
> > --
> > Yugo Nagata <nagata at sraoss.co.jp>
> >
> 
> 
> 
> -- 
> Best Regards,
> Gintautas Sulskus

-- 
Yugo Nagata <nagata at sraoss.co.jp>