[pgpool-general: 4428] [Problem] pgpool-II 3.5.0 rhel5 pg94

Stefano C. djcenox at gmail.com
Wed Feb 10 06:19:23 JST 2016


Hello Everyone,
due to problem with database master overload, i've decided to move one and
use pgpool especially for load balacing.
We already bumped our postgres to 9.4.1 version last year and performances
got a litte better.

First problem was that we still have rhel 5 in prod, so i've downloaded
sources compiled over a rh5 and pached to rpm using .spec from sources.

Our configuration has 2 nodes in streaming replica all-in-one that means
webapp+db for each node for HA purpose. At the moment we are using just 1
pgpool for testing, then we will use WD with query health check to sync
another pgpool instance on second server(we know is deprecated, but we just
need to keep connection if server 1 die).

Anyway we were good until at application start we had this and the
application crash:


2016-02-09 18:05:53: pid 7898: LOG:  Backend status file /tmp/pgpool_status
does not exist
2016-02-09 18:05:53: pid 7898: LOG:  Setting up socket for 0.0.0.0:9999
2016-02-09 18:05:53: pid 7898: LOG:  pgpool-II successfully started.
version 3.5.0 (ekieboshi)
2016-02-09 18:05:53: pid 7898: LOG:  find_primary_node: checking backend no
0
2016-02-09 18:05:53: pid 7898: LOG:  find_primary_node: checking backend no
1
2016-02-09 18:05:53: pid 7921: FATAL:  Backend throw an error message
2016-02-09 18:05:53: pid 7921: DETAIL:  Exiting current session because of
an error from backend
2016-02-09 18:05:53: pid 7921: HINT:  BACKEND Error: "recovery is in
progress"
2016-02-09 18:05:53: pid 7921: CONTEXT:  while checking replication time lag
2016-02-09 18:05:53: pid 7898: LOG:  find_primary_node: primary node id is 1
2016-02-09 18:05:53: pid 7898: LOG:  worker child process with pid: 7921
exits with status 256
2016-02-09 18:05:53: pid 7898: LOG:  fork a new worker child process with
pid: 7924
2016-02-09 18:06:41: pid 7917: WARNING:  packet kind of backend 1 ['Z']
does not match with master/majority nodes packet kind ['1']
2016-02-09 18:06:41: pid 7917: FATAL:  failed to read kind from backend
2016-02-09 18:06:41: pid 7917: DETAIL:  kind mismatch among backends.
Possible last query was: "COMMIT" kind details are: 0[1] 1[Z]
2016-02-09 18:06:41: pid 7917: HINT:  check data consistency among db nodes
2016-02-09 18:06:41: pid 7898: LOG:  child process with pid: 7917 exits
with status 512
2016-02-09 18:06:41: pid 7898: LOG:  fork a new child process with pid: 8103
2016-02-09 18:07:10: pid 7916: WARNING:  packet kind of backend 1 ['Z']
does not match with master/majority nodes packet kind ['1']
2016-02-09 18:07:10: pid 7916: FATAL:  failed to read kind from backend
2016-02-09 18:07:10: pid 7916: DETAIL:  kind mismatch among backends.
Possible last query was: "COMMIT" kind details are: 0[1] 1[Z]
2016-02-09 18:07:10: pid 7916: HINT:  check data consistency among db nodes
2016-02-09 18:07:10: pid 7898: LOG:  child process with pid: 7916 exits
with status 512
2016-02-09 18:07:10: pid 7898: LOG:  fork a new child process with pid: 8213
2016-02-09 18:07:11: pid 7913: WARNING:  packet kind of backend 1 ['Z']
does not match with master/majority nodes packet kind ['1']
2016-02-09 18:07:11: pid 7913: FATAL:  failed to read kind from backend
2016-02-09 18:07:11: pid 7913: DETAIL:  kind mismatch among backends.
Possible last query was: "COMMIT" kind details are: 0[1] 1[Z]
2016-02-09 18:07:11: pid 7913: HINT:  check data consistency among db nodes

We forced the replicafrom async to sync thought that maybe was the problem,
but nothing changed.


This is our postgresql.conf:

listen_addresses = '*'
max_connections = 100
shared_buffers = 32MB
log_destination = 'stderr'
logging_collector = on
log_directory = 'pg_log'
log_filename = 'postgresql-%a.log'
log_truncate_on_rotation = on
log_rotation_age = 1d
log_rotation_size = 0
log_line_prefix = '%m                           '
datestyle = 'iso, mdy'
lc_messages = 'en_US.UTF-8'
lc_monetary = 'en_US.UTF-8'
lc_numeric = 'en_US.UTF-8'
lc_time = 'en_US.UTF-8'
default_text_search_config = 'pg_catalog.english'
ssl = on
ssl_renegotiation_limit = 0
wal_level = hot_standby
archive_mode = off
max_wal_senders = 1
wal_keep_segments = 100
hot_standby = on
work_mem = 100MB

#Sync edit edit
synchronous_standby_names = 'walreceiver'
synchronous_commit = on


the pgpool.conf is just the sample-streaming example conf:
 we added sr_check_user properly and two postgres.

Anyway if the stanby db is stopped everything works ok.

Thanks for your patience,
Stefano.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20160209/f614602f/attachment.html>


More information about the pgpool-general mailing list