[Pgpool-general] Question about client connections
Tatsuo Ishii
ishii at sraoss.co.jp
Fri Dec 21 10:31:12 UTC 2007
Sorry for delay.
The behavior you had is an expected one.
(Maybe I should write this somewhere in the pgpool docs).
pgbench establishes connections to PostgreSQL as specified by -c
option in the first place. If the number specified by -c exceeds
num_init_children, pgbench stacks because there's no listening process
in pgpool children any more. For pgpool to accept next connection from
clients, one of already connected clients sents transactions then
disconects from pgpool. Unfortunately pgpool will not send any
transaction until all connections required by -c, this is a kind dead
lock situation.
If you want to test more real world's behavior of pgpool, I recommend
to use ab coming with Apache.
Here is a sample script written in PHP(you could write simalar nasty
program by using Java, perl, ruby or whatever).
<?php
ini_set("track_errors", "1");
define_syslog_variables();
$con = pg_connect("dbname=test user=postgres port=9999");
if ($con == FALSE) {
syslog(LOG_ERR, "could not connect $php_errormsg");
trigger_error("Could not connect to DB", E_USER_ERROR);
exit(1);
}
$aid = rand(1,10000);
pg_query($con, "SELECT * FROM accounts WHERE aid = $aid");
pg_close($con);
?>
And ab run:
$ /usr/local/apache/bin/ab -c 100 -n 1000 "http://localhost/bench.php"
where -c is similar to pgbench's -c. You could specify -c 100 even
num_init_children is 32.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
> Hello,
>
> What is the behavior of pgpool when all of its available connections are
> used? Today I walked through the replication tutorial and was able to
> successfully replicate the pgbench data to all of the backends. Then I
> tried running the pgbench benchmark through pgpool and ran into a
> problem. I ran the test several times, each time increasing the number
> of clients. This was working without any problems until the number of
> clients exceeded the num_init_children value in the pgpool.conf file.
> At this point pgbench seems to hang, and a listing of the pgpool and
> postgres processes show them all as IDLE. The behvior is the same even
> if I run pgbench in select-only mode. I read in another post to the
> list that postgres should be configured so that the number of
> connections it accepts is larger than (num_init_children * max_pool).
> However, even after making this change to each of the backends and
> restarting them (as well as pgpool) the pgbench test still hangs.
>
> Is this a bug in some portion of my setup (either pgpool or pgbench), or
> is this the expected behavior? I am new to pgpool so it's quite
> possible that something is configured incorrectly.
>
>
> The pgbench command I am running is:
>
> pgbench -c 33 -p 9797 bench_replication
>
>
> Here is the pgpool.conf file I am using:
>
> #
> # pgpool-II configuration file sample
> # $Header: /cvsroot/pgpool/pgpool-II/pgpool.conf.sample,v 1.15
> 2007/10/31 10:45:52 y-asaba Exp $
>
> # Host name or IP address to listen on: '*' for all, '' for no TCP/IP
> # connections
> listen_addresses = 'localhost'
>
> # Port number for pgpool
> port = 9797
>
> # Port number for pgpool communication manager
> pcp_port = 9898
>
> # Unix domain socket path. (The Debian package defaults to
> # /var/run/postgresql.)
> socket_dir = '/tmp'
>
> # Unix domain socket path for pgpool communication manager.
> # (Debian package defaults to /var/run/postgresql)
> pcp_socket_dir = '/tmp'
>
> # Unix domain socket path for the backend. Debian package defaults to
> /var/run/postgresql!
> backend_socket_dir = '/tmp'
>
> # pgpool communication manager timeout. 0 means no timeout, but strongly
> not rec
> ommended!
> pcp_timeout = 10
>
> # number of pre-forked child process
> num_init_children = 32
>
> # Number of connection pools allowed for a child process
> max_pool = 4
>
> # If idle for this many seconds, child exits. 0 means no timeout.
> child_life_time = 300
>
> # If idle for this many seconds, connection to PostgreSQL closes.
> # 0 means no timeout.
> connection_life_time = 0
>
> # If child_max_connections connections were received, child exits.
> # 0 means no exit.
> child_max_connections = 0
>
> # If client_idle_limit is n (n > 0), the client is forced to be
> # disconnected whenever after n seconds idle (even inside an explicit
> # transactions!)
> # 0 means no disconnect.
> client_idle_limit = 0
>
> # Maximum time in seconds to complete client authentication.
> # 0 means no timeout.
> authentication_timeout = 60
>
> # Logging directory
> logdir = '/tmp'
>
> # Replication mode
> replication_mode = true
>
> # Set this to nonzero (in milliseconds) to detect this situation and
> # resolve the deadlock by aborting current session.
> replication_timeout = 5000
>
> # Load balancing mode, i.e., all SELECTs except in a transaction block
> # are load balanced. This is ignored if replication_mode is false.
> load_balance_mode = true
>
> # if there's a data mismatch between master and secondary
> # start degeneration to stop replication mode
> replication_stop_on_mismatch = false
>
> # If true, replicate SELECT statement when load balancing is disabled.
> # If false, it is only sent to the master node.
> replicate_select = false
>
> # Semicolon separated list of queries to be issued at the end of a
> session
> reset_query_list = 'ABORT; RESET ALL; SET SESSION AUTHORIZATION DEFAULT'
>
> # If true print timestamp on each log line.
> print_timestamp = true
>
> # If true, operate in master/slave mode.
> master_slave_mode = false
>
> # If true, cache connection pool.
> connection_cache = true
>
> # Health check timeout. 0 means no timeout.
> health_check_timeout = 20
>
> # Health check period. 0 means no health check.
> health_check_period = 60
>
> # Health check user
> health_check_user = 'postgres'
>
> # Execute command by failover.
> # special values: %d = node id
> # %h = host name
> # %p = port number
> # %D = database cluster path
> # %% = '%' character
> #
> failover_command = ''
>
> # Execute command by failback.
> # special values: %d = node id
> # %h = host name
> # %p = port number
> # %D = database cluster path
> # %% = '%' character
> #
> failback_command = ''
>
> # If true, automatically lock table with INSERT statements to keep
> SERIAL
> # data consistency. An /*INSERT LOCK*/ comment has the same effect. A
> # /NO INSERT LOCK*/ comment disables the effect.
> insert_lock = false
>
> # If true, ignore leading white spaces of each query while pgpool judges
> # whether the query is a SELECT so that it can be load balanced. This
> # is useful for certain APIs such as DBI/DBD which is known to adding an
> # extra leading white space.
> ignore_leading_white_space = true
>
> # If true, print all statements to the log. Like the log_statement
> option
> # to PostgreSQL, this allows for observing queries without engaging in
> full
> # debugging.
> log_statement = false
>
> # If true, incoming connections will be printed to the log.
> log_connections = false
>
> # If true, hostname will be shown in ps status. Also shown in
> # connection log if log_connections = true.
> # Be warned that this feature will add overhead to look up hostname.
> log_hostname = false
>
> # if non 0, run in parallel query mode
> parallel_mode = false
>
> # if non 0, use query cache
> enable_query_cache = false
>
> #set pgpool2 hostname
> pgpool2_hostname = ''
>
> # system DB info
> system_db_hostname = 'localhost'
> system_db_port = 5432
> system_db_dbname = 'pgpool'
> system_db_schema = 'pgpool_catalog'
> system_db_user = 'pgpool'
> system_db_password = ''
>
> # backend_hostname, backend_port, backend_weight
> # here are examples
> #backend_hostname0 = 'host1'
> #backend_port0 = 5432
> #backend_weight0 = 1
> #backend_data_directory0 = '/data'
> #backend_hostname1 = 'host2'
> #backend_port1 = 5433
> #backend_weight1 = 1
> #backend_data_directory1 = '/data1'
>
> backend_hostname0=localhost
> backend_port0=5433
> backend_weight0=1
> backend_data_directory0='/var/lib/pgsql/testdb1'
>
> backend_hostname1=localhost
> backend_port1=5434
> backend_weight1=1
> backend_data_directory1='/var/lib/pgsql/testdb2'
>
> backend_hostname2=localhost
> backend_port2=5435
> backend_weight2=1
> backend_data_directory2='/var/lib/pgsql/testdb3'
>
> backend_hostname3=localhost
> backend_port3=5436
> backend_weight3=1
> backend_data_directory3='/var/lib/pgsql/testdb4'
>
>
> # - HBA -
>
> # If true, use pool_hba.conf for client authentication. In pgpool-II
> # 1.1, the default value is false. The default value will be true in
> # 1.2.
> enable_pool_hba = false
>
> # - online recovery -
> # online recovery user
> recovery_user = 'nobody'
>
> # online recovery password
> recovery_password = ''
>
> # execute a command in first stage.
> recovery_1st_stage_command = ''
>
> # execute a command in second stage.
> recovery_2nd_stage_command = ''
>
>
>
> Thanks!
> _______________________________________________
> Pgpool-general mailing list
> Pgpool-general at pgfoundry.org
> http://pgfoundry.org/mailman/listinfo/pgpool-general
More information about the Pgpool-general
mailing list