[pgpool-committers: 2706] Re: pgpool: Mega patch to enhance performance in extended protocol mode.
nagata at sraoss.co.jp
Tue Sep 29 11:20:23 JST 2015
Thank you. I can build this now.
On Tue, 29 Sep 2015 10:57:21 +0900 (JST)
Tatsuo Ishii <ishii at postgresql.org> wrote:
> Sorry my bad. I have committed CommandComplete.c.
> Best regards,
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> English: http://www.sraoss.co.jp/index_en.php
> > Ishii-san,
> > Is this commit missing CommandComplete.c?
> > On Thu, 24 Sep 2015 02:13:16 +0000
> > Tatsuo Ishii <ishii at postgresql.org> wrote:
> >> Mega patch to enhance performance in extended protocol mode.
> >> For now it only affects to streaming replication mode, that is, the
> >> performance of other modes remains same. In the future we may be able
> >> to enhance this.
> >> Anyway, basic idea here is, removing "flush" message which used to be
> >> sent in each step of extended protocol messages (parse, bind, describe
> >> and execute). This brought significant communication overhead. To
> >> achieve the goal following modifications have been made.
> >> - New data structure called "pending message queue" is created to
> >> manage which response of particular message is pending. When
> >> Parse/Bind/Close message are received, each message is en-queued.
> >> The information is used to process those response messages, when
> >> Parse complete/Bind completes and Close compete message are received
> >> because they don't have any information regarding statement/portal.
> >> - New state variable to represent that there's any "pending response"
> >> exists. If an extended protocol message, for example, parse is sent
> >> and still the response to parse (i.e. parse complete) is not yet
> >> received, the variable is set to true.
> >> - If there's any pending response, do_query() sends flush message to
> >> retrieve any pending data from primary backend and save it. After
> >> finishing the job, the saved data is restored. This will keep the
> >> original response message sequence.
> >> - Deal with a special case in read_kind_from_backend(). Certain client
> >> (at least JDBC driver has this habit) does not send a sync message
> >> after an execute message. See the comment in
> >> read_kind_from_backend() for more details.
> >> - New data structure called "sync map" to manage sync message response
> >> is added. Used for managing sync message response in streaming
> >> replication mode. In streaming replication mode, there are at most
> >> two nodes involved. One is always primary, and the other (if any)
> >> could be standby. If the load balancing manager chooses the primary
> >> as the load balancing node, only the primary node is involved. Every
> >> time extended protocol is sent to backend, we set true on the member
> >> of array below. It is cleared after sync message is processed and
> >> command complete or error response message is processed.
> >> - New parameter "bool nowait" is added to
> >> pool_extended_send_and_wait() to control whether flush message is
> >> sent or not after sending extended protocol command.
> >> - Move CommandComplete() to new file src/protocol/CommandComplete.c to
> >> make maintainer's life easier.
> >> Initial performance evaluation:
> >> - Hard ware: Let's note CF-SX3, mem 16GB, SSD 512GB, Core i7-4600U (2 cores)
> >> - OS: Ubuntu 14.04 LTS
> >> - 2 PostgreSQL streaming replication setup using pgpool_setup
> >> - pgbench options: pgbench -M extended -c 16 -j 8 -S -T 30 test (scale=1)
> >> - Results (excluding connections establishing):
> >> pgpool-II 3.4.4: 7274.926104 TPS, this commit: 11290.026468 TPS
> >> So, this committed version is 55.2% faster than 3.4.3.
> >> TODOs:
> >> - parse_before_bind() in pool_proto_modules is ifdef'ed out. Even
> >> without this, all regression tests are passed. But I am not sure if
> >> it's really harmless.
> >> - When parse message is sent, it is possible that deadlock occurs
> >> according to the comment. I'm not sure if it's still true or not. If
> >> true, we need to fix it.
> >> Branch
> >> ------
> >> master
> >> Details
> >> -------
> >> http://git.postgresql.org/gitweb?p=pgpool2.git;a=commitdiff;h=e7b4de8d457e223234975b4ceaf8c345acdc2492
> >> Modified Files
> >> --------------
> >> src/Makefile.am | 3 +-
> >> src/Makefile.in | 7 +-
> >> src/context/pool_query_context.c | 82 +++--
> >> src/context/pool_session_context.c | 295 +++++++++++++++-
> >> src/include/context/pool_query_context.h | 2 +-
> >> src/include/context/pool_session_context.h | 63 ++++
> >> src/include/pool.h | 1 +
> >> src/include/protocol/pool_proto_modules.h | 5 +
> >> src/protocol/pool_process_query.c | 282 +++++++++++++--
> >> src/protocol/pool_proto_modules.c | 372 +++++++-------------
> >> src/query_cache/pool_memqcache.c | 192 +++++++---
> >> .../regression/tests/006.memqcache/jdbctest.java | 2 +-
> >> src/test/regression/tests/051.bug60/test.sh | 6 +-
> >> src/utils/pool_stream.c | 54 ++-
> >> 14 files changed, 983 insertions(+), 383 deletions(-)
> >> _______________________________________________
> >> pgpool-committers mailing list
> >> pgpool-committers at pgpool.net
> >> http://www.pgpool.net/mailman/listinfo/pgpool-committers
> > --
> > Yugo Nagata <nagata at sraoss.co.jp>
Yugo Nagata <nagata at sraoss.co.jp>
More information about the pgpool-committers