From avi.raboah at gmail.com Wed Aug 3 14:48:25 2022 From: avi.raboah at gmail.com (Avi Raboah) Date: Wed, 3 Aug 2022 08:48:25 +0300 Subject: [pgpool-general: 8349] Jdbc driver harm cache hit ratio Message-ID: Hi, I ran query load with two different drivers via pgpool with cache enabled. 1. With psychopg2 which generates simple query and I got 91% cache hit ratio. 2. I ran the same load with jdbc which generates extended query and I got 0.03% cache hit ratio. Any thoughts? Am I missing some configuration maybe? Thanks, Avi. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ishii at sraoss.co.jp Wed Aug 3 17:27:08 2022 From: ishii at sraoss.co.jp (Tatsuo Ishii) Date: Wed, 03 Aug 2022 17:27:08 +0900 (JST) Subject: [pgpool-general: 8350] Re: Jdbc driver harm cache hit ratio In-Reply-To: References: Message-ID: <20220803.172708.1362353369418538287.t-ishii@sranhm.sra.co.jp> > Hi, > > I ran query load with two different drivers via pgpool with cache enabled. > > 1. With psychopg2 which generates simple query and I got 91% cache hit > ratio. > 2. I ran the same load with jdbc which generates extended query and I got > 0.03% cache hit ratio. > > Any thoughts? Am I missing some configuration maybe? Please a section in the manual: https://www.pgpool.net/docs/latest/en/html/runtime-in-memory-query-cache.html As explained in the doc, JDBC by default operates in extended query mode. In the mode the chance to create query cache tends to be smaller than simple query because query cache will not be created until sync message is sent from frontend. Also you might want to inspect pgpool log with log_client_messages and log_statement enabled to find the timing of sync message being sent from frontend. Best reagards, -- Tatsuo Ishii SRA OSS LLC English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp From ishii at sraoss.co.jp Wed Aug 3 21:50:27 2022 From: ishii at sraoss.co.jp (Tatsuo Ishii) Date: Wed, 03 Aug 2022 21:50:27 +0900 (JST) Subject: [pgpool-general: 8351] Re: Jdbc driver harm cache hit ratio In-Reply-To: References: <20220803.172708.1362353369418538287.t-ishii@sranhm.sra.co.jp> Message-ID: <20220803.215027.1423806576388096545.t-ishii@sranhm.sra.co.jp> > Thanks for your email. > > So in case I am using extended query the query will be cached only when the > sync message received? Yes. Note that in extended query message, sending sync message at some point is required. See the PostgreSQL manual. Anyway sending sync message should be automatically done in the driver level and you don't need to worry about it. > If this is the case why I see so much difference in cache hit ratio between > the drivers? I think because psychopg2 uses simple query protocol as you said. > If it?s not the case, do you know what can be done in order to decrease the > gap? No. I am not an expert of JDBC. However if you provide pgpool log with log_client_messages and log_statement enabled while using JDBC, maybe I can find some hints to fill the gap. > Thanks, > > Avi > > On Wed, 3 Aug 2022 at 11:27 Tatsuo Ishii wrote: > >> > Hi, >> > >> > I ran query load with two different drivers via pgpool with cache >> enabled. >> > >> > 1. With psychopg2 which generates simple query and I got 91% cache hit >> > ratio. >> > 2. I ran the same load with jdbc which generates extended query and I got >> > 0.03% cache hit ratio. >> > >> > Any thoughts? Am I missing some configuration maybe? >> >> Please a section in the manual: >> >> >> https://www.pgpool.net/docs/latest/en/html/runtime-in-memory-query-cache.html >> >> As explained in the doc, JDBC by default operates in extended query >> mode. In the mode the chance to create query cache tends to be smaller >> than simple query because query cache will not be created until sync >> message is sent from frontend. >> >> Also you might want to inspect pgpool log with log_client_messages and >> log_statement enabled to find the timing of sync message being sent >> from frontend. >> >> Best reagards, >> -- >> Tatsuo Ishii >> SRA OSS LLC >> English: http://www.sraoss.co.jp/index_en.php >> Japanese:http://www.sraoss.co.jp >> From avi.raboah at gmail.com Mon Aug 15 18:25:51 2022 From: avi.raboah at gmail.com (Avi Raboah) Date: Mon, 15 Aug 2022 12:25:51 +0300 Subject: [pgpool-general: 8352] Cache entries Message-ID: Hi, There is a way to see what entries are stored in shared memory cache? Thanks, Avi -------------- next part -------------- An HTML attachment was scrubbed... URL: From ishii at sraoss.co.jp Tue Aug 16 10:27:36 2022 From: ishii at sraoss.co.jp (Tatsuo Ishii) Date: Tue, 16 Aug 2022 10:27:36 +0900 (JST) Subject: [pgpool-general: 8353] Re: Cache entries In-Reply-To: References: Message-ID: <20220816.102736.1187728706556188733.t-ishii@sranhm.sraoss.co.jp> > Hi, > > There is a way to see what entries are stored in shared memory cache? No. I have to admit that such a feature could be useful, though. Best reagards, -- Tatsuo Ishii SRA OSS LLC English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp From avi.raboah at gmail.com Tue Aug 16 17:49:50 2022 From: avi.raboah at gmail.com (Avi Raboah) Date: Tue, 16 Aug 2022 11:49:50 +0300 Subject: [pgpool-general: 8354] Re: Cache entries In-Reply-To: <20220816.102736.1187728706556188733.t-ishii@sranhm.sraoss.co.jp> References: <20220816.102736.1187728706556188733.t-ishii@sranhm.sraoss.co.jp> Message-ID: Ok thanks. Moreover I found something very weird. In case you add something to unsafe table in order not to cache queries from this table. Pgpool still increment num selects with those queries and this increase harm the cache hit ratio. I am missing something? Unsafe = city Select from city ? let?s say 10 times Show pool_cache will show Num selects = 10 Thanks, Avi On Tue, 16 Aug 2022 at 4:27 Tatsuo Ishii wrote: > > Hi, > > > > There is a way to see what entries are stored in shared memory cache? > > No. I have to admit that such a feature could be useful, though. > > Best reagards, > -- > Tatsuo Ishii > SRA OSS LLC > English: http://www.sraoss.co.jp/index_en.php > Japanese:http://www.sraoss.co.jp > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ishii at sraoss.co.jp Wed Aug 17 11:09:33 2022 From: ishii at sraoss.co.jp (Tatsuo Ishii) Date: Wed, 17 Aug 2022 11:09:33 +0900 (JST) Subject: [pgpool-general: 8355] Re: Cache entries In-Reply-To: References: <20220816.102736.1187728706556188733.t-ishii@sranhm.sraoss.co.jp> Message-ID: <20220817.110933.567525600659681410.t-ishii@sranhm.sraoss.co.jp> > Moreover I found something very weird. > In case you add something to unsafe table in order not to cache queries > from this table. > Pgpool still increment num selects with those queries and this increase > harm the cache hit ratio. > > I am missing something? > > Unsafe = city > Select from city ? let?s say 10 times > Show pool_cache will show > Num selects = 10 That's an expected behavior. cache_hit_ratio = num_cache_hits / (num_cache_hits + num_selects) where num_cache_hits is the total number of queries hitting query cache, and num_selects is the total number of queries (regardless hitting query cache or not). So the number you showed looks correct. Best reagards, -- Tatsuo Ishii SRA OSS LLC English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp From pengbo at sraoss.co.jp Thu Aug 18 13:58:07 2022 From: pengbo at sraoss.co.jp (Bo Peng) Date: Thu, 18 Aug 2022 13:58:07 +0900 Subject: [pgpool-general: 8356] Pgpool-II 4.3.3, 4.2.10, 4.1.13, 4.0.20 and 3.7.25 are now officially released. Message-ID: <20220818135807.c97cf619ac0df85d34416bd0@sraoss.co.jp> Pgpool Global Development Group is pleased to announce the availability of Pgpool-II 4.3.3, 4.2.10, 4.1.13, 4.0.20 and 3.7.25. For more details please see the release notes: https://www.pgpool.net/docs/latest/en/html/release.html You can download the source code and packages from: https://pgpool.net/mediawiki/index.php/Downloads -- Bo Peng SRA OSS LLC https://www.sraoss.co.jp/ From tony.albers at gmx.com Thu Aug 18 21:19:12 2022 From: tony.albers at gmx.com (Tony Albers) Date: Thu, 18 Aug 2022 14:19:12 +0200 Subject: [pgpool-general: 8357] Separate backend(!) network? Message-ID: <20220818141912.a96bd39ac7600745d892020c@gmx.com> Hi guys, Would it be possible to set up 3 postgresql nodes running pgpool-II so that: We have two networks: A, a frontend network(192.168.100.0/24), where clients connect to pgpool-II on port 9999 and get the postgresql service, but nothing else is available on that network. B, a 'heartbeat' network(192.168.110.0/24), where we do watchdog, pcp, heartbeat and failover. Could this be done? I thought so, but I seem to have misunderstood the term 'backend_network' in the docs, which seem to refer to postgresql servers _behind_ a pgpool2 cluster, and not a dedicated failover/heartbeat network. So what I've done is I have 3 nodes, each running PostgreSQL and Pgpool-II, with networks and hostnames like this: adbtopo01.front 192.168.100.143 adbtopo01.back 192.168.110.143 adbtopo02.front 192.168.100.162 adbtopo02.back 192.168.110.162 adbtopo03.front 192.168.100.197 adbtopo03.back 192.168.110.197 VIP 192.168.100.100 And I'd like clients to connect to the VIP, and heartbeat, watchdog etc. to use the .back hostnames/network. Any suggestions are much appreciated. Thanks, Tony -- Tony Albers From imbtmn at gccorp.com Fri Aug 19 08:38:04 2022 From: imbtmn at gccorp.com (=?ks_c_5601-1987?B?w9bB9rz2?=) Date: Thu, 18 Aug 2022 23:38:04 +0000 Subject: [pgpool-general: 8358] RHEL 9 support Message-ID: Hello, We just recently updated our servers from Rocky Linux from 8.6 to 9.0 and discovered that PG-Pool has yet to support RHEL 9. Was just wondering if there were any plans for RHEL 9 anytime soon. Thanks in advance for whoever answers my question. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pengbo at sraoss.co.jp Fri Aug 19 17:43:39 2022 From: pengbo at sraoss.co.jp (Bo Peng) Date: Fri, 19 Aug 2022 17:43:39 +0900 Subject: [pgpool-general: 8359] Re: RHEL 9 support In-Reply-To: References: Message-ID: <20220819174339.75c3b8a845ad83c0c5632a5b@sraoss.co.jp> Hello, On Thu, 18 Aug 2022 23:38:04 +0000 ??? wrote: > Hello, > > We just recently updated our servers from Rocky Linux from 8.6 to 9.0 and discovered that PG-Pool has yet to support RHEL 9. > > Was just wondering if there were any plans for RHEL 9 anytime soon. Yes. We plan to support for RHEL 9 in future releases. > Thanks in advance for whoever answers my question. -- Bo Peng SRA OSS LLC https://www.sraoss.co.jp/ From j.emerlik at gmail.com Mon Aug 22 19:14:25 2022 From: j.emerlik at gmail.com (j. e.) Date: Mon, 22 Aug 2022 12:14:25 +0200 Subject: [pgpool-general: 8360] Active connection Message-ID: <31E8A9BF-6FD9-476D-A615-97CBBFA6FD2B@gmail.com> Hi, I am using pgoool version 4.2.5 and noticed somerhing disturbing. pg_stat_statements showed one query in active state for long time, then I checked network connecrions between pgpool and postgresql using netstat and connection between pgpool and frontend. Connection between pgpool and postgresql is established but between pgpool and frontend is missed. command on pgpool: ps aux showed IP and port of ?connected? frontend but connection in netstat is missed. Why and what could have happened ? How I can diagnose that ? Regards Jack From pengbo at sraoss.co.jp Tue Aug 23 13:59:17 2022 From: pengbo at sraoss.co.jp (Bo Peng) Date: Tue, 23 Aug 2022 13:59:17 +0900 Subject: [pgpool-general: 8361] Re: Active connection In-Reply-To: <31E8A9BF-6FD9-476D-A615-97CBBFA6FD2B@gmail.com> References: <31E8A9BF-6FD9-476D-A615-97CBBFA6FD2B@gmail.com> Message-ID: <20220823135917.1a3d1597b7bd09bf1c3e68f6@sraoss.co.jp> Hello, > Hi, > I am using pgoool version 4.2.5 and noticed somerhing disturbing. > > pg_stat_statements showed one query in active state for long time, then I checked network connecrions between pgpool and postgresql using netstat and connection between pgpool and frontend. > > Connection between pgpool and postgresql is established but between pgpool and frontend is missed. > > command on pgpool: > > ps aux showed IP and port of ?connected? frontend but connection in netstat is missed. > > Why and what could have happened ? > How I can diagnose that ? Your question is why the established connections was remained even there aren't any clients? This behavior is due to connection pooling feature. You can disable this feature using: connection_cache = off -- Bo Peng SRA OSS LLC https://www.sraoss.co.jp/ From j.emerlik at gmail.com Tue Aug 23 16:47:03 2022 From: j.emerlik at gmail.com (j. e.) Date: Tue, 23 Aug 2022 09:47:03 +0200 Subject: [pgpool-general: 8362] Re: Active connection In-Reply-To: <20220823135917.1a3d1597b7bd09bf1c3e68f6@sraoss.co.jp> References: <20220823135917.1a3d1597b7bd09bf1c3e68f6@sraoss.co.jp> Message-ID: Wys?ane z iPhone'a > Wiadomo?? napisana przez Bo Peng w dniu 23.08.2022, o godz. 06:59: > > ?Hello, > >> Hi, >> I am using pgoool version 4.2.5 and noticed somerhing disturbing. >> >> pg_stat_statements showed one query in active state for long time, then I checked network connecrions between pgpool and postgresql using netstat and connection between pgpool and frontend. >> >> Connection between pgpool and postgresql is established but between pgpool and frontend is missed. >> >> command on pgpool: >> >> ps aux showed IP and port of ?connected? frontend but connection in netstat is missed. >> >> Why and what could have happened ? >> How I can diagnose that ? > > Your question is why the established connections was remained even there aren't any clients? > This behavior is due to connection pooling feature. > > You can disable this feature using: > > connection_cache = off > > -- > Bo Peng > SRA OSS LLC > https://www.sraoss.co.jp/ Connection cache is enabled in my configuration it?s a fact, but in my understanding, connection cache should work for connections with "idle" state with substring "DISCARD ALL" what means frontend is diskonected and pgpool cacheing this connection for some period of time (cached connection will be finished after time defined by child_life_time and I have it set to 60s). In my case connection was in "active" state for few hours, client was disconeted and pgpool kept this connection, from postgresql point of view it was wiat_event ClientRead. Something strange from my point of view. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jerabkin at post.cz Wed Aug 31 22:28:46 2022 From: jerabkin at post.cz (Michal Jerabek) Date: Wed, 31 Aug 2022 15:28:46 +0200 (CEST) Subject: [pgpool-general: 8363] =?utf-8?q?pgpoolAdmin_4=2E2_doesnt_see_state_of_postgres=2C_but_?= =?utf-8?q?pcp=5Fstatus_and_=22show_pool=5Fnodes=22_works?= Message-ID: <3gs.fJkT.2GQC2y}yMTs.1Z3s8E@scif.cz> Hello, i? made install of 2 nodes with postgresql-14 (4.6), pgpool-II (4.3.2.) and pgpoolAdmin on RHEL 8.6 (selinux permissive), all postgres related components from repos (https://download.postgresql.org/pub/repos/yum/14/ redhat/rhel-8-x86_64/ and https://apt.postgresql.org/pub/repos/yum/common/ redhat/rhel-8-x86_64/). And I have problem that pgpoolAdmin component show state of postgres always "down" either in pgpool status and Node status , but postgres is running, standard database connection to db backend throught? pgpool works and I am able to attach and dettach nodes from command line. PCP status command shows right state of components, and also query to "show pool_nodes" works, where should be a problem? where pgpoolAdmin get states from? pcp_status Node 0: Hostname?????????????? : 172.27.8.249 Port?????????????????? : 5432 Status???????????????? : 2 Weight???????????????? : 0.500000 Status Name??????????? : up Backend Status Name??? : up Role?????????????????? : primary Backend Role?????????? : primary Replication Delay????? : 0 Replication State????? : none Replication Sync State : none Last Status Change???? : 2022-08-31 15:07:04 Node 1: Hostname?????????????? : 172.27.8.250 Port?????????????????? : 5432 Status???????????????? : 2 Weight???????????????? : 0.500000 Status Name??????????? : up Backend Status Name??? : up Role?????????????????? : standby Backend Role?????????? : primary Replication Delay????? : 402654288 Replication State????? : none Replication Sync State : none Last Status Change???? : 2022-08-31 15:07:04 (of course there is no correct active/standby state, both backend are primary, prepared for recovery/sync) $ psql -h VIPADDRESS -p 5433 -U pgpool postgres -c "show pool_nodes" ?node_id |?? hostname?? | port | status | pg_status | lb_weight |? role?? | pg_role | select_cnt | load_balance_node | replication_delay | replication_ state | replication_sync_state | last_sta tus_change ---------+--------------+------+--------+-----------+-----------+---------+- --------+------------+-------------------+-------------------+-------------- -----+------------------------+--------- ------------ ?0?????? | 172.27.8.249 | 5432 | up???? | up??????? | 0.500000? | primary | primary | 72???????? | false???????????? | 0???????????????? |?????????????? ???? |??????????????????????? | 2022-08- 31 15:07:04 ?1?????? | 172.27.8.250 | 5432 | up???? | up??????? | 0.500000? | standby | primary | 83???????? | true????????????? | 402654288???????? |?????????????? ???? |??????????????????????? | 2022-08- 31 15:07:04 (2 rows) Thank you for help Michal I have similiar expierence wiht installing same versions of psql, pgpool-II and pgpoolAdmin on Centos8 stream, where there is no problem there. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pgpool.png Type: image/png Size: 39106 bytes Desc: not available URL: