[Pgpool-hackers] First pcp command as PostgreSQL function
Jehan-Guillaume (ioguix) de Rorthais
jgdr at dalibo.com
Fri Apr 22 20:57:39 UTC 2011
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 22/04/2011 09:40, Tatsuo Ishii wrote:
>>>>> Sorry but I couldn't apply your patches nor make working it.
>>>>> Your patches produced lots of following messages:
>>>>>
>>>>> Reversed (or previously applied) patch detected! Assume -R? [n]
>>>>
>>>> Look like you tried to apply them multiple times and had an unstable
>>>> source tree ?
>>>
>>> I tried to apply split_reporting_functions-2.patch after
>>> split_reporting_functions-1.patch, or just
>>> split_reporting_functions-2.patch without success.
>>
>> it was split_reporting_functions-2.patch, then pcp_pool_status.patch-1.patch
>> Anyway doesn't matter now.
>
> Oh I seem to do in reverse order.
> Anyway, I have committed your patches. Thanks!
>
> Regarding pgpool_adm unfortunately I got PostgreSQL backend crash with
> double free memory:
Mh, actually I couldn't reproduce it when the connexion is OK:
pgpool=# \x
Expanded display is on.
pgpool=# SELECT version();
-[ RECORD 1 ]---
version | PostgreSQL 9.1devel on x86_64-unknown-linux-gnu, compiled by
GCC gcc (GCC) 4.5.2 20110127 (prerelease), 64-bit
pgpool=# \des+
List of foreign servers
-[ RECORD 1 ]--------+--------------------------------------
Name | pgpool0
Owner | postgres
Foreign-data wrapper | pgpool
Access privileges |
Type |
Version |
Options | {host=127.0.0.1,port=9898,timeout=10}
pgpool=# \deu+
List of user mappings
-[ RECORD 1 ]--------------------------------
Server | pgpool0
User name | pgpool
Options | {user=pgpool_adm,password=pgpass}
pgpool=# SET ROLE pgpool;
SET
pgpool=> \timing
Timing is on.
pgpool=> \o /dev/null
pgpool=> SELECT *
FROM generate_series(1,100000),
pcp_node_info(0, '127.0.0.1', 10, 9898, 'pgpool_adm', 'pgpass');
Time: 141.146 ms
pgpool=> SELECT *
FROM generate_series(1,100000),
pcp_pool_status('127.0.0.1', 10, 9898, 'pgpool_adm', 'pgpass');
Time: 5743.456 ms
pgpool=> SELECT *
FROM generate_series(1,100000), pcp_node_info(0, 'pgpool0');
Time: 146.723 ms
pgpool=> SELECT *
FROM generate_series(1,100000), pcp_pool_status('pgpool0');
Time: 5502.824 ms
However, while writing this email, I thought about trying with bad
connexions parameters:
pgpool=# SELECT * FROM generate_series(1,100),
pcp_pool_status('127.0.0.1', 10, 9898, 'BAD', 'pgpass');
ERROR: Cannot connect to PCP server.
pgpool=# SELECT * FROM generate_series(1,100),
pcp_pool_status('127.0.0.1', 10, 9898, 'BAD', 'pgpass');
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
So I did a quick test with the command pcp_node_info.c:
diff -r1.9 pcp_node_info.c
49a50
> int i;
134a136
> for(i=0; i<2; i++) {
138c140,141
< myexit(errorcode);
---
> /*myexit(errorcode);*/
> continue;
165a169
> }
Compile, then:
$ ./pcp_node_info 10 127.0.0.1 9898 pgpool_adm pgpass 0
/tmp/ 5484 2 1.000000
/tmp/ 5484 2 1.000000
$ ./pcp_node_info 10 127.0.0.1 9898 BAD pgpass 0
AuthorizationError
I0
^C ### I Killed myself here.
$ ./pcp_node_info 10 127.0.0.1 9898 pgpool_adm BAD 0
AuthorizationError
WriteError
*** glibc detected *** /home/ioguix/src/cvs/pgpool/pgpool-II.status
/pcp/.libs/lt-pcp_node_info: double free or corruption (!prev):
0x0000000001929460 ***
======= Backtrace: =========
/lib/libc.so.6(+0x71b96)[0x7f0a87f09b96]
/lib/libc.so.6(cfree+0x6c)[0x7f0a87f0e96c]
/home/ioguix/src/cvs/pgpool/pgpool-II.status/pcp/.libs/libpcp.so.0(pcp_close+0x14)[0x7f0a88ae6bc4]
/home/ioguix/src/cvs/pgpool/pgpool-II.status/pcp/.libs/libpcp.so.0(pcp_disconnect+0x65)[0x7f0a88ae43b5]
/home/ioguix/src/cvs/pgpool/pgpool-II.status/pcp/.libs/lt-pcp_node_info[0x401094]
/lib/libc.so.6(__libc_start_main+0xfd)[0x7f0a87eb6dcd]
/home/ioguix/src/cvs/pgpool/pgpool-II.status/pcp/.libs/lt-pcp_node_info[0x400aa9]
> 30628 2011-04-22 16:04:33 JST LOG: statement: SELECT * FROM pcp_node_info(0, 'localhost', 10, 9898, 'postgres', 'pgpoolAdmin');
> *** glibc detected *** postgres: t-ishii test [local] SELECT: double free or corruption (out): 0x00007f8c4188cc10 ***
> ======= Backtrace: =========
> /lib64/libc.so.6[0x7f8c415aca42]
> /lib64/libc.so.6(cfree+0x8c)[0x7f8c415b024c]
> /usr/local/pgsql/lib/libpcp.so.0(pcp_close+0x14)[0x7f8c33b94af4]
> /usr/local/pgsql/lib/libpcp.so.0(pcp_disconnect+0x63)[0x7f8c33b92653]
> /usr/local/pgsql/lib/pgpool_adm.so(_pcp_node_info+0x5db)[0x7f8c33d994bb]
> [...]
> /lib64/libc.so.6(__libc_start_main+0xf4)[0x7f8c41556304]
It seems this double free happened in pcp_close(), where we have a
free() on pc->fd, pc->wbuf, pc->hp and pc, anyway, and I never deal with
them in my code...
I tried to investigate the pcp.c and pcp_stream.c code, I hadn't find
the bug yet
>
>>>>> (I fixed rejecting patches caused by Toshihiro's commit)
>>>>>
>>>>> I couldn't solve undefined reference to get_config().
>>>>>
>>>>> /home/t-ishii/work/pgfoundry/tmp/pgpool-II/pcp_child.c:911: undefined
>>>>> reference to `get_config'
>>>>
>>>> get_config is in the first patch about splitting the report functions
>>>>
>>>>> It seems the function body does not exist in your patches at all.
>>>>>
>>>>> Please re-submit complete patches against CVS HEAD so that I can test
>>>>> it.
>>>>
>>>> PFA a cumulative patch synched with current CVS
>>>>
>>>> I was able to build pgpool after applying the patch with the following
>>>> command:
>>>>
>>>> patch -p1 < /tmp/split_report_funcs+pcp_pool_status.patch
>>>>
>>>> Originally, I decided to split both patches for ease of review, sorry if
>>>> it was confusing :(
>>>
>>> Thanks! This is really helpfull! Regarding the patches size, one large
>>> patch is not problem at least for me.
>>
>> Ok
>>
>>> --
>>> Tatsuo Ishii
>>> SRA OSS, Inc. Japan
>>> English: http://www.sraoss.co.jp/index_en.php
>>> Japanese: http://www.sraoss.co.jp
>>>
>>>>> On 03/04/2011 01:38, Tatsuo Ishii wrote:
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> Following our previous discussions I created a PostgreSQL contrib module
>>>>>>>>> which brings pgpool's pcp command as function in PostgreSQL. I called it
>>>>>>>>> pgpool_adm, any better ideas ?
>>>>>>>>
>>>>>>>> Great! I think "pgpool_adm" is fine.
>>>>>>>>
>>>>>>>>> As a first step, I decided to focus on one function only: pcp_node_info.
>>>>>>>>> I'll add other ones as soon as this one will be committable.
>>>>>>>>
>>>>>>>> I see. BTW, are you going to implement a function for "show
>>>>>>>> pool_status"?
>>>>>
>>>>> Here is a new version of the contrib module, including support for
>>>>> pcp_pool_status.
>>>>>
>>>>> It requires a patched version of pgpool with my 2 patches
>>>>> "split_reporting_functions-1.patch" and "pcp_pool_status-2.patch".
>>>>>
>>>>> Follow the same installation instructions than in my previous emails to
>>>>> install this module and, optionally some foreign server wrapper.
>>>>>
>>>>> Then:
>>>>>
>>>>> pgpool=> SELECT * FROM pcp_pool_status('127.0.0.1', 10, 9898,
>>>>> 'pgpool_adm', 'password');
>>>>>
>>>>> or, if you are using foreign server capability in postgresql 8.4+:
>>>>>
>>>>> pgpool=> SELECT * FROM pcp_pool_status('pgpool0');
>>>>>
>>>>> Feedbacks and comments ?
>>>>>
>>>>>>>>> INSTALL: (tested against current 9.1devel HEAD)
>>>>>>>>>
>>>>>>>>> $ cd $PGSRC/contrib
>>>>>>>>> $ tar zxf pgpool_adm.tgz
>>>>>>>>> $ cd pgpool_adm
>>>>>>>>> $ make && sudo make install
>>>>>>>>>
>>>>>>>>> then:
>>>>>>>>>
>>>>>>>>> $ psql pgpool
>>>>>>>>> CREATE SCHEMA pgpool;
>>>>>>>>> CREATE EXTENSION pgpool_adm WITH SCHEMA pgpool ;
>>>>>>>>> SELECT * FROM pgpool.pcp_node_info(0, 'localhost', 10, 9898,
>>>>>>>>> 'pgpool_adm', 'password');
>>>>>>>>> host | port | status | weight
>>>>>>>>> -----------+------+---------------------------------+-------------
>>>>>>>>> localhost | 5491 | Waiting for connection to start | 1.46602e+13
>>>>>>>>> (1 ligne)
>>>>>>>>>
>>>>>>>>> If you don't want/cannot test with 9.1, just adapt and run the "CREATE
>>>>>>>>> FUNCTION" query located in pgpool_adm/pgpool_adm--1.0.sql.
>>>>>>>>>
>>>>>>>>> NOTE:
>>>>>>>>> I had to mess a bit to make the compilation work out of the box because
>>>>>>>>> of my setup of pgpool installing stuffs in /usr/local/{lib,include}.
>>>>>>>>> Basically editing my system ldd setup and configuring postgresql with
>>>>>>>>> "--with-libraries" and "--with-includes". Drop me a mail if you have any
>>>>>>>>> trouble.
>>>>>>>>>
>>>>>>>>> TODO:
>>>>>>>>> * I will try to make it work with foreign server and user mappings so
>>>>>>>>> we will not have to give all servers parameters (host, port, timeout),
>>>>>>>>> but most importantly, not have to write in clear text authentication
>>>>>>>>> parameters.
>>>>>>>>> * Why do we backend_weight/RAND_MAX in the code ?
>>>>>>>>
>>>>>>>> To normalize backend weight.
>>>>>>>>
>>>>>>>>> Feedback & comments ?
>>>>>>>>> - --
>>>>>>>>> Jehan-Guillaume (ioguix) de Rorthais
>>>>>>>>> DBA
>>>>>>>>> http://www.dalibo.com
>>>>>>>>>
>>>>
>>>>
>>>> - --
>>>> Jehan-Guillaume (ioguix) de Rorthais
>>>> DBA
>>>> http://www.dalibo.com
>> - --
>> Jehan-Guillaume (ioguix) de Rorthais
>> DBA
>> http://www.dalibo.com
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.11 (GNU/Linux)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>
>> iEYEARECAAYFAk2xLBYACgkQXu9L1HbaT6IzhQCfTUItDy2JiE8dMop7HszW6xYR
>> iMYAoNTQtd2PvI5nxkpWaBC++4Vtp0DX
>> =VsS5
>> -----END PGP SIGNATURE-----
- --
Jehan-Guillaume (ioguix) de Rorthais
DBA
http://www.dalibo.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk2x68MACgkQXu9L1HbaT6Ly6wCeKaAfD0dleEEvVVBnGuQtaldM
4bYAmgKMPMR+W1WXaibAmiKB3I66g8SP
=KTaV
-----END PGP SIGNATURE-----
More information about the Pgpool-hackers
mailing list