View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000015 | Pgpool-II | Bug | public | 2012-06-07 15:41 | 2013-09-12 13:21 |
| Reporter | spork | Assigned To | t-ishii | ||
| Priority | normal | Severity | major | Reproducibility | always |
| Status | resolved | Resolution | open | ||
| OS | FreeBSD | OS Version | 8.2 | ||
| Summary | 0000015: pgpool-II 3.1.3 on FreeBSD regex patterns for blacklist/whitelist report errors | ||||
| Description | Using the default config which is just a simple list of comma separated keywords causes pgpool to throw errors and likely ignore the configuration settings. config statement in pgpool.conf: black_function_list = 'currval,lastval,nextval,setval' # Comma separated list of function names # that write to database # Regexp are accepted log: Jun 7 01:20:22 devrep pgpool[52608]: add_to_patterns: invalid regex pattern: ^currval$ Jun 7 01:20:22 devrep pgpool[52608]: add_to_patterns: invalid regex pattern: ^lastval$ Jun 7 01:20:22 devrep pgpool[52608]: add_to_patterns: invalid regex pattern: ^nextval$ Jun 7 01:20:22 devrep pgpool[52608]: add_to_patterns: invalid regex pattern: ^setval$ This is on a FreeBSD 8.2 box. I see a similar report from about a year ago: http://answerpot.com/showthread.php?3105812-Pgpool+3.1+errors | ||||
| Steps To Reproduce | Setup a config with sample config file, including the "black_function_list" statement. Start pgpool, watch log output. | ||||
| Tags | No tags attached. | ||||
|
|
Works for me (Linux). Maybe FreeBSD specific issue? I don't have FreeBSD at this point, and I cannot confirm now. Sorry. BTW, the link you are referring is not related to the problem you are facing, I think. Probably the problem stated by the link has been fixed by: 3.1.1 (hatsuiboshi) 2011/12/06 * Version 3.1.1 This is a bugfix release against pgpool-II 3.1. * Bug fixes - Fix add_regex_pattern(). It does not allocate enough memory for each black/white_function_list items. The function adds "^" and "$" to each function items which do not contain those characters. Unfortunately the function forgot to add extra 2 bytes for those characters. This may lead to memory corruption errors when pgpool starting up. |
|
|
Still present in 3.1.2. |
|
|
I have tested pgpool-II 3.2.1 and 3.1.5 on FreeBSD 9.0 box. I don't see no problem with black function list so far. |
|
|
Still see it on 3.2.1 (although there are some build issues on FreeBSD in the watchdog code). Maybe this debug info helps. This is FreeBSD 8.3. Oct 18 01:49:18 k16 pgpool: DEBUG: pid 6545: key: white_function_list Oct 18 01:49:18 k16 pgpool: DEBUG: pid 6545: value: '' kind: 4 Oct 18 01:49:18 k16 pgpool: DEBUG: pid 6545: key: black_function_list Oct 18 01:49:18 k16 pgpool: DEBUG: pid 6545: value: 'currval,lastval,nextval,setval' kind: 4 Oct 18 01:49:18 k16 pgpool: DEBUG: pid 6545: extract_string_tokens: token: currval Oct 18 01:49:18 k16 pgpool: DEBUG: pid 6545: extract_string_tokens: token: lastval Oct 18 01:49:18 k16 pgpool: DEBUG: pid 6545: extract_string_tokens: token: nextval Oct 18 01:49:18 k16 pgpool: DEBUG: pid 6545: extract_string_tokens: token: setval Oct 18 01:49:18 k16 pgpool: DEBUG: pid 6545: add_to_patterns: regex pattern: ^currval$ Oct 18 01:49:18 k16 pgpool: ERROR: pid 6545: add_to_patterns: invalid regex pattern: ^currval$ Oct 18 01:49:18 k16 pgpool: DEBUG: pid 6545: add_to_patterns: regex pattern: ^lastval$ Oct 18 01:49:18 k16 pgpool: ERROR: pid 6545: add_to_patterns: invalid regex pattern: ^lastval$ Oct 18 01:49:18 k16 pgpool: DEBUG: pid 6545: add_to_patterns: regex pattern: ^nextval$ Oct 18 01:49:18 k16 pgpool: ERROR: pid 6545: add_to_patterns: invalid regex pattern: ^nextval$ Oct 18 01:49:18 k16 pgpool: DEBUG: pid 6545: add_to_patterns: regex pattern: ^setval$ Oct 18 01:49:18 k16 pgpool: ERROR: pid 6545: add_to_patterns: invalid regex pattern: ^setval$ At least it shows the config parser is getting the values right. :) Just thought I'd add this. 3.2.1 doesn't quite work on FreeBSD 8.3 (can't connect to any db hosts for the health checks), but I wanted to at least document the regex issue. |
|
|
I have fixed some FreeBSD build issues. Next time please grab 3.2-STABLE head. |
|
|
Just downloaded, built and installed 3.2 head. Build issues are totally resolved, but the regex errors are still there: DEBUG: pid 18118: add_to_patterns: regex pattern: ^currval$ ERROR: pid 18118: add_to_patterns: invalid regex pattern: ^currval$ DEBUG: pid 18118: add_to_patterns: regex pattern: ^lastval$ ERROR: pid 18118: add_to_patterns: invalid regex pattern: ^lastval$ DEBUG: pid 18118: add_to_patterns: regex pattern: ^nextval$ ERROR: pid 18118: add_to_patterns: invalid regex pattern: ^nextval$ DEBUG: pid 18118: add_to_patterns: regex pattern: ^setval$ ERROR: pid 18118: add_to_patterns: invalid regex pattern: ^setval$ |
|
|
I confirmed that with FreeBSD 8.3 the problem reproduced. Strange thing is, nm command against pgpool shows following result: [t-ishii@freebsd /usr/local/src/pgpool/pgpool-II-3.2.1]$ nm ~/bin/pgpool |grep regcomp 0000000000476df0 T regcomp "T" means regcomp is a global symbol in pgpool. This should be like this: [t-ishii@localhost tmp]$ nm a.out|grep regcomp U regcomp@@GLIBC_2.2.5 a.out is produced from test program: #include <regex.h> #include <stdio.h> main() { regex_t regexv; int rtn; int regex_flags = REG_NOSUB; regex_flags |= REG_ICASE; regex_flags |= REG_EXTENDED; rtn = regcomp(®exv, "^nextval$", regex_flags); if (rtn != 0) { printf("invalid regex pattern %d\n", rtn); } } This program does exactly same thing in pgpool and works fine of courese. My wild guess is, pgpool uses wrong regcomp module in FreeBSD 8.3. I don't know why though. |
|
|
Just noting this still exists. We've had to stop using pgpool for now, as we have some queries that use "nextval()" and they get sent to the slave, which causes problems for us: Aug 19 10:18:24 k29.i postgres[83999]: [17-1] db=xxx,user=xxx ERROR: cannot execute nextval() in a read-only transaction Aug 19 10:18:24 k29.i postgres[83999]: [17-2] db=xxx,user=xxx STATEMENT: select nextval('signup_id_seq'); I've tested again, now on FreeBSD 8.4 and pgpool 3.3.0. Same error message when pgpool exits: DEBUG: pid 96388: key: black_function_list DEBUG: pid 96388: value: 'currval,lastval,nextval,setval' kind: 4 DEBUG: pid 96388: extract_string_tokens: token: currval DEBUG: pid 96388: extract_string_tokens: token: lastval DEBUG: pid 96388: extract_string_tokens: token: nextval DEBUG: pid 96388: extract_string_tokens: token: setval DEBUG: pid 96388: add_to_patterns: regex pattern: ^currval$ ERROR: pid 96388: add_to_patterns: invalid regex pattern: ^currval$ DEBUG: pid 96388: add_to_patterns: regex pattern: ^lastval$ ERROR: pid 96388: add_to_patterns: invalid regex pattern: ^lastval$ DEBUG: pid 96388: add_to_patterns: regex pattern: ^nextval$ ERROR: pid 96388: add_to_patterns: invalid regex pattern: ^nextval$ DEBUG: pid 96388: add_to_patterns: regex pattern: ^setval$ ERROR: pid 96388: add_to_patterns: invalid regex pattern: ^setval$ I also added a call to the error routine to see what the regex flags are. I don't understand bitwise stuff at all (and barely understand C), so I don't know if "7" represents the correct flags or not: DEBUG: pid 3484: add_to_patterns: regex pattern: ^currval$ ERROR: pid 3484: add_to_patterns: invalid regex pattern: ^currval$ ERROR: pid 3484: add_to_patterns: regex flags: 7 DEBUG: pid 3484: add_to_patterns: regex pattern: ^lastval$ ERROR: pid 3484: add_to_patterns: invalid regex pattern: ^lastval$ ERROR: pid 3484: add_to_patterns: regex flags: 7 DEBUG: pid 3484: add_to_patterns: regex pattern: ^nextval$ ERROR: pid 3484: add_to_patterns: invalid regex pattern: ^nextval$ ERROR: pid 3484: add_to_patterns: regex flags: 7 DEBUG: pid 3484: add_to_patterns: regex pattern: ^setval$ ERROR: pid 3484: add_to_patterns: invalid regex pattern: ^setval$ ERROR: pid 3484: add_to_patterns: regex flags: 7 What else can I give you to check this out? Is "7" the correct flag to bitwise combine 0001 0002 and 0004? Putting another debug in there shows me "errno 2" which maps to "invalid regex". I'm stumped. Your test program above works as expected. |
|
|
I have no idea how to debug further. As I said before, I think the problem must be in the compiler tool chain, including gcc and linker on FreeBSD, not the pgpool source itself. To verify this, you can do: - cd to pgpool source code - compile - see "nm pool_config.o|grep regcomp". If it's not something like: U regcomp You are in trouble. |
|
|
[root@client1 /usr/local/src/pgpool-II-3.3.0]# nm pool_config.o |grep regcomp U regcomp [root@client1 /usr/local/src/pgpool-II-3.3.0]# |
|
|
It turned out if we remove the linking of compat library, everything goes fine. I did it by hand: gcc -g -O2 -Wall -Wmissing-prototypes -Wmissing-declarations -o .libs/pgpool main.o child.o pool_auth.o pool_config.o pool_error.o pool_process_query.o pool_stream.o pool_connection_pool.o pool_params.o pool_signal.o pcp_child.o md5.o pool_shmem.o pool_sema.o pool_system.o pool_rewrite_query.o pool_rewrite_outfuncs.o pool_query_cache.o pool_hba.o pool_path.o pool_ip.o ps_status.o strlcpy.o recovery.o pool_relcache.o pool_process_reporting.o pool_ssl.o pool_timestamp.o pool_proto2.o pool_proto_modules.o pool_lobj.o pool_process_context.o pool_memqcache.o pool_session_context.o pool_query_context.o pool_worker_child.o pool_passwd.o pool_globals.o pool_select_walker.o getopt_long.o parser/nodes.o -L/usr/local/pgsql/lib -lpq parser/libsql-parser.a pcp/.libs/libpcp.so watchdog/lib-watchdog.a -lpthread -lcrypt -lm -Wl,-rpath -Wl,/usr/local/lib -Wl,-rpath -Wl,/usr/local/pgsql/lib However, I don't know how to tell autoconf do that. |
|
|
Just confirmed here that if I pull "-lcompat" from all Makefiles I get a clean build and the error goes away. I have no idea how the whole autotools suite works though. I'm also not really sure what the compat lib is for, trying to read up on it a bit. Thanks! |
|
|
|
|
|
It seems "-lcompat" is in configure.in. Attached patch is just removing it. Unfortunately I couldn't get neccessary tool chains such as autoconf to test it put on my FreeBSD box. Can you please try it out? |
|
|
I missed your last reply... I can confirm that removing the compat check from configure.in results in a working build. I also think I sort of understand why the compat libs break things. Looking at the source, I'm guessing this is for really, really old backward compatibility and it appears to include it's own regex functions, which I imagine is how we end up with the errors - the compat regex functions probably take arguments in a different order or something. Does that sound about right? [root@client1 /usr/local/src/pgpool-II-3.3.0]# ls -R /usr/src/lib/libcompat/ 4.1 4.3 4.4 Makefile regexp /usr/src/lib/libcompat/4.1: ascftime.c cftime.c ftime.c getpw.c cftime.3 ftime.3 getpw.3 /usr/src/lib/libcompat/4.3: cfree.3 re_comp.3 rexec.3 cfree.c regex.c rexec.c /usr/src/lib/libcompat/4.4: cuserid.3 cuserid.c /usr/src/lib/libcompat/regexp: COPYRIGHT regerror.c regexp.c regsub.c README regexp.3 regmagic.h |
|
|
Ok, I'm going to remove -lcompat from configure.in. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2012-06-07 15:41 | spork | New Issue | |
| 2012-06-07 16:58 | t-ishii | Note Added: 0000037 | |
| 2012-09-25 13:48 | spork | Note Added: 0000091 | |
| 2012-10-14 12:21 | t-ishii | Assigned To | => t-ishii |
| 2012-10-14 12:21 | t-ishii | Status | new => assigned |
| 2012-10-14 12:22 | t-ishii | Note Added: 0000104 | |
| 2012-10-18 14:55 | spork | Note Added: 0000109 | |
| 2012-10-18 15:15 | t-ishii | Note Added: 0000110 | |
| 2012-10-20 10:48 | spork | Note Added: 0000114 | |
| 2012-10-20 17:39 | t-ishii | Note Added: 0000118 | |
| 2013-08-28 15:54 | spork | Note Added: 0000313 | |
| 2013-08-28 18:54 | t-ishii | Note Added: 0000314 | |
| 2013-08-29 00:38 | spork | Note Added: 0000315 | |
| 2013-08-29 10:47 | t-ishii | Note Added: 0000316 | |
| 2013-08-29 12:12 | spork | Note Added: 0000317 | |
| 2013-08-29 14:14 | t-ishii | File Added: configure.in.patch | |
| 2013-08-29 14:16 | t-ishii | Note Added: 0000318 | |
| 2013-09-12 10:32 | spork | Note Added: 0000327 | |
| 2013-09-12 13:21 | t-ishii | Note Added: 0000329 | |
| 2013-09-12 13:21 | t-ishii | Status | assigned => resolved |
