[pgpool-hackers: 2354] segfault with pg_md5
Tatsuo Ishii
ishii at sraoss.co.jp
Mon May 22 17:49:01 JST 2017
Usama,
>From commit:
https://git.postgresql.org/gitweb/?p=pgpool2.git;a=commit;h=18f4d2ea08c2562361858b09534490a6fa09d496
"Allow to configure health check parameters for each individual node."
I have been experincing regression failure on my Ubuntu 17 laptop.
Actually it is caused by pg_md5 segfault. For example following
command reliably segfaults.
$ temp/installed/bin/pg_md5 -m -f tests/001.load_balance/testdir/etc/pgpool.conf -u aaa aaa
Here's the stack trace info.
$ gdb temp/installed/bin/pg_md5
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from temp/installed/bin/pg_md5...done.
(gdb) run -m -f tests/001.load_balance/testdir/etc/pgpool.conf -u aaa aaa
Starting program: /home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5 -m -f tests/001.load_balance/testdir/etc/pgpool.conf -u aaa aaa
*** Error in `/home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5': free(): invalid next size (fast): 0x0000000000652c60 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7ffff7a857e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x7fe0a)[0x7ffff7a8de0a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7ffff7a9198c]
/lib/x86_64-linux-gnu/libc.so.6(+0xe0224)[0x7ffff7aee224]
/lib/x86_64-linux-gnu/libc.so.6(regcomp+0xbf)[0x7ffff7af569f]
/home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5[0x408b6b]
/home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5[0x404e84]
/home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5[0x407507]
/home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5[0x40887e]
/home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5[0x401d6c]
/home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5[0x401aa8]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7ffff7a2e830]
/home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5[0x401c29]
======= Memory map: ========
00400000-00411000 r-xp 00000000 08:05 16694994 /home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5
00610000-00611000 r--p 00010000 08:05 16694994 /home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5
00611000-00617000 rw-p 00011000 08:05 16694994 /home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5
00617000-00655000 rw-p 00000000 00:00 0 [heap]
7ffff0000000-7ffff0021000 rw-p 00000000 00:00 0
7ffff0021000-7ffff4000000 ---p 00000000 00:00 0
7ffff77f8000-7ffff780e000 r-xp 00000000 08:05 12062567 /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffff780e000-7ffff7a0d000 ---p 00016000 08:05 12062567 /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffff7a0d000-7ffff7a0e000 rw-p 00015000 08:05 12062567 /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffff7a0e000-7ffff7bcd000 r-xp 00000000 08:05 12183870 /lib/x86_64-linux-gnu/libc-2.23.so
7ffff7bcd000-7ffff7dcd000 ---p 001bf000 08:05 12183870 /lib/x86_64-linux-gnu/libc-2.23.so
7ffff7dcd000-7ffff7dd1000 r--p 001bf000 08:05 12183870 /lib/x86_64-linux-gnu/libc-2.23.so
7ffff7dd1000-7ffff7dd3000 rw-p 001c3000 08:05 12183870 /lib/x86_64-linux-gnu/libc-2.23.so
7ffff7dd3000-7ffff7dd7000 rw-p 00000000 00:00 0
7ffff7dd7000-7ffff7dfd000 r-xp 00000000 08:05 12183871 /lib/x86_64-linux-gnu/ld-2.23.so
7ffff7fd2000-7ffff7fd5000 rw-p 00000000 00:00 0
7ffff7ff5000-7ffff7ff8000 rw-p 00000000 00:00 0
7ffff7ff8000-7ffff7ffa000 r--p 00000000 00:00 0 [vvar]
7ffff7ffa000-7ffff7ffc000 r-xp 00000000 00:00 0 [vdso]
7ffff7ffc000-7ffff7ffd000 r--p 00025000 08:05 12183871 /lib/x86_64-linux-gnu/ld-2.23.so
7ffff7ffd000-7ffff7ffe000 rw-p 00026000 08:05 12183871 /lib/x86_64-linux-gnu/ld-2.23.so
7ffff7ffe000-7ffff7fff000 rw-p 00000000 00:00 0
7ffffffde000-7ffffffff000 rw-p 00000000 00:00 0 [stack]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
Program received signal SIGABRT, Aborted.
0x00007ffff7a43428 in __GI_raise (sig=sig at entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
54 ../sysdeps/unix/sysv/linux/raise.c: そのようなファイルやディレクトリはありません.
(gdb) bt
#0 0x00007ffff7a43428 in __GI_raise (sig=sig at entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1 0x00007ffff7a4502a in __GI_abort () at abort.c:89
#2 0x00007ffff7a857ea in __libc_message (do_abort=do_abort at entry=2,
fmt=fmt at entry=0x7ffff7b9e2e0 "*** Error in `%s': %s: 0x%s ***\n")
at ../sysdeps/posix/libc_fatal.c:175
#3 0x00007ffff7a8de0a in malloc_printerr (ar_ptr=<optimized out>, ptr=<optimized out>,
str=0x7ffff7b9e358 "free(): invalid next size (fast)", action=3) at malloc.c:5004
#4 _int_free (av=<optimized out>, p=<optimized out>, have_lock=0) at malloc.c:3865
#5 0x00007ffff7a9198c in __GI___libc_free (mem=<optimized out>) at malloc.c:2966
#6 0x00007ffff7aee224 in analyze (preg=0x7fffffff7e10) at regcomp.c:1166
#7 re_compile_internal (preg=preg at entry=0x7fffffff7e10, pattern=pattern at entry=0x651e90 "^nextval$",
length=<optimized out>, syntax=syntax at entry=4436732) at regcomp.c:791
#8 0x00007ffff7af569f in __regcomp (preg=preg at entry=0x7fffffff7e10, pattern=0x651e90 "^nextval$",
cflags=<optimized out>) at regcomp.c:498
#9 0x0000000000408b6b in add_regex_pattern (type=<optimized out>, s=<optimized out>)
at pool_config.l:177
#10 0x0000000000404e84 in initialize_variables_with_default (
gconf=0x6141b0 <ConfigureNamesStringList+304>) at pool_config_variables.c:2363
#11 0x0000000000407507 in InitializeConfigOptions () at pool_config_variables.c:2449
#12 0x000000000040887e in pool_init_config () at pool_config.l:119
#13 0x0000000000401d6c in update_pool_passwd (
conf_file=conf_file at entry=0x7fffffffc110 "tests/001.load_balance/testdir/etc/pgpool.conf",
username=username at entry=0x7fffffffbff0 "aaa", password=0x7fffffffe6b8 "aaa") at pg_md5.c:199
#14 0x0000000000401aa8 in main (argc=7, argv=0x7fffffffe238) at pg_md5.c:179
(gdb)
Also valgrind shows some errors.
t-ishii at localhost: valgrind temp/installed/bin/pg_md5 -m -f tests/001.load_balance/testdir/etc/pgpool.conf -u aaa aaa
==16881== Memcheck, a memory error detector
==16881== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==16881== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==16881== Command: temp/installed/bin/pg_md5 -m -f tests/001.load_balance/testdir/etc/pgpool.conf -u aaa aaa
==16881==
==16881== Invalid write of size 8
==16881== at 0x4074A2: build_variable_groups (pool_config_variables.c:2075)
==16881== by 0x4074A2: build_config_variables (pool_config_variables.c:2016)
==16881== by 0x4074A2: InitializeConfigOptions (pool_config_variables.c:2441)
==16881== by 0x40887D: pool_init_config (pool_config.l:119)
==16881== by 0x401D6B: update_pool_passwd (pg_md5.c:199)
==16881== by 0x401AA7: main (pg_md5.c:179)
==16881== Address 0x5227018 is 0 bytes after a block of size 40 alloc'd
==16881== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==16881== by 0x4094B4: pg_malloc (fe_memutils.c:33)
==16881== by 0x4094DD: pg_malloc0 (fe_memutils.c:47)
==16881== by 0x407432: build_variable_groups (pool_config_variables.c:2064)
==16881== by 0x407432: build_config_variables (pool_config_variables.c:2016)
==16881== by 0x407432: InitializeConfigOptions (pool_config_variables.c:2441)
==16881== by 0x40887D: pool_init_config (pool_config.l:119)
==16881== by 0x401D6B: update_pool_passwd (pg_md5.c:199)
==16881== by 0x401AA7: main (pg_md5.c:179)
==16881==
==16881== Invalid write of size 8
==16881== at 0x4074B4: build_variable_groups (pool_config_variables.c:2077)
==16881== by 0x4074B4: build_config_variables (pool_config_variables.c:2016)
==16881== by 0x4074B4: InitializeConfigOptions (pool_config_variables.c:2441)
==16881== by 0x40887D: pool_init_config (pool_config.l:119)
==16881== by 0x401D6B: update_pool_passwd (pg_md5.c:199)
==16881== by 0x401AA7: main (pg_md5.c:179)
==16881== Address 0x5227020 is 8 bytes after a block of size 40 alloc'd
==16881== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==16881== by 0x4094B4: pg_malloc (fe_memutils.c:33)
==16881== by 0x4094DD: pg_malloc0 (fe_memutils.c:47)
==16881== by 0x407432: build_variable_groups (pool_config_variables.c:2064)
==16881== by 0x407432: build_config_variables (pool_config_variables.c:2016)
==16881== by 0x407432: InitializeConfigOptions (pool_config_variables.c:2441)
==16881== by 0x40887D: pool_init_config (pool_config.l:119)
==16881== by 0x401D6B: update_pool_passwd (pg_md5.c:199)
==16881== by 0x401AA7: main (pg_md5.c:179)
==16881==
==16881== Invalid write of size 8
==16881== at 0x4074C6: build_variable_groups (pool_config_variables.c:2079)
==16881== by 0x4074C6: build_config_variables (pool_config_variables.c:2016)
==16881== by 0x4074C6: InitializeConfigOptions (pool_config_variables.c:2441)
==16881== by 0x40887D: pool_init_config (pool_config.l:119)
==16881== by 0x401D6B: update_pool_passwd (pg_md5.c:199)
==16881== by 0x401AA7: main (pg_md5.c:179)
==16881== Address 0x5227028 is 16 bytes after a block of size 40 alloc'd
==16881== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==16881== by 0x4094B4: pg_malloc (fe_memutils.c:33)
==16881== by 0x4094DD: pg_malloc0 (fe_memutils.c:47)
==16881== by 0x407432: build_variable_groups (pool_config_variables.c:2064)
==16881== by 0x407432: build_config_variables (pool_config_variables.c:2016)
==16881== by 0x407432: InitializeConfigOptions (pool_config_variables.c:2441)
==16881== by 0x40887D: pool_init_config (pool_config.l:119)
==16881== by 0x401D6B: update_pool_passwd (pg_md5.c:199)
==16881== by 0x401AA7: main (pg_md5.c:179)
==16881==
==16881==
==16881== HEAP SUMMARY:
==16881== in use at exit: 159,848 bytes in 3,700 blocks
==16881== total heap usage: 5,872 allocs, 2,172 frees, 483,653 bytes allocated
==16881==
==16881== LEAK SUMMARY:
==16881== definitely lost: 4,586 bytes in 1,034 blocks
==16881== indirectly lost: 0 bytes in 0 blocks
==16881== possibly lost: 0 bytes in 0 blocks
==16881== still reachable: 155,262 bytes in 2,666 blocks
==16881== suppressed: 0 bytes in 0 blocks
==16881== Rerun with --leak-check=full to see details of leaked memory
==16881==
==16881== For counts of detected and suppressed errors, rerun with: -v
==16881== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0)
Any idea?
Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp
More information about the pgpool-hackers
mailing list