Thread: double free issue in 2.4 (and previous versions)
Running into "python: double free or corruption " that causing my app to terminate while doing inserts. It seems to happen when inserts are happing at a fairly high velocity. I've included some info that might be helpful in tracking it down. From the info, it looks like it may be on returning a ProgrammingError but I do not receive any errors in the PostgreSQL logs and the query that is executed right before the double free or corruption issue completes in Postgres without problem. 100% of queries are inserts and the debug output compared to the database validates the last command executed is successful.
Does anyone have any suggestions?
My code for initializing psycopg2 and getting a cursor: https://gmr.privatepaste.com/776e15d184
It's just a wrapper to deal with a repeating pattern of use in our apps.
Use of that module looks like:
import mypackage.pgsql as pgsql
connection = pgsql.connect(host, port, dbname, user, password)
cursor = pgsql.get_cursor(connection)
Environment:
CentOS 5.3
Python 2.6.6
pgbouncer version 1.3.4
ldd /usr/lib64/python2.6/site-packages/psycopg2/_psycopg.so
linux-vdso.so.1 => (0x00007ffffddff000)
libpq.so.4 => /usr/lib64/libpq.so.4 (0x00007f6ef5811000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f6ef55f6000)
libc.so.6 => /lib64/libc.so.6 (0x00007f6ef529f000)
libssl.so.6 => /lib64/libssl.so.6 (0x00007f6ef5055000)
libcrypto.so.6 => /lib64/libcrypto.so.6 (0x00007f6ef4d04000)
libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00007f6ef4a6e000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f6ef4836000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f6ef4621000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f6ef4408000)
/lib64/ld-linux-x86-64.so.2 (0x000000307ba00000)
libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00007f6ef41da000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f6ef3fd8000)
libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00007f6ef3db2000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f6ef3bae000)
libz.so.1 => /usr/lib64/libz.so.1 (0x00007f6ef399a000)
libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00007f6ef3791000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f6ef358f000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f6ef3376000)
libsepol.so.1 => /lib64/libsepol.so.1 (0x00007f6ef3130000)
(gdb) bt
#0 0x000000307be30215 in call_gmon_start ()
#1 0x000000307be31cc0 in pq_raise (conn=<value optimized out>, curs=0x419ccd80, pgres=<value optimized out>) at psycopg/pqpath.c:121
#2 0x00000000419ccb60 in ?? ()
(no other symbols resolve as I do not have python compiled with gdb support)
(gdb) frame 1
#1 0x000000307be31cc0 in pq_raise (conn=<value optimized out>, curs=0x419ccd80, pgres=<value optimized out>) at psycopg/pqpath.c:121
121 return ProgrammingError;
(gdb) info frame 1
Stack frame at 0x419cca90:
rip = 0x307be31cc0 in pq_raise (psycopg/pqpath.c:121); saved rip 0x419ccb60
called by frame at 0x419cca98, caller of frame at 0x419cca60
source language c.
Arglist at 0x419cca58, args: conn=<value optimized out>, curs=0x419ccd80, pgres=<value optimized out>
Locals at 0x419cca58, Previous frame's sp is 0x419cca90
Saved registers:
rbx at 0x419cca68, rbp at 0x419cca70, r12 at 0x419cca78, r13 at 0x419cca80, rip at 0x419cca88
output from pg_config:
[root@rejected01 rejected]# pg_config
BINDIR = /usr/bin
DOCDIR = /usr/share/doc/pgsql
INCLUDEDIR = /usr/include
PKGINCLUDEDIR = /usr/include/pgsql
INCLUDEDIR-SERVER = /usr/include/pgsql/server
LIBDIR = /usr/lib64
PKGLIBDIR = /usr/lib64/pgsql
LOCALEDIR = /usr/share/locale
MANDIR = /usr/share/man
SHAREDIR = /usr/share/pgsql
SYSCONFDIR = /etc/sysconfig/pgsql
PGXS = /usr/lib64/pgsql/pgxs/src/makefiles/pgxs.mk
CONFIGURE = '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--target=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/usr/com' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--disable-rpath' '--with-perl' '--with-tcl' '--with-tclconfig=/usr/lib64' '--with-python' '--with-openssl' '--with-pam' '--with-krb5' '--enable-nls' '--enable-thread-safety' '--sysconfdir=/etc/sysconfig/pgsql' '--datadir=/usr/share/pgsql' '--with-docdir=/usr/share/doc' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'target_alias=x86_64-redhat-linux-gnu'
CC = gcc
CPPFLAGS = -D_GNU_SOURCE
CFLAGS = -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wall -Wmissing-prototypes -Wpointer-arith -Winline -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing
CFLAGS_SL = -fpic
LDFLAGS =
LDFLAGS_SL =
LIBS = -lpgport -lpam -lssl -lcrypto -lkrb5 -lz -lreadline -ltermcap -lcrypt -lresolv -lnsl -ldl -lm -lbsd
VERSION = PostgreSQL 8.1.11
stdout from app:
*** glibc detected *** python: double free or corruption (!prev): 0x000000000104c520 ***
======= Backtrace: =========
/lib64/libc.so.6[0x307be71ce2]
/lib64/libc.so.6(cfree+0x8c)[0x307be7590c]
/usr/lib64/python2.6/site-packages/psycopg2/_psycopg.so[0x7fce8d616915]
/usr/lib64/python2.6/site-packages/psycopg2/_psycopg.so[0x7fce8d61bc49]
/usr/lib64/python2.6/site-packages/psycopg2/_psycopg.so[0x7fce8d61c321]
python(PyObject_Call+0x68)[0x418a78]
python(PyEval_CallObjectWithKeywords+0x56)[0x485d96]
python[0x4e2216]
python(PyObject_Call+0x68)[0x418a78]
python(PyEval_EvalFrameEx+0x1127)[0x487ae7]
python(PyEval_EvalCodeEx+0x940)[0x48e180]
python(PyEval_EvalFrameEx+0x56b3)[0x48c073]
python(PyEval_EvalCodeEx+0x940)[0x48e180]
python(PyEval_EvalFrameEx+0x56b3)[0x48c073]
python(PyEval_EvalFrameEx+0x679a)[0x48d15a]
python(PyEval_EvalFrameEx+0x679a)[0x48d15a]
python(PyEval_EvalFrameEx+0x679a)[0x48d15a]
python(PyEval_EvalCodeEx+0x940)[0x48e180]
python(PyEval_EvalFrameEx+0x56b3)[0x48c073]
python(PyEval_EvalFrameEx+0x679a)[0x48d15a]
python(PyEval_EvalFrameEx+0x679a)[0x48d15a]
python(PyEval_EvalCodeEx+0x940)[0x48e180]
python[0x4e566d]
python(PyObject_Call+0x68)[0x418a78]
python[0x41fa8f]
python(PyObject_Call+0x68)[0x418a78]
python(PyEval_CallObjectWithKeywords+0x56)[0x485d96]
python[0x4bd6da]
/lib64/libpthread.so.0[0x307ce06367]
/lib64/libc.so.6(clone+0x6d)[0x307bed2f7d]
On 18/03/11 19:31, Gavin M. Roy wrote: > Running into "python: double free or corruption " that causing my app to > terminate while doing inserts. It seems to happen when inserts are > happing at a fairly high velocity. I've included some info that might be > helpful in tracking it down. From the info, it looks like it may be on > returning a ProgrammingError but I do not receive any errors in the > PostgreSQL logs and the query that is executed right before the double > free or corruption issue completes in Postgres without problem. 100% of > queries are inserts and the debug output compared to the database > validates the last command executed is successful. > > Does anyone have any suggestions? First of all enable debug logging in setup.cfg to get an exact trace of what psycopg is doing just before the crash and send it to this list. Also: - which psycopg version are you using? - is your application multithreaded? - can you provide us the code that crashes? (not everything but a small enough piece of code that reproduce the error) federico -- Federico Di Gregorio federico.digregorio@dndg.it Studio Associato Di Nunzio e Di Gregorio http://dndg.it I did appreciate the irony that I was whining about encoding issues on a mailing list that was unable to show those chars, too. -- Antti S. Lankila to mono-devel-list@