Thread: Threading crash using ODBC
Here's what I have:
I have an application with 1 main thread and a number of worker threads.
At startup, I allocate a number of ODBC connections.
I have an internal queue that is protected by a mutex.
When one of the worker threads wants to use the database, it gets an un
occupied SQL connection from the queue.
At startup, I can execte a query just fine. When I try to execute a
query from one of the worker threads, I get a core dump.
I am using UNIX ODBC and the PostgreSQL driver, as shipped with RedHat 7.3.
The stack trace looks like this:
0 0x402fba4c in CC_send_query () from /usr/lib/libodbcpsql.so
(gdb) where
#0 0x402fba4c in CC_send_query () from /usr/lib/libodbcpsql.so
#1 0x40314842 in SC_execute () from /usr/lib/libodbcpsql.so
#2 0x40304c45 in PG_SQLExecute () from /usr/lib/libodbcpsql.so
#3 0x40304c6b in SQLExecute () from /usr/lib/libodbcpsql.so
#4 0x402a13f4 in SQLExecute () from /usr/lib/libodbc.so.1
#5 0x40023d6f in MSqlODBC::ExecResult (this=0x806bfd4,
sql=0x4037f83c "select value, timeout from msession_sessions where
session = '2b7atuahma6u9igcorr7c6q8ld7ne86o'") at modbcsql.cpp:360
The ExecResult function looks like this:
SQLRESULT MSqlODBC::ExecResult(char *sql)
{
TRACE_FUNCT("ExecResult");
unsigned long err;
HSTMT hstmt;
TRACE_STR(sql);
err = SQLAllocStmt(m_hdbc, &hstmt);
if(err != SQL_SUCCESS)
{
fprintf(stderr, "Could not get Stmt\n");
return NULL;
}
err = SQLPrepare(hstmt, (SQLCHAR *)sql, SQL_NTS);
if(err != SQL_SUCCESS)
{
fprintf(stderr, "Could not Prepare Statement [%s]\n", sql);
return NULL;
}
err = SQLExecute(hstmt);
if(err != SQL_SUCCESS)
{
fprintf(stderr, "Could not ExecDirect\n");
SQLFreeStmt(hstmt, SQL_DROP);
hstmt = NULL;
}
return (SQLRESULT)build_result(hstmt);
}
markw wrote: > > Here's what I have: > > I have an application with 1 main thread and a number of worker threads. > At startup, I allocate a number of ODBC connections. > I have an internal queue that is protected by a mutex. > When one of the worker threads wants to use the database, it gets an un > occupied SQL connection from the queue. > > At startup, I can execte a query just fine. When I try to execute a > query from one of the worker threads, I get a core dump. > > I am using UNIX ODBC and the PostgreSQL driver, as shipped with RedHat 7.3. > > The stack trace looks like this: > > 0 0x402fba4c in CC_send_query () from /usr/lib/libodbcpsql.so > (gdb) where > #0 0x402fba4c in CC_send_query () from /usr/lib/libodbcpsql.so > #1 0x40314842 in SC_execute () from /usr/lib/libodbcpsql.so > #2 0x40304c45 in PG_SQLExecute () from /usr/lib/libodbcpsql.so > #3 0x40304c6b in SQLExecute () from /usr/lib/libodbcpsql.so > #4 0x402a13f4 in SQLExecute () from /usr/lib/libodbc.so.1 > #5 0x40023d6f in MSqlODBC::ExecResult (this=0x806bfd4, > sql=0x4037f83c "select value, timeout from msession_sessions where > session = '2b7atuahma6u9igcorr7c6q8ld7ne86o'") at modbcsql.cpp:360 It's the driver provided by unixODBC itself. Please inquire unixODBC team about it. Or try the (maybe) thread-safe driver on *nix recently provided at http://odbc.postgresql.org/. regards, Hiroshi Inoue http://w2422.nsk.ne.jp/~inoue/
Hiroshi Inoue wrote:
markw wrote:Here's what I have: I have an application with 1 main thread and a number of worker threads. At startup, I allocate a number of ODBC connections. I have an internal queue that is protected by a mutex. When one of the worker threads wants to use the database, it gets an un occupied SQL connection from the queue. At startup, I can execte a query just fine. When I try to execute a query from one of the worker threads, I get a core dump. I am using UNIX ODBC and the PostgreSQL driver, as shipped with RedHat 7.3. The stack trace looks like this: 0 0x402fba4c in CC_send_query () from /usr/lib/libodbcpsql.so (gdb) where #0 0x402fba4c in CC_send_query () from /usr/lib/libodbcpsql.so #1 0x40314842 in SC_execute () from /usr/lib/libodbcpsql.so #2 0x40304c45 in PG_SQLExecute () from /usr/lib/libodbcpsql.so #3 0x40304c6b in SQLExecute () from /usr/lib/libodbcpsql.so #4 0x402a13f4 in SQLExecute () from /usr/lib/libodbc.so.1 #5 0x40023d6f in MSqlODBC::ExecResult (this=0x806bfd4, sql=0x4037f83c "select value, timeout from msession_sessions where session = '2b7atuahma6u9igcorr7c6q8ld7ne86o'") at modbcsql.cpp:360It's the driver provided by unixODBC itself. Please inquire unixODBC team about it. Or try the (maybe) thread-safe driver on *nix recently provided at http://odbc.postgresql.org/.
Actually, I tracked it down. It may be something that you are interested in, I had my stack set to 64K, when I bumped it up to 256K, the problem went away.
Both the unixODBC and PostgreSQL drivers are similar, or at least have a similar origin. The stack utilization deserve at least a little scrutiny.
markw wrote: > > Hiroshi Inoue wrote: > > > markw wrote: > > > > > >> Here's what I have: > >> > >> I have an application with 1 main thread and a number of worker > >> threads. > >> At startup, I allocate a number of ODBC connections. > >> I have an internal queue that is protected by a mutex. > >> When one of the worker threads wants to use the database, it gets > >> an un > >> occupied SQL connection from the queue. > >> > >> At startup, I can execte a query just fine. When I try to execute > >> a > >> query from one of the worker threads, I get a core dump. > >> > >> I am using UNIX ODBC and the PostgreSQL driver, as shipped with > >> RedHat 7.3. > >> > >> The stack trace looks like this: > >> > >> 0 0x402fba4c in CC_send_query () from /usr/lib/libodbcpsql.so > >> (gdb) where > >> #0 0x402fba4c in CC_send_query () from /usr/lib/libodbcpsql.so > >> #1 0x40314842 in SC_execute () from /usr/lib/libodbcpsql.so > >> #2 0x40304c45 in PG_SQLExecute () from /usr/lib/libodbcpsql.so > >> #3 0x40304c6b in SQLExecute () from /usr/lib/libodbcpsql.so > >> #4 0x402a13f4 in SQLExecute () from /usr/lib/libodbc.so.1 > >> #5 0x40023d6f in MSqlODBC::ExecResult (this=0x806bfd4, > >> sql=0x4037f83c "select value, timeout from msession_sessions > >> where > >> session = '2b7atuahma6u9igcorr7c6q8ld7ne86o'") at modbcsql.cpp:360 > >> > >> > > It's the driver provided by unixODBC itself. > > Please inquire unixODBC team about it. > > Or try the (maybe) thread-safe driver on *nix > > recently provided at http://odbc.postgresql.org/. > > > > Actually, I tracked it down. It may be something that you are > interested in, I had my stack set to 64K, when I bumped it up to 256K, > the problem went away. > > Both the unixODBC and PostgreSQL drivers are similar, or at least have > a similar origin. The stack utilization deserve at least a little > scrutiny. I've improved a little about the stack utilization of our ODBC driver and some applications could work with the improvement though I'm not sure if it can work with your application. regards, Hiroshi Inoue http://w2422.nsk.ne.jp/~inoue/
Hiroshi Inoue wrote: >markw wrote: > > >>Hiroshi Inoue wrote: >> >> >> >>>markw wrote: >>> >>> >>> >>> >>>>Here's what I have: >>>> >>>>I have an application with 1 main thread and a number of worker >>>>threads. >>>>At startup, I allocate a number of ODBC connections. >>>>I have an internal queue that is protected by a mutex. >>>>When one of the worker threads wants to use the database, it gets >>>>an un >>>>occupied SQL connection from the queue. >>>> >>>>At startup, I can execte a query just fine. When I try to execute >>>>a >>>>query from one of the worker threads, I get a core dump. >>>> >>>>I am using UNIX ODBC and the PostgreSQL driver, as shipped with >>>>RedHat 7.3. >>>> >>>>The stack trace looks like this: >>>> >>>>0 0x402fba4c in CC_send_query () from /usr/lib/libodbcpsql.so >>>>(gdb) where >>>>#0 0x402fba4c in CC_send_query () from /usr/lib/libodbcpsql.so >>>>#1 0x40314842 in SC_execute () from /usr/lib/libodbcpsql.so >>>>#2 0x40304c45 in PG_SQLExecute () from /usr/lib/libodbcpsql.so >>>>#3 0x40304c6b in SQLExecute () from /usr/lib/libodbcpsql.so >>>>#4 0x402a13f4 in SQLExecute () from /usr/lib/libodbc.so.1 >>>>#5 0x40023d6f in MSqlODBC::ExecResult (this=0x806bfd4, >>>> sql=0x4037f83c "select value, timeout from msession_sessions >>>>where >>>>session = '2b7atuahma6u9igcorr7c6q8ld7ne86o'") at modbcsql.cpp:360 >>>> >>>> >>>> >>>> >>>It's the driver provided by unixODBC itself. >>>Please inquire unixODBC team about it. >>>Or try the (maybe) thread-safe driver on *nix >>>recently provided at http://odbc.postgresql.org/. >>> >>> >>> >>Actually, I tracked it down. It may be something that you are >>interested in, I had my stack set to 64K, when I bumped it up to 256K, >>the problem went away. >> >>Both the unixODBC and PostgreSQL drivers are similar, or at least have >>a similar origin. The stack utilization deserve at least a little >>scrutiny. >> >> > >I've improved a little about the stack utilization of >our ODBC driver and some applications could work with >the improvement though I'm not sure if it can work with >your application. > > well, I'm flexable, and I'm sure it would be helpful to many other developers, what is a good estimate for thread stack size? > >
markw wrote:
>
> Hiroshi Inoue wrote:
>
> >markw wrote:
[snip]
> >>Actually, I tracked it down. It may be something that you are
> >>interested in, I had my stack set to 64K, when I bumped it up to 256K,
> >>the problem went away.
> >>
> >>Both the unixODBC and PostgreSQL drivers are similar, or at least have
> >>a similar origin. The stack utilization deserve at least a little
> >>scrutiny.
> >>
> >>
> >
> >I've improved a little about the stack utilization of
> >our ODBC driver and some applications could work with
> >the improvement though I'm not sure if it can work with
> >your application.
> >
> >
> well, I'm flexable, and I'm sure it would be helpful to many other
> developers, what is a good estimate for thread stack size?
Though the unixODBC and PostgreSQL drivers have a similar origin,
they are pretty different now. As for stack utilization, I removed
MAX_MESSAGE_LEN(65536) byte buffers completely and so most
applications may be able to work with 64K stack size.
regards,
Hiroshi Inoue
http://w2422.nsk.ne.jp/~inoue/
Hiroshi Inoue wrote: >markw wrote: > > >>Hiroshi Inoue wrote: >> >> >> >>>markw wrote: >>> >>> > >[snip] > > > >>>>Actually, I tracked it down. It may be something that you are >>>>interested in, I had my stack set to 64K, when I bumped it up to 256K, >>>>the problem went away. >>>> >>>>Both the unixODBC and PostgreSQL drivers are similar, or at least have >>>>a similar origin. The stack utilization deserve at least a little >>>>scrutiny. >>>> >>>> >>>> >>>> >>>I've improved a little about the stack utilization of >>>our ODBC driver and some applications could work with >>>the improvement though I'm not sure if it can work with >>>your application. >>> >>> >>> >>> >>well, I'm flexable, and I'm sure it would be helpful to many other >>developers, what is a good estimate for thread stack size? >> >> > >Though the unixODBC and PostgreSQL drivers have a similar origin, >they are pretty different now. As for stack utilization, I removed >MAX_MESSAGE_LEN(65536) byte buffers completely and so most >applications may be able to work with 64K stack size. > > That's great news. Did you use malloc or alloca? The reason I ask, I know alloca is "discouraged" and not "posix" but it is supported on a lot of platforms (Including Windows as _alloca), and if wrapped in a macro, should be extreamly portable. Given a choice, I'd rather use stack memory for short term allocations and just beef up the stack, than use malloc. For server programs that expect to be running long term, think months or years, heap fragmentation is a serious issue. Stack based allocations are a good way to avoid this problem. > >
markw wrote:
>
> Hiroshi Inoue wrote:
>
> >markw wrote:
> >
[snip]
> >
> >Though the unixODBC and PostgreSQL drivers have a similar origin,
> >they are pretty different now. As for stack utilization, I removed
> >MAX_MESSAGE_LEN(65536) byte buffers completely and so most
> >applications may be able to work with 64K stack size.
> >
> >
> That's great news. Did you use malloc or alloca?
I didn't use malloc to remove MAX_MESSAGE_LEN buffers though
I'm using malloc to build long queries.
regards,
Hiroshi Inoue
http://w2422.nsk.ne.jp/~inoue/