Thread: ECPG, threading and pooling
Hi all, I would like to use ECPG as it is relatively easy to code. However my application is multithreaded and also uses connecion pools. I would like to know if ECPG is thread safe and I can use an arbitrary PGconn* object pulled from a connection pool. Do I need to use connection name? I mean how do I store connection name in an array or so? Otherwise I would have to fiddle with libpq which is bit more difficult than ECPG. TIA Shridhar
On Thu, Jan 23, 2003 at 02:40:33PM +0530, Shridhar Daithankar<shridhar_daithankar@persistent.co.in> wrote: > I would like to use ECPG as it is relatively easy to code. However my > application is multithreaded and also uses connecion pools. I'm afraid it needs some work to be thread-safe. sqlca is defined statically. No big deal it seems to implement a thread safe version but I haven't yet found the time. > I would like to know if ECPG is thread safe and I can use an arbitrary PGconn* > object pulled from a connection pool. Not sure what you mean with this? > Do I need to use connection name? I mean how do I store connection name in an > array or so? How else would you decide against whcih connection to run the statement? Michael -- Michael Meskes Email: Michael@Fam-Meskes.De ICQ: 179140304 Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!
On Thursday 23 January 2003 08:40 pm, you wrote: > On Thu, Jan 23, 2003 at 02:40:33PM +0530, Shridhar Daithankar<shridhar_daithankar@persistent.co.in> wrote: > > I would like to know if ECPG is thread safe and I can use an arbitrary > > PGconn* object pulled from a connection pool. > Not sure what you mean with this? Well, when using libpq, I create a pool of PGconn*. What pool I create with ecpg? Looking at ecpglib.h, it seems that I need to create a pool of char * > > Do I need to use connection name? I mean how do I store connection name > > in an array or so? > How else would you decide against whcih connection to run the statement? That is correct but I did not find any example of how to use a particular connection in an SQL statement. It would be a fairly trivial guess that the connection name is one of the arguments to ECPGdo. But how the SQL statement look like, I don't have a clue. Also I don't know know how to use ECPGsetconn to set a connection. No documentation on that. I can guess that ECPG maintains a mapping between connection name and PGconn* object for a user. If possible it would be very helpful to form a ecpg connection on top a user supplied PGconn* object. I just downloaded 7.3.1 to take a look at. I hope to make a contribution if possible.Shridhar
On Thu, Jan 23, 2003 at 08:58:24PM +0530, Shridhar Daithankar wrote: > Well, when using libpq, I create a pool of PGconn*. What pool I create with > ecpg? Looking at ecpglib.h, it seems that I need to create a pool of char * Yes. You use it as EXEC SQL AT :connection_var SELECT ... > That is correct but I did not find any example of how to use a particular > connection in an SQL statement. It would be a fairly trivial guess that the > connection name is one of the arguments to ECPGdo. But how the SQL statement > look like, I don't have a clue. Also I don't know know how to use ECPGsetconn > to set a connection. No documentation on that. You don't call either one yourself. Let ecpg do that job. Some of the test cases in the source tree use AT. Just look under .../ecpg/test. > I can guess that ECPG maintains a mapping between connection name and PGconn* > object for a user. If possible it would be very helpful to form a ecpg > connection on top a user supplied PGconn* object. Why? It's possible, but you have to work with structures not meant to be used outside the lib. Michael -- Michael Meskes Email: Michael@Fam-Meskes.De ICQ: 179140304 Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!
Michael, Michael Meskes writes: > On Thu, Jan 23, 2003 at 02:40:33PM +0530, Shridhar Daithankar<shridhar_daithankar@persistent.co.in> wrote: > > I would like to use ECPG as it is relatively easy to code. However my > > application is multithreaded and also uses connecion pools. > > I'm afraid it needs some work to be thread-safe. sqlca is defined > statically. No big deal it seems to implement a thread safe version but > I haven't yet found the time. I've spent a bit of time on making ecpg thread safe over Christmas, while it's not finished i'm sure the attached patch is at least useful and a step in the right direction. Lee.
Attachment
On Mon, Jan 27, 2003 at 11:52:18AM +0000, Lee Kindness wrote: Content-Description: message body text > Michael, > ... > I've spent a bit of time on making ecpg thread safe over Christmas, > while it's not finished i'm sure the attached patch is at least useful > and a step in the right direction. Thanks a lot. I have no experience in multithreaded software development so I cannot completely check your patch but it surely looks good. Shall I commit it, or will you? I think we should get it into CVS for all to test. Just two questions: - Is it neccessary to patch c.h for ecpg? - Not sure with the version numbering but I can fix that later. Michael -- Michael Meskes Email: Michael@Fam-Meskes.De ICQ: 179140304 Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!
Michael Meskes writes:> On Mon, Jan 27, 2003 at 11:52:18AM +0000, Lee Kindness wrote:> > I've spent a bit of time on makingecpg thread safe over Christmas,> > while it's not finished i'm sure the attached patch is at least useful> > and astep in the right direction.> Thanks a lot. I have no experience in multithreaded software development> so I cannot completelycheck your patch but it surely looks good. Shall> I commit it, or will you? I think we should get it into CVS forall to> test. Problem with it is it needs some changes to configure.in (and associated files) for it to be worthwhile. Checks would need to be added to determine is threads are supported on the build platform and the needed compile and link flags. I'm trying to get this together for libpq too. > Just two questions:> - Is it neccessary to patch c.h for ecpg? No, but each ecpg source file would need something like: #ifdef HAVE_PTHREAD_H# include <pthread.h># include <errno.h>#endif It's in c.h just now for ease of testing. > - Not sure with the version numbering but I can fix that later. As discussed on pgsql-hackers after the 7.3 release any "binary incompatible" library change needs a major version number increase. Lee.
On 27 Jan 2003 at 14:06, Lee Kindness wrote: > Michael Meskes writes: > > On Mon, Jan 27, 2003 at 11:52:18AM +0000, Lee Kindness wrote: > > > I've spent a bit of time on making ecpg thread safe over Christmas, > > > while it's not finished i'm sure the attached patch is at least useful > > > and a step in the right direction. > > Thanks a lot. I have no experience in multithreaded software development > > so I cannot completely check your patch but it surely looks good. Shall > > I commit it, or will you? I think we should get it into CVS for all to > > test. > > Problem with it is it needs some changes to configure.in (and > associated files) for it to be worthwhile. Checks would need to be > added to determine is threads are supported on the build platform and > the needed compile and link flags. I'm trying to get this together for > libpq too. Just out of curiosity, what happens when there are more than one competing threading libraries? Like native threads and linuxthreads on freeBSD? ByeShridhar -- Reliable source, n.: The guy you just met.
Shridhar Daithankar writes:> On 27 Jan 2003 at 14:06, Lee Kindness wrote:> > Michael Meskes writes:> > > Thanks a lot. Ihave no experience in multithreaded software development> > > so I cannot completely check your patch but it surely looksgood. Shall> > > I commit it, or will you? I think we should get it into CVS for all to> > > test.> > Problem withit is it needs some changes to configure.in (and> > associated files) for it to be worthwhile. Checks would need to be>> added to determine is threads are supported on the build platform and> > the needed compile and link flags. I'm tryingto get this together for> > libpq too.> Just out of curiosity, what happens when there are more than one competing> threading libraries? Like native threads and linuxthreads on freeBSD? It's all down to the checks in configure... Looking at many packages which have threads check in configure (e.g. openldap, mysql, mozilla, glib, ...) some let the user specify which theading library to use (e.g --with-threads=linuxthreads (or something)) while others just seem to pick up the first available... To be honest I am quite daunted by the checks needed (every other package seems to be doing the check differently!) and any/all help would be welcome! L.
Michael Meskes <meskes@postgresql.org> writes:
> - Is it neccessary to patch c.h for ecpg?
If you commit that part, it will be reverted forthwith (especially the
hardwired #define HAVE_PTHREAD_H ;-)).
Keep the thread hacking on the client side, please. Isn't there one of
the ecpg/include/ files that would be suitable?
regards, tom lane
On Mon, Jan 27, 2003 at 02:06:26PM +0000, Lee Kindness wrote: > > - Not sure with the version numbering but I can fix that later. > > As discussed on pgsql-hackers after the 7.3 release any "binary > incompatible" library change needs a major version number increase. Sorry, I was not precise enough. The major number has to be inceased, it's the minor numbers that need fixing. :-) Michael -- Michael Meskes Email: Michael@Fam-Meskes.De ICQ: 179140304 Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!
On Mon, Jan 27, 2003 at 09:30:50AM -0500, Tom Lane wrote: > Michael Meskes <meskes@postgresql.org> writes: > > - Is it neccessary to patch c.h for ecpg? > > If you commit that part, it will be reverted forthwith (especially the > hardwired #define HAVE_PTHREAD_H ;-)). > > Keep the thread hacking on the client side, please. Isn't there one of > the ecpg/include/ files that would be suitable? Yes, there is. Better make that there are. There is a file called extern.h both in preproc as well as in lib and each file in the directory does include the extern.h in its own directory. So we just need to add it twice I guess. Michael -- Michael Meskes Email: Michael@Fam-Meskes.De ICQ: 179140304 Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!
Michael Meskes <meskes@postgresql.org> writes:
>> Keep the thread hacking on the client side, please. Isn't there one of
>> the ecpg/include/ files that would be suitable?
> Yes, there is. Better make that there are. There is a file called
> extern.h both in preproc as well as in lib and each file in the
> directory does include the extern.h in its own directory. So we just
> need to add it twice I guess.
The preproc code doesn't need to be thread-safe does it?
One issue we already fought with for large-file support is that that
#define _REENTRANT probably needs to appear before you start to include
any system header files. You may find that the best way to handle it
is to make it a "-D_REENTRANT" added to CPPFLAGS (only within ecpg/lib),
rather than trying to find a safe place to put it in the .h files.
regards, tom lane
On Mon, Jan 27, 2003 at 10:26:53AM -0500, Tom Lane wrote: > The preproc code doesn't need to be thread-safe does it? You're right of cause. Michael -- Michael Meskes Email: Michael@Fam-Meskes.De ICQ: 179140304 Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!