Thread: ECPG, threading and pooling

ECPG, threading and pooling

From
"Shridhar Daithankar"
Date:
Hi all,

I would like to use ECPG as it is relatively easy to code. However my 
application is multithreaded and also uses connecion pools.

I would like to know if ECPG is thread safe and I can use an arbitrary PGconn* 
object pulled from a connection pool.

Do I need to use connection name? I mean how do I store connection name in an 
array or so?

Otherwise I would have to fiddle with libpq which is bit more difficult than 
ECPG.

TIA 
Shridhar


Re: ECPG, threading and pooling

From
Michael Meskes
Date:
On Thu, Jan 23, 2003 at 02:40:33PM +0530, Shridhar Daithankar<shridhar_daithankar@persistent.co.in> wrote:
> I would like to use ECPG as it is relatively easy to code. However my 
> application is multithreaded and also uses connecion pools.

I'm afraid it needs some work to be thread-safe. sqlca is defined
statically. No big deal it seems to implement a thread safe version but
I haven't yet found the time.

> I would like to know if ECPG is thread safe and I can use an arbitrary PGconn* 
> object pulled from a connection pool.

Not sure what you mean with this?

> Do I need to use connection name? I mean how do I store connection name in an 
> array or so?

How else would you decide against whcih connection to run the statement?

Michael
-- 
Michael Meskes
Email: Michael@Fam-Meskes.De
ICQ: 179140304
Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!


Re: ECPG, threading and pooling

From
Shridhar Daithankar
Date:
On Thursday 23 January 2003 08:40 pm, you wrote:
> On Thu, Jan 23, 2003 at 02:40:33PM +0530, Shridhar 
Daithankar<shridhar_daithankar@persistent.co.in> wrote:
> > I would like to know if ECPG is thread safe and I can use an arbitrary
> > PGconn* object pulled from a connection pool.
> Not sure what you mean with this?

Well, when using libpq, I create a pool of PGconn*. What pool I create with 
ecpg? Looking at ecpglib.h, it seems that I need to create a pool of char *

> > Do I need to use connection name? I mean how do I store connection name
> > in an array or so?
> How else would you decide against whcih connection to run the statement?

That is correct but I did not find any example of how to use a particular 
connection in an SQL statement. It would be a fairly trivial guess that the 
connection name is one of the arguments to ECPGdo. But how the SQL statement 
look like, I don't have a clue. Also I don't know know how to use ECPGsetconn 
to set a connection. No documentation on that.

I can guess that ECPG maintains a mapping between connection name and PGconn* 
object for a user. If possible it would be very helpful to form a ecpg 
connection on top a user supplied PGconn* object.

I just downloaded 7.3.1 to take a look at. I hope to make a contribution if 
possible.Shridhar




Re: ECPG, threading and pooling

From
Michael Meskes
Date:
On Thu, Jan 23, 2003 at 08:58:24PM +0530, Shridhar Daithankar wrote:
> Well, when using libpq, I create a pool of PGconn*. What pool I create with 
> ecpg? Looking at ecpglib.h, it seems that I need to create a pool of char *

Yes. You use it as EXEC SQL AT :connection_var SELECT ...

> That is correct but I did not find any example of how to use a particular 
> connection in an SQL statement. It would be a fairly trivial guess that the 
> connection name is one of the arguments to ECPGdo. But how the SQL statement 
> look like, I don't have a clue. Also I don't know know how to use ECPGsetconn 
> to set a connection. No documentation on that.

You don't call either one yourself. Let ecpg do that job. Some of the
test cases in the source tree use AT. Just look under .../ecpg/test.

> I can guess that ECPG maintains a mapping between connection name and PGconn* 
> object for a user. If possible it would be very helpful to form a ecpg 
> connection on top a user supplied PGconn* object.

Why? It's possible, but you have to work with structures not meant to be
used outside the lib.

Michael
-- 
Michael Meskes
Email: Michael@Fam-Meskes.De
ICQ: 179140304
Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!


Re: ECPG, threading and pooling

From
Lee Kindness
Date:
Michael,

Michael Meskes writes:
 > On Thu, Jan 23, 2003 at 02:40:33PM +0530, Shridhar Daithankar<shridhar_daithankar@persistent.co.in> wrote:
 > > I would like to use ECPG as it is relatively easy to code. However my
 > > application is multithreaded and also uses connecion pools.
 >
 > I'm afraid it needs some work to be thread-safe. sqlca is defined
 > statically. No big deal it seems to implement a thread safe version but
 > I haven't yet found the time.

I've spent a bit of time on making ecpg thread safe over Christmas,
while it's not finished i'm sure the attached patch is at least useful
and a step in the right direction.

Lee.


Attachment

Re: ECPG, threading and pooling

From
Michael Meskes
Date:
On Mon, Jan 27, 2003 at 11:52:18AM +0000, Lee Kindness wrote:
Content-Description: message body text
> Michael,
> ...
> I've spent a bit of time on making ecpg thread safe over Christmas,
> while it's not finished i'm sure the attached patch is at least useful
> and a step in the right direction.

Thanks a lot. I have no experience in multithreaded software development
so I cannot completely check your patch but it surely looks good. Shall
I commit it, or will you? I think we should get it into CVS for all to
test.

Just two questions:
- Is it neccessary to patch c.h for ecpg?
- Not sure with the version numbering but I can fix that later.

Michael
-- 
Michael Meskes
Email: Michael@Fam-Meskes.De
ICQ: 179140304
Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!


Re: ECPG, threading and pooling

From
Lee Kindness
Date:
Michael Meskes writes:> On Mon, Jan 27, 2003 at 11:52:18AM +0000, Lee Kindness wrote:> > I've spent a bit of time on
makingecpg thread safe over Christmas,> > while it's not finished i'm sure the attached patch is at least useful> > and
astep in the right direction.> Thanks a lot. I have no experience in multithreaded software development> so I cannot
completelycheck your patch but it surely looks good. Shall> I commit it, or will you? I think we should get it into CVS
forall to> test.
 

Problem with it is it needs some changes to configure.in (and
associated files) for it to be worthwhile. Checks would need to be
added to determine is threads are supported on the build platform and
the needed compile and link flags. I'm trying to get this together for
libpq too.
> Just two questions:> - Is it neccessary to patch c.h for ecpg?

No, but each ecpg source file would need something like:
#ifdef HAVE_PTHREAD_H# include <pthread.h># include <errno.h>#endif

It's in c.h just now for ease of testing.
> - Not sure with the version numbering but I can fix that later.

As discussed on pgsql-hackers after the 7.3 release any "binary
incompatible" library change needs a major version number increase.

Lee.


Re: ECPG, threading and pooling

From
"Shridhar Daithankar"
Date:
On 27 Jan 2003 at 14:06, Lee Kindness wrote:

> Michael Meskes writes:
>  > On Mon, Jan 27, 2003 at 11:52:18AM +0000, Lee Kindness wrote:
>  > > I've spent a bit of time on making ecpg thread safe over Christmas,
>  > > while it's not finished i'm sure the attached patch is at least useful
>  > > and a step in the right direction.
>  > Thanks a lot. I have no experience in multithreaded software development
>  > so I cannot completely check your patch but it surely looks good. Shall
>  > I commit it, or will you? I think we should get it into CVS for all to
>  > test.
> 
> Problem with it is it needs some changes to configure.in (and
> associated files) for it to be worthwhile. Checks would need to be
> added to determine is threads are supported on the build platform and
> the needed compile and link flags. I'm trying to get this together for
> libpq too.

Just out of curiosity, what happens when there are more than one competing 
threading libraries? Like native threads and linuxthreads on freeBSD?

ByeShridhar

--
Reliable source, n.:    The guy you just met.



Re: ECPG, threading and pooling

From
Lee Kindness
Date:
Shridhar Daithankar writes:> On 27 Jan 2003 at 14:06, Lee Kindness wrote:> > Michael Meskes writes:> >  > Thanks a lot.
Ihave no experience in multithreaded software development> >  > so I cannot completely check your patch but it surely
looksgood. Shall> >  > I commit it, or will you? I think we should get it into CVS for all to> >  > test.> > Problem
withit is it needs some changes to configure.in (and> > associated files) for it to be worthwhile. Checks would need to
be>> added to determine is threads are supported on the build platform and> > the needed compile and link flags. I'm
tryingto get this together for> > libpq too.> Just out of curiosity, what happens when there are more than one
competing> threading libraries? Like native threads and linuxthreads on freeBSD?
 

It's all down to the checks in configure... Looking at many packages
which have threads check in configure (e.g. openldap, mysql, mozilla,
glib, ...) some let the user specify which theading library to use
(e.g --with-threads=linuxthreads (or something)) while others just
seem to pick up the first available...

To be honest I am quite daunted by the checks needed (every other
package seems to be doing the check differently!) and any/all help
would be welcome!

L.


Re: ECPG, threading and pooling

From
Tom Lane
Date:
Michael Meskes <meskes@postgresql.org> writes:
> - Is it neccessary to patch c.h for ecpg?

If you commit that part, it will be reverted forthwith (especially the
hardwired #define HAVE_PTHREAD_H ;-)).

Keep the thread hacking on the client side, please.  Isn't there one of
the ecpg/include/ files that would be suitable?
        regards, tom lane


Re: ECPG, threading and pooling

From
Michael Meskes
Date:
On Mon, Jan 27, 2003 at 02:06:26PM +0000, Lee Kindness wrote:
>  > - Not sure with the version numbering but I can fix that later.
> 
> As discussed on pgsql-hackers after the 7.3 release any "binary
> incompatible" library change needs a major version number increase.

Sorry, I was not precise enough. The major number has to be inceased,
it's the minor numbers that need fixing. :-)

Michael
-- 
Michael Meskes
Email: Michael@Fam-Meskes.De
ICQ: 179140304
Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!


Re: ECPG, threading and pooling

From
Michael Meskes
Date:
On Mon, Jan 27, 2003 at 09:30:50AM -0500, Tom Lane wrote:
> Michael Meskes <meskes@postgresql.org> writes:
> > - Is it neccessary to patch c.h for ecpg?
> 
> If you commit that part, it will be reverted forthwith (especially the
> hardwired #define HAVE_PTHREAD_H ;-)).
> 
> Keep the thread hacking on the client side, please.  Isn't there one of
> the ecpg/include/ files that would be suitable?

Yes, there is. Better make that there are. There is a file called
extern.h both in preproc as well as in lib and each file in the
directory does include the extern.h in its own directory. So we just
need to add it twice I guess.

Michael
-- 
Michael Meskes
Email: Michael@Fam-Meskes.De
ICQ: 179140304
Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!


Re: ECPG, threading and pooling

From
Tom Lane
Date:
Michael Meskes <meskes@postgresql.org> writes:
>> Keep the thread hacking on the client side, please.  Isn't there one of
>> the ecpg/include/ files that would be suitable?

> Yes, there is. Better make that there are. There is a file called
> extern.h both in preproc as well as in lib and each file in the
> directory does include the extern.h in its own directory. So we just
> need to add it twice I guess.

The preproc code doesn't need to be thread-safe does it?

One issue we already fought with for large-file support is that that
#define _REENTRANT probably needs to appear before you start to include
any system header files.  You may find that the best way to handle it
is to make it a "-D_REENTRANT" added to CPPFLAGS (only within ecpg/lib),
rather than trying to find a safe place to put it in the .h files.
        regards, tom lane


Re: ECPG, threading and pooling

From
Michael Meskes
Date:
On Mon, Jan 27, 2003 at 10:26:53AM -0500, Tom Lane wrote:
> The preproc code doesn't need to be thread-safe does it?

You're right of cause.

Michael
-- 
Michael Meskes
Email: Michael@Fam-Meskes.De
ICQ: 179140304
Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!