Thread: libpq + multiple connections ...

libpq + multiple connections ...

From
The Hermit Hacker
Date:
Anyone able to comment on this?  I haven' tused libpq in so long that I
can't :(

======

Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
function to libpq and I stumbled across some problems.

It seems that libpq makes use of some static variables, meaning i'm not
sure if it's safe to use libpq for multiple database connections.

What i'm refering to is:

postgresql-6.5.3/src/interfaces/libpq/fe-connect.c

line 79 has a structure that seems to be shared amongst the entire
library, am I likely to stumble upon more stuff that makes it somewhat
dangerous to have more than one active database connection in my program?

=======

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 



Re: [INTERFACES] libpq + multiple connections ...

From
Tom Lane
Date:
The Hermit Hacker <scrappy@hub.org> forwards:
> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
> function to libpq and I stumbled across some problems.

Someone else was already working on that --- check the archives from
a few months back.

> It seems that libpq makes use of some static variables, meaning i'm not
> sure if it's safe to use libpq for multiple database connections.
> What i'm refering to is:
> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
> line 79 has a structure that seems to be shared amongst the entire
> library, am I likely to stumble upon more stuff that makes it somewhat
> dangerous to have more than one active database connection in my program?

The PQconnectdb() function uses that static array, meaning that you can't
safely run two PQconnectdb() calls in parallel.  But you can open two
connections in sequence and then use them in parallel; and you can do
the opens in parallel if you use one of the older, less-friendly
connection-opening calls.  There aren't any other non-constant statics
in libpq AFAIR.

The static array sucks, I agree, but I don't see any way to get rid of
it without changing libpq's API for PQconnectdb() and PQconndefaults().
Do we want to consider doing that (and breaking some apps) for 7.0?
        regards, tom lane


Re: [INTERFACES] libpq + multiple connections ...

From
The Hermit Hacker
Date:
On Mon, 29 Nov 1999, Tom Lane wrote:

> The Hermit Hacker <scrappy@hub.org> forwards:
> > Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
> > function to libpq and I stumbled across some problems.
> 
> Someone else was already working on that --- check the archives from
> a few months back.
> 
> > It seems that libpq makes use of some static variables, meaning i'm not
> > sure if it's safe to use libpq for multiple database connections.
> > What i'm refering to is:
> > postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
> > line 79 has a structure that seems to be shared amongst the entire
> > library, am I likely to stumble upon more stuff that makes it somewhat
> > dangerous to have more than one active database connection in my program?
> 
> The PQconnectdb() function uses that static array, meaning that you can't
> safely run two PQconnectdb() calls in parallel.  But you can open two
> connections in sequence and then use them in parallel; and you can do
> the opens in parallel if you use one of the older, less-friendly
> connection-opening calls.  There aren't any other non-constant statics
> in libpq AFAIR.
> 
> The static array sucks, I agree, but I don't see any way to get rid of
> it without changing libpq's API for PQconnectdb() and PQconndefaults().
> Do we want to consider doing that (and breaking some apps) for 7.0?

IMHO...7.0 let's us pretty much go hog-wild as far as API changes are
concerned...if it is something that a) can be done and b) shoudl be
done...now is about the best time to do it...



Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 



database GUI library for C,C++

From
Madarasz Karoly
Date:
Hello!

I try to make a PostgreSQL based GUI aplication.
I find a lot of GUI libraries, but none of them
have 'high level' classes for accessing and displaying
dates from databases in a more convenient mode.
(like TTable, TDBGrid in Delphi)
Is there something like that?

Thanks,
Karesz.


unsubscribe

From
Michael Katkin
Date:


Re: [INTERFACES] libpq + multiple connections ...

From
"E.E. Mellor"
Date:
--On 29 November 1999, 02:45 -0500 Tom Lane <tgl@sss.pgh.pa.us> wrote:

> The Hermit Hacker <scrappy@hub.org> forwards:
>> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
>> function to libpq and I stumbled across some problems.
> 
> Someone else was already working on that --- check the archives from
> a few months back.

'twas me.  In fact, it's been ready for ages, but I've had so much on that
I've not had time to write the docs.  I'll do them today for you.

>> It seems that libpq makes use of some static variables, meaning i'm not
>> sure if it's safe to use libpq for multiple database connections.
>> What i'm refering to is:
>> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
>> line 79 has a structure that seems to be shared amongst the entire
>> library, am I likely to stumble upon more stuff that makes it somewhat
>> dangerous to have more than one active database connection in my program?
> 
> The PQconnectdb() function uses that static array, meaning that you can't
> safely run two PQconnectdb() calls in parallel.  But you can open two
> connections in sequence and then use them in parallel; and you can do
> the opens in parallel if you use one of the older, less-friendly
> connection-opening calls.  There aren't any other non-constant statics
> in libpq AFAIR.

This is correct.  Someone else was looking at thread-safety across the
whole of libpq, but as far as I'm aware, it's not _guaranteed_ to be
thread-safe anywhere.  My changes allow one to use libpq in a single
thread, but with multiple connections and without blocking the thread at
all.

> The static array sucks, I agree, but I don't see any way to get rid of
> it without changing libpq's API for PQconnectdb() and PQconndefaults().
> Do we want to consider doing that (and breaking some apps) for 7.0?

I think it should be possible to arrange it so that these functions still
exist (but are deprecated and marked as non-thread-safe).  I'll leave that
for others to decide though.

Ewan Mellor.


Re: [INTERFACES] libpq + multiple connections ...

From
Roberto Joao Lopes Garcia
Date:
>> safely run two PQconnectdb() calls in parallel.  But you can open two
>> connections in sequence and then use them in parallel; 
>

So I could open, for exemple, 5 db connections (with PQconnectdb()) and
then create 5 threads and use those connections in eache thread. But how
about use PQreset() or PQfinish() inside the new created threads?

I used to test connection before send a query like bellow:

if(PQstatus(con)==CONNECTION_BAD) {  PQreset(con);  if(PQstatus(con)==CONNECTION_BAD) erro_fatal();
}

Should I protect the calls to those functions by an mutex?? Could it
work??? Which functions read or alter Qconnectdb()'s  static array??

I think it is important to mark wich  functions in Libpq is thread safe or
not.

Roberto




Re: [INTERFACES] libpq + multiple connections ...

From
The Hermit Hacker
Date:
Is this something that could *safely* be back-patched to a v6.5.4 release?

On Mon, 29 Nov 1999, E.E. Mellor wrote:

> --On 29 November 1999, 02:45 -0500 Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> > The Hermit Hacker <scrappy@hub.org> forwards:
> >> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
> >> function to libpq and I stumbled across some problems.
> > 
> > Someone else was already working on that --- check the archives from
> > a few months back.
> 
> 'twas me.  In fact, it's been ready for ages, but I've had so much on that
> I've not had time to write the docs.  I'll do them today for you.
> 
> >> It seems that libpq makes use of some static variables, meaning i'm not
> >> sure if it's safe to use libpq for multiple database connections.
> >> What i'm refering to is:
> >> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
> >> line 79 has a structure that seems to be shared amongst the entire
> >> library, am I likely to stumble upon more stuff that makes it somewhat
> >> dangerous to have more than one active database connection in my program?
> > 
> > The PQconnectdb() function uses that static array, meaning that you can't
> > safely run two PQconnectdb() calls in parallel.  But you can open two
> > connections in sequence and then use them in parallel; and you can do
> > the opens in parallel if you use one of the older, less-friendly
> > connection-opening calls.  There aren't any other non-constant statics
> > in libpq AFAIR.
> 
> This is correct.  Someone else was looking at thread-safety across the
> whole of libpq, but as far as I'm aware, it's not _guaranteed_ to be
> thread-safe anywhere.  My changes allow one to use libpq in a single
> thread, but with multiple connections and without blocking the thread at
> all.
> 
> > The static array sucks, I agree, but I don't see any way to get rid of
> > it without changing libpq's API for PQconnectdb() and PQconndefaults().
> > Do we want to consider doing that (and breaking some apps) for 7.0?
> 
> I think it should be possible to arrange it so that these functions still
> exist (but are deprecated and marked as non-thread-safe).  I'll leave that
> for others to decide though.
> 
> Ewan Mellor.
> 

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 


************




Re: [INTERFACES] libpq + multiple connections ...

From
eem21@cam.ac.uk
Date:
On 29 Nov, The Hermit Hacker wrote:
> 
> Is this something that could *safely* be back-patched to a v6.5.4 release?

Well, it's not _supposed_ to have changed the behaviour for existing
functions.  However, there's been quite a lot of chopping and changing,
and I might have lost something somewhere.  That said, it would be of
great help to me if it made it into a 6.5.x release, and I would thus
be willing to test it.  I can only test ix86/Linux, unfortunately. 
When would it be released, if that were the case?

BTW, how do I find out about the SGML tags we are using (I've never
written any SGML before, only HTML).

Ewan.

> 
> On Mon, 29 Nov 1999, E.E. Mellor wrote:
> 
>> --On 29 November 1999, 02:45 -0500 Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> 
>> > The Hermit Hacker <scrappy@hub.org> forwards:
>> >> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
>> >> function to libpq and I stumbled across some problems.
>> > 
>> > Someone else was already working on that --- check the archives from
>> > a few months back.
>> 
>> 'twas me.  In fact, it's been ready for ages, but I've had so much on that
>> I've not had time to write the docs.  I'll do them today for you.
>> 
>> >> It seems that libpq makes use of some static variables, meaning i'm not
>> >> sure if it's safe to use libpq for multiple database connections.
>> >> What i'm refering to is:
>> >> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
>> >> line 79 has a structure that seems to be shared amongst the entire
>> >> library, am I likely to stumble upon more stuff that makes it somewhat
>> >> dangerous to have more than one active database connection in my program?
>> > 
>> > The PQconnectdb() function uses that static array, meaning that you can't
>> > safely run two PQconnectdb() calls in parallel.  But you can open two
>> > connections in sequence and then use them in parallel; and you can do
>> > the opens in parallel if you use one of the older, less-friendly
>> > connection-opening calls.  There aren't any other non-constant statics
>> > in libpq AFAIR.
>> 
>> This is correct.  Someone else was looking at thread-safety across the
>> whole of libpq, but as far as I'm aware, it's not _guaranteed_ to be
>> thread-safe anywhere.  My changes allow one to use libpq in a single
>> thread, but with multiple connections and without blocking the thread at
>> all.
>> 
>> > The static array sucks, I agree, but I don't see any way to get rid of
>> > it without changing libpq's API for PQconnectdb() and PQconndefaults().
>> > Do we want to consider doing that (and breaking some apps) for 7.0?
>> 
>> I think it should be possible to arrange it so that these functions still
>> exist (but are deprecated and marked as non-thread-safe).  I'll leave that
>> for others to decide though.
>> 
>> Ewan Mellor.
>> 
> 
> Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
> Systems Administrator @ hub.org 
> primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 
> 



************




Re: [INTERFACES] libpq + multiple connections ...

From
The Hermit Hacker
Date:
On Mon, 29 Nov 1999 eem21@cam.ac.uk wrote:

> On 29 Nov, The Hermit Hacker wrote:
> > 
> > Is this something that could *safely* be back-patched to a v6.5.4 release?
> 
> Well, it's not _supposed_ to have changed the behaviour for existing
> functions.  However, there's been quite a lot of chopping and changing,
> and I might have lost something somewhere.  That said, it would be of
> great help to me if it made it into a 6.5.x release, and I would thus
> be willing to test it.  I can only test ix86/Linux, unfortunately. 
> When would it be released, if that were the case?

If you can get a patch in so that we can test it, would love to get a
release out by ... oh ... end of week? :)

Anyone object overly?

> BTW, how do I find out about the SGML tags we are using (I've never
> written any SGML before, only HTML).
> 
> Ewan.
> 
> > 
> > On Mon, 29 Nov 1999, E.E. Mellor wrote:
> > 
> >> --On 29 November 1999, 02:45 -0500 Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> 
> >> > The Hermit Hacker <scrappy@hub.org> forwards:
> >> >> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
> >> >> function to libpq and I stumbled across some problems.
> >> > 
> >> > Someone else was already working on that --- check the archives from
> >> > a few months back.
> >> 
> >> 'twas me.  In fact, it's been ready for ages, but I've had so much on that
> >> I've not had time to write the docs.  I'll do them today for you.
> >> 
> >> >> It seems that libpq makes use of some static variables, meaning i'm not
> >> >> sure if it's safe to use libpq for multiple database connections.
> >> >> What i'm refering to is:
> >> >> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
> >> >> line 79 has a structure that seems to be shared amongst the entire
> >> >> library, am I likely to stumble upon more stuff that makes it somewhat
> >> >> dangerous to have more than one active database connection in my program?
> >> > 
> >> > The PQconnectdb() function uses that static array, meaning that you can't
> >> > safely run two PQconnectdb() calls in parallel.  But you can open two
> >> > connections in sequence and then use them in parallel; and you can do
> >> > the opens in parallel if you use one of the older, less-friendly
> >> > connection-opening calls.  There aren't any other non-constant statics
> >> > in libpq AFAIR.
> >> 
> >> This is correct.  Someone else was looking at thread-safety across the
> >> whole of libpq, but as far as I'm aware, it's not _guaranteed_ to be
> >> thread-safe anywhere.  My changes allow one to use libpq in a single
> >> thread, but with multiple connections and without blocking the thread at
> >> all.
> >> 
> >> > The static array sucks, I agree, but I don't see any way to get rid of
> >> > it without changing libpq's API for PQconnectdb() and PQconndefaults().
> >> > Do we want to consider doing that (and breaking some apps) for 7.0?
> >> 
> >> I think it should be possible to arrange it so that these functions still
> >> exist (but are deprecated and marked as non-thread-safe).  I'll leave that
> >> for others to decide though.
> >> 
> >> Ewan Mellor.
> >> 
> > 
> > Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
> > Systems Administrator @ hub.org 
> > primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 
> > 
> 
> 

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 


************




Re: [INTERFACES] libpq + multiple connections ...

From
eem21@cam.ac.uk
Date:
On 29 Nov, The Hermit Hacker wrote:
> On Mon, 29 Nov 1999 eem21@cam.ac.uk wrote:
> 
>> On 29 Nov, The Hermit Hacker wrote:
>> > 
>> > Is this something that could *safely* be back-patched to a v6.5.4 release?
>> 
>> Well, it's not _supposed_ to have changed the behaviour for existing
>> functions.  However, there's been quite a lot of chopping and changing,
>> and I might have lost something somewhere.  That said, it would be of
>> great help to me if it made it into a 6.5.x release, and I would thus
>> be willing to test it.  I can only test ix86/Linux, unfortunately. 
>> When would it be released, if that were the case?
> 
> If you can get a patch in so that we can test it

I'm doin' it, I'm doin' it... ;-)

> would love to get a release out by ... oh ... end of week? :)
> 
> Anyone object overly?

Nope.

> [Snip async functions]

Ewan.



************




Re: [INTERFACES] libpq + multiple connections ...

From
eem21@cam.ac.uk
Date:
On 29 Nov, The Hermit Hacker wrote:
> On Mon, 29 Nov 1999 eem21@cam.ac.uk wrote:
> 
>> On 29 Nov, The Hermit Hacker wrote:
>> > 
>> > Is this something that could *safely* be back-patched to a v6.5.4 release?
>> 
>> Well, it's not _supposed_ to have changed the behaviour for existing
>> functions.  However, there's been quite a lot of chopping and changing,
>> and I might have lost something somewhere.  That said, it would be of
>> great help to me if it made it into a 6.5.x release, and I would thus
>> be willing to test it.  I can only test ix86/Linux, unfortunately. 
>> When would it be released, if that were the case?
> 
> If you can get a patch in so that we can test it, would love to get a
> release out by ... oh ... end of week? :)

Patch submitted.

> [Snip]

Ewan.


************




Re: [INTERFACES] libpq + multiple connections ...

From
eem21@cam.ac.uk
Date:
On 29 Nov, The Hermit Hacker wrote:
> On Mon, 29 Nov 1999 eem21@cam.ac.uk wrote:
> 
>> On 29 Nov, The Hermit Hacker wrote:
>> > 
>> > Is this something that could *safely* be back-patched to a v6.5.4 release?
>> 
>> Well, it's not _supposed_ to have changed the behaviour for existing
>> functions.  However, there's been quite a lot of chopping and changing,
>> and I might have lost something somewhere.  That said, it would be of
>> great help to me if it made it into a 6.5.x release, and I would thus
>> be willing to test it.  I can only test ix86/Linux, unfortunately. 
>> When would it be released, if that were the case?
> 
> If you can get a patch in so that we can test it

I'm doin' it, I'm doin' it... ;-)

> would love to get a release out by ... oh ... end of week? :)
> 
> Anyone object overly?

Nope.

> [Snip async functions]

Ewan.



************




Re: [INTERFACES] libpq + multiple connections ...

From
The Hermit Hacker
Date:
Is this something that could *safely* be back-patched to a v6.5.4 release?

On Mon, 29 Nov 1999, E.E. Mellor wrote:

> --On 29 November 1999, 02:45 -0500 Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> > The Hermit Hacker <scrappy@hub.org> forwards:
> >> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
> >> function to libpq and I stumbled across some problems.
> > 
> > Someone else was already working on that --- check the archives from
> > a few months back.
> 
> 'twas me.  In fact, it's been ready for ages, but I've had so much on that
> I've not had time to write the docs.  I'll do them today for you.
> 
> >> It seems that libpq makes use of some static variables, meaning i'm not
> >> sure if it's safe to use libpq for multiple database connections.
> >> What i'm refering to is:
> >> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
> >> line 79 has a structure that seems to be shared amongst the entire
> >> library, am I likely to stumble upon more stuff that makes it somewhat
> >> dangerous to have more than one active database connection in my program?
> > 
> > The PQconnectdb() function uses that static array, meaning that you can't
> > safely run two PQconnectdb() calls in parallel.  But you can open two
> > connections in sequence and then use them in parallel; and you can do
> > the opens in parallel if you use one of the older, less-friendly
> > connection-opening calls.  There aren't any other non-constant statics
> > in libpq AFAIR.
> 
> This is correct.  Someone else was looking at thread-safety across the
> whole of libpq, but as far as I'm aware, it's not _guaranteed_ to be
> thread-safe anywhere.  My changes allow one to use libpq in a single
> thread, but with multiple connections and without blocking the thread at
> all.
> 
> > The static array sucks, I agree, but I don't see any way to get rid of
> > it without changing libpq's API for PQconnectdb() and PQconndefaults().
> > Do we want to consider doing that (and breaking some apps) for 7.0?
> 
> I think it should be possible to arrange it so that these functions still
> exist (but are deprecated and marked as non-thread-safe).  I'll leave that
> for others to decide though.
> 
> Ewan Mellor.
> 

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 


************




Re: [INTERFACES] libpq + multiple connections ...

From
eem21@cam.ac.uk
Date:
On 29 Nov, The Hermit Hacker wrote:
> 
> Is this something that could *safely* be back-patched to a v6.5.4 release?

Well, it's not _supposed_ to have changed the behaviour for existing
functions.  However, there's been quite a lot of chopping and changing,
and I might have lost something somewhere.  That said, it would be of
great help to me if it made it into a 6.5.x release, and I would thus
be willing to test it.  I can only test ix86/Linux, unfortunately. 
When would it be released, if that were the case?

BTW, how do I find out about the SGML tags we are using (I've never
written any SGML before, only HTML).

Ewan.

> 
> On Mon, 29 Nov 1999, E.E. Mellor wrote:
> 
>> --On 29 November 1999, 02:45 -0500 Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> 
>> > The Hermit Hacker <scrappy@hub.org> forwards:
>> >> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
>> >> function to libpq and I stumbled across some problems.
>> > 
>> > Someone else was already working on that --- check the archives from
>> > a few months back.
>> 
>> 'twas me.  In fact, it's been ready for ages, but I've had so much on that
>> I've not had time to write the docs.  I'll do them today for you.
>> 
>> >> It seems that libpq makes use of some static variables, meaning i'm not
>> >> sure if it's safe to use libpq for multiple database connections.
>> >> What i'm refering to is:
>> >> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
>> >> line 79 has a structure that seems to be shared amongst the entire
>> >> library, am I likely to stumble upon more stuff that makes it somewhat
>> >> dangerous to have more than one active database connection in my program?
>> > 
>> > The PQconnectdb() function uses that static array, meaning that you can't
>> > safely run two PQconnectdb() calls in parallel.  But you can open two
>> > connections in sequence and then use them in parallel; and you can do
>> > the opens in parallel if you use one of the older, less-friendly
>> > connection-opening calls.  There aren't any other non-constant statics
>> > in libpq AFAIR.
>> 
>> This is correct.  Someone else was looking at thread-safety across the
>> whole of libpq, but as far as I'm aware, it's not _guaranteed_ to be
>> thread-safe anywhere.  My changes allow one to use libpq in a single
>> thread, but with multiple connections and without blocking the thread at
>> all.
>> 
>> > The static array sucks, I agree, but I don't see any way to get rid of
>> > it without changing libpq's API for PQconnectdb() and PQconndefaults().
>> > Do we want to consider doing that (and breaking some apps) for 7.0?
>> 
>> I think it should be possible to arrange it so that these functions still
>> exist (but are deprecated and marked as non-thread-safe).  I'll leave that
>> for others to decide though.
>> 
>> Ewan Mellor.
>> 
> 
> Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
> Systems Administrator @ hub.org 
> primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 
> 



************




Re: [INTERFACES] libpq + multiple connections ...

From
The Hermit Hacker
Date:
On Mon, 29 Nov 1999 eem21@cam.ac.uk wrote:

> On 29 Nov, The Hermit Hacker wrote:
> > 
> > Is this something that could *safely* be back-patched to a v6.5.4 release?
> 
> Well, it's not _supposed_ to have changed the behaviour for existing
> functions.  However, there's been quite a lot of chopping and changing,
> and I might have lost something somewhere.  That said, it would be of
> great help to me if it made it into a 6.5.x release, and I would thus
> be willing to test it.  I can only test ix86/Linux, unfortunately. 
> When would it be released, if that were the case?

If you can get a patch in so that we can test it, would love to get a
release out by ... oh ... end of week? :)

Anyone object overly?

> BTW, how do I find out about the SGML tags we are using (I've never
> written any SGML before, only HTML).
> 
> Ewan.
> 
> > 
> > On Mon, 29 Nov 1999, E.E. Mellor wrote:
> > 
> >> --On 29 November 1999, 02:45 -0500 Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> 
> >> > The Hermit Hacker <scrappy@hub.org> forwards:
> >> >> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
> >> >> function to libpq and I stumbled across some problems.
> >> > 
> >> > Someone else was already working on that --- check the archives from
> >> > a few months back.
> >> 
> >> 'twas me.  In fact, it's been ready for ages, but I've had so much on that
> >> I've not had time to write the docs.  I'll do them today for you.
> >> 
> >> >> It seems that libpq makes use of some static variables, meaning i'm not
> >> >> sure if it's safe to use libpq for multiple database connections.
> >> >> What i'm refering to is:
> >> >> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
> >> >> line 79 has a structure that seems to be shared amongst the entire
> >> >> library, am I likely to stumble upon more stuff that makes it somewhat
> >> >> dangerous to have more than one active database connection in my program?
> >> > 
> >> > The PQconnectdb() function uses that static array, meaning that you can't
> >> > safely run two PQconnectdb() calls in parallel.  But you can open two
> >> > connections in sequence and then use them in parallel; and you can do
> >> > the opens in parallel if you use one of the older, less-friendly
> >> > connection-opening calls.  There aren't any other non-constant statics
> >> > in libpq AFAIR.
> >> 
> >> This is correct.  Someone else was looking at thread-safety across the
> >> whole of libpq, but as far as I'm aware, it's not _guaranteed_ to be
> >> thread-safe anywhere.  My changes allow one to use libpq in a single
> >> thread, but with multiple connections and without blocking the thread at
> >> all.
> >> 
> >> > The static array sucks, I agree, but I don't see any way to get rid of
> >> > it without changing libpq's API for PQconnectdb() and PQconndefaults().
> >> > Do we want to consider doing that (and breaking some apps) for 7.0?
> >> 
> >> I think it should be possible to arrange it so that these functions still
> >> exist (but are deprecated and marked as non-thread-safe).  I'll leave that
> >> for others to decide though.
> >> 
> >> Ewan Mellor.
> >> 
> > 
> > Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
> > Systems Administrator @ hub.org 
> > primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 
> > 
> 
> 

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 


************




Re: [INTERFACES] libpq + multiple connections ...

From
eem21@cam.ac.uk
Date:
On 29 Nov, The Hermit Hacker wrote:
> On Mon, 29 Nov 1999 eem21@cam.ac.uk wrote:
> 
>> On 29 Nov, The Hermit Hacker wrote:
>> > 
>> > Is this something that could *safely* be back-patched to a v6.5.4 release?
>> 
>> Well, it's not _supposed_ to have changed the behaviour for existing
>> functions.  However, there's been quite a lot of chopping and changing,
>> and I might have lost something somewhere.  That said, it would be of
>> great help to me if it made it into a 6.5.x release, and I would thus
>> be willing to test it.  I can only test ix86/Linux, unfortunately. 
>> When would it be released, if that were the case?
> 
> If you can get a patch in so that we can test it, would love to get a
> release out by ... oh ... end of week? :)

Patch submitted.

> [Snip]

Ewan.


************




Re: [INTERFACES] libpq + multiple connections ...

From
eem21@cam.ac.uk
Date:
On 29 Nov, The Hermit Hacker wrote:
> On Mon, 29 Nov 1999 eem21@cam.ac.uk wrote:
> 
>> On 29 Nov, The Hermit Hacker wrote:
>> > 
>> > Is this something that could *safely* be back-patched to a v6.5.4 release?
>> 
>> Well, it's not _supposed_ to have changed the behaviour for existing
>> functions.  However, there's been quite a lot of chopping and changing,
>> and I might have lost something somewhere.  That said, it would be of
>> great help to me if it made it into a 6.5.x release, and I would thus
>> be willing to test it.  I can only test ix86/Linux, unfortunately. 
>> When would it be released, if that were the case?
> 
> If you can get a patch in so that we can test it, would love to get a
> release out by ... oh ... end of week? :)

Patch submitted.

> [Snip]

Ewan.


************




Re: [INTERFACES] libpq + multiple connections ...

From
eem21@cam.ac.uk
Date:
On 29 Nov, The Hermit Hacker wrote:
> On Mon, 29 Nov 1999 eem21@cam.ac.uk wrote:
> 
>> On 29 Nov, The Hermit Hacker wrote:
>> > 
>> > Is this something that could *safely* be back-patched to a v6.5.4 release?
>> 
>> Well, it's not _supposed_ to have changed the behaviour for existing
>> functions.  However, there's been quite a lot of chopping and changing,
>> and I might have lost something somewhere.  That said, it would be of
>> great help to me if it made it into a 6.5.x release, and I would thus
>> be willing to test it.  I can only test ix86/Linux, unfortunately. 
>> When would it be released, if that were the case?
> 
> If you can get a patch in so that we can test it

I'm doin' it, I'm doin' it... ;-)

> would love to get a release out by ... oh ... end of week? :)
> 
> Anyone object overly?

Nope.

> [Snip async functions]

Ewan.



************




Re: [INTERFACES] libpq + multiple connections ...

From
The Hermit Hacker
Date:
Is this something that could *safely* be back-patched to a v6.5.4 release?

On Mon, 29 Nov 1999, E.E. Mellor wrote:

> --On 29 November 1999, 02:45 -0500 Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> > The Hermit Hacker <scrappy@hub.org> forwards:
> >> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
> >> function to libpq and I stumbled across some problems.
> > 
> > Someone else was already working on that --- check the archives from
> > a few months back.
> 
> 'twas me.  In fact, it's been ready for ages, but I've had so much on that
> I've not had time to write the docs.  I'll do them today for you.
> 
> >> It seems that libpq makes use of some static variables, meaning i'm not
> >> sure if it's safe to use libpq for multiple database connections.
> >> What i'm refering to is:
> >> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
> >> line 79 has a structure that seems to be shared amongst the entire
> >> library, am I likely to stumble upon more stuff that makes it somewhat
> >> dangerous to have more than one active database connection in my program?
> > 
> > The PQconnectdb() function uses that static array, meaning that you can't
> > safely run two PQconnectdb() calls in parallel.  But you can open two
> > connections in sequence and then use them in parallel; and you can do
> > the opens in parallel if you use one of the older, less-friendly
> > connection-opening calls.  There aren't any other non-constant statics
> > in libpq AFAIR.
> 
> This is correct.  Someone else was looking at thread-safety across the
> whole of libpq, but as far as I'm aware, it's not _guaranteed_ to be
> thread-safe anywhere.  My changes allow one to use libpq in a single
> thread, but with multiple connections and without blocking the thread at
> all.
> 
> > The static array sucks, I agree, but I don't see any way to get rid of
> > it without changing libpq's API for PQconnectdb() and PQconndefaults().
> > Do we want to consider doing that (and breaking some apps) for 7.0?
> 
> I think it should be possible to arrange it so that these functions still
> exist (but are deprecated and marked as non-thread-safe).  I'll leave that
> for others to decide though.
> 
> Ewan Mellor.
> 

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 


************




Re: [INTERFACES] libpq + multiple connections ...

From
eem21@cam.ac.uk
Date:
On 29 Nov, The Hermit Hacker wrote:
> 
> Is this something that could *safely* be back-patched to a v6.5.4 release?

Well, it's not _supposed_ to have changed the behaviour for existing
functions.  However, there's been quite a lot of chopping and changing,
and I might have lost something somewhere.  That said, it would be of
great help to me if it made it into a 6.5.x release, and I would thus
be willing to test it.  I can only test ix86/Linux, unfortunately. 
When would it be released, if that were the case?

BTW, how do I find out about the SGML tags we are using (I've never
written any SGML before, only HTML).

Ewan.

> 
> On Mon, 29 Nov 1999, E.E. Mellor wrote:
> 
>> --On 29 November 1999, 02:45 -0500 Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> 
>> > The Hermit Hacker <scrappy@hub.org> forwards:
>> >> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
>> >> function to libpq and I stumbled across some problems.
>> > 
>> > Someone else was already working on that --- check the archives from
>> > a few months back.
>> 
>> 'twas me.  In fact, it's been ready for ages, but I've had so much on that
>> I've not had time to write the docs.  I'll do them today for you.
>> 
>> >> It seems that libpq makes use of some static variables, meaning i'm not
>> >> sure if it's safe to use libpq for multiple database connections.
>> >> What i'm refering to is:
>> >> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
>> >> line 79 has a structure that seems to be shared amongst the entire
>> >> library, am I likely to stumble upon more stuff that makes it somewhat
>> >> dangerous to have more than one active database connection in my program?
>> > 
>> > The PQconnectdb() function uses that static array, meaning that you can't
>> > safely run two PQconnectdb() calls in parallel.  But you can open two
>> > connections in sequence and then use them in parallel; and you can do
>> > the opens in parallel if you use one of the older, less-friendly
>> > connection-opening calls.  There aren't any other non-constant statics
>> > in libpq AFAIR.
>> 
>> This is correct.  Someone else was looking at thread-safety across the
>> whole of libpq, but as far as I'm aware, it's not _guaranteed_ to be
>> thread-safe anywhere.  My changes allow one to use libpq in a single
>> thread, but with multiple connections and without blocking the thread at
>> all.
>> 
>> > The static array sucks, I agree, but I don't see any way to get rid of
>> > it without changing libpq's API for PQconnectdb() and PQconndefaults().
>> > Do we want to consider doing that (and breaking some apps) for 7.0?
>> 
>> I think it should be possible to arrange it so that these functions still
>> exist (but are deprecated and marked as non-thread-safe).  I'll leave that
>> for others to decide though.
>> 
>> Ewan Mellor.
>> 
> 
> Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
> Systems Administrator @ hub.org 
> primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 
> 



************




Re: [INTERFACES] libpq + multiple connections ...

From
The Hermit Hacker
Date:
On Mon, 29 Nov 1999 eem21@cam.ac.uk wrote:

> On 29 Nov, The Hermit Hacker wrote:
> > 
> > Is this something that could *safely* be back-patched to a v6.5.4 release?
> 
> Well, it's not _supposed_ to have changed the behaviour for existing
> functions.  However, there's been quite a lot of chopping and changing,
> and I might have lost something somewhere.  That said, it would be of
> great help to me if it made it into a 6.5.x release, and I would thus
> be willing to test it.  I can only test ix86/Linux, unfortunately. 
> When would it be released, if that were the case?

If you can get a patch in so that we can test it, would love to get a
release out by ... oh ... end of week? :)

Anyone object overly?

> BTW, how do I find out about the SGML tags we are using (I've never
> written any SGML before, only HTML).
> 
> Ewan.
> 
> > 
> > On Mon, 29 Nov 1999, E.E. Mellor wrote:
> > 
> >> --On 29 November 1999, 02:45 -0500 Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> 
> >> > The Hermit Hacker <scrappy@hub.org> forwards:
> >> >> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
> >> >> function to libpq and I stumbled across some problems.
> >> > 
> >> > Someone else was already working on that --- check the archives from
> >> > a few months back.
> >> 
> >> 'twas me.  In fact, it's been ready for ages, but I've had so much on that
> >> I've not had time to write the docs.  I'll do them today for you.
> >> 
> >> >> It seems that libpq makes use of some static variables, meaning i'm not
> >> >> sure if it's safe to use libpq for multiple database connections.
> >> >> What i'm refering to is:
> >> >> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
> >> >> line 79 has a structure that seems to be shared amongst the entire
> >> >> library, am I likely to stumble upon more stuff that makes it somewhat
> >> >> dangerous to have more than one active database connection in my program?
> >> > 
> >> > The PQconnectdb() function uses that static array, meaning that you can't
> >> > safely run two PQconnectdb() calls in parallel.  But you can open two
> >> > connections in sequence and then use them in parallel; and you can do
> >> > the opens in parallel if you use one of the older, less-friendly
> >> > connection-opening calls.  There aren't any other non-constant statics
> >> > in libpq AFAIR.
> >> 
> >> This is correct.  Someone else was looking at thread-safety across the
> >> whole of libpq, but as far as I'm aware, it's not _guaranteed_ to be
> >> thread-safe anywhere.  My changes allow one to use libpq in a single
> >> thread, but with multiple connections and without blocking the thread at
> >> all.
> >> 
> >> > The static array sucks, I agree, but I don't see any way to get rid of
> >> > it without changing libpq's API for PQconnectdb() and PQconndefaults().
> >> > Do we want to consider doing that (and breaking some apps) for 7.0?
> >> 
> >> I think it should be possible to arrange it so that these functions still
> >> exist (but are deprecated and marked as non-thread-safe).  I'll leave that
> >> for others to decide though.
> >> 
> >> Ewan Mellor.
> >> 
> > 
> > Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
> > Systems Administrator @ hub.org 
> > primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 
> > 
> 
> 

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 


************




Re: [INTERFACES] libpq + multiple connections ...

From
"E.E. Mellor"
Date:
On Mon, 29 Nov 1999, Roberto Joao Lopes Garcia wrote:

> >> safely run two PQconnectdb() calls in parallel.  But you can open two
> >> connections in sequence and then use them in parallel; 
> >
> 
> So I could open, for exemple, 5 db connections (with PQconnectdb()) and
> then create 5 threads and use those connections in eache thread. But how
> about use PQreset() or PQfinish() inside the new created threads?
> 
> I used to test connection before send a query like bellow:
> 
> if(PQstatus(con)==CONNECTION_BAD) {
>    PQreset(con);
>    if(PQstatus(con)==CONNECTION_BAD) erro_fatal();
> }
> 
> Should I protect the calls to those functions by an mutex?? Could it
> work??? Which functions read or alter Qconnectdb()'s  static array??

You cannot mix calls to _any_ of PQconnectdb, PQreset, or their
newly-added asynchronous analogues in a thread-safe manner.  You need to
use a mutex to protect the calls above.

> I think it is important to mark wich  functions in Libpq is thread safe or
> not.

I agree.  I'll add a note to that effect to the docs today.

Ewan.