Thread: library policy question

library policy question

From
Michael Meskes
Date:
What exactly is our policy towards global variables in libraries? I take is
these variables are harmless in multi tasking operation as each process has
its seperate data space. But they may do harm in multi threading. But then
libpq are is not suitable for multi threading, is it?

Michael
-- 
Michael Meskes                         | Go SF 49ers!
Th.-Heuss-Str. 61, D-41812 Erkelenz    | Go Rhein Fire!
Tel.: (+49) 2431/72651                 | Use Debian GNU/Linux!
Email: Michael@Fam-Meskes.De           | Use PostgreSQL!


Re: [HACKERS] library policy question

From
Tom Lane
Date:
Michael Meskes <meskes@postgreSQL.org> writes:
> What exactly is our policy towards global variables in libraries?

Avoid them.

> But then libpq are is not suitable for multi threading, is it?

AFAIK, libpq's only use of non-constant globals is the ill-designed
PQconninfoOption array.  I'd get rid of that if it didn't mean breaking
the library API.  But as things stand, you can't safely use concurrent
calls to PQconnectdb or PQconndefaults.  Everything else should be OK,
unless someone has broken it recently.

(idly examines code...)

Hmm, we do have a bit of a problem here.  While PQconnectdb can be
replaced by PQsetdb to avoid the concurrency issue, there is no
thread-safe equivalent for the new routines
PQconnectStart/PQconnectPoll.  That may not matter much, because
probably you would only need those in a single-threaded environment,
but it's still kinda ugly.  In any case it'd be a lot nicer to be
able to say "libpq is thread safe" rather than "almost thread safe".

At one point we had discussed going ahead and breaking compatibility
in order to get rid of the static PQconninfoOption array.  It wouldn't
be a big change in the API: we'd only need to make PQconndefaults return
a malloc'd array instead of a static.  That probably wouldn't really
break any existing code, just create a small memory leak in applications
that didn't know to free the result when they were done with it.  My bet
is that very few apps use PQconndefaults anyway.

7.0 would be a good time to do that if we were gonna do it.  Comments?
        regards, tom lane


Re: [HACKERS] library policy question

From
Thomas Lockhart
Date:
> 7.0 would be a good time to do that if we were gonna do it.  Comments?

Yup.
                     - Thomas

-- 
Thomas Lockhart                lockhart@alumni.caltech.edu
South Pasadena, California


Re: [HACKERS] library policy question

From
Lamar Owen
Date:
Tom Lane wrote:
> but it's still kinda ugly.  In any case it'd be a lot nicer to be
> able to say "libpq is thread safe" rather than "almost thread safe".
> 7.0 would be a good time to do that if we were gonna do it.  Comments?

If time is available to do that, I agree that now is an great time to do
so. As a user of a multithreaded web front end to PostgreSQL
(AOLserver), I personally am affected by the result.  The AOLserver
PostgreSQL driver avoids the PQconnectdb() issue by using
PQsetdbLogin().

HOWEVER, it was a hunt to find that information -- it would have been
nice for the docs to say 'libpq {is|is not} threadsafe' -- even 'libpq
is threadsafe if and only if the following API calls are used:' would be
nice.
In fact, even if libpq is not touched, a documentation note to libpq's
threadsafeness would be nice.

--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11


Re: [HACKERS] library policy question

From
The Hermit Hacker
Date:
On Tue, 7 Mar 2000, Thomas Lockhart wrote:

> > 7.0 would be a good time to do that if we were gonna do it.  Comments?
> 
> Yup.

Ditto ... 




Re: [HACKERS] library policy question

From
Tom Lane
Date:
The Hermit Hacker <scrappy@hub.org> writes:
> On Tue, 7 Mar 2000, Thomas Lockhart wrote:
>>>> 7.0 would be a good time to do that if we were gonna do it.  Comments?
>> 
>> Yup.

> Ditto ... 

OK, I'll take a look at doing it later this week.
        regards, tom lane


Re: [HACKERS] library policy question

From
Bruce Momjian
Date:
> Hmm, we do have a bit of a problem here.  While PQconnectdb can be
> replaced by PQsetdb to avoid the concurrency issue, there is no
> thread-safe equivalent for the new routines
> PQconnectStart/PQconnectPoll.  That may not matter much, because
> probably you would only need those in a single-threaded environment,
> but it's still kinda ugly.  In any case it'd be a lot nicer to be
> able to say "libpq is thread safe" rather than "almost thread safe".
> 
> At one point we had discussed going ahead and breaking compatibility
> in order to get rid of the static PQconninfoOption array.  It wouldn't
> be a big change in the API: we'd only need to make PQconndefaults return
> a malloc'd array instead of a static.  That probably wouldn't really
> break any existing code, just create a small memory leak in applications
> that didn't know to free the result when they were done with it.  My bet
> is that very few apps use PQconndefaults anyway.
> 
> 7.0 would be a good time to do that if we were gonna do it.  Comments?
> 

Seems like a good time to do it.

--  Bruce Momjian                        |  http://www.op.net/~candle pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: [HACKERS] library policy question

From
Michael Meskes
Date:
On Tue, Mar 07, 2000 at 11:34:04AM -0500, Tom Lane wrote:
> > What exactly is our policy towards global variables in libraries?
> 
> Avoid them.

And what shall I do with sqlca? Make every program define it in its own space?

Michael
-- 
Michael Meskes                         | Go SF 49ers!
Th.-Heuss-Str. 61, D-41812 Erkelenz    | Go Rhein Fire!
Tel.: (+49) 2431/72651                 | Use Debian GNU/Linux!
Email: Michael@Fam-Meskes.De           | Use PostgreSQL!


Re: [HACKERS] library policy question

From
Michael Meskes
Date:
On Tue, Mar 07, 2000 at 11:34:04AM -0500, Tom Lane wrote:
> > What exactly is our policy towards global variables in libraries?
> 
> Avoid them.

Maybe I've got a brain lock for the moment but what do I do to the list of
connections I have to handle? Since it is used in several functions I cannot
see how to program this without a global variable albeit a static one. What
I need is a mapping of the SQL conenction name to the PGconn structure.

I also wonder if multiple threads should be allowed access to the same
structure, i.e. if one thread opens a new connection should the other one be
allowed to access this too?

BTW libpgeasy does not seem to be able to be used in multi-threading
environments either. Is this library still supported?

Michael
-- 
Michael Meskes                         | Go SF 49ers!
Th.-Heuss-Str. 61, D-41812 Erkelenz    | Go Rhein Fire!
Tel.: (+49) 2431/72651                 | Use Debian GNU/Linux!
Email: Michael@Fam-Meskes.De           | Use PostgreSQL!


Re: [HACKERS] library policy question

From
Michael Meskes
Date:
On Wed, Mar 08, 2000 at 02:06:35PM +0000, Thomas Lockhart wrote:
> My vague recollection is that embedded SQL doesn't multithread very
> well for exactly this reason. You may be stuck with a global variable
> for that case...

I was afraid you'd say that. So I can use lots of global vars since libecpg
does not multithread anyway. :-)

Michael
-- 
Michael Meskes                         | Go SF 49ers!
Th.-Heuss-Str. 61, D-41812 Erkelenz    | Go Rhein Fire!
Tel.: (+49) 2431/72651                 | Use Debian GNU/Linux!
Email: Michael@Fam-Meskes.De           | Use PostgreSQL!


Re: [HACKERS] library policy question

From
Thomas Lockhart
Date:
> And what shall I do with sqlca? Make every program define it in its own 
> space?

My vague recollection is that embedded SQL doesn't multithread very
well for exactly this reason. You may be stuck with a global variable
for that case...
                     - Thomas

-- 
Thomas Lockhart                lockhart@alumni.caltech.edu
South Pasadena, California


Re: [HACKERS] library policy question

From
Tom Lane
Date:
Thomas Lockhart <lockhart@alumni.caltech.edu> writes:
>> And what shall I do with sqlca? Make every program define it in its own 
>> space?

> My vague recollection is that embedded SQL doesn't multithread very
> well for exactly this reason. You may be stuck with a global variable
> for that case...

Aside from the unthreadable API, ecpg has another potential threading
problem: it depends on a lexer and parser that might or might not be
thread-safe, depending on what they were generated with.

I'd advise just labeling ecpg "not threadable" :-(.  Not much point in
breaking existing apps by changing the API, when you still won't be able
to guarantee thread safeness...
        regards, tom lane