Thread: threads stuff/UnixWare
At the risk of getting my butt kicked again, is there any way we can talk about how to deal with threads on UnixWare and the libpq stuff? Has any other platform come up with a need to look for the real pthread_* calls from libpq? I would REALLY like us to support threaded programs in UnixWare, and would like to NOT antagonize the community, but would like to have a direction I can go to make this happen in 7.5 before we freeze. Thanks, LER -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On Wed, 12 May 2004, Larry Rosenman wrote: > At the risk of getting my butt kicked again, is there any way we can > talk about how to deal with threads on UnixWare and the libpq stuff? > > Has any other platform come up with a need to look for the real pthread_* > calls from libpq? > > I would REALLY like us to support threaded programs in UnixWare, and would > like to NOT antagonize the community, but would like to have a direction > I can go to make this happen in 7.5 before we freeze. Perfect time to start talking about it, at least :) Do you have a proposed patch to look at for doing this? ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
--On Wednesday, May 12, 2004 12:57:10 -0300 "Marc G. Fournier" <scrappy@postgresql.org> wrote: > On Wed, 12 May 2004, Larry Rosenman wrote: > >> At the risk of getting my butt kicked again, is there any way we can >> talk about how to deal with threads on UnixWare and the libpq stuff? >> >> Has any other platform come up with a need to look for the real pthread_* >> calls from libpq? >> >> I would REALLY like us to support threaded programs in UnixWare, and >> would like to NOT antagonize the community, but would like to have a >> direction I can go to make this happen in 7.5 before we freeze. > > Perfect time to start talking about it, at least :) > > Do you have a proposed patch to look at for doing this? No, as I believe the way I want to do it was frowned upon. I'd LIKE to be able to have PG wrappers for those functions, and have the first invocation of them look via dlsym() for the real ones, and if they are NOT there, use fake functions that assume we are NOT threaded. If they ARE present, indirect to the real functions. I am looking for direction before expending a lot of effort on it, if it will ultimately be rejected. > > ---- > Marc G. Fournier Hub.Org Networking Services > (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy > ICQ: 7615664 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Larry Rosenman <ler@lerctr.org> writes:
> At the risk of getting my butt kicked again, is there any way we can
> talk about how to deal with threads on UnixWare and the libpq stuff?
In what way does the current thread stuff not work for you?
regards, tom lane
--On Wednesday, May 12, 2004 12:58:58 -0400 Tom Lane <tgl@sss.pgh.pa.us> wrote: > Larry Rosenman <ler@lerctr.org> writes: >> At the risk of getting my butt kicked again, is there any way we can >> talk about how to deal with threads on UnixWare and the libpq stuff? > > In what way does the current thread stuff not work for you? > > regards, tom lane In the initdb compile: Undefined first referenced symbol in file pthread_mutex_unlock libpq.so pthread_getspecific libpq.so pthread_mutex_lock libpq.so pthread_key_create libpq.so pthread_once libpq.so pthread_setspecific libpq.so UX:ld: ERROR: Symbol referencing errors. No output written to initdb gmake[3]: *** [initdb] Error 1 gmake[3]: Leaving directory `/home/ler/pg-dev/pgsql-server/src/bin/initdb' gmake[2]: *** [all] Error 2 gmake[2]: Leaving directory `/home/ler/pg-dev/pgsql-server/src/bin' gmake[1]: *** [all] Error 2 gmake[1]: Leaving directory `/home/ler/pg-dev/pgsql-server/src' gmake: *** [all] Error 2 $ -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On Wed, 12 May 2004, Larry Rosenman wrote: > I'd LIKE to be able to have PG wrappers for those functions, and have > the first invocation of them look via dlsym() for the real ones, and if > they are NOT there, use fake functions that assume we are NOT threaded. Wouldn't it be easier to have a #define? Correct me if I'm wrong here, but the problem is that we are expecting thread functinos to be called x_, while Unixware defines them as pthread_, right? so make Unixware specific #defines that map x_ to pthread_ ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
--On Wednesday, May 12, 2004 14:14:30 -0300 "Marc G. Fournier" <scrappy@postgresql.org> wrote: > On Wed, 12 May 2004, Larry Rosenman wrote: > >> I'd LIKE to be able to have PG wrappers for those functions, and have >> the first invocation of them look via dlsym() for the real ones, and if >> they are NOT there, use fake functions that assume we are NOT threaded. > > Wouldn't it be easier to have a #define? > > Correct me if I'm wrong here, but the problem is that we are expecting > thread functinos to be called x_, while Unixware defines them as pthread_, > right? so make Unixware specific #defines that map x_ to pthread_ ... No, we are expecting them to have pthread_*. On UnixWare, to get the pthread_* functions, you need to invoke a compiler option (-Kpthread), to cause libpthread to be linked in. Since libpq.so now REQUIRES the function, we need to either: 1) force ANY program that uses libpq to be compiled/linked with -Kpthread or 2) dynamically determine if libpthread is loaded and use the real pthread_* functions if they are there (libpthread is in the image), or use fake singlethread versions of the pthread_* calls if libpthread is NOT in the image. > > > > ---- > Marc G. Fournier Hub.Org Networking Services > (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy > ICQ: 7615664 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Larry Rosenman <ler@lerctr.org> writes:
> --On Wednesday, May 12, 2004 12:58:58 -0400 Tom Lane <tgl@sss.pgh.pa.us>
>> In what way does the current thread stuff not work for you?
> In the initdb compile:
> Undefined first referenced
> symbol in file
> pthread_mutex_unlock libpq.so
> pthread_getspecific libpq.so
> pthread_mutex_lock libpq.so
> pthread_key_create libpq.so
> pthread_once libpq.so
> pthread_setspecific libpq.so
Hmm. And that means what? Does Unixware not have these functions?
Are we just failing to suck in the needed library?
regards, tom lane
--On Wednesday, May 12, 2004 13:18:35 -0400 Tom Lane <tgl@sss.pgh.pa.us> wrote: > Larry Rosenman <ler@lerctr.org> writes: >> --On Wednesday, May 12, 2004 12:58:58 -0400 Tom Lane <tgl@sss.pgh.pa.us> >>> In what way does the current thread stuff not work for you? > >> In the initdb compile: > >> Undefined first referenced >> symbol in file >> pthread_mutex_unlock libpq.so >> pthread_getspecific libpq.so >> pthread_mutex_lock libpq.so >> pthread_key_create libpq.so >> pthread_once libpq.so >> pthread_setspecific libpq.so > > Hmm. And that means what? Does Unixware not have these functions? > Are we just failing to suck in the needed library? > > regards, tom lane No, we don't pass the -Kpthread (thread CFLAGS) to the C links. This is the whole discussion we had back in January/February about forcing -Kpthread for *ALL* libpq using programs, or dynamically determining if the image already is linked -Kpthread, or not supporting threads at all on UW. #3 is unacceptable to me. LER -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On Wed, 12 May 2004, Larry Rosenman wrote: > > > --On Wednesday, May 12, 2004 14:14:30 -0300 "Marc G. Fournier" > <scrappy@postgresql.org> wrote: > > > On Wed, 12 May 2004, Larry Rosenman wrote: > > > >> I'd LIKE to be able to have PG wrappers for those functions, and have > >> the first invocation of them look via dlsym() for the real ones, and if > >> they are NOT there, use fake functions that assume we are NOT threaded. > > > > Wouldn't it be easier to have a #define? > > > > Correct me if I'm wrong here, but the problem is that we are expecting > > thread functinos to be called x_, while Unixware defines them as pthread_, > > right? so make Unixware specific #defines that map x_ to pthread_ ... > No, we are expecting them to have pthread_*. > > On UnixWare, to get the pthread_* functions, you need to invoke a compiler > option (-Kpthread), to cause libpthread to be linked in. > > Since libpq.so now REQUIRES the function, we need to either: > > 1) force ANY program that uses libpq to be compiled/linked with -Kpthread Ummm, shouldn't that be added to the port specific Makefile? ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
--On Wednesday, May 12, 2004 15:02:30 -0300 "Marc G. Fournier" <scrappy@postgresql.org> wrote: > On Wed, 12 May 2004, Larry Rosenman wrote: > >> >> >> --On Wednesday, May 12, 2004 14:14:30 -0300 "Marc G. Fournier" >> <scrappy@postgresql.org> wrote: >> >> > On Wed, 12 May 2004, Larry Rosenman wrote: >> > >> >> I'd LIKE to be able to have PG wrappers for those functions, and have >> >> the first invocation of them look via dlsym() for the real ones, and >> >> if they are NOT there, use fake functions that assume we are NOT >> >> threaded. >> > >> > Wouldn't it be easier to have a #define? >> > >> > Correct me if I'm wrong here, but the problem is that we are expecting >> > thread functinos to be called x_, while Unixware defines them as >> > pthread_, right? so make Unixware specific #defines that map x_ to >> > pthread_ ... >> No, we are expecting them to have pthread_*. >> >> On UnixWare, to get the pthread_* functions, you need to invoke a >> compiler option (-Kpthread), to cause libpthread to be linked in. >> >> Since libpq.so now REQUIRES the function, we need to either: >> >> 1) force ANY program that uses libpq to be compiled/linked with -Kpthread > > Ummm, shouldn't that be added to the port specific Makefile? See my reply to Tom. It forces ALL libpq using programs to be linked with -Kpthread, which was deemed unacceptable. > > ---- > Marc G. Fournier Hub.Org Networking Services > (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy > ICQ: 7615664 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On Wed, 12 May 2004, Larry Rosenman wrote: > > > --On Wednesday, May 12, 2004 15:02:30 -0300 "Marc G. Fournier" > <scrappy@postgresql.org> wrote: > > > On Wed, 12 May 2004, Larry Rosenman wrote: > > > >> > >> > >> --On Wednesday, May 12, 2004 14:14:30 -0300 "Marc G. Fournier" > >> <scrappy@postgresql.org> wrote: > >> > >> > On Wed, 12 May 2004, Larry Rosenman wrote: > >> > > >> >> I'd LIKE to be able to have PG wrappers for those functions, and have > >> >> the first invocation of them look via dlsym() for the real ones, and > >> >> if they are NOT there, use fake functions that assume we are NOT > >> >> threaded. > >> > > >> > Wouldn't it be easier to have a #define? > >> > > >> > Correct me if I'm wrong here, but the problem is that we are expecting > >> > thread functinos to be called x_, while Unixware defines them as > >> > pthread_, right? so make Unixware specific #defines that map x_ to > >> > pthread_ ... > >> No, we are expecting them to have pthread_*. > >> > >> On UnixWare, to get the pthread_* functions, you need to invoke a > >> compiler option (-Kpthread), to cause libpthread to be linked in. > >> > >> Since libpq.so now REQUIRES the function, we need to either: > >> > >> 1) force ANY program that uses libpq to be compiled/linked with -Kpthread > > > > Ummm, shouldn't that be added to the port specific Makefile? > See my reply to Tom. It forces ALL libpq using programs to be > linked with -Kpthread, which was deemed unacceptable. deemed unacceptable by whom? Sounds to me alot simpler of a solution then making wrappers for the pthread_* functions just to accommodate one OS ... I could see it if this was a wide-spread problem, but it doesn't appear to be ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
--On Wednesday, May 12, 2004 15:39:34 -0300 "Marc G. Fournier" <scrappy@postgresql.org> wrote: > On Wed, 12 May 2004, Larry Rosenman wrote: > >> >> >> --On Wednesday, May 12, 2004 15:02:30 -0300 "Marc G. Fournier" >> <scrappy@postgresql.org> wrote: >> >> > On Wed, 12 May 2004, Larry Rosenman wrote: >> > >> >> >> >> >> >> --On Wednesday, May 12, 2004 14:14:30 -0300 "Marc G. Fournier" >> >> <scrappy@postgresql.org> wrote: >> >> >> >> > On Wed, 12 May 2004, Larry Rosenman wrote: >> >> > >> >> >> I'd LIKE to be able to have PG wrappers for those functions, and >> >> >> have the first invocation of them look via dlsym() for the real >> >> >> ones, and if they are NOT there, use fake functions that assume we >> >> >> are NOT threaded. >> >> > >> >> > Wouldn't it be easier to have a #define? >> >> > >> >> > Correct me if I'm wrong here, but the problem is that we are >> >> > expecting thread functinos to be called x_, while Unixware defines >> >> > them as pthread_, right? so make Unixware specific #defines that >> >> > map x_ to pthread_ ... >> >> No, we are expecting them to have pthread_*. >> >> >> >> On UnixWare, to get the pthread_* functions, you need to invoke a >> >> compiler option (-Kpthread), to cause libpthread to be linked in. >> >> >> >> Since libpq.so now REQUIRES the function, we need to either: >> >> >> >> 1) force ANY program that uses libpq to be compiled/linked with >> >> -Kpthread >> > >> > Ummm, shouldn't that be added to the port specific Makefile? >> See my reply to Tom. It forces ALL libpq using programs to be >> linked with -Kpthread, which was deemed unacceptable. > > deemed unacceptable by whom? Sounds to me alot simpler of a solution then > making wrappers for the pthread_* functions just to accommodate one OS ... > I could see it if this was a wide-spread problem, but it doesn't appear to > be ... Bruce didn't want to muck ip all the .c programs in the distribution to do that. As well as any users using libpq. I'm all ears if the COMMUNITY wants to do it. LER > > ---- > Marc G. Fournier Hub.Org Networking Services > (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy > ICQ: 7615664 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On Wed, 12 May 2004, Larry Rosenman wrote: > >> > Ummm, shouldn't that be added to the port specific Makefile? > >> See my reply to Tom. It forces ALL libpq using programs to be > >> linked with -Kpthread, which was deemed unacceptable. > > > > deemed unacceptable by whom? Sounds to me alot simpler of a solution then > > making wrappers for the pthread_* functions just to accommodate one OS ... > > I could see it if this was a wide-spread problem, but it doesn't appear to > > be ... > Bruce didn't want to muck ip all the .c programs in the distribution to > do that. I'm confused now ... what .c programs would have to be mucked up if you add -Kpthread to the Unixware specific Makefile? ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
--On Wednesday, May 12, 2004 15:59:19 -0300 "Marc G. Fournier" <scrappy@postgresql.org> wrote: > On Wed, 12 May 2004, Larry Rosenman wrote: > >> >> > Ummm, shouldn't that be added to the port specific Makefile? >> >> See my reply to Tom. It forces ALL libpq using programs to be >> >> linked with -Kpthread, which was deemed unacceptable. >> > >> > deemed unacceptable by whom? Sounds to me alot simpler of a solution >> > then making wrappers for the pthread_* functions just to accommodate >> > one OS ... I could see it if this was a wide-spread problem, but it >> > doesn't appear to be ... >> Bruce didn't want to muck ip all the .c programs in the distribution to >> do that. > > I'm confused now ... what .c programs would have to be mucked up if you > add -Kpthread to the Unixware specific Makefile? initdb, et al. See these comments in src/template/unixware: # Disabled because flags are required for all apps using libpq. # Waiting to see if other platforms need this too. 2004-03-22 THREAD_SUPPORT=yes # verified UnixWare 7.1.4 2004-03-18 STRERROR_THREADSAFE=yes GETPWUID_THREADSAFE=yes GETHOSTBYNAME_THREADSAFE=yes $ I turned it back on to see if the new stuff had fixed it. > > ---- > Marc G. Fournier Hub.Org Networking Services > (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy > ICQ: 7615664 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Larry Rosenman <ler@lerctr.org> writes:
> This is the whole discussion we had back in January/February about forcing
> -Kpthread for *ALL* libpq using programs, or dynamically determining
> if the image already is linked -Kpthread, or not supporting threads
> at all on UW.
Oh, that business :-(. AFAIR we did not much care for any of the
proposed alternatives. Have you thought of any new options?
At this point I'd settle for saying that --enable-thread-safety on
Unixware will generate a library that requires -Kpthread. This is
kinda grungy but it seems that any more-pleasant solution would
require a disproportionate amount of work.
regards, tom lane
--On Wednesday, May 12, 2004 15:39:54 -0400 Tom Lane <tgl@sss.pgh.pa.us> wrote: > Larry Rosenman <ler@lerctr.org> writes: >> This is the whole discussion we had back in January/February about >> forcing -Kpthread for *ALL* libpq using programs, or dynamically >> determining if the image already is linked -Kpthread, or not supporting >> threads at all on UW. > > Oh, that business :-(. AFAIR we did not much care for any of the > proposed alternatives. Have you thought of any new options? > > At this point I'd settle for saying that --enable-thread-safety on > Unixware will generate a library that requires -Kpthread. This is > kinda grungy but it seems that any more-pleasant solution would > require a disproportionate amount of work. If I did the work for the dlsym() stuff would you and the rest of core@ accept it? > > regards, tom lane -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On Wed, 12 May 2004, Larry Rosenman wrote: > > > --On Wednesday, May 12, 2004 15:59:19 -0300 "Marc G. Fournier" > <scrappy@postgresql.org> wrote: > > > On Wed, 12 May 2004, Larry Rosenman wrote: > > > >> >> > Ummm, shouldn't that be added to the port specific Makefile? > >> >> See my reply to Tom. It forces ALL libpq using programs to be > >> >> linked with -Kpthread, which was deemed unacceptable. > >> > > >> > deemed unacceptable by whom? Sounds to me alot simpler of a solution > >> > then making wrappers for the pthread_* functions just to accommodate > >> > one OS ... I could see it if this was a wide-spread problem, but it > >> > doesn't appear to be ... > >> Bruce didn't want to muck ip all the .c programs in the distribution to > >> do that. > > > > I'm confused now ... what .c programs would have to be mucked up if you > > add -Kpthread to the Unixware specific Makefile? > initdb, et al. > > See these comments in src/template/unixware: > # Disabled because flags are required for all apps using libpq. > # Waiting to see if other platforms need this too. 2004-03-22 > THREAD_SUPPORT=yes > # verified UnixWare 7.1.4 2004-03-18 > STRERROR_THREADSAFE=yes > GETPWUID_THREADSAFE=yes > GETHOSTBYNAME_THREADSAFE=yes > $ > > I turned it back on to see if the new stuff had fixed it. k, I just looked at the template/unixware file myself, and it looks to me like -Kpthread is already defined, and included as part of THREAD_CPPFLAGS ... is THREAD_CPPFLAGS not honor'd in all of the various Makefile's (ie. initdb?) ... Please correct me if I'm wrong here, but so far, the issue seems to be that now that libpq uses pthread_* by default, Unixware requires -Kpthread to be used when building all the various support programs that use libpq ... and -Kpthread is defined in template/unixware ... so I'm *really* confused as to what is actually broken, except maybe the Makefile's themselves ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Larry Rosenman <ler@lerctr.org> writes:
> --On Wednesday, May 12, 2004 15:39:54 -0400 Tom Lane <tgl@sss.pgh.pa.us>=20
> wrote:
>> At this point I'd settle for saying that --enable-thread-safety on
>> Unixware will generate a library that requires -Kpthread. This is
>> kinda grungy but it seems that any more-pleasant solution would
>> require a disproportionate amount of work.
> If I did the work for the dlsym() stuff would you and the rest of core@
> accept it?
How invasive a change are we talking about? I'd be inclined to reject a
patch that makes libpq materially less readable ...
regards, tom lane
--On Wednesday, May 12, 2004 16:00:48 -0400 Tom Lane <tgl@sss.pgh.pa.us> wrote: > Larry Rosenman <ler@lerctr.org> writes: >> --On Wednesday, May 12, 2004 15:39:54 -0400 Tom Lane >> <tgl@sss.pgh.pa.us>=20 wrote: >>> At this point I'd settle for saying that --enable-thread-safety on >>> Unixware will generate a library that requires -Kpthread. This is >>> kinda grungy but it seems that any more-pleasant solution would >>> require a disproportionate amount of work. > >> If I did the work for the dlsym() stuff would you and the rest of core@ >> accept it? > > How invasive a change are we talking about? I'd be inclined to reject a > patch that makes libpq materially less readable ... > > regards, tom lane I was thinking of pq_pthread_* calls, and that function would set a static flag for calling either the real pthread_* function or a statically named version in libpgport.a that is a single thread wrapper. I know, this sucks, but, I don't see any other way, other than linking *ALL* libpq-using programs (including initdb and friends) with -K pthread. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Larry Rosenman <ler@lerctr.org> writes:
> I was thinking of pq_pthread_* calls, and that function would
> set a static flag for calling either the real pthread_* function
> or a statically named version in libpgport.a that is a single thread
> wrapper.
And how will you avoid having a link-time dependency on the real pthread
function? You muttered about dlsym but how much code will that take,
and what kind of runtime penalty will we incur? (IIRC the pthread
functions are performance critical.)
Even more to the point, can you make it work at all? I seem to recall
from the prior discussion that -Kpthread actually changes some code
generation details on that platform. Are -Kpthread and non -Kpthread
libraries interoperable at all?
> I know, this sucks, but, I don't see any other way, other than linking
> *ALL* libpq-using programs (including initdb and friends) with -K pthread.
-Kpthread doesn't sound that bad to me, as long as it's documented.
regards, tom lane
--On Wednesday, May 12, 2004 16:22:58 -0400 Tom Lane <tgl@sss.pgh.pa.us> wrote: > Larry Rosenman <ler@lerctr.org> writes: >> I was thinking of pq_pthread_* calls, and that function would >> set a static flag for calling either the real pthread_* function >> or a statically named version in libpgport.a that is a single thread >> wrapper. > > And how will you avoid having a link-time dependency on the real pthread > function? You muttered about dlsym but how much code will that take, > and what kind of runtime penalty will we incur? (IIRC the pthread > functions are performance critical.) The first call to ANY of the pthread_* functions would set the flag, and would cache the dlsym() info. > > Even more to the point, can you make it work at all? I seem to recall > from the prior discussion that -Kpthread actually changes some code > generation details on that platform. Are -Kpthread and non -Kpthread > libraries interoperable at all? Yes, this is how libc deals with it. We wouldn't have a hard symbol for the pthread_* calls. I can ask the compiler guys at SCO if you want. > >> I know, this sucks, but, I don't see any other way, other than linking >> *ALL* libpq-using programs (including initdb and friends) with -K >> pthread. > > -Kpthread doesn't sound that bad to me, as long as it's documented. > > regards, tom lane -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On Wed, 12 May 2004, Larry Rosenman wrote: > > > --On Wednesday, May 12, 2004 16:00:48 -0400 Tom Lane <tgl@sss.pgh.pa.us> > wrote: > > > Larry Rosenman <ler@lerctr.org> writes: > >> --On Wednesday, May 12, 2004 15:39:54 -0400 Tom Lane > >> <tgl@sss.pgh.pa.us>=20 wrote: > >>> At this point I'd settle for saying that --enable-thread-safety on > >>> Unixware will generate a library that requires -Kpthread. This is > >>> kinda grungy but it seems that any more-pleasant solution would > >>> require a disproportionate amount of work. > > > >> If I did the work for the dlsym() stuff would you and the rest of core@ > >> accept it? > > > > How invasive a change are we talking about? I'd be inclined to reject a > > patch that makes libpq materially less readable ... > > > > regards, tom lane > I was thinking of pq_pthread_* calls, and that function would > set a static flag for calling either the real pthread_* function > or a statically named version in libpgport.a that is a single thread > wrapper. > > I know, this sucks, but, I don't see any other way, other than linking > *ALL* libpq-using programs (including initdb and friends) with -K pthread. k, a change that 'sucks', vs linking against -Kpthread ... I'm for the -Kpthread route myself, which still sounds the 'clean' solution ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
--On Wednesday, May 12, 2004 17:29:30 -0300 "Marc G. Fournier" <scrappy@postgresql.org> wrote: > On Wed, 12 May 2004, Larry Rosenman wrote: > >> >> >> --On Wednesday, May 12, 2004 16:00:48 -0400 Tom Lane <tgl@sss.pgh.pa.us> >> wrote: >> >> > Larry Rosenman <ler@lerctr.org> writes: >> >> --On Wednesday, May 12, 2004 15:39:54 -0400 Tom Lane >> >> <tgl@sss.pgh.pa.us>=20 wrote: >> >>> At this point I'd settle for saying that --enable-thread-safety on >> >>> Unixware will generate a library that requires -Kpthread. This is >> >>> kinda grungy but it seems that any more-pleasant solution would >> >>> require a disproportionate amount of work. >> > >> >> If I did the work for the dlsym() stuff would you and the rest of >> >> core@ accept it? >> > >> > How invasive a change are we talking about? I'd be inclined to reject >> > a patch that makes libpq materially less readable ... >> > >> > regards, tom lane >> I was thinking of pq_pthread_* calls, and that function would >> set a static flag for calling either the real pthread_* function >> or a statically named version in libpgport.a that is a single thread >> wrapper. >> >> I know, this sucks, but, I don't see any other way, other than linking >> *ALL* libpq-using programs (including initdb and friends) with -K >> pthread. > > k, a change that 'sucks', vs linking against -Kpthread ... I'm for the > -Kpthread route myself, which still sounds the 'clean' solution ... that was rejected back in Jan-Mar. BUT, I agree it would work. I tried to submit the patch, and it was killed. LER > > ---- > Marc G. Fournier Hub.Org Networking Services > (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy > ICQ: 7615664 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On Wed, 12 May 2004, Larry Rosenman wrote: > > k, a change that 'sucks', vs linking against -Kpthread ... I'm for the > > -Kpthread route myself, which still sounds the 'clean' solution ... > that was rejected back in Jan-Mar. > > BUT, I agree it would work. > > I tried to submit the patch, and it was killed. Please re-submit since I don't recall the patch ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Larry Rosenman <ler@lerctr.org> writes:
>> Please save us all time searching by providing a URL ...
> I can't find my posts on archives.postgresql.org, but can find it in
> MY archives.
Well, then we won't be able to find 'em either, so please repost.
> This is heading down the same path I was back on 3/22.
I think at the time we were hoping for a cleaner solution. Since none
has emerged, it would be good of you to refresh our memories of where
things stood.
regards, tom lane
Tom Lane wrote: > Larry Rosenman <ler@lerctr.org> writes: > >> Please save us all time searching by providing a URL ... > > > I can't find my posts on archives.postgresql.org, but can find it in > > MY archives. > > Well, then we won't be able to find 'em either, so please repost. > > > This is heading down the same path I was back on 3/22. > > I think at the time we were hoping for a cleaner solution. Since none > has emerged, it would be good of you to refresh our memories of where > things stood. [ Sorry I have been away from email today. ] Larry, now that I have put the thread testing into configure, I am ready to deal with Unixware. In fact I posted to the list asking you about it but was too lazy to look up your email address. Anyway, I think the only solution is going to be to use the -K flag on all compiles on that platform if you want threads, so anything that uses libpq will need it. We don't have a client-level flags option at this point, but we can add it, or just the flag for the backend compile too. I think that is our best option. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
--On Wednesday, May 12, 2004 21:08:25 -0400 Bruce Momjian <pgman@candle.pha.pa.us> wrote: > Tom Lane wrote: >> Larry Rosenman <ler@lerctr.org> writes: >> >> Please save us all time searching by providing a URL ... >> >> > I can't find my posts on archives.postgresql.org, but can find it in >> > MY archives. >> >> Well, then we won't be able to find 'em either, so please repost. >> >> > This is heading down the same path I was back on 3/22. >> >> I think at the time we were hoping for a cleaner solution. Since none >> has emerged, it would be good of you to refresh our memories of where >> things stood. > > [ Sorry I have been away from email today. ] > > Larry, now that I have put the thread testing into configure, I am ready > to deal with Unixware. In fact I posted to the list asking you about it > but was too lazy to look up your email address. > > Anyway, I think the only solution is going to be to use the -K flag on > all compiles on that platform if you want threads, so anything that uses > libpq will need it. We don't have a client-level flags option at this > point, but we can add it, or just the flag for the backend compile too. I think a client-level flag would be best, that way the backend doesn't get the wrapped functions from libthread. Can you do the patch? > > I think that is our best option. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Larry Rosenman wrote: > > [ Sorry I have been away from email today. ] > > > > Larry, now that I have put the thread testing into configure, I am ready > > to deal with Unixware. In fact I posted to the list asking you about it > > but was too lazy to look up your email address. > > > > Anyway, I think the only solution is going to be to use the -K flag on > > all compiles on that platform if you want threads, so anything that uses > > libpq will need it. We don't have a client-level flags option at this > > point, but we can add it, or just the flag for the backend compile too. > I think a client-level flag would be best, that way the backend doesn't get > the wrapped functions from libthread. > > Can you do the patch? OK. Can someone think of a clean way to add a flag to everything but the backend build? Hold, what about /port. Larry, if we compile anything with threads does everything that links with that object file require the flag, even if there are no thread flags in the object file? I don't think so, but I am just checking. Actually, my bet is that it might do that because if it didn't, there would be no problem with using that flag on the backend because there are no threading calls in there. I am afraid of libpgport. Those object files get used by the backend, and by the client stuff. Larry can -K and non-K object files be mixed in the same binary? -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
--On Wednesday, May 12, 2004 21:55:40 -0400 Bruce Momjian <pgman@candle.pha.pa.us> wrote: > Larry Rosenman wrote: >> > [ Sorry I have been away from email today. ] >> > >> > Larry, now that I have put the thread testing into configure, I am >> > ready to deal with Unixware. In fact I posted to the list asking you >> > about it but was too lazy to look up your email address. >> > >> > Anyway, I think the only solution is going to be to use the -K flag on >> > all compiles on that platform if you want threads, so anything that >> > uses libpq will need it. We don't have a client-level flags option at >> > this point, but we can add it, or just the flag for the backend >> > compile too. >> I think a client-level flag would be best, that way the backend doesn't >> get the wrapped functions from libthread. >> >> Can you do the patch? > > OK. Can someone think of a clean way to add a flag to everything but > the backend build? Hold, what about /port. Larry, if we compile > anything with threads does everything that links with that object file > require the flag, even if there are no thread flags in the object file? > I don't think so, but I am just checking. Actually, my bet is that it > might do that because if it didn't, there would be no problem with using > that flag on the backend because there are no threading calls in there. Yes, there would still be the overhead, because the functions that libthread wraps would go through that overhead since libthread does it's magic at _ini time. Y'all were concerned with overhead in previous discussions. If you want to link the backend with -Kpthread, you can, but basically the client code (initdb, et al) should be -Kpthread, and the backend should NOT since there are no threads calls in the backend. > > I am afraid of libpgport. Those object files get used by the backend, > and by the client stuff. Larry can -K and non-K object files be mixed > in the same binary? Yes, they can. But, you MUST link ANY pthread_* using code with -K pthread. See above discussion. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Larry Rosenman wrote: > Yes, there would still be the overhead, because the functions that > libthread wraps would go through that overhead since libthread does it's > magic at _ini time. > > Y'all were concerned with overhead in previous discussions. > > If you want to link the backend with -Kpthread, you can, but basically the > client code (initdb, et al) should be -Kpthread, and the backend should NOT > since there are no threads calls in the backend. > > > > > I am afraid of libpgport. Those object files get used by the backend, > > and by the client stuff. Larry can -K and non-K object files be mixed > > in the same binary? > Yes, they can. > > But, you MUST link ANY pthread_* using code with -K pthread. OK, can someone come up with a cute way to add flags only to things that use libpq, meaning /interfaces and /bin? -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
--On Wednesday, May 12, 2004 22:26:03 -0400 Bruce Momjian <pgman@candle.pha.pa.us> wrote: > Larry Rosenman wrote: >> Yes, there would still be the overhead, because the functions that >> libthread wraps would go through that overhead since libthread does it's >> magic at _ini time. >> >> Y'all were concerned with overhead in previous discussions. >> >> If you want to link the backend with -Kpthread, you can, but basically >> the client code (initdb, et al) should be -Kpthread, and the backend >> should NOT since there are no threads calls in the backend. >> >> > >> > I am afraid of libpgport. Those object files get used by the backend, >> > and by the client stuff. Larry can -K and non-K object files be mixed >> > in the same binary? >> Yes, they can. >> >> But, you MUST link ANY pthread_* using code with -K pthread. > > OK, can someone come up with a cute way to add flags only to things that > use libpq, meaning /interfaces and /bin? What about where -lpq is specified? (I'm NOT a GNU Make guru). LER -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
> I know, this sucks, but, I don't see any other way, other than linking > *ALL* libpq-using programs (including initdb and friends) with -K pthread. How about making a libpq.so (without pthread) and a thread safe libpq_r.so ? Andreas
--On Thursday, May 13, 2004 09:54:02 +0200 Zeugswetter Andreas SB SD <ZeugswetterA@spardat.at> wrote: > >> I know, this sucks, but, I don't see any other way, other than linking >> *ALL* libpq-using programs (including initdb and friends) with -K > pthread. > > How about making a libpq.so (without pthread) and a thread safe > libpq_r.so ? That's against the way this platform does things. I.E. the only library is libc. All the other system supplied libs do the right thing in the presence or absence of libthread in the image. I did get a note from my SCO contacts that they are looking into how to make it easier for stuff to be threads ready, but I don't expect that to be ready for 7.5 release. The -Kpthread on all libpq using programs is the easiest way FOR NOW. Thanks, LER > > Andreas -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Larry Rosenman <ler@lerctr.org> writes:
> I did get a note from my SCO contacts that they are looking into how
> to make it easier for stuff to be threads ready, but I don't expect
> that to be ready for 7.5 release.
> The -Kpthread on all libpq using programs is the easiest way FOR NOW.
Hmm. If there is work happening at the platform level to improve
matters, then I'd definitely vote for taking the simplest solution
(ie -Kpthread everywhere) for now. We can always do all that other
work later if nothing gets fixed; but why invest a lot of work for
what might be only a short-term problem?
regards, tom lane
--On Thursday, May 13, 2004 09:18:21 -0400 Tom Lane <tgl@sss.pgh.pa.us> wrote: > Larry Rosenman <ler@lerctr.org> writes: >> I did get a note from my SCO contacts that they are looking into how >> to make it easier for stuff to be threads ready, but I don't expect >> that to be ready for 7.5 release. > >> The -Kpthread on all libpq using programs is the easiest way FOR NOW. > > Hmm. If there is work happening at the platform level to improve > matters, then I'd definitely vote for taking the simplest solution > (ie -Kpthread everywhere) for now. We can always do all that other > work later if nothing gets fixed; but why invest a lot of work for > what might be only a short-term problem? I agree. the only issue is how to set up our makefiles to only do the -Kpthread/-pthreads(gcc) flags on the client code, and not do it for the backend itself. > > regards, tom lane -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Larry Rosenman wrote: -- Start of PGP signed section. > > > --On Thursday, May 13, 2004 09:18:21 -0400 Tom Lane <tgl@sss.pgh.pa.us> > wrote: > > > Larry Rosenman <ler@lerctr.org> writes: > >> I did get a note from my SCO contacts that they are looking into how > >> to make it easier for stuff to be threads ready, but I don't expect > >> that to be ready for 7.5 release. > > > >> The -Kpthread on all libpq using programs is the easiest way FOR NOW. > > > > Hmm. If there is work happening at the platform level to improve > > matters, then I'd definitely vote for taking the simplest solution > > (ie -Kpthread everywhere) for now. We can always do all that other > > work later if nothing gets fixed; but why invest a lot of work for > > what might be only a short-term problem? > > [ added blank line here ] > I agree. the only issue is how to set up our makefiles to only do the > -Kpthread/-pthreads(gcc) flags on the client code, and not do it for > the backend itself. I think mixing a pgport that has thread flags with a backend that does not is more risky than just compiling everything with the same thread flags. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Larry Rosenman wrote:
>> I agree. the only issue is how to set up our makefiles to only do the
>> -Kpthread/-pthreads(gcc) flags on the client code, and not do it for
>> the backend itself.
> I think mixing a pgport that has thread flags with a backend that does
> not is more risky than just compiling everything with the same thread
> flags.
Can we get this straight: is -Kpthread a compile flag or just a link flag?
If the latter then it should not be needed in building the libpgport files.
If the former then adding it to LDFLAGS is the wrong thing.
regards, tom lane
Tom Lane wrote: > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > Larry Rosenman wrote: > >> I agree. the only issue is how to set up our makefiles to only do the > >> -Kpthread/-pthreads(gcc) flags on the client code, and not do it for > >> the backend itself. > > > I think mixing a pgport that has thread flags with a backend that does > > not is more risky than just compiling everything with the same thread > > flags. > > Can we get this straight: is -Kpthread a compile flag or just a link flag? > If the latter then it should not be needed in building the libpgport files. > If the former then adding it to LDFLAGS is the wrong thing. -Kpthread is needed for both compile and link, and the PTHREAD_CFLAGS is used for both compile and link phases. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
--On Thursday, May 13, 2004 10:05:22 -0400 Bruce Momjian <pgman@candle.pha.pa.us> wrote: > Basically, as things set right now in CVS, Unixware is ready to go > because it thread for everything. We don't have per-template thread > settings anymore because we test all of it in configure. Was a change made to link initdb et al with -Kpthread? It doesn't seem to be in AnonCVS yet. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Larry Rosenman wrote: -- Start of PGP signed section. > > > --On Thursday, May 13, 2004 10:05:22 -0400 Bruce Momjian > <pgman@candle.pha.pa.us> wrote: > > > > Basically, as things set right now in CVS, Unixware is ready to go > > because it thread for everything. We don't have per-template thread > > settings anymore because we test all of it in configure. > > Was a change made to link initdb et al with -Kpthread? It doesn't seem to > be in AnonCVS yet. Really? You are configuring with --enable-thread-safety? I just updated your template in CVS, and it is attached. However, any old CVS should work fine. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073 if test "$GCC" != yes; then then # The -Kno_host is for a bug in the compiler. See -hackers # discussion on 7-8/Aug/2003. cat >conftest.c <<__EOF__ extern char *strcpy(char *, const char *); static void f(char *p, int n){ strcpy(p+n,""); } void g(void){ f(0, 0); } __EOF__ if $CC -c -O -Kinline conftest.c >conftest.err 2>&1; then CFLAGS="-O -Kinline" else CFLAGS="-O -Kinline,no_host" fi rm -f conftest.* PTHREAD_CFLAGS="-Kpthread" fi
--On Thursday, May 13, 2004 11:44:59 -0400 Bruce Momjian <pgman@candle.pha.pa.us> wrote: > Larry Rosenman wrote: > -- Start of PGP signed section. >> >> >> --On Thursday, May 13, 2004 10:05:22 -0400 Bruce Momjian >> <pgman@candle.pha.pa.us> wrote: >> >> >> > Basically, as things set right now in CVS, Unixware is ready to go >> > because it thread for everything. We don't have per-template thread >> > settings anymore because we test all of it in configure. >> >> Was a change made to link initdb et al with -Kpthread? It doesn't seem >> to be in AnonCVS yet. > > Really? You are configuring with --enable-thread-safety? I just > updated your template in CVS, and it is attached. However, any old CVS > should work fine. Nope, initdb is where we still die: cc -O -Kinline initdb.o exec.o -L../../../src/interfaces/libpq -lpq -L../../../src/port -L/usr/local/lib -Wl,-R/usr/local/pgsql/lib -lz -lreadline -ltermcap -lresolv -lgen -lld -lsocket -lnsl -ldl -lm -lpgport -o initdb Undefined first referenced symbol in file pthread_mutex_unlock libpq.so pthread_getspecific libpq.so pthread_mutex_lock libpq.so pthread_key_create libpq.so pthread_once libpq.so pthread_setspecific libpq.so UX:ld: ERROR: Symbol referencing errors. No output written to initdb gmake[3]: *** [initdb] Error 1 gmake[3]: Leaving directory `/home/ler/pg-dev/pgsql-server/src/bin/initdb' gmake[2]: *** [all] Error 2 gmake[2]: Leaving directory `/home/ler/pg-dev/pgsql-server/src/bin' gmake[1]: *** [all] Error 2 gmake[1]: Leaving directory `/home/ler/pg-dev/pgsql-server/src' gmake: *** [all] Error 2 $ -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Larry Rosenman wrote: > > Really? You are configuring with --enable-thread-safety? I just > > updated your template in CVS, and it is attached. However, any old CVS > > should work fine. > Nope, initdb is where we still die: > OH! I remember now. What we have to do for this platform only is to pass the thread flags into all compile/links. I forgot that step. Let me work on that and send you a patch. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073