Thread: Re: [HACKERS] UnixWare/CVS Tip/initdb.c needs to use threads
[moved to -patches because of the patch]
--On Friday, March 19, 2004 08:01:53 -0500 Bruce Momjian
<pgman@candle.pha.pa.us> wrote:
> Larry Rosenman wrote:
>> > I thought that once you include libpthread in libpq, that you don't
>> > have to mention it again then you use libpq. Is your platform
>> > different somehow in this regard?
>> >
>> > I seem to remember this problem with libcrypt and libpq. Is this the
>> > same problem?
>> >
>> > I see that initdb is just the first of many /bin programs to be
>> > compiled, so if we have to add the thread lib, we will have to do it
>> > for all the bin programs. Yikes. Why wasn't this a problem for 7.4?
>> 7.4 had initdb as a Shell Script.
>> the 7.4.x libpq didn't have any pthread_* references in it, that I see
>> on my box.
>
> Ah, yes. We added the thread-local storage to handle SIGPIPE. The
> problem is that initdb isn't the only place. If you comment out initdb
> from the Makefile in src/bin, does the next make fail too? I bet it
> does.
Apparently, because of the way the wrappers work, having -lpthread on
libpq.so does NOT add it to the NEEDED list.
I made the following patch, and all compiles now:
Index: src/bin/initdb/Makefile
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/bin/initdb/Makefile,v
retrieving revision 1.35
diff -u -r1.35 Makefile
--- src/bin/initdb/Makefile 23 Dec 2003 21:56:20 -0000 1.35
+++ src/bin/initdb/Makefile 19 Mar 2004 13:35:19 -0000
@@ -20,7 +20,7 @@
all: submake-libpq submake-libpgport initdb
initdb: $(OBJS) $(libpq_builddir)/libpq.a
- $(CC) $(CFLAGS) $(OBJS) $(libpq) $(LDFLAGS) $(LIBS) -o $@
+ $(CC) $(CFLAGS) $(OBJS) $(libpq) $(LDFLAGS) $(LIBS) $(THREAD_LIBS) -o $@
install: all installdirs
$(INSTALL_PROGRAM) initdb$(X) $(DESTDIR)$(bindir)/initdb$(X)
Index: src/bin/pg_dump/Makefile
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/bin/pg_dump/Makefile,v
retrieving revision 1.44
diff -u -r1.44 Makefile
--- src/bin/pg_dump/Makefile 7 Feb 2004 07:20:12 -0000 1.44
+++ src/bin/pg_dump/Makefile 19 Mar 2004 13:35:19 -0000
@@ -25,13 +25,13 @@
all: submake-libpq submake-libpgport submake-backend pg_dump pg_restore
pg_dumpall
pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS)
$(libpq_builddir)/libpq.a
- $(CC) $(CFLAGS) pg_dump.o common.o pg_dump_sort.o $(OBJS) $(EXTRA_OBJS)
$(libpq) $(LDFLAGS) $(LIBS) -o $@
+ $(CC) $(CFLAGS) pg_dump.o common.o pg_dump_sort.o $(OBJS) $(EXTRA_OBJS)
$(libpq) $(LDFLAGS) $(LIBS) $(THREAD_LIBS) -o $@
pg_restore: pg_restore.o $(OBJS) $(libpq_builddir)/libpq.a
- $(CC) $(CFLAGS) pg_restore.o $(OBJS) $(EXTRA_OBJS) $(libpq) $(LDFLAGS)
$(LIBS) -o $@
+ $(CC) $(CFLAGS) pg_restore.o $(OBJS) $(EXTRA_OBJS) $(libpq) $(LDFLAGS)
$(LIBS) $(THREAD_LIBS) -o $@
pg_dumpall: pg_dumpall.o dumputils.o $(libpq_builddir)/libpq.a
- $(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(EXTRA_OBJS) $(libpq)
$(LDFLAGS) $(LIBS) -o $@
+ $(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(EXTRA_OBJS) $(libpq)
$(LDFLAGS) $(LIBS) $(THREAD_LIBS) -o $@
.PHONY: submake-backend
submake-backend:
Index: src/bin/pg_encoding/Makefile
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/bin/pg_encoding/Makefile,v
retrieving revision 1.16
diff -u -r1.16 Makefile
--- src/bin/pg_encoding/Makefile 29 Nov 2003 19:52:05 -0000 1.16
+++ src/bin/pg_encoding/Makefile 19 Mar 2004 13:35:19 -0000
@@ -17,7 +17,7 @@
all: submake-libpq submake-libpgport pg_encoding
pg_encoding: $(OBJS)
- $(CC) $(CFLAGS) $^ $(libpq) $(LDFLAGS) $(LIBS) -o $@
+ $(CC) $(CFLAGS) $^ $(libpq) $(LDFLAGS) $(LIBS) $(THREAD_LIBS) -o $@
install: all installdirs
$(INSTALL_PROGRAM) pg_encoding$(X) $(DESTDIR)$(bindir)/pg_encoding$(X)
Index: src/bin/pgtclsh/Makefile
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/bin/pgtclsh/Makefile,v
retrieving revision 1.43
diff -u -r1.43 Makefile
--- src/bin/pgtclsh/Makefile 19 Dec 2003 11:54:25 -0000 1.43
+++ src/bin/pgtclsh/Makefile 19 Mar 2004 13:35:20 -0000
@@ -33,10 +33,10 @@
all: submake $(PROGRAMS)
pgtclsh: pgtclAppInit.o
- $(CC) $(CFLAGS) $^ $(libpgtcl) $(libpq) $(TCL_LIB_SPEC) $(TCL_LIBS)
$(LDFLAGS) $(LIBS) -o $@
+ $(CC) $(CFLAGS) $^ $(libpgtcl) $(libpq) $(TCL_LIB_SPEC) $(TCL_LIBS)
$(LDFLAGS) $(LIBS) $(THREAD_LIBS) -o $@
pgtksh: pgtkAppInit.o
- $(CC) $(CFLAGS) $^ $(libpgtcl) $(libpq) $(TK_LIB_SPEC) $(TK_LIBS)
$(TCL_LIB_SPEC) $(LDFLAGS) $(LIBS) -o $@
+ $(CC) $(CFLAGS) $^ $(libpgtcl) $(libpq) $(TK_LIB_SPEC) $(TK_LIBS)
$(TCL_LIB_SPEC) $(LDFLAGS) $(LIBS) $(THREAD_LIBS) -o $@
.PHONY: submake
submake:
Index: src/bin/psql/Makefile
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/bin/psql/Makefile,v
retrieving revision 1.40
diff -u -r1.40 Makefile
--- src/bin/psql/Makefile 9 Mar 2004 19:47:05 -0000 1.40
+++ src/bin/psql/Makefile 19 Mar 2004 13:35:20 -0000
@@ -27,7 +27,7 @@
all: submake-libpq submake-libpgport psql
psql: $(OBJS) $(libpq_builddir)/libpq.a
- $(CC) $(CFLAGS) $(OBJS) $(libpq) $(LDFLAGS) $(LIBS) -o $@
+ $(CC) $(CFLAGS) $(OBJS) $(libpq) $(LDFLAGS) $(LIBS) $(THREAD_LIBS) -o $@
help.o: $(srcdir)/sql_help.h
Index: src/bin/scripts/Makefile
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/bin/scripts/Makefile,v
retrieving revision 1.25
diff -u -r1.25 Makefile
--- src/bin/scripts/Makefile 29 Nov 2003 19:52:07 -0000 1.25
+++ src/bin/scripts/Makefile 19 Mar 2004 13:35:20 -0000
@@ -20,7 +20,7 @@
all: submake-libpq submake-backend $(PROGRAMS)
%: %.o
- $(CC) $(CFLAGS) $^ $(libpq) $(LDFLAGS) $(LIBS) -o $@
+ $(CC) $(CFLAGS) $^ $(libpq) $(LDFLAGS) $(LIBS) $(THREAD_LIBS) -o $@
createdb: createdb.o common.o dumputils.o
$(top_builddir)/src/backend/parser/keywords.o
createlang: createlang.o common.o print.o mbprint.o
Index: src/template/unixware
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/template/unixware,v
retrieving revision 1.30
diff -u -r1.30 unixware
--- src/template/unixware 11 Feb 2004 21:44:06 -0000 1.30
+++ src/template/unixware 19 Mar 2004 13:35:21 -0000
@@ -24,5 +24,10 @@
THREAD_CPPFLAGS="-K pthread"
fi
-# tools/thread/thread_test must be run
+THREAD_SUPPORT=yes
THREAD_CPPFLAGS="$THREAD_CPPFLAGS -D_REENTRANT"
+STRERROR_THREADSAFE=yes
+GETPWUID_THREADSAFE=yes
+GETHOSTBYNAME_THREADSAFE=yes
+THREAD_LIBS='-lpthread'
+
Index: src/tools/fsync/Makefile
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/tools/fsync/Makefile,v
retrieving revision 1.1
diff -u -r1.1 Makefile
--- src/tools/fsync/Makefile 18 Mar 2004 03:56:59 -0000 1.1
+++ src/tools/fsync/Makefile 19 Mar 2004 13:35:21 -0000
@@ -4,7 +4,7 @@
#
TARGET = test_fsync
XFLAGS =
-CFLAGS = -g -Wall
+CFLAGS = -O
LIBS =
$(TARGET) : test_fsync.o
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Attachment
> [moved to -patches because of the patch] > > > --On Friday, March 19, 2004 08:01:53 -0500 Bruce Momjian > <pgman@candle.pha.pa.us> wrote: > > > Larry Rosenman wrote: > >> > I thought that once you include libpthread in libpq, that you don't > >> > have to mention it again then you use libpq. Is your platform > >> > different somehow in this regard? > >> > > >> > I seem to remember this problem with libcrypt and libpq. Is this the > >> > same problem? > >> > > >> > I see that initdb is just the first of many /bin programs to be > >> > compiled, so if we have to add the thread lib, we will have to do it > >> > for all the bin programs. Yikes. Why wasn't this a problem for 7.4? > >> 7.4 had initdb as a Shell Script. > >> the 7.4.x libpq didn't have any pthread_* references in it, that I see > >> on my box. > > > > Ah, yes. We added the thread-local storage to handle SIGPIPE. The > > problem is that initdb isn't the only place. If you comment out initdb > > from the Makefile in src/bin, does the next make fail too? I bet it > > does. > > Apparently, because of the way the wrappers work, having -lpthread on > libpq.so does NOT add it to the NEEDED list. > > I made the following patch, and all compiles now: Yes, I was afraid of that. Is there any way to make it work? If not, evey libpq program you create will need -lpthread added. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
--On Friday, March 19, 2004 10:15:56 -0500 Bruce Momjian <pgman@candle.pha.pa.us> wrote: >> [moved to -patches because of the patch] >> >> >> --On Friday, March 19, 2004 08:01:53 -0500 Bruce Momjian >> <pgman@candle.pha.pa.us> wrote: >> >> > Larry Rosenman wrote: >> >> > I thought that once you include libpthread in libpq, that you don't >> >> > have to mention it again then you use libpq. Is your platform >> >> > different somehow in this regard? >> >> > >> >> > I seem to remember this problem with libcrypt and libpq. Is this >> >> > the same problem? >> >> > >> >> > I see that initdb is just the first of many /bin programs to be >> >> > compiled, so if we have to add the thread lib, we will have to do it >> >> > for all the bin programs. Yikes. Why wasn't this a problem for >> >> > 7.4? >> >> 7.4 had initdb as a Shell Script. >> >> the 7.4.x libpq didn't have any pthread_* references in it, that I see >> >> on my box. >> > >> > Ah, yes. We added the thread-local storage to handle SIGPIPE. The >> > problem is that initdb isn't the only place. If you comment out initdb >> > from the Makefile in src/bin, does the next make fail too? I bet it >> > does. >> >> Apparently, because of the way the wrappers work, having -lpthread on >> libpq.so does NOT add it to the NEEDED list. >> >> I made the following patch, and all compiles now: > > Yes, I was afraid of that. Is there any way to make it work? If not, > evey libpq program you create will need -lpthread added. I think we need to mention if you --enable-thread-safety you MUST use -lpthread on UnixWare, at least. I don't know about other platforms. I'll ask the compiler guys, but I suspect we're going to have to do it this way. LER > > -- > Bruce Momjian | http://candle.pha.pa.us > pgman@candle.pha.pa.us | (610) 359-1001 > + If your life is a hard drive, | 13 Roberts Road > + Christ can be your backup. | Newtown Square, Pennsylvania > 19073 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Attachment
--On Friday, March 19, 2004 09:18:03 -0600 Larry Rosenman <ler@lerctr.org> wrote: > > > --On Friday, March 19, 2004 10:15:56 -0500 Bruce Momjian > <pgman@candle.pha.pa.us> wrote: > >>> [moved to -patches because of the patch] >>> >>> >>> --On Friday, March 19, 2004 08:01:53 -0500 Bruce Momjian >>> <pgman@candle.pha.pa.us> wrote: >>> >>> > Larry Rosenman wrote: >>> >> > I thought that once you include libpthread in libpq, that you don't >>> >> > have to mention it again then you use libpq. Is your platform >>> >> > different somehow in this regard? >>> >> > >>> >> > I seem to remember this problem with libcrypt and libpq. Is this >>> >> > the same problem? >>> >> > >>> >> > I see that initdb is just the first of many /bin programs to be >>> >> > compiled, so if we have to add the thread lib, we will have to do >>> >> > it for all the bin programs. Yikes. Why wasn't this a problem for >>> >> > 7.4? >>> >> 7.4 had initdb as a Shell Script. >>> >> the 7.4.x libpq didn't have any pthread_* references in it, that I >>> >> see on my box. >>> > >>> > Ah, yes. We added the thread-local storage to handle SIGPIPE. The >>> > problem is that initdb isn't the only place. If you comment out >>> > initdb from the Makefile in src/bin, does the next make fail too? I >>> > bet it does. >>> >>> Apparently, because of the way the wrappers work, having -lpthread on >>> libpq.so does NOT add it to the NEEDED list. >>> >>> I made the following patch, and all compiles now: >> >> Yes, I was afraid of that. Is there any way to make it work? If not, >> evey libpq program you create will need -lpthread added. > I think we need to mention if you --enable-thread-safety you MUST use > -lpthread on UnixWare, at least. I don't know about other platforms. > > I'll ask the compiler guys, but I suspect we're going to have to do it > this way. > From the Compiler Guys: The a.out (not any library) should be linked with -Kpthread (not -lpthread). This will force libthread to be linked in the right order relative to libc, libC, networking libraries, etc. In other words, the entire application either is or is not linked with threads; it's not a property of an individual library. SO, IF we are using the threads flags, we need to use them on ALL libpq-using programs, ours or the users. Not surprising. LER -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Attachment
Larry Rosenman wrote: > > I'll ask the compiler guys, but I suspect we're going to have to do it > > this way. > > > From the Compiler Guys: > > The a.out (not any library) should be linked with -Kpthread (not > -lpthread). > This will force libthread to be linked in the right order relative to > libc, libC, networking libraries, etc. > > In other words, the entire application either is or is not linked with > threads; it's not a property of an individual library. > > > SO, IF we are using the threads flags, we need to use them on ALL > libpq-using programs, ours or the users. > > Not surprising. Is this problem with just the native compiler, or gcc too? Are enough other platforms going to have this problem or do I make a platform-specific hack for this? -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Larry Rosenman <ler@lerctr.org> writes:
> In other words, the entire application either is or is not linked with
> threads; it's not a property of an individual library.
> SO, IF we are using the threads flags, we need to use them on ALL=20
> libpq-using programs, ours or the users.
Yeek. This is an example of the sort of thing that makes people want to
build two versions of every library.
I'm not excited about doing that (at least not unless it pops up on more
platforms). It seems that what we have to do for Unixware is add
-Kpthread to LDFLAGS; is that correct?
regards, tom lane
Tom Lane wrote:
> Larry Rosenman <ler@lerctr.org> writes:
> > In other words, the entire application either is or is not linked with
> > threads; it's not a property of an individual library.
>
> > SO, IF we are using the threads flags, we need to use them on ALL=20
> > libpq-using programs, ours or the users.
>
> Yeek. This is an example of the sort of thing that makes people want to
> build two versions of every library.
>
> I'm not excited about doing that (at least not unless it pops up on more
> platforms). It seems that what we have to do for Unixware is add
> -Kpthread to LDFLAGS; is that correct?
I am attaching a new bin/Makefile that should fix it. The new code is:
# this platform needs the thread compiler flag for all binaries
# to override libc
ifeq ($(PORTNAME), unixware)
CPPFLAGS += "$THREAD_CPPFLAGS"
endif
Larry, does this fix it?
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073
#-------------------------------------------------------------------------
#
# Makefile for src/bin (client programs)
#
# Portions Copyright (c) 1996-2002, PostgreSQL Global Development Group
# Portions Copyright (c) 1994, Regents of the University of California
#
# $PostgreSQL: pgsql-server/src/bin/Makefile,v 1.41 2003/12/17 18:44:08 petere Exp $
#
#-------------------------------------------------------------------------
subdir = src/bin
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
DIRS := initdb initlocation ipcclean pg_ctl pg_dump \
psql scripts pg_config pg_controldata pg_resetxlog \
pg_encoding
# this platforms needs the thread compiler flag for all binaries to override libc
ifeq ($(PORTNAME), unixware)
CPPFLAGS += "$THREAD_CPPFLAGS"
endif
ifeq ($(with_tcl), yes)
DIRS += pgtclsh
endif
all install installdirs uninstall depend distprep:
@for dir in $(DIRS); do $(MAKE) -C $$dir $@ || exit; done
clean distclean maintainer-clean:
-@for dir in $(DIRS); do $(MAKE) -C $$dir $@; done
Bruce Momjian wrote: > Tom Lane wrote: > > Larry Rosenman <ler@lerctr.org> writes: > > > In other words, the entire application either is or is not linked with > > > threads; it's not a property of an individual library. > > > > > SO, IF we are using the threads flags, we need to use them on ALL=20 > > > libpq-using programs, ours or the users. > > > > Yeek. This is an example of the sort of thing that makes people want to > > build two versions of every library. > > > > I'm not excited about doing that (at least not unless it pops up on more > > platforms). It seems that what we have to do for Unixware is add > > -Kpthread to LDFLAGS; is that correct? > > I am attaching a new bin/Makefile that should fix it. The new code is: > Sorry, here is the right patch. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073 Index: src/bin/Makefile =================================================================== RCS file: /cvsroot/pgsql-server/src/bin/Makefile,v retrieving revision 1.41 diff -c -c -r1.41 Makefile *** src/bin/Makefile 17 Dec 2003 18:44:08 -0000 1.41 --- src/bin/Makefile 19 Mar 2004 16:52:54 -0000 *************** *** 17,22 **** --- 17,27 ---- psql scripts pg_config pg_controldata pg_resetxlog \ pg_encoding + # this platforms needs the thread compiler flag for all binaries to override libc + ifeq ($(PORTNAME), unixware) + CPPFLAGS += $(THREAD_CPPFLAGS) + endif + ifeq ($(with_tcl), yes) DIRS += pgtclsh endif
Larry Rosenman wrote: -- Start of PGP signed section. > [moved to -patches because of the patch] > > > --On Friday, March 19, 2004 08:01:53 -0500 Bruce Momjian > <pgman@candle.pha.pa.us> wrote: > > > Larry Rosenman wrote: > >> > I thought that once you include libpthread in libpq, that you don't > >> > have to mention it again then you use libpq. Is your platform > >> > different somehow in this regard? > >> > > >> > I seem to remember this problem with libcrypt and libpq. Is this the > >> > same problem? > >> > > >> > I see that initdb is just the first of many /bin programs to be > >> > compiled, so if we have to add the thread lib, we will have to do it > >> > for all the bin programs. Yikes. Why wasn't this a problem for 7.4? > >> 7.4 had initdb as a Shell Script. > >> the 7.4.x libpq didn't have any pthread_* references in it, that I see > >> on my box. > > > > Ah, yes. We added the thread-local storage to handle SIGPIPE. The > > problem is that initdb isn't the only place. If you comment out initdb > > from the Makefile in src/bin, does the next make fail too? I bet it > > does. > > Apparently, because of the way the wrappers work, having -lpthread on > libpq.so does NOT add it to the NEEDED list. Clarification. The flags are "-pthread" or "-K pthread", not -lpthread, which is clearly a linker flag. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073