Thread: peripatus build failures....
I noticed my buildfarm member peripatus hadn't been building due to me missing a perl library. After I fixed that I get failures on: Buildfarm member peripatus failed on REL9_3_STABLE stage Make Buildfarm member peripatus failed on REL9_4_STABLE stage Make Buildfarm member peripatus failed on REL9_5_STABLE stage Make Buildfarm member peripatus failed on REL9_6_STABLE stage Make Buildfarm member peripatus failed on REL_10_STABLE stage Make HEAD and REL_11_STABLE build fine. These all fail on relocation issues in pgport(strcase*) and possibly others. This is FreeBSD HEAD as of today, clang 6.0.1, lld as the linker. Can someone take a look at these? I've tried making sure all the cache's are clear, etc, same results. Thanks. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
Attachment
On Thu, Jul 5, 2018 at 11:43 AM, Larry Rosenman <ler@lerctr.org> wrote: > I noticed my buildfarm member peripatus hadn't been building due to me > missing a perl library. After I fixed that I get failures on: > > Buildfarm member peripatus failed on REL9_3_STABLE stage Make > Buildfarm member peripatus failed on REL9_4_STABLE stage Make > Buildfarm member peripatus failed on REL9_5_STABLE stage Make > Buildfarm member peripatus failed on REL9_6_STABLE stage Make > Buildfarm member peripatus failed on REL_10_STABLE stage Make > > HEAD and REL_11_STABLE build fine. > > These all fail on relocation issues in pgport(strcase*) and possibly > others. > > This is FreeBSD HEAD as of today, clang 6.0.1, lld as the linker. > > Can someone take a look at these? > I've tried making sure all the cache's are clear, etc, same results. Seems to have something to do with the recent linker toolchain changes in FreeBSD. I don't actually understand what's going on here or why it doesn't affect REL_11_STABLE and HEAD, but this seems to be some kind of explanation + workaround (as used by the ports): https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219758 https://svnweb.freebsd.org/ports?view=revision&revision=456635 -- Thomas Munro http://www.enterprisedb.com
On Thu, Jul 05, 2018 at 12:30:37PM +1200, Thomas Munro wrote: > On Thu, Jul 5, 2018 at 11:43 AM, Larry Rosenman <ler@lerctr.org> wrote: > > I noticed my buildfarm member peripatus hadn't been building due to me > > missing a perl library. After I fixed that I get failures on: > > > > Buildfarm member peripatus failed on REL9_3_STABLE stage Make > > Buildfarm member peripatus failed on REL9_4_STABLE stage Make > > Buildfarm member peripatus failed on REL9_5_STABLE stage Make > > Buildfarm member peripatus failed on REL9_6_STABLE stage Make > > Buildfarm member peripatus failed on REL_10_STABLE stage Make > > > > HEAD and REL_11_STABLE build fine. > > > > These all fail on relocation issues in pgport(strcase*) and possibly > > others. > > > > This is FreeBSD HEAD as of today, clang 6.0.1, lld as the linker. > > > > Can someone take a look at these? > > I've tried making sure all the cache's are clear, etc, same results. > > Seems to have something to do with the recent linker toolchain changes > in FreeBSD. I don't actually understand what's going on here or why > it doesn't affect REL_11_STABLE and HEAD, but this seems to be some > kind of explanation + workaround (as used by the ports): > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219758 > https://svnweb.freebsd.org/ports?view=revision&revision=456635 > I agree. Is there an easy way I can add this work around to peripatus' source tree: It may be that adding "LDFLAGS+= -Wl,-z,notext" (and removing LLD_UNSAFE) will let the port build with lld. Thanks, Thomas. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
Attachment
On Wed, Jul 04, 2018 at 07:35:28PM -0500, Larry Rosenman wrote: > On Thu, Jul 05, 2018 at 12:30:37PM +1200, Thomas Munro wrote: > > On Thu, Jul 5, 2018 at 11:43 AM, Larry Rosenman <ler@lerctr.org> wrote: > > > I noticed my buildfarm member peripatus hadn't been building due to me > > > missing a perl library. After I fixed that I get failures on: > > > > > > Buildfarm member peripatus failed on REL9_3_STABLE stage Make > > > Buildfarm member peripatus failed on REL9_4_STABLE stage Make > > > Buildfarm member peripatus failed on REL9_5_STABLE stage Make > > > Buildfarm member peripatus failed on REL9_6_STABLE stage Make > > > Buildfarm member peripatus failed on REL_10_STABLE stage Make > > > > > > HEAD and REL_11_STABLE build fine. > > > > > > These all fail on relocation issues in pgport(strcase*) and possibly > > > others. > > > > > > This is FreeBSD HEAD as of today, clang 6.0.1, lld as the linker. > > > > > > Can someone take a look at these? > > > I've tried making sure all the cache's are clear, etc, same results. > > > > Seems to have something to do with the recent linker toolchain changes > > in FreeBSD. I don't actually understand what's going on here or why > > it doesn't affect REL_11_STABLE and HEAD, but this seems to be some > > kind of explanation + workaround (as used by the ports): > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219758 > > https://svnweb.freebsd.org/ports?view=revision&revision=456635 > > > I agree. Is there an easy way I can add this work around to peripatus' > source tree: > > It may be that adding "LDFLAGS+= -Wl,-z,notext" (and removing LLD_UNSAFE) will let the port build with lld. > > Thanks, Thomas. BTW: If some hacker wants to play on this box, I'm happy to make an account and give ssh access. It's a playground. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
Attachment
On Thu, Jul 5, 2018 at 12:35 PM, Larry Rosenman <ler@lerctr.org> wrote:
> I agree. Is there an easy way I can add this work around to peripatus'
> source tree:
>
> It may be that adding "LDFLAGS+= -Wl,-z,notext" (and removing LLD_UNSAFE) will let the port build with lld.
Maybe something like this at the end of your build-farm.conf?
if ($branch =~ /^REL(9|_10)/)
{
$conf{config_env}->{"LDFLAGS"} = "...something something...";
}
--
Thomas Munro
http://www.enterprisedb.com
On Thu, Jul 05, 2018 at 12:56:49PM +1200, Thomas Munro wrote:
> On Thu, Jul 5, 2018 at 12:35 PM, Larry Rosenman <ler@lerctr.org> wrote:
> > I agree. Is there an easy way I can add this work around to peripatus'
> > source tree:
> >
> > It may be that adding "LDFLAGS+= -Wl,-z,notext" (and removing LLD_UNSAFE) will let the port build with lld.
>
> Maybe something like this at the end of your build-farm.conf?
>
> if ($branch =~ /^REL(9|_10)/)
> {
> $conf{config_env}->{"LDFLAGS"} = "...something something...";
> }
>
Good news. I ran a quick build test on my checked out FreeBSD ports
tree and with Ed Maste's suggestion, it builds.
Ed's suggestion:
remove LLD_UNSAFE, and add to the LDFLAGS+= line in the port
-Wl,-z,notext.
So, that is indeed a fix for us. My question is:
how to add this LDFLAG for FreeBSD >= 12 and PostgreSQL <= 11's configure et al
I'm more than willing to try and generate a patch, but would like some
guidance. I'm also willing to give access to my box.
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: ler@lerctr.org
US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
Attachment
On Wed, Jul 04, 2018 at 08:19:48PM -0500, Larry Rosenman wrote:
> On Thu, Jul 05, 2018 at 12:56:49PM +1200, Thomas Munro wrote:
> > On Thu, Jul 5, 2018 at 12:35 PM, Larry Rosenman <ler@lerctr.org> wrote:
> > > I agree. Is there an easy way I can add this work around to peripatus'
> > > source tree:
> > >
> > > It may be that adding "LDFLAGS+= -Wl,-z,notext" (and removing LLD_UNSAFE) will let the port build with lld.
> >
> > Maybe something like this at the end of your build-farm.conf?
> >
> > if ($branch =~ /^REL(9|_10)/)
> > {
> > $conf{config_env}->{"LDFLAGS"} = "...something something...";
> > }
> >
>
> Good news. I ran a quick build test on my checked out FreeBSD ports
> tree and with Ed Maste's suggestion, it builds.
>
> Ed's suggestion:
> remove LLD_UNSAFE, and add to the LDFLAGS+= line in the port
> -Wl,-z,notext.
>
> So, that is indeed a fix for us. My question is:
> how to add this LDFLAG for FreeBSD >= 12 and PostgreSQL <= 11's configure et al
>
> I'm more than willing to try and generate a patch, but would like some
> guidance. I'm also willing to give access to my box.
>
>
I also filed FreeBSD pr
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229523 to make the
change in the FreeBSD port.
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: ler@lerctr.org
US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
Attachment
On Wed, Jul 04, 2018 at 08:37:40PM -0500, Larry Rosenman wrote:
> On Wed, Jul 04, 2018 at 08:19:48PM -0500, Larry Rosenman wrote:
> > On Thu, Jul 05, 2018 at 12:56:49PM +1200, Thomas Munro wrote:
> > > On Thu, Jul 5, 2018 at 12:35 PM, Larry Rosenman <ler@lerctr.org> wrote:
> > > > I agree. Is there an easy way I can add this work around to peripatus'
> > > > source tree:
> > > >
> > > > It may be that adding "LDFLAGS+= -Wl,-z,notext" (and removing LLD_UNSAFE) will let the port build with lld.
> > >
> > > Maybe something like this at the end of your build-farm.conf?
> > >
> > > if ($branch =~ /^REL(9|_10)/)
> > > {
> > > $conf{config_env}->{"LDFLAGS"} = "...something something...";
> > > }
> > >
> >
> > Good news. I ran a quick build test on my checked out FreeBSD ports
> > tree and with Ed Maste's suggestion, it builds.
> >
> > Ed's suggestion:
> > remove LLD_UNSAFE, and add to the LDFLAGS+= line in the port
> > -Wl,-z,notext.
> >
> > So, that is indeed a fix for us. My question is:
> > how to add this LDFLAG for FreeBSD >= 12 and PostgreSQL <= 11's configure et al
> >
> > I'm more than willing to try and generate a patch, but would like some
> > guidance. I'm also willing to give access to my box.
> >
> >
>
> I also filed FreeBSD pr
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229523 to make the
> change in the FreeBSD port.
>
I suspect HEAD and REL_11 are ok due to changes in the pgport source.
(Someone with better git foo would have to check that).
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: ler@lerctr.org
US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
Attachment
On Thu, Jul 05, 2018 at 07:47:39AM -0500, Larry Rosenman wrote:
> On Wed, Jul 04, 2018 at 08:37:40PM -0500, Larry Rosenman wrote:
> > On Wed, Jul 04, 2018 at 08:19:48PM -0500, Larry Rosenman wrote:
> > > On Thu, Jul 05, 2018 at 12:56:49PM +1200, Thomas Munro wrote:
> > > > On Thu, Jul 5, 2018 at 12:35 PM, Larry Rosenman <ler@lerctr.org> wrote:
> > > > > I agree. Is there an easy way I can add this work around to peripatus'
> > > > > source tree:
> > > > >
> > > > > It may be that adding "LDFLAGS+= -Wl,-z,notext" (and removing LLD_UNSAFE) will let the port build with lld.
> > > >
> > > > Maybe something like this at the end of your build-farm.conf?
> > > >
> > > > if ($branch =~ /^REL(9|_10)/)
> > > > {
> > > > $conf{config_env}->{"LDFLAGS"} = "...something something...";
> > > > }
> > > >
> > >
> > > Good news. I ran a quick build test on my checked out FreeBSD ports
> > > tree and with Ed Maste's suggestion, it builds.
> > >
> > > Ed's suggestion:
> > > remove LLD_UNSAFE, and add to the LDFLAGS+= line in the port
> > > -Wl,-z,notext.
> > >
> > > So, that is indeed a fix for us. My question is:
> > > how to add this LDFLAG for FreeBSD >= 12 and PostgreSQL <= 11's configure et al
> > >
> > > I'm more than willing to try and generate a patch, but would like some
> > > guidance. I'm also willing to give access to my box.
> > >
> > >
> >
> > I also filed FreeBSD pr
> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229523 to make the
> > change in the FreeBSD port.
> >
>
> I suspect HEAD and REL_11 are ok due to changes in the pgport source.
> (Someone with better git foo would have to check that).
>
>
anyone want to look at this, or at least give me a clue on how to add
this to 10 & below?
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: ler@lerctr.org
US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
Attachment
Larry Rosenman <ler@lerctr.org> writes:
> anyone want to look at this, or at least give me a clue on how to add
> this to 10 & below?
I do not like the "-Wl,-z,notext" thing at all. It fails to explain
why things are working OK in v11/HEAD, which makes me think that it's
band-aiding something rather than really fixing it.
Perhaps you could spend a bit of time with git bisect and find out
which commit un-broke things? That should help us narrow down the
true explanation.
regards, tom lane
On Fri, Jul 06, 2018 at 06:40:49PM -0400, Tom Lane wrote: > Larry Rosenman <ler@lerctr.org> writes: > > anyone want to look at this, or at least give me a clue on how to add > > this to 10 & below? > > I do not like the "-Wl,-z,notext" thing at all. It fails to explain > why things are working OK in v11/HEAD, which makes me think that it's > band-aiding something rather than really fixing it. > > Perhaps you could spend a bit of time with git bisect and find out > which commit un-broke things? That should help us narrow down the > true explanation. > > regards, tom lane Following the advice in the error message, the following ALSO fixes it (REL_10_STABLE): borg.lerctr.org /home/ler/Git/postgresql/src/port $ git diff diff --git a/src/port/Makefile b/src/port/Makefile index 81f01b25bb..9ef00b3d54 100644 --- a/src/port/Makefile +++ b/src/port/Makefile @@ -28,6 +28,7 @@ top_builddir = ../.. include $(top_builddir)/src/Makefile.global override CPPFLAGS := -I$(top_builddir)/src/port -DFRONTEND $(CPPFLAGS) +override CFLAGS := -fPIC LIBS += $(PTHREAD_LIBS) OBJS = $(LIBOBJS) $(PG_CRC32C_OBJS) chklocale.o erand48.o inet_net_ntop.o \ borg.lerctr.org /home/ler/Git/postgresql/src/port $ Is that more acceptable? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
Attachment
On Fri, Jul 06, 2018 at 06:05:36PM -0500, Larry Rosenman wrote: > On Fri, Jul 06, 2018 at 06:40:49PM -0400, Tom Lane wrote: > > Larry Rosenman <ler@lerctr.org> writes: > > > anyone want to look at this, or at least give me a clue on how to add > > > this to 10 & below? > > > > I do not like the "-Wl,-z,notext" thing at all. It fails to explain > > why things are working OK in v11/HEAD, which makes me think that it's > > band-aiding something rather than really fixing it. > > > > Perhaps you could spend a bit of time with git bisect and find out > > which commit un-broke things? That should help us narrow down the > > true explanation. > > > > regards, tom lane > > Following the advice in the error message, the following ALSO fixes it > (REL_10_STABLE): > > borg.lerctr.org /home/ler/Git/postgresql/src/port $ git diff > diff --git a/src/port/Makefile b/src/port/Makefile > index 81f01b25bb..9ef00b3d54 100644 > --- a/src/port/Makefile > +++ b/src/port/Makefile > @@ -28,6 +28,7 @@ top_builddir = ../.. > include $(top_builddir)/src/Makefile.global > > override CPPFLAGS := -I$(top_builddir)/src/port -DFRONTEND $(CPPFLAGS) > +override CFLAGS := -fPIC > LIBS += $(PTHREAD_LIBS) > > OBJS = $(LIBOBJS) $(PG_CRC32C_OBJS) chklocale.o erand48.o inet_net_ntop.o \ > borg.lerctr.org /home/ler/Git/postgresql/src/port $ > > Is that more acceptable? Err, add a $(CFLAGS) to the end of the added line. > -- > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 E-Mail: ler@lerctr.org > US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
Attachment
Larry Rosenman <ler@lerctr.org> writes:
> On Fri, Jul 06, 2018 at 06:40:49PM -0400, Tom Lane wrote:
>> I do not like the "-Wl,-z,notext" thing at all. It fails to explain
>> why things are working OK in v11/HEAD, which makes me think that it's
>> band-aiding something rather than really fixing it.
> Following the advice in the error message, the following ALSO fixes it
> (REL_10_STABLE):
> +override CFLAGS := -fPIC
Yeah, I wondered about whether that wasn't a cleaner answer, but the same
problem remains: if we need that in src/port/, why don't all the branches
need it? It would be unsurprising if we'd gained a requirement for -fPIC
somewhere along the line, but I don't entirely believe that we lost one.
So I'd still like to know when this changed.
regards, tom lane
On Fri, Jul 06, 2018 at 07:18:46PM -0400, Tom Lane wrote: > Larry Rosenman <ler@lerctr.org> writes: > > On Fri, Jul 06, 2018 at 06:40:49PM -0400, Tom Lane wrote: > >> I do not like the "-Wl,-z,notext" thing at all. It fails to explain > >> why things are working OK in v11/HEAD, which makes me think that it's > >> band-aiding something rather than really fixing it. > > > Following the advice in the error message, the following ALSO fixes it > > (REL_10_STABLE): > > +override CFLAGS := -fPIC > > Yeah, I wondered about whether that wasn't a cleaner answer, but the same > problem remains: if we need that in src/port/, why don't all the branches > need it? It would be unsurprising if we'd gained a requirement for -fPIC > somewhere along the line, but I don't entirely believe that we lost one. > So I'd still like to know when this changed. > BROKE:0c62356cc8777961221a643fa77f62e1c7361085 GOOD: f044d71e331d77a0039cec0a11859b5a3c72bc95 f044d71e331d77a0039cec0a11859b5a3c72bc95 fixed it. Here's the revs I tried: BROKE:9d4649ca49416111aee2c84b7e4441a0b7aa2fac GOOD: 3b8f6e75f3c8c6d192621f21624cc8cee04ec3cb GOOD: 3a5e0a91bb324ad2b2b1a0623a3f2e37772b43fc GOOD: 8989f52b1b0636969545e6c8f6c813bc563ebcf5 BROKE:0c62356cc8777961221a643fa77f62e1c7361085 GOOD: f044d71e331d77a0039cec0a11859b5a3c72bc95 > regards, tom lane -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
Attachment
Larry Rosenman <ler@lerctr.org> writes:
> f044d71e331d77a0039cec0a11859b5a3c72bc95 fixed it.
Don't think I believe that conclusion; that patch shouldn't
have affected anything at all for non-ARM architectures.
regards, tom lane
On Sat, Jul 07, 2018 at 11:11:24AM -0400, Tom Lane wrote: > Larry Rosenman <ler@lerctr.org> writes: > > f044d71e331d77a0039cec0a11859b5a3c72bc95 fixed it. > > Don't think I believe that conclusion; that patch shouldn't > have affected anything at all for non-ARM architectures. > > regards, tom lane Those are the 2 adjacent commits in src/port. gitlog, and the 2 gmake outputs at: https://www.lerctr.org/~ler/PostgreSQL/ -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
Attachment
Larry Rosenman <ler@lerctr.org> writes:
> On Sat, Jul 07, 2018 at 11:11:24AM -0400, Tom Lane wrote:
>> Larry Rosenman <ler@lerctr.org> writes:
>>> f044d71e331d77a0039cec0a11859b5a3c72bc95 fixed it.
>> Don't think I believe that conclusion; that patch shouldn't
>> have affected anything at all for non-ARM architectures.
> Those are the 2 adjacent commits in src/port.
You're assuming something not in evidence, which is that the critical
change was textually within src/port/. It might have been in configure,
for instance, or in Makefile.global, or some other upper-level makefile.
I'd just do a straight bisect run without any assumptions about which part
of the tree matters.
regards, tom lane
On Sat, Jul 07, 2018 at 11:24:48AM -0400, Tom Lane wrote:
> Larry Rosenman <ler@lerctr.org> writes:
> > On Sat, Jul 07, 2018 at 11:11:24AM -0400, Tom Lane wrote:
> >> Larry Rosenman <ler@lerctr.org> writes:
> >>> f044d71e331d77a0039cec0a11859b5a3c72bc95 fixed it.
>
> >> Don't think I believe that conclusion; that patch shouldn't
> >> have affected anything at all for non-ARM architectures.
>
> > Those are the 2 adjacent commits in src/port.
>
> You're assuming something not in evidence, which is that the critical
> change was textually within src/port/. It might have been in configure,
> for instance, or in Makefile.global, or some other upper-level makefile.
> I'd just do a straight bisect run without any assumptions about which part
> of the tree matters.
>
> regards, tom lane
And the winner is:
dddfc4cb2edcfa5497f5d50190a7fb046c51da16 is the first bad commit
commit dddfc4cb2edcfa5497f5d50190a7fb046c51da16
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Tue Apr 3 16:26:05 2018 -0400
Prevent accidental linking of system-supplied copies of libpq.so etc.
We were being careless in some places about the order of -L switches in
link command lines, such that -L switches referring to external directories
could come before those referring to directories within the build tree.
This made it possible to accidentally link a system-supplied library, for
example /usr/lib/libpq.so, in place of the one built in the build tree.
Hilarity ensued, the more so the older the system-supplied library is.
To fix, break LDFLAGS into two parts, a sub-variable LDFLAGS_INTERNAL
and the main LDFLAGS variable, both of which are "recursively expanded"
so that they can be incrementally adjusted by different makefiles.
Establish a policy that -L switches for directories in the build tree
must always be added to LDFLAGS_INTERNAL, while -L switches for external
directories must always be added to LDFLAGS. This is sufficient to
ensure a safe search order. For simplicity, we typically also put -l
switches for the respective libraries into those same variables.
(Traditional make usage would have us put -l switches into LIBS, but
cleaning that up is a project for another day, as there's no clear
need for it.)
This turns out to also require separating SHLIB_LINK into two variables,
SHLIB_LINK and SHLIB_LINK_INTERNAL, with a similar rule about which
switches go into which variable. And likewise for PG_LIBS.
Although this change might appear to affect external users of pgxs.mk,
I think it doesn't; they shouldn't have any need to touch the _INTERNAL
variables.
In passing, tweak src/common/Makefile so that the value of CPPFLAGS
recorded in pg_config lacks "-DFRONTEND" and the recorded value of
LDFLAGS lacks "-L../../../src/common". Both of those things are
mistakes, apparently introduced during prior code rearrangements,
as old versions of pg_config don't print them. In general we don't
want anything that's specific to the src/common subdirectory to
appear in those outputs.
This is certainly a bug fix, but in view of the lack of field
complaints, I'm unsure whether it's worth the risk of back-patching.
In any case it seems wise to see what the buildfarm makes of it first.
Discussion: https://postgr.es/m/25214.1522604295@sss.pgh.pa.us
:040000 040000 49374c1617930b2ff57015188d2f06ff15198fb0 c07d24d934ac3f8e8a60e533953b1ea1a1411824 M contrib
:040000 040000 d00cbb0746730fabc32828a839033fbe6bb1aba8 51f1c15d4802aaae90e527eeca38611b73802f47 M src
borg.lerctr.org /home/ler/Git/postgresql $
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: ler@lerctr.org
US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
Attachment
Larry Rosenman <ler@lerctr.org> writes:
> And the winner is:
> dddfc4cb2edcfa5497f5d50190a7fb046c51da16 is the first bad commit
> commit dddfc4cb2edcfa5497f5d50190a7fb046c51da16
> Author: Tom Lane <tgl@sss.pgh.pa.us>
> Date: Tue Apr 3 16:26:05 2018 -0400
>
> Prevent accidental linking of system-supplied copies of libpq.so etc.
Huh. So what that suggests is that the problem is related to picking
up copies of our libraries from outside the build tree. Do you have
any copies of libpgport.a/.so or libpgcommon.a/.so in
/usr/local/lib or /usr/lib or /lib ?
regards, tom lane
On Sat, Jul 07, 2018 at 02:28:17PM -0400, Tom Lane wrote: > Larry Rosenman <ler@lerctr.org> writes: > > And the winner is: > > > dddfc4cb2edcfa5497f5d50190a7fb046c51da16 is the first bad commit > > commit dddfc4cb2edcfa5497f5d50190a7fb046c51da16 > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > Date: Tue Apr 3 16:26:05 2018 -0400 > > > > Prevent accidental linking of system-supplied copies of libpq.so etc. > > Huh. So what that suggests is that the problem is related to picking > up copies of our libraries from outside the build tree. Do you have > any copies of libpgport.a/.so or libpgcommon.a/.so in > /usr/local/lib or /usr/lib or /lib ? > > regards, tom lane Yes.... borg.lerctr.org /home/ler $ ls /usr/local/lib/libpg* /usr/local/lib/libpgcommon.a /usr/local/lib/libpgport.a /usr/local/lib/libpgtypes.a /usr/local/lib/libpgtypes.so /usr/local/lib/libpgtypes.so.3 borg.lerctr.org /home/ler $ ls -l /usr/local/lib/libpg* -rw-r--r-- 1 root wheel 190410 Jun 29 12:11 /usr/local/lib/libpgcommon.a -rw-r--r-- 1 root wheel 80234 May 11 19:08 /usr/local/lib/libpgport.a -rw-r--r-- 1 root wheel 112312 May 11 19:08 /usr/local/lib/libpgtypes.a lrwxr-xr-x 1 root wheel 15 May 11 19:08 /usr/local/lib/libpgtypes.so -> libpgtypes.so.3 -rwxr-xr-x 1 root wheel 77726 May 11 19:08 /usr/local/lib/libpgtypes.so.3 borg.lerctr.org /home/ler $<Paste> -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
Attachment
On Sat, Jul 07, 2018 at 01:33:26PM -0500, Larry Rosenman wrote: > On Sat, Jul 07, 2018 at 02:28:17PM -0400, Tom Lane wrote: > > Larry Rosenman <ler@lerctr.org> writes: > > > And the winner is: > > > > > dddfc4cb2edcfa5497f5d50190a7fb046c51da16 is the first bad commit > > > commit dddfc4cb2edcfa5497f5d50190a7fb046c51da16 > > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > > Date: Tue Apr 3 16:26:05 2018 -0400 > > > > > > Prevent accidental linking of system-supplied copies of libpq.so etc. > > > > Huh. So what that suggests is that the problem is related to picking > > up copies of our libraries from outside the build tree. Do you have > > any copies of libpgport.a/.so or libpgcommon.a/.so in > > /usr/local/lib or /usr/lib or /lib ? > > > > regards, tom lane > > Yes.... > borg.lerctr.org /home/ler $ ls /usr/local/lib/libpg* > /usr/local/lib/libpgcommon.a /usr/local/lib/libpgport.a /usr/local/lib/libpgtypes.a /usr/local/lib/libpgtypes.so /usr/local/lib/libpgtypes.so.3 > borg.lerctr.org /home/ler $ ls -l /usr/local/lib/libpg* > -rw-r--r-- 1 root wheel 190410 Jun 29 12:11 /usr/local/lib/libpgcommon.a > -rw-r--r-- 1 root wheel 80234 May 11 19:08 /usr/local/lib/libpgport.a > -rw-r--r-- 1 root wheel 112312 May 11 19:08 /usr/local/lib/libpgtypes.a > lrwxr-xr-x 1 root wheel 15 May 11 19:08 /usr/local/lib/libpgtypes.so -> libpgtypes.so.3 > -rwxr-xr-x 1 root wheel 77726 May 11 19:08 /usr/local/lib/libpgtypes.so.3 > borg.lerctr.org /home/ler $<Paste> > And: borg.lerctr.org /home/ler $ pkg which -g /usr/local/lib/libpgport.a /usr/local/lib/libpgport.a was installed by package postgresql10-client-10.4 borg.lerctr.org /home/ler $ -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
I wrote:
> Huh. So what that suggests is that the problem is related to picking
> up copies of our libraries from outside the build tree. Do you have
> any copies of libpgport.a/.so or libpgcommon.a/.so in
> /usr/local/lib or /usr/lib or /lib ?
Ah, no, scratch that, I see the problem. In v10, peripatus builds
insert_username.so like this:
ccache clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels
-Wmissing-format-attribute-Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -g -O2 -fPIC
-DPIC-L../../src/port -L../../src/common -L/usr/local/lib -L/usr/local/lib -L/usr/lib -Wl,--as-needed
-Wl,-R'/home/pgbuildfarm/buildroot/REL_10_STABLE/inst/lib'-L../../src/port -lpgport -shared -o insert_username.so
insert_username.o
while v11 does it like this:
ccache clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels
-Wmissing-format-attribute-Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -g -O2 -fPIC
-DPIC-L../../src/port -L../../src/common -L/usr/local/lib -L/usr/local/lib -L/usr/lib -Wl,--as-needed
-Wl,-R'/home/pgbuildfarm/buildroot/REL_11_STABLE/inst/lib' -shared -o insert_username.so insert_username.o
In the first case, we're trying to include code from libpgport.a directly
into the .so, and since libpgport.a isn't built with -fPIC, it blows up.
In the second case, we *don't* link libpgport.a here at all. Rather,
any symbols that insert_username.so needs from that library will be
resolved in the main backend's copy of the library, for which
relocatability isn't required.
Note that there's a second problem with the way this is happening pre-v11:
for any src/port/ file that compiles different logic for frontend and
backend, insert_username.so would be picking up the wrong logic. Perhaps
we've not noticed because that module doesn't use any files in which
there's a meaningful difference, but there's an obvious hazard there.
Fooling with -fPIC isn't enough to fix it.
I'd been hesitant to back-patch dddfc4cb2 back in April; but now that
it's survived some beta testing, I think that doing so seems like the
most appropriate way to fix this.
regards, tom lane
I wrote:
> I'd been hesitant to back-patch dddfc4cb2 back in April; but now that
> it's survived some beta testing, I think that doing so seems like the
> most appropriate way to fix this.
Done. Hopefully I didn't break anything; a lot of this code has mutated
to some extent since 9.3. But I expect the buildfarm will point out any
problems.
regards, tom lane
On Mon, Jul 09, 2018 at 05:25:50PM -0400, Tom Lane wrote: > I wrote: > > I'd been hesitant to back-patch dddfc4cb2 back in April; but now that > > it's survived some beta testing, I think that doing so seems like the > > most appropriate way to fix this. > > Done. Hopefully I didn't break anything; a lot of this code has mutated > to some extent since 9.3. But I expect the buildfarm will point out any > problems. The reason you might not have seen it on FreeBSD before is that FreeBSD 12 now uses lld (llvm's LD) to link, and it changed the default for -z. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
Attachment
On 2018-Jul-09, Larry Rosenman wrote: > On Mon, Jul 09, 2018 at 05:25:50PM -0400, Tom Lane wrote: > > I wrote: > > > I'd been hesitant to back-patch dddfc4cb2 back in April; but now that > > > it's survived some beta testing, I think that doing so seems like the > > > most appropriate way to fix this. > > > > Done. Hopefully I didn't break anything; a lot of this code has mutated > > to some extent since 9.3. But I expect the buildfarm will point out any > > problems. > > The reason you might not have seen it on FreeBSD before is that FreeBSD > 12 now uses lld (llvm's LD) to link, and it changed the default for -z. So, has the hack for -z notext been removed from the freebsd port build? -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
It was never put into the build, and I have a PR open to remove the LLD_UNSAFE flag for 10.5 and the rest of today's releases. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229523 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106 On 8/9/18, 3:25 PM, "Alvaro Herrera" <alvherre@2ndquadrant.com> wrote: On 2018-Jul-09, Larry Rosenman wrote: > On Mon, Jul 09, 2018 at 05:25:50PM -0400, Tom Lane wrote: > > I wrote: > > > I'd been hesitant to back-patch dddfc4cb2 back in April; but now that > > > it's survived some beta testing, I think that doing so seems like the > > > most appropriate way to fix this. > > > > Done. Hopefully I didn't break anything; a lot of this code has mutated > > to some extent since 9.3. But I expect the buildfarm will point out any > > problems. > > The reason you might not have seen it on FreeBSD before is that FreeBSD > 12 now uses lld (llvm's LD) to link, and it changed the default for -z. So, has the hack for -z notext been removed from the freebsd port build? -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services