Thread: Re: LLVM strip -x fails

Re: LLVM strip -x fails

From
Andres Freund
Date:
Hi,

Moving this to -hackers.

On 2023-04-20 11:49:23 +0200, Palle Girgensohn wrote:
> I was recently made aware of a problem building postgresql using LLVM binutils.
> 
> A summary:
> 
> --
> 
> pgsql's build has requested to strip all non-global symbols (strip -x), but
> there is at least one non-global symbol that in fact cannot be stripped
> because it is referenced by a relocation.

An important detail here is that this happens when stripping static
libraries. It's not too surprising that one needs the symbols referenced by
relocations until after linking with the static lib, even if they're not
globally visible symbols.


> Both GNU strip and ELF Tool Chain strip silently handle this case (and just retain the local symbol), but LLVM strip
isstricter and emits an error upon request to strip a non-removable local symbol.
 
> 
> There is an LLVM ticket open for this at https://github.com/llvm/llvm-project/issues/47468, and it may make sense for
LLVMstrip to behave the same as GNU and ELF Tool Chain strip. That said, pgsql should just not use strip -x when there
aresymbols that cannot be stripped.
 

Personally I'd say stripping symbols is something that should just not be done
anymore, but ...

> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270769
> 
> https://reviews.freebsd.org/D39590
> 
> 
> Any toughts about this? Should send the suggested patch upstreams? Or do we just consider LLVM to behave badly?

Peter, it's unlikely given the timeframe, but do you happen to remember why
you specified -x when stripping static libs? This seems to be all the way back
from

commit 563673e15db995b6f531b44be7bb162330ac157a
Author: Peter Eisentraut <peter_e@gmx.net>
Date:   2002-04-10 16:45:25 +0000

    Add make install-strip target.


Afaict the only safe thing to use when stripping static libs is
-g/--strip-debug.

Greetings,

Andres Freund



Re: LLVM strip -x fails

From
Tom Lane
Date:
Andres Freund <andres@anarazel.de> writes:
> Afaict the only safe thing to use when stripping static libs is
> -g/--strip-debug.

The previous complaint about this [1] suggested we use --strip-unneeded
for all cases with GNU strip, same as we've long done for shared libs.
It's an easy enough change, but I wonder if anyone will complain.

            regards, tom lane

[1] https://www.postgresql.org/message-id/17898-5308d09543463266%40postgresql.org



Re: LLVM strip -x fails

From
Andres Freund
Date:
Hi,

On 2023-04-20 12:43:48 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > Afaict the only safe thing to use when stripping static libs is
> > -g/--strip-debug.
>
> The previous complaint about this [1] suggested we use --strip-unneeded
> for all cases with GNU strip, same as we've long done for shared libs.
> It's an easy enough change, but I wonder if anyone will complain.

--strip-unneeded output is smaller than -x output:

19M  src/interfaces/libpq/libpq.a
364K src/interfaces/libpq/libpq.a.strip.gnu.g
352K src/interfaces/libpq/libpq.a.strip.gnu.unneeded
356K src/interfaces/libpq/libpq.a.strip.gnu.x
352K src/interfaces/libpq/libpq.a.strip.gnu.x.g

strip --version
GNU strip (GNU Binutils for Debian) 2.40

Partially that's because --strip-unneeded implies -g. Interestingly -x -g
output isn't quite the same as --strip-unneeded. The latter also removes the
_GLOBAL_OFFSET_TABLE_ symbol.


I doubt anybody wants to strip symbols and keep debug information, so I doubt
there's much ground for complaints?


Oddly the output of llvm-strip confuses binutils objdump enough that it claims
that "file format not recognized". Not sure which side is broken there.

llvm-strip's output is a lot larger than gnu strip's:
 19M src/interfaces/libpq/libpq.a
 19M src/interfaces/libpq/libpq.a.strip.llvm.X
908K src/interfaces/libpq/libpq.a.strip.llvm.g
892K src/interfaces/libpq/libpq.a.strip.llvm.unneeded
892K src/interfaces/libpq/libpq.a.strip.llvm.unneeded.g
364K src/interfaces/libpq/libpq.a.strip.gnu.g
356K src/interfaces/libpq/libpq.a.strip.gnu.x
352K src/interfaces/libpq/libpq.a.strip.gnu.x.g
352K src/interfaces/libpq/libpq.a.strip.gnu.unneeded

Greetings,

Andres Freund



Re: LLVM strip -x fails

From
Tom Lane
Date:
Andres Freund <andres@anarazel.de> writes:
> On 2023-04-20 12:43:48 -0400, Tom Lane wrote:
>> The previous complaint about this [1] suggested we use --strip-unneeded
>> for all cases with GNU strip, same as we've long done for shared libs.
>> It's an easy enough change, but I wonder if anyone will complain.

> I doubt anybody wants to strip symbols and keep debug information, so I doubt
> there's much ground for complaints?

Agreed.  It doesn't look like --strip-unneeded is a worse choice than -x.

> Oddly the output of llvm-strip confuses binutils objdump enough that it claims
> that "file format not recognized". Not sure which side is broken there.

Not our problem, I'd say ...

> llvm-strip's output is a lot larger than gnu strip's:

... nor that.  These things do suggest that llvm-strip isn't all that
close to being ready for prime time, but if FreeBSD wants to push the
envelope on toolchain usage, who are we to stand in the way?

I'll go make it so.

            regards, tom lane



Re: LLVM strip -x fails

From
Peter Eisentraut
Date:
On 20.04.23 17:33, Andres Freund wrote:
> Peter, it's unlikely given the timeframe, but do you happen to remember why
> you specified -x when stripping static libs? This seems to be all the way back
> from
> 
> commit 563673e15db995b6f531b44be7bb162330ac157a
> Author: Peter Eisentraut<peter_e@gmx.net>
> Date:   2002-04-10 16:45:25 +0000
> 
>      Add make install-strip target.
> 
> 
> Afaict the only safe thing to use when stripping static libs is
> -g/--strip-debug.

I suspect this was copied from GNU Libtool.  Libtool still has that but 
later changed the stripping of static libraries on darwin to "strip -S". 
  Maybe should adopt that.




Re: LLVM strip -x fails

From
Peter Eisentraut
Date:
On 21.04.23 18:41, Peter Eisentraut wrote:
> On 20.04.23 17:33, Andres Freund wrote:
>> Peter, it's unlikely given the timeframe, but do you happen to 
>> remember why
>> you specified -x when stripping static libs? This seems to be all the 
>> way back
>> from
>>
>> commit 563673e15db995b6f531b44be7bb162330ac157a
>> Author: Peter Eisentraut<peter_e@gmx.net>
>> Date:   2002-04-10 16:45:25 +0000
>>
>>      Add make install-strip target.
>>
>>
>> Afaict the only safe thing to use when stripping static libs is
>> -g/--strip-debug.
> 
> I suspect this was copied from GNU Libtool.  Libtool still has that but 
> later changed the stripping of static libraries on darwin to "strip -S". 
>   Maybe should adopt that.

Here is the current logic in GNU Libtool:

https://github.com/autotools-mirror/libtool/blob/master/m4/libtool.m4#L2214




Re: LLVM strip -x fails

From
Tom Lane
Date:
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
> On 20.04.23 17:33, Andres Freund wrote:
>> Peter, it's unlikely given the timeframe, but do you happen to remember why
>> you specified -x when stripping static libs?

> I suspect this was copied from GNU Libtool.  Libtool still has that but 
> later changed the stripping of static libraries on darwin to "strip -S". 
>   Maybe should adopt that.

I tried that, but it seems strictly worse on output file size:

$ ll lib*/libpq.a
-rw-r--r--  1 tgl  staff  715312 Apr 21 12:52 lib-no-strip/libpq.a
-rw-r--r--  1 tgl  staff  209984 Apr 21 12:51 lib-strip-S/libpq.a
-rw-r--r--  1 tgl  staff  208456 Apr 21 12:50 lib-strip-x/libpq.a
$ ll lib*/libecpg.a
-rw-r--r--  1 tgl  staff  324952 Apr 21 12:52 lib-no-strip/libecpg.a
-rw-r--r--  1 tgl  staff  102752 Apr 21 12:51 lib-strip-S/libecpg.a
-rw-r--r--  1 tgl  staff  102088 Apr 21 12:50 lib-strip-x/libecpg.a

If you use both -x and -S, you get the same file sizes as with -x
alone.  Not sure why we should change anything here.

            regards, tom lane



Re: LLVM strip -x fails

From
Peter Eisentraut
Date:
On 21.04.23 19:00, Tom Lane wrote:
> Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
>> On 20.04.23 17:33, Andres Freund wrote:
>>> Peter, it's unlikely given the timeframe, but do you happen to remember why
>>> you specified -x when stripping static libs?
> 
>> I suspect this was copied from GNU Libtool.  Libtool still has that but
>> later changed the stripping of static libraries on darwin to "strip -S".
>>    Maybe should adopt that.
> 
> I tried that, but it seems strictly worse on output file size:
> 
> $ ll lib*/libpq.a
> -rw-r--r--  1 tgl  staff  715312 Apr 21 12:52 lib-no-strip/libpq.a
> -rw-r--r--  1 tgl  staff  209984 Apr 21 12:51 lib-strip-S/libpq.a
> -rw-r--r--  1 tgl  staff  208456 Apr 21 12:50 lib-strip-x/libpq.a
> $ ll lib*/libecpg.a
> -rw-r--r--  1 tgl  staff  324952 Apr 21 12:52 lib-no-strip/libecpg.a
> -rw-r--r--  1 tgl  staff  102752 Apr 21 12:51 lib-strip-S/libecpg.a
> -rw-r--r--  1 tgl  staff  102088 Apr 21 12:50 lib-strip-x/libecpg.a
> 
> If you use both -x and -S, you get the same file sizes as with -x
> alone.  Not sure why we should change anything here.

The complaint was that -x doesn't work correctly, no?




Re: LLVM strip -x fails

From
Tom Lane
Date:
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
> On 21.04.23 19:00, Tom Lane wrote:
>> If you use both -x and -S, you get the same file sizes as with -x
>> alone.  Not sure why we should change anything here.

> The complaint was that -x doesn't work correctly, no?

The complaint was that it doesn't work correctly if you are using
llvm-strip.  However, llvm-strip will be picked up by the previous
test for GNU strip, independently of what platform you're on.
The code in question here is only concerned with Apple's strip.

            regards, tom lane