Thread: [HACKERS] building libpq.a static library

I maintain static libraries for libpq for the R programming language
(we need static linking to ship with the binary packages).
Unfortunately currently the standard postgres makefile only generates
a shared library for libpq, not a static one.

In order to make a static library I always manually edit
./src/interfaces/libpq/Makefile and add a target:

libpq.a: $(OBJS) ar rcs $@ $^

And then run 'make libpq.a' in the libpq directory.

This works but it's a bit of a pain to maintain. I was wondering if
this hack could be merged so that the standard 'configure
--enable-static' script would install a static library for libpq
alongside the shared one.

On Wednesday, July 12, 2017 6:31:09 AM EDT Jeroen Ooms wrote:
> I maintain static libraries for libpq for the R programming language
> (we need static linking to ship with the binary packages).
> Unfortunately currently the standard postgres makefile only generates
> a shared library for libpq, not a static one.
> 
> In order to make a static library I always manually edit
> ./src/interfaces/libpq/Makefile and add a target:
> 
> libpq.a: $(OBJS)
>   ar rcs $@ $^
> 
> And then run 'make libpq.a' in the libpq directory.
> 
> This works but it's a bit of a pain to maintain. I was wondering if
> this hack could be merged so that the standard 'configure
> --enable-static' script would install a static library for libpq
> alongside the shared one.

I have no idea what the consensus about this is, but providing a patch file and 
registering it at https://commitfest.postgresql.org/ increases your chances.

I wish you good luck hacking autoconf hell (which is what you'll need to do to 
get the command line switch) though (*shudder*).

Jeroen Ooms <jeroen@berkeley.edu> writes:
> I maintain static libraries for libpq for the R programming language
> (we need static linking to ship with the binary packages).

How do you get that past vendor packaging policies?  When I worked at
Red Hat, there was a very strong policy against allowing any package
to statically embed parts of another one, because it creates serious
management problems if e.g. the other one needs a security update.
I'm sure Red Hat isn't the only distro that feels that way.

I think you'd be better advised to fix things so you can link with
the standard shared-library version of libpq (and whatever else
you're doing this with).

> This works but it's a bit of a pain to maintain. I was wondering if
> this hack could be merged so that the standard 'configure
> --enable-static' script would install a static library for libpq
> alongside the shared one.

FWIW, we used to have support for building static libpq, but
we got rid of it a long time ago.  I couldn't find the exact
spot in some desultory trawling of the commit history.
        regards, tom lane

On Wed, Jul 12, 2017 at 5:11 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Jeroen Ooms <jeroen@berkeley.edu> writes:
>> I maintain static libraries for libpq for the R programming language
>> (we need static linking to ship with the binary packages).
>
> How do you get that past vendor packaging policies?  When I worked at
> Red Hat, there was a very strong policy against allowing any package
> to statically embed parts of another one, because it creates serious
> management problems if e.g. the other one needs a security update.
> I'm sure Red Hat isn't the only distro that feels that way.

We only use this on Windows. On platforms with a decent package
manager we indeed link to a shared library.

> FWIW, we used to have support for building static libpq, but
> we got rid of it a long time ago.

OK that's too bad. I agree that shared libs are often preferable but
in some environments dynamic linking is simply not possible and you
need to static link the library into the application. Most C/C++
libraries do support --enable-static and even for most linux distros
the static libs are included with the lib-dev / lib-devel package.

On 12 July 2017 at 16:11, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Jeroen Ooms <jeroen@berkeley.edu> writes:
>
>> This works but it's a bit of a pain to maintain. I was wondering if
>> this hack could be merged so that the standard 'configure
>> --enable-static' script would install a static library for libpq
>> alongside the shared one.
>
> FWIW, we used to have support for building static libpq, but
> we got rid of it a long time ago.  I couldn't find the exact
> spot in some desultory trawling of the commit history.

Fwiw I think the real problem is that building static libraries
"properly" requires different compiler options -- notably they're not
normally built with -fPIC. So that means building every object twice
which kind of breaks make's build model which has a simple dependency
graph where each object appears once. Some packages do this by
inventing a foo-shared.o and foo-static.o but that introduces its own
weirdness.

I don't know what the downsides would be of creating a static library
out of objects built with -fPIC. It might just be a small performance
penalty which might be no big deal for libpq. That may be a good
compromise.

-- 
greg

On 2017-07-12 23:55:56 +0100, Greg Stark wrote:
> Fwiw I think the real problem is that building static libraries
> "properly" requires different compiler options -- notably they're not
> normally built with -fPIC. So that means building every object twice
> which kind of breaks make's build model which has a simple dependency
> graph where each object appears once. Some packages do this by
> inventing a foo-shared.o and foo-static.o but that introduces its own
> weirdness.
> 
> I don't know what the downsides would be of creating a static library
> out of objects built with -fPIC. It might just be a small performance
> penalty which might be no big deal for libpq. That may be a good
> compromise.

FWIW, most linux distributions build everything with -fPIC/PIE anyway
these days, to allow address space randomization. So I don't think this
is a huge concern for modern platforms.

- Andres

On Thu, Jul 13, 2017 at 4:58 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
> You shouldn't ever need static libraries on Windows, though. Because it
> searches the CWD first on its linker search path, you can just drop
> libpq.dll in the same directory as your binary/library and link to the stub
> libpq.lib .

This is not possible in our case. The R programming language binaries
are installed in the "program files" directory which is only writable
by the sys admin.

There are over 10.000 contributed packages (including one with
postgres drivers) which are installed by the user in the home dir and
the package DLL's need to get dynamically loaded at runtime. We have
been working with this system for a very long time and static linking
external libs to the package DLL is really the only reliable way to
prevent DLL problems across Windows versions/configurations

> I'm not trying to block support for a static libpq, I just don't see the
> point.

This is a bit overgeneralized, there are many use cases where static
linking is the only feasible way. Most libs support --enable static
and many distros ship both static and shared libs and leave it up to
user or developer author how to configure their application.

On 7/12/17 11:11, Tom Lane wrote:
> FWIW, we used to have support for building static libpq, but
> we got rid of it a long time ago.  I couldn't find the exact
> spot in some desultory trawling of the commit history.

We still build and install static libraries.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
> On 7/12/17 11:11, Tom Lane wrote:
>> FWIW, we used to have support for building static libpq, but
>> we got rid of it a long time ago.  I couldn't find the exact
>> spot in some desultory trawling of the commit history.

> We still build and install static libraries.

Hm, I'd taken Jeroen's complaint at face value, but that sure
explains why I couldn't find where it'd been removed ;-)

The answer may then be that he's working with a vendor-packaged version
and the vendor chose to enforce their distro policy about not shipping
static libraries by patching that build step out of libpq's Makefile.
        regards, tom lane