Thread: Splitting libpq build

Splitting libpq build

From
Igor Korot
Date:
Hi, ALL,
I wonder who ade the decision not to provide an independent build
of the client library of PostgreSQL? And what was the reason behind it?

Its very annoying to build the whole server, just to create a client library
to connect to a remote DB. And then kill the build after the install as
useless.

Could someone please recommend a best way to have it build?

Thank you.



Re: Splitting libpq build

From
Tom Lane
Date:
Igor Korot <ikorot01@gmail.com> writes:
> I wonder who ade the decision not to provide an independent build
> of the client library of PostgreSQL? And what was the reason behind it?

It wasn't worth the maintenance trouble.  It still isn't.  On any
remotely-modern machine, a build of the whole tree is only going
to take a minute or two.  And libpq doesn't change so fast that
you'd need to do this often.

Having said that, it does work on non-Windows machines to do
    ./configure ...
    cd src/interfaces/libpq
    make -s install

The Windows case is exactly what we killed as not being worth
the maintenance effort, so I doubt you'll find much interest
in resurrecting that one.

            regards, tom lane



Re: Splitting libpq build

From
Igor Korot
Date:
Hi Tom,

On Sun, Oct 3, 2021 at 3:11 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Igor Korot <ikorot01@gmail.com> writes:
> > I wonder who ade the decision not to provide an independent build
> > of the client library of PostgreSQL? And what was the reason behind it?
>
> It wasn't worth the maintenance trouble.  It still isn't.  On any
> remotely-modern machine, a build of the whole tree is only going
> to take a minute or two.  And libpq doesn't change so fast that
> you'd need to do this often.

But building the whole tree is just that - useless.
All I need is the libpq to connect to the remote server.

So I don't need the server to be built - I need a client.

>
> Having said that, it does work on non-Windows machines to do
>         ./configure ...
>         cd src/interfaces/libpq
>         make -s install

Well, asking people to download the whole source tree just to build
client library 1000 times and then 1000 times delete the source code...

I don't know - I'd rather have to install just the library.

>
> The Windows case is exactly what we killed as not being worth
> the maintenance effort, so I doubt you'll find much interest
> in resurrecting that one.

So instead of simplifying the life of maintainers in the different distros
you made the life easier for yourself. ;;-)

Thank you.

>
>                         regards, tom lane



Re: Splitting libpq build

From
Adrian Klaver
Date:
On 10/3/21 1:20 PM, Igor Korot wrote:
> Hi Tom,
> 
> On Sun, Oct 3, 2021 at 3:11 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>
>> Igor Korot <ikorot01@gmail.com> writes:
>>> I wonder who ade the decision not to provide an independent build
>>> of the client library of PostgreSQL? And what was the reason behind it?

> But building the whole tree is just that - useless.
> All I need is the libpq to connect to the remote server.
> 
> So I don't need the server to be built - I need a client.

That is available from many of the packaging systems.

> 
>>
>> Having said that, it does work on non-Windows machines to do
>>          ./configure ...
>>          cd src/interfaces/libpq
>>          make -s install
> 
> Well, asking people to download the whole source tree just to build
> client library 1000 times and then 1000 times delete the source code...

If you are deploying to the same client you only need to build once.

If not then you need to be more specific about what you are doing?

> 
> I don't know - I'd rather have to install just the library.
> 

> So instead of simplifying the life of maintainers in the different distros
> you made the life easier for yourself. ;;-)

Not following:

apt-cache show libpq5
Package: libpq5
Source: postgresql-14
Version: 14.0-1.pgdg20.04+1
Architecture: amd64
Maintainer: Debian PostgreSQL Maintainers 
<team+postgresql@tracker.debian.org>
Installed-Size: 748
Depends: libc6 (>= 2.25), libgssapi-krb5-2 (>= 1.17), libldap-2.4-2 (>= 
2.4.7), libssl1.1 (>= 1.1.0)
Multi-Arch: same
Homepage: http://www.postgresql.org/
Priority: optional
Section: libs
Filename: pool/main/p/postgresql-14/libpq5_14.0-1.pgdg20.04+1_amd64.deb
Size: 169756
SHA256: 59be24749b3ee996af762dfad6ac8e915498f0bdb431612308548bac6a5324e0
SHA1: d002223d362a45959a7832bdddbd2ec04c4611ba
MD5sum: 2834edac0af08b7ffb4cc5093bb236b6
Description-en: PostgreSQL C client library
  libpq is a C library that enables user programs to communicate with
  the PostgreSQL database server.  The server can be on another machine
  and accessed through TCP/IP.  This version of libpq is compatible
  with servers from PostgreSQL 8.2 or later.
  .
  This package contains the run-time library, needed by packages using
  libpq.
  .
  PostgreSQL is an object-relational SQL database management system.
Description-md5: 4109d08c168c8058d09c0cc8bf8b18b1

> 
> Thank you.
> 
>>
>>                          regards, tom lane
> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com



Re: Splitting libpq build

From
Tom Lane
Date:
Igor Korot <ikorot01@gmail.com> writes:
> On Sun, Oct 3, 2021 at 3:11 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Having said that, it does work on non-Windows machines to do
>> ./configure ...
>> cd src/interfaces/libpq
>> make -s install

> Well, asking people to download the whole source tree just to build
> client library 1000 times and then 1000 times delete the source code...

Uh ... why are you installing from source at all, if that's your
use-case?  Most major distributions provide packages that subdivide
Postgres into client and server packages, and usually there's even
a package with just the library(s).

> So instead of simplifying the life of maintainers in the different distros
> you made the life easier for yourself. ;;-)

Well, yeah.  Packaging is a different business.  And, having *been*
a packager for Red Hat for a decade, it's not like I've never heard
of it.  But I don't see anything we need to do differently there.

            regards, tom lane



Re: Splitting libpq build

From
Igor Korot
Date:
Hi, Adrian,

On Sun, Oct 3, 2021 at 3:28 PM Adrian Klaver <adrian.klaver@aklaver.com> wrote:
>
> On 10/3/21 1:20 PM, Igor Korot wrote:
> > Hi Tom,
> >
> > On Sun, Oct 3, 2021 at 3:11 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >>
> >> Igor Korot <ikorot01@gmail.com> writes:
> >>> I wonder who ade the decision not to provide an independent build
> >>> of the client library of PostgreSQL? And what was the reason behind it?
>
> > But building the whole tree is just that - useless.
> > All I need is the libpq to connect to the remote server.
> >
> > So I don't need the server to be built - I need a client.
>
> That is available from many of the packaging systems.
>
> >
> >>
> >> Having said that, it does work on non-Windows machines to do
> >>          ./configure ...
> >>          cd src/interfaces/libpq
> >>          make -s install
> >
> > Well, asking people to download the whole source tree just to build
> > client library 1000 times and then 1000 times delete the source code...
>
> If you are deploying to the same client you only need to build once.
>
> If not then you need to be more specific about what you are doing?

Imagine you are a developer/maintainer of sme software. This software needs to
connect to different DBMSes perform soe operations and then disconnect.

It is very easy to say that I have SQLite as a submodule in my Git tree,
I have mysql-connector as a submodule, but I can't have libpq this way
because it is hidden deep down inside PostgreSQL sources.

So in order too package the software and let the users build it on
their machine,
I need to explicitly document that the user need to get the PostgreSQL sources,
build it, copy the headers and libraries as appropriate and then build
the software
I provide.

Do you know of a better way of handling this?

O, and of course I need this on all 3 major platforms - Windows, nix and OSX.

Thank you.

>
> >
> > I don't know - I'd rather have to install just the library.
> >
>
> > So instead of simplifying the life of maintainers in the different distros
> > you made the life easier for yourself. ;;-)
>
> Not following:
>
> apt-cache show libpq5
> Package: libpq5
> Source: postgresql-14
> Version: 14.0-1.pgdg20.04+1
> Architecture: amd64
> Maintainer: Debian PostgreSQL Maintainers
> <team+postgresql@tracker.debian.org>
> Installed-Size: 748
> Depends: libc6 (>= 2.25), libgssapi-krb5-2 (>= 1.17), libldap-2.4-2 (>=
> 2.4.7), libssl1.1 (>= 1.1.0)
> Multi-Arch: same
> Homepage: http://www.postgresql.org/
> Priority: optional
> Section: libs
> Filename: pool/main/p/postgresql-14/libpq5_14.0-1.pgdg20.04+1_amd64.deb
> Size: 169756
> SHA256: 59be24749b3ee996af762dfad6ac8e915498f0bdb431612308548bac6a5324e0
> SHA1: d002223d362a45959a7832bdddbd2ec04c4611ba
> MD5sum: 2834edac0af08b7ffb4cc5093bb236b6
> Description-en: PostgreSQL C client library
>   libpq is a C library that enables user programs to communicate with
>   the PostgreSQL database server.  The server can be on another machine
>   and accessed through TCP/IP.  This version of libpq is compatible
>   with servers from PostgreSQL 8.2 or later.
>   .
>   This package contains the run-time library, needed by packages using
>   libpq.
>   .
>   PostgreSQL is an object-relational SQL database management system.
> Description-md5: 4109d08c168c8058d09c0cc8bf8b18b1
>
> >
> > Thank you.
> >
> >>
> >>                          regards, tom lane
> >
> >
>
>
> --
> Adrian Klaver
> adrian.klaver@aklaver.com



Re: Splitting libpq build

From
Tom Lane
Date:
Igor Korot <ikorot01@gmail.com> writes:
> Imagine you are a developer/maintainer of sme software. This software needs to
> connect to different DBMSes perform soe operations and then disconnect.
> It is very easy to say that I have SQLite as a submodule in my Git tree,
> I have mysql-connector as a submodule, but I can't have libpq this way
> because it is hidden deep down inside PostgreSQL sources.

To be blunt, that is an absolutely horrid way to distribute software.
You should not, not, not be redistributing other people's code,
because that puts you on the hook to keep up with their bug fixes ---
particularly security-critical fixes, which do happen from time to time.
That's a loop you don't want to be in.

In this case I don't even see what's the point.  If somebody wants to
use your code with Postgres, it's just about certain that they already
have an installed copy of libpq, from wherever they get Postgres from.
The same argument applies to mysql or SQLite.  So you should be
expecting to link against already-present libraries, not bring your
own.  Do you bring your own libc, or openssl, or C compiler?

I can tell you for certain that if your package were to be picked up
and redistributed by Red Hat, or Debian, or any other open-source
vendor, the very first thing they'd do is strip it of any such copied
code.  They don't want the management hassle of having to update
multiple copies of libpq or mysql-connector or other stuff.

            regards, tom lane



Re: Splitting libpq build

From
Igor Korot
Date:
Hi, Tom,

On Sun, Oct 3, 2021 at 5:27 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Igor Korot <ikorot01@gmail.com> writes:
> > Imagine you are a developer/maintainer of sme software. This software needs to
> > connect to different DBMSes perform soe operations and then disconnect.
> > It is very easy to say that I have SQLite as a submodule in my Git tree,
> > I have mysql-connector as a submodule, but I can't have libpq this way
> > because it is hidden deep down inside PostgreSQL sources.
>
> To be blunt, that is an absolutely horrid way to distribute software.
> You should not, not, not be redistributing other people's code,
> because that puts you on the hook to keep up with their bug fixes ---
> particularly security-critical fixes, which do happen from time to time.
> That's a loop you don't want to be in.

I am not.
I don't know if you are familiar, but Git has a notion of using submodules
Which means all I as a developer can do is to say "My program is using moduleA,
moduleB and moduleC". And if the user configure my program not to use
system libraries
but use mine instead (maybe because mine is newer?), I'll let them.

>
> In this case I don't even see what's the point.  If somebody wants to
> use your code with Postgres, it's just about certain that they already
> have an installed copy of libpq, from wherever they get Postgres from.
> The same argument applies to mysql or SQLite.  So you should be
> expecting to link against already-present libraries, not bring your
> own.  Do you bring your own libc, or openssl, or C compiler?

Imagine a situation where someone is using Ubuntu 16 and libpq 5.
I can provide a newer version of libpq because it has security fixes
and sme additional features.
Remember - release life cycle for different distros are different.
And what about Windows?
Usually when you work with Windows you lin statically.
Therefore it doesn't matter much - you provide the code that the user builds
and that's it.

>
> I can tell you for certain that if your package were to be picked up
> and redistributed by Red Hat, or Debian, or any other open-source
> vendor, the very first thing they'd do is strip it of any such copied
> code.  They don't want the management hassle of having to update
> multiple copies of libpq or mysql-connector or other stuff.

Agreed.
But my repository will not contain a copy of libpq - it will merely be
pointing to
the original source code of the library.
See the difference?
And if the user wants to clone my repository - great.
He will just grab the cde from the PostgreSQL repo and build it as
part of the build process.

Thank you.

>
>                         regards, tom lane