Thread: libpq 11.10 / 11.11 are not propertly released for bionic

libpq 11.10 / 11.11 are not propertly released for bionic

From
Saif Hakim
Date:
Hey there,

I noticed that version 11.11 of libpq5 and libpq-dev were released yesterday, but the release seems incomplete.

libpq-dev_11.11-1.pgdg18.04+1_amd64.deb            11-Feb-2021 10:39              165700
libpq-dev_11.11-1.pgdg18.04+1_arm64.deb            11-Feb-2021 10:40              154792
libpq-dev_11.11-1.pgdg18.04+1_i386.deb             11-Feb-2021 10:39              175292
libpq-dev_11.11-1.pgdg18.04+1_ppc64el.deb          11-Feb-2021 10:40              179288


However, in https://apt.postgresql.org/pub/repos/apt/dists/bionic-pgdg/11/binary-amd64/ , Packages was last updated on 12-Nov-2020 12:10, listing filenames for the two packages for the *11.10* release:
pool/11/p/postgresql-11/libpq-dev_11.10-1.pgdg18.04+1_amd64.deb
pool/11/p/postgresql-11/libpq5_11.10-1.pgdg18.04+1_amd64.deb

Furthermore, the referenced files are *not* found in the pool. The only 11.10 files are as follows:
libpq-dev_11.10-1.pgdg110+1+b1_amd64.deb           20-Nov-2020 09:24              164420
libpq-dev_11.10-1.pgdg110+1+b1_arm64.deb           20-Nov-2020 09:24              161744
libpq-dev_11.10-1.pgdg110+1+b1_ppc64el.deb         20-Nov-2020 09:24              179880
libpq-dev_11.10-1.pgdg110+1+b2_amd64.deb           04-Jan-2021 12:10              164424
libpq-dev_11.10-1.pgdg110+1+b2_arm64.deb           04-Jan-2021 12:10              161788


This prevents me from installing libpq 11 on bionic, as reproduced via `docker run -it ubuntu:18.04 bash` below:

root@62d4e1ddd8bf:/# apt-get update -qq && apt-get install -qq curl gnupg2
...
root@62d4e1ddd8bf:/# curl -sf https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
OK
root@62d4e1ddd8bf:/# echo 'deb http://apt.postgresql.org/pub/repos/apt/ bionic-pgdg 11' >> /etc/apt/sources.list.d/postgresql.list
root@62d4e1ddd8bf:/# apt-cache policy libpq5
libpq5:
  Installed: (none)
  Candidate: 10.15-0ubuntu0.18.04.1
  Version table:
     10.15-0ubuntu0.18.04.1 500
        500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
     10.3-1 500
        500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages
root@62d4e1ddd8bf:/# apt-get update -qq
root@62d4e1ddd8bf:/# apt-cache policy libpq5
libpq5:
  Installed: (none)
  Candidate: 11.10-1.pgdg18.04+1
  Version table:
     11.10-1.pgdg18.04+1 500
        500 http://apt.postgresql.org/pub/repos/apt bionic-pgdg/11 amd64 Packages
     10.15-0ubuntu0.18.04.1 500
        500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
     10.3-1 500
        500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages
root@62d4e1ddd8bf:/# apt-get install libpq5=11\*
Reading package lists... Done
Building dependency tree
Reading state information... Done
Selected version '11.10-1.pgdg18.04+1' (PostgreSQL for Debian/Ubuntu repository:bionic-pgdg [amd64]) for 'libpq5'
The following NEW packages will be installed:
  libpq5
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 169 kB of archives.
After this operation, 767 kB of additional disk space will be used.
Err:1 http://apt.postgresql.org/pub/repos/apt bionic-pgdg/11 amd64 libpq5 amd64 11.10-1.pgdg18.04+1
  404  Not Found [IP: 87.238.57.227 80]
E: Failed to fetch http://apt.postgresql.org/pub/repos/apt/pool/11/p/postgresql-11/libpq5_11.10-1.pgdg18.04+1_amd64.deb  404  Not Found [IP: 87.238.57.227 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?


I *think* I can manually install these packages via the following, but I'm not sure if there are any other release-related issues:
curl https://apt.postgresql.org/pub/repos/apt/pool/11/p/postgresql-11/libpq-dev_11.11-1.pgdg18.04%2B1_amd64.deb -o /tmp/libpq-dev.debcurl https://apt.postgresql.org/pub/repos/apt/pool/11/p/postgresql-11/libpq5_11.11-1.pgdg18.04%2B1_amd64.deb -o /tmp/libpq.debdpkg -i /tmp/libpq-dev.deb
dpkg -i /tmp/libpq.deb

Thanks,
Saif

Re: libpq 11.10 / 11.11 are not propertly released for bionic

From
Christoph Berg
Date:
Re: Saif Hakim
> I noticed that version 11.11 of libpq5 and libpq-dev were released
> yesterday, but the release seems incomplete.

Hi Saif,

> However, in
> https://apt.postgresql.org/pub/repos/apt/dists/bionic-pgdg/11/binary-amd64/
> , Packages was last updated on 12-Nov-2020 12:10, listing filenames for the
> two packages for the *11.10* release:

that is indeed stale, thanks for spotting that. The problem is present
in the other distributions as well. I have now manually updated
bionic-pgdg, but I will leave the other untouched until I can figure
out why reprepro isn't maintaining those files properly by itself.

> root@62d4e1ddd8bf:/# echo 'deb http://apt.postgresql.org/pub/repos/apt/
> bionic-pgdg 11' >> /etc/apt/sources.list.d/postgresql.list

That said, you shouldn't worry about libpq5 11 - using the 13 version
from the "main" repository component is the recommended way. I guess
the fact that no one has spotted that problem before means no one is
actually using the "11" and the other versioned components.

So the recommendation would be to use "bionic-pgdg main" in
sources.list.

Christoph



Re: libpq 11.10 / 11.11 are not propertly released for bionic

From
Saif Hakim
Date:
Hi Christoph,

> that is indeed stale, thanks for spotting that. The problem is present
> in the other distributions as well. I have now manually updated
> bionic-pgdg, but I will leave the other untouched until I can figure

Thanks!, I've confirmed that my repro no longer works and version 11 is installed.

> That said, you shouldn't worry about libpq5 11 - using the 13 version
> from the "main" repository component is the recommended way. 

For my own understanding, what are the ramifications of using the non-"main" repository component / what are the differences?

I was also under the impression that since the version number tracks the postgres release it is compatible with, that I should be choosing the version that matches the version of postgres I'm running, i.e. since I'm running Postgres 11, I should be using libpq5-11.xx. I'm primarily installing libpq for python; however, taking a closer look at psycopg2 docs (https://www.psycopg.org/docs/install.html#runtime-requirements), I see that they are in agreement with what you're saying:

> Note: Whatever version of libpq psycopg2 is compiled with, it will
> be possible to connect to PostgreSQL servers of any supported
> version: just install the most recent libpq version or the most
> practical, without trying to match it to the version of the
> PostgreSQL server you will have to connect to.

Out of curiosity, if we were running libpq5-11 with Postgres 13, would we see any issues or does postgres maintain backwards compatibility with these libs as well?

Thanks again!

Best,
Saif

Re: libpq 11.10 / 11.11 are not propertly released for bionic

From
Aaron Pavely
Date:
On Sun, Feb 14, 2021 at 2:53 PM Christoph Berg <myon@debian.org> wrote:
That said, you shouldn't worry about libpq5 11 - using the 13 version
from the "main" repository component is the recommended way. I guess
the fact that no one has spotted that problem before means no one is
actually using the "11" and the other versioned components.
 
Actually, I received a few complaints the other day from developers for the bionic-pgdg/9.6 component when 9.6.21 was pushed, so Saif's issue was not an isolated incident. At the time, I saw that the 'Packages' file was similarly last updated in November. Everything looks good now, though.

While I recommend to my teams use the 'main' component for libpq, there continues to be fear among many that libpq versions mismatched from the parent database cluster version may cause unexpected issues — even after I point out that the database cluster itself is using the most recent release of libpq from the 'main' component.

Aaron

Re: libpq 11.10 / 11.11 are not propertly released for bionic

From
Christoph Berg
Date:
Re: Saif Hakim
> > that is indeed stale, thanks for spotting that. The problem is present
> > in the other distributions as well. I have now manually updated
> > bionic-pgdg, but I will leave the other untouched until I can figure
> 
> Thanks!, I've confirmed that my repro no longer works and version 11 is
> installed.

In the meantime I found the issue: to speed up getting the different
PG major versions out for the release, I had told reprepro not to
"export" the Packages files except on the last version processed.
Normally this is fine, since the last run would export all the files
touched, but as the extra components are only touched by a single PG
major version each, the last run had only exported the extra files for
that PG version, leaving the other versions in the old state.

I have now exported all the dists manually again and things will be
back on track in a few minutes with the next mirror run.

> > That said, you shouldn't worry about libpq5 11 - using the 13 version
> > from the "main" repository component is the recommended way.
> 
> For my own understanding, what are the ramifications of using the
> non-"main" repository component / what are the differences?

The non-main components contain the lib packages only, so you can't
actually install postgresql-11 from there since that is contained in
"main".

If you want libpq5 only, you can of course go with "11" only (or
"main 11"), but I have yet to hear from a case where libpq5 version 13
isn't just working as well.

> Out of curiosity, if we were running libpq5-11 with Postgres 13, would we
> see any issues or does postgres maintain backwards compatibility with these
> libs as well?

On the client side, you should be able to see much of a difference,
the protocol is the same. Possibly there are some functions in libpq5
that are only available with 13, but dpkg's dependency tracking will
tell you if a newer version is required.


Re: Aaron Pavely
> Actually, I received a few complaints the other day from developers for the
> bionic-pgdg/9.6 component when 9.6.21 was pushed, so Saif's issue was not
> an isolated incident. At the time, I saw that the 'Packages' file was
> similarly last updated in November. Everything looks good now, though.

Nod.

> While I recommend to my teams use the 'main' component for libpq, there
> continues to be fear among many that libpq versions mismatched from the
> parent database cluster version may cause unexpected issues — even after I
> point out that the database cluster itself is using the most recent release
> of libpq from the 'main' component.

Right. (But note that the PG server itself doesn't actually use libpq,
it's only required for things like postgres_fdw and dblink.)

Christoph