Thread: libpq 11.10 / 11.11 are not propertly released for bionic
Hey there,
I noticed that version 11.11 of libpq5 and libpq-dev were released yesterday, but the release seems incomplete.
The new releases are found at https://apt.postgresql.org/pub/repos/apt/pool/11/p/postgresql-11/ , e.g.:
libpq-dev_11.11-1.pgdg18.04+1_amd64.deb 11-Feb-2021 10:39 165700
libpq-dev_11.11-1.pgdg18.04+1_arm64.deb 11-Feb-2021 10:40 154792
libpq-dev_11.11-1.pgdg18.04+1_i386.deb 11-Feb-2021 10:39 175292
libpq-dev_11.11-1.pgdg18.04+1_ppc64el.deb 11-Feb-2021 10:40 179288
libpq-dev_11.11-1.pgdg18.04+1_arm64.deb 11-Feb-2021 10:40 154792
libpq-dev_11.11-1.pgdg18.04+1_i386.deb 11-Feb-2021 10:39 175292
libpq-dev_11.11-1.pgdg18.04+1_ppc64el.deb 11-Feb-2021 10:40 179288
However, in https://apt.postgresql.org/pub/repos/apt/dists/bionic-pgdg/11/binary-amd64/ , Packages was last updated on 12-Nov-2020 12:10, listing filenames for the two packages for the *11.10* release:
pool/11/p/postgresql-11/libpq-dev_11.10-1.pgdg18.04+1_amd64.deb
pool/11/p/postgresql-11/libpq5_11.10-1.pgdg18.04+1_amd64.deb
Furthermore, the referenced files are *not* found in the pool. The only 11.10 files are as follows:
libpq-dev_11.10-1.pgdg110+1+b1_amd64.deb 20-Nov-2020 09:24 164420
libpq-dev_11.10-1.pgdg110+1+b1_arm64.deb 20-Nov-2020 09:24 161744
libpq-dev_11.10-1.pgdg110+1+b1_ppc64el.deb 20-Nov-2020 09:24 179880
libpq-dev_11.10-1.pgdg110+1+b2_amd64.deb 04-Jan-2021 12:10 164424
libpq-dev_11.10-1.pgdg110+1+b2_arm64.deb 04-Jan-2021 12:10 161788
libpq-dev_11.10-1.pgdg110+1+b1_amd64.deb 20-Nov-2020 09:24 164420
libpq-dev_11.10-1.pgdg110+1+b1_arm64.deb 20-Nov-2020 09:24 161744
libpq-dev_11.10-1.pgdg110+1+b1_ppc64el.deb 20-Nov-2020 09:24 179880
libpq-dev_11.10-1.pgdg110+1+b2_amd64.deb 04-Jan-2021 12:10 164424
libpq-dev_11.10-1.pgdg110+1+b2_arm64.deb 04-Jan-2021 12:10 161788
This prevents me from installing libpq 11 on bionic, as reproduced via `docker run -it ubuntu:18.04 bash` below:
root@62d4e1ddd8bf:/# apt-get update -qq && apt-get install -qq curl gnupg2
...
root@62d4e1ddd8bf:/# curl -sf https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
OK
root@62d4e1ddd8bf:/# echo 'deb http://apt.postgresql.org/pub/repos/apt/ bionic-pgdg 11' >> /etc/apt/sources.list.d/postgresql.list
root@62d4e1ddd8bf:/# apt-cache policy libpq5
libpq5:
Installed: (none)
Candidate: 10.15-0ubuntu0.18.04.1
Version table:
10.15-0ubuntu0.18.04.1 500
500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
10.3-1 500
500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages
root@62d4e1ddd8bf:/# apt-get update -qq
root@62d4e1ddd8bf:/# apt-cache policy libpq5
libpq5:
Installed: (none)
Candidate: 11.10-1.pgdg18.04+1
Version table:
11.10-1.pgdg18.04+1 500
500 http://apt.postgresql.org/pub/repos/apt bionic-pgdg/11 amd64 Packages
10.15-0ubuntu0.18.04.1 500
500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
10.3-1 500
500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages
root@62d4e1ddd8bf:/# apt-get install libpq5=11\*
Reading package lists... Done
Building dependency tree
Reading state information... Done
Selected version '11.10-1.pgdg18.04+1' (PostgreSQL for Debian/Ubuntu repository:bionic-pgdg [amd64]) for 'libpq5'
The following NEW packages will be installed:
libpq5
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 169 kB of archives.
After this operation, 767 kB of additional disk space will be used.
Err:1 http://apt.postgresql.org/pub/repos/apt bionic-pgdg/11 amd64 libpq5 amd64 11.10-1.pgdg18.04+1
404 Not Found [IP: 87.238.57.227 80]
E: Failed to fetch http://apt.postgresql.org/pub/repos/apt/pool/11/p/postgresql-11/libpq5_11.10-1.pgdg18.04+1_amd64.deb 404 Not Found [IP: 87.238.57.227 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
OK
root@62d4e1ddd8bf:/# echo 'deb http://apt.postgresql.org/pub/repos/apt/ bionic-pgdg 11' >> /etc/apt/sources.list.d/postgresql.list
root@62d4e1ddd8bf:/# apt-cache policy libpq5
libpq5:
Installed: (none)
Candidate: 10.15-0ubuntu0.18.04.1
Version table:
10.15-0ubuntu0.18.04.1 500
500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
10.3-1 500
500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages
root@62d4e1ddd8bf:/# apt-get update -qq
root@62d4e1ddd8bf:/# apt-cache policy libpq5
libpq5:
Installed: (none)
Candidate: 11.10-1.pgdg18.04+1
Version table:
11.10-1.pgdg18.04+1 500
500 http://apt.postgresql.org/pub/repos/apt bionic-pgdg/11 amd64 Packages
10.15-0ubuntu0.18.04.1 500
500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
10.3-1 500
500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages
root@62d4e1ddd8bf:/# apt-get install libpq5=11\*
Reading package lists... Done
Building dependency tree
Reading state information... Done
Selected version '11.10-1.pgdg18.04+1' (PostgreSQL for Debian/Ubuntu repository:bionic-pgdg [amd64]) for 'libpq5'
The following NEW packages will be installed:
libpq5
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 169 kB of archives.
After this operation, 767 kB of additional disk space will be used.
Err:1 http://apt.postgresql.org/pub/repos/apt bionic-pgdg/11 amd64 libpq5 amd64 11.10-1.pgdg18.04+1
404 Not Found [IP: 87.238.57.227 80]
E: Failed to fetch http://apt.postgresql.org/pub/repos/apt/pool/11/p/postgresql-11/libpq5_11.10-1.pgdg18.04+1_amd64.deb 404 Not Found [IP: 87.238.57.227 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
I *think* I can manually install these packages via the following, but I'm not sure if there are any other release-related issues:
curl https://apt.postgresql.org/pub/repos/apt/pool/11/p/postgresql-11/libpq-dev_11.11-1.pgdg18.04%2B1_amd64.deb -o /tmp/libpq-dev.debcurl https://apt.postgresql.org/pub/repos/apt/pool/11/p/postgresql-11/libpq5_11.11-1.pgdg18.04%2B1_amd64.deb -o /tmp/libpq.debdpkg -i /tmp/libpq-dev.deb
dpkg -i /tmp/libpq.deb
Thanks,
Saif
Saif
Re: Saif Hakim > I noticed that version 11.11 of libpq5 and libpq-dev were released > yesterday, but the release seems incomplete. Hi Saif, > However, in > https://apt.postgresql.org/pub/repos/apt/dists/bionic-pgdg/11/binary-amd64/ > , Packages was last updated on 12-Nov-2020 12:10, listing filenames for the > two packages for the *11.10* release: that is indeed stale, thanks for spotting that. The problem is present in the other distributions as well. I have now manually updated bionic-pgdg, but I will leave the other untouched until I can figure out why reprepro isn't maintaining those files properly by itself. > root@62d4e1ddd8bf:/# echo 'deb http://apt.postgresql.org/pub/repos/apt/ > bionic-pgdg 11' >> /etc/apt/sources.list.d/postgresql.list That said, you shouldn't worry about libpq5 11 - using the 13 version from the "main" repository component is the recommended way. I guess the fact that no one has spotted that problem before means no one is actually using the "11" and the other versioned components. So the recommendation would be to use "bionic-pgdg main" in sources.list. Christoph
Hi Christoph,
Out of curiosity, if we were running libpq5-11 with Postgres 13, would we see any issues or does postgres maintain backwards compatibility with these libs as well?
> that is indeed stale, thanks for spotting that. The problem is present
> in the other distributions as well. I have now manually updated
> bionic-pgdg, but I will leave the other untouched until I can figure
> in the other distributions as well. I have now manually updated
> bionic-pgdg, but I will leave the other untouched until I can figure
Thanks!, I've confirmed that my repro no longer works and version 11 is installed.
> That said, you shouldn't worry about libpq5 11 - using the 13 version
> from the "main" repository component is the recommended way. For my own understanding, what are the ramifications of using the non-"main" repository component / what are the differences?
I was also under the impression that since the version number tracks the postgres release it is compatible with, that I should be choosing the version that matches the version of postgres I'm running, i.e. since I'm running Postgres 11, I should be using libpq5-11.xx. I'm primarily installing libpq for python; however, taking a closer look at psycopg2 docs (https://www.psycopg.org/docs/install.html#runtime-requirements), I see that they are in agreement with what you're saying:
> Note: Whatever version of libpq psycopg2 is compiled with, it will
> be possible to connect to PostgreSQL servers of any supported
> version: just install the most recent libpq version or the most
> practical, without trying to match it to the version of the
> PostgreSQL server you will have to connect to.
> be possible to connect to PostgreSQL servers of any supported
> version: just install the most recent libpq version or the most
> practical, without trying to match it to the version of the
> PostgreSQL server you will have to connect to.
Out of curiosity, if we were running libpq5-11 with Postgres 13, would we see any issues or does postgres maintain backwards compatibility with these libs as well?
Thanks again!
Best,
Saif
On Sun, Feb 14, 2021 at 2:53 PM Christoph Berg <myon@debian.org> wrote:
That said, you shouldn't worry about libpq5 11 - using the 13 version
from the "main" repository component is the recommended way. I guess
the fact that no one has spotted that problem before means no one is
actually using the "11" and the other versioned components.
Actually, I received a few complaints the other day from developers for the bionic-pgdg/9.6 component when 9.6.21 was pushed, so Saif's issue was not an isolated incident. At the time, I saw that the 'Packages' file was similarly last updated in November. Everything looks good now, though.
While I recommend to my teams use the 'main' component for libpq, there continues to be fear among many that libpq versions mismatched from the parent database cluster version may cause unexpected issues — even after I point out that the database cluster itself is using the most recent release of libpq from the 'main' component.
Aaron
Re: Saif Hakim > > that is indeed stale, thanks for spotting that. The problem is present > > in the other distributions as well. I have now manually updated > > bionic-pgdg, but I will leave the other untouched until I can figure > > Thanks!, I've confirmed that my repro no longer works and version 11 is > installed. In the meantime I found the issue: to speed up getting the different PG major versions out for the release, I had told reprepro not to "export" the Packages files except on the last version processed. Normally this is fine, since the last run would export all the files touched, but as the extra components are only touched by a single PG major version each, the last run had only exported the extra files for that PG version, leaving the other versions in the old state. I have now exported all the dists manually again and things will be back on track in a few minutes with the next mirror run. > > That said, you shouldn't worry about libpq5 11 - using the 13 version > > from the "main" repository component is the recommended way. > > For my own understanding, what are the ramifications of using the > non-"main" repository component / what are the differences? The non-main components contain the lib packages only, so you can't actually install postgresql-11 from there since that is contained in "main". If you want libpq5 only, you can of course go with "11" only (or "main 11"), but I have yet to hear from a case where libpq5 version 13 isn't just working as well. > Out of curiosity, if we were running libpq5-11 with Postgres 13, would we > see any issues or does postgres maintain backwards compatibility with these > libs as well? On the client side, you should be able to see much of a difference, the protocol is the same. Possibly there are some functions in libpq5 that are only available with 13, but dpkg's dependency tracking will tell you if a newer version is required. Re: Aaron Pavely > Actually, I received a few complaints the other day from developers for the > bionic-pgdg/9.6 component when 9.6.21 was pushed, so Saif's issue was not > an isolated incident. At the time, I saw that the 'Packages' file was > similarly last updated in November. Everything looks good now, though. Nod. > While I recommend to my teams use the 'main' component for libpq, there > continues to be fear among many that libpq versions mismatched from the > parent database cluster version may cause unexpected issues — even after I > point out that the database cluster itself is using the most recent release > of libpq from the 'main' component. Right. (But note that the PG server itself doesn't actually use libpq, it's only required for things like postgres_fdw and dblink.) Christoph