Thread: logical decoding: ABI break in 10.5 et al

Hello

In the process of fixing a bug, I inadvertently broke ABI compatibility
in all released branches that sport logical decoding.

[TL;DR: I propose to set up abidiff.postgresql.org.]

This is the commit:

Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
Branch: master Release: REL_11_BR [f49a80c48] 2018-06-26 16:48:10 -0400
Branch: REL_10_STABLE Release: REL_10_5 [b767b3f2e] 2018-06-26 16:38:34 -0400
Branch: REL9_6_STABLE Release: REL9_6_10 [da10d6a8a] 2018-06-26 16:38:34 -0400
Branch: REL9_5_STABLE Release: REL9_5_14 [4cb6f7837] 2018-06-26 16:38:34 -0400
Branch: REL9_4_STABLE Release: REL9_4_19 [962313558] 2018-06-26 16:38:34 -0400

    Fix "base" snapshot handling in logical decoding
    [long commit message elided]
https://git.postgresql.org/pg/commitdiff/f49a80c48

This commit made modules dependent on structs ReorderBuffer and
ReorderBufferTXN compiled for prior minor releases no longer work with
the new minors, because some new struct members were inserted in the
middle of the struct.  Third-party modules such as pglogical, BDR,
wal2json are affected and need to be recompiled; failing to do so can
make them misbehave or crash.

Obviously, there's nothing to be done about it at this point; users must
recompile.  Modules have the option of providing code improvements that
will enable them to work with either API definition, so upgrading the
module immediately enables them to run before the new minors are
installed.  Reverting to the original struct definition in Postgres is
of course no longer an option.

In order to forestall this from happening again, I'm proposing a new
community service that will let us know of ABI incompatibilities
introduced by commits, hopefully before they become full-fledged minor
releases, so that we can apply fixes.  Initially, I'll just use the
tools abidumper and abi-compliance-checker from https://abi-laboratory.pro/
and set up a website that displays relevant reports for each branch.
I'm not yet sure about sending email alerts for failures; I'll
experiment a bit first.

(FWIW I ran the tools to compare 9.6.9 and 9.6.10 and it did indeed
detect the problem.)

I'm thinking of naming this abidiff.postgresql.org.  Bikeshedding
opportunity if you don't like that name -- this week only.

-- 
Álvaro Herrera

Re: Alvaro Herrera 2018-08-28 <20180828153806.fgfnul2imeltzmib@alvherre.pgsql>
> In order to forestall this from happening again, I'm proposing a new
> community service that will let us know of ABI incompatibilities
> introduced by commits, hopefully before they become full-fledged minor
> releases, so that we can apply fixes.  Initially, I'll just use the
> tools abidumper and abi-compliance-checker from https://abi-laboratory.pro/
> and set up a website that displays relevant reports for each branch.
> I'm not yet sure about sending email alerts for failures; I'll
> experiment a bit first.

Fwiw, there's already a setup, but it's restricted to the shared
libraries, and of course only for released versions:

https://abi-laboratory.pro/index.php?view=objects_report&l=postgresql&v1=10.4&v2=10.5

A big +1 on setting up an instance.

Christoph

Re: Alvaro Herrera 2018-08-28 <20180828153806.fgfnul2imeltzmib@alvherre.pgsql>
> This commit made modules dependent on structs ReorderBuffer and
> ReorderBufferTXN compiled for prior minor releases no longer work with
> the new minors, because some new struct members were inserted in the
> middle of the struct.  Third-party modules such as pglogical, BDR,
> wal2json are affected and need to be recompiled; failing to do so can
> make them misbehave or crash.

Fwiw, I haven't seen any problems with the wal2json packages in
apt.postgresql.org that were compiled with the old PG packages. They
still pass the regression tests with the new ABI.


https://pgdgbuild.dus.dg-i.net/view/Testsuite/job/wal2json-adt/lastSuccessfulBuild/architecture=amd64,distribution=sid/console

Christoph

On Tue, Aug 28, 2018 at 11:38 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
> [TL;DR: I propose to set up abidiff.postgresql.org.]

Nice idea.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company