Re: pg_upgrade does not upgrade pg_stat_statements properly - Mailing list pgsql-hackers

From Dave Cramer
Subject Re: pg_upgrade does not upgrade pg_stat_statements properly
Date
Msg-id CADK3HHLZP=3HYNY_qsVBDQt83F7QpY_WrqXGN=KkQZCxraOyLA@mail.gmail.com
Whole thread Raw
In response to Re: pg_upgrade does not upgrade pg_stat_statements properly  (Bruce Momjian <bruce@momjian.us>)
Responses Re: pg_upgrade does not upgrade pg_stat_statements properly  ("David G. Johnston" <david.g.johnston@gmail.com>)
Re: pg_upgrade does not upgrade pg_stat_statements properly  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers

Dave Cramer


On Thu, 29 Jul 2021 at 15:06, Bruce Momjian <bruce@momjian.us> wrote:
On Thu, Jul 29, 2021 at 01:43:09PM -0400, Álvaro Herrera wrote:
> On 2021-Jul-29, Bruce Momjian wrote:
>
> > +     If the old cluster used extensions, whether from
> > +     <filename>contrib</filename> or some other source, it used
> > +     shared object files (or DLLs) to implement these extensions, e.g.,
> > +     <filename>pgcrypto.so</filename>.  Now, shared object files matching
> > +     the new server binary must be installed in the new cluster, usually
> > +     via operating system commands.  Do not load the schema definitions,
> > +     e.g., <command>CREATE EXTENSION pgcrypto</command>, because these
> > +     will be copied from the old cluster.  (Extensions should be upgraded
> > +     later using <literal>ALTER EXTENSION ... UPGRADE</literal>.)
>
> I propose this:
>
> <para>
>   If the old cluster used shared-object files (or DLLs) for extensions
>   or other loadable modules, install recompiled versions of those files
>   onto the new cluster.
>   Do not install the extension themselves (i.e., do not run
>   <command>CREATE EXTENSION</command>), because extension definitions
>   will be carried forward from the old cluster.
> </para>
>
> <para>
>   Extensions can be upgraded after pg_upgrade completes using
>   <command>ALTER EXTENSION ... UPGRADE</command>, on a per-database
>   basis.
> </para>
>
> I suggest " ... for extensions or other loadable modules" because
> loadable modules aren't necessarily for extensions.  Also, it's
> perfectly possible to have extension that don't have a loadable module.

Yes, good point.

> I suggest "extension definitions ... carried forward" instead of
> "extensions ... copied" (your proposed text) to avoid the idea that
> files are copied; use it instead of "schema definitions ... upgraded"
> (the current docs) to avoid the idea that they are actually upgraded;
> also, "schema definition" seems a misleading term to use here.

I used the term "duplicated".

> I suggest "can be upgraded" rather than "should be upgraded" because
> we're not making a recommendation, merely stating the fact that it is
> possible to do so.

Agreed.  Most extensions don't have updates between major versions.

> I suggest using the imperative mood, to be consistent with the
> surrounding text.  (Applies to the first para; the second para is
> informative.)

OK.

> I haven't seen it mentioned in the thread, but I think the final phrase
> of this <step> should be a separate step,
>
> <step>
>  <title>Copy custom full-text search files</title>
>  <para>
>   Copy any custom full text search file (dictionary, synonym, thesaurus,
>   stop word list) to the new server.
>  </para>
> </step>
>
> While this is closely related to extensions, it's completely different.

Agreed.  See attached patch.

So back to the original motivation for bringing this up. Recall that a cluster was upgraded. Everything appeared to work fine, except that the definitions of the functions were slightly different enough to cause a fatal issue on restoring a dump from pg_dump.
Since pg_upgrade is now part of the core project, we need to make sure this is not possible or be much more insistent that the user needs to upgrade any extensions that require it.

I believe we should be doing more than making a recommendation.

Dave

pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: needless complexity in StartupXLOG
Next
From: "David G. Johnston"
Date:
Subject: Re: pg_upgrade does not upgrade pg_stat_statements properly