Re: pg_upgrade: transfer pg_largeobject_metadata's files when possible - Mailing list pgsql-hackers

From Andres Freund
Subject Re: pg_upgrade: transfer pg_largeobject_metadata's files when possible
Date
Msg-id jaqwqwwiy57bhg36jru4mvjznzemrhgobyynpqvyp573m7h3rt@2kpgm5sfkbyy
Whole thread
In response to Re: pg_upgrade: transfer pg_largeobject_metadata's files when possible  (Nathan Bossart <nathandbossart@gmail.com>)
Responses Re: pg_upgrade: transfer pg_largeobject_metadata's files when possible
List pgsql-hackers
Hi,

On 2026-02-05 15:29:55 -0600, Nathan Bossart wrote:
> Here is what I have so far.

Thanks!


> --- a/src/bin/pg_dump/pg_dump.c
> +++ b/src/bin/pg_dump/pg_dump.c
> @@ -214,12 +214,6 @@ static int    nbinaryUpgradeClassOids = 0;
>  static SequenceItem *sequences = NULL;
>  static int    nsequences = 0;
>  
> -/*
> - * For binary upgrade, the dump ID of pg_largeobject_metadata is saved for use
> - * as a dependency for pg_shdepend and any large object comments/seclabels.
> - */
> -static DumpId lo_metadata_dumpId;
> -
>  /* Maximum number of relations to fetch in a fetchAttributeStats() call. */
>  #define MAX_ATTR_STATS_RELS 64
>  
> @@ -1121,27 +1115,20 @@ main(int argc, char **argv)
>          getTableData(&dopt, tblinfo, numTables, RELKIND_SEQUENCE);
>  
>      /*
> -     * For binary upgrade mode, dump pg_largeobject_metadata and the
> -     * associated pg_shdepend rows. This is faster to restore than the
> -     * equivalent set of large object commands.  We can only do this for
> -     * upgrades from v12 and newer; in older versions, pg_largeobject_metadata
> -     * was created WITH OIDS, so the OID column is hidden and won't be dumped.
> +     * For binary upgrade mode, dump the pg_shdepend rows for large objects
> +     * and maybe even pg_largeobject_metadata (see comment below for details).
> +     * This is faster to restore than the equivalent set of large object
> +     * commands.  We can only do this for upgrades from v12 and newer; in
> +     * older versions, pg_largeobject_metadata was created WITH OIDS, so the
> +     * OID column is hidden and won't be dumped.
>       */

It's not really related to this change, but what is that WITH OIDS bit about?
Sure, they aren't shown by default, but all it takes to change that is to
explicitly add the output column? I'm not saying we have to do that, I just
don't understand the reasoning as written here.


> @@ -3979,7 +3964,25 @@ getLOs(Archive *fout)
>      appendPQExpBufferStr(loQry,
>                           "SELECT oid, lomowner, lomacl, "
>                           "acldefault('L', lomowner) AS acldefault "
> -                         "FROM pg_largeobject_metadata "
> +                         "FROM pg_largeobject_metadata ");
> +
> +    /*
> +     * For upgrades from v12 or newer, we transfer pg_largeobject_metadata via

*binary upgrades.


The improvements in pg_upgrade time and memory in a cluster with 50M LOs [1]
are really quite impressive, even if upgrading from < 16. It's rare to improve
memory usage by several orders of magnitude.

Greetings,

Andres Freund

[1] c=20;pgbench -n -c$c -j$c -P1 -t $((50000000 / $c)) -f <(echo -e "SELECT lo_create(0) AS loid\n") 50m_los



pgsql-hackers by date:

Previous
From: "Jelte Fennema-Nio"
Date:
Subject: Re: Correct documentation for protocol version
Next
From: Nathan Bossart
Date:
Subject: Re: pg_upgrade: transfer pg_largeobject_metadata's files when possible