Re: optimize file transfer in pg_upgrade - Mailing list pgsql-hackers

From Nathan Bossart
Subject Re: optimize file transfer in pg_upgrade
Date
Msg-id Z8IWHJ7OdMnQ45ur@nathan
Whole thread Raw
In response to Re: optimize file transfer in pg_upgrade  (Robert Haas <robertmhaas@gmail.com>)
Responses Re: optimize file transfer in pg_upgrade
List pgsql-hackers
On Fri, Feb 28, 2025 at 02:41:22PM -0500, Robert Haas wrote:
> On Fri, Feb 28, 2025 at 2:40 PM Robert Haas <robertmhaas@gmail.com> wrote:
>> Maybe we should rethink the decision not to transfer relfilenodes for
>> sequences. Or have more than one way to do it. pg_upgrade
>> --binary-upgrade --binary-upgrade-even-for-sequences, or whatever.
> 
> Sorry, I meant: pg_dump --binary-upgrade --binary-upgrade-even-for-sequences
> 
> i.e. pg_upgrade could decide which way to ask pg_dump to do it,
> depending on versions and flags.

That's exactly where I landed (see v3-0002).  I haven't measured whether
transferring relfilenodes or dumping the sequence data is faster for the
existing modes, but for now I've left those alone, i.e., they still dump
sequence data.  The new "swap" mode just uses the old cluster's sequence
files, and I've disallowed using swap mode for upgrades from <v10 to avoid
the sequence tuple format change (along with other incompatible changes).

I'll admit I'm a bit concerned that this will cause problems if and when
someone wants to change the sequence tuple format again.  But that hasn't
happened for a while, AFAIK nobody's planning to change it, and even if it
does happen, we just need to have my proposed new mode transfer the
sequence files like it transfers the catalog files.  That will make this
mode slower, especially if you have a ton of sequences, but maybe it'll
still be a win in most cases.  Of course, we probably will need to have
pg_upgrade handle other kinds of format changes, too, but IMHO it's still
worth trying to speed up pg_upgrade despite the potential future
complexities.

-- 
nathan



pgsql-hackers by date:

Previous
From: Mircea Cadariu
Date:
Subject: Metadata and record block access stats for indexes
Next
From: Thom Brown
Date:
Subject: Re: making EXPLAIN extensible