Re: 64-bit MultiXactOffset vs. 9.3->9.4 upgrade, pg_resetwal, "wraparound" msg - Mailing list pgsql-hackers

On 16/09/2026 14:33, Heikki Linnakangas wrote:
> On 28/08/2026 02:17, Noah Misch wrote:
>> commit bd8d9c9 wrote:
>>> Commit:     Heikki Linnakangas <heikki.linnakangas@iki.fi>
>>> CommitDate: Tue Dec 9 13:53:03 2025 +0200
>>>
>>>      Widen MultiXactOffset to 64 bits
>>
>>> --- a/src/bin/pg_upgrade/pg_upgrade.c
>>> +++ b/src/bin/pg_upgrade/pg_upgrade.c
>>
>>> +        nxtmulti = old_cluster.controldata.chkpnt_nxtmulti;
>>> +        if (old_cluster.controldata.cat_ver >= 
>>> MULTIXACT_FORMATCHANGE_CAT_VER)
>>> +        {
>>> +            /* Versions 9.3 - 18: convert all multixids  */
>>> +            oldstMulti = old_cluster.controldata.chkpnt_oldstMulti;
>>
>> If a cluster's upgrade history includes an upgrade from 9.3 to early 
>> 9.4, it
>> may have a wrong value here.  Specifically, upgrades done before a61daa14
>> (2014-07 commit) have that hazard.  We still have backend code to 
>> detect such
>> cases and reduce damage:
>>
>>         ereport(LOG,
>>                 (errmsg("cannot truncate up to MultiXact %u because it 
>> does not exist on disk, skipping truncation",
>>                         newOldestMulti)));
>>
>> However, the pg_upgrade side from the v19 commit lacks such 
>> protection.  If
>> heap tuples still reference older multixacts than the faulty control data
>> suggests, pg_upgrade will copy too small a range, making affected tuples
>> unreadable.
> 
> Thanks, I'll look into this. My first reaction is that I think if 
> oldstMulti is incorrectly too old, the upgrade will fail because the 
> conversion routine will fail to find it. If it's too new, i.e. "in the 
> future", it will also fail to find it.
> 
> The third possibility is that the bogus oldstMulti value is within the 
> range of the "real" range. That can happen if multixid wraparound had 
> already happened before the (broken) 9.3 -> 9.4 upgrade. In that case, 
> even if the multixids are still readable on disk, you're one vacuum away 
> from truncating them. In other words, the damage has already been done, 
> or could be done at any minute. That's the scenario for which the commit 
> message 78db307bb2 says "this mechanism cannot save us".
> 
> But I'll do some testing of that. Let's ensure that the error message 
> makes sense at the very least.

Ok, I reconstructed that scenario artificially with the attached script. 
It creates a new cluster, runs a test program to create a bunch of 
multixids, runs vacuum freeze to truncate the multixids SLRU and advance 
oldestMulti. Finally, it uses pg_resetwal to reset oldestMulti back to 
1, to simulate what you'd get with the buggy 9.3 pg_upgrade. I tested 
this with a v18 source cluster, but I don't think the source version 
matters. (It must've been pg_upgraded to 9.3 in the past, but it 
could've gone through more pg_upgrades since)

When you run pg_upgrade on that, it fails like this:

> Converting pg_multixact files                                 
> could not open file "/tmp/pgcluster/pg_multixact/offsets/0000": No such file or directory

Not an ideal error message, but at least you get an error rather than 
silently losing data.

With the attached patch, you get an error message crafted specifically 
for that case.

> Converting pg_multixact files                                 fatal
> 
> Segment file containing multixid 1 does not exist.
> This can happen if an old version of pg_upgrade was used in the past
> to upgrade the cluster from version 9.3 to 9.4. If that is the cause,
> it can be fixed by running VACUUM FREEZE before upgrade.
> Failure, exiting

I'm not sure this is worth committing though. How many clusters are 
there out there that have been pg_upgraded to version 9.3.0-9.3.4, and 
haven't since been running for long enough to fix the oldestMultiXid 
value? Having that error message seems pretty harmless too, but if you 
actually hit that case that the file is missing, it might actually be 
more likely that you've hit some other bug than that old pg_upgrade issue.

Another approach would be to ignore the oldestMulti in the control file 
and scan pg_multixact/offsets for the oldest segment that actually 
exists. But I don't like that. Unless you actually hit that old 
pg_upgrade bug, a missing file is a sign of trouble, and it's better 
that you get alerted about it than upgrading the broken cluster to a new 
version.

- Heikki