Re: BUG #19519: REPACK can fail due to missing chunk for toast value - Mailing list pgsql-bugs
| From | Matthias van de Meent |
|---|---|
| Subject | Re: BUG #19519: REPACK can fail due to missing chunk for toast value |
| Date | |
| Msg-id | CAEze2WgBjhf6KTKuJ-UqHHGj9Z66f62grmJgZ1EazvMH9HU+XA@mail.gmail.com Whole thread |
| In response to | Re: BUG #19519: REPACK can fail due to missing chunk for toast value (Michael Paquier <michael@paquier.xyz>) |
| Responses |
Re: BUG #19519: REPACK can fail due to missing chunk for toast value
|
| List | pgsql-bugs |
On Tue, 7 Jul 2026, 01:19 Michael Paquier, <michael@paquier.xyz> wrote:
On Mon, Jul 06, 2026 at 08:21:41PM +0200, Matthias van de Meent wrote:
> So, in effect, this is an issue that's quite similar to the CIC/RIC
> bug of PG14<14.4, except that
>
> 1.) in this case it's SnapshotAny instead of a
> registered-but-invisible MVCC snapshot;
> 2.) the issue in this case only happens when toast pointers were
> cleaned up and then are dereferenced, rather than HOT chains updated
> and reclaimed since the scan started; and finally
> 3.) this issue dates back a very long time, likely since TOAST, but
> defininitely since PG14 introduced the GlobalVisTestFor apis.
You mean 042b584c7f7d, revert of d9d076222f5b. That rings a bell.
Indeed
> So I think the only way to fix this is either
> 1.) we update every maintenance process that makes use of SnapshotAny
> and could access TOAST tables to handle errors caused by missing TOAST
> data when the base relation's tuple is RECENTLY_DEAD, and treat those
> as "tuple cleanup has already started, we just weren't aware of it
> yet, so ignore this tuple" (as I mentioned in [1]), or
That seems like an option worth exploring to me, but I doubt that
we'll be able to get something that can be backpatched due to how
invasive it is. I strongly suspect that it would require at least one
table AM change to cope with the fact that we want to let the callers
be OK with TOAST chunks missing in some contexts when grabbing a toast
slice. That's perhaps for the best if we think about potential
regressions, this is scary and old enough that we may still be OK by
living without a backpatch.
I suspect that for indexing we're lucky, in that we have full control internally over which tuple data get fed into IndexBuildCallback. We can make sure only fully detoasted attributes get passed on, and can be implemented with some (a lot) of TRY/CATCH work in or around FormIndexDatum. That said, I don't think that's very nice, and I'm also a bit concerned about leaking things (memory, buffers, ...) in such an approach.
For REPACK/VACUUM FULL/CLUSTER, I think we can do something similar, given that this code too has a central loop processing the tuple data, though I suspect it'll need a bit more work than the reindex one.
> 2.) we set the xmin of the CI/R backend to the current OldestXmin for
> the relation(s) we're currently processing, and so block all cleanup
> of the relevant (toast) data for the duration of this statement or
> whenever we check for a newer xmin for the relation (whilst ignoring
> our current one).
Err, I don't think that would be safe anyway? It's rather uncommon in
practice in schemas, but we could trigger function calls, like index
expressions, domains, whatever, that access data of other tables while
processing one table with a SnapshotAny. The CIC revert of 14.4 was
about such cases.
True, but in indexes that is forbidden: mislabeling of non-INMUTABLE functions causes broken indexes every day, and we can't be expected to make it work there. Similarly, for REPACK it shouldn't evaluate any non-indexable expressions, right?
Either way, if we can register a snapshot in the REPACK/INDEX backend whose xmin is (*correctly*) registered with the OldestNonRemovableTransactionId() of the relation, then concurrent cleanup won't happen for toast, and we'd also avoid dereferencing those dangling toast pointers. We won't need to actually use that snapshot for any scans; it'll avoid the issue as long as our backend advertises an xmin <= the OldestXmin used in maintenance to hold back pruning/vacuum on the relation.
(we can adjust that xmin every once in a while with a newer OldestNonRemovableTransactionId (excl. our own backend) as long as we also adjust the OldestXmin cutoff for the SnapshotAny visibility tests, but I don't suggest to backpatch that even newer system, it'd add significant complications to an already complicated system)
- Matthias
ps. sorry for the formatting, I'm writing this on my phone.
pgsql-bugs by date:
Previous
From: Fujii MasaoDate:
Subject: Re: BUG #18876: HINT messages for mxid wrap-around say "drop stale slots", but that may not be appropriate
Next
From: Gabriele BartoliniDate:
Subject: Re: BUG #19506: LOAD '$libdir/...' inside extension scripts ignores dynamic_library_path with extension_control_path