Re: Tepid: selective index updates for heap relations - Mailing list pgsql-hackers

From: Greg Burd <greg@burd.me>
To: pgsql-hackers@postgresql.org
Subject: Re: Tepid: selective index updates for heap relations
In-Reply-To: <af02b486-6adc-494c-a357-1dd72f655dcf@app.fastmail.com>
References: <af02b486-6adc-494c-a357-1dd72f655dcf@app.fastmail.com>

Attached is v71, rebased on master through f58ab18a.

I took the time to split the table/index AM contract and executor
changes out from the heap implementation. This makes the series longer,
but should make it easier to review the interface without first
reviewing the other changes for Tepid.

In thinking this through I might pull out the locator contract into
a new thread to discuss separately along with the other changes to
core, but for now they live here.

Each of these have their own patch near the beginning so that we can
discuss them separately:

* BitmapAnd now learns the table relation from each bitmap index scan,
  including scans nested under BitmapOr. For a group where entries for
  the same row can name different slots (as can happen with Tepid due to
  mid-chain references in indexes), the AM's callback requests a union
  and recheck instead of an exact intersection. The locator contract
  also has an optional BitmapOr recheck hook.

* The locator descriptor states width, maximum offset, locator stability
  and whether an UPDATE preserves the old version. A stable locator does
  not imply unchanged contents. The LOCKED_VERSION flags distinguish a
  snapshot-visible version from one the command has already locked;
  retargeted tells callers when they must re-evaluate a newer version.

* For AMs that overwrite rows (none of which exist in core today, but
  many have experimented with this including me in the UNDO[1] proposal
  with the FLUX table AM), the executor preserves OLD before the
  write. AFTER ROW events can carry OLD and event-time NEW, with spill
  and subtransaction cleanup. Foreign-key checks and unique rechecks
  still fetch the current version. The contract requires a deleted row
  to remain fetchable with SnapshotAny until the deleting transaction
  ends.

* Index creation and rebuilds check locator width. The amcanvarlocator
  capability remains, but this does not add variable-width transport to
  the existing ItemPointerData interfaces. max_offset also bounds the
  planner's bitmap paths and GIN's supported offsets.

In the v68 patch set I overloaded the meaning of bit 14 in a TID as a
signal used during BitmapAnd, I never felt that was a good idea and it
was that struggle that forced my hand to either a) give up or b) write
the locator contract.  I did (b), but that meant that I needed somewhere
else to hide the hint I'd burried in bit 14 before.  In this series I've
done that via the visibility map and I feel a bit less dirty, but I do
have a sense of guilt for doubling its size.

I moved heap's locator-split visibility-map bit and pg_upgrade rewrite
into the heap portion. This widens the visibility map from two to four
bits per heap block, as I said before doubling its size, and requires
rewriting old _vm forks during upgrade which isn't great, but we're
living in a world of not-great trade-offs if we want to solve this so
help me find a better trade-off, get comfortable with this one, or we
put a pin in this idea alltogether and call it a day.

Heap's update and prune/vacuum changes now land together, so no
intermediate patch creates chains that pruning cannot handle. I also
fixed index-only-scan identification in the common scan constructor so
btree retains its leaf pin for genuine index-only scans.

The patch order is:
  01-07  Locator contract, version naming, triggers and bitmap hooks
  08     Capturing/codifying the existing HOT behavior in tests
  09-12  Attribute comparison and generic selective index maintenance
  13-15  Heap format, visibility map, update and prune/vacuum support
  16-18  Statistics, amcheck and logical-replication apply option
  19-20  [DO NOT MERGE] Test AMs and benchmark harness

The test AMs exercise contract properties that heap does not normally
use. They do not validate LOCKED_VERSION semantics for a real in-place
AM; they still use heap's lock and update implementation.

I need to re-benchmark this version and also measure the impact on the
visibility map.

There remains many things that need to be quantified, reviewed, and
deemed "acceptable trade-offs" including:
 * the visibility map 2->4 bits
 * rewriting old _vm forks during upgrade
 * the overhead of additional tuples in longer chains on heap pages
 * making it harder to declare a heap page all visible
 * the negative impact of 01-07 patches on performance
 * the positive impact of 09-18 on performance, table and index bloat
 * I'm surely missing something...

At this point, I'd particularly appreciate review of the locator
contract, the rest of the 01-07 changes, and the visibility-map
trade-off. TM_FailureData still exposes cmax, xmax and ctid to callers
outside heap; I have left abstracting those fields for follow-up work
(because I think they fall into the same bucket of "heap-specific leak
into the surrounding code" or "leaky abstraction" or whatever you'd like
to call it) rather than adding it to this (ever growing, now 20 patches)
series.

best.

-greg