pgsql: Track unpruned relids to avoid processing pruned relations - Mailing list pgsql-committers

Track unpruned relids to avoid processing pruned relations

This commit introduces changes to track unpruned relations explicitly,
making it possible for top-level plan nodes, such as ModifyTable and
LockRows, to avoid processing partitions pruned during initial
pruning.  Scan-level nodes, such as Append and MergeAppend, already
avoid the unnecessary processing by accessing partition pruning
results directly via part_prune_index. In contrast, top-level nodes
cannot access pruning results directly and need to determine which
partitions remain unpruned.

To address this, this commit introduces a new bitmapset field,
es_unpruned_relids, which the executor uses to track the set of
unpruned relations.  This field is referenced during plan
initialization to skip initializing certain nodes for pruned
partitions. It is initialized with PlannedStmt.unprunableRelids,
a new field that the planner populates with RT indexes of relations
that cannot be pruned during runtime pruning. These include relations
not subject to partition pruning and those required for execution
regardless of pruning.

PlannedStmt.unprunableRelids is computed during set_plan_refs() by
removing the RT indexes of runtime-prunable relations, identified
from PartitionPruneInfos, from the full set of relation RT indexes.
ExecDoInitialPruning() then updates es_unpruned_relids by adding
partitions that survive initial pruning.

To support this, PartitionedRelPruneInfo and PartitionedRelPruningData
now include a leafpart_rti_map[] array that maps partition indexes to
their corresponding RT indexes. The former is used in set_plan_refs()
when constructing unprunableRelids, while the latter is used in
ExecDoInitialPruning() to convert partition indexes returned by
get_matching_partitions() into RT indexes, which are then added to
es_unpruned_relids.

These changes make it possible for ModifyTable and LockRows nodes to
process only relations that remain unpruned after initial pruning.
ExecInitModifyTable() trims lists, such as resultRelations,
withCheckOptionLists, returningLists, and updateColnosLists, to
consider only unpruned partitions. It also creates ResultRelInfo
structs only for these partitions. Similarly, child RowMarks for
pruned relations are skipped.

By avoiding unnecessary initialization of structures for pruned
partitions, these changes improve the performance of updates and
deletes on partitioned tables during initial runtime pruning.

Due to ExecInitModifyTable() changes as described above, EXPLAIN on a
plan for UPDATE and DELETE that uses runtime initial pruning no longer
lists partitions pruned during initial pruning.

Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions)
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/cbc127917e04a978a788b8bc9d35a70244396d5b

Modified Files
--------------
src/backend/commands/copyfrom.c               |  3 +-
src/backend/executor/execMain.c               | 19 +++++-
src/backend/executor/execParallel.c           |  1 +
src/backend/executor/execPartition.c          | 83 +++++++++++++++++++++++----
src/backend/executor/execUtils.c              | 12 +++-
src/backend/executor/nodeAppend.c             |  8 +--
src/backend/executor/nodeLockRows.c           |  9 ++-
src/backend/executor/nodeMergeAppend.c        |  2 +-
src/backend/executor/nodeModifyTable.c        | 70 ++++++++++++++++++----
src/backend/optimizer/plan/planner.c          |  2 +
src/backend/optimizer/plan/setrefs.c          | 29 +++++++++-
src/backend/partitioning/partprune.c          | 15 +++++
src/backend/replication/logical/worker.c      |  3 +-
src/backend/replication/pgoutput/pgoutput.c   |  3 +-
src/include/executor/execPartition.h          |  5 +-
src/include/executor/executor.h               |  3 +-
src/include/nodes/execnodes.h                 | 10 ++++
src/include/nodes/pathnodes.h                 | 13 +++++
src/include/nodes/plannodes.h                 | 35 +++++++----
src/test/regress/expected/partition_prune.out | 46 +++++++++++++++
src/test/regress/sql/partition_prune.sql      | 20 +++++++
21 files changed, 340 insertions(+), 51 deletions(-)