From 2b32a7e2b4d50892256c9ddf37cde6d85fe83c57 Mon Sep 17 00:00:00 2001 From: Hannu Krosing Date: Sun, 23 Aug 2026 20:17:07 +0000 Subject: [PATCH v2 6/8] Add documentation for Direct TOAST Update the PostgreSQL SGML documentation to describe Direct TOAST: - doc/src/sgml/storage.sgml: Document Plain and Direct on-disk TOAST formats, TID-based direct addressing, single chunk, flat arrays, and hierarchical DAGs. - doc/src/sgml/config.sgml: Document the toast_flavour configuration parameter. - doc/src/sgml/ref/create_table.sgml: Document the toast_flavour storage parameter. --- doc/src/sgml/config.sgml | 19 +++++++++ doc/src/sgml/ref/create_table.sgml | 18 ++++++++ doc/src/sgml/storage.sgml | 66 +++++++++++++++++++++++++----- 3 files changed, 92 insertions(+), 11 deletions(-) diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 0165eb9ec02..705c7d243b1 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -10508,6 +10508,25 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv; + + toast_flavour (enum) + + toast_flavour configuration parameter + + + + + This variable sets the default TOAST storage flavour + to use for new writes. Supported values are plain + (the default, which uses index-based chunk lookups) and direct + (which stores physical TID pointers to bypass index lookups). + This can be overridden for individual tables by setting the + toast_flavour storage parameter in + CREATE TABLE or ALTER TABLE. + + + + temp_tablespaces (string) diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml index e342585c7f0..c70d366fb01 100644 --- a/doc/src/sgml/ref/create_table.sgml +++ b/doc/src/sgml/ref/create_table.sgml @@ -1653,6 +1653,24 @@ WITH ( MODULUS numeric_literal, REM + + toast_flavour (enum) + + toast_flavour storage parameter + + + + + Sets the TOAST storage flavour for out-of-line data + written to this table. Supported values are plain (the + default, which uses index-based chunk lookup) and direct + (which stores physical TID pointers to bypass index lookup). + If not set, the default is determined by the configuration parameter + . + + + + parallel_workers (integer) diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml index 19924b98d71..f709249ec7f 100644 --- a/doc/src/sgml/storage.sgml +++ b/doc/src/sgml/storage.sgml @@ -424,17 +424,61 @@ belonging to the owning table. Every TOAST table has the columns chunk_id (an OID identifying the particular TOASTed value), chunk_seq (a sequence number for the chunk within its value), -and chunk_data (the actual data of the chunk). A unique index -on chunk_id and chunk_seq provides fast -retrieval of the values. A pointer datum representing an out-of-line on-disk -TOASTed value therefore needs to store the OID of the -TOAST table in which to look and the OID of the specific value -(its chunk_id). For convenience, pointer datums also store the -logical datum size (original uncompressed data length), physical stored size -(different if compression was applied), and the compression method used, if -any. Allowing for the varlena header bytes, -the total size of an on-disk TOAST pointer datum is therefore 18 -bytes regardless of the actual size of the represented value. +chunk_data (the actual data of the chunk), +chunk_tids (an array of chunk tuple identifiers, used by Direct TOAST), +and chunk_tid_offsets (an array of byte offsets, used for hierarchical Direct TOAST trees). + + + +PostgreSQL supports two flavours of on-disk TOAST +storage: + + + + Plain TOAST (the default): A unique index + on chunk_id and chunk_seq provides + retrieval of the values. A pointer datum representing a plain on-disk + TOASTed value stores the OID of the TOAST table + and the OID of the specific value (chunk_id). + Allowing for the varlena header bytes, the total size of a plain on-disk + TOAST pointer datum is 18 bytes. + + + + + Direct TOAST (toast_flavour = 'direct'): + Instead of looking up chunks through a B-Tree index, the TOAST + pointer directly stores the physical Tuple Identifier (TID) of the root/final chunk + in a 19-byte pointer datum. For single-chunk values (up to ~2 kB), reading the data + requires only a single buffer pin without any index access. For multi-chunk values up to 100 chunks, + the final chunk contains a chunk_tids array pointing directly + to all data chunks. For larger values (> 100 chunks), an internal hierarchical tree + is constructed with intermediate index nodes storing child TIDs and byte offsets in + chunk_tid_offsets, enabling efficient tree traversal and slice pruning. + In Direct TOAST, chunk_id is set to NULL, + and the unique index on the TOAST table is a partial index (WHERE chunk_id IS NOT NULL), + completely avoiding index maintenance and WAL logging overhead for direct chunks. + + + + + + +For convenience, pointer datums also store the logical datum size (original +uncompressed data length), physical stored size (different if compression was applied), +and the compression method used, if any. + + + +Direct TOAST is fully compatible with +pg_upgrade (including mode). +Because pg_upgrade --link preserves the physical block +and offset layout of heap and TOAST relation files, +physical TID pointers remain valid across major version upgrades. +Furthermore, plain and direct TOAST +pointers can seamlessly coexist within the same table, allowing tables upgraded from +older PostgreSQL versions to be read transparently and +gradually adopt Direct TOAST for new writes. -- 2.55.0.970.g62bdec98f9-goog