pgsql: Add EXPLAIN (IO) infrastructure with BitmapHeapScan support - Mailing list pgsql-committers

From Tomas Vondra
Subject pgsql: Add EXPLAIN (IO) infrastructure with BitmapHeapScan support
Date
Msg-id E1wAD8C-003Rvs-04@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Add EXPLAIN (IO) infrastructure with BitmapHeapScan support

Allows collecting details about AIO / prefetch for scan nodes backed by
a ReadStream. This may be enabled by a new "IO" option in EXPLAIN, and
it shows information about the prefetch distance and I/O requests.

As of this commit this applies only to BitmapHeapScan, because that's
the only scan node using a ReadStream and collecting instrumentation
from workers in a parallel query. Support for SeqScan and TidRangeScan,
the other scan nodes using ReadStream, will be added in subsequent
commits.

The stats are collected only when required by EXPLAIN ANALYZE, with the
IO option (disabled by default). The amount of collected statistics is
very limited, but we don't want to clutter EXPLAIN with too much data.

The IOStats struct is stored in the scan descriptor as a field, next to
other fields used by table AMs. A pointer to the field is passed to the
ReadStream, and updated directly.

It's the responsibility of the table AM to allocate the struct (e.g. in
ambeginscan) whenever the flag SO_SCAN_INSTRUMENT flag is passed to the
scan, so that the executor and ReadStream has access to it.

The collected stats are designed for ReadStream, but are meant to be
reasonably generic in case a TAM manages I/Os in different ways.

Author: Tomas Vondra <tomas@vondra.me>
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/flat/a177a6dd-240b-455a-8f25-aca0b1c08c6e%40vondra.me

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/681daed93169ff5bed2796dd5c8c76ee2fae1ebd

Modified Files
--------------
doc/src/sgml/ref/explain.sgml             |  12 +++
src/backend/access/heap/heapam.c          |  13 +++
src/backend/commands/explain.c            | 128 ++++++++++++++++++++++++++++++
src/backend/commands/explain_state.c      |   8 ++
src/backend/executor/nodeBitmapHeapscan.c |  19 ++++-
src/backend/storage/aio/read_stream.c     |  87 ++++++++++++++++++++
src/include/access/relscan.h              |   6 ++
src/include/access/tableam.h              |   3 +
src/include/commands/explain_state.h      |   1 +
src/include/executor/instrument.h         |   1 +
src/include/executor/instrument_node.h    |  50 ++++++++++++
src/include/storage/read_stream.h         |   2 +
src/tools/pgindent/typedefs.list          |   2 +
13 files changed, 330 insertions(+), 2 deletions(-)


pgsql-committers by date:

Previous
From: Tomas Vondra
Date:
Subject: pgsql: Switch EXPLAIN to unaligned output for json/xml/yaml
Next
From: Tomas Vondra
Date:
Subject: pgsql: auto_explain: Add new GUC auto_explain.log_io