F.53. pgpro_iheap — a table access method with an integrated BRIN index #
F.53.1. Description #
The pgpro_iheap extension provides the iheap (indexed heap) table access method that is based on the built-in heap access method and integrates a BRIN index.
The iheap access method is designed to accelerate table scans when the planner selects a sequential scan plan. This scenario typically occurs when no suitable indexes are available for the query. A BRIN index is built into the iheap access method and is not user-controllable. In the rest of the document, the BRIN index integrated within iheap is referred to as the iheap index.
The iheap index is stored in a dedicated index fork of a table. This fork is an additional storage structure, analogous to the visibility map and the free space map. Within this fork, a group of BRIN indexes defined on columns is created for every N pages of the table. The BRIN index group stores the minimum and maximum values and is created for all compatible table columns. Compatible columns are those with fixed or variable length not exceeding 64 bytes. The BRIN index is not utilized for other column types. The number of pages per BRIN index group is configured using the pages_per_index index storage parameter. The default value is 256, and it can be set to any integer between 64 and 512, inclusive. A single iheap index page can store multiple BRIN index groups. The number of groups per page is determined dynamically based on the total size of indexed table columns. Consequently, one iheap page can store index information for tens or even hundreds of thousands of table pages with the standard heap access method.
When a sequential scan is executed, the system evaluates scan conditions against the BRIN index group associated with the iheap index columns prior to reading table data. If the conditions are not satisfied by the index group, the corresponding table block consisting of the number of pages set in pages_per_index is completely skipped during read operations. Skipping such blocks reduces the amount of data read from disk, thereby improving sequential scan performance.
The iheap index is most effective in the following scenarios:
As a faster alternative to the standard
heapaccess method for sequential scans when a B-tree index cannot be created, for example, due to insufficient disk space. In such cases, theiheapindex can be thousands or even tens of thousands of times smaller than the B-tree index.For queries that use the
<,<=,>, and>=comparison operators and target large sample subsets (typically 10–15% of a table) based on the predicate, in situations where a B-tree index cannot be utilized.For indexing monotonically increasing or decreasing column values, as well as values that vary per group of data pages, where the number of pages per group is configured using the pages_per_index index storage parameter. These characteristics make such values suitable for filtering. This scenario most commonly applies to tables whose rows include an insertion timestamp or an incrementing unique ID, for example, logs or tables containing historical data. In these cases, using
iheapis preferable to having multiple B-tree indexes for the following reasons:iheapis thousands or even tens of thousands of times smaller than the B-tree index.Unlike the B-tree index,
iheapindexes all compatible table columns.Although search performance with
iheapis generally slower than with the B-tree index, it is tens of times faster than with the standardheapaccess method.
F.53.2. Considerations and Limitations #
When using the
iheapaccess method, only columns with fixed or variable length are indexed, provided that the length does not exceed 64 bytes. For columns of thevarlenadata type, this size includes the column header.The column data type must be supported by the B-tree index, as the
iheapaccess method uses the comparison operators of the B-tree index, given its status as the most widely used index access method.A BRIN index group defined on all table columns must fit into a single
iheapindex page, which approximately corresponds to a single table with 1,000 columns of theintegerdata type. If there is not enough space to store the column data, an error message is returned. The message includes the number of bytes that were attempted to be allocated foriheapindex columns.The pages_per_index index storage parameter can only be changed by following these steps:
Change the access method from
iheaptoheapusing theiheap_set_amfunction.Change the
pages_per_indexvalue.Change the access method back to
iheapby callingiheap_set_amagain.
The
iheapindex is then rebuilt with the new index storage parameter value.When
DELETEorUPDATEoperations are performed, data on table pages changes, and the correspondingiheapindex groups are marked as obsolete. Although data from these groups remains accessible, the index groups must be updated by executingVACUUMor by relying on autovacuum.For tables using the
iheapaccess method, theVACUUMcommand, when executed with the disabledDISABLE_PAGE_SKIPPINGparameter, processes only those pages that belong toiheapindex groups marked asobsolete. Autovacuum uses the same approach.When the
VACUUMcommand is executed with theDISABLE_PAGE_SKIPPINGparameter enabled, it iterates through all data pages and builds theiheapindex for those pages.For the
booleandata type, only theconditions are supported.column= true
F.53.3. Installation #
The pgpro_iheap extension is included in Postgres Pro Enterprise. To enable pgpro_iheap, create the extension using the following query:
CREATE EXTENSION pgpro_iheap;
F.53.4. Functions #
The pgpro_iheap extension provides the following functions:
-
iheap_set_am(#rel_nametext,am_nametext) returnsbool Changes the table access method, for example, from
iheaptoheapor vice versa. When changing fromheaptoiheap, theiheapindex is built. When the access method is changed fromiheaptoheap, theiheapindex is dropped. Thus, the function works similarly to theCREATE INDEXandDROP INDEXcommands.Note that the
iheapindex, i.e., the table index fork, is created or dropped only after the transaction is committed, not immediately upon calling theiheap_set_amfunction. This behavior eliminates the need to roll back changes in the event of an error. Consequently, when changing the table access method fromheaptoiheap, the table can be accessed only in the subsequent transaction.After calling this function, the
PREPARE TRANSACTIONcommand fails with an error.-
iheap_page_info(#rel_nametext,blk_numint4) returnsrecord Returns general information about the
blk_numpage of theiheapindex for the tablerel_name. The output contains the following columns:col_group_size: The size of a BRIN index group defined on a column.col_groups_num: The number of BRIN index groups on aniheappage.
-
iheap_page_groups_info(#rel_nametext,blk_numint4) returnssetof record Returns BRIN index groups defined on columns of the
blk_numpage of theiheapindex for the tablerel_name. The output contains the following columns:group_num: The group number starting from0.flags: The flags set for the group:set: The group contains values of BRIN indexes defined on columns, meaning that the group is not empty.obsolete: The group data became outdated and will be updated when the vacuum is performed.
-
iheap_page_items_info(#rel_nametext,blk_numint4,group_numint4) returnssetof record Returns the contents of the
group_numBRIN index group of theblk_numpage of theiheapindex for the tablerel_name. The output contains the following columns:attno: The number of the indexed column starting from1.flags: The flags set for the column:set: The column has BRIN index values.has nulls: The column has one or moreNULLvalues.
minval: The minimum BRIN index value.maxval: The maximum BRIN index value.
F.53.5. Usage #
In the following example, the extension is created, a table with the iheap access method is created, and various operations are performed on that table.
-- Create the pgpro_iheap extension
CREATE EXTENSION pgpro_iheap;
-- Create a table with the iheap access method
CREATE TABLE test (i int, t text, b bigint) USING iheap;
-- Add data (NULL is added to the third column in the first page)
INSERT INTO test (i, t, b) VALUES (1, '1', NULL);
INSERT INTO test (SELECT i, i::text, i % 100 FROM generate_series(2, 100000) AS i);
-- Retrieve headers of the iheap index pages
SELECT i AS blkno, (info).col_group_size, (info).col_groups_num FROM
(SELECT i, iheap_page_info('test', i) AS info FROM
(SELECT i FROM generate_series(0, ((pg_relation_size('test', 'idx') / 8192) - 1)::int) AS i));
blkno | col_group_size | col_groups_num
-------+----------------+----------------
0 | 32 | 255
(1 row)
-- Retrieve information about the iheap index groups
SELECT i AS blkno, (info).group_num, (info).flags FROM
(SELECT i, iheap_page_groups_info('test', i) AS info FROM
(SELECT i FROM generate_series(0, ((pg_relation_size('test', 'idx') / 8192) - 1)::int) AS i))
WHERE array_position((info).flags, 'set') > 0;
blkno | group_num | flags
-------+-----------+-------
0 | 0 | {set}
0 | 1 | {set}
0 | 2 | {set}
(3 rows)
-- Retrieve information about columns of the iheap index groups
SELECT i AS blkno, group_num, (info).attno, (info).flags FROM
(SELECT i, group_num, iheap_page_items_info('test', i, group_num) AS info FROM
(SELECT i, (info).group_num AS group_num FROM
(SELECT i, iheap_page_groups_info('test', i) AS info FROM
(SELECT i FROM generate_series(0, ((pg_relation_size('test', 'idx') / 8192) - 1)::int) AS i))
WHERE array_position((info).flags, 'set') > 0));
blkno | group_num | attno | flags
-------+-----------+-------+-------------------
0 | 0 | 1 | {set}
0 | 0 | 2 | {}
0 | 0 | 3 | {set,"has nulls"}
0 | 1 | 1 | {set}
0 | 1 | 2 | {}
0 | 1 | 3 | {set}
0 | 2 | 1 | {set}
0 | 2 | 2 | {}
0 | 2 | 3 | {set}
(9 rows)
-- Search for a value using the iheap index
SELECT * FROM test WHERE i = 99999;
i | t | b
-------+-------+----
99999 | 99999 | 99
(1 row)