F.68. ptrack — a block-level incremental backup engine for Postgres Pro #
PTRACK is a block-level incremental backup engine for Postgres Pro. If PTRACK is enabled, backup tools like pg_probackup can use its API to get information on the changed blocks on the fly when taking incremental backups. Copying only those blocks that have changed since the previous backup can significantly speed up the creation and minimize the size of backups.
PTRACK is designed to allow false positives, but to never allow false negatives: it means that all changes within the data directory except hint bits are guaranteed to be marked in the PTRACK map file, although some unchanged blocks might be included as well.
F.68.1. Setting up PTRACK #
Once you complete Postgres Pro Enterprise installation, do the following:
Add
ptrackto the shared_preload_libraries parameter in thepostgresql.conffile:shared_preload_libraries = 'ptrack'
Set ptrack.map_size parameter to a positive integer.
For optimal performance, it is recommended to set
ptrack.map_sizetoN/ 1024Nis the size of the Postgres Pro cluster, inMB. If you set this parameter to a lower value, PTRACK is more likely to map several blocks together, which leads to false-positive results when tracking changed blocks and increases the incremental backup size as unchanged blocks can also be copied into the incremental backup.Setting
ptrack.map_sizeto a higher value does not affect PTRACK operation, but keep in mind that you need up toptrack.map_size* 3 MBN> 1TB, it makes sense to use a higher value only if the efficiency and sizes of backups are more critical than the potential overhead of maintaining a large PTRACK map.Check the wal_level setting. When using PTRACK, it is required to set the wal_level parameter to
replicaor higher. Otherwise, you can lose some tracked changes if crash-recovery occurs: some commands do not write WAL at all ifwal_levelisminimal, and PTRACK map files are flushed to disk only at checkpoint time.Restart the Postgres Pro Enterprise instance for the changes to take effect, and then create the PTRACK extension:
CREATE EXTENSION ptrack;
As a result, several PTRACK functions are created, which are required for accessing PTRACK data.
Once this setup is complete, PTRACK starts tracking all the page changes in the Postgres Pro cluster and creates a ptrack.map file that stores the latest LSN values for these pages.
Note
The ptrack.map_size parameter can only be set at server start. If you change this parameter, the previously created PTRACK map file is cleared, and tracking newly changed blocks starts from scratch. To avoid losing recent changes, it is recommended to retake a full backup after modifying this setting.
F.68.2. PTRACK Configuration Parameters #
ptrack.map_size(integer) #Specifies the size of a PTRACK map file and the amount of shared memory allocated for this file, in
MB. The PTRACK map file stores the latest LSN values for all pages of the Postgres Pro cluster that have changed since PTRACK was enabled. It is not recommended to set this parameter to a value higher than 1GB. The 0 value disables PTRACK and cleans up all the related service files.The
ptrack.map_sizeparameter can only be set at server start. If you change this parameter, the previously created PTRACK map file is cleared, and tracking newly changed blocks starts from scratch. To avoid losing recent changes, it is recommended to retake a full backup after modifying this setting.Default: 0
F.68.3. PTRACK Functions #
ptrack_init_lsn() returns#pg_lsnReturns the LSN of the last PTRACK map initialization.
ptrack_get_pagemapset(#start_lsnpg_lsn) returnssetof recordReturns a list of data files changed since the specified
start_lsnwith the number and bitmap of changed pages for each file.For example:
postgres=# SELECT * FROM ptrack_get_pagemapset('0/185C8C0'); path | pagecount | pagemap ---------------------+-----------+---------------------------------------- base/16384/1255 | 3 | \x001000000005000000000000 base/16384/2674 | 3 | \x0000000900010000000000000000 base/16384/2691 | 1 | \x00004000000000000000000000 base/16384/2608 | 1 | \x000000000000000400000000000000000000 base/16384/2690 | 1 | \x000400000000000000000000ptrack_get_change_stat(#start_lsnpg_lsn) returnsrecordReturns the statistics of changes (number of files, number of changed pages and their total size in
MB) since the specifiedstart_lsn.For example:
postgres=# SELECT * FROM ptrack_get_change_stat('0/285C8C8'); files | pages | size, MB -------+-------+------------------------ 20 | 25 | 0.19531250000000000000ptrack_version() returns#textReturns PTRACK version.