Postgres Pro
Downloads
F.39. pgpro_bindump — a replication protocol module for backup and restore
Prev UpAppendix F. Additional Supplied Modules and Extensions Shipped in postgrespro-std-14-contribHome Next

F.39. pgpro_bindump — a replication protocol module for backup and restore

F.39.1. Architecture
F.39.2. Limitations and Requirements
F.39.3. Installation and Setup
F.39.4. Replication Commands
F.39.5. Authors

pgpro_bindump is a module designed to implement additional commands for the Postgres Pro replication protocol, specifically for use with the pg_probackup3 utility. pgpro_bindump offers the following benefits:

  • Simplified communication: pgpro_bindump does not require the SSH protocol and associated configurations to connect to a database server.

  • Integrated WAL transfer: WAL files are transferred together with data files, reducing the risk of data loss during backups.

  • Optimized storage: Files are not copied as they are, but are archived using the pg_probackup3 own format, CBOR (RFC 8949 Concise Binary Object Representation), which enhances the storage efficiency.

F.39.1. Architecture

A dedicated backend process is created to handle connections from external applications. You can use the pg_probackup3 utility or a custom application to send requests to this backend.

Multiple connections can be established for the same application if needed.

Since the pgpro_bindump module has direct access to the database files, it does not require any additional permissions.

pgpro_bindump uses the libpgprobackup, along with its built-in encoder module, to send data to the application.

The walsender_plugin_libraries plugin has been added to simplify the process of adding and utilizing new replication commands, including custom ones.

F.39.2. Limitations and Requirements

pgpro_bindump currently has the following limitations and requirements:

  • libpgprobackup.so and libpb3_encoder.so are required for pgpro_bindump operation.

F.39.3. Installation and Setup

Once you complete Postgres Pro installation, follow the steps below.

  1. Specify the following parameters in the postgresql.conf file: 

    shared_preload_libraries = 'pgpro_bindump'
wal_level = 'replica' # or 'logical'
walsender_plugin_libraries = 'pgpro_bindump'

  2. Restart the Postgres Pro instance.

F.39.4. Replication Commands

The pgpro_bindump module implements the following replication commands:

  • just-noop — a do-nothing command.

  • start_backup — starts the backup process.

  • copy_files — copies files in the CBOR format.

  • stop_backup — stops the backup process.

The backup process runs in the PRO mode — an advanced data source mode that incorporates all available optimizations and enhancements, including the proprietary replication protocol and specialized replication commands.

To send commands from a user application to pgpro_bindump and to receive the output data from pgpro_bindump by the application, use libpq.

Note

An established connection to a running Postgres Pro server is required.

F.39.4.1. start_backup

start_backup LABEL label, INCR_MODE incr_mode, START_LSN start_lsn, TRANSFER_MODE transfer_mode,
[TRANSFER_BUFFER_SIZE buffer_size]

Starts the backup process. Sends the following data to the application:

  • start_lsn — LSN at the start of the backup.

  • start_tli — the timeline ID at the start of the backup.

  • min_xid — the minimal transaction ID at the start of the backup.

  • min_multixact — the minimal multixact ID at the start of the backup.

  • tablespace_list — a list of tablespaces included in the backup.

  • database_list — a list of databases included in the backup.

LABEL label

The name of the backup.

INCR_MODE incr_mode

The type of incremental backup. Possible values: PTRACK, DELTA.

START_LSN start_lsn

An XLogRecPtr value representing LSN at the start of the backup.

TRANSFER_MODE transfer_mode

The method to be used for sending data from a server to an application. Possible values:

  • raw — unpacked data is sent in random blocks of arbitrary size.

  • packed — packed data is sent in blocks of 128 KB with a common header.

packed is the default value.

TRANSFER_BUFFER_SIZE buffer_size

A numeric value representing the buffer size. The preset (unchanged) value is 131072 (128 KB).

F.39.4.2. copy_files

copy_files VERIFY_CHECKSUMS [TRUE | FALSE], COMPRESS_ALG compress_alg, COMPRESS_LVL compress_lvl, TRANSFER_MODE transfer_mode,
[INCR_MODE incr_mode], [START_LSN start_lsn], [TRANSFER_BUFFER_SIZE buffer_size],
[WORKER_INFO number_of_workers:worker_id]

Forwards blocks of data files and WAL files packaged in the CBOR format.

VERIFY_CHECKSUMS

A boolean value (true or false) indicating whether to verify checksums.

COMPRESS_ALG compress_alg

The compression algorithm to be used. Possible values: zlib, zstd, lz4, none.

COMPRESS_LVL compress_lvl

The compression level. Possible values: 1, 2, 3, 4, 5.

TRANSFER_MODE transfer_mode

The method to be used for sending data from a server to an application. Possible values:

  • raw — unpacked data is sent in random blocks of arbitrary size.

  • packed — packed data is sent in blocks of 128 KB with a common header.

packed is the default value.

INCR_MODE incr_mode

The type of incremental backup. Possible values: PTRACK, DELTA.

START_LSN start_lsn

An XLogRecPtr value representing LSN at the start of the backup.

TRANSFER_BUFFER_SIZE buffer_size

A numeric value representing the buffer size. The preset (unchanged) value is 131072 (128 KB).

WORKER_INFO number_of_workers:worker_id

The number of workers and the worker ID.

F.39.4.3. stop_backup

stop_backup STREAM [TRUE | FALSE], COMPRESS_ALG compress_alg, COMPRESS_LVL compress_lvl,
[EXTERNALS externals] [TRANSFER_BUFFER_SIZE buffer_size], [TRANSFER_MODE transfer_mode]

Completes the copy process and closes the existing connection. Sends the following data to the application:

  • stop_lsn — LSN at the end of the backup.

  • stop_tli — the timeline ID at the end of the backup.

  • The backup.control file.

  • A set of WAL files (only if WAL streaming is enabled).

STREAM

A boolean value (true or false) indicating whether WAL streaming is enabled.

COMPRESS_ALG compress_alg

The compression algorithm to be used. Possible values: zlib, zstd, lz4, none.

COMPRESS_LVL compress_lvl

The compression level. Possible values: 1, 2, 3, 4, 5.

EXTERNALS externals

A list of external directories.

TRANSFER_BUFFER_SIZE buffer_size

A numeric value representing the buffer size. The preset (unchanged) value is 131072 (128 KB).

TRANSFER_MODE transfer_mode

The method to be used for sending data from a server to an application. Possible values:

  • raw — unpacked data is sent in random blocks of arbitrary size.

  • packed — packed data is sent in blocks of 128 KB with a common header.

packed is the default value.

F.39.5. Authors

Postgres Professional, Moscow, Russia.

Prev Up Next
F.38. pg_proaudit Home F.40. pg_prewarm
pdf epub