A.10. Postgres Pro Shardman 14.11.1 #
Release date: 2024-03-14
This release provides new features, optimizations and bug fixes. Major changes are as follows:
A.10.1. Core and Extensions #
Enhanced the pgpro_stats extension to give a better understanding of what system resources are used for distributed queries. Now the regular
pgpro_stats_statements
view shows the gathered statistics for individual statements on the current Shardman node (they can be part of some distributed query), while thepgpro_stats_sdm_statements
view shows the gathered statistics for the distributed queries originating from the current node, that is, aggregated from all the participating nodes.Added the pgpro_pwr package compatible with Shardman. This allows Shardman users to build workload reports.
Improved the
EXPLAIN
output. If a query plan containsForeignScan
nodes, theEXPLAIN
output for queries executed on the remote server can now be included.Added a new configuration parameter
enable_partition_pruning_extra
that enables extended subplan pruning logic when building and executing generic plans where the set of useful partitions depends on the prepared query parameters. This allows Shardman to do initial pruning of complex subplans, joins and partial aggregates, in particular.Added metrics to the
shardman.pg_stat_csn
view that show delays of the global horizon and the transaction that may cause that delay. They may be useful to research autovacuum issues.
A.10.2. Management Utilities #
Considerably improved backup and restore with the
shardmanctl probackup
command. Notable changes are as follows:Added support of backups to an S3-compatible object storage.
Implemented selective WAL archiving on the specified shards by the
probackup
subcommand.Added two new commands
shardmanctl probackup delete
andshardmanctl probackup merge
. Thedelete
command deletes a backup with a specified ID and the archived WAL files that are no longer in use. Themerge
command merges the backups that belong to a common incremental backup chain.Added new
shardmanctl probackup set-config
command that adds the specified settings to thepg_probackup.conf
or modifies the existing ones.Added a new option
log-to-console
for thevalidate
subcommand. Set the log rotation file size to 20 MB. If this value is reached, the log file is rotated once avalidate
orbackup
subcommand is launched.Increased the number of retries for some subcommands to avoid backup failures caused by large database sizes.
Added topology compatibility checks between the current Shardman cluster and the one in the backup directory to
backup
andrestore
subcommands.Set the default value for the number of concurrent pg_probackup processes to the number of logical CPUs of the system.
Fixed data cleanup after a failure of a
backup
subcommand. Previously, some data of a failed backup could still remain in the repository.Fixed hanging that could occur during metadata-only restore of a Shardman cluster.
Fixed the pg_probackup issue that could occur during the schema recovery process.
Changed the behavior of metadata-only restore to avoid losing a cluster. Now the cluster is stopped before such a restore and restarted after it, a cluster that has no nodes cannot be restored from the etcd dump, and if cluster IDs of the dump and the current cluster are different, the user is asked whether restoring the cluster with the changed ID is OK.
Added new options for
archive-command
:--compress
,--compress-algorithm
,--compress-level
,--batch-size
, and-j|--jobs
. This helps to reduce the WAL size.Improved the
show
subcommand output. Added new flags-archive
to output the log information,-instance
and-i|--backup-id
to output information for the specified backups and instances.
Updated the
getconnstr
andcluster topology
commands so that they do not issue a lock on other processes. Previously, some commands failed to receive a connection string because of the locks.Fixed a panic that could occur on a Shardman cluster configured with
PlacementPolicy
=manual
when a user executed the commandshardmanctl cluster repfactor set
.Hid uninformative warnings that pg_dump displayed during execution of
shardmanctl nodes add
andshardmanctl probackup backup
commands.Removed a lock that was required by the
shardmanctl status
command. Previouslyshardmanctl status
did not provide any useful information in case a process hung as it was waiting for the lock from that process.Added the
forceSuUserLocalPeerAuth
configuration parameter. When enabled, it sets a peer authentication via unix socket for thepostgres
user unlessstrictUserHBA
is set totrue
. See sdmspec.json for details.Added a URL for Prometheus automatic service discovery metrics to shardmand.