A.18. Postgres Pro Shardman 14.11.1 #
Release date: 2024-03-14
This release provides new features, optimizations and bug fixes. Major changes are as follows:
A.18.1. Core and Extensions #
Enhanced the pgpro_stats extension to give a better understanding of what system resources are used for distributed queries. Now the regular
pgpro_stats_statementsview shows the gathered statistics for individual statements on the current Shardman node (they can be part of some distributed query), while thepgpro_stats_sdm_statementsview shows the gathered statistics for the distributed queries originating from the current node, that is, aggregated from all the participating nodes.Added the pgpro_pwr package compatible with Shardman. This allows Shardman users to build workload reports.
Improved the
EXPLAINoutput. If a query plan containsForeignScannodes, theEXPLAINoutput for queries executed on the remote server can now be included.Added a new configuration parameter
enable_partition_pruning_extrathat enables extended subplan pruning logic when building and executing generic plans where the set of useful partitions depends on the prepared query parameters. This allows Shardman to do initial pruning of complex subplans, joins and partial aggregates, in particular.Added metrics to the
shardman.pg_stat_csnview that show delays of the global horizon and the transaction that may cause that delay. They may be useful to research autovacuum issues.
A.18.2. Management Utilities #
Considerably improved backup and restore with the
shardmanctl probackupcommand. Notable changes are as follows:Added support of backups to an S3-compatible object storage.
Implemented selective WAL archiving on the specified shards by the
probackupsubcommand.Added two new commands
shardmanctl probackup deleteandshardmanctl probackup merge. Thedeletecommand deletes a backup with a specified ID and the archived WAL files that are no longer in use. Themergecommand merges the backups that belong to a common incremental backup chain.Added new
shardmanctl probackup set-configcommand that adds the specified settings to thepg_probackup.confor modifies the existing ones.Added a new option
log-to-consolefor thevalidatesubcommand. Set the log rotation file size to 20 MB. If this value is reached, the log file is rotated once avalidateorbackupsubcommand is launched.Increased the number of retries for some subcommands to avoid backup failures caused by large database sizes.
Added topology compatibility checks between the current Shardman cluster and the one in the backup directory to
backupandrestoresubcommands.Set the default value for the number of concurrent pg_probackup processes to the number of logical CPUs of the system.
Fixed data cleanup after a failure of a
backupsubcommand. Previously, some data of a failed backup could still remain in the repository.Fixed hanging that could occur during metadata-only restore of a Shardman cluster.
Fixed the pg_probackup issue that could occur during the schema recovery process.
Changed the behavior of metadata-only restore to avoid losing a cluster. Now the cluster is stopped before such a restore and restarted after it, a cluster that has no nodes cannot be restored from the etcd dump, and if cluster IDs of the dump and the current cluster are different, the user is asked whether restoring the cluster with the changed ID is OK.
Added new options for
archive-command:--compress,--compress-algorithm,--compress-level,--batch-size, and-j|--jobs. This helps to reduce the WAL size.Improved the
showsubcommand output. Added new flags-archiveto output the log information,-instanceand-i|--backup-idto output information for the specified backups and instances.
Updated the
getconnstrandcluster topologycommands so that they do not issue a lock on other processes. Previously, some commands failed to receive a connection string because of the locks.Fixed a panic that could occur on a Shardman cluster configured with
PlacementPolicy=manualwhen a user executed the commandshardmanctl cluster repfactor set.Hid uninformative warnings that pg_dump displayed during execution of
shardmanctl nodes addandshardmanctl probackup backupcommands.Removed a lock that was required by the
shardmanctl statuscommand. Previouslyshardmanctl statusdid not provide any useful information in case a process hung as it was waiting for the lock from that process.Added the
forceSuUserLocalPeerAuthconfiguration parameter. When enabled, it sets a peer authentication via unix socket for thepostgresuser unlessstrictUserHBAis set totrue. See sdmspec.json for details.Added a URL for Prometheus automatic service discovery metrics to shardmand.