7.7. Syncpoints and Consistent Backup #
To ensure that cluster binary backup is consistent, Shardman implements the syncpoints mechanism.
To achieve consistent visibility of distributed transactions, the technique of global snapshots based on physical clocks is used. Similarly, it is possible to get a consistent snapshot for backups, only the time corresponding to the global snapshot must be mapped to a set of LSN for each node. Such a set of consistent LSN in a cluster is called a syncpoint.
In a Shardman cluster, each node can generate its own independent local CSN, which does not guarantee the global ordering of values in time. Therefore, we cannot take this arbitrary local CSN as the basis for a syncpoint. Instead, Shardman chooses only those CSNs that match distributed transaction commit records as the basis of the syncpoint. These CSNs have the property of global ordering and can be used to obtain a syncpoint. The main points of this mechanism are described below.
The commit record of each completed transaction in Shardman contains the assigned CSN for this transaction. This value, together with the LSN of this record, forms a pair of values (CSN, LSN). Each of the cluster nodes stores a certain number of such pairs in RAM in a special structure - the CSNLSNMap. CSNLSNMap is a circular buffer. Each element of the map is a (CSN, LSN) pair. The map size is set by the configuration settings csn_lsn_map_size. A (CSN, LSN) pair can be added to the map only if there are no transactions on the node that can receive a CSN less than the one added. This important condition guarantees monotonous growth of CSN and LSN in CSNLSNmap, but does not guarantee that every commit record will get into the map.
When a user submits a request to create a syncpoint, a search by every CSNLSNMap is made for a largest possible CSNg for which there is an entry (CSN n, LSN) in each node and the condition CSNn <= CSNg is true. The monotonic growth property of every CSNLSNMap ensures that each found pair (CSNn, LSN) corresponds to the state of the global data at the time corresponding to CSNg. If no such value of CSNg is found, the get syncpoint operation fails and can be retried later. If such a value CSNg is found, then a syncpoint is generated as a special type of WAL record, which is duplicated on all nodes of the cluster.
By getting a syncpoint and taking the LSN for each node in the cluster from it, we can make a backup of each node, which must necessarily contain that LSN. We can also recover to this LSN using the point in time recovery (PITR) mechanism.