Request a Demo
27.4. Reference for the biha Extension
Prev UpChapter 27. Built-in High Availability (BiHA)Home Next

27.4. Reference for the biha Extension #

27.4.1. Configuration Parameters
27.4.2. Functions
27.4.3. Views

biha is a Postgres Pro extension that allows managing a BiHA cluster.

This section contains information about the biha extension configuration parameters, functions, and views.

27.4.1. Configuration Parameters #

The biha extension provides a set of configuration parameters described below that are specific to the BiHA cluster.

27.4.1.1. Cluster Configuration #

Important

  • When setting the cluster configuration parameters, you must ensure network reliability for the changes to affect all cluster nodes without any errors.

  • It is not recommended to modify any cluster configuration parameters while one of the nodes is in the NODE_ERROR state, as the changes may fail to be applied on that node.

biha.autorewind (boolean) #

An optional parameter that controls the automatic rewind policy for that node, against which the pg_rewind must be executed. For example, for the old leader when synchronizing it with the new leader. The default value is false meaning that the automatic rewind is not performed. When the value is set to true, the automatic rewind is performed after the error that usually causes the NODE_ERROR state of the node.

For more information, see Automatic Cluster Synchronization after Failover.

biha.callbacks_timeout (integer) #

Sets time for execution of all callbacks for a single event, in milliseconds. The default value is 10000 (10 seconds). The minimum value is 1000 (1 second).

The value of the biha.callbacks_timeout parameter can be different for different nodes. Changing the value for the leader does not change it for the followers.

biha.can_be_leader (boolean) #

Determines the ability of a node to become the leader. The default value is true. If set to false, the node cannot stand as a candidate in elections of the new leader.

A node that cannot become the leader may have the largest WAL. In this case, if elections are required, those nodes that can stand as candidates attempt to receive missing data from that node. If they succeed, one of these nodes becomes the leader. If they fail to receive missing data, the automatic rewind process is launched on the node that cannot become the leader. If biha.autorewind is not enabled, the node state changes to NODE_ERROR.

biha.can_vote (boolean) #

Determines whether a node is allowed to vote. The default value is true. If set to false, the node cannot vote, as well as cannot stand as a candidate in elections of the new leader.

biha.flw_ro (boolean) #

Determines whether a follower is available for read operations. If set to off, reading from this follower is prohibited. The default value is on.

biha.heartbeat_max_lost (integer) #

Specifies the maximum number of heartbeats that can be missed before the node is considered offline. This parameter can be set with the biha.set_heartbeat_max_lost function. The default value is 10.

Nodes in all states use the heartbeat timeout value calculated as biha.heartbeat_max_lost * biha.heartbeat_send_period to determine if other nodes are online. For example, if Node A receives no heartbeats from Node B within the heartbeat timeout, the Node B state changes to UNKNOWN in the biha.status_v view of Node A indicating that Node A considers Node B offline. The leader in the LEADER_RW state additionally uses the timeout specified in the biha.no_wal_on_follower parameter.

biha.heartbeat_send_period (integer) #

Specifies the heartbeat sending frequency, in milliseconds. This parameter can be set with the biha.set_heartbeat_send_period function. The default value is 1000.

Nodes in all states use the heartbeat timeout value calculated as biha.heartbeat_max_lost * biha.heartbeat_send_period to determine if other nodes are online. For example, if Node A receives no heartbeats from Node B within the heartbeat timeout, the Node B state changes to UNKNOWN in the biha.status_v view of Node A indicating that Node A considers Node B offline. The leader in the LEADER_RW state additionally uses the timeout specified in the biha.no_wal_on_follower parameter.

biha.host (text) #

Specifies the host of the cluster node. It is not recommended to modify this parameter.

biha.id (integer) #

Specifies the ID of the high-availability cluster node. This parameter is unique for each node of the cluster. It is not recommended to modify this parameter.

biha.minnodes (integer) #

Specifies the minimum number of operational nodes for the leader node to be open for write transactions. This parameter can be set with the biha.set_minnodes function. If you do not specify the --minnodes option in bihactl init command, the biha.minnodes value equals the biha.nquorum value.

When setting up this value, consider the split-brain risk. It is recommended to use the following formula: (total_number_of_nodes + 1)/2. For example, if your cluster has 3 nodes, the minnodes value must be 2.

Nodes with the biha.can_vote parameter set to false are ignored.

biha.no_wal_on_follower (integer) #

Specifies the timeout of the replication slot position advancement, in milliseconds. This parameter can be set with the biha.set_no_wal_on_follower function. The default value is 20000.

BiHA uses this configuration parameter for the following purposes:

  • When you nominate a follower as the new leader using the biha.set_leader function, the follower state changes to FOLLOWER_OFFERED. The follower in this state uses biha.no_wal_on_follower to determine if it must stop waiting for missing WAL records from the current leader and can become the new leader.

    A follower in the FOLLOWER_OFFERED state checks whether its WAL location has been updated. Update of WAL location confirms that the follower keeps receiving replication data from the leader. If the follower does not receive WAL records longer than biha.no_wal_on_follower, the follower state changes to LEADER_RO. In this case, the log contains the following message: “Timed out waiting for new WAL records from current leader”.

  • The leader in the LEADER_RW state uses biha.no_wal_on_follower in conjunction with the heartbeat timeout value (which is calculated as biha.heartbeat_max_lost * biha.heartbeat_send_period) to determine if other nodes are online.

    The leader checks the replication slot positions of other nodes. If the replication slot position updates, it means that the node receives replication data from the leader and the node is online. If the leader does not receive heartbeats from the node within the above mentioned heartbeat timeout and its replication slot position does not change within the biha.no_wal_on_follower timeout value, the leader considers the node offline and its state changes to UNKNOWN in the biha.status_v view of the leader.

biha.node_priority (integer) #

Sets the node priority in a cluster with synchronous replication, in seconds. The value determines the timeout that must be reached before the node stands as a candidate in elections. The zero value indicates the highest priority. The default value is -1, meaning that the parameter is ignored.

Important

To ensure correct operation of this parameter, set the --sync-standbys value one unit less than the total number of cluster nodes.

biha.nquorum (integer) #

Specifies the minimum number of nodes, which must vote for the new leader node if the current leader is down. This parameter can be set with the biha.set_nquorum function.

When setting up this value, consider the split-brain risk. It is recommended to use the following formula: (total_number_of_nodes + 1)/2. For example, if your cluster has 3 nodes, the nquorum value must be 2.

Nodes with the biha.can_vote parameter set to false are ignored.

biha.port (integer) #

Specifies the port used to exchange service information between nodes. This parameter is required to establish a connection with the cluster. It is not recommended to modify this parameter.

biha.use_ssl (boolean) #

Specifies whether to use the protected mode of the service information exchange between cluster nodes with SSL/TLS over the biha control channel. It is not recommended to modify this parameter.

biha.autowaltrim (boolean) #

Manages the automatic WAL trimming policy within a single node. When set to true (default), BiHA automatically schedules and executes the WAL trimming procedure if possible. If set to false, the WAL trimming procedure is not executed and cluster synchronization is performed by means of pg_rewind.

For more information, see Automatic Cluster Synchronization after Failover.

27.4.1.2. SSL Authentication Configuration #

Configuration parameters listed below relate to the SSL authentication of the biha_replication_user role. For more information about SSL authentication configuration, see Configuring SSL for User Authentication.

biha.user_biha_cert (text) #

Specifies the path to the SSL public certificate used for the biha_replication_user role authentication.

biha.user_biha_key (text) #

Specifies the path to the SSL private key used for the biha_replication_user role authentication.

biha.ssl_mode (text) #

Specifies the authentication policy for the biha_replication_user role. The following modes are supported:

  • verify-full (default)

  • require

  • verify-ca

For more information about modes, see sslmode.

27.4.1.3. biha Logging Levels #

biha.BihaLog_log_level (enum) #

Specifies the logging level to provide general information about the operation of biha components. The default value is LOG.

biha.BcpTransportDebug_log_level (enum) #

Specifies the logging level to provide debugging information about the control channel operation. The default value is DEBUG4.

biha.BcpTransportDetails_log_level (enum) #

Specifies the logging level to provide detailed information about the control channel operation. The default value is DEBUG4.

biha.BcpTransportLog_log_level (enum) #

Specifies the logging level to provide general information about the control channel operation. The default value is DEBUG4.

biha.BcpTransportWarn_log_level (enum) #

Specifies the logging level to provide warnings of likely problems in the control channel. The default value is DEBUG4.

biha.NodeControllerDebug_log_level (enum) #

Specifies the logging level to provide debugging information about the node controller operation. The default value is DEBUG4.

biha.NodeControllerDetails_log_level (enum) #

Specifies the logging level to provide detailed information about the node controller operation. The default value is DEBUG4.

biha.NodeControllerLog_log_level (enum) #

Specifies the logging level to provide general information about the node controller operation. The default value is DEBUG4.

biha.NodeControllerWarn_log_level (enum) #

Specifies the logging level to provide warnings of likely problems in the node controller. The default value is DEBUG4.

27.4.2. Functions #

All functions listed below must be called from the biha_db database, for example: 

psql biha_db -c "SELECT biha.set_leader(2)"

27.4.2.1. Cluster Composition #

biha.set_leader (id integer) returns boolean #

Sets the leader node manually. It is recommended to call this function on the node that you want to set as the new leader.

Note

Calling the biha.set_leader function on the current leader is not recommended. If you call the function on the current leader in the LEADER_RW state, in case of a successful switchover request, there may not be enough time for the request result to be sent to the client before the current leader reboots for demotion.

biha.remove_node (id integer) returns boolean #

Removes the node from the cluster. Before removing, the node must be stopped. This function can only be called on the leader node.

27.4.2.2. Common Cluster Configuration #

The following functions are used to change configuration parameters that must have the same values on all cluster nodes. These functions must only be called on the leader node. The changes are applied to all cluster nodes.

You do not need to restart the cluster nodes after calling a function for the changes to take effect unless the function description explicitly specifies otherwise.

biha.set_heartbeat_max_lost (integer) returns boolean #

Sets the biha.heartbeat_max_lost value.

biha.set_heartbeat_send_period (integer) returns boolean #

Sets the biha.heartbeat_send_period value, in milliseconds.

biha.set_no_wal_on_follower (integer) returns boolean #

Sets the biha.no_wal_on_follower value, in milliseconds.

biha.set_minnodes (integer) returns boolean #

Sets the biha.minnodes value.

biha.set_nquorum (integer) returns boolean #

Sets the biha.nquorum value.

biha.set_nquorum_and_minnodes (integer, integer) returns boolean #

Sets the biha.nquorum and biha.minnodes values.

27.4.2.3. Replication #

The following functions are used to configure and manage replication.

biha.add_to_ssn (id integer) returns boolean #

Note

You can call this function only when quorum-based synchronous replication is enabled.

Adds a node with the specified ID to the list of synchronous standbys of the synchronous_standby_names parameter. You must specify the node which name is not yet in the list.

biha.get_ssn() returns varchar #

Displays the current value of the synchronous_standby_names parameter.

biha.remove_from_ssn (id integer) returns boolean #

Note

You can call this function only when quorum-based synchronous replication is enabled.

Removes the node with the specified ID from the list of synchronous standbys of the synchronous_standby_names parameter. You must specify the node which name is already in the list.

biha.set_ssn (VARIADIC ids integer) returns boolean #

Note

You can call this function only when quorum-based synchronous replication is enabled.

Sets nodes with the IDs specified in the ids array as the list of synchronous standbys of the synchronous_standby_names parameter.

biha.set_sync_standbys (integer) returns boolean #

Enables quorum-based synchronous replication by setting the synchronous_standby_names parameter and specifying the number of synchronous standbys (quorum) with the ANY method. The value must be positive, higher than the MIN field value of the synchronous_standby_names parameter if set, and must not exceed the number of followers excluding referee. For more information, see Replication Configuration.

biha.set_sync_standbys_min (integer) returns boolean #

Note

You can call this function only when quorum-based synchronous replication is enabled.

Sets the MIN field value of the synchronous_standby_names parameter, i.e. the minimum number of synchronous standbys that must be available for the leader node to continue operation.

Using this function, you can also manage relaxed quorum-based synchronous replication. To disable relaxed quorum-based synchronous replication, set the -1 value.

27.4.2.4. Cluster Monitoring #

biha.config () returns setof record #

Returns the cluster configuration parameter values: id, term, nquorum, minnodes, heartbeat_send_period, heartbeat_max_lost, no_wal_on_follower, sync_standbys_min, priority, can_be_leader, can_vote, mode.

biha.error_details () returns setof record #

Returns the description of why the node transferred to the NODE_ERROR state. The returned record contains the type of the error, its details, the place it occurred specifying begin_lsn, end_lsn, and identifier of the current and the next timeline, as well as replay_lsn.

biha.nodes () returns setof record #

Defines the biha.nodes_v view, which is described in detail in the biha.nodes_v section.

biha.status () returns setof record #

Defines the biha.status_v view, which is described in detail in the biha.status_v section. It is not recommended to use this function as it provides raw data for the view.

27.4.2.5. Callback Management #

biha.register_callback (event text, func text, database text, executor text, priority integer) returns integer #

Adds a new callback and returns its unique ID. This function can only be called from the leader node in the LEADER_RW state. The new callback is replicated to the followers.

Note

On the biha side, there is no check that func exists in database. If the specified function does not exist, the callback fails.

See Registering Callbacks for the example of using the biha.register_callback function.

Table 27.3. Variable Definitions

NameTypeDescription
eventtext Event of the BiHA cluster that triggers a callback. For more information about events and corresponding callback types, see Callback Types.
functext Name of the SQL function that biha executes when event happens. This function must be located in database, otherwise it cannot be executed.
databasetextDatabase where func is executed.
executortext

User that executes func. This parameter is optional. The default value is biha_callbacks_user.

priorityinteger The lower the value, the sooner the callback is executed. This parameter is optional. The default value is 0.

biha.unregister_callback(callback_id) #

Deletes a callback. This function can only be called on the leader node in the LEADER_RW state. See Unregistering Callbacks for the example of using the biha.unregister_callback function.

27.4.2.6. Other Functions #

biha.get_magic_string () returns string #

Generates a magic string for the cluster node.

biha.reset_node_error () returns boolean #

Resets the NODE_ERROR state on a node. Use this function after you fix the issue that caused the NODE_ERROR state. For more information, see Restoring the Node from the NODE_ERROR State.

27.4.3. Views #

To query the views listed below, you must first connect to the biha_db database.

27.4.3.1. biha.nodes_v #

This view displays the connection status of nodes in the cluster. For the node that the view is queried on, the following columns contain NULL: state, since_conn_start, conn_count.

Table 27.4. The biha.nodes_v view

Column NameDescription
id The node ID.
host The host of the node.
port The port of the node.
state The connection state of the node. This column may contain one of the following values: ACTIVE, CONNECTING, IDLE, or INIT.
since_conn_start The time since the node connection.
conn_count The number of times the node was connected since the start of the cluster.

27.4.3.2. biha.status_v #

This view displays the state of nodes in the cluster.

Table 27.5. The biha.status_v View

Column NameDescription
id The node ID.
leader_id The leader node ID.
term The term of the node. This is used for the purposes of the leader election. The data type is integer.
online Shows whether the node is online. The data type is boolean (t or f).
state

The state of the node. This column may contain one of the following values:

  • PRESTARTUP — the initial state of the node at the BiHA cluster launch. The node sends heartbeats and executes pg_rewind if scheduled. Otherwise, the node proceeds to STARTUP.

  • STARTUP — the node waits for the Postgres Pro startup process to reach the consistency point.

  • CSTATE_FORMING — the node receives and sends heartbeats to determine what state it must transfer to.

  • LEADER_RO — the node is the leader available for read-only operations.

  • LEADER_RW — the node is the leader available for read and write operations.

  • FOLLOWER — the node is a replica of the leader. If biha.can_be_leader and biha.can_vote are set to true, the follower can be elected as the new leader.

  • FOLLOWER_OFFERED — the node was manually nominated as the new leader by means of the biha.set_leader function. This state is required for the nominated node to receive missing data from the old leader.

  • CANDIDATE — the node stands as a candidate in elections of the new leader.

  • REFEREE — the node is the cluster referee. The state is the same both for the referee and referee_with_wal modes.

  • NODE_ERROR — the node is non-operational due to an error. Nodes in this state are unable to vote during elections. Reading from such nodes is prohibited. To get more information about the error, use the biha.error_details function. To restore the faulty node, see Restoring the Node from the NODE_ERROR State.

  • UNKNOWN — the node is offline for the current node.

last_known_state The last known state of the node.
since_last_hb The time since the last received heartbeat message. The data type is interval.

Prev Up Next
27.3. Administration Home 27.5. Reference for the bihactl Utility
epub pdf