Postgres Pro
Downloads
18.11. RDMA Connections
Prev UpChapter 18. Server Setup and OperationHome Next

18.11. RDMA Connections

18.11.1. Enabling Postgres Pro Enterprise Client/Server RDMA Connections
18.11.2. Using RDMA with pg_dump
18.11.3. Using RDMA for Master-Standby Replication

Postgres Pro Enterprise provides support for client/server connections that use remote direct memory access (RDMA) technology. You must have an RDMA implementation set up on both client and server systems.

With RDMA, data can be sent directly into the memory of the remote system, bypassing the operating system kernel. As a result, this reduces the CPU load, ensures low network latency in distributed systems, and yields better performance.

18.11.1. Enabling Postgres Pro Enterprise Client/Server RDMA Connections

For client/server RDMA connections, Postgres Pro Enterprise uses libpq with rsocket API. To enable RDMA connections for your Postgres Pro Enterprise database via rsocket, you must configure both client and server systems, as follows:

  1. Disable memory swapping on systems you are going to connect. For example, if you are starting Postgres Pro Enterprise as a systemd service, complete these steps:

    1. Add the following line to the postgrespro-ent-10.service unit file: 

      LimitMEMLOCK=infinity

    2. Reload systemd configuration for the new setting to take effect: 

      systemctl daemon-reload

  2. On the server side, add the listen_rdma_addresses GUC variable to the postgresql.conf configuration file to specify TCP/IP address(es) on which the server is to listen for new RDMA connections via rsocket from client applications. For example: 

    listen_rdma_addresses = 'server1,172.17.3.21'

    You can specify a comma-separated list of host names and/or numeric IP addresses. If set to *, this variable will enable RDMA connections via rsocket to all the available IP interfaces.

    Important

    Since all connections use the same port specified in the port variable, make sure to specify different IP addresses in listen_addresses and listen_rdma_addresses to avoid conflicts.

  3. On the client side, modify the libpq connection parameters to use rsocket API. This can be done in one of the following ways:

    • Add the following rsocket settings to the libpq connection service file pg_service.conf: 

      # rsocket configuration
[rsocket]
host=hostname
port=5433
user=username
with_rsocket=true

      When the client is connecting to the server via libpq, the name of the service provided in square brackets in the pg_service.conf file must be specified in the service parameter keyword, or in the corresponding PGSERVICE environment variable. This approach enables you to dynamically control client/server connection type.

    • Alternatively, you can set the WITH_RSOCKET environment variable: 

      export WITH_RSOCKET=true

      In this case, all the client/server connections will use rsocket API.

18.11.2. Using RDMA with pg_dump

To connect pg_dump to the server over RDMA via rsocket:

  1. Set up the server as described in Section 18.11.1.

  2. Set the WITH_RSOCKET environment variable: 

    export WITH_RSOCKET=true

Once the setup is complete, launch pg_dump as usual. For example: 

pg_dump dbname -h hostname > db.sql

18.11.3. Using RDMA for Master-Standby Replication

To enable master-standby replication over RDMA connections via rsocket, you need to configure master and standby nodes. For example, to allow client connections to the server 172.17.3.21 from 172.17.3.0/24 addresses on behalf of the postgres user, follow the steps below:

To set up the master node for replication over RDMA connections via rsocket:

  1. Edit the pg_hba.conf file to allow client connections and replication from the specified IP addresses: 

    host    all            postgres     172.17.3.0/24         md5
host    replication    postgres     172.17.3.0/24         md5

  2. Edit the postgresql.conf file: 

    listen_rdma_addresses = '172.17.3.21'
wal_level = hot_standby
hot_standby = on
max_wal_senders = 1

    The listen_rdma_addresses variable specifies the TCP/IP address(es) on which the server is to listen for new RDMA connections via rsocket.

  3. Restart the master node.

Tip

If you set log_connections and log_disconnections GUC variables, the log file will include detailed information on each client connection and disconnection, respectively. For example: 

LOG:  connection received: host=172.17.3.22 port=37709 with_rsocket=true
LOG:  disconnection: session time: 0:00:00.622 user=postgres database= host=172.17.3.22 port=37709 with_rsocket=true

To set up the standby node:

  1. Set the WITH_RSOCKET environment variable: 

    export WITH_RSOCKET=true

  2. Copy the data from the master node using pg_basebackup: 

    pg_basebackup -D datadir -x -R -h 172.17.3.21 -U postgres

    All the data appears on the standby node under the specified datadir directory.

  3. Make sure the datadir/recovery.conf contains the with_rsocket parameter. 

    standby_mode = 'on'
primary_conninfo = 'user=postgres host=172.17.3.21 port=5432 with_rsocket=true'

  4. In the postgresql.conf file, clear the listen_rdma_addresses parameter: 

    listen_rdma_addresses = ''

Once the setup is complete, start the standby node. The streaming replication is now performed over the RDMA connections using rsocket API.

Prev Up Next
18.10. Secure TCP/IP Connections with SSH Tunnels Home 18.12. Registering Event Log on Windows
epub pdf