18.11. RDMA Connections
Postgres Pro Enterprise provides support for client/server connections that use remote direct memory access (RDMA) technology. You must have an RDMA implementation set up on both client and server systems.
With RDMA, data can be sent directly into the memory of the remote system, bypassing the operating system kernel. As a result, this reduces the CPU load, ensures low network latency in distributed systems, and yields better performance.
18.11.1. Enabling Postgres Pro Enterprise Client/Server RDMA Connections
For client/server RDMA connections, Postgres Pro Enterprise uses libpq with
rsocket API. To enable RDMA connections for your Postgres Pro Enterprise database via
rsocket, you must configure both client and server systems, as follows:
Disable memory swapping on systems you are going to connect. For example, if you are starting Postgres Pro Enterprise as a systemd service, complete these steps:
Add the following line to the
Reload systemd configuration for the new setting to take effect:
On the server side, add the listen_rdma_addresses GUC variable to the
postgresql.confconfiguration file to specify TCP/IP address(es) on which the server is to listen for new RDMA connections via
rsocketfrom client applications. For example:
listen_rdma_addresses = 'server1,172.17.3.21'
You can specify a comma-separated list of host names and/or numeric IP addresses. If set to
*, this variable will enable RDMA connections via
rsocketto all the available IP interfaces.
Since all connections use the same port specified in the
portvariable, make sure to specify different IP addresses in
listen_rdma_addressesto avoid conflicts.
On the client side, modify the libpq connection parameters to use
rsocketAPI. This can be done in one of the following ways:
Add the following
rsocketsettings to the libpq connection service file
# rsocket configuration [rsocket] host=
When the client is connecting to the server via libpq, the name of the service provided in square brackets in the
pg_service.conffile must be specified in the
serviceparameter keyword, or in the corresponding
PGSERVICEenvironment variable. This approach enables you to dynamically control client/server connection type.
Alternatively, you can set the
In this case, all the client/server connections will use
18.11.2. Using RDMA with pg_dump
To connect pg_dump to the server over RDMA via
Set up the server as described in Section 18.11.1.
Once the setup is complete, launch pg_dump as usual. For example:
18.11.3. Using RDMA for Master-Standby Replication
To enable master-standby replication over RDMA connections via
rsocket, you need to configure master and standby nodes. For example, to allow client connections to the server 172.17.3.21 from 172.17.3.0/24 addresses on behalf of the
postgres user, follow the steps below:
To set up the master node for replication over RDMA connections via
pg_hba.conffile to allow client connections and replication from the specified IP addresses:
host all postgres 172.17.3.0/24 md5 host replication postgres 172.17.3.0/24 md5
listen_rdma_addresses = '172.17.3.21' wal_level = hot_standby hot_standby = on max_wal_senders = 1
listen_rdma_addressesvariable specifies the TCP/IP address(es) on which the server is to listen for new RDMA connections via
Restart the master node.
If you set
log_disconnections GUC variables, the log file will include detailed information on each client connection and disconnection, respectively. For example:
LOG: connection received: host=172.17.3.22 port=37709 with_rsocket=true LOG: disconnection: session time: 0:00:00.622 user=postgres database= host=172.17.3.22 port=37709 with_rsocket=true
To set up the standby node:
Copy the data from the master node using pg_basebackup:
datadir-x -R -h 172.17.3.21 -U postgres
All the data appears on the standby node under the specified
Make sure the
standby_mode = 'on' primary_conninfo = 'user=postgres host=172.17.3.21 port=5432 with_rsocket=true'
postgresql.conffile, clear the
listen_rdma_addresses = ''
Once the setup is complete, start the standby node. The streaming replication is now performed over the RDMA connections using