Thread: pgsql: Add new replication mode synchronous_commit = 'remote_apply'.
Add new replication mode synchronous_commit = 'remote_apply'. In this mode, the master waits for the transaction to be applied on the remote side, not just written to disk. That means that you can count on a transaction started on the standby to see all commits previously acknowledged by the master. To make this work, the standby sends a reply after replaying each commit record generated with synchronous_commit >= 'remote_apply'. This introduces a small inefficiency: the extra replies will be sent even by standbys that aren't the current synchronous standby. But previously-existing synchronous_commit levels make no attempt at all to optimize which replies are sent based on what the primary cares about, so this is no worse, and at least avoids any extra replies for people not using the feature at all. Thomas Munro, reviewed by Michael Paquier and by me. Some additional tweaks by me. Branch ------ master Details ------- http://git.postgresql.org/pg/commitdiff/314cbfc5da988eff8998655158f84c9815ecfbcd Modified Files -------------- doc/src/sgml/config.sgml | 14 ++-- doc/src/sgml/high-availability.sgml | 18 ++++- src/backend/access/transam/twophase.c | 6 +- src/backend/access/transam/xact.c | 16 +++- src/backend/access/transam/xlog.c | 23 ++++++ src/backend/replication/README | 18 +++-- .../libpqwalreceiver/libpqwalreceiver.c | 26 +++---- src/backend/replication/syncrep.c | 35 +++++++-- src/backend/replication/walreceiver.c | 85 ++++++++++++++++++---- src/backend/utils/misc/guc.c | 5 +- src/backend/utils/misc/postgresql.conf.sample | 2 +- src/include/access/xact.h | 7 +- src/include/access/xlog.h | 2 + src/include/access/xlog_internal.h | 2 +- src/include/replication/syncrep.h | 5 +- src/include/replication/walreceiver.h | 12 ++- 16 files changed, 209 insertions(+), 67 deletions(-)
On Wed, Mar 30, 2016 at 10:31 AM, Robert Haas <rhaas@postgresql.org> wrote: > Add new replication mode synchronous_commit = 'remote_apply'. > > In this mode, the master waits for the transaction to be applied on > the remote side, not just written to disk. That means that you can > count on a transaction started on the standby to see all commits > previously acknowledged by the master. > > To make this work, the standby sends a reply after replaying each > commit record generated with synchronous_commit >= 'remote_apply'. > This introduces a small inefficiency: the extra replies will be sent > even by standbys that aren't the current synchronous standby. But > previously-existing synchronous_commit levels make no attempt at all > to optimize which replies are sent based on what the primary cares > about, so this is no worse, and at least avoids any extra replies for > people not using the feature at all. > > Thomas Munro, reviewed by Michael Paquier and by me. Some additional > tweaks by me. The commit message does not directly mention that the spec of walrcv_receive has been changed in a backward-incompatible way so as the wait control can be done with a latch directly in walreceiver.c and not in libpqwalreceiver.c. That's not worth a mention in the release notes as this is really low-level and compilation on any code using this hook would simply fail on 9.6, so I am just mentioning it for the sake of the archives. -- Michael
On Tue, Mar 29, 2016 at 9:37 PM, Michael Paquier <michael.paquier@gmail.com> wrote: > On Wed, Mar 30, 2016 at 10:31 AM, Robert Haas <rhaas@postgresql.org> wrote: >> Add new replication mode synchronous_commit = 'remote_apply'. >> >> In this mode, the master waits for the transaction to be applied on >> the remote side, not just written to disk. That means that you can >> count on a transaction started on the standby to see all commits >> previously acknowledged by the master. >> >> To make this work, the standby sends a reply after replaying each >> commit record generated with synchronous_commit >= 'remote_apply'. >> This introduces a small inefficiency: the extra replies will be sent >> even by standbys that aren't the current synchronous standby. But >> previously-existing synchronous_commit levels make no attempt at all >> to optimize which replies are sent based on what the primary cares >> about, so this is no worse, and at least avoids any extra replies for >> people not using the feature at all. >> >> Thomas Munro, reviewed by Michael Paquier and by me. Some additional >> tweaks by me. > > The commit message does not directly mention that the spec of > walrcv_receive has been changed in a backward-incompatible way so as > the wait control can be done with a latch directly in walreceiver.c > and not in libpqwalreceiver.c. That's not worth a mention in the > release notes as this is really low-level and compilation on any code > using this hook would simply fail on 9.6, so I am just mentioning it > for the sake of the archives. Yeah, I didn't really think that mattered much. I'm not really sure what you even mean by backward-incompatible -- AFAIK, that's a private interface which we can whack around whenever we like. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Wed, Mar 30, 2016 at 10:39 AM, Robert Haas <robertmhaas@gmail.com> wrote: > On Tue, Mar 29, 2016 at 9:37 PM, Michael Paquier > <michael.paquier@gmail.com> wrote: >> On Wed, Mar 30, 2016 at 10:31 AM, Robert Haas <rhaas@postgresql.org> wrote: >>> Add new replication mode synchronous_commit = 'remote_apply'. >>> >>> In this mode, the master waits for the transaction to be applied on >>> the remote side, not just written to disk. That means that you can >>> count on a transaction started on the standby to see all commits >>> previously acknowledged by the master. >>> >>> To make this work, the standby sends a reply after replaying each >>> commit record generated with synchronous_commit >= 'remote_apply'. >>> This introduces a small inefficiency: the extra replies will be sent >>> even by standbys that aren't the current synchronous standby. But >>> previously-existing synchronous_commit levels make no attempt at all >>> to optimize which replies are sent based on what the primary cares >>> about, so this is no worse, and at least avoids any extra replies for >>> people not using the feature at all. >>> >>> Thomas Munro, reviewed by Michael Paquier and by me. Some additional >>> tweaks by me. >> >> The commit message does not directly mention that the spec of >> walrcv_receive has been changed in a backward-incompatible way so as >> the wait control can be done with a latch directly in walreceiver.c >> and not in libpqwalreceiver.c. That's not worth a mention in the >> release notes as this is really low-level and compilation on any code >> using this hook would simply fail on 9.6, so I am just mentioning it >> for the sake of the archives. > > Yeah, I didn't really think that mattered much. I'm not really sure > what you even mean by backward-incompatible -- AFAIK, that's a private > interface which we can whack around whenever we like. By "Backward-incompatible", I mean that any custom library using this walrcv hook is not going to compile anymore and we don't provide a pre-9.5 equivalent. I don't think that's worth worrying though, I have yet to see this interface being used for something else than libpqwalreceiver to be honest. -- Michael
On Tue, Mar 29, 2016 at 9:43 PM, Michael Paquier <michael.paquier@gmail.com> wrote: > On Wed, Mar 30, 2016 at 10:39 AM, Robert Haas <robertmhaas@gmail.com> wrote: >> On Tue, Mar 29, 2016 at 9:37 PM, Michael Paquier >> <michael.paquier@gmail.com> wrote: >>> On Wed, Mar 30, 2016 at 10:31 AM, Robert Haas <rhaas@postgresql.org> wrote: >>>> Add new replication mode synchronous_commit = 'remote_apply'. >>>> >>>> In this mode, the master waits for the transaction to be applied on >>>> the remote side, not just written to disk. That means that you can >>>> count on a transaction started on the standby to see all commits >>>> previously acknowledged by the master. >>>> >>>> To make this work, the standby sends a reply after replaying each >>>> commit record generated with synchronous_commit >= 'remote_apply'. >>>> This introduces a small inefficiency: the extra replies will be sent >>>> even by standbys that aren't the current synchronous standby. But >>>> previously-existing synchronous_commit levels make no attempt at all >>>> to optimize which replies are sent based on what the primary cares >>>> about, so this is no worse, and at least avoids any extra replies for >>>> people not using the feature at all. >>>> >>>> Thomas Munro, reviewed by Michael Paquier and by me. Some additional >>>> tweaks by me. >>> >>> The commit message does not directly mention that the spec of >>> walrcv_receive has been changed in a backward-incompatible way so as >>> the wait control can be done with a latch directly in walreceiver.c >>> and not in libpqwalreceiver.c. That's not worth a mention in the >>> release notes as this is really low-level and compilation on any code >>> using this hook would simply fail on 9.6, so I am just mentioning it >>> for the sake of the archives. >> >> Yeah, I didn't really think that mattered much. I'm not really sure >> what you even mean by backward-incompatible -- AFAIK, that's a private >> interface which we can whack around whenever we like. > > By "Backward-incompatible", I mean that any custom library using this > walrcv hook is not going to compile anymore and we don't provide a > pre-9.5 equivalent. I don't think that's worth worrying though, I have > yet to see this interface being used for something else than > libpqwalreceiver to be honest. Yeah, I'd be very surprised if anyone did that. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company