Re: Logical replication and multimaster - Mailing list pgsql-hackers

From Craig Ringer
Subject Re: Logical replication and multimaster
Date
Msg-id CAMsr+YEfTEp35bQ5ia__nDwc2MwzYUQcuQ__vT_b=UNOHyooMA@mail.gmail.com
Whole thread Raw
In response to Re: Logical replication and multimaster  (Simon Riggs <simon@2ndQuadrant.com>)
Responses Re: Logical replication and multimaster  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers

There are definitely two clear places where additional help would be useful and welcome right now.

Here's a shortlist of replication-related open items I put together earlier. These are all independent things it'd be helpful to have when working with logical replication in PostgreSQL. There are a lot of medium-sized pieces of independent work still to be done to make some things work well.

For example, you can't have multimaster logical replication with physical failover nodes for each MM node unless we replicate slot create/drop/advance over physical replication and copy them with pg_basebackup.

Handling big transactions better:

* logical decoding interleaved transaction streaming as discussed earlier

Automatic DDL replication:

* DDL deparse extension (Álvaro started some work on this)
* Better way to link the pg_temp_nnn tables generated during table rewrite to the original table in decoding
* logical decoding support for prepared xacts (before commit prepared). Useful for DDL replication, other consensus operations.

Using physical replication/PITR with logical replication:

* replication slots replicated to physical standbys ("Failover slots")
* pg_basebackup should copy slots
* slots should follow timeline changes

Failover between logical replicas:

* logical decoding support for sequences
* logical decoding support for slot create/drop/advance
* Separate slot WAL retention from confirmed commit point so you can say "I've replayed up to 1/0000AB but you should keep from 1/000001". Needed in async MM to cope with node loss properly. Will write it up separately.

Multimaster:

* Sequence Access Method

Other logical decoding enhancements:

* Support exporting a new snapshot from an existing logical slot. Useful for COPYing new tables not previously replicated when added to a replication set, for resync'ing tables, comparing tables, etc.
* WAL messages. Useful for voting, replay confirmation, etc. Rendered largely unnecessary by xact interleaved streaming.

Misc:

* An API to enumerate currently registered bgworkers
* A way to securely make a libpq connection from a bgworker without messing with passwords etc. Generate one-time cookies, sometihng like that.
* (unimportant but nice API fix): BGW_NO_RESTART_ON_CRASH

  

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

pgsql-hackers by date:

Previous
From: Andreas Karlsson
Date:
Subject: Re: Re: Reusing abbreviated keys during second pass of ordered [set] aggregates
Next
From: Peter Geoghegan
Date:
Subject: Re: Re: Reusing abbreviated keys during second pass of ordered [set] aggregates