Re: Replication Node Identifiers and crashsafe Apply - Mailing list pgsql-hackers
From | Andres Freund |
---|---|
Subject | Re: Replication Node Identifiers and crashsafe Apply |
Date | |
Msg-id | 20131211153833.GB25227@awork2.anarazel.de Whole thread Raw |
List | pgsql-hackers |
Progress - v0.2 Reply-To: In-Reply-To: <20131114172632.GE7522@alap2.anarazel.de> Hi, So, the reaction to the first version of the patch was primarily concerned with the format of the replication identifiers and the lack of an SQL accessible API. With some doubts about the neccessity of the feature added. I've changed stuff around so replication identifiers are externally just identified by a string, without a predefined structure. The only limit is that it needs to fit into a btree. All lookups are done using varchar_pattern_ops to make sure we don't have a problem with differing collations in different databases. There's currently no limit to the encoding stored which should be fixed to be ascii-only, but I haven't found appropriate functionality. And we don't seem to put restriction into place for content in pg_shdescription either? Perhaps the identifier should be a bytea instead, but they are so annoying to use, especially since the default output format has become hex. I've also added a full sql API to the functionality. It's currently superuser only, not sure if we need to change that? Example usage is: postgres=# SELECT pg_replication_identifier_create('bdr: this-is-my-identifier'); pg_replication_identifier_create ---------------------------------- 4 (1 row) postgres=# SELECT * FROM pg_replication_identifier WHERE riident = 4; riident | riname ---------+---------------------------- 4 | bdr: this-is-my-identifier Table "pg_catalog.pg_replication_identifier" Column | Type | Modifiers ---------+------+----------- riident | oid | not null riname | text | Indexes: "pg_replication_identifier_riiident_index" UNIQUE, btree (riident), tablespace "pg_global" "pg_replication_identifier_riname_index" UNIQUE, btree (riname varchar_pattern_ops), tablespace "pg_global" Tablespace: "pg_global" postgres=# SELECT pg_replication_identifier_setup_replaying_from('bdr: this-is-my-identifier'); pg_replication_identifier_setup_replaying_from ------------------------------------------------ (1 row) postgres=# BEGIN; BEGIN postgres=# SELECT pg_replication_identifier_setup_tx_origin('0/123456', '2013-12-11 15:14:59.219737+01') pg_replication_identifier_setup_tx_origin ------------------------------------------- (1 row) postgres=# INSERT INTO frak VALUES (11, 1); INSERT 0 1 postgres=# SELECT pg_current_xlog_insert_location(); pg_current_xlog_insert_location --------------------------------- 0/18E6FB8 (1 row) postgres=# commit; COMMIT Note the end of the generated wal record, that's not normally included: rmgr: Transaction len (rec/tot): 56/ 88, tx: 708, lsn: 0/018E7B88, prev 0/018E7B48, bkp: 0000, desc: commit:2013-12-11 16:09:14.931325 CET origin 4, lsn 0/123456, at 2013-12-11 15:14:59.219737 CET To see where we are at there's a SRF: postgres=# SELECT * FROM pg_replication_identifier_progress; local_id | external_id | remote_lsn | local_lsn ----------+-----------------------------------------+------------+----------- ... 4 | bdr: this-is-my-identifier | 0/123456 | 0/18E7BE0 ... Note that when performing a checkpoint, the local_lsn is XLogFlush()ed for proper consistency when using async commit. This is most definitely not close to ready for commit even if it didn't have loads of prerequisite patches, but I think the current state is at a point where feedback would be useful again. Patch applies ontop of lots of other patches (changeset extraction, committs), so it's probably easiest to checkout the http://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git;a=summary branch replication-identifiers . I've also attached this patch solely. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Attachment
pgsql-hackers by date: