Re: pglogical_output - a general purpose logical decoding output plugin - Mailing list pgsql-hackers

From Craig Ringer
Subject Re: pglogical_output - a general purpose logical decoding output plugin
Date
Msg-id CAMsr+YHvYQL_uY=O6G6GaVEJyCs_-paTq4FWQ1wo3NS8FYrnAA@mail.gmail.com
Whole thread Raw
In response to Re: pglogical_output - a general purpose logical decoding output plugin  (Greg Stark <stark@mit.edu>)
Responses Re: pglogical_output - a general purpose logical decoding output plugin  (Craig Ringer <craig@2ndquadrant.com>)
List pgsql-hackers
On 7 January 2016 at 02:32, Greg Stark <stark@mit.edu> wrote:
 
However you also need to keep in mind that any of these other purposes
will be more or less equally large projects as logical replication.
There's no particular reason to expect one to be able to start up
today and provide feedback faster than the replication code that's
already been under development for ages.

Good point. Though it's been out there for a while now.
 
The one project that does seem like it should be fairly fast to get
going and provide a relatively easy way to test the APIs separately
would be an auditing tool.

Yep. The json stream is very simple to consume for that purpose too, providing pre-jsonified data from the upstream so you don't have to deal with all the mess of maintaining matching type and table definitions on the downstream to construct HeapTuple s, etc.

You generally don't want to write audit logs into a relation of the same form as the original table plus some extra columns. It's nicer to work with, sure, but it means you're almost totally locked in to your original table definition. You can add new nullable columns but that's about it, since there's no way to retroactively construct audit data for already logged entries. So I think the json output mode is more interesting for auditing.

An interesting hiccup for audit is that AFAIK nothing in WAL associates user identity information with a change, so we can't decode and send info on which user performed which xact or changed a given tuple. SET ROLE and SECURITY DEFINER mean there isn't a 1:1 mapping of xact to user either, and info would probably have to be tuple level. Fun. The WAL messages patch would be helpful for this sort of thing, allowing apps to use triggers to add arbitrary info to xlog; XLOG_NOOP could help too, just in an uglier way with more room for confusion between unrelated products using it.

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

pgsql-hackers by date:

Previous
From: Craig Ringer
Date:
Subject: Re: pglogical_output - a general purpose logical decoding output plugin
Next
From: Pavel Stehule
Date:
Subject: Re: Add numeric_trim(numeric)