Re: Cmpact commits and changeset extraction - Mailing list pgsql-hackers

From Andres Freund
Subject Re: Cmpact commits and changeset extraction
Date
Msg-id 20131011200535.GE4056218@alap2.anarazel.de
Whole thread Raw
In response to Re: Cmpact commits and changeset extraction  (Robert Haas <robertmhaas@gmail.com>)
Responses Re: Cmpact commits and changeset extraction  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
On 2013-10-01 10:12:13 -0400, Robert Haas wrote:
> On Tue, Oct 1, 2013 at 7:26 AM, Andres Freund <andres@2ndquadrant.com> wrote:
> > On 2013-10-01 06:20:20 -0400, Robert Haas wrote:
> >> On Mon, Sep 30, 2013 at 5:34 PM, Andres Freund <andres@2ndquadrant.com> wrote:
> >> >> What's wrong with #1?
> >> >
> >> > It seems confusing that a changeset stream in database #1 will contain
> >> > commits (without corresponding changes) from database #2. Seems like aaa
> >> > pola violation to me.
> >>
> >> I don't really see the problem.  A transaction could be empty for lots
> >> of reasons; it may have obtained an XID without writing any data, or
> >> whatever it's changed may be outside the bounds of logical rep.
> >
> > Sure. But all of them will have had a corresponding action in the
> > database. If your replication stream suddenly sees commits that you
> > cannot connect to any application activity... And it would depend on the
> > kind of commit, you won't see it if a non-compact commit was used.
> > It also means we need to do work to handle that commit. If you have a
> > busy and a less so database and you're only replicating the non-busy
> > one, that might be noticeable.
> 
> Maybe.  The original reason we added compact commits was because we
> thought that making unlogged tables logged and visca/versa was going
> to require adding still more stuff to the commit record.  I'm no
> longer sure that's the case and, in any case, nobody's worked out the
> design details.  But I can't help thinking more stuff's likely to come
> up in the future.  I'm certainly willing to entertain proposals for
> restructuring that, but I don't really want to just throw it out.

Well, what I am thinking of - including & reading data depending on a
flag in ->xinfo - would give you extensibility without requiring
different types of commits. And it would only blow up the size by
whatever needs to be included.

> >> Maybe you should just skip replay of transactions with no useful
> >> content.
> >
> > Yes, I have thought about that as well. But I dislike it - how do we
> > define "no useful content"?
> 
> The only action we detected for that XID was the commit itself.

What if the transaction was intentionally done to get an xid + timestamp
in a multimaster system? What if it includes DDL but no logged data? Do
we replay a transaction because of the pg_shdepend entry when creating a
table in another database?

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services



pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: [COMMITTERS] pgsql: Replace duplicate_oids with Perl implementation
Next
From: Andres Freund
Date:
Subject: Re: drop-index-concurrently-1 on master fails at serializable