Re: Cmpact commits and changeset extraction - Mailing list pgsql-hackers

From Andres Freund
Subject Re: Cmpact commits and changeset extraction
Date
Msg-id 20130930213419.GD30125@awork2.anarazel.de
Whole thread Raw
In response to Re: Cmpact commits and changeset extraction  (Robert Haas <robertmhaas@gmail.com>)
Responses Re: Cmpact commits and changeset extraction
List pgsql-hackers
On 2013-09-30 14:22:22 -0400, Robert Haas wrote:
> On Mon, Sep 30, 2013 at 10:50 AM, Andres Freund <andres@2ndquadrant.com> wrote:
> > Changeset extraction only works in the context of a single database but
> > has to scan through xlog records from multiple databases. Most records
> > are easy to skip because they contain the database in the relfilenode or
> > are just not interesting for logical replication. The only exception are
> > compact commits.
> > So we have some alternatives:
> > 1) don't do anything, in that case empty transactions will get replayed since the changes
> >   themselves will get skipped.
> > 2) Don't use compact commits if wal_level=logical
> > 3) unify compact and non-compact commits, trying to get the normal one
> >    smaller.
> >
> > For 3) I am thinking of using 'xinfo' to store whether we have the other
> > information or not. E.g. if there are subxacts in a compact commit we
> > signal that by the flag 'XACT_COMMIT_CONTAINS_SUBXACTS' and store the
> > number of subxacts after the xlog record. Similarly with relations,
> > invalidation messages and the database id. That should leave compact
> > commits without any subxacts at the former size, and those with at the
> > former size + 4. Normal commits would get smaller in many cases since we
> > don't store the empty fields.
> >
> > I personally think 3) is the best solution, any other opinions?
> 
> What's wrong with #1?

It seems confusing that a changeset stream in database #1 will contain
commits (without corresponding changes) from database #2. Seems like aaa
pola violation to me.

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services



pgsql-hackers by date:

Previous
From: Huchev
Date:
Subject: Re: pluggable compression support
Next
From: Kevin Grittner
Date:
Subject: Re: logical changeset generation v6.1