Re: [PATCH 10/16] Introduce the concept that wal has a 'origin' node - Mailing list pgsql-hackers

From Heikki Linnakangas
Subject Re: [PATCH 10/16] Introduce the concept that wal has a 'origin' node
Date
Msg-id 4FE21A44.8090701@enterprisedb.com
Whole thread Raw
In response to Re: [PATCH 10/16] Introduce the concept that wal has a 'origin' node  (Simon Riggs <simon@2ndQuadrant.com>)
Responses Re: [PATCH 10/16] Introduce the concept that wal has a 'origin' node
List pgsql-hackers
On 20.06.2012 16:46, Simon Riggs wrote:
> The proposal now includes flag bits that would allow the addition of a
> variable length header, should that ever become necessary. So the
> unused space in the fixed header is not being "used up" as you say. In
> any case, the fixed header still has 4 wasted bytes on 64bit systems
> even after the patch is applied. So this claim of short sightedness is
> just plain wrong.
>> ...>
> We need to add information to every WAL record that is used as the
> source for generating LCRs. It is also possible to add this to HEAP
> and HEAP2 records, but doing that *will* bloat the WAL stream, whereas
> using the *currently wasted* bytes on a WAL record header does *not*
> bloat the WAL stream.

Or, we could provide a mechanism for resource managers to use those 
padding bytes for whatever data the wish to use. Or modify the record 
format so that the last 4 bytes of the data in the WAL record are always 
automatically stored in those padding bytes, thus making all WAL records 
4 bytes shorter. That would make the WAL even more compact, with only a 
couple of extra CPU instructions in the critical path.

My point is that it's wrong to think that it's free to use those bytes, 
just because they're currently unused. If we use them for one thing, we 
can't use them for other things anymore. If we're so concerned about WAL 
bloat that we can't afford to add any more bytes to the WAL record 
header or heap WAL records, then it would be equally fruitful to look at 
ways to use those padding bytes to save that precious WAL space.

I don't think we're *that* concerned about the WAL bloat, however. So 
let's see what is the most sensible place to add whatever extra 
information we need in the WAL, from the point of view of 
maintainability, flexibility, readability etc. Then we can decide where 
to put it.

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com


pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: [PATCH 10/16] Introduce the concept that wal has a 'origin' node
Next
From: Simon Riggs
Date:
Subject: Re: [PATCH 10/16] Introduce the concept that wal has a 'origin' node