On 02/24/2015 04:55 AM, Robert Haas wrote:
> On Mon, Feb 23, 2015 at 12:52 PM, Heikki Linnakangas
> <hlinnakangas@vmware.com> wrote:
>> Dunno, but Jim's got a point. This is a maintenance burden to all indexams,
>> if they all have to remember to update the LSN map separately. It needs to
>> be done in some common code, like in PageSetLSN or XLogInsert or something.
>>
>> Aside from that, isn't this horrible from a performance point of view? The
>> patch doubles the buffer manager traffic, because any update to any page
>> will also need to modify the LSN map. This code is copied from the
>> visibility map code, but we got away with it there because the VM only needs
>> to be updated the first time a page is modified. Subsequent updates will
>> know the visibility bit is already cleared, and don't need to access the
>> visibility map.
>>
>> Ans scalability: Whether you store one value for every N pages, or the LSN
>> of every page, this is going to have a huge effect of focusing contention to
>> the LSN pages. Currently, if ten backends operate on ten different heap
>> pages, for example, they can run in parallel. There will be some contention
>> on the WAL insertions (much less in 9.4 than before). But with this patch,
>> they will all fight for the exclusive lock on the single LSN map page.
>>
>> You'll need to find a way to not update the LSN map on every update. For
>> example, only update the LSN page on the first update after a checkpoint
>> (although that would still have a big contention focusing effect right after
>> a checkpoint).
>
> I think it would make more sense to do this in the background.
> Suppose there's a background process that reads the WAL and figures
> out which buffers it touched, and then updates the LSN map
> accordingly. Then the contention-focusing effect disappears, because
> all of the updates to the LSN map are being made by the same process.
> You need some way to make sure the WAL sticks around until you've
> scanned it for changed blocks - but that is mighty close to what a
> physical replication slot does, so it should be manageable.
If you implement this as a background process that reads WAL, as Robert
suggested, you could perhaps implement this completely in an extension.
That'd be nice, even if we later want to integrate this in the backend,
in order to get you started quickly.
This is marked in the commitfest as "Needs Review", but ISTM this got
its fair share of review back in February. Marking as Returned with
Feedback.
- Heikki
