Re: XLogInsert scaling, revisited - Mailing list pgsql-hackers

From Andres Freund
Subject Re: XLogInsert scaling, revisited
Date
Msg-id 20130708104327.GA7242@awork2.anarazel.de
Whole thread Raw
In response to Re: XLogInsert scaling, revisited  (Heikki Linnakangas <hlinnakangas@vmware.com>)
List pgsql-hackers
On 2013-07-08 10:45:41 +0300, Heikki Linnakangas wrote:
> On 01.07.2013 16:40, Andres Freund wrote:
> >On 2013-06-26 18:52:30 +0300, Heikki Linnakangas wrote:
> >>>* Could you document the way slots prevent checkpoints from occurring
> >>>   when XLogInsert rechecks for full page writes? I think it's correct -
> >>>   but not very obvious on a glance.
> >>
> >>There's this in the comment near the top of the file:
> >>
> >>  * To update RedoRecPtr or fullPageWrites, one has to make sure that all
> >>  * subsequent inserters see the new value. This is done by reserving all the
> >>  * insertion slots before changing the value. XLogInsert reads RedoRecPtr
> >>and
> >>  * fullPageWrites after grabbing a slot, so by holding all the slots
> >>  * simultaneously, you can ensure that all subsequent inserts see the new
> >>  * value.  Those fields change very seldom, so we prefer to be fast and
> >>  * non-contended when they need to be read, and slow when they're changed.
> >>
> >>Does that explain it well enough? XLogInsert holds onto a slot while it
> >>rechecks for full page writes.
> >
> >I am a bit worried about that logic. We're basically reverting to the
> >old logic whe xlog writing is an exlusive affair. We will have to wait
> >for all the other queued inserters before we're finished. I am afraid
> >that that will show up latencywise.
> 
> A single stall of the xlog-insertion "pipeline" at a checkpoint is hardly
> going to be a problem. I wish PostgreSQL was real-time enough for that to
> matter, but I think we're very very far from that.

Well, the stall won't necessarily be that short. There might be several
backends piling on every insertion slot and waiting - and thus put to
sleep by the kerenl. I am pretty sure it's easy enough to get stalls in
the second range that way.

Sure, there are lots of reasons we don't have all that reliable response
times, but IME the amount of response time jitter is one of the bigger
pain points of postgres. And this feature has a good chance of reducing
that pain noticeably...

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services



pgsql-hackers by date:

Previous
From: Heikki Linnakangas
Date:
Subject: Re: XLogInsert scaling, revisited
Next
From: Hari Babu
Date:
Subject: Re: Review: Patch to compute Max LSN of Data Pages