Re: Scaling XLog insertion (was Re: Moving more work outside WALInsertLock) - Mailing list pgsql-hackers

From Alvaro Herrera
Subject Re: Scaling XLog insertion (was Re: Moving more work outside WALInsertLock)
Date
Msg-id 1331129422-sup-5323@alvh.no-ip.org
Whole thread Raw
In response to Re: Scaling XLog insertion (was Re: Moving more work outside WALInsertLock)  (Simon Riggs <simon@2ndQuadrant.com>)
Responses Re: Scaling XLog insertion (was Re: Moving more work outside WALInsertLock)  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
Excerpts from Simon Riggs's message of mié mar 07 05:35:44 -0300 2012:
> On Tue, Mar 6, 2012 at 8:32 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:
> >> On 06.03.2012 17:12, Tom Lane wrote:
> >>> How long is the current locked code exactly --- does it contain a loop?
> >
> >> Perhaps best if you take a look for yourself, the function is called
> >> ReserveXLogInsertLocation() in patch. It calls a helper function called
> >>   AdvanceXLogRecPtrToNextPage(ptr), which is small and could be inlined.
> >> It does contain one loop, which iterates once for every WAL page the
> >> record crosses.
> >
> > Hm.  The loop makes me a tad uncomfortable, because it is possible for
> > WAL records to be very long (many pages).  I see the point that
> > replacing the spinlock with an LWLock would likely negate any
> > performance win from this patch, but having other processes arrive and
> > spin while somebody is busy calculating the size of a multi-megabyte
> > commit record would be bad too.
>
> I would have thought the existence of a multi-megabyte commit record
> would already imply a huge performance effect elsewhere and we
> wouldn't care too much about a few spinlock cycles. But I think
> they're as rare as Higgs bosons.

Just to keep things in perspective -- For a commit record to reach one
megabyte, it would have to be a transaction that drops over 43k tables.
Or have 64k smgr inval messages (for example, a TRUNCATE might send half
a dozen of these messages). Or have 262k subtransactions.  Or
combinations thereof.

Now admittedly, a page is only 8 kB, so for a commit record to be "many
pages long" (that is, >=3) it would require about 1500 smgr inval
messages, or, say, about 250 TRUNCATEs (of permanent tables with at
least one toastable field and at least one index).

So they are undoubtely rare.  Not sure if as rare as Higgs bosons.

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


pgsql-hackers by date:

Previous
From: Noah Misch
Date:
Subject: Re: RFC: Making TRUNCATE more "MVCC-safe"
Next
From: Alvaro Herrera
Date:
Subject: Re: elegant and effective way for running jobs inside a database