Re: LWLock Queue Jumping - Mailing list pgsql-hackers

From Jeff Janes
Subject Re: LWLock Queue Jumping
Date
Msg-id f67928030908301044w77311615h9462869814509bfd@mail.gmail.com
Whole thread Raw
In response to LWLock Queue Jumping  (Jeff Janes <jeff.janes@gmail.com>)
Responses Re: LWLock Queue Jumping
List pgsql-hackers
---------- Forwarded message ----------
From: Stefan Kaltenbrunner <stefan@kaltenbrunner.cc>
To: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
Date: Sun, 30 Aug 2009 11:48:47 +0200
Subject: Re: LWLock Queue Jumping
Heikki Linnakangas wrote:
Greg Stark wrote:
On Fri, Aug 28, 2009 at 8:07 PM, Simon Riggs<simon@2ndquadrant.com> wrote:
WALInsertLock is heavily contended and likely always will be even if we
apply some of the planned fixes.
I've lost any earlier messages, could you resend the raw data on which
this is based?

I don't have any pointers right now, but WALInsertLock does often show
up as a bottleneck in write-intensive benchmarks.

yeah I recently ran accross that issue with testing concurrent COPY performance:

http://www.kaltenbrunner.cc/blog/index.php?/archives/27-Benchmarking-8.4-Chapter-2bulk-loading.html
discussed here:

http://archives.postgresql.org/pgsql-hackers/2009-06/msg01019.php


It looks like this is the bulk loading of data into unindexed tables.  How good is that as a target for optimization?  I can see several (quite difficult to code and maintain) ways to make bulk loading into unindexed tables faster, but they would not speed up the more general cases. 
 
and (iirc) also here:

http://archives.postgresql.org/pgsql-hackers/2009-06/msg01133.php


I played around a little with this, parallel bulk loads into a unindexed, very skinny table.  If I hacked XLogInsert so that it did nothing but take the WALInsertLock, release it, then return a fake RecPtr, it scaled better but still not very well.  So giant leaps in throughput would need to involve calling XLogInsert less often (or at least taking the WALInsertLock less often).  You could nibble around the edges by tweaking what happens under the WALInsertLock, but I don't think that that will get you big wins by doing that for this case.  But again, how important is this case?  Are bulk loads into skinny unindexed tables the best test-bed for improving XLogInsert?

(Sorry, I think I forgot to change the subject on previous message.  Digests are great if you only read, but for contributing I guess I have to change to receiving each message)

Jeff

pgsql-hackers by date:

Previous
From: Grzegorz Jaskiewicz
Date:
Subject: Re: clang's static checker report.
Next
From: Grzegorz Jaskiewicz
Date:
Subject: Re: clang's static checker report.