Re: WAL Bypass for indexes - Mailing list pgsql-hackers

From Martin Scholes
Subject Re: WAL Bypass for indexes
Date
Msg-id 3ckNY4hVsDfg.cwBbLpCH@mail.iicolo.com
Whole thread Raw
In response to WAL Bypass for indexes  ("Martin Scholes" <marty@iicolo.com>)
List pgsql-hackers
<div align="LEFT">Simon,</div><div align="LEFT"> </div><div align="LEFT">>The WAL becomes more of a hotspot as you
scaleup numbers of CPUs.</div><div align="LEFT"> </div><div align="LEFT">I tend to agree and the original idea came
whenI was working on a Sun quad-CPU system for a highly parallelized web application. Each page was broken into several
dynamicimages, each created "on the fly" from dozens of vector objects and all of the data, as well as state
informationwas held in Pg.</div><div align="LEFT"> </div><div align="LEFT">As a complex app, each page load would spawn
adozen or so processes, each hitting the DB pretty hard, where all of the business logic resided.</div><div
align="LEFT"> </div><divalign="LEFT">It didn't take too many concurrent users to bring the server to its
knees.</div><divalign="LEFT"> </div><div align="LEFT">Here's the point: some inspection showed that a lot of time was
beingspent on index output. At that point I realized that there had to be a better way.</div><div
align="LEFT"> </div><divalign="LEFT">My simple home system is not capable of recreating those conditions. If you have
anSMP box, please run some tests.</div><div align="LEFT"> </div><div align="LEFT">M</div><div align="LEFT"> </div><div
align="LEFT">_____Original message _____</div><div align="LEFT">Subject: Re: [HACKERS] WAL Bypass for indexes</div><div
align="LEFT">Author:Simon Riggs <simon@2ndquadrant.com></div><div align="LEFT">Date: 05th April 2006 11:0:34
AM</div><divalign="LEFT"> </div><div align="LEFT">On Wed, 2006-04-05 at 09:40 -0700, Martin Scholes wrote:<br
/></div><divalign="LEFT">> > I will run multiple tests and post the actual numbers.<br />> <br />> I did
runmore extensive tests and did not bother writing down the<br />> numbers, and here's why: the unmodified Pg ran
pgbenchwith a whopping<br />> average of 6.3% time in IO wait.<br />> <br />> If I was able to totally
eliminatethat time (which is impossible),<br />> then the best we could hope for is a 7% increase in performance
by<br/>> skipping WAL of indexes.</div><div align="LEFT"> </div><div align="LEFT">The WAL becomes more of a hotspot
asyou scale up numbers of CPUs. I<br />guess this idea doesn't make much difference for smaller systems.<br
/></div><divalign="LEFT">I think your idea still has merit, Martin. I'll do some tests also.<br /></div><div
align="LEFT">BestRegards, Simon Riggs<br /></div><div align="LEFT"> </div><div
align="LEFT">---------------------------(endof broadcast)---------------------------<br />TIP 2: Don't 'kill -9' the
postmaster<br/></div> 

pgsql-hackers by date:

Previous
From: Josh Berkus
Date:
Subject: Anyone want to finish BEFORE COMMIT triggers?
Next
From: Philip Yarra
Date:
Subject: Re: psql \c error