Re: Load Distributed Checkpoints, final patch - Mailing list pgsql-patches

From Bruce Momjian
Subject Re: Load Distributed Checkpoints, final patch
Date
Msg-id 200803112106.m2BL6Rp09789@momjian.us
Whole thread Raw
In response to Re: Load Distributed Checkpoints, final patch  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-patches
Added to TODO:

* Test to see if calling PreallocXlogFiles() from the background writer
  will help with WAL segment creation latency

  http://archives.postgresql.org/pgsql-patches/2007-06/msg00340.php


---------------------------------------------------------------------------

Tom Lane wrote:
> Heikki Linnakangas <heikki@enterprisedb.com> writes:
> > Here's latest revision of Itagaki-sans Load Distributed Checkpoints patch:
>
> Applied with some minor revisions to make some of the internal APIs a
> bit cleaner; mostly, it seemed like a good idea to replace all those
> bool parameters with a flag-bits approach, so that you could have
> something like "CHECKPOINT_FORCE | CHECKPOINT_WAIT" instead of
> "false, true, true, false" ...
>
> For the moment I removed all the debugging elog's in the patch.
> We still have Greg Smith's checkpoint logging patch to look at
> (which I suppose needs adjustment now), and that seems like the
> appropriate venue to consider what to put in.
>
> Also, the question of redesigning the bgwriter's LRU scan is
> still open.  I believe that's on Greg's plate, too.
>
> One other closely connected item that might be worth looking at is the
> code for creating new future xlog segments (PreallocXlogFiles).  Greg
> was griping upthread about xlog segment creation being a real
> performance drag.  I realized that as we currently have it set up, the
> checkpoint code is next to useless for high-WAL-volume installations,
> because it only considers making *one* future XLOG segment.  Once you've
> built up enough XLOG segments, the system isn't too bad about recycling
> them, but there will be a nasty startup transient where foreground
> processes have to stop and make the things.  I wonder whether it would
> help if we (a) have the bgwriter call PreallocXlogFiles during its
> normal loop, and (b) back the slop in PreallocXlogFiles way off, so that
> it will make a future segment as soon as we start using the last
> existing segment, instead of only when we're nearly done.  This would at
> least make it more likely that the bgwriter does the work instead of a
> foreground process.  I'm hesitant to go much further than that, because
> I don't want to bloat the minimum disk footprint for low-volume
> installations, but the minimum footprint is really 2 xlog files anyway...
>
>             regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
>                http://archives.postgresql.org

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

pgsql-patches by date:

Previous
From: Tom Lane
Date:
Subject: Re: [PERFORM] Very slow (2 tuples/second) sequential scan after bulk insert; speed returns to ~500 tuples/second after commit
Next
From: Bruce Momjian
Date:
Subject: Re: SPI-header-files safe for C++-compiler