Re: Design proposal: fsync absorb linear slider - Mailing list pgsql-hackers

From didier
Subject Re: Design proposal: fsync absorb linear slider
Date
Msg-id CAJRYxuJoBrqZ4DCx2j7NCn860zva5o5bGvFqP0aL07VN_=jCcw@mail.gmail.com
Whole thread Raw
In response to Re: Design proposal: fsync absorb linear slider  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
Hi,


Sure, that's what the WAL does.  But you still have to checkpoint eventually.

Sure, when you run  pg_ctl stop.
Unlike the WAL it only needs two files, shared_buffers size.
 
I did bogus tests by replacing  mask |= BM_PERMANENT; with mask = -1 in BufferSync() and simulating checkpoint with a periodic dd if=/dev/zero of=foo  conv=fsync

On a saturated storage with %usage locked solid at 100% I got up to 30% speed improvement and fsync latency down by one order of magnitude, some fsync were still slow of course if buffers were already in OS cache.

But it's the upper bound, it's was done one a slow storage with bad ratios : (OS cache write)/(disk sequential write) in 50, (sequential write)/(effective random write) in 10 range and a proper implementation would have a 'little' more work to do... (only checkpoint task can write BM_CHECKPOINT_NEEDED buffers keeping them dirty and so on)

Didier

pgsql-hackers by date:

Previous
From: Craig Ringer
Date:
Subject: Repeated array type info caching code
Next
From: Andres Freund
Date:
Subject: Re: Condition to become the standby mode.