Re: Massive I/O spikes during checkpoint - Mailing list pgsql-performance

From Jeff Janes
Subject Re: Massive I/O spikes during checkpoint
Date
Msg-id CAMkU=1zRS0fObMNyuH50UxA170GfqwvfRvc3s6gq-wgAhMtjog@mail.gmail.com
Whole thread Raw
In response to Massive I/O spikes during checkpoint  (David Kerr <dmk@mr-paradox.net>)
Responses Re: Massive I/O spikes during checkpoint  (David Kerr <dmk@mr-paradox.net>)
List pgsql-performance
On Mon, Jul 9, 2012 at 10:39 PM, David Kerr <dmk@mr-paradox.net> wrote:
>
> I thought that the idea of checkpoint_completion_target was that we try to
> finish writing
> out the data throughout the entire checkpoint (leaving some room to spare,
> in my case 30%
> of the total estimated checkpoint time)
>
> But what appears to be happening is that all of the data is being written
> out at the end of the checkpoint.

Postgres is writing data out to the kernel throughout the checkpoint.
But the kernel is just buffering it up dirty, until the end of the
checkpoint when the fsyncs start landing like bombs.

>
> This happens at every checkpoint while the system is under load.
>
> I get the feeling that this isn't the correct behavior and i've done
> something wrong.
>
> Also, I didn't see this sort of behavior in PG 8.3, however unfortunately, I
> don't have data to back that
> statement up.

Did you have less RAM back when you were running PG 8.3?

> Any suggestions. I'm willing and able to profile, or whatever.

Who much RAM do you have?  What are your settings for /proc/sys/vm/dirty_* ?

Cheers,

Jeff

pgsql-performance by date:

Previous
From: Maxim Boguk
Date:
Subject: Re: Massive I/O spikes during checkpoint
Next
From: David Kerr
Date:
Subject: Re: Massive I/O spikes during checkpoint