Re: Checkpoint distribution - Mailing list pgsql-performance

From Borodin Vladimir
Subject Re: Checkpoint distribution
Date
Msg-id D7EAF9CA-83DB-4CE8-9742-192BB1B8C90C@simply.name
Whole thread Raw
In response to Re: Checkpoint distribution  (Jeff Janes <jeff.janes@gmail.com>)
List pgsql-performance

14 апр. 2014 г., в 22:09, Jeff Janes <jeff.janes@gmail.com> написал(а):

On Mon, Apr 14, 2014 at 9:42 AM, Borodin Vladimir <root@simply.name> wrote:
14 апр. 2014 г., в 19:11, Jeff Janes <jeff.janes@gmail.com> написал(а):


During the writing phase of the checkpoint, PostgreSQL passes the dirty data to the OS.  At the end, it then tells the OS to make sure that that data has actually reached disk.  If your OS stored up too much dirty data in memory then it kind of freaks out once it is notified it needs to actually write that data to disk.  The best solution for this may be to lower dirty_background_bytes or dirty_background_ratio so the OS doesn't store up so much trouble for itself.


Actually, I have already tuned them to different values. Test results above have been obtained with such settings for page cache:

vm.dirty_background_ratio = 5

If you have 64GB of RAM, that is 3.2GB of allowed dirty data, which is probably too much. But I think I've heard rumors that the kernel ignores settings below 5, so probably switch to dirty_background_bytes.


Actually, I have even more :) 128 GB of RAM. I’ve set such settings for page cache:

# 100 MB
vm.dirty_background_bytes = 104857600
vm.dirty_ratio = 40
vm.dirty_expire_centisecs = 100
vm.dirty_writeback_centisecs = 100

And tried 2 GB, 4 GB, 8 GB for shared_buffers size (when I wrote first letter, it was 2 GB). Shared buffers size does not matter with above page cache settings. But it really affects the distribution of checkpoint over time. 
Right now test results (for 1000 tps and checkpoint every 5 minutes) look like that:

Thank you very much, Jeff.

Cheers,

Jeff


--
Vladimir




Attachment

pgsql-performance by date:

Previous
From: Ryan Johnson
Date:
Subject: Re: SSI slows down over time
Next
From: desmodemone
Date:
Subject: Re: HFS+ pg_test_fsync performance