Re: Redesigning checkpoint_segments - Mailing list pgsql-hackers

From Josh Berkus
Subject Re: Redesigning checkpoint_segments
Date
Msg-id 54D25F4C.7070902@agliodbs.com
Whole thread Raw
In response to Redesigning checkpoint_segments  (Heikki Linnakangas <hlinnakangas@vmware.com>)
Responses Re: Redesigning checkpoint_segments  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
On 02/04/2015 09:28 AM, Robert Haas wrote:
> On Tue, Feb 3, 2015 at 4:18 PM, Josh Berkus <josh@agliodbs.com> wrote:
>>>> That's different from our current checkpoint_segments setting. With
>>>> checkpoint_segments, the documented formula for calculating the disk usage
>>>> is (2 + checkpoint_completion_target) * checkpoint_segments * 16 MB. That's
>>>> a lot less intuitive to set.
>>>
>>> Hmm, that's different from what I was thinking.  We probably shouldn't
>>> call that max_checkpoint_segments, then.  I got confused and thought
>>> you were just trying to decouple the number of segments that it takes
>>> to trigger a checkpoint from the number we keep preallocated.
>>
>> Wait, what?  Because the new setting is an actual soft maximum, we
>> *shouldn't* call it a maximum?  Or are you saying something else?
> 
> I am saying that I proposed calling it max_checkpoint_segments because
> I thought it was the maximum number of segments between checkpoints.
> But it's not.

That's good, though, isn't it? Knowing the number of segments between
checkpoints is useful only to postgres experts with experience.  What
the patch defines is what most users actually want to know: how much
disk space, total, do I need to allocate?

Let me push "max_wal_size" and "min_wal_size" again as our new parameter
names, because:

* does what it says on the tin
* new user friendly
* encourages people to express it in MB, not segments
* very different from the old name, so people will know it works differently

> We are too often far too conservative about these things.  If we make
> the default 192MB, it will only ever get tuned in one direction: up.
> It is not a bad thing for us to set the settings high enough that once
> in a great while someone might find them to be too high rather than
> too low.
> 
> I find it amazing that anyone here thinks that a user would be OK with
> using 192MB of space for WAL, but 384MB would break the bank.  The
> hard drive in my laptop is 456GB.  The point is, with Heikki's work
> here, you're only going to use the maximum amount of space if you have
> massive write activity.  And if you have massive write activity, it's
> extremely likely that you will be OK with using a very modest amount
> of disk space to have that be fast.  Right now, we have to be really
> conservative because we're going to use the full allocation all the
> time, but this fixes that.  I think.

Hmmm, I see your point.  I spend a lot of time on AWS and in
container-world, where disk space is a lot more constrained.  However,
it probably makes more sense to recommend non-default settings for that
environment, since it requires non-default settings anyway.

So, 384MB?

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com



pgsql-hackers by date:

Previous
From: Peter Geoghegan
Date:
Subject: Re: INSERT ... ON CONFLICT {UPDATE | IGNORE} 2.0
Next
From: Josh Berkus
Date:
Subject: Re: Getting rid of wal_level=archive and default to hot_standby + wal_senders