Re: [PATCHES] ARC Memory Usage analysis - Mailing list pgsql-hackers

From Greg Stark
Subject Re: [PATCHES] ARC Memory Usage analysis
Date
Msg-id 87r7nlts34.fsf@stark.xeocode.com
Whole thread Raw
In response to Re: [PATCHES] ARC Memory Usage analysis  (Curt Sampson <cjs@cynic.net>)
Responses Re: [PATCHES] ARC Memory Usage analysis
List pgsql-hackers
Curt Sampson <cjs@cynic.net> writes:

> On Tue, 26 Oct 2004, Greg Stark wrote:
> 
> > I see mmap or O_DIRECT being the only viable long-term stable states. My
> > natural inclination was the former but after the latest thread on the subject
> > I suspect it'll be forever out of reach. That makes O_DIRECT And a Postgres
> > managed cache the only real choice. Having both caches is just a waste of
> > memory and a waste of cpu cycles.
> 
> I don't see why mmap is any more out of reach than O_DIRECT; it's not
> all that much harder to implement, and mmap (and madvise!) is more
> widely available.

Because there's no way to prevent a write-out from occurring and no way to be
notified by mmap before a write-out occurs, and Postgres wants to do its WAL
logging at that time if it hasn't already happened.

> But if using two caches is only costing us 1% in performance, there's
> not really much point....

Well firstly it depends on the work profile. It can probably get much higher
than we saw in that profile if your work load is causing more fresh buffers to
be fetched.

Secondly it also reduces the amount of cache available. If you have 256M of
ram with about 200M free, and 40Mb of ram set aside for Postgres's buffer
cache then you really only get 160Mb. It's costing you 20% of your cache, and
reducing the cache hit rate accordingly.

Thirdly the kernel doesn't know as much as Postgres about the load. Postgres
could optimize its use of cache based on whether it knows the data is being
loaded by a vacuum or sequential scan rather than an index lookup. In practice
Postgres has gone with ARC which I suppose a kernel could implement anyways,
but afaik neither linux nor BSD choose to do anything like it.

-- 
greg



pgsql-hackers by date:

Previous
From: Kris Jurka
Date:
Subject: Re: making pdf of docs
Next
From: Mike Mascari
Date:
Subject: Re: making pdf of docs