Re: a question about Direct I/O and double buffering - Mailing list pgsql-performance

From Erik Jones
Subject Re: a question about Direct I/O and double buffering
Date
Msg-id 0F61A03E-DF77-46C2-B7E1-2E7A5F985FB8@myemma.com
Whole thread Raw
In response to Re: a question about Direct I/O and double buffering  (Xiaoning Ding <dingxn@cse.ohio-state.edu>)
Responses Re: a question about Direct I/O and double buffering
List pgsql-performance
On Apr 5, 2007, at 1:22 PM, Xiaoning Ding wrote:

Erik Jones wrote:
On Apr 5, 2007, at 12:09 PM, Xiaoning Ding wrote:
Hi,

A page may be double buffered in PG's buffer pool and in OS's buffer cache.
Other DBMS like DB2 and Oracle has provided Direct I/O option to eliminate
double buffering. I noticed there were discusses on the list. But
I can not find similar option in PG. Does PG support direct I/O now?

The tuning guide of PG usually recommends a small shared buffer pool
(compared
to the size of physical memory).  I think it is to avoid swapping. If
there were
swapping, OS kernel may swap out some pages in PG's buffer pool even PG
want to keep them in memory. i.e. PG would loose full control over
buffer pool.
A large buffer pool is not good because it may
1. cause more pages double buffered, and thus decrease the efficiency of
buffer
cache and buffer pool.
2. may cause swapping.
Am I right?

If PG's buffer pool is small compared with physical memory, can I say
that the
hit ratio of PG's buffer pool is not so meaningful because most misses
can be
satisfied by OS Kernel's buffer cache?

Thanks!
To the best of my knowledge, Postgres itself does not have a direct IO option (although it would be a good addition).  So, in order to use direct IO with postgres you'll need to consult your filesystem docs for how to set the forcedirectio mount option.  I believe it can be set dynamically, but if you want it to be permanent you'll to add it to your fstab/vfstab file.

I use Linux.  It supports direct I/O on a per-file basis only.  To bypass OS buffer cache,
files should be opened with O_DIRECT option.  I afraid that I have to modify PG.

Xiaoning

Looks like it.  I just did a cursory search of the archives and it seems that others have looked at this before so you'll probably want to start there if your up to it.

erik jones <erik@myemma.com>
software developer
615-296-0838
emma(r)



pgsql-performance by date:

Previous
From: "Jonathan Ellis"
Date:
Subject: Premature view materialization in 8.2?
Next
From: "Alex Deucher"
Date:
Subject: Re: a question about Direct I/O and double buffering