Re: a question about Direct I/O and double buffering

From: Xiaoning Ding
Subject: Re: a question about Direct I/O and double buffering
Date: ,
Msg-id: 461562F5.8070600@cse.ohio-state.edu
(view: Whole thread, Raw)
In response to: Re: a question about Direct I/O and double buffering  ("Alex Deucher")
Responses: Re: a question about Direct I/O and double buffering  ("Alex Deucher")
Re: a question about Direct I/O and double buffering  ()
List: pgsql-performance

Tree view

a question about Direct I/O and double buffering  (Xiaoning Ding <-state.edu>, )
 Re: a question about Direct I/O and double buffering  (Erik Jones, )
  Re: a question about Direct I/O and double buffering  (Xiaoning Ding <-state.edu>, )
   Re: a question about Direct I/O and double buffering  (Erik Jones, )
    Re: a question about Direct I/O and double buffering  ("Alex Deucher", )
     Re: a question about Direct I/O and double buffering  (Xiaoning Ding <-state.edu>, )
      Re: a question about Direct I/O and double buffering  ("Alex Deucher", )
      Re: a question about Direct I/O and double buffering  (, )
   Re: a question about Direct I/O and double buffering  (, )
    Re: a question about Direct I/O and double buffering  (Erik Jones, )
     Re: a question about Direct I/O and double buffering  (, )
  Re: a question about Direct I/O and double buffering  (Mark Lewis, )
   Re: a question about Direct I/O and double buffering  (Erik Jones, )
    Re: a question about Direct I/O and double buffering  (Mark Lewis, )
     Re: a question about Direct I/O and double buffering  (Erik Jones, )
      Re: a question about Direct I/O and double buffering  ("Jim C. Nasby", )

Alex Deucher wrote:
> On 4/5/07, Erik Jones <> wrote:
>>
>> On Apr 5, 2007, at 1:22 PM, Xiaoning Ding wrote:
>>
>> Erik Jones wrote:
>> On Apr 5, 2007, at 12:09 PM, Xiaoning Ding wrote:
>> Hi,
>>
>> A page may be double buffered in PG's buffer pool and in OS's buffer
>> cache.
>> Other DBMS like DB2 and Oracle has provided Direct I/O option to
>> eliminate
>> double buffering. I noticed there were discusses on the list. But
>> I can not find similar option in PG. Does PG support direct I/O now?
>>
>> The tuning guide of PG usually recommends a small shared buffer pool
>> (compared
>> to the size of physical memory).  I think it is to avoid swapping. If
>> there were
>> swapping, OS kernel may swap out some pages in PG's buffer pool even PG
>> want to keep them in memory. i.e. PG would loose full control over
>> buffer pool.
>> A large buffer pool is not good because it may
>> 1. cause more pages double buffered, and thus decrease the efficiency of
>> buffer
>> cache and buffer pool.
>> 2. may cause swapping.
>> Am I right?
>>
>> If PG's buffer pool is small compared with physical memory, can I say
>> that the
>> hit ratio of PG's buffer pool is not so meaningful because most misses
>> can be
>> satisfied by OS Kernel's buffer cache?
>>
>> Thanks!
>> To the best of my knowledge, Postgres itself does not have a direct IO
>> option (although it would be a good addition).  So, in order to use
>> direct
>> IO with postgres you'll need to consult your filesystem docs for how
>> to set
>> the forcedirectio mount option.  I believe it can be set dynamically,
>> but if
>> you want it to be permanent you'll to add it to your fstab/vfstab file.
>>
>> I use Linux.  It supports direct I/O on a per-file basis only.  To
>> bypass OS
>> buffer cache,
>> files should be opened with O_DIRECT option.  I afraid that I have to
>> modify
>> PG.
>>
>> Xiaoning
>> Looks like it.  I just did a cursory search of the archives and it seems
>> that others have looked at this before so you'll probably want to start
>> there if your up to it.
>>
>
> Linux used to have (still does?) a RAW interface which might also be
> useful.  I think the original code was contributed by oracle so they
> could support direct IO.
>
> Alex
I am more concerned with reads , and how to do direct I/O under Linux here.
Reading raw devices in linux bypasses OS buffer cache. But how can you
mount a raw device( it is a character device) as a file system?

  Xiaoning


pgsql-performance by date:

From: david@lang.hm
Date:
Subject: Re: SCSI vs SATA
From: Ron
Date:
Subject: Re: SCSI vs SATA