Re: Direct I/O - Mailing list pgsql-hackers

From Andres Freund
Subject Re: Direct I/O
Date
Msg-id 20221115025015.yf2uk2bj2bwmt7m6@awork3.anarazel.de
Whole thread Raw
In response to Re: Direct I/O  (John Naylor <john.naylor@enterprisedb.com>)
List pgsql-hackers
Hi,

On 2022-11-10 14:26:20 +0700, John Naylor wrote:
> On Tue, Nov 1, 2022 at 2:37 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> 
> > Memory alignment patches:
> >
> > Direct I/O generally needs to be done to/from VM page-aligned
> > addresses, but only "standard" 4KB pages, even when larger VM pages
> > are in use (if there is an exotic system where that isn't true, it
> > won't work).  We need to deal with buffers on the stack, the heap and
> > in shmem.  For the stack, see patch 0001.  For the heap and shared
> > memory, see patch 0002, but David Rowley is going to propose that part
> > separately, as MemoryContext API adjustments are a specialised enough
> > topic to deserve another thread; here I include a copy as a
> > dependency.  The main direct I/O patch is 0003.
> 
> One thing to note: Currently, a request to aset above 8kB must go into a
> dedicated block. Not sure if it's a coincidence that that matches the
> default PG page size, but if allocating pages on the heap is hot enough,
> maybe we should consider raising that limit. Although then, aligned-to-4kB
> requests would result in 16kB chunks requested unless a different allocator
> was used.

With one exception, there's only a small number of places that allocate pages
dynamically and we only do it for a small number of buffers. So I don't think
we should worry too much about this for now.

The one exception to this: GetLocalBufferStorage(). But it already batches
memory allocations by increasing sizes, so I think we're good as well.

Greetings,

Andres Freund



pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: Assertion failure in SnapBuildInitialSnapshot()
Next
From: Thomas Munro
Date:
Subject: Re: Assertion failure in SnapBuildInitialSnapshot()