Re: Extended Prefetching using Asynchronous IO - proposal and patch - Mailing list pgsql-hackers

From Heikki Linnakangas
Subject Re: Extended Prefetching using Asynchronous IO - proposal and patch
Date
Msg-id 538652E5.70303@vmware.com
Whole thread Raw
In response to Re: Extended Prefetching using Asynchronous IO - proposal and patch  (John Lumby <johnlumby@hotmail.com>)
Responses Re: Extended Prefetching using Asynchronous IO - proposal and patch
Re: Extended Prefetching using Asynchronous IO - proposal and patch
List pgsql-hackers
On 05/28/2014 11:52 PM, John Lumby wrote:
>
> The patch is attached.
> It is based on clone of today's 9.4dev source.
> I have noticed that this source is
> (not suprisingly) quite a moving target at present,
> meaning that this patch becomes stale quite quickly.
> So although this copy is fine for reviewing,
> it may quite probably soon not be correct
> for the current source tree.
>
> As mentioned before,  if anyone wishes to try this feature out
> on 9.3.4,   I will be making a patch for that soon
> which I can supply on request.

Wow, that's a huge patch. I took a very brief look, focusing on the 
basic design. ignoring the style & other minor things for now:

The patch seems to assume that you can put the aiocb struct in shared 
memory, initiate an asynchronous I/O request from one process, and wait 
for its completion from another process. I'm pretty surprised if that 
works on any platform.

How portable is POSIX aio nowadays? Googling around, it still seems that 
on Linux, it's implemented using threads. Does the thread-emulation 
implementation cause problems with the rest of the backend, which 
assumes that there is only a single thread? In any case, I think we'll 
want to encapsulate the AIO implementation behind some kind of an API, 
to allow other implementations to co-exist.

Benchmarks?
- Heikki



pgsql-hackers by date:

Previous
From: Bruce Momjian
Date:
Subject: Re: Re: popen and pclose redefinitions causing many warning in Windows build
Next
From: Peter Geoghegan
Date:
Subject: Re: Extended Prefetching using Asynchronous IO - proposal and patch