Re: Multi CPU Queries - Feedback and/or suggestions wanted! - Mailing list pgsql-hackers

From Greg Stark
Subject Re: Multi CPU Queries - Feedback and/or suggestions wanted!
Date
Msg-id D4703268-79F4-48F0-8A05-2D12774481A7@enterprisedb.com
Whole thread Raw
In response to Re: Multi CPU Queries - Feedback and/or suggestions wanted!  (Bruce Momjian <bruce@momjian.us>)
Responses Re: Multi CPU Queries - Feedback and/or suggestions wanted!
List pgsql-hackers
I thought about how to support both and ran into probblems that would  
make the resulting solutions quite complex.

In the libaio view of the world you initiate io and either get a  
callback or call another syscall to test if it's complete. Either  
approach has problems for postgres. If the process that initiated io  
is in the middle of a long query it might takr a long time ot even  
never get back to complete the io. The callbacks use threads...

And polling for completion has the problem that another process could  
be waiting on the io and can't issue a read as long as the first  
process has the buffer locked and io in progress. I think aio makes a  
lot more sense if you're using threads so you can start a thread to  
wait for the io to complete.

Actually I think it might be doable with a lot of work but I'm worried  
that it would be a lot of extra complexity even when you're not using  
it. The current patch doesn't change anything when you're not using it  
and actually is quite simple.

greg

On 24 Oct 2008, at 03:18 PM, Bruce Momjian <bruce@momjian.us> wrote:

> Jonah H. Harris wrote:
>> On Fri, Oct 24, 2008 at 7:59 AM, Hannu Krosing  
>> <hannu@2ndquadrant.com> wrote:
>>> On Fri, 2008-10-24 at 00:52 -0400, Jonah H. Harris wrote:
>>>> While we could build an
>>>> abstract prefetch interface and simply use fadvise for it now  
>>>> (rather
>>>> than OS-specific code), I don't see an easy win in any case.
>>>
>>> When building an abstract interface, always use at least two
>>> implementations (I guess that would be fadvise on linux and AIO on
>>> solaris in this case). You are much more likely to get the interface
>>> right this way.
>>
>> I agree, I just wasn't sure as to whether Greg's patch supported  
>> both methods.
>
> It does not, and probably will not for the near future;  we can only
> hope Solaris suports posix_fadvise() at some point.
>
> -- 
>  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
>  EnterpriseDB                             http://enterprisedb.com
>
>  + If your life is a hard drive, Christ can be your backup. +


pgsql-hackers by date:

Previous
From: Stefan Kaltenbrunner
Date:
Subject: Re: [pgsql-www] tribble.postgresql.org maintenance/hardware replacement
Next
From: Greg Stark
Date:
Subject: Re: Multi CPU Queries - Feedback and/or suggestions wanted!