Re: Multi CPU Queries - Feedback and/or suggestions wanted! - Mailing list pgsql-hackers

From Aidan Van Dyk
Subject Re: Multi CPU Queries - Feedback and/or suggestions wanted!
Date
Msg-id 20081024150532.GK16893@yugib.highrise.ca
Whole thread Raw
In response to Re: Multi CPU Queries - Feedback and/or suggestions wanted!  (Greg Stark <greg.stark@enterprisedb.com>)
List pgsql-hackers
* Greg Stark <greg.stark@enterprisedb.com> [081024 10:48]:
> I thought about how to support both and ran into probblems that would  
> make the resulting solutions quite complex.
> 
> In the libaio view of the world you initiate io and either get a  
> callback or call another syscall to test if it's complete. Either  
> approach has problems for postgres. If the process that initiated io  
> is in the middle of a long query it might takr a long time ot even  
> never get back to complete the io. The callbacks use threads...
> 
> And polling for completion has the problem that another process could  
> be waiting on the io and can't issue a read as long as the first  
> process has the buffer locked and io in progress. I think aio makes a  
> lot more sense if you're using threads so you can start a thread to  
> wait for the io to complete.
> 
> Actually I think it might be doable with a lot of work but I'm worried  
> that it would be a lot of extra complexity even when you're not using  
> it. The current patch doesn't change anything when you're not using it  
> and actually is quite simple.

In the Solaris async IO, are you bound by direct IO?  Does the OS page-cache
still get primed by async reads?  If so, how about starting async IO
into a "throwaway" local buffer;  treat async IO in the same way as
fadvise, a "pre-load the OS page cache so the real read is quick".

Sure, I understand it's not the "perfect model", but it I don't see
PostgreSQL being refactored enough to have a pure async model happening
any time in the near future...

-- 
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

pgsql-hackers by date:

Previous
From: Greg Stark
Date:
Subject: Re: Multi CPU Queries - Feedback and/or suggestions wanted!
Next
From: Simon Riggs
Date:
Subject: Reducing some DDL Locks to ShareLock