Re: GSoC 2011: Fast GiST index build - Mailing list pgsql-hackers

From Heikki Linnakangas
Subject Re: GSoC 2011: Fast GiST index build
Date
Msg-id 4DB7C563.9060605@enterprisedb.com
Whole thread Raw
In response to Re: GSoC 2011: Fast GiST index build  (Alexander Korotkov <aekorotkov@gmail.com>)
Responses Re: GSoC 2011: Fast GiST index build  (Alexander Korotkov <aekorotkov@gmail.com>)
List pgsql-hackers
On 27.04.2011 09:51, Alexander Korotkov wrote:
> On Tue, Apr 26, 2011 at 1:10 PM, Alexander Korotkov<aekorotkov@gmail.com>wrote:
>
>> Since algorithm is focused to reduce I/O, we should expect best
>> acceleration in the case when index doesn't fitting to memory. Size of
>> buffers is comparable to size of whole index. It means that if we can hold
>> buffers in memory then we mostly can hold whole index in memory. That's why
>> I think we should have simple on-disk buffers management for first
>> representative benchmark.
>>
> Since we need to free all buffers after index built, I believe that buffers
> should be stored separately. If not, index become bloat immediatly after
> creation. We probably need to create a temporary relation to store buffers
> in it. If my thought is right, then is there any example of using temporary
> relation?

A temporary relation is a bit heavy-weight for this, a temporary file 
should be enough. See src/backend/storage/file/buffile.c, 
BufFileCreateTemp() function in particular. Or perhaps a tuplestore 
suits better, see src/backend/utils/sort/tuplestore.c, that's simpler to 
use if you're storing tuples. tuplestore only supports storing heap 
tuples at the moment, but it could easily be extended to store index 
tuples, like tuplesort.c does.

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com


pgsql-hackers by date:

Previous
From: Markus Wanner
Date:
Subject: Re: Proposal - asynchronous functions
Next
From: Heikki Linnakangas
Date:
Subject: Re: Memory leak in FDW