Re: Batch update of indexes on data loading - Mailing list pgsql-hackers

From Simon Riggs
Subject Re: Batch update of indexes on data loading
Date
Msg-id 1209036501.4259.1499.camel@ebony.site
Whole thread Raw
In response to Re: Batch update of indexes on data loading  (Simon Riggs <simon@2ndquadrant.com>)
List pgsql-hackers
On Tue, 2008-02-26 at 09:08 +0000, Simon Riggs wrote:

> I very much like the idea of index merging, or put another way: batch
> index inserts. How big do the batch of index inserts have to be for us
> to gain benefit from this technique? Would it be possible to just buffer
> the index inserts inside the indexam module so that we perform a batch
> of index inserts every N rows? Maybe use work_mem? Or specify a batch
> size as a parameter on COPY? 

Itagaki,

I think the index merging idea is still useful even if we do not build a
whole new index. ISTM we can do this without locking the table also.

I understand it is most efficient when you do rebuild the index by
merging the old index with the incoming data, but it does seem there are
other problems associated with doing that.

Your idea still has a great future, IMHO.

--  Simon Riggs 2ndQuadrant  http://www.2ndQuadrant.com



pgsql-hackers by date:

Previous
From: Martijn van Oosterhout
Date:
Subject: Re: Index AM change proposals, redux
Next
From: Simon Riggs
Date:
Subject: Re: Index AM change proposals, redux