Postgres Pro
Downloads
Home   >   mailing lists

Thread: WAL-support for Pluggable Indexes

WAL-support for Pluggable Indexes

From
Simon Riggs
Date:
We've just rejected Knn-gist indexes as "not enough time for 9.0", which
is a considerable disappointment for many people.

We already have a pluggable index API, but not one that supports
recoverability.

It is a simple patch to add recoverability to the index API, if we have
the will to do so.

Let's add this into 9.0 now and let index development flourish without
the need for integration with core. PostgreSQL will benefit from having
index types grow alongside it. There will at times be additional changes
in core to optimise certain index use cases, that can come later. Let's
allow Postgres to be what it was always intended to be: extensible for
real world applications.

The must-have list of requirements are:
* must be possible to test whether rmgrid is set before allowing
XLogInsert()
* must allow normal rmgr APIs as well as index AM API

Not looking for the ability to redefine existing rmgrs, just ability to
add new ones.

I'm looking for agreement to proceed now and some help from those with
an interest.

-- Simon Riggs           www.2ndQuadrant.com

Re: WAL-support for Pluggable Indexes

From
Tom Lane
Date:
Simon Riggs <simon@2ndQuadrant.com> writes:
> We already have a pluggable index API, but not one that supports
> recoverability.

> It is a simple patch to add recoverability to the index API, if we have
> the will to do so.

I suggest you go re-read the archives before asserting this is a simple
no-thought-required fix.  If it were, it'd have been done before.

The killer problem as I recall it is how to identify the plugin rmgrs
to use, bearing in mind that you can't rely on looking at the catalogs.
We don't have a design for that, and I don't want one that's been thrown
together under intense schedule pressure.
        regards, tom lane

Re: WAL-support for Pluggable Indexes

From
Robert Haas
Date:
On Sun, Feb 21, 2010 at 12:54 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
> We've just rejected Knn-gist indexes as "not enough time for 9.0", which
> is a considerable disappointment for many people.
>
> We already have a pluggable index API, but not one that supports
> recoverability.
>
> It is a simple patch to add recoverability to the index API, if we have
> the will to do so.
>
> Let's add this into 9.0 now and let index development flourish without
> the need for integration with core. PostgreSQL will benefit from having
> index types grow alongside it. There will at times be additional changes
> in core to optimise certain index use cases, that can come later. Let's
> allow Postgres to be what it was always intended to be: extensible for
> real world applications.
>
> The must-have list of requirements are:
> * must be possible to test whether rmgrid is set before allowing
> XLogInsert()
> * must allow normal rmgr APIs as well as index AM API
>
> Not looking for the ability to redefine existing rmgrs, just ability to
> add new ones.
>
> I'm looking for agreement to proceed now and some help from those with
> an interest.

I am also disappointed that knngist didn't make it into 9.0, but it
seems somewhat orthogonal to the issue you're raising here.  Knngist
can't exist outside of core because it requires planner support and
changes to the opclass machinery; so even if we did this, it wouldn't
actually benefit the proposed use case.  That doesn't mean this is a
bad idea, of course, just that it doesn't solve that particular
problem.

...Robert