Re: [HACKERS] [WIP]Vertical Clustered Index (columnar store extension) - Mailing list pgsql-hackers

From Haribabu Kommi
Subject Re: [HACKERS] [WIP]Vertical Clustered Index (columnar store extension)
Date
Msg-id CAJrrPGeyuc7LZmp1MsPQkAgGFa1CPf8m9pNe4aQv5HtdmQBHrw@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] [WIP]Vertical Clustered Index (columnar storeextension)  (Peter Eisentraut <peter.eisentraut@2ndquadrant.com>)
List pgsql-hackers


On Wed, Jan 18, 2017 at 2:25 PM, Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote:
On 12/29/16 10:55 PM, Haribabu Kommi wrote:
> Fujitsu was interested in developing a columnar storage extension with
> minimal
> changes the server backend.
>
> The columnar store is implemented as an extension using index access
> methods.
> This can be easily enhanced with pluggable storage methods once they are
> available.
>
> A new index method (VCI) is added to create columnar index on the table.

I'm confused.  You say that you are adding an index access method, for
which we have a defined extension mechanism, but the code doesn't do
that.  Instead, it sprinkles a bunch of hooks through the table access
code.  So you are really adding ways to add alternatives to heap
storage, except we have no way to know whether these hooks have been
designed with any kind of generality in mind.  So is it an index access
method or a table access method?

Yes, it is a mix of both index and table access methods. The current design
of Vertical clustered index needs both access methods, because of this reason
we used both access methods.

Either way, you shouldn't need a new relkind.  Note that all indexes
have the same relkind, even if they use different access methods.

I think there are two ways to integrate column storage into PostgreSQL:
One is to use the FDW interface.  That has been done before, see
cstore_fdw.  The other is to define a storage manager extension
interface.  That has been tried but has not been completed yet.  Adding
a bunch of custom hooks all over the place seems worse than both of those.

Thanks for your suggestion. Yes, I also agree that the best way to integrate
column storage for a better performance is through storage manager extension
interface.

It is better first try to finish the pluggable storage interface and integrate this
columnar store is a good way to proceed.

Regards,
Hari Babu
Fujitsu Australia

pgsql-hackers by date:

Previous
From: Kyotaro HORIGUCHI
Date:
Subject: Re: [HACKERS] [BUGS] Bug in Physical Replication Slots (at least 9.5)?
Next
From: Tom Lane
Date:
Subject: Re: [HACKERS] Possible issue with expanded object infrastructure on Postgres 9.6.1