Re: On columnar storage - Mailing list pgsql-hackers

From Jim Nasby
Subject Re: On columnar storage
Date
Msg-id 557F8448.5070104@BlueTreble.com
Whole thread Raw
In response to Re: On columnar storage  (Alvaro Herrera <alvherre@2ndquadrant.com>)
Responses Re: On columnar storage  (Alvaro Herrera <alvherre@2ndquadrant.com>)
List pgsql-hackers
On 6/14/15 10:22 AM, Alvaro Herrera wrote:
>> >To me, it feels like there are two different features here that would
>> >be better separated.  First, there's the idea of having a table that
>> >gets auto-joined to other tables whenever you access it, so that the
>> >user sees one really wide table but really the data is segregated by
>> >column groups under the hood.  That's a neat idea.
> Thanks.  (It also seems pretty tricky to implement.)

I look at it as a form of vertical partitioning; the big difference 
being whether you normalize the columns out or not (or to use data 
warehouse parlance, slow vs fast changing dimensions).

Perhaps it would be useful to vet this out as a userspace extension 
first since that would presumably be much easier. I believe we actually 
have all the backend infrastructure that would be needed for this now 
that views are smart enough to exclude tables that aren't referenced at 
all. I suspect that even a 'dumb userspace' approach would still expose 
a lot of the planner problems we'll run into (join explosion and 
filtering through the join come to mind).

Related to idea of an 'auto join', I do wish we had the ability to 
access columns in a referenced FK table from a referring key; something 
like SELECT customer_id.first_name FROM invoice (which would be 
translated to SELECT first_name FROM invoice JOIN customer USING( 
customer_id )).
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Data in Trouble? Get it in Treble! http://BlueTreble.com



pgsql-hackers by date:

Previous
From: Jim Nasby
Date:
Subject: Re: Need Multixact Freezing Docs
Next
From: Jim Nasby
Date:
Subject: Re: pg_stat_*_columns?