Re: [HACKERS] [WIP]Vertical Clustered Index (columnar store extension) - Mailing list pgsql-hackers

From Haribabu Kommi
Subject Re: [HACKERS] [WIP]Vertical Clustered Index (columnar store extension)
Date
Msg-id CAJrrPGdjQ0MOS_LqjSu5=nw-Od1p2voMHhOrWYu8=SL5vAL6aA@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] [WIP]Vertical Clustered Index (columnar storeextension)  (Konstantin Knizhnik <k.knizhnik@postgrespro.ru>)
Responses Re: [HACKERS] [WIP]Vertical Clustered Index (columnar store extension)  (David Steele <david@pgmasters.net>)
List pgsql-hackers


On Tue, Feb 14, 2017 at 2:57 AM, Konstantin Knizhnik <k.knizhnik@postgrespro.ru> wrote:
Hi,

I wonder if it is possible to somehow benchmark your clustered index implementation.
I tried to create VCI index for lineitem table from TPC and run Q6 query.
After index creation Postgres is not using parallel execution plan any more but speed of sequential plan is not changed
and nothing in query execution plan indicates that VCI index is used:


postgres=# explain select
    sum(l_extendedprice*l_discount) as revenue
from
    lineitem_projection
where
    l_shipdate between '1996-01-01' and '1997-01-01'
    and l_discount between 0.08 and 0.1
    and l_quantity < 24;
                                                                                                                 QUERY PLAN                           
                                                                                    
-------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------
 Finalize Aggregate  (cost=608333.85..608333.86 rows=1 width=4)
   ->  Gather  (cost=608333.23..608333.84 rows=6 width=4)
         Workers Planned: 6
         ->  Partial Aggregate  (cost=607333.23..607333.24 rows=1 width=4)
               ->  Parallel Seq Scan on lineitem_projection  (cost=0.00..607024.83 rows=61680 width=8)
                     Filter: ((l_shipdate >= '1996-01-01'::date) AND (l_shipdate <= '1997-01-01'::date) AND (l_discount >= '0.08'::double precision) AN
D (l_discount <= '0.1'::double precision) AND (l_quantity < '24'::double precision))
(6 rows)

postgres=# select
    sum(l_extendedprice*l_discount) as revenue
from
    lineitem_projection
where
    l_shipdate between '1996-01-01' and '1997-01-01'
    and l_discount between 0.08 and 0.1
    and l_quantity < 24;
   revenue  
-------------
 6.21111e+08
(1 row)

Time: 1171.324 ms (00:01.171)

postgres=# create index vci_idx on lineitem_projection using vci(l_shipdate,l_quantity,l_extendedprice,l_discount,l_tax,l_returnflag,l_linestatus);
CREATE INDEX
Time: 4.705 ms


postgres=# explain select
    * from
    lineitem_projection
where                 
    l_shipdate between '1996-01-01' and '1997-01-01'
    and l_discount between 0.08 and 0.1
    and l_quantity < 24;
                                                                                                        QUERY PLAN                                    
                                                                  
-------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------
 Seq Scan on lineitem_projection  (cost=0.00..382077.00 rows=1 width=22)
   Filter: ((l_shipdate >= '1996-01-01'::date) AND (l_shipdate <= '1997-01-01'::date) AND (l_discount >= '0.08'::double precision) AND (l_discount <= '
0.1'::double precision) AND (l_quantity < '24'::double precision))
(2 rows)

postgres=# select                                                                                                                                 
    sum(l_extendedprice*l_discount) as revenue
from
    lineitem_projection
where
    l_shipdate between '1996-01-01' and '1997-01-01'
    and l_discount between 0.08 and 0.1
    and l_quantity < 24;
  revenue  
------------
 6.2112e+08
(1 row)

Time: 4304.355 ms (00:04.304)


I wonder if there is any query which can demonstrate advantages of using VCI index?

The current patch that I shared doesn't contains the plan and executor changes to show
the performance benefit of the clustered index. we used custom plan to generate the plan
for the clustered index. Currently I am working on it to rebase it to current master and
other necessary changes.

In the current state of the patch, I cannot take any performance tests, as it needs some
major changes according to the latest PostgreSQL version. I have an old performance
report that is took on 9.5 attached for your reference.

The current patch that is shared is to find out the best approach in developing a columnar
storage in PostgreSQL, by adopting Index access methods + additional hooks or pluggable
storage access methods?

The only problem I can think of pluggable storage methods is, to use the proper benefits of
columnar storage, the planner and executor needs to be changed to support vector processing,
But whereas in the current model, we implemented the same with custom plan and additional
hooks. The same may be possible with pluggable storage methods also.


Regards,
Hari Babu
Fujitsu Australia
Attachment

pgsql-hackers by date:

Previous
From: Haribabu Kommi
Date:
Subject: Re: [HACKERS] Parallel bitmap heap scan
Next
From: Amit Kapila
Date:
Subject: Re: [HACKERS] Parallel Index Scans