Re: Postgres as In-Memory Database? - Mailing list pgsql-general

From Stefan Keller
Subject Re: Postgres as In-Memory Database?
Date
Msg-id CAFcOn29A3BE6d=Babg-Mn57f3Lfam97inM=NrKH7SDvu3EFy8A@mail.gmail.com
Whole thread Raw
In response to Re: Postgres as In-Memory Database?  (Stefan Keller <sfkeller@gmail.com>)
Responses Re: Postgres as In-Memory Database?  (Hadi Moshayedi <hadi@citusdata.com>)
List pgsql-general
Hi,

I wrote
Finally the paper is mostly about column stores - nothing about persistence. 

Regarding column store, Hadi wrote 2014-04-03 18:43 GMT+02:00 about the release of a PostgreSQL Columnar Store called "cstore_fdw" [1]!

@Hadi: Can you say something about usage of cstore FDW in-memory?

Regards, S.





2014-04-02 0:32 GMT+02:00 Stefan Keller <sfkeller@gmail.com>:
Hi Yeb

Thanks for the pointers.

Of course disk access is not obsolete: As I said, I suppose changes are streamed to disk. 

When I mentioned "no disk access" I meant the indices of RDBMS which designed to handle disk access - which seems to me different in in-memory dabases.

The paper referred by you is coming from SAP's chief scientist and it confirms actually my claim, that there's no need for a primary index since the primary attribute (i.e. all attributes) is already kept sorted in-memory.

It also mentions an insert-only technique: "This approach has been adopted before in POSTGRES [21] in 1987 and was called "time-travel". 
I would be interested what "time-travel" is and if this is still used by Postgres. 
Finally the paper is mostly about column stores - nothing about persistence. In mentions Disaster recovery" in the last section about future work, though. 

-S.




2014-04-01 21:57 GMT+02:00 Yeb Havinga <yebhavinga@gmail.com>:

On 2014-04-01 04:20, Jeff Janes wrote:
On Sunday, March 30, 2014, Stefan Keller <sfkeller@gmail.com> wrote:
Hi Jeff

2013/11/20 Jeff Janes <jeff.janes@gmail.com>

I don't know what you mean about enhancements in the buffer pool.  For an in-memory database, there shouldn't be a buffer pool in the first place, as it is *all* in memory.  

You are right: In-memory DBs are making buffer-pooling obsolete - except for making data persistent (see below).


I would be very reluctant to use any database engine which considered disk access obsolete.

The disk is not obsolete but something called 'anti-caching' is used:
http://www.vldb.org/pvldb/vol6/p1942-debrabant.pdf


 
Are there any show cases out there?

What did the HANA users have to say?  Seems like they would be in the best position to provide the test cases.

This paper provides some insights into the research behind HANA http://www.sigmod09.org/images/sigmod1ktp-plattner.pdf

regards
Yeb



pgsql-general by date:

Previous
From: Stefan Keller
Date:
Subject: Re: Postgres as In-Memory Database?
Next
From: Rebecca Clarke
Date:
Subject: Initial queries of day slow