Re: [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL - Mailing list pgsql-hackers

From Jim Nasby
Subject Re: [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL
Date
Msg-id 52CF0EEF.2050408@nasby.net
Whole thread Raw
In response to Re: [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL  (knizhnik <knizhnik@garret.ru>)
List pgsql-hackers
On 1/9/14, 1:18 PM, knizhnik wrote:
> So it is clear why do we need shared memory for parallel query execution. But why it has to be dynamic? Why it can
notbe preallocated at start time as most of other resources used by PostgreSQL?
 

That would limit us to doing something like allocating a fixed maximum of parallel processes (which might be workable)
andonly allocating a very small amount of memory for IPC. Small as in can only handle a small number of tuples. That
soundslike a really inefficient way to shuffle data to and from parallel processes, especially because one or both
sideswould probably have to actually copy the data if we're doing it that way.
 

With DSM if you want to do something like a parallel sort each process can put their results into memory that the
parentprocess can directly access.
 

Of course the other enormous win for DSM is it's the foundation for finally being able to resize things without a
restart.For large dollar sites that ability would be hugely beneficial.
 
-- 
Jim C. Nasby, Data Architect                       jim@nasby.net
512.569.9461 (cell)                         http://jim.nasby.net



pgsql-hackers by date:

Previous
From: Greg Stark
Date:
Subject: Re: Planning time in explain/explain analyze
Next
From: Steeve Lennmark
Date:
Subject: Re: [PATCH] Relocation of tablespaces in pg_basebackup