Re: Atomic access to large arrays - Mailing list pgsql-performance

From Tom Lane
Subject Re: Atomic access to large arrays
Date
Msg-id 19329.1248270877@sss.pgh.pa.us
Whole thread Raw
In response to Atomic access to large arrays  ("Victor de Buen (Bayes)" <vdebuen@bayesinf.com>)
Responses Re: Atomic access to large arrays  (Victor de Buen <vdebuen@gmail.com>)
Re: Atomic access to large arrays  ("Victor de Buen (Bayes)" <vdebuen@bayesinf.com>)
List pgsql-performance
"Victor de Buen (Bayes)" <vdebuen@bayesinf.com> writes:
> I'm storing historical meteorological gridded data from GFS (
> http://www.nco.ncep.noaa.gov/pmb/products/gfs/) into an array field in a
> table like this:

>   vl_grid smallint[361][720],

>    - It's posible to tune some TOAST parameters to get faster atomic access
>    to large arrays?

It might save a little bit to make the toast chunk size larger, but I'm
not sure you could gain much from that.

>    - Using "EXTERNAL" strategy for storing TOAST-able columns could solve
>    the problem?

Nope, wouldn't help --- AFAIR array access is not optimized for slice
access.  In any case, doing that would give up the compression savings
that you were so happy about.

If your normal access patterns involve "vertical" rather than
"horizontal" scans of the data, maybe you should rethink the choice
of table layout.  Or maybe the compression is enough to allow you
to consider storing the data twice, once in the current layout and
once in a "vertical" format.

            regards, tom lane

pgsql-performance by date:

Previous
From: Scott Marlowe
Date:
Subject: Re: Master/Slave, DB separation or just spend $$$?
Next
From: Victor de Buen
Date:
Subject: Re: Atomic access to large arrays