Re: Strategy for doing number-crunching - Mailing list pgsql-novice

From Matthew Foster
Subject Re: Strategy for doing number-crunching
Date
Msg-id CAP1ZYZGiic53RbHdj5vC=3QVXpmZ01CwB_b5_dcPrYdn7=G2Aw@mail.gmail.com
Whole thread Raw
In response to Re: Strategy for doing number-crunching  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Strategy for doing number-crunching  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-novice
On Wed, Jan 4, 2012 at 10:48 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Matthew Foster <matthew.foster@noaa.gov> writes:
> We have a database with approximately 130M rows, and we need to produce
> statistics (e.g. mean, standard deviation, etc.) on the data.  Right now,
> we're generating these stats via a single SELECT, and it is extremely
> slow...like it can take hours to return results.

What datatype are the columns being averaged?  If "numeric", consider
casting to float8 before applying the aggregates.  You'll lose some
precision but it'll likely be orders of magnitude faster.

                       regards, tom lane

Tom,

The data are type double.

Matt

pgsql-novice by date:

Previous
From: "Jean-Yves F. Barbier"
Date:
Subject: Re: Strategy for doing number-crunching
Next
From: Tom Lane
Date:
Subject: Re: Strategy for doing number-crunching