Re: Strategy for doing number-crunching - Mailing list pgsql-novice

From Tom Lane
Subject Re: Strategy for doing number-crunching
Date
Msg-id 16291.1325695712@sss.pgh.pa.us
Whole thread Raw
In response to Strategy for doing number-crunching  (Matthew Foster <matthew.foster@noaa.gov>)
Responses Re: Strategy for doing number-crunching  (Matthew Foster <matthew.foster@noaa.gov>)
List pgsql-novice
Matthew Foster <matthew.foster@noaa.gov> writes:
> We have a database with approximately 130M rows, and we need to produce
> statistics (e.g. mean, standard deviation, etc.) on the data.  Right now,
> we're generating these stats via a single SELECT, and it is extremely
> slow...like it can take hours to return results.

What datatype are the columns being averaged?  If "numeric", consider
casting to float8 before applying the aggregates.  You'll lose some
precision but it'll likely be orders of magnitude faster.

            regards, tom lane

pgsql-novice by date:

Previous
From: Matthew Foster
Date:
Subject: Strategy for doing number-crunching
Next
From: Sean Davis
Date:
Subject: Re: Strategy for doing number-crunching