Re: Postgres for a "data warehouse", 5-10 TB - Mailing list pgsql-performance

From Scott Marlowe
Subject Re: Postgres for a "data warehouse", 5-10 TB
Date
Msg-id CAOR=d=0U=+0v0UkpBacBTJh06rAwScpWMCvFrDO26OUA7jsctw@mail.gmail.com
Whole thread Raw
In response to Re: Postgres for a "data warehouse", 5-10 TB  (Robert Klemme <shortcutter@googlemail.com>)
List pgsql-performance
On Mon, Sep 12, 2011 at 11:04 AM, Robert Klemme
<shortcutter@googlemail.com> wrote:
> On 11.09.2011 22:10, Scott Marlowe wrote:
>
>> Another data point.  We had a big Oracle installation at my last job,
>> and OLAP queries were killing it midday, so I built a simple
>> replication system to grab rows from the big iron Oracle SUN box and
>> shove into a single core P IV 2.xGHz machine with 4 120G SATA drives
>> in SW RAID-10.
>>
>> That machine handily beat the big iron Oracle machine at OLAP queries,
>> running in 20 minutes what was taking well over an hour for the big
>> Oracle machine to do, even during its (Oracle machine) off peak load
>> times.
>
> Um, that sounds as if the SUN setup was really bad.  Do you remember any
> details about the drive configuration there?

It was actually setup quite well.  A very fast SAN with individual
drive arrays etc.  It was VERY fast at transactional throughput.  BUT
it was not setup for massive OLAP work.  The drives that housed the
statistical data we were running OLAP against were the slowest in the
set, since they were made to mostly just take in a small amount of
data each minute from the java servers.  Originally the stats had been
on a pg server in production and very fast, but some political
decision moved it onto the Oracle server.  The Oracle DBA wasn't any
happier with this move than me, btw.

pgsql-performance by date:

Previous
From: Andy Colson
Date:
Subject: Re: Allow sorts to use more available memory
Next
From: pasman pasmański
Date:
Subject: Re: Allow sorts to use more available memory