Re: bulk data loading - Mailing list pgsql-performance

From Magnus Hagander
Subject Re: bulk data loading
Date
Msg-id 20080408094017.349949af@mha-laptop
Whole thread Raw
In response to bulk data loading  ("Potluri Srikanth" <spotluri@ismartpanache.com>)
List pgsql-performance
Potluri  Srikanth wrote:
> Hi all,
>
> I need to do a bulk data loading around 704GB (log file size) at
> present  in 8 hrs (1 am - 9am). The data file size may increase 3 to
> 5 times in future.
>
> Using COPY it takes 96 hrs to finish the task.
> What is the best way to do it ?
>
> HARDWARE: SUN THUMPER/ RAID10
> OS : SOLARIS 10.
> DB: Greenplum/Postgres

If you're using Greenplum, you should probably be talking to the
Greenplum folks. IIRC, they have made some fairly large changes to the
load process, so they'll be the ones with the proper answers for you.

//Magnus

pgsql-performance by date:

Previous
From: Ow Mun Heng
Date:
Subject: Re: Forcing more agressive index scans for BITMAP AND
Next
From: hubert depesz lubaczewski
Date:
Subject: Re: bulk insert performance problem