Postgres Pro
Downloads
Home   >   mailing lists

Thread: TPC H data

TPC H data

From
Shalu Gupta
Date:
Hello,

We are trying to import the TPC-H data into postgresql using the COPY
command and for the larger files we get an error due to insufficient
memory space.

We are using a linux system with Postgresql-7.3.4

Is it that Postgresql cannot handle such large files or is there some
other possible reason.

Thanks
Shalu Gupta
NC State University.

Re: TPC H data

From
Andrew Dunstan
Date:
Shalu Gupta wrote:

>Hello,
>
>We are trying to import the TPC-H data into postgresql using the COPY
>command and for the larger files we get an error due to insufficient
>memory space.
>
>We are using a linux system with Postgresql-7.3.4
>
>Is it that Postgresql cannot handle such large files or is there some
>other possible reason.
>
>Thanks
>Shalu Gupta
>NC State University.
>
>
>

Shalu,

I loaded the largest TPC-H table (lineitem, roughly 6 million rows) the
other day into a completely untuned 7.5devel PostgreSQL instance running
on RH 9, and it didn't raise a sweat. I delayed creating the indexes
until after the load. Data load took roughly 10 minutes, index creation
took a further 35 minutes (there are 13 of them).

HTH. (I'm just down the road from NCSU, would be happy to help out)

cheers

andrew

Re: TPC H data

From
"scott.marlowe"
Date:
On Wed, 21 Apr 2004, Shalu Gupta wrote:

> Hello,
>
> We are trying to import the TPC-H data into postgresql using the COPY
> command and for the larger files we get an error due to insufficient
> memory space.
>
> We are using a linux system with Postgresql-7.3.4
>
> Is it that Postgresql cannot handle such large files or is there some
> other possible reason.

what method(s) are you using to load the data?

Re: [GENERAL] TPC H data

From
Mark Kirkwood
Date:
What scale factor TPC H are you importing?

additionally -  might be worth giving the specs of the machine you are
doing this on.

(I seem to recall trying this with Pg 7.2 a while ago without this
issue, mind you -  think I had ~1G of Ram and used the scale fact 1
dataset, i.e 1G)

regards

Mark


Shalu Gupta wrote:

>Hello,
>
>We are trying to import the TPC-H data into postgresql using the COPY
>command and for the larger files we get an error due to insufficient
>memory space.
>
>We are using a linux system with Postgresql-7.3.4
>
>Is it that Postgresql cannot handle such large files or is there some
>other possible reason.
>
>Thanks
>Shalu Gupta
>NC State University.
>
>---------------------------(end of broadcast)---------------------------
>TIP 5: Have you checked our extensive FAQ?
>
>               http://www.postgresql.org/docs/faqs/FAQ.html
>
>
>
>