Large table load (40 millon rows) - WAL hassles - Mailing list pgsql-general

From simon@gonzo.magellic.com (simon lai)
Subject Large table load (40 millon rows) - WAL hassles
Date
Msg-id slrn9k7af7.1cl.simon@gonzo.magellic.com
Whole thread Raw
Responses Re: Large table load (40 millon rows) - WAL hassles  (Bruce Momjian <pgman@candle.pha.pa.us>)
List pgsql-general
Version: Postgres 7.1.2

A product we are developing requires frequent loading
of a large number of rows into a table.  We are using
the "copy file" command, but with WAL we are effectively
doubling the amount of disk writing we are doing.

After the rows are loaded we do a "create index".

Is there a way to turn off WAL or otherwise speeding up
the table loading process? Would db_restore be faster?

The rows are loaded in sorted order.  Does this impact
index creation negatively or positively?

We are currently working with test data but we estimate
production data to be 6 - 9 billion rows.  Is anyone
else running with these volumes?

Simon

pgsql-general by date:

Previous
From: "Thalis A. Kalfigopoulos"
Date:
Subject: Re: Stored Procedure Newbie
Next
From: Alex Pilosov
Date:
Subject: Re: Why is it not using the other processor?