Re: large inserts and fsync - Mailing list pgsql-general

From Sam Mason
Subject Re: large inserts and fsync
Date
Msg-id 20080905151047.GY7271@frubble.xen.chris-lamb.co.uk
Whole thread Raw
In response to large inserts and fsync  (Aaron Burnett <aburnett@bzzagent.com>)
Responses Re: large inserts and fsync  (Aaron Burnett <aburnett@bzzagent.com>)
List pgsql-general
On Fri, Sep 05, 2008 at 09:16:41AM -0400, Aaron Burnett wrote:
> For an upcoming release there is a 16 million row insert that on our test
> cluster takes about 2.5 hours to complete with all indices dropped
> beforehand.
>
> If I turn off fsync, it completes in under 10 minutes.

Have you tried bundling all the INSERT statements into a single
transaction?  If you haven't then PG will run each statement in its own
transaction and then commit each INSERT statement to disk separately,
incurring large overheads.


  Sam

pgsql-general by date:

Previous
From: "Filip Rembiałkowski"
Date:
Subject: Re: a performence question
Next
From: Aaron Burnett
Date:
Subject: Re: large inserts and fsync