Re: slow speeds after 2 million rows inserted - Mailing list pgsql-general

From Nikola Milutinovic
Subject Re: slow speeds after 2 million rows inserted
Date
Msg-id 20061231161817.91257.qmail@web58710.mail.re1.yahoo.com
Whole thread Raw
In response to slow speeds after 2 million rows inserted  (James Neff <jneff@tethyshealth.com>)
Responses Re: slow speeds after 2 million rows inserted
Re: slow speeds after 2 million rows inserted
List pgsql-general
> 1. There is no difference (speed-wise) between committing every 1K or every 250K rows.

It was really some time ago, since I have experimented with this. My las experiment was on PG 7.2 or 7.3. I was inserting cca 800,000 rows. Inserting without transactions took 25 hrs. Inserting with 10,000 rows per transaction took about 2.5 hrs. So, the speedup was 10x. I have not experimented with the transaction batch size, but I suspect that 1,000 would not show much speedup.

> 2. Vacuuming also makes no difference for a heavy insert-only table, only slows it down.

Makes sense. Since my application was dumping all records each month and inserting new ones, vacuum was really needed, but no speedup.

> 3. Table size plays no real factor.

The reason I saw speedup, must have to do with the fact that without transactions, each insert was it's own transaction. That was eating resources.

Nix.

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

pgsql-general by date:

Previous
From: Christopher Browne
Date:
Subject: Re: Autovacuum Improvements
Next
From: novnov
Date:
Subject: Re: Generic timestamp function for updates where field