Re: feeding big script to psql - Mailing list pgsql-general

From Scott Marlowe
Subject Re: feeding big script to psql
Date
Msg-id 1122998231.21793.11.camel@state.g2switchworks.com
Whole thread Raw
In response to Re: feeding big script to psql  (Havasvölgyi Ottó <h.otto@freemail.hu>)
List pgsql-general
On Tue, 2005-08-02 at 04:24, Havasvölgyi Ottó wrote:
> Tom,
>
> Thanks for the suggestion. I have just applied both switch , -f (I have
> applied this in the previous case too) and -n, but it becomes slow again. At
> the beginning it reads about 300 KB a second, and when it has read 1.5 MB,
> it reads only about 10 KB a second, it slows down gradually. Maybe others
> should also try this scenario. Can I help anything?

I be you've got an issue where a seq scan on an fk field or something
works fine for the first few thousand rows.  At some point, pgsql should
switch to an index scan, but it just doesn't know it.

Try wrapping every 10,000 or so inserts with

begin;
<insert 10,000 rows>
commit;
analyze;
begin;
rinse, wash repeat.

You probably won't need an analyze after the first one though.

pgsql-general by date:

Previous
From: Dan Armbrust
Date:
Subject: Re: Slow Inserts on 1 table?
Next
From: Dan Armbrust
Date:
Subject: Force PostgreSQL to use indexes on foreign key lookups - Was: Slow Inserts on 1 table?