Postgres Pro
Downloads
Home   >   mailing lists

Thread: slow insert into very large table

slow insert into very large table

From
Wolfgang Gehner
Date:
Hi there,

I need a simple but large table with several million records. I do batch
inserts with JDBC. After the first million or so records,
the inserts degrade to become VERY slow (like 8 minutes vs initially 20
secondes).

The table has no indices except PK while I do the inserts.

This is with PostgreSQL 8.0 final for WindowsXP on a Pentium 1.86 GHz,
1GB Memory. HD is fast IDE.

I already have shared buffers already set to 25000.

I wonder what else I can do. Any ideas?

Kindest regards,

Wolfgang Gehner

--
Infonoia SA
7 rue de Berne
1211 Geneva 1
Tel: +41 22 9000 009
Fax: +41 22 9000 018
http://www.infonoia.com

Re: slow insert into very large table

From
Andreas Pflug
Date:
Wolfgang Gehner wrote:
> Hi there,
>
> I need a simple but large table with several million records. I do batch
> inserts with JDBC. After the first million or so records,
> the inserts degrade to become VERY slow (like 8 minutes vs initially 20
> secondes).
>
> The table has no indices except PK while I do the inserts.
>
> This is with PostgreSQL 8.0 final for WindowsXP on a Pentium 1.86 GHz,
> 1GB Memory. HD is fast IDE.
>
> I already have shared buffers already set to 25000.
>
> I wonder what else I can do. Any ideas?

Run VACUUM ANALYZE to have statistics reflect the growth of the table.
The planner probably still assumes your table to be small, and thus
takes wrong plans to check PK indexes or so.

Regards,
Andreas

Re: slow insert into very large table

From
Tom Lane
Date:
Wolfgang Gehner <wgehner@infonoia.com> writes:
> This is with PostgreSQL 8.0 final for WindowsXP on a Pentium 1.86 GHz,
> 1GB Memory. HD is fast IDE.

Try something more recent, like 8.0.3 or 8.0.4.  IIRC we had some
performance issues in 8.0.0 with tables that grew from zero to large
size during a single session.

            regards, tom lane