Re: Re: Slowdown problem when writing 1.7million records - Mailing list pgsql-general

From Tom Lane
Subject Re: Re: Slowdown problem when writing 1.7million records
Date
Msg-id 7787.983302490@sss.pgh.pa.us
Whole thread Raw
In response to Re: Re: Slowdown problem when writing 1.7million records  ("Gordan Bobic" <gordan@freeuk.com>)
List pgsql-general
"Gordan Bobic" <gordan@freeuk.com> writes:
> Are you using autocommit? Are you using fsync? You should probably
> periodically commit the data every few thousand inserts if you don't
> have autocommit enabled.
> [ various other speedup suggestions from other people ]

The thing that interests me here is the apparent slowdown as more data
is inserted.  I don't see a good reason for that, and none of the
suggested remedies seem likely to eliminate it if we don't know what's
causing it.

INSERT per se should be an essentially constant-time operation, since
it's just slapping another record at the end of the table.  Unless
you're using advanced features like triggers/rules/foreign keys,
it seems like any progressive slowdown would have to be blamed on
updates of indexes.  But that's not normally a performance problem
except in pathological cases (zillions of equal keys for example ---
but Stephen's only index is a PRIMARY KEY, therefore UNIQUE, therefore
no equal keys).  So I'm confused, and think this deserves more careful
examination as to *why* the slowdown, rather than just looking for
a quick&dirty workaround.

            regards, tom lane

pgsql-general by date:

Previous
From: "Brent R. Matzelle"
Date:
Subject: Re: vacuum and backup
Next
From: Tom Lane
Date:
Subject: Re: Slowdown problem when writing 1.7million records