Re: huge price database question.. - Mailing list pgsql-general

From Steve Crawford
Subject Re: huge price database question..
Date
Msg-id 4F69ED33.3050706@pinpointresearch.com
Whole thread Raw
In response to Re: huge price database question..  (Jim Green <student.northwestern@gmail.com>)
Responses Re: huge price database question..  (Jim Green <student.northwestern@gmail.com>)
List pgsql-general
On 03/20/2012 06:16 PM, Jim Green wrote:
>
> It looks like alternatives are kind of complex to me, right now my
> approach(perl dbi and prepared insert) would take about 8/9 mins to
> insert  a day's data.  I think I'll probably just stick with it and
> wait.
>
> the autovacuum processes does a lot of io and make my pc unusable
> while I do the data inserts.. and I tested autovacuum off with not
> much success because of they are launched due to the transaction id
> wrap around issue.
>
Something sounds wrong, here. XID wraps around somewhere around
4-billion transactions which is a substantial multiple of the entire
number of records you are trying to insert. Do you have any unusual
vacuum settings?

As mentioned by others and myself, anything that batches lots of inserts
together into a single transaction instead of each record being its own
transaction will help whether this is COPY or BEGIN, INSERT......, COMMIT.

Cheers,
Steve

pgsql-general by date:

Previous
From: David Kerr
Date:
Subject: Re: postgresql.conf evaluation of duplicate keys
Next
From: salah jubeh
Date:
Subject: strange result with union