Re: Deteriorating performance when loading large objects - Mailing list pgsql-performance

From Vegard Bønes
Subject Re: Deteriorating performance when loading large objects
Date
Msg-id 49300F08.3000304@met.no
Whole thread Raw
In response to Re: Deteriorating performance when loading large objects  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-performance
Tom Lane wrote:
>> INFO:  vacuuming "pg_catalog.pg_largeobject"
>> INFO:  index "pg_largeobject_loid_pn_index" now contains 11060658 row
>> versions in 230587 pages
>> DETAIL:  178683 index pages have been deleted, 80875 are currently reusable.
>> CPU 0.92s/0.10u sec elapsed 199.38 sec.
>> INFO:  "pg_largeobject": found 0 removable, 11060658 nonremovable row
>> versions in 6849398 pages
>> DETAIL:  0 dead row versions cannot be removed yet.
>> There were 84508215 unused item pointers.
>> 0 pages are entirely empty.
>> CPU 0.98s/0.10u sec elapsed 4421.17 sec.
>> VACUUM
>
> Hmm ... although you have no dead rows now, the very large number of
> unused item pointers suggests that there were times in the past when
> pg_largeobject didn't get vacuumed often enough.  You need to look at
> your vacuuming policy.  If you're using autovacuum, it might need to have
> its parameters adjusted.  Otherwise, how often are you vacuuming, and
> are you doing it as superuser?
>
>> I will try to run VACUUM ANALYZE FULL after the next delete tonight, as
>> suggested by Ivan Voras in another post.
>
> Actually, a CLUSTER might be more effective.
>
>             regards, tom lane
>

I have autovacuum turned on, but the high number of unused item pointers
may be related to some experimentation I did earlier on the same
database with only vacuuming after the nightly delete. I have, however,
seen the same performance degradation on a database that had only ever
been vacuumed by autovacuum.

I am a bit unsure about what parameters to adjust in order to maintain a
good loading performance for bulk loading. Do you have any suggestions?

Also, VACUUM ANALYZE FULL has been running for 15 hours now, blocking
the loading of today's data. It will be interesting to see how the
database will work once it is completed. Is there any point in trying to
use CLUSTER instead if this does not give any result?


Regards,

Vegard Bønes


pgsql-performance by date:

Previous
From: "Andrus"
Date:
Subject: Increasing GROUP BY CHAR columns speed
Next
From: Scott Carey
Date:
Subject: Re: Increasing GROUP BY CHAR columns speed