Thread: Database-wide vacuum can take a long time, during which tables are not being analyzed

Steven Flatt wrote:
> Here is a potential problem with the auto-vacuum daemon, and I'm
> wondering if anyone has considered this.  To avoid transaction ID
> wraparound, the auto-vacuum daemon will periodically determine that it
> needs to do a DB-wide vacuum, which takes a long time.  On our system,
> it is on the order of a couple of weeks.  (The system is very busy and
> there is a lot of I/O going on pretty much 24/7).  During this period of
> time, there is nothing to automatically analyze any of the tables,
> leading to further performance problems.  What are your thoughts on
> having the DB-wide vacuum running on a separate thread so that the
> daemon can concurrently wake up and take care of analyzing tables?

Two issues here:
1)XID Wraparound:  There has been work done on this already, and in 8.2
I believe there will no longer be a requirement that a database wide
vacuum be issued, rather, XID wraparound will be managed on a per table
basis rather than per database, so that will solve this problem.

2)Concurrent Vacuuming:  There has been a lot of talk about
multiple-concurrent vacuums and I believe that this is required in the
long run, but it's not here yet, and won't be in 8.2, hopefully it will
get done for 8.3.


Matt

On Wed, 2006-11-01 at 14:15 -0500, Steven Flatt wrote:
> Here is a potential problem with the auto-vacuum daemon, and I'm
> wondering if anyone has considered this.  To avoid transaction ID
> wraparound, the auto-vacuum daemon will periodically determine that it
> needs to do a DB-wide vacuum, which takes a long time.  On our system,
> it is on the order of a couple of weeks.  (The system is very busy and
> there is a lot of I/O going on pretty much 24/7).  During this period
> of time, there is nothing to automatically analyze any of the tables,
> leading to further performance problems.  What are your thoughts on
> having the DB-wide vacuum running on a separate thread so that the
> daemon can concurrently wake up and take care of analyzing tables?

Yes, do it.

Every couple of weeks implies a transaction rate of ~~500tps, so I'd be
interested to hear more about your system.

--
  Simon Riggs
  EnterpriseDB   http://www.enterprisedb.com

Steven Flatt wrote:
> Sorry, I think there's a misunderstanding here.  Our system is not doing
> near that number of transactions per second.  I meant that the duration of a
> single DB-wide vacuum takes on the order of a couple of weeks.  The time
> between DB-wide vacuums is a little over a year, I believe.

I wonder if this is using some vacuum delay setting?

If that's the case, I think you could manually run a database-wide
vacuum with a zero vacuum delay setting, so that said vacuum takes less
time to finish (say, once every 8 months).

(8.2 pretty much solves this issue BTW, by not requiring database-wide
vacuums).

--
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.