Thread: Autoanalyze settings with zero scale factor

Some of my very large tables (10 million rows) need to be analyzed by
autovacuum on a frequent basis.  Rather than specifying this as a
percentage of table size + base threshold, I wanted to specify it as an
explicit number of rows.

I changed the table-specific settings so that the ANALYZE base threshold
was 5000 and the ANALYZE scale factor is 0.  According to the documented
formula: analyze threshold = analyze base threshold + analyze scale
factor * number of tuples, I assumed that this would cause the table to
be analyzed everytime 5000 tuples were inserted/updated/deleted.

However, the tables have been updated with tens of thousands of inserts
and the table has still not been analyzed (according to
pg_stat_user_tables).  Does a scale factor of 0 cause the table to never
be analyzed?  What am I doing wrong?

I'm using PG 8.2.1.

Thanks,
Jeremy Haile

Jeremy Haile wrote:
> I changed the table-specific settings so that the ANALYZE base threshold
> was 5000 and the ANALYZE scale factor is 0.  According to the documented
> formula: analyze threshold = analyze base threshold + analyze scale
> factor * number of tuples, I assumed that this would cause the table to
> be analyzed everytime 5000 tuples were inserted/updated/deleted.

That is right, and exactly how the scaling factor / base value are
supposed to work, so this should be fine.

> However, the tables have been updated with tens of thousands of inserts
> and the table has still not been analyzed (according to
> pg_stat_user_tables).  Does a scale factor of 0 cause the table to never
> be analyzed?  What am I doing wrong?  I'm using PG 8.2.1.

No a scaling factor of 0 shouldn't stop the table from being analyzed.

Unless it's just a bug, my only guess is that autovacuum may be getting
busy at times (vacuuming large tables for example) and hasn't had a
chance to even look at that table for a while, and by the time it gets
to it, there have been tens of thousands of inserts.  Does that sounds
plausible?

Also, are other auto-vacuums and auto-analyzes showing up in the
pg_stats table?  Maybe it's a stats system issue.

> Unless it's just a bug, my only guess is that autovacuum may be getting
> busy at times (vacuuming large tables for example) and hasn't had a
> chance to even look at that table for a while, and by the time it gets
> to it, there have been tens of thousands of inserts.  Does that sounds
> plausible?

Possible, but I think your next suggestion is more likely.

> Also, are other auto-vacuums and auto-analyzes showing up in the
> pg_stats table?  Maybe it's a stats system issue.

No tables have been vacuumed or analyzed today.  I had thought that this
problem was due to my pg_autovacuum changes, but perhaps not.  I
restarted PostgreSQL (in production - yikes)  About a minute after being
restarted, the autovac process fired up.

What could get PG in a state where autovac isn't running?  Is there
anything I should watch to debug or monitor for this problem in the
future?  I wish I'd noticed whether or not the stats collector process
was running before I restarted.

Jeremy Haile wrote:
>> Also, are other auto-vacuums and auto-analyzes showing up in the
>> pg_stats table?  Maybe it's a stats system issue.
>>
>
> No tables have been vacuumed or analyzed today.  I had thought that this
> problem was due to my pg_autovacuum changes, but perhaps not.  I
> restarted PostgreSQL (in production - yikes)  About a minute after being
> restarted, the autovac process fired up.
>
> What could get PG in a state where autovac isn't running?  Is there
> anything I should watch to debug or monitor for this problem in the
> future?  I wish I'd noticed whether or not the stats collector process
> was running before I restarted.

First off you shouldn't need to restart PG.  When it wasn't working did
you ever check the autovacuum_enabled setting? For example within psql:
"show autovacuum;".

I would venture to guess that autovacuum was disabled for some reason.
Perhaps last time you started the server the stats settings weren't enabled?

"Jeremy Haile" <jhaile@fastmail.fm> writes:
> No tables have been vacuumed or analyzed today.  I had thought that this
> problem was due to my pg_autovacuum changes, but perhaps not.  I
> restarted PostgreSQL (in production - yikes)  About a minute after being
> restarted, the autovac process fired up.

> What could get PG in a state where autovac isn't running?

Um, are you sure it wasn't?  The autovac process is not an always-there
thing, it quits after each pass and then the postmaster starts a new one
awhile later.

            regards, tom lane

Well - it hadn't run on any table in over 24 hours (according to
pg_stat_user_tables).  My tables are constantly being inserted into and
deleted from, and the autovacuum settings are pretty aggressive.  I also
had not seen the autovac process running in the past 24 hours. (although
I wasn't watching it *all* the time)

So - as far as I could tell it wasn't running.


On Thu, 18 Jan 2007 16:30:17 -0500, "Tom Lane" <tgl@sss.pgh.pa.us> said:
> "Jeremy Haile" <jhaile@fastmail.fm> writes:
> > No tables have been vacuumed or analyzed today.  I had thought that this
> > problem was due to my pg_autovacuum changes, but perhaps not.  I
> > restarted PostgreSQL (in production - yikes)  About a minute after being
> > restarted, the autovac process fired up.
>
> > What could get PG in a state where autovac isn't running?
>
> Um, are you sure it wasn't?  The autovac process is not an always-there
> thing, it quits after each pass and then the postmaster starts a new one
> awhile later.
>
>             regards, tom lane