Re: Autovacuum on partitioned table (autoanalyze) - Mailing list pgsql-hackers

From Álvaro Herrera
Subject Re: Autovacuum on partitioned table (autoanalyze)
Date
Msg-id 0794d7ca-5183-486b-9c5e-6d434867cecd@www.fastmail.com
Whole thread Raw
In response to Re: Autovacuum on partitioned table (autoanalyze)  (Andres Freund <andres@anarazel.de>)
Responses Re: Autovacuum on partitioned table (autoanalyze)  (Álvaro Herrera <alvherre@alvh.no-ip.org>)
List pgsql-hackers
Here is a proposal for 14.  This patch has four main changes:

* The mod counts are only propagated to the topmost parent, not to each ancestor.  This means that we'll only analyze
thetopmost partitioned table and not each intermediate partitioned table; seems a good compromise to avoid sampling all
partitionsmultiple times per round.
 

* One pgstat message is sent containing many partition/parent pairs, not just one. This reduces the number of messages
sent. 123 partitions fit in one message (messages are 1000 bytes).  This is done once per autovacuum worker run, so it
shouldn'tbe too bad.
 

* There's a sleep between sending the message and re-reading stats.  It would be great to have a mechanism by which
pgstatcollector says "I've received and processed up to this point", but we don't have that; what we can do is sleep
PGSTAT_STAT_INTERVALand then reread the file, so we're certain that the file we read is at least as new as that time.
Thisis far longer than it takes to process the messages.  Note that if the messages do take longer than that to be
processedby the collector, it's not a big loss anyway; those tables will be processed by the next autovacuum run.
 

* I changed vacuum_expand_rel to put the main-rel OID at the end. (This is a variation of Horiguchi-san proposed patch;
insteadof making the complete list be in the opposite order, it's just that one OID that appears at the other end).
Thishas the same effect as his patch: any error reports thrown by vacuum/analyze mention the first partition rather
thanthe main table.  This part is in 0002 and I'm not totally convinced it's a sane idea.
 

Minor changes:
* I reduced autovacuum from three passes over pg_class to two passes, per your observation that we can acquire toast
associationtogether with processing partitions, and then use that in the second pass to collect everything.
 

* I moved the catalog-accessing code to partition.c, so we don't need to have pgstat.c doing it.

Some doc changes are pending, and some more commentary in parts of the code, but I think this is much more sensible.  I
dolament the lack of a syscache for pg_inherits. 
Attachment

pgsql-hackers by date:

Previous
From: "David G. Johnston"
Date:
Subject: Re: Default to TIMESTAMP WITH TIME ZONE?
Next
From: David Zhang
Date:
Subject: Re: [UNVERIFIED SENDER] Re: [bug] Logical Decoding of relation rewrite with toast does not reset toast_hash