Re: autovacuum stress-testing our system - Mailing list pgsql-hackers

From Tom Lane
Subject Re: autovacuum stress-testing our system
Date
Msg-id 26698.1348676957@sss.pgh.pa.us
Whole thread Raw
In response to Re: autovacuum stress-testing our system  (Alvaro Herrera <alvherre@2ndquadrant.com>)
Responses Re: autovacuum stress-testing our system  (Jeff Janes <jeff.janes@gmail.com>)
Re: autovacuum stress-testing our system  (Tomas Vondra <tv@fuzzy.cz>)
Re: autovacuum stress-testing our system  (Tomas Vondra <tv@fuzzy.cz>)
List pgsql-hackers
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> Excerpts from Euler Taveira's message of mié sep 26 11:53:27 -0300 2012:
>> On 26-09-2012 09:43, Tomas Vondra wrote:
>>> 5) splitting the single stat file into multiple pieces - e.g. per database,
>>> written separately, so that the autovacuum workers don't need to read all
>>> the data even for databases that don't need to be vacuumed. This might be
>>> combined with (4).

>> IMHO that's the definitive solution. It would be one file per database plus a
>> global one. That way, the check would only read the global.stat and process
>> those database that were modified. Also, an in-memory map could store that
>> information to speed up the checks.

> +1

That would help for the case of hundreds of databases, but how much
does it help for lots of tables in a single database?

I'm a bit suspicious of the idea that we should encourage people to use
hundreds of databases per installation anyway: the duplicated system
catalogs are going to be mighty expensive, both in disk space and in
their cache footprint in shared buffers.  There was some speculation
at the last PGCon about how we might avoid the duplication, but I think
we're years away from any such thing actually happening.

What seems to me like it could help more is fixing things so that the
autovac launcher needn't even launch a child process for databases that
haven't had any updates lately.  I'm not sure how to do that, but it
probably involves getting the stats collector to produce some kind of
summary file.
        regards, tom lane



pgsql-hackers by date:

Previous
From: Dimitri Fontaine
Date:
Subject: Re: [9.1] 2 bugs with extensions
Next
From: Andrew Dunstan
Date:
Subject: data to json enhancements