Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem - Mailing list pgsql-hackers

From Alvaro Herrera
Subject Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem
Date
Msg-id 20180207235226.zygu4r3yv3yfcnmc@alvherre.pgsql
Whole thread Raw
In response to Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem  (Claudio Freire <klaussfreire@gmail.com>)
Responses Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem  (Claudio Freire <klaussfreire@gmail.com>)
List pgsql-hackers
Claudio Freire wrote:
> On Wed, Feb 7, 2018 at 7:57 PM, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > Claudio Freire wrote:

> > Hmm, this solution is not very friendly to the goal of reducing test
> > runtime, particularly since the new test creates a nontrivial-sized
> > table.  Maybe we can find a better alternative.  Can we use some wait
> > logic instead?  Maybe something like grab a snapshot of running VXIDs
> > and loop waiting until they're all gone before doing the vacuum?
> 
> I'm not sure there's any alternative. I did some tests and any active
> snapshot on any other table, not necessarily on the one being
> vacuumed, distorted the test. And it makes sense, since that snapshot
> makes those deleted tuples unvacuumable.

Sure.

> Waiting as you say would be akin to what the patch does by putting
> vacuum on its own parallel group.

I don't think it's the same.  We don't need to wait until all the
concurrent tests are done -- we only need to wait until the transactions
that were current when the delete finished are done, which is very
different since each test runs tons of small transactions rather than
one single big transaction.

> > Also, I don't understand why pg_relation_size() is not a better solution
> > to determining the table size compared to explain.
> 
> I was told pg_relation_size can return stale information. I didn't
> check, I took that at face value.

Hmm, it uses stat() on the table files.  I think those files would be
truncated at the time the transaction commits, so they shouldn't be
stale.  (I don't think the system waits for a checkpoint to flush a
truncation.)  Maybe relying on that is not reliable or future-proof
enough.  Anyway this is a minor point -- the one above worries me most.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


pgsql-hackers by date:

Previous
From: Claudio Freire
Date:
Subject: Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem
Next
From: Thomas Munro
Date:
Subject: Re: [COMMITTERS] pgsql: Rearm statement_timeout after each executed query.