Re: Testing autovacuum wraparound (including failsafe) - Mailing list pgsql-hackers

From Heikki Linnakangas
Subject Re: Testing autovacuum wraparound (including failsafe)
Date
Msg-id 64524348-b10e-9904-4858-8fe62d058b14@iki.fi
Whole thread Raw
In response to Re: Testing autovacuum wraparound (including failsafe)  (Ian Lawrence Barwick <barwick@gmail.com>)
Responses Re: Testing autovacuum wraparound (including failsafe)  (Heikki Linnakangas <hlinnaka@iki.fi>)
Re: Testing autovacuum wraparound (including failsafe)  (Masahiko Sawada <sawada.mshk@gmail.com>)
Re: Testing autovacuum wraparound (including failsafe)  (Peter Geoghegan <pg@bowt.ie>)
List pgsql-hackers
On 16/11/2022 06:38, Ian Lawrence Barwick wrote:
> Thanks for the patch. While reviewing the patch backlog, we have determined that
> the latest version of this patch was submitted before meson support was
> implemented, so it should have a "meson.build" file added for consideration for
> inclusion in PostgreSQL 16.

I wanted to do some XID wraparound testing again, to test the 64-bit 
SLRUs patches [1], and revived this.

I took a different approach to consuming the XIDs. Instead of setting 
nextXID directly, bypassing GetNewTransactionId(), this patch introduces 
a helper function to call GetNewTransactionId() repeatedly. But because 
that's slow, it does include a shortcut to skip over "uninteresting" 
XIDs. Whenever nextXid is close to an SLRU page boundary or XID 
wraparound, it calls GetNewTransactionId(), and otherwise it bumps up 
nextXid close to the next "interesting" value. That's still a lot slower 
than just setting nextXid, but exercises the code more realistically.

I've written some variant of this helper function many times over the 
years, for ad hoc testing. I'd love to have it permanently in the git tree.

In addition to Masahiko's test for emergency vacuum, this includes two 
other tests. 002_limits.pl tests the "warn limit" and "stop limit" in 
GetNewTransactionId(), and 003_wraparound.pl burns through 10 billion 
transactions in total, exercising XID wraparound in general. 
Unfortunately these tests are pretty slow; the tests run for about 4 
minutes on my laptop in total, and use about 20 GB of disk space. So 
perhaps these need to be put in a special test suite that's not run as 
part of "check-world". Or perhaps leave out the 003_wraparounds.pl test, 
that's the slowest of the tests. But I'd love to have these in the git 
tree in some form.

[1] 
https://www.postgresql.org/message-id/CAJ7c6TPKf0W3MfpP2vr=kq7-NM5G12vTBhi7miu_5m8AG3Cw-w@mail.gmail.com)

- Heikki




pgsql-hackers by date:

Previous
From: Amit Kapila
Date:
Subject: Re: Fix comments in gistxlogDelete, xl_heap_freeze_page and xl_btree_delete
Next
From: David Rowley
Date:
Subject: Re: Allow ordered partition scans in more cases