Re: Add a test for index_rebuild_count of REPACK (CONCURRENTLY) - Mailing list pgsql-hackers

From Manu
Subject Re: Add a test for index_rebuild_count of REPACK (CONCURRENTLY)
Date
Msg-id 179018818545.3031865.5809261654654798774@gmail.com
Whole thread
In response to Re: Add a test for index_rebuild_count of REPACK (CONCURRENTLY)  (Manu <manuelreyesbravo@gmail.com>)
List pgsql-hackers
The CFBot still could not apply the series, and this time it was the
mail, not the tree: the attachments of my last message went out as
text/plain without base64, and on the way the first line of each was
escaped from "From <sha1>" to ">From <sha1>", which git am does not
recognize ("Patch format detection failed" in the CFBot log).  They
are base64-encoded this time.

v3, rebased on master (374522aa63a), changes two things.

0005 now runs every command that reports progress, which Álvaro asked
for.  New since v2:

- REINDEX TABLE and REINDEX INDEX CONCURRENTLY
- CREATE INDEX on a partitioned table (partitions_total/done)
- ANALYZE of a partitioned table with extended statistics
- COPY FROM with WHERE and ON_ERROR ignore (tuples_excluded/skipped)
- VACUUM FULL
- disabling and enabling data checksums online, launcher and workers

Data checksums needed one addition to the rules: their counters start
at -1, which the view shows as NULL, so a counter may now go to and
come back from a command's NULL value.

The new cases earn their keep on the old bugs, too.  With 0001-0004
reverted, the checker used to fail 10 checks; it now fails 14, the new
ones in REINDEX TABLE, REINDEX CONCURRENTLY, CREATE INDEX on a
partitioned table and VACUUM FULL, all of them the same unreset
counters.

0006 changes in three ways:

- It no longer compares the order of the phase tables in
  monitoring.sgml with the order of execution.  Sami made the point in
  the REPACK phases thread [1] that the documented order is not a
  guarantee, and 001 already checks the actual succession of phases.
  What remains is the drift check: every phase a command reports must
  be named by the view and listed in the documentation.
- That check used to run only when it found a problem, so a clean run
  passed without testing anything; it is now one test per command.
- 002_doc_phases.pl only ran when PROGRESS_DEBUG_BUILD=1 was set in the
  environment, and nothing sets it, so on a server compiled with
  -DPROGRESS_DEBUG -- the buildfarm animal Álvaro suggested -- it would
  have been skipped every time.  It now detects the build the same way
  001_progress.pl does; that check moved into ProgressCheck.pm as
  compiled_with_progress_debug().

0001-0004 are unchanged apart from the rebase.

Tested with meson:

- with -DPROGRESS_DEBUG: 001 passes 80 subtests (53 in v2), and 002
  passes 7;
- 0005 alone passes the same 80;
- 002 fails, naming the phase, when a row of a phase table is renamed;
- without PROGRESS_DEBUG: 001 only checks progress.h against its
  description, and 002 skips, saying why;
- the main regression suite passes, with no compiler warnings.

Regards,
Manu

[1]
https://postgr.es/m/CAN12+YKo-vjvdPQts6QnHoB3ET5A2or137oN_FGMviQdtYLN6Q@mail.gmail.com

Attachment

pgsql-hackers by date:

Previous
From: Masahiko Sawada
Date:
Subject: Re: REPACK (CONCURRENTLY) can silently lose updates when the toast table is rewritten
Next
From: Sami Imseih
Date:
Subject: Re: REPACK (CONCURRENTLY) loses missing values of columns added without a rewrite