Re: pg_walinspect: add functions to locate and list WAL by time and LSN - Mailing list pgsql-hackers

From surya poondla
Subject Re: pg_walinspect: add functions to locate and list WAL by time and LSN
Date
Msg-id CAOVWO5qzZMzKtjZZd+zyua6JFnU56pO541d6=O7vyL+iNW6fJQ@mail.gmail.com
Whole thread
In response to Re: pg_walinspect: add functions to locate and list WAL by time and LSN  (Chao Li <li.evan.chao@gmail.com>)
Responses Re: pg_walinspect: add functions to locate and list WAL by time and LSN
List pgsql-hackers
Hi Chao,

Thanks for v3. Moving to outer anchors with the fallback to the current WAL position addresses my concern with v2, and I appreciate the new tests
covering both the empty-window case and the DELETE scenario.

v3 applies cleanly to master (f884f359f5a) and builds without warnings.
Locally, the regression tests and both TAP tests pass, and so does my earlier reproducer. The returned range now brackets the DELETE:

   start_lsn  |  end_lsn   | delete_lsn
  ------------+------------+------------
   0/017F0220 | 0/017F02E0 | 0/017F0280


However, CFBot's macOS - Meson job fails on v3 (https://github.com/postgres/postgres-cfbot/actions/runs/37738280802/job/113183259446) in 001_timeline.pl, during the archive_mode=0 pass:

  ERROR:  WAL segment needed for the requested range is missing
  DETAIL:  Segment 4 is not present in pg_wal.
  ... FROM pg_get_wal_files('0/04001790', pg_current_wal_lsn())

The standby's server log from that run points to a race in the test rather than a bug in pg_get_wal_files():

  06:52:52.657  received promote request
  06:52:52.701  checkpoint starting: force
  06:52:52.938  checkpoint complete: ... 0 WAL file(s) added, 0 removed, 2 recycled
  06:52:52.972  ERROR:  WAL segment needed for the requested range is missing

The checkpoint forced by the promotion recycled two WAL segments, and since segment 4, which holds start_lsn, was gone 34 ms later, it must have been
one of them. On a faster machine the query runs before that checkpoint completes, which would explain why the test passes locally.

Keeping that WAL on the standby makes the test deterministic. I confirmed this locally with a copy of the test that forces a CHECKPOINT right after
promote(): it fails every time with the same "Segment 4 is not present" error, and passes once the standby has wal_keep_size set:

$standby->init_from_backup($primary, $backup_name, has_streaming => 1);
$standby->append_conf('postgresql.conf', "wal_keep_size = '64MB'");

src/test/recovery/t/004_timeline_switch.pl uses wal_keep_size the same way.

One small question on 0001. For a point lookup at the current LSN on a segment boundary, v3 now returns the preceding segment, which makes sense
given that the next file may not exist yet. pg_walfile_name(), though, maps the same LSN to the next segment (it uses XLByteToSeg). So at that one LSN
the two functions name different files. I'm fine with the choice, since the docs explain it, but it may be worth a sentence noting the difference from
pg_walfile_name().

Regards,
Surya Poondla

pgsql-hackers by date:

Previous
From: Daniel Gustafsson
Date:
Subject: Re: Fix detection of truncated zstd-compressed backups
Next
From: Chao Li
Date:
Subject: Re: Fix detection of truncated zstd-compressed backups