Re: Making pg_rewind faster - Mailing list pgsql-hackers

From Srinath Reddy Sadipiralla
Subject Re: Making pg_rewind faster
Date
Msg-id CAFC+b6rcHL3M3s7poZT1iHKc9Tzw5hqihiKpBx6KJCY1f6R5mg@mail.gmail.com
Whole thread Raw
In response to Re: Making pg_rewind faster  (Srinath Reddy Sadipiralla <srinath2133@gmail.com>)
List pgsql-hackers


On Wed, Oct 15, 2025 at 7:56 PM Srinath Reddy Sadipiralla <srinath2133@gmail.com> wrote:

thanks for updating, i have attached a diff patch on
top of v9-0001 patch , where i tried to add more tests
to improve the validation that we copy the WAL file even
when it exists on both source and target but the size differs.
I also tried to rename some variables for readability,while
testing this out i found that usleep(1000); is not enough
to show that there has been a copy which changed the
stat->mtime of the file because i think the copy was very
fast (less than 1 millisecond) so the mtime of the
file after rewind's copy didn't change or filesystem precision
didn't caught the change, the copying of the file is confirmed
because the same file has different size before rewind and
same after rewind which these tests proved  
 
ok($corrupt_wal_seg_stat_before_rewind->size != $corrupt_wal_seg_source_stat->size,"WAL segment $corrupt_wal_seg has different size in source vs target before rewind");
ok 11 - WAL segment 000000010000000000000004 has different size in source vs target before rewind

ok($corrupt_wal_seg_stat_after_rewind->size == $corrupt_wal_seg_source_stat->size, "WAL segment $corrupt_wal_seg was copied: file sizes are same between target and source after rewind");
ok 12 - WAL segment 000000010000000000000004 was copied: file sizes are same between target and source after rewind

and more over the pg_rewind confirmed this with a new debug
message which I added in this diff patch

diff --git a/src/bin/pg_rewind/filemap.c b/src/bin/pg_rewind/filemap.c
index 69d7728ce44..743484f4833 100644
--- a/src/bin/pg_rewind/filemap.c
+++ b/src/bin/pg_rewind/filemap.c
@@ -732,11 +732,11 @@ decide_wal_file_action(const char *fname, XLogSegNo last_common_segno,
  */
  if (file_segno < last_common_segno && source_size == target_size)
  {
  pg_log_debug("WAL segment \"%s\" not copied to target", fname);
  return FILE_ACTION_NONE;
  }
 
+ pg_log_debug("WAL segment \"%s\" is copied to target", fname);
  return FILE_ACTION_COPY;
 }
ok 6 - run pg_rewind stderr /(?^:WAL segment \"000000010000000000000004\" is copied to target)/
 
 
 
btw i forgot to let you know that i have increased the
usleep(1000) to usleep(1000000) for the above reason
which helped to show the difference in mtime after rewind
adding the below extra validation that the file has been copied from
source to target ,this is already done in the v9-0001.diff patch. 

ok($corrupt_wal_seg_stat_after_rewind->mtime > $corrupt_wal_seg_stat_before_rewind->mtime,"WAL segment $corrupt_wal_seg was copied: mtime after rewind > mtime before rewind");
 ok 12 - WAL segment 000000010000000000000004 was copied: mtime after rewind > mtime before rewind


On Fri, Oct 10, 2025 at 3:18 AM John H <johnhyvr@gmail.com> wrote:
Hey Srinath

On Thu, Oct 9, 2025 at 12:09 PM Srinath Reddy Sadipiralla
<srinath2133@gmail.com> wrote:
> ...
> 1) if the timeline switch happens in middle of segment ,copy data from the last WAL segment and create WAL file with same segno but different timelineID,in this case the starting WAL file for the new timeline will be 000000020000000000000003
> 2) if the timeline switch happens at segment boundary , just create next segment for this case the starting WAL file for the new timeline will be 000000020000000000000004
>
> so basically the files which exists in source and not in target like the new timeline WAL segments will be copied to target in total before we reach decide_wal_file_action , so i think we don't need to think about copying WAL files after divergence point by calculating and checking against last_common_segno which we are doing in our current approach , i think we can just do
>
> ...

That's a great point. I need to think about it some more but the
reasoning makes sense to me.
I think 'last_common_segno ' is only useful as another sanity check
but we already have the size
ones.

it would be awesome to know ,if you have any update on this.

--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/

pgsql-hackers by date:

Previous
From: Heikki Linnakangas
Date:
Subject: Re: get rid of RM_HEAP2_ID
Next
From: "Matheus Alcantara"
Date:
Subject: postgres_fdw: Use COPY to speed up batch inserts