Don't keep closed WAL segment in page cache after replay - Mailing list pgsql-hackers

From Anthonin Bonnefoy
Subject Don't keep closed WAL segment in page cache after replay
Date
Msg-id CAO6_XqriMRHs0ma1oLYhudH6cYyjz_8v7Q2M83XZPdLv7=zW4g@mail.gmail.com
Whole thread Raw
Responses Re: Don't keep closed WAL segment in page cache after replay
List pgsql-hackers
Hi,

I've been looking at page cache usage as some of our replicas were
under memory pressure (no inactive pages available) which led to WAL
replay lag as the recovery process had to read from disk. One thing
I've noticed was that the last WAL files are in the pagecache even
after having been replayed.

This can be checked with vmtouch:
vmtouch  pg_wal/*
           Files: 141
     Directories: 2
  Resident Pages: 290816/290816  1G/1G  100%

And page-types shows a replayed WAL file in the active LRU:
page-types  -Cl -f 000000010000001B00000076
page-count       MB  long-symbolic-flags
      4096       16  referenced,uptodate,lru,active

From my understanding, once replayed on a replica, WAL segment files
won't be re-read. So keeping it in the pagecache seems like an
unnecessary strain on the memory (more so that they appear to be in
the active LRU).

This patch adds a POSIX_FADV_DONTNEED before closing a WAL segment,
immediately releasing cached pages. With this, the page cache usage of
pg_wal stays under the wal_segment_size:

vmtouch  pg_wal/*
           Files: 88
     Directories: 2
  Resident Pages: 3220/262144  12M/1G  1.23%

Regards,
Anthonin

Attachment

pgsql-hackers by date:

Previous
From: Etsuro Fujita
Date:
Subject: Re: Problem with transition tables on partitioned tables with foreign-table partitions
Next
From: Peter Eisentraut
Date:
Subject: Re: Replace magic numbers with strategy numbers for B-tree indexes