Re: [HACKERS] pg_upgrade to clusters with a different WAL segmentsize - Mailing list pgsql-hackers

From Bossart, Nathan
Subject Re: [HACKERS] pg_upgrade to clusters with a different WAL segmentsize
Date
Msg-id 57923032-5892-4454-BC00-CA69B695EAEE@amazon.com
Whole thread Raw
In response to Re: [HACKERS] pg_upgrade to clusters with a different WAL segment size  (Robert Haas <robertmhaas@gmail.com>)
Responses Re: [HACKERS] pg_upgrade to clusters with a different WAL segmentsize  (Peter Eisentraut <peter.eisentraut@2ndquadrant.com>)
List pgsql-hackers
Apologies for the delay.

Here is a patch to allow users to change the WAL segment size of a cluster with
pg_resetwal.  Like the '--wal-segize' option in initdb, the new '-s' option
accepts segment sizes in megabytes.  This patch also fixes two small issues that
I noticed.

The first is a division-by-zero error caused when the control data must be
guessed.  If the control data is damaged, WalSegSz will not be set to the
guessed WAL segment size, resulting in an error during XLogFromFileName().  The
attached patch resolves this issue by ensuring WalSegSz is set to a valid value
after either reading or guessing the control values.

The second issue is with the automatically determined new WAL starting address
calculation.  Currently, the starting address is determined by looking at the
last WAL file name, rounding the WAL byte position up to the next segment of the
new size, and then incrementing to the next WAL segment for the new size.  I
believe this can cause the WAL byte position to go backwards in some cases (e.g.
WAL position is towards the end of a 1024 MB segment and pg_resetwal is used to
switch the segment size to 1 MB).  Since the "current" WAL byte position
calculated in pg_resetwal is always the beginning of the latest log, the current
strategy may not yield a byte position sufficiently far ahead.  The attached
patch changes this logic to determine the last byte position in the latest log
(for the old WAL segment size) prior to aligning to the new segment size and
incrementing.

Note that some segment size changes will cause old WAL file names to be reused.
The new documentation for '-s' contains a note warning of this side effect.  I
considered a couple of strategies to prevent this, but I chose not to include
any in the patch based on previous correspondence in this thread.  If file name
overlap is an issue, users can already use the '-l' option to set a safe WAL
starting address.

Nathan


Attachment

pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: [HACKERS] Transactions involving multiple postgres foreign servers
Next
From: Andres Freund
Date:
Subject: Re: [HACKERS] [POC] Faster processing at Gather node