Re: BUG #19032: In restore_command %f parameter does not support WAL partial files. - Mailing list pgsql-bugs

From Wojciech Szenajch
Subject Re: BUG #19032: In restore_command %f parameter does not support WAL partial files.
Date
Msg-id CAEcP0DH7Z2n0N_jzum1rNEaOjSi4P3qw4baaW4qZaoyrCHvAkg@mail.gmail.com
Whole thread Raw
In response to Re: BUG #19032: In restore_command %f parameter does not support WAL partial files.  ("David G. Johnston" <david.g.johnston@gmail.com>)
List pgsql-bugs
I used pg_basebackup for full backups. pg_receivewal for collecting latest WALs from Primary Server.
The scenario I described was for full database restoration in a situation of complete Primary database server unrecoverable crash.
So, it is not the situation when orignial Primary may return back.
I did not use archive_command, because it seemed to be obsolete when pg_receivewal is is used.

In my procedures there is nothing different from standard usage described in Postgres manual.

"pg_receivewal creates files with ".partial" extensions that are semantically different from what the core system ".partial" files" - does it mean that files are useless for database restoring from backup? 

From pg_receivewal documentation:
"pg_receivewal streams the write-ahead log in real time as it's being generated on the server, and does not wait for segments to complete like archive_command and archive_library do. For this reason, it is not necessary to set archive_timeout when using pg_receivewal." - What is it for? Especially with this "archive_timeout" and " does not wait for segments to complete" remarks.

"When using pg_receivewal instead of archive_command or archive_library as the main WAL backup method, it is strongly recommended to use replication slots" and "The archive_command or archive_library is only invoked for completed WAL segments."
In a case of server crash usable WAL part file is not accessible (putting pg_wal directory on i.e. NFS Server is not a good idea in my opinion).

It seems that advantage of Postgres database with almost continuous WAL backup compared to some commercial databases when equivalent to archive_timeout must be set to i.e. 10 minutes and end up  with a lot of "empty" WALs i.e. during weekends is not true.

It is a pity, because Postgres has all tools to have this advantage, but they must be properly modified. Or there is something I do not understand and this functionality is somehow available.

Wojciech S.

śr., 27 sie 2025 o 02:48 David G. Johnston <david.g.johnston@gmail.com> napisał(a):
On Tue, Aug 26, 2025 at 5:10 PM PG Bug reporting form <noreply@postgresql.org> wrote:
The following bug has been logged on the website:

Bug reference:      19032
Logged by:          Wojciech Szenajch
Email address:      wszenajch@gmail.com
PostgreSQL version: 17.6
Operating system:   Ubuntu 24.04.03 LTS
Description:       

Using restore_command = 'cp /tmp/receivewal/%f %p' does not use WAL partial
file during backup restore process.

While this seems un-documented in the user-facing documentation the source code makes it clear this is intended behavior.  If your point-in-time-recovery flow requires the use of these files you will need to manually manipulate them.


(link to article about a custom archiving process relying on pg_receivewal)

It seems unfortunate that the pg_receivewal creates files with ".partial" extensions that are semantically different from what the core system ".partial" files are.  But if your archive process is custom (i.e., uses something other than archive_command) it seems reasonable that any restore_command you need to create would also be something custom.  Going by the documentation for only half of the process is going to lead to these kinds of inconsistent results.

David J.

pgsql-bugs by date:

Previous
From: Arseniy Mukhin
Date:
Subject: Re: BUG #19031: pg_trgm infinite loop on certain cases
Next
From: "Hayato Kuroda (Fujitsu)"
Date:
Subject: RE: [CAUTION: SUSPECT SENDER] RE: [CAUTION: SUSPECT SENDER] RE: [CAUTION: SUSPECT SENDER] RE: BUG #19029: Replication Slot size keeps increasing while logical subscription works fine