Re: Use durable_unlink for .ready and .done files for WAL segmentremoval - Mailing list pgsql-hackers

From Michael Paquier
Subject Re: Use durable_unlink for .ready and .done files for WAL segmentremoval
Date
Msg-id 20180928231143.GC1823@paquier.xyz
Whole thread Raw
In response to Re: Use durable_unlink for .ready and .done files for WAL segmentremoval  (Stephen Frost <sfrost@snowman.net>)
Responses Re: Use durable_unlink for .ready and .done files for WAL segmentremoval
List pgsql-hackers
On Fri, Sep 28, 2018 at 02:36:19PM -0400, Stephen Frost wrote:
> Is there an issue with making the archiver able to understand that
> situation instead of being confused by it..?  Seems like that'd probably
> be a good thing to do regardless of this, but that would then remove the
> need for this kind of change..

I thought about that a bit, and there is as well a lot which can be done
within the archive_command itself regarding that, so I am not sure that
there is the argument to make pgarch.c more complicated than it should.
Now it is true that for most users having a .ready file but no segment
would most likely lead in a failure.  I suspect that a large user base
is still just using plain cp in archive_command, which would cause the
archiver to be stuck.  So we could actually just tweak pgarch_readyXlog
to check if the segment fetched actually exists (see bottom of the
so-said function).  If it doesn't, then the archiver removes the .ready
file and retries fetching a new segment.
--
Michael

Attachment

pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: Odd 9.4, 9.3 buildfarm failure on s390x
Next
From: Stephen Frost
Date:
Subject: Re: [PATCH] Include application_name in "connection authorized" logmessage