Thread: Re: [COMMITTERS] pgsql: At promotion, archive last segment from old timeline with .parti

On 05/22/2015 12:35 PM, Fujii Masao wrote:
> Doesn't this change break the case where we want to PITR to the recovery
> target location in the last partial WAL file with the old timeline?
> In this case, that partial WAL file needs to be read and replayed. But
> since the suffix of its filename is .partial, unless DBA gets rid of the suffix,
> the WAL file cannot be restored and PITR would fail. No?

PITR to a specific location always requires manual intervention by the 
DBA anyway. It's not something you'd automate. Copying the .partial file 
manually into pg_xlog is just one small extra step.

Even if there are some downsides to this, I think it's just plain evil 
to archive a partial segment that looks indistinguishable from a 
complete one. We have had reports of that causing confusion in 
production systems. What if the master had already archived the complete 
version of the segment before dying? The standby will try to archive a 
partial version of the same, which will fail, or worse, overwrite the 
complete version with the partial one.

Note that PITR in that scenario was always hit-and-miss. First of all, 
if the master died, there is no guarantee that it archived all the 
previous segments successfully before dying. (archive_mode=always 
alleviates that in 9.5, as the standby will archive them even if the 
master didn't).

(See discussion on this point at 
http://www.postgresql.org/message-id/5535FE71.1010905@iki.fi)

- Heikki

On Fri, May 22, 2015 at 6:59 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
> On 05/22/2015 12:35 PM, Fujii Masao wrote:
>>
>> Doesn't this change break the case where we want to PITR to the recovery
>> target location in the last partial WAL file with the old timeline?
>> In this case, that partial WAL file needs to be read and replayed. But
>> since the suffix of its filename is .partial, unless DBA gets rid of the
>> suffix,
>> the WAL file cannot be restored and PITR would fail. No?
>
>
> PITR to a specific location always requires manual intervention by the DBA
> anyway. It's not something you'd automate. Copying the .partial file
> manually into pg_xlog is just one small extra step.

We should document this? Otherwise no DBA can complete such PITR scenario.

Also as a safeguard, if the required WAL file is not found but the file with
.parital suffix found during recovery, maybe we should cause the recovery to
fail at that moment. Otherwise since the required file is not found, the server
would end the recovery before the .partial file and start normal processing.
DBA may not be able to notice this incompletion of the recovery.
Maybe this is overkill against the small use case, though.

> Even if there are some downsides to this, I think it's just plain evil to
> archive a partial segment that looks indistinguishable from a complete one.
> We have had reports of that causing confusion in production systems. What if
> the master had already archived the complete version of the segment before
> dying? The standby will try to archive a partial version of the same, which
> will fail, or worse, overwrite the complete version with the partial one.
>
> Note that PITR in that scenario was always hit-and-miss. First of all, if
> the master died, there is no guarantee that it archived all the previous
> segments successfully before dying. (archive_mode=always alleviates that in
> 9.5, as the standby will archive them even if the master didn't).

So we don't need to rename the last WAL file basically in archive recovery
case even if it's read from pg_xlog, i.e., it's partial. No?

Regards,

-- 
Fujii Masao