On Sat, Sep 1, 2012 at 2:32 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
> On Fri, Aug 31, 2012 at 5:03 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>> When a cascading standby launches a new walsender, it fetches the current
>> recovery timeline:
>>
>> /*
>> * Use the recovery target timeline ID during recovery
>> */
>> if (am_cascading_walsender)
>> ThisTimeLineID = GetRecoveryTargetTLI();
>>
>> Comment in GetRecoveryTargetTLI() does this:
>>
>> /* RecoveryTargetTLI doesn't change so we need no lock to copy it */
>> return XLogCtl->RecoveryTargetTLI;
>>
>>
>> That comment is not true. RecoveryTargetTLI can change during recovery, if
>> you set recovery_target_timeline='latest'. In 'latest' mode, when the
>> (apparent) end of WAL is reached, the archive is scanned for any new
>> timeline history files that may have appeared. If a new timeline is found,
>> RecoveryTargetTLI is updated, and recovery is continued on the new timeline.
>
> Right. We need lock there for now.
>
>> Aside from the missing locking, I wonder what that does to a cascaded
>> standby. If there is an active walsender running while RecoveryTargetTLI is
>> changed, I think what will happen is that the walsender will continue to
>> stream WAL from the old timeline, but because the startup process is now
>> actually replaying from a different timeline, the walsender will send bogus
>> WAL to the standby.
>
> Good catch! That's really problem. To address that, we should terminate
> all cascading walsenders when the timeline history file is read and
> the recovery target timeline is changed?
This is not right fix. After terminating cascading walsenders, it
might take them
some time to come to an end, and during that time they might send bogus WAL
from old timeline. Currently there is no safeguard against sending bogus WAL
from old timeline. To implement such a safeguard, cascading walsender needs
to know when the timeline is updated and which is the last valid WAL file of
the timeline as the startup process knows. IOW, we need to change cascading
walsenders so that they also read and understand the timeline history files.
This is not easy fix at this stage (9.2.0 is about to be released...).
So, as one idea, I'm thiking to just forbid cascading replication when
recovery_target_timeline is set to 'latest'. Thought?
Regards,
--
Fujii Masao
