Re: Enable WAL archiving even in standby - Mailing list pgsql-hackers

On Tue, Aug 19, 2014 at 7:33 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
> On Fri, Aug 15, 2014 at 4:30 AM, Robert Haas <robertmhaas@gmail.com> wrote:
>> On Wed, Aug 13, 2014 at 6:42 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
>>> I'd propose the attached WIP patch which allows us to enable WAL archiving
>>> even in standby. The patch adds "always" as the valid value of archive_mode.
>>> If it's set to "always", the archiver is started when the server is in standby
>>> mode and all the WAL files that walreceiver wrote to the disk are archived by
>>> using archive_command. Then, even after the server is promoted to master,
>>> the archiver keeps archiving WAL files. The patch doesn't change the meanings
>>> of the setting values "on" and "off" of archive_mode.
>>
>> I like the feature, but I don't much like this as a control mechanism.
>> Having archive_command and standby_archive_command, as you propose
>> further down, seems saner.
>
> Okay, that's fine. One question is; Which WAL files should be archived by
> standby_archive_command? There are following kinds of WAL files.
>
> (1) WAL files which were fully written and closed by walreceiver
>      Curently they are not archived at all.
>
> (2) WAL file which is being written by walreceiver
>      This file will be closed before it's fully written because of,
>      for example, standby promotion.
>      Currently this is archived by archive_command.
>
> (3) WAL file with new timeline, which is copied from (2)
>       At the end of recovery, after new timeline is assigned,
>       this latest WAL file with new timeline is created by being copied
>       from (2) (i.e., latest WAL file with old timeline). WAL data of
>       end-of-recovery checkpoint is written to this latest WAL file.
>       Currently this is archived by archive_command.
>
> (4) Timeline history files
>      When standby is promoted to the master, the imeline is incremented
>      and the timeline history file is created.
>      Currently the timeline history files are archived by archive_command.
>
> (5) WAL files generated in normal processing mode
>       Currently they are archived by archive_command.
>
> I'm thinking to use standby_archive_command only for (1) because
> the others are currently archived by archive_command. That means
> that even if there are type (1) WAL files which have not been archived
> yet after the standby promotion (i.e., the situation where WAL archiving
> was delayed for some reasons in the standby), they are archived by
> standby_archive_command. IOW, the archiver uses both archive commands
> as the situation demands.
>
> OTOH, maybe there are people who want to use standby_archive_command
> for all the WAL files with old timeline, i.e., (1) and (2). Thought?

Boy, that's quite confusing.  I didn't think we ever ran
archive_command on the standby right now, so then it would make sense
to have a way to do that.  And it makes sense for it to be separate
from the mode used on the master to avoid breaking existing
configurations, so that a user assuming that a certain setting will
only take effect after promotion will not be disappointed.  However,
if what you're saying is that we do archiving on the standby sometimes
but not others, I'm not quite sure what the best way forward is.  It
seems rather inconsistent to do it for some types of WAL files but not
others.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company