Re: pg_basebackup and pg_receivewal timing, missing WAL files - Mailing list pgsql-admin

From Stephen Frost
Subject Re: pg_basebackup and pg_receivewal timing, missing WAL files
Date
Msg-id 20220610163325.GW9030@tamriel.snowman.net
Whole thread Raw
In response to Re: pg_basebackup and pg_receivewal timing, missing WAL files  (Tim <timfosho@gmail.com>)
List pgsql-admin
Greetings,

* Tim (timfosho@gmail.com) wrote:
> > However, you can use repo-retention-archive and
> > repo-retention-archive-type to change this behavior, see
> >
> > https://pgbackrest.org/configuration.html#section-repository/option-repo-retention-archive
> > .
> > Basically, if you need to keep a number of full backups for compliance
> > you can set repo-retention-archive-type=full and
> > repo-retention-archive-type=1 in that repo and WAL in between the
> > backups will only be kept after the last backup. WAL required to make
> > backups consistent is always kept until the backup expires.
>
> So as I understand it, it would keep WALs from the most current backup up
> to the most current WALs.
> In my case, the only purpose of this 2nd repo is to keep monthly backups
> that are 2-6 months old (and occasionally test restores)
> for legal purposes.
> The  most recent backup in this repo would still be 2 months old, so it
> would be keeping an unnecessary 2 months of WALs.

You don't need to have the second repo be enabled all the time, just
enable it when you do the next full backup into the new repo, and
disable it afterwards, so you don't continue to push WAL to it.

> We've had requests for repo-retention-archive-type=0, i.e. only archive
> > WAL during a backup, but we have not had time to implement that. Some
> > users have implemented a work-around based on checking the backup lock.
> > There is an issue that describes the method but I was not able to find
> > it after a brief search. This is safer than it sounds because pgbackrest
> > will check that all WAL required to make the backup consistent reached
> > the archive. So, if the wrapper is broken you'll get errors.
>
> I'm guessing you're referring to: optional wal archiving · Issue #900 ·
> pgbackrest/pgbackrest (github.com)
> <https://github.com/pgbackrest/pgbackrest/issues/900>,

This was from before we had multi-repo support and I'd think simply
enabling/disabling that second repo would be the simpler approach.

I see that it sounds like you're going to take a look at trying that
approach- would be great to get your feedback on how hard it was and how
well it did or didn't work out.

Thanks,

Stephen

Attachment

pgsql-admin by date:

Previous
From: Tom Lane
Date:
Subject: Re: ERROR: type "my_user_type" does not exist on REFRESH MATERIALIZED VIEW
Next
From: Teja Jakkidi
Date:
Subject: N_live_tup value is less than actual row count in a table