Thread: Archive directory

2020年12月7日(月) 22:23 Yambu <hyambu@gmail.com>:
>
> Hello
>
> Does postgres have a way of removing wal files stored in archive directory?
> Or do I have to manually clean up this directory?
>
> The directory im talking about is the one on master server configured as
>
> archive_command= cp %p /replication/archive/%f

Is "archive_cleanup_command" what you're looking for?

  https://www.postgresql.org/docs/current/runtime-config-wal.html#GUC-ARCHIVE-CLEANUP-COMMAND

Regards

Ian Barwick


--
EnterpriseDB: https://www.enterprisedb.com

> On Dec 7, 2020, at 6:47 AM, Yambu <hyambu@gmail.com> wrote:
>
> Does the command ,archive_cleanup_command, delete the files from the master server? i understand this config is done
onthe slave server 

The point of the archive command is to copy to the replica. If you're archiving on master, then using some script to
copyto replica, then postgres has no way to know when it is safe to delete WAL files. 

Hi Scott,

> On 07. Dec, 2020, at 14:53, Scott Ribe <scott_ribe@elevated-dev.com> wrote:
>
> The point of the archive command is to copy to the replica. If you're archiving on master, then using some script to
copyto replica, then postgres has no way to know when it is safe to delete WAL files. 

in my understanding, this is only true if you don't use replication slots. If you use replication slots then the
primaryshould know when a WAL file is no longer needed and abandoned or ready to be recycled. 

Cheers,
Paul

hmmm, I disagree with that statement:

The point of the archive_command is to release WALs from the WAL 
directory that have already been applied to the actual data files during 
checkpoints, and send these archived WALs elsewhere, whether it be a 
replica, another location on the master, or to a backup server like 
Barman and pgbackrest.  The ultimate goal is to preserve point in time 
recovery.

So postgres doesn't care where you send the WALs, but that you just did 
something with it so it can remove them from its own internal directory, 
pg_wal/pg_xlog.  So you can even do this: archive_command = '/bin/true' 
which actually does nothing but lets PG know it can remove that WAL from 
its WALDIR.

Regards,
Michael Vitale




Scott Ribe wrote on 12/7/2020 8:53 AM:
>> On Dec 7, 2020, at 6:47 AM, Yambu <hyambu@gmail.com> wrote:
>>
>> Does the command ,archive_cleanup_command, delete the files from the master server? i understand this config is done
onthe slave server
 
> The point of the archive command is to copy to the replica. If you're archiving on master, then using some script to
copyto replica, then postgres has no way to know when it is safe to delete WAL files.
 
>
>
>

> On Dec 7, 2020, at 7:02 AM, Paul Förster <paul.foerster@gmail.com> wrote:
>
> in my understanding, this is only true if you don't use replication slots. If you use replication slots then the
primaryshould know when a WAL file is no longer needed and abandoned or ready to be recycled. 

I don't think OP is talking about the WAL files PG is managing; I think he's talking about WAL files copied into an
archivedirectory. 

Hi Scott, hi Yambu,

> On 07. Dec, 2020, at 15:11, Scott Ribe <scott_ribe@elevated-dev.com> wrote:
>
>> On Dec 7, 2020, at 7:02 AM, Paul Förster <paul.foerster@gmail.com> wrote:
>>
>> in my understanding, this is only true if you don't use replication slots. If you use replication slots then the
primaryshould know when a WAL file is no longer needed and abandoned or ready to be recycled. 
>
> I don't think OP is talking about the WAL files PG is managing; I think he's talking about WAL files copied into an
archivedirectory. 

if those files already archived to some destination are referred to, then yes, you can do with them whatever you want.
Theycan be removed if they are not needed for a PITR. 

For example, we remove them time-based. Files older than a certain number of days are removed (cron job, note: the job
onlyremoves files in the archive destination, NOT in ${PGDATA}/pg_wal). 

Cheers,
Paul