RE: Replication: slave server has 3x size of production server? - Mailing list pgsql-general

From Edson Richter
Subject RE: Replication: slave server has 3x size of production server?
Date
Msg-id MN2PR01MB53276044B68C6DDD848E4BE1CFEF0@MN2PR01MB5327.prod.exchangelabs.com
Whole thread Raw
In response to Re: Replication: slave server has 3x size of production server?  (Adrian Klaver <adrian.klaver@aklaver.com>)
List pgsql-general


De: Adrian Klaver <adrian.klaver@aklaver.com>
Enviado: domingo, 23 de fevereiro de 2020 15:42
Para: Edson Richter <edsonrichter@hotmail.com>; pgsql-general <pgsql-general@postgresql.org>
Assunto: Re: Replication: slave server has 3x size of production server?
 
On 2/23/20 8:04 AM, Edson Richter wrote:
>     ------------------------------------------------------------------------
>
>     *De:* Adrian Klaver <adrian.klaver@aklaver.com>
>     *Enviado:* sábado, 22 de fevereiro de 2020 20:34
>     *Para:* Edson Richter <edsonrichter@hotmail.com>; pgsql-general
>     <pgsql-general@postgresql.org>
>     *Assunto:* Re: Replication: slave server has 3x size of production
>     server?
>     On 2/22/20 2:51 PM, Edson Richter wrote:
>
>     >
>     > Yes, it is working. Last X'log file is present on all thee servers.
>     > Also, comparting last transaction number on master and slave shows that
>     > all are in sync.
>     > Last, but not least, select max(id) from a busy table shows same id
>     > (when queried almost simultaneously using a simple test routine).
>
>     Well something is keeping those WAL file around. You probably should
>     analyze your complete setup to see what else is touching those servers.
>
>
> It is safe to add a "--remove-source-files" into my archive_command as
> folows into my slave server?

I would say not. See:

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.postgresql.org%2Fdocs%2F12%2Fwal-configuration.html&amp;data=02%7C01%7C%7Cb49e9c01f11a4b9fe4d108d7b8902bd2%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637180801653706393&amp;sdata=QY24D6R%2B%2BJ7VgnctERdK964reKEp7XbxERiXGC2XL8Y%3D&amp;reserved=0

"Checkpoints are points in the sequence of transactions at which it is
guaranteed that the heap and index data files have been updated with all
information written before that checkpoint. At checkpoint time, all
dirty data pages are flushed to disk and a special checkpoint record is
written to the log file. (The change records were previously flushed to
the WAL files.) In the event of a crash, the crash recovery procedure
looks at the latest checkpoint record to determine the point in the log
(known as the redo record) from which it should start the REDO
operation. Any changes made to data files before that point are
guaranteed to be already on disk. Hence, after a checkpoint, log
segments preceding the one containing the redo record are no longer
needed and can be recycled or removed. (When WAL archiving is being
done, the log segments must be archived before being recycled or removed.)"

So there is a window where a WAL is written but before the data it
represents is check pointed, so it still needed.

I see. Makes sense.
I suppose that long lifed xlog files are of no use then... I would expect PostgreSQL delete them automatically.
Perhaps, since I have full backups happening every odd days, I can create a "post backup command" in barman script so it will delete files above 1 week from the server it is backup up from...
I understand there is no guarantee that these files have already been processed... but if they are needed, they can be recovered from the barman server...

Thanks,

Edson

>
>
> archive_command = 'rsync --remove-source-files -e "ssh -2 -C -p 2022"
> -az %p barman@192.168.0.2:/dados/barman/dbcluster/incoming/%f'
>
>
> and remove the xlog file after copy to barman?
> I mean, whem the archive command starts, the wal has been already
> processed by the slave server, so we don't need them after copying to
> backup server, right?
>
>
> Regards,
>
> Edson
>
>     >
>     >     >
>     >     >
>     >     > The files are about 7 months old.
>     >
>     >     Are there newer files that would indicate that the streaming is working?
>     >
>     >
>     > Yes, streaming is working properly (as stated above).
>     >
>     > Thanks,
>     >
>     >
>     > Edson Richter
>     >
>     >
>     >>
>
>
>
>     --
>     Adrian Klaver
>     adrian.klaver@aklaver.com
>


--
Adrian Klaver
adrian.klaver@aklaver.com

pgsql-general by date:

Previous
From: Adrian Klaver
Date:
Subject: Re: Reset DB stats suggestion pg_stat_reset()
Next
From: "sivapostgres@yahoo.com"
Date:
Subject: Re: Can I trigger an action from a coalesce ?