Re: Reliable WAL file shipping over unreliable network - Mailing list pgsql-admin

From Rui DeSousa
Subject Re: Reliable WAL file shipping over unreliable network
Date
Msg-id E6AB850C-D05E-405B-8D4E-DE18E128F402@icloud.com
Whole thread Raw
In response to Re: Reliable WAL file shipping over unreliable network  (scott ribe <scott_ribe@elevated-dev.com>)
Responses Re: Reliable WAL file shipping over unreliable network  (Mark Kirkwood <mark.kirkwood@catalyst.net.nz>)
Re: Reliable WAL file shipping over unreliable network  (Stephen Frost <sfrost@snowman.net>)
List pgsql-admin


On Mar 1, 2018, at 12:21 AM, scott ribe <scott_ribe@elevated-dev.com> wrote:

The false report of success is not good, but it's not the root problem.

A false success if a problem; especially in this use case as the source WAL file will be deleted by Postgres before it was truly successful.  While monitoring is nice to avoid the issue it is not a fix for the issue.

I personally cannot recommend the use of rsync in this application for two reasons.  

1. It adds no value; it’s a more complex cp command (no bandwidth saved, etc as archive processes a single file at a time).
2. It lies on success/failure — Period.


I have use “cat” longer than I have used rsync to archive WALs.  I can say that I’ve lost zero WAL files using cat; I can not say the same for rsync.

The following code is more reliable than rsync and works with across multiple platforms and filesystems without fail.  

STS=3

OUTPUT=$(cat $XLOGFILE | $SSH_CMD "(mkdir -p $ARCH_DIR && cat > $ARCH_DIR/$WALFILE.swap) && mv $ARCH_DIR/$WALFILE.swap $ARCH_DIR/$WALFILE")
if [ $? == 0 ]; then 
   STS=0
fi

exit $STS


pgsql-admin by date:

Previous
From: Ibrahim Edib Kokdemir
Date:
Subject: Re: Separate log for a specfic database
Next
From: MichaelDBA
Date:
Subject: Re: Postgresql | Vacuum information