Re: robust archiving of WAL segments - Mailing list pgsql-admin

From Drazen Kacar
Subject Re: robust archiving of WAL segments
Date
Msg-id CAFxrd3tbAiZ8ygHDgtA4iA6VtcJbTCnmRRtjq1rXbXQc696fUQ@mail.gmail.com
Whole thread Raw
In response to robust archiving of WAL segments  ("Andrew W. Gibbs" <awgibbs@awgibbs.com>)
List pgsql-admin
On 8 April 2014 15:56, Andrew W. Gibbs <awgibbs@awgibbs.com> wrote:

  * I'm not even sure what a good solution is for guaranteeing that
    transmitted files have been durably persisted using common tools;
    it doesn't seem that commonly available rsync implementations
    support a "please call fsync" option, though some Googling yields
    discussion of a patched version that someone created for such
    purposes; maybe I could invoke the shell command "sync" as part of
    the dance, but that doesn't seem that great either, since the
    first transfer is happening on the master database and I don't
    want to issue a sync request to all file systems as that will kill
    database performance, and the second transfer is happening via
    rsync and you wouldn't be able to call "sync" until "rsync" had
    already deleted the source files, thus creating a race condition

 For copying to the same machine I'm using dd instead of cp because dd can fsync just that one file. Something like this:

dd if="$path" of="$localstore/$file" bs=8192 conv=fsync,excl

I don't have a good solution for copying to the remote site. The only thing that pops to my mind that doesn't require configuration with the C compiler is NFS. It's supposed to use fsync on the remote site, so it should do the trick. Most Linux distributions cheat and don't have NFS configured in this way out of the box, but you should be able to configure it to be reliable.

I've never tried this and don't recommend doing it if you don't know what you're doing.

pgsql-admin by date:

Previous
From: "Andrew W. Gibbs"
Date:
Subject: robust archiving of WAL segments
Next
From: Jason Mathis
Date:
Subject: Re: How to find table creation time