Re: Streaming replication with sync slave, but disconnects due to missing WAL segments - Mailing list pgsql-general

From Shaun Thomas
Subject Re: Streaming replication with sync slave, but disconnects due to missing WAL segments
Date
Msg-id 51B08DD1.4030009@optionshouse.com
Whole thread Raw
In response to Streaming replication with sync slave, but disconnects due to missing WAL segments  (Mads.Tandrup@schneider-electric.com)
List pgsql-general
On 06/04/2013 08:25 AM, Mads.Tandrup@schneider-electric.com wrote:

> It all seems to work fine. But I have noticed that sometimes when I
> restore backups created by pg_dump. The slave node will disconnect
> with the message in the postgresql log:

You need to increase wal_keep_segments on the master. Any pg_dump or
pg_restore is one giant transaction, and the dirty little secret about
PG replication, is that no slave can replay while a transaction is in
progress.

The slave keeps waiting while the master is doing all this extra work,
but without enough segments in reserve, the master may archive or delete
what it considers "excess" segments before slaves can receive or process
them.

One way to avoid keeping these around is to set your archive_command to
transmit archived WAL files to a location where the slave can read them.
Then in your recovery.conf, you can make it look something like this:

standby_mode     = 'on'
primary_conninfo = 'host=master-node user=replication'
trigger_file     = '/db/pgdata/promote'
restore_command  = 'cp -f /db/wal/pg_archived/%f %p'

So if your connection gets disrupted, or the slave falls behind, it can
always catch up again.

Honestly in sync rep, I'm surprised the master doesn't keep segments
until a waiting slave either disconnects, or can consume WAL files being
held up by a long-running transaction. Not that it matters, since you
can fake that behavior this way.

--
Shaun Thomas
OptionsHouse | 141 W. Jackson Blvd. | Suite 500 | Chicago IL, 60604
312-676-8870
sthomas@optionshouse.com

______________________________________________

See http://www.peak6.com/email_disclaimer/ for terms and conditions related to this email


pgsql-general by date:

Previous
From: Richard Huxton
Date:
Subject: Re: PostgreSQL Synchronous Replication in production
Next
From: Ioana Danes
Date:
Subject: Re: Function use in query