Thread: How to clone a running master cluster?

How to clone a running master cluster?

From
Moshe Jacobson
Date:
I have a master database cluster on one server, and it is configured to ship logs via scp to an archive directory on my slave server. The slave server is configured for streaming replication, and also is configured to delete the archived xlogs when they are no longer needed (using pg_archivecleanup).

I have a third machine on which I'd like to get another master cluster running, and I'd like it to start with a copy of my current master. I'd cannot restart my master, and would prefer not to restart my slave either.

Given my xlog archive configuration, Is there a way to clone my master cluster to another machine, including all of the necessary xlogs, without bringing down the original master or slave? Step-by-step instructions would be much appreciated.

Thank you!

--
Moshe Jacobson
Nead Werx, Inc. | Manager of Systems Engineering
2323 Cumberland Parkway, Suite 201 | Atlanta, GA 30339
moshe@neadwerx.com | 
www.neadwerx.com

"Quality is not an act, it is a habit." -- Aristotle

Re: How to clone a running master cluster?

From
Michael Nolan
Date:
On 5/11/13, Moshe Jacobson <moshe@neadwerx.com> wrote:
> I have a master database cluster on one server, and it is configured to
> ship logs via scp to an archive directory on my slave server. The slave
> server is configured for streaming replication, and also is configured to
> delete the archived xlogs when they are no longer needed (using
> pg_archivecleanup).
>
> I have a third machine on which I'd like to get another master cluster
> running, and I'd like it to start with a copy of my current master. I'd
> cannot restart my master, and would prefer not to restart my slave either.
>
> Given my xlog archive configuration, Is there a way to clone my master
> cluster to another machine, including all of the necessary xlogs, without
> bringing down the original master or slave? Step-by-step instructions would
> be much appreciated.
>
> Thank you!
>
> --
> Moshe Jacobson
> Nead Werx, Inc. | Manager of Systems Engineering
> 2323 Cumberland Parkway, Suite 201 | Atlanta, GA 30339
> moshe@neadwerx.com | www.neadwerx.com
>
> "Quality is not an act, it is a habit." -- Aristotle

>

Moshe, if you need a detailed cookbook tailored to your specific
requirements, you may need to hire a PostgreSQL expert as a consultant
to write it for you. Generalized guidelines can't possibly cover every
possible situation.

The Binary Replication Tutorial at
http://wiki.postgresql.org/wiki/Binary_Replication_Tutorial (which is
slightly out of date as it does not cover 9.2 and 9.3 improvements
yet) is probably going to cover most of what you need.  The 'long
method' is going to be pretty much what you need, you will still need
to do a pg_start_backup() and pg_stop_backup() while you copy the data
directory files, but you probably won't need to restart the master to
change the master configuration files since you've already got
replication working to one server and you're apparently not planning
to have the second slave server poll the master for updates.
--
Mike Nolan


Re: How to clone a running master cluster?

From
Jeff Janes
Date:
On Sat, May 11, 2013 at 7:56 AM, Moshe Jacobson <moshe@neadwerx.com> wrote:
I have a master database cluster on one server, and it is configured to ship logs via scp to an archive directory on my slave server. The slave server is configured for streaming replication, and also is configured to delete the archived xlogs when they are no longer needed (using pg_archivecleanup).

I have a third machine on which I'd like to get another master cluster running, and I'd like it to start with a copy of my current master. I'd cannot restart my master, and would prefer not to restart my slave either.

Given my xlog archive configuration, Is there a way to clone my master cluster to another machine, including all of the necessary xlogs, without bringing down the original master or slave?

I'd probably use "pg_basebackup --xlog-method=stream ..."

Or temporarily disable pg_archivecleanup.

Cheers,

Jeff

Re: How to clone a running master cluster?

From
Moshe Jacobson
Date:
That option doesn't seem to exist, but pg_basebackup does seem to be the way to go.
Thanks for the help, Jeff!


On Sat, May 11, 2013 at 2:15 PM, Jeff Janes <jeff.janes@gmail.com> wrote:
On Sat, May 11, 2013 at 7:56 AM, Moshe Jacobson <moshe@neadwerx.com> wrote:
I have a master database cluster on one server, and it is configured to ship logs via scp to an archive directory on my slave server. The slave server is configured for streaming replication, and also is configured to delete the archived xlogs when they are no longer needed (using pg_archivecleanup).

I have a third machine on which I'd like to get another master cluster running, and I'd like it to start with a copy of my current master. I'd cannot restart my master, and would prefer not to restart my slave either.

Given my xlog archive configuration, Is there a way to clone my master cluster to another machine, including all of the necessary xlogs, without bringing down the original master or slave?

I'd probably use "pg_basebackup --xlog-method=stream ..."

Or temporarily disable pg_archivecleanup.

Cheers,

Jeff



--
Moshe Jacobson
Nead Werx, Inc. | Manager of Systems Engineering
2323 Cumberland Parkway, Suite 201 | Atlanta, GA 30339
moshe@neadwerx.com | 
www.neadwerx.com

"Quality is not an act, it is a habit." -- Aristotle

Re: How to clone a running master cluster?

From
Jeff Janes
Date:
On Sun, May 12, 2013 at 5:54 AM, Moshe Jacobson <moshe@neadwerx.com> wrote:
That option doesn't seem to exist, but pg_basebackup does seem to be the way to go.
Thanks for the help, Jeff!

--xlog-method was add in 9.2.

Cheers,

Jeff