Thread: Estimating WAL usage during pg_basebackup

Estimating WAL usage during pg_basebackup

From

Mike Blackwell

Date:

30 October 2014, 21:52:50

I need to get an idea of how much WAL space will be required during a long (many hours) pg_basebackup over a relatively slow network connection. This is for a server that's not yet running PITR / streaming.

Any thoughts?

Re: Estimating WAL usage during pg_basebackup

From

Soni M

Date:

31 October 2014, 05:22:56

This is hard to tell, but You can get some estimation.

1. You can have WAL rate estimation from pg_xlog/ dir, i.e. How many WAL generated per minutes

2. How long this pg_basebackup will last. Lets say for 3 hours.

Then You can multiple values in #1 and #2 to get rough estimation.

Hope this would help.

On Fri, Oct 31, 2014 at 1:52 AM, Mike Blackwell <mike.blackwell@rrd.com> wrote:

I need to get an idea of how much WAL space will be required during a long (many hours) pg_basebackup over a relatively slow network connection. This is for a server that's not yet running PITR / streaming.

Any thoughts?

Regards,

Soni Maula Harriz

Re: Estimating WAL usage during pg_basebackup

From

Sameer Kumar

Date:

26 November 2014, 05:21:28

On Fri, Oct 31, 2014 at 2:52 AM, Mike Blackwell <mike.blackwell@rrd.com> wrote:

I need to get an idea of how much WAL space will be required during a long (many hours) pg_basebackup over a relatively slow network connection. This is for a server that's not yet running PITR / streaming.

Any thoughts?

You may want to explore

--xlog-method=method

This will enable you to 'fetch' wals at end of backup (which means you should guess wal_keep_segment properly)

Or you can 'stream' them which means WALs will be backed up in parallel. This may introduce contention on network and hence may slow it down but you will not have to guess wal_keep_segment. See if this helps you.