Re: [9.3 bug] disk space in pg_xlog increases during archive recovery - Mailing list pgsql-hackers

From Fujii Masao
Subject Re: [9.3 bug] disk space in pg_xlog increases during archive recovery
Date
Msg-id CAHGQGwG_TK7nS=wSYAjorzH_sJCxnwt_fdPKSLM13QMumyzrJA@mail.gmail.com
Whole thread Raw
In response to [9.3 bug] disk space in pg_xlog increases during archive recovery  ("MauMau" <maumau307@gmail.com>)
Responses Re: [9.3 bug] disk space in pg_xlog increases during archive recovery
Re: [9.3 bug] disk space in pg_xlog increases during archive recovery
List pgsql-hackers
On Sun, Jul 28, 2013 at 7:59 AM, MauMau <maumau307@gmail.com> wrote:
> Hello,
>
> I'm sorry I've been touching several things recently before fixing any of
> them.
>
> I've noticed undesirable disk space increase while performing archive
> recovery with PostgreSQL 9.3.  This happens with 9.2, too.
>
> I just performed archived recovery with the following parameters in
> recovery.conf.  I'm not using replication.
>
> restore_command = 'cp ...'
> recovery_target_timeline = 'latest'
>
> As the archive recovery progresses, the disk space used by $PGDATA/pg_xlog
> increases.  It seems that restored archive WAL files are accumulated there.
> This is considerable amount depending on the number of archived WAL files.
> In my case, the recovery failed because of the shortage of disk space.  This
> did not happen with 9.1
>
> The cause appears to be KeepFileRestoredFromArchive().  This function saves
> restored archive WAL files in pg_xlog/.  I guess this is for cascading
> replication, a new feature added in 9.2.

Yes. Those accumulated WAL files will be removed basically when
restartpoint ends.
But if restartpoint is slow compared to the WAL reply, the problem you described
would happen.

> So, I think it is a bug that the disk space increases if not using cascading
> replication.  Those who migrated from 9.1 and do not use 9.2 features would
> be surprised like me.
>
>
> Do you think this should be fixed?

I think so.

> How should it be fixed?

What about removing the restored archived file as soon as it's replayed
if cascading replication is not enabled (i.e., max_wal_senders = 0 or
hot_standby = off)? This doesn't seem to break the existing behavior
in 9.2.

Regards,

-- 
Fujii Masao



pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])
Next
From: Fujii Masao
Date:
Subject: Re: comment for "fast promote"