Re: PostgreSQL data loads - turn off WAL - Mailing list pgsql-general

From Albe Laurenz
Subject Re: PostgreSQL data loads - turn off WAL
Date
Msg-id D960CB61B694CF459DCFB4B0128514C20874BE42@exadv11.host.magwien.gv.at
Whole thread Raw
In response to PostgreSQL data loads - turn off WAL  (hartrc <rhart2@mt.gov>)
List pgsql-general
hartrc wrote:
> Basically my question is:
> Is there currently any way to avoid wal generation during data load
for
> given tables and then have point in time recovery after that?
>
> Background and blurb
> The situation i'm referring to here is for a development environment.
I
> require point in time recovery because if there is crash etc I don't
want to
> lose up to a days work for 12 developers. I'm fairly new to PostgreSQL
so
> please forgive any gaps in my knowledge.
>
> A developer did a data load yesterday of approximately 5GB of data
into a
> new schema. This generated approximately 7GB of wal.  The situation
arises
> where if something is incorrect in the data load the data load may
need to
> be repeated 2 or 3 times (thus generating 20GB +of WAL). For a data
load i
> don't want wal to be generated. I accept the fact before there was
nothing
> and from the point of the next pg_basebackup there was everything. It
is
> from the point i say ok that is everything (the next backup) that i
want
> point in time recovery to apply to that table.
> It is doesn't seem practical, and appears very risky to turn off
wal_archive
> during the data load.

But that's exactly what I'd recommend.

Set archive_mode=off, restart the server, load your data,
set archive_mode=on, restart the server, perform a backup.

If something goes wrong during data load, restore the
previous backup and PITR to the end.

Turning off WAL archiving is no data corruption risk.
The server will still be able to recover from crashes.

Yours,
Laurenz Albe


pgsql-general by date:

Previous
From: Amit Kapila
Date:
Subject: Re: In one of negative test row-level trigger results into loop
Next
From: "Albe Laurenz"
Date:
Subject: Re: Multiple Schema and extensions