Re: Disable WAL logging to speed up data loading - Mailing list pgsql-hackers

From Simon Riggs
Subject Re: Disable WAL logging to speed up data loading
Date
Msg-id CANP8+jJ-pKFtHpkv_W7-=rLU_MO+LRnugHvcPAw-daXTY+OVPg@mail.gmail.com
Whole thread Raw
In response to Re: Disable WAL logging to speed up data loading  (Michael Paquier <michael@paquier.xyz>)
Responses Re: Disable WAL logging to speed up data loading
RE: Disable WAL logging to speed up data loading
List pgsql-hackers
On Fri, 25 Dec 2020 at 07:09, Michael Paquier <michael@paquier.xyz> wrote:
>
> On Thu, Dec 03, 2020 at 03:52:47AM +0000, tsunakawa.takay@fujitsu.com wrote:
> > The code looks good, and the performance seems to be nice, so I
> > marked this ready for committer.
>
> FWIW, I am extremely afraid of this proposal because this is basically
> a footgun able to corrupt customer instances, and I am ready to bet
> that people would switch wal_level to none because they see a lot of
> benefits in doing so at first sight, until the host running the server
> is plugged off and they need to use pg_resetwal in urgency to bring an
> instance online.

Agreed, it is a footgun. -1 to commit the patch as-is.

The patch to avoid WAL is simple but it is dangerous for both the user
and the PostgreSQL project.

In my experience, people will use this option and when it crashes and
they lose their data, they will claim PostgreSQL broke and that they
were not stupid enough to use this option. Data loss has always been
the most serious error for PostgreSQL and our reputation for
protecting data has been hard won; it can easily be lost in a moment
of madness. Please consider how the headlines will read, bearing in
mind past incidents and negative press. Yes, we did think of this
feature already and rejected it.

If we ever did allow such an option, it must contain these things (IMHO):
* the option should be called "unsafe" or "allows_data_loss", not just
"none" (anything less than "minimal" must be insufficient or
unsafe...)
* the option must be set in the control file and be part of the same
patch, so users cannot easily edit things to hide their unsafe usage
* we must not support the option of going down to "unsafe" and then
back up again. It must be a one-way transition from "unsafe" to a
higher level, so if people want to use this for temp reporting servers
or initial loading, great, but they can't use it as a quick speed-up
for databases containing needs-to-be-safe data. Possibly the state
change might be "unsafe" -> "needs_backup" -> "minimal"... or some
other way to signal to backup.

-- 
Simon Riggs                http://www.EnterpriseDB.com/



pgsql-hackers by date:

Previous
From: Fujii Masao
Date:
Subject: Re: postgres_fdw - cached connection leaks if the associated user mapping/foreign server is dropped
Next
From: Masahiko Sawada
Date:
Subject: Re: Rename of triggers for partitioned tables