RE: Disable WAL logging to speed up data loading - Mailing list pgsql-hackers

From tsunakawa.takay@fujitsu.com
Subject RE: Disable WAL logging to speed up data loading
Date
Msg-id TYAPR01MB2990E9EDA7F5CD0F88959B37FED50@TYAPR01MB2990.jpnprd01.prod.outlook.com
Whole thread Raw
In response to Re: Disable WAL logging to speed up data loading  (Simon Riggs <simon@2ndquadrant.com>)
List pgsql-hackers
From: Simon Riggs <simon@2ndquadrant.com>
> Agreed, it is a footgun. -1 to commit the patch as-is.
> 
> The patch to avoid WAL is simple but it is dangerous for both the user
> and the PostgreSQL project.
> 
> In my experience, people will use this option and when it crashes and
> they lose their data, they will claim PostgreSQL broke and that they
> were not stupid enough to use this option. Data loss has always been
> the most serious error for PostgreSQL and our reputation for
> protecting data has been hard won; it can easily be lost in a moment
> of madness. Please consider how the headlines will read, bearing in
> mind past incidents and negative press. Yes, we did think of this
> feature already and rejected it.

Could you share the negative press that blames Postgres due to the user's misuse of some feature like fsync?


> If we ever did allow such an option, it must contain these things (IMHO):
> * the option should be called "unsafe" or "allows_data_loss", not just
> "none" (anything less than "minimal" must be insufficient or
> unsafe...)

One idea I proposed is "wal_level = unrecoverable" because it clearly states the bad consequence and Oracle also uses
UNRECOVERABLEas a data loading option (and I found it intuitive.)  But others here commented that "none" would be OK.
Idon't have a strong opinion on naming, as I think what's important is warn the user in the documentation.
 


> * the option must be set in the control file and be part of the same
> patch, so users cannot easily edit things to hide their unsafe usage

wal_level setting is stored in the control file as before.  If the server crashes while wal_level is set to none, the
serverrefuses to start saying that's because wal_level is set to none, which is like MySQL.  So, the user cannot hide
theirmisuse.
 


> * we must not support the option of going down to "unsafe" and then
> back up again. It must be a one-way transition from "unsafe" to a
> higher level, so if people want to use this for temp reporting servers
> or initial loading, great, but they can't use it as a quick speed-up
> for databases containing needs-to-be-safe data. Possibly the state
> change might be "unsafe" -> "needs_backup" -> "minimal"... or some
> other way to signal to backup.

I'm afraid I don't get a clear image.  Could you elaborate on that?  But anyway, I think that could be an overreaction
anda prominent caution would suffice (like the one in this patch.)
 


Regards
Takayuki Tsunakawa



pgsql-hackers by date:

Previous
From: "tsunakawa.takay@fujitsu.com"
Date:
Subject: RE: Disable WAL logging to speed up data loading
Next
From: Bruce Momjian
Date:
Subject: Key management with tests