RE: Disable WAL logging to speed up data loading - Mailing list pgsql-hackers

From tsunakawa.takay@fujitsu.com
Subject RE: Disable WAL logging to speed up data loading
Date
Msg-id TYAPR01MB2990128F309EF1CBD16EFB4DFEE20@TYAPR01MB2990.jpnprd01.prod.outlook.com
Whole thread Raw
In response to Re: Disable WAL logging to speed up data loading  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
From: Robert Haas <robertmhaas@gmail.com>
> I'm also concerned about the way that this proposed feature interacts with
> incremental backup capabilities that already exist in tools like pgBackRest,
> EDB's BART, pg_probackup, and future things we might want to introduce into
> core, along the lines of what I have previously proposed. Now, I think
> pgBackRest uses only timestamps and checksums, so it probably doesn't care,
> but some of the other solutions rely on WAL-scanning to gather a list of
> changed blocks. I guess there's no reason that they can't notice the wal_level
> being changed and do the right thing; they should probably have that kind of
> capability already. Still, it strikes me that it might be useful if we had a stronger
> mechanism.

Having a quick look, those backup tools seem to require setting wal_level to replica or higher.  That's no wonder,
becauserecovering the database needs WAL for non-relation resources such as pg_control and relation map.  So, I think
wal_level= none won't introduce new issues (compared to wal_level = minimal, which also can lack WAL records for some
dataupdates.)
 


> By the way, another problem here is that some AMs - e.g. GiST, IIRC - use LSNs
> to figure out whether a block has changed. For temporary and unlogged tables,
> we use "fake" LSNs that are generated using a counter, but that approach only
> works because such relations are never really WAL-logged. Mixing fake LSNs
> and real LSNs will break stuff, and not bumping the LSN when the page
> changes probably will, too.

Unlogged GiST indexes use fake LSNs that are instance-wide.  Unlogged temporary GiST indexes use backend-local sequence
values. Other unlogged and temporary relations don't set LSNs on pages.  So, I think it's enough to call
GetFakeLSNForUnloggedRel()when wal_level = none as well.
 


Regards
Takayuki Tsunakawa


pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: Skip ExecCheckRTPerms in CTAS with no data
Next
From: Michael Paquier
Date:
Subject: Re: Tab complete for CREATE OR REPLACE TRIGGER statement