Re: Disable WAL logging to speed up data loading - Mailing list pgsql-hackers

From Kyotaro Horiguchi
Subject Re: Disable WAL logging to speed up data loading
Date
Msg-id 20210113.171255.1152741719204412097.horikyota.ntt@gmail.com
Whole thread Raw
In response to RE: Disable WAL logging to speed up data loading  ("tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>)
Responses RE: Disable WAL logging to speed up data loading  ("tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>)
List pgsql-hackers
At Wed, 13 Jan 2021 06:01:34 +0000, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com> wrote in 
> From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
> > XLogBeginInsert();
> > XLogSetRecrodFlags(XLOG_MARK_ESSENTIAL);  # new flag value
> > XLOGInsert(....);
> 
> Oh, sounds like a nice idea.  That's more flexible by allowing WAL-emitting modules to specify which WAL records are
mandatoryeven when wal_level is none.
 
> 
> For example, gistXLogAssignLSN() adds the above flag like this:
> 
>     XLogBeginInsert();
>     XLogSetRecordFlags(XLOG_MARK_UNIMPORTANT | XLOG_MARK_ESSENTIAL);
>     XLogRegisterData((char *) &dummy, sizeof(dummy));
> 
> (Here's a word play - unimportant but essential, what's that?)

Hmm. Food may not be important to someone but it is essential for
survival.  I think this is somethig like that :p

> And the filter in XLogInsert() becomes:
> 
> +    if (wal_level == WAL_LEVEL_NONE &&
> +        !((rmid == RM_XLOG_ID && info == XLOG_CHECKPOINT_SHUTDOWN) ||
> +          (rmid == RM_XLOG_ID && info == XLOG_PARAMETER_CHANGE) ||
> +          (rmid == RM_XACT_ID && info == XLOG_XACT_PREPARE) ||
> +          (curinsert_flags & XLOG_MARK_ESSENTIAL)))
> 
> Or,
> 
> +    if (wal_level == WAL_LEVEL_NONE &&
> +         !(curinsert_flags & XLOG_MARK_ESSENTIAL))
>
> and add the new flag when emitting XLOG_CHECKPOINT_ONLINE,

Yeah, I was going to comment about that.  Skipping the record makes
controlfile filled by a bogus checkpoint location.

> XLOG_PARAMETER_CHANGE and XLOG_PREPARE records.  I think both have
> good reasons: the former centralizes the handling of XACT and XLOG
> RM WAL records (as the current XLOG module does already), and the
> latter delegates the decision to each module.  Which would you
> prefer?  (I kind of like the former, but this is a weak opinion.)

Unfortunately, I prefer the latter as it is simple because it is in a
hot path.  As I think I mentioned upthread, I think the xlog stuff
should refrain from being conscious of things deeper than RMger-ID
level.  One of other reasons is that generally the issuer site is the
authoritative about the importance and essentiality of a record being
issued.  And I don't think it's very good to do the same thing in
different ways at the same time.  Fortunately each type of the recrods
has only few issuer places.

By the way, I noticed that pg_switch_wal() silently skips its
task. Desn't it need to give any sort of ERROR?

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center



pgsql-hackers by date:

Previous
From: "Hou, Zhijie"
Date:
Subject: RE: Single transaction in the tablesync worker?
Next
From: Peter Eisentraut
Date:
Subject: Re: Improper use about DatumGetInt32