Re: Disable WAL logging to speed up data loading - Mailing list pgsql-hackers

From Kyotaro Horiguchi
Subject Re: Disable WAL logging to speed up data loading
Date
Msg-id 20201127.173524.596465781453214648.horikyota.ntt@gmail.com
Whole thread Raw
In response to RE: Disable WAL logging to speed up data loading  ("tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>)
Responses RE: Disable WAL logging to speed up data loading  ("osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>)
List pgsql-hackers
At Fri, 27 Nov 2020 07:01:16 +0000, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com> wrote in 
> From: Masahiko Sawada <sawada.mshk@gmail.com>
> > While testing the patch on some workload, I realized that
> > XLOG_FPI_FOR_HINT record could still be emitted even when wal_level =
> > none. IIUC that WAL record is not necessary during wal_level = none
> > since the server cannot be the primary server and the server crash
> > ends up requiring to restore the whole database.
> 
> Nice catch!  XLOG_FPI_FOR_HINT and XLOG_FPI should be eliminated, otherwise large amount of WAL may be written.  (It
seemsthat other RMIDs than RM_XLOG_ID and RM_XACT_ID do not have to be written.)
 
> 
> I'm afraid "none" doesn't represent the behavior because RM_XLOG_ID and RM_XACT_ID WAL records, except for
XLOG_FPI_*,are emitted.  What's the good name?  IIUC, "minimal" is named after the fact that the minimal amount of WAL
necessaryfor crash recovery is generated.  "norecovery" or "unrecoverable"?
 

I haven't seen a criteria of whether a record is emitted or not for
wal_leve=none.

We're emitting only redo logs.  So I think theoretically we don't need
anything other than the shutdown checkpoint record because we don't
perform recovery and checkpoint record is required at startup.

RM_XLOG_ID:
 XLOG_FPI_FOR_HINT  - not needed?
 XLOG_FPI           - not needed?

 XLOG_CHECKPOINT_SHUTDOWN - must have

So how about the followings?
 XLOG_CHECKPOINT_ONLINE
 XLOG_NOOP
 XLOG_NEXTOID
 XLOG_SWITCH
 XLOG_BACKUP_END
 XLOG_PARAMETER_CHANGE
 XLOG_RESTORE_POINT
 XLOG_FPW_CHANGE
 XLOG_END_OF_RECOVERY


RM_XACT_ID:
 XLOG_XACT_COMMIT
 XLOG_XACT_PREPARE
 XLOG_XACT_ABORT
 XLOG_XACT_COMMIT_PREPARED
 XLOG_XACT_ABORT_PREPARED
 XLOG_XACT_ASSIGNMENT
 XLOG_XACT_INVALIDATIONS

Do we need all of these?

And, currenly what decides whether to emit a wal record according to
wal_level is the caller of XLogInsert. So doing this at
XLogInsert-level means that we bring the criteria of the necessity of
wal-record into xlog layer only for wal_level=none. I'm not sure it is
the right direction.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center



pgsql-hackers by date:

Previous
From: Anastasia Lubennikova
Date:
Subject: Re: [proposal] de-TOAST'ing using a iterator
Next
From: Alexander Korotkov
Date:
Subject: Re: range_agg