Re: Disable WAL logging to speed up data loading - Mailing list pgsql-hackers

At Fri, 27 Nov 2020 13:34:49 +0000, "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com> wrote in 
> Thank you, Horiguchi-San
> 
> > I haven't seen a criteria of whether a record is emitted or not for
> > wal_leve=none.
> > 
> > We're emitting only redo logs.  So I think theoretically we don't need anything
> > other than the shutdown checkpoint record because we don't perform
> > recovery and checkpoint record is required at startup.
> > 
> > RM_XLOG_ID:
> >  XLOG_FPI_FOR_HINT  - not needed?
> >  XLOG_FPI           - not needed?
> > 
> >  XLOG_CHECKPOINT_SHUTDOWN - must have
> > 
> > So how about the followings?
> >  XLOG_CHECKPOINT_ONLINE
> >  XLOG_NOOP
> >  XLOG_NEXTOID
> >  XLOG_SWITCH
> >  XLOG_BACKUP_END
> >  XLOG_PARAMETER_CHANGE
> >  XLOG_RESTORE_POINT
> >  XLOG_FPW_CHANGE
> >  XLOG_END_OF_RECOVERY
> > 
> > 
> > RM_XACT_ID:
> >  XLOG_XACT_COMMIT
> >  XLOG_XACT_PREPARE
> >  XLOG_XACT_ABORT
> >  XLOG_XACT_COMMIT_PREPARED
> >  XLOG_XACT_ABORT_PREPARED
> >  XLOG_XACT_ASSIGNMENT
> >  XLOG_XACT_INVALIDATIONS
> > 
> > Do we need all of these?
> No. Strictly speaking, you are right.
> We still have types of WAL that are not necessarily needed.
> For example, XLOG_END_OF_RECOVERY is not useful
> because wal_level=none doesn't recover from any accidents.
> Or, XLOG_CHECKPOINT_ONLINE is used when we execute CHECKPOINT
> not for shutting down. Thus we could eliminate more.

Yeah, although it's enough only to restrict non-harmful records
practically, if we find that only a few kinds of records are needed,
maybe it's cleaner to allow only required record type(s).

> > And, currenly what decides whether to emit a wal record according to
> > wal_level is the caller of XLogInsert. 
> Yes.
> 
> > So doing this at XLogInsert-level means
> > that we bring the criteria of the necessity of wal-record into xlog layer only for
> > wal_level=none. I'm not sure it is the right direction.
> I'm sorry. I didn't understand what "doing this" and "xlog layer" meant.

"doing this" meant filtering record types.

> Did you mean that fixing the caller side of XLogInsert (e.g. CreateCheckPoint)
> is not the right direction ? Or, fixing the function of XLogInsert is not the right direction ?

Maybe it's right that if we can filter-out records looking only rmid,
since the xlog facility doesn't need to know about record types of a
resource manager.  But if we need to finer-grained control on the
record types, I'm afraid that that's wrong.  However, if we need only
the XLOG_CHECKPOINT_SHUTDOWN record, it might be better to let
XLogInsert filter records rather than inserting that filtering code to
all the caller sites.

> > At Fri, 27 Nov 2020 07:01:16 +0000, "tsunakawa.takay@fujitsu.com"
> > <tsunakawa.takay@fujitsu.com> wrote in
> > > I'm afraid "none" doesn't represent the behavior because RM_XLOG_ID and
> > RM_XACT_ID WAL records, except for XLOG_FPI_*, are emitted.  What's the
> > good name?  IIUC, "minimal" is named after the fact that the minimal
> > amount of WAL necessary for crash recovery is generated.  "norecovery" or
> > "unrecoverable"?
> Lastly, I found another name which expresses the essential characteristic of this wal_level.
> How about the name of wal_level="crash_unsafe" ?
> What did you think ?

I don't dislike "none" since it seems to me practically "none".  It
seems rather correct if we actually need only the shutdown checkpoint
record.

"unrecoverable" is apparently misleading. "crash_unsafe" is precise
but seems somewhat alien being among "logical", "replica" and
"minimal".

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center