RE: Disable WAL logging to speed up data loading - Mailing list pgsql-hackers

From osumi.takamichi@fujitsu.com
Subject RE: Disable WAL logging to speed up data loading
Date
Msg-id OSBPR01MB48884BCEEBC453BA0DA8FEAFEDF80@OSBPR01MB4888.jpnprd01.prod.outlook.com
Whole thread Raw
In response to Re: Disable WAL logging to speed up data loading  (Kyotaro Horiguchi <horikyota.ntt@gmail.com>)
Responses Re: Disable WAL logging to speed up data loading  (Kyotaro Horiguchi <horikyota.ntt@gmail.com>)
List pgsql-hackers
Thank you, Horiguchi-San

> I haven't seen a criteria of whether a record is emitted or not for
> wal_leve=none.
>
> We're emitting only redo logs.  So I think theoretically we don't need anything
> other than the shutdown checkpoint record because we don't perform
> recovery and checkpoint record is required at startup.
>
> RM_XLOG_ID:
>  XLOG_FPI_FOR_HINT  - not needed?
>  XLOG_FPI           - not needed?
>
>  XLOG_CHECKPOINT_SHUTDOWN - must have
>
> So how about the followings?
>  XLOG_CHECKPOINT_ONLINE
>  XLOG_NOOP
>  XLOG_NEXTOID
>  XLOG_SWITCH
>  XLOG_BACKUP_END
>  XLOG_PARAMETER_CHANGE
>  XLOG_RESTORE_POINT
>  XLOG_FPW_CHANGE
>  XLOG_END_OF_RECOVERY
>
>
> RM_XACT_ID:
>  XLOG_XACT_COMMIT
>  XLOG_XACT_PREPARE
>  XLOG_XACT_ABORT
>  XLOG_XACT_COMMIT_PREPARED
>  XLOG_XACT_ABORT_PREPARED
>  XLOG_XACT_ASSIGNMENT
>  XLOG_XACT_INVALIDATIONS
>
> Do we need all of these?
No. Strictly speaking, you are right.
We still have types of WAL that are not necessarily needed.
For example, XLOG_END_OF_RECOVERY is not useful
because wal_level=none doesn't recover from any accidents.
Or, XLOG_CHECKPOINT_ONLINE is used when we execute CHECKPOINT
not for shutting down. Thus we could eliminate more.

> And, currenly what decides whether to emit a wal record according to
> wal_level is the caller of XLogInsert.
Yes.

> So doing this at XLogInsert-level means
> that we bring the criteria of the necessity of wal-record into xlog layer only for
> wal_level=none. I'm not sure it is the right direction.
I'm sorry. I didn't understand what "doing this" and "xlog layer" meant.
Did you mean that fixing the caller side of XLogInsert (e.g. CreateCheckPoint)
is not the right direction ? Or, fixing the function of XLogInsert is not the right direction ?


> At Fri, 27 Nov 2020 07:01:16 +0000, "tsunakawa.takay@fujitsu.com"
> <tsunakawa.takay@fujitsu.com> wrote in
> > I'm afraid "none" doesn't represent the behavior because RM_XLOG_ID and
> RM_XACT_ID WAL records, except for XLOG_FPI_*, are emitted.  What's the
> good name?  IIUC, "minimal" is named after the fact that the minimal
> amount of WAL necessary for crash recovery is generated.  "norecovery" or
> "unrecoverable"?
Lastly, I found another name which expresses the essential characteristic of this wal_level.
How about the name of wal_level="crash_unsafe" ?
What did you think ?


Best,
    Takamichi Osumi




pgsql-hackers by date:

Previous
From: Ajin Cherian
Date:
Subject: Re: [HACKERS] logical decoding of two-phase transactions
Next
From: Fujii Masao
Date:
Subject: Re: Feature improvement for pg_stat_statements