RE: Disable WAL logging to speed up data loading - Mailing list pgsql-hackers

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
> We're emitting only redo logs.  So I think theoretically we don't need
> anything other than the shutdown checkpoint record because we don't
> perform recovery and checkpoint record is required at startup.
>
> RM_XLOG_ID:
>  XLOG_FPI_FOR_HINT  - not needed?
>  XLOG_FPI           - not needed?
>
>  XLOG_CHECKPOINT_SHUTDOWN - must have
>
> So how about the followings?
>  XLOG_CHECKPOINT_ONLINE
>  XLOG_NOOP
>  XLOG_NEXTOID
>  XLOG_SWITCH
>  XLOG_BACKUP_END
>  XLOG_PARAMETER_CHANGE
>  XLOG_RESTORE_POINT
>  XLOG_FPW_CHANGE
>  XLOG_END_OF_RECOVERY
>
>
> RM_XACT_ID:
>  XLOG_XACT_COMMIT
>  XLOG_XACT_PREPARE
>  XLOG_XACT_ABORT
>  XLOG_XACT_COMMIT_PREPARED
>  XLOG_XACT_ABORT_PREPARED
>  XLOG_XACT_ASSIGNMENT
>  XLOG_XACT_INVALIDATIONS
>
> Do we need all of these?

What need to be emitted even when wal_level = none are:

RM_XLOG_ID:
- XLOG_CHECKPOINT_SHUTDOWN
- XLOG_PARAMETER_CHANGE

RM_XACT_ID:
-  XLOG_XACT_PREPARE

XLOG_PARAMETER_CHANGE is necessary to detect during archive recovery that wal_level was changed from >= replica to
none,thus failing the archive recovery.  This is for the fix discussed in this thread to change the error level from
WARNINGto FATAL. 


> Yeah, although it's enough only to restrict non-harmful records
> practically, if we find that only a few kinds of records are needed,
> maybe it's cleaner to allow only required record type(s).
>
> Maybe it's right that if we can filter-out records looking only rmid,
> since the xlog facility doesn't need to know about record types of a
> resource manager.  But if we need to finer-grained control on the
> record types, I'm afraid that that's wrong.  However, if we need only
> the XLOG_CHECKPOINT_SHUTDOWN record, it might be better to let
> XLogInsert filter records rather than inserting that filtering code to
> all the caller sites.

Agreed.  As the kind of WAL records to be emitted is very limited, I think XLogInsert() can filter them where the
currentpatch does.  (OTOH, the boot strap mode filters WAL coarsely as follows.  I thought we may follow this coarse
RMID-basedfiltering as a pessimistic safeguard against new kind of WAL records that are necessary even in wal_level =
none.)

    /*
     * In bootstrap mode, we don't actually log anything but XLOG resources;
     * return a phony record pointer.
     */
    if (IsBootstrapProcessingMode() && rmid != RM_XLOG_ID)
    {
        XLogResetInsertion();
        EndPos = SizeOfXLogLongPHD; /* start of 1st chkpt record */
        return EndPos;
    }


> I don't dislike "none" since it seems to me practically "none".  It
> seems rather correct if we actually need only the shutdown checkpoint
> record.
>
> "unrecoverable" is apparently misleading. "crash_unsafe" is precise
> but seems somewhat alien being among "logical", "replica" and
> "minimal".

OK, I'm happy with "none" too.  We can describe in the manual that some limited amount of WAL is emitted.


Regards
Takayuki Tsunakawa