RE: Disable WAL logging to speed up data loading - Mailing list pgsql-hackers

From tsunakawa.takay@fujitsu.com
Subject RE: Disable WAL logging to speed up data loading
Date
Msg-id TYAPR01MB2990DA2A5FDA5B094CD708EBFE310@TYAPR01MB2990.jpnprd01.prod.outlook.com
Whole thread Raw
In response to Re: Disable WAL logging to speed up data loading  (Fujii Masao <masao.fujii@oss.nttdata.com>)
Responses RE: Disable WAL logging to speed up data loading
List pgsql-hackers
From: Fujii Masao <masao.fujii@oss.nttdata.com>
> > The speedup has already been achieved with higher durability by
> > wal_level=minimal in that case.
>
> I was thinking the same, i.e., wal_level=minimal + wal_skip_threshold would
> speed up that initial data loading.

First of all, thank you Horiguchi-san for trying to improve ALTER TABLE SET UNLOGGED/LOGGED.  That should also be
appealing.

At the same time, as I said before, both features have good points.  TBH, as a user, I'm kind of attracted by MySQL's
approachbecause of its simplicity for users (although DBMS developers may be worried about this and that.)  What tempts
meis that I can just switch on the feature with a single configuration parameter, and continue to use existing SQL
scriptsand other data integration software without knowing what tables those load data into.  In the same context, I
don'thave to add or delete ALTER TABLE statements when I have to change the set of tables to be loaded.  For the same
reason,I'm also interested in Oracle's another feature ALTER TABLESPACE LOGGING/NOLOGGING. 

BTW, does ALTER TABLE LOGGED/UNLOGGED on a partitioned table get the change to its all partitions?  It would be a bit
tediousto add/delete ALTER TABLE LOGGED/UNLOGGED when I add/drop a partition. 

Regarding data migration, data movement is not limited only to major upgrades.  It will be convenient to speed up the
migrationof the entire database cluster into a new instance for testing and new deployment.  (I'm not sure about recent
pg_upgrade,but pg_upgrade sometimes cannot upgrade too older versions.) 

To conclude, I hope both features will be realized, and wish we won't fall in a situation where the words fly such as
"Mineis enough. Yours is risky and not necessary." 

With that said, I think we may as well separate the thread some time later for CF entry.  Otherwise, we will have
troublein finding the latest patch from the CF entry. 


> No, I have no strong objection against your trial. But I was thinking
> that it's not so easy to design and implement wal_level=none.
> For example, there are some functions and commands depending on
> the existence of WAL, like pg_switch_wal(), PREPARE TRANSACTION
> and COMMIT PREPARED. Probably you need to define how they should
> work in wal_level=none, e.g., emit an error.

    Yeah, we thought pg_switch_wal() may need some treatment.  We'll check PREPARE and COMMIT PREPARED as well.  I'd
appreciateit if you share what you notice at any time.  It is possible that we should emit WAL records of some resource
managers,like the bootstrap mode emits WAL only for RM_XLOG_ID. 


Regards
Takayuki Tsunakawa






pgsql-hackers by date:

Previous
From: Kyotaro Horiguchi
Date:
Subject: Re: Disable WAL logging to speed up data loading
Next
From: Kyotaro Horiguchi
Date:
Subject: Re: Disable WAL logging to speed up data loading