RE: Disable WAL logging to speed up data loading - Mailing list pgsql-hackers

From tsunakawa.takay@fujitsu.com
Subject RE: Disable WAL logging to speed up data loading
Date
Msg-id TYAPR01MB299099E8FEBE79F6D2054194FEE20@TYAPR01MB2990.jpnprd01.prod.outlook.com
Whole thread Raw
In response to Re: Disable WAL logging to speed up data loading  ("David G. Johnston" <david.g.johnston@gmail.com>)
List pgsql-hackers

# It'd be helpful if you could send mails in text format, not HTML.

 

From: David G. Johnston <david.g.johnston@gmail.com>

> For this case the fundamental feature that would seem to be required is an ability for a transaction commit to return only after the system has ensured that all of the new pages added to the relation during the scope of the transaction have made it to disk.  Something like:

>

> BEGIN UNLOGGED TRANSACTION FOR table1, table2;

> -- locking probably allows reads, definitely disallows concurrent writes, to the named tables

> -- Disallow updates and deletes, do not use dead tuple space, for the tables named.  Should be able to do normal stuff for other tables?

> -- Always create new pages

> COPY TO table1;

> COPY TO table2;

> COMMIT; -- wait here until data files for table1 and table2 are completely written and the transaction alive flag is committed to the WAL.

>

> I suppose the above could be written "BEGIN UNLOGGED TRANSACTION FOR ALL TABLES" and you'd get the initial database population optimization capability.

>

> If the commit doesn't complete all of the newly created pages are junk.  Otherwise, you have a crash-recoverable state for those tables as regards those specific pages.

 

As Steven-san said, I don't want to go this complicated direction.  Plus, putting my feet in the user's shoes, I want to try to avoid introducing a new SQL syntax for this kind of performance boost, which requires applications and maintenance scripts and testing.

 

 

Regards

Takayuki Tsunakawa

 

pgsql-hackers by date:

Previous
From: Fujii Masao
Date:
Subject: Re: Add statistics to pg_stat_wal view for wal related parameter tuning
Next
From: Fujii Masao
Date:
Subject: Re: Add statistics to pg_stat_wal view for wal related parameter tuning