RE: Disable WAL logging to speed up data loading - Mailing list pgsql-hackers

From tsunakawa.takay@fujitsu.com
Subject RE: Disable WAL logging to speed up data loading
Date
Msg-id TYAPR01MB29906BC112B9639F4679D92BFEF30@TYAPR01MB2990.jpnprd01.prod.outlook.com
Whole thread Raw
In response to RE: Disable WAL logging to speed up data loading  ("osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>)
List pgsql-hackers
From: Osumi, Takamichi/大墨 昂道 <osumi.takamichi@fujitsu.com>
> I executed each wal_level three times and calculated the average time
> and found that disabling WAL logging reduced about 73 % of the minimal's
> loading speed
> in this test. This speed-up came from the difference of generated WAL sizes.

So, it's 4x speedup when the WAL buffer and/or disk is likely to be saturated.  That's nice!


> In this test, to load the data generated more than 10GB of WALs with
> wal_level=minimal
> while wal_level=none emits just 760 bytes of WALs.
>
> I expect this size for none will become smaller when
> I take the modification to filter out the types of WAL which is discussed above.

I don't expect so, because the WAL volume is already very small in this workload (probably only the commit records.)


> Also, I monitored numbers of iostat's 'util' and noticed that
> util's spike to use I/O reduced from twice to once when I changed the level
> from minimal to none, which should be the effect of the patch.

It's a bit mysterious.  I thought %util would be constantly 100% when wal_level = minimal.  That may be because
wal_buffersis large. 


Regards
Takayuki Tsunakawa





pgsql-hackers by date:

Previous
From: Mark Dilger
Date:
Subject: Re: room for improvement in amcheck btree checking?
Next
From: Michael Paquier
Date:
Subject: Re: scram-sha-256 broken with FIPS and OpenSSL 1.0.2