Re: Disable WAL logging to speed up data loading - Mailing list pgsql-hackers

From Stephen Frost
Subject Re: Disable WAL logging to speed up data loading
Date
Msg-id 20210322134603.GR20766@tamriel.snowman.net
Whole thread Raw
In response to RE: Disable WAL logging to speed up data loading  ("tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>)
Responses Re: Disable WAL logging to speed up data loading  (Laurenz Albe <laurenz.albe@cybertec.at>)
List pgsql-hackers
Greetings,

* tsunakawa.takay@fujitsu.com (tsunakawa.takay@fujitsu.com) wrote:
> From: Stephen Frost <sfrost@snowman.net>
> > The argument here seems to stem from the idea that issueing a 'TRUNCATE'
> > inside the transaction before starting the 'COPY' command is 'too hard'.
>
> > I could be wrong and perhaps opinions will change in the future, but it really
> > doesn't seem like the there's much support for adding a new WAL level just to
> > avoid doing the TRUNCATE.  Appealing to what other database systems
> > support can be helpful in some cases but that's usually when we're looking at a
> > new capability which multiple other systems have implemented.  This isn't
> > actually a new capability at all- the WAL level that you're looking for already
> > exists and already gives the optimization you're looking for, as long as you issue
> > a TRUNCATE at the start of the transaction.
>
> No, we can't ask using TRUNCATE because the user wants to add data to a table.

First- what are you expecting would actually happen during crash
recovery in this specific case with your proposed new WAL level?

Second, use partitioning, or unlogged tables (with the patch discussed
elsewhere to allow flipping them to logged without writing the entire
thing to WAL).

Thanks,

Stephen

Attachment

pgsql-hackers by date:

Previous
From: Dean Rasheed
Date:
Subject: Re: pgbench - add pseudo-random permutation function
Next
From: Robert Haas
Date:
Subject: Re: New IndexAM API controlling index vacuum strategies