Re: Disable WAL logging to speed up data loading - Mailing list pgsql-hackers

From Stephen Frost
Subject Re: Disable WAL logging to speed up data loading
Date
Msg-id 20210324114357.GP20766@tamriel.snowman.net
Whole thread Raw
In response to RE: Disable WAL logging to speed up data loading  ("tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>)
Responses RE: Disable WAL logging to speed up data loading  ("osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>)
List pgsql-hackers
Greetings,

* tsunakawa.takay@fujitsu.com (tsunakawa.takay@fujitsu.com) wrote:
> From: Stephen Frost <sfrost@snowman.net>
> > * tsunakawa.takay@fujitsu.com (tsunakawa.takay@fujitsu.com) wrote:
> > As for data loading tools, surely they support loading data into UNLOGGED
> > tables and it's certainly not hard to have a script run around and flip those
> > tables to LOGGED after they're loaded, and I do actually believe some of those
> > tools support building processes of which one step could be such a command
> > (I'm fairly confident Pentaho, in particular, does as I remember building such
> > pipelines myself...).
>
> Oh, Pentaho has such a feature, doesn't it?  But isn't it a separate step from the data output step?  Here, I assume
ETLtools allow users to compose a data loading job from multiple steps: data input, transformation, data output, etc.
Iguess the user can't directly incorporate ALTER TABLE into the data output step, and has to add separate custom steps
forALTER TABLE.  That's burdonsome and forgettable, I think. 

None of the arguments presented here has done anything to change my
opinion that adding a 'none' WAL level is a bad idea.

Thanks,

Stephen

Attachment

pgsql-hackers by date:

Previous
From: Andrew Dunstan
Date:
Subject: Re: multi-install PostgresNode
Next
From: Dilip Kumar
Date:
Subject: Re: [HACKERS] Custom compression methods