RE: Disable WAL logging to speed up data loading - Mailing list pgsql-hackers

From osumi.takamichi@fujitsu.com
Subject RE: Disable WAL logging to speed up data loading
Date
Msg-id OSBPR01MB4888372E2590774B1311F4B1ED300@OSBPR01MB4888.jpnprd01.prod.outlook.com
Whole thread Raw
In response to Re: Disable WAL logging to speed up data loading  (Fujii Masao <masao.fujii@oss.nttdata.com>)
Responses Re: Disable WAL logging to speed up data loading
List pgsql-hackers
Hello.

> >> Can they use a database with all unlogged tables?
> > Unfortunately, no. They want to switch a cluster condition to "without WAL
> logging"
> > only when they execute night bulk loading for their data warehouse.
> > In other words, they would like to keep their other usual operations with WAL.
> > In addition, using all tables as unlogged brings about the risk to
> > lose data warehouse's data caused by an unexpected server crash or power
> outage.
> 
> But the same issue can happen even in the proposed approach because
> Tsunakawa-san explains as follows?
Sorry, my last expression about "to lose data" was not precise.

When unlogged tables are used and the server is crashed unexpectedly,
all data after the last backup is truncated without WAL, right ?
In this case, sequential commands for unlogged tables disappear.
On the other hand, wal_level='none' can be changed to 'minimal' for example.
Therefore, such other sequential operations after the last backup
could be stored in WAL file, when a user applies such a change of wal_level.
I meant this is helpful but didn't this make sense ?

> > The server refuses to start (pg_ctl start fails) after an abnormal shutdown due
> to power outage, pg_ctl's immediate shutdown, etc, showing a straightforward
> message like MySQL.
This is the behavior during wal_level='none'.

> * The user can use all features again if you shut down the server successfully
> after data loading and reset wal_level to a value other than none.  He needs to
> take a base backup or rebuild the replication standby after restarting the server.
And, we expect we can change the wal_level even after using wal_level='none'
and can use other features again like I mentioned above. What did you think ?

Regards,
    Takamichi Osumi

pgsql-hackers by date:

Previous
From: "tsunakawa.takay@fujitsu.com"
Date:
Subject: RE: [Patch] Optimize dropping of relation buffers using dlist
Next
From: Kyotaro Horiguchi
Date:
Subject: Re: [Patch] Optimize dropping of relation buffers using dlist