Re: silent data loss with ext4 / all current versions - Mailing list pgsql-hackers

From Greg Stark
Subject Re: silent data loss with ext4 / all current versions
Date
Msg-id CAM-w4HNdUkN7=8ob3xX-t613L_mWEuzFcRCFZ6hWVCjwM_16_g@mail.gmail.com
Whole thread Raw
In response to Re: silent data loss with ext4 / all current versions  (Tomas Vondra <tomas.vondra@2ndquadrant.com>)
Responses Re: silent data loss with ext4 / all current versions  (Michael Paquier <michael.paquier@gmail.com>)
List pgsql-hackers
On Fri, Jan 22, 2016 at 8:26 AM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
> On 01/22/2016 06:45 AM, Michael Paquier wrote:
>
>> So, I have been playing with a Linux VM with VMware Fusion and on
>> ext4 with data=ordered the renames are getting lost if the root
>> folder is not fsync. By killing-9 the VM I am able to reproduce that
>> really easily.
>
>
> Yep. Same experience here (with qemu-kvm VMs).

I still think a better approach for this is to run the database on an
LVM volume and take lots of snapshots. No VM needed, though it doesn't
hurt. LVM volumes are below the level of the filesystem and a snapshot
captures the state of the raw blocks the filesystem has written to the
block layer. The block layer does no caching though the drive may but
neither the VM solution nor LVM would capture that.

LVM snapshots would have the advantage that you can keep running the
database and you can take lots of snapshots with relatively little
overhead. Having dozens or hundreds of snapshots would be unacceptable
performance drain in production but for testing it should be practical
and they take relatively little space -- just the blocks changed since
the snapshot was taken.


-- 
greg



pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: silent data loss with ext4 / all current versions
Next
From: Andres Freund
Date:
Subject: Re: silent data loss with ext4 / all current versions