RE: Complete data erasure - Mailing list pgsql-hackers

From asaba.takanori@fujitsu.com
Subject RE: Complete data erasure
Date
Msg-id OSBPR01MB4728E6BE5B022146088B85868CF70@OSBPR01MB4728.jpnprd01.prod.outlook.com
Whole thread Raw
In response to RE: Complete data erasure  ("asaba.takanori@fujitsu.com" <asaba.takanori@fujitsu.com>)
Responses RE: Complete data erasure  ("asaba.takanori@fujitsu.com" <asaba.takanori@fujitsu.com>)
List pgsql-hackers
Hello Tom,

From: asaba.takanori@fujitsu.com <asaba.takanori@fujitsu.com>
> Hello Tom,
>
> From: Tom Lane <tgl@sss.pgh.pa.us>
> > Tomas Vondra <tomas.vondra@2ndquadrant.com> writes:
> > > I think it depends how exactly it's implemented. As Tom pointed out in
> > > his message [1], we can't do the erasure itself in the post-commit is
> > > not being able to handle errors. But if the files are renamed durably,
> > > and the erasure happens in a separate process, that could be OK. The
> > > COMMIT may wayt for it or not, that's mostly irrelevant I think.
> >
> > How is requiring a file rename to be completed post-commit any less
> > problematic than the other way?  You still have a non-negligible
> > chance of failure.
>
> I think that errors of rename(2) listed in [1] cannot occur or can be handled.
> What do you think?
>
> [1] http://man7.org/linux/man-pages/man2/rename.2.html
>

I have another idea.
How about managing status of data file like the WAL archiver?
For example,

1. Create a status file "...ready" in a transaction that has DROP TABLE. (not rename the data file)
2. Background worker scans the directory that has status file.
3. Rename the status file to "...progress" when the erase of the data file starts.
4. Rename the status file to "...done" when the erase of the data file finished.

I think that it's OK because step1 is not post-commit and background worker can handle error of the erase.

Regards,

--
Takanori Asaba





pgsql-hackers by date:

Previous
From: Julien Rouhaud
Date:
Subject: Re: Online checksums verification in the backend
Next
From: Justin Pryzby
Date:
Subject: Re: control max length of parameter values logged