RE: Complete data erasure - Mailing list pgsql-hackers

From asaba.takanori@fujitsu.com
Subject RE: Complete data erasure
Date
Msg-id OSBPR01MB4728FC5244A9823507AB572A8CDE0@OSBPR01MB4728.jpnprd01.prod.outlook.com
Whole thread Raw
In response to RE: Complete data erasure  ("asaba.takanori@fujitsu.com" <asaba.takanori@fujitsu.com>)
Responses Re: Complete data erasure  (Tomas Vondra <tomas.vondra@2ndquadrant.com>)
List pgsql-hackers
Hello,

I was off the point.
I want to organize the discussion and suggest feature design.

There are two opinions.
1. COMMIT should not take a long time because errors are more likely to occur.
2. The data area should be released when COMMIT is completed because COMMIT has to be an atomic action.

These opinions are correct.
But it is difficult to satisfy them at the same time.
So I suggest that users have the option to choose.
DROP TABLE works as following two patterns:

1. Rename data file to "...del" instead of ftruncate(fd,0).
  After that, bgworker scan the directory and run erase_command.
  (erase_command is command set by user like archive_command.
   For example, shred on Linux.)

2. Run erase_command for data file immediately before ftruncate(fd,0).
  Wait until it completes, then reply COMMIT to the client.
  After that, it is the same as normal processing.

If error of erase_command occurs, it issues WARNING and don't request unlink to CheckPointer.
It’s not a security failure because I think that there is a risk when data area is returned to OS.

I will implement from pattern 2 because it's more similar to user experience than pattern 1.
This method has been pointed out as follows.

From Stephen
> We certainly can't run external commands during transaction COMMIT, so
> this can't be part of a regular DROP TABLE.

I think it means that error of external commands can't be handled.
If so, it's no problem because I determined behavior after error.
Are there any other problems?

Regards,

--
Takanori Asaba





pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: pg_basebackup, manifests and backends older than ~12
Next
From: Jeremy Morton
Date:
Subject: Support for DATETIMEOFFSET