Re: XTS cipher mode for cluster file encryption - Mailing list pgsql-hackers

Greetings,

* Bruce Momjian (bruce@momjian.us) wrote:
> On Tue, Oct 19, 2021 at 02:54:56PM -0400, Stephen Frost wrote:
> > * Sasasu (i@sasa.su) wrote:
> > > A unified block-based I/O API sounds great. Has anyone tried to do this
> > > before? It would be nice if the front-end tools could also use these API.
> >
> > The TDE patch from Cybertec did go down this route, but the API ended up
> > being rather different which menat a lot of changes in other parts of
> > the system.  If we can get a block-based temporary file method that
> > maintains more-or-less the same API, that'd be great, but I'm not sure
> > that we can really do so and I am not entirely convinced that we should
> > make the TDE effort depend on an otherwise quite independent effort of
> > making all temp files usage be block based.
>
> Uh, I thought people felt the Cybertec patch was too large and that a
> unified API for temporary file I/O-encryption was a requirement.  Would
> a CTR-steaming-encryption API for temporary tables be easier to
> implement?

Having a unified API for temporary file I/O (that could then be extended
to provide encryption) would definitely help with reducing the size of a
TDE patch.  The approach used in the Cybertec patch was to make
temporary file access block based, but the way that was implemented was
with an API different from pread/pwrite and that meant changing pretty
much all of the call sites for temporary file access, which naturally
resulted in changes in a lot of otherwise unrelated code.

There was an argument put forth that a block-based API for temporary
file access would generally be good as it would mean fewer syscalls.  If
we can get behind that and make it happen in (relatively) short order
then we'd certainly be better off when it comes to implementing TDE
which also deals with temporary files.  I'm a bit concerned about that
approach due to the changes needed but I'm also not against it.  I do
think that an API which was more-or-less the same as what's used today
would be a smaller change and therefore might be easier to get in and
that it'd also make a TDE patch smaller.  Perhaps both could be
accomplished (an API that's similar to today, but the actual file access
being block-based).

Either way, we should get that unification done in the core code first
as an independent effort.  That's what I hope the Cybertec folks have
had a chance to work on.

As for the specific encryption method to use, using CTR would be simpler
as it doesn't require access to be block-based, though we would need to
make sure to not re-use the IV across any of the temporary files being
created (potentially concurrently).  Probably not that hard to do but
just something to make sure we do.  Of course, if we arrange for
block-based access then we could use XTS or perhaps GCM/GCM-SIV if we
wanted to.

Thanks,

Stephen