Re: [Proposal] Table-level Transparent Data Encryption (TDE) and KeyManagement Service (KMS) - Mailing list pgsql-hackers

From Bruce Momjian
Subject Re: [Proposal] Table-level Transparent Data Encryption (TDE) and KeyManagement Service (KMS)
Date
Msg-id 20191107011949.GA5682@momjian.us
Whole thread Raw
In response to Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)  (Antonin Houska <ah@cybertec.at>)
List pgsql-hackers
On Sat, Nov  2, 2019 at 01:34:41PM +0100, Antonin Houska wrote:
> Robert Haas <robertmhaas@gmail.com> wrote:
> 
> > On Tue, Aug 6, 2019 at 10:36 AM Bruce Momjian <bruce@momjian.us> wrote:
> 
> > Seems reasonable (not that I am an encryption expert).
> > 
> > > For WAL, we effectively create a 16MB bitstream, though we can create it
> > > in parts as needed.  (Creating it in parts is easier in CTR mode.)  The
> > > nonce is the segment number, but each 16-byte chunk uses a different
> > > counter.  Therefore, even if you are encrypting the same 8k page several
> > > times in the WAL, the 8k page would be different because of the LSN (and
> > > other changes), and the bitstream you encrypt/XOR it with would be
> > > different because the counter would be different for that offset in the
> > > WAL.
> > 
> > But, if you encrypt the same WAL page several times, the LSN won't
> > change, because a WAL page doesn't have an LSN on it, and if it did,
> > it wouldn't be changing, because an LSN is just a position within the
> > WAL stream, so any given byte on any given WAL page always has the
> > same LSN, whatever it is.
> > 
> > And if the counter value changed on re-encryption, I don't see how
> > we'd know what counter value to use when decrypting.  There's no way
> > for the code that is decrypting to know how many times the page got
> > rewritten as it was being filled.
> > 
> > Please correct me if I'm being stupid here.
> 
> In my implementation (I haven't checked whether Masahiko Sawada changed this
> in his patch) I avoided repeated encryption of different data using the same
> key+IV by omitting the unused part of the WAL page from encryption. Already
> written records can be encrypted repeatedly because they do not change.

Right.  Even though AES with CTR generates encryption bit patterns in
16-byte chunks, you only XOR the bytes you have written.  So, if the WAL
record is 167 bytes, you generate 11 16-byte patterns, but you only XOR
the first seven bytes of the 11th 16-byte block.  CTR is not like CBC
which has to encrypt in 16-byte chunks.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +



pgsql-hackers by date:

Previous
From: Andrew Dunstan
Date:
Subject: Re: TAP tests aren't using the magic words for Windows file access
Next
From: Bruce Momjian
Date:
Subject: Re: ssl passphrase callback