Re: [Proposal] Table-level Transparent Data Encryption (TDE) and KeyManagement Service (KMS) - Mailing list pgsql-hackers

On Sat, Jul 13, 2019 at 09:38:06AM -0400, Joe Conway wrote:
> On 7/11/19 9:05 PM, Bruce Momjian wrote:
> >> Regardless of the method to split the heap into different keys, I think
> >> there should be an option for some tables to not be encrypted. If we
> >> decide it must be all or nothing for the first implementation I guess I
> >> could live with it but would be very disappointed.
> > 
> > What does it mean you "could live this it"?  Why do you consider having
> > some tables unencrypted important?
> 
> 
> I think it is pretty obvious isn't it? You have talked about the
> performance impact. Why would I want to encrypt, for example a lookup
> table, if there is nothing in that table warranting encryption?

Well, a lookup table is not going to get many writes, and will usually
stay in shared buffers, so I don't see the value in skipping encryption
of it.  However, the big issue is that having encryption on only some
tables has a cost, both in code complexity, and in looking up pg_class
or pg_tablespace rows to find out what needs encryption.  Also, it would
lead to information leakage if we don't encrypt all of WAL but instead
only encrypt the entries for tables/tablespaces that need encryption. 
(I don't think we discussed how WAL would be encrypted in such cases.)

My point is that doing encryption of only some data might actually make
the system slower due to the lookups, so I think we need to implement
all-cluster encryption and then see what the overhead is, and if there
are use-cases for not encrypting only some data.

> I think in many if not most applications the sensitive data is limited
> to much less than all of the tables, and I'd rather not take the hit for
> those tables.

Agreed, but let's see what the overhead it first, and also decide how
WAL would be handled in such cases.

> >> Thinking out loud (and I believe somewhere in this massive thread
> >> someone else already said this), if we had a way to flag "key version"
> >> at the page level it seems like we could potentially rekey page-by-page
> >> while online, locking only one page at a time. We really only need to
> >> support 2 key versions and could ping-pong between them as they change.
> >> Or maybe this is a crazy idea.
> > 
> > Yes, we did talk about this.  It is certainly possible, but we would
> > still need a tool to guarantee all pages are using the new version, so I
> > am not sure what per-page buys us except making the later check faster. 
> > I don't see this as a version-1 feature, frankly.
> 
> If we allow for say, 2 versions of the key to exist at any given time,
> and if we could store that key version information on each page, we
> could change the key from old to new without locking the entire table at
> once, just locking one page at a time. Or at least that was my thinking.

Yes, that is true, but eventually we will need to do key rotation again,
so we have to be sure the old key is no longer being used, so we would
still need a tool to check everything to be sure all the pages are using
the new key, and that could be done a page at a time.

It does feel like our checksum problem and I am hoping the
infrastructure to allow that to be done online can be used for this too.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +