Thread: Value of Transparent Data Encryption (TDE)
For plan for full-cluster Transparent Data Encryption (TDE) is here:
https://wiki.postgresql.org/wiki/Transparent_Data_Encryption#TODO_for_Full-Cluster_Encryption
The values it has, I think, are:
* encrypts data for anyone with read-access to the file system (but not
memory)
* I think write access would allow access to the encryption keys
by modifying postgresql.conf or other files
* This is particularly useful if the storage is remote
* encrypts non-logical/non-pg_dump-like backups
* fulfills several security compliance requirements
* encrypts storage
* perhaps easier to implement than file system encryption
Is that accurate?
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On Mon, Sep 30, 2019 at 05:40:52PM -0400, Bruce Momjian wrote: >For plan for full-cluster Transparent Data Encryption (TDE) is here: > > https://wiki.postgresql.org/wiki/Transparent_Data_Encryption#TODO_for_Full-Cluster_Encryption > >The values it has, I think, are: > >* encrypts data for anyone with read-access to the file system (but not > memory) > > * I think write access would allow access to the encryption keys > by modifying postgresql.conf or other files > > * This is particularly useful if the storage is remote > >* encrypts non-logical/non-pg_dump-like backups > >* fulfills several security compliance requirements > >* encrypts storage > Maybe. I think this is approaching the problem from the wrong angle. Encryption is more a means of achieving something. OK, for compliance purposes it's useful to be able to tick "encryption" checkbox. But other than that, people really care about threat models and how encryption improves them (or does not). So I think it'd be valuable to improve the "thread models" section on that wiki page, with more detailed cases. We need to explain what capabilities the attacker has (can he read files?can he interact with the database? can he read memory? ..) and then explain how that works with encrypted cluster. >* perhaps easier to implement than file system encryption > Not sure. IMO filesystem encryption is fairly simple to use, to the extent that it's hard to beat. The problem is usually people can't use it for various reasons - lack of support on their OS, no access to the block device, problems with obtaining the privileges etc. Having it built into the database menas you can sidestep most of those issue (e.g. you can deploy it as a DBA, on arbitrary OS, ...). Plus it allows features you can't easily achieve with fs encryption, because the filesystem only sees opaque data files. So having keys per database/user/... is easier from within the database. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Tue, Oct 1, 2019 at 03:43:05PM +0200, Tomas Vondra wrote: > On Mon, Sep 30, 2019 at 05:40:52PM -0400, Bruce Momjian wrote: > Maybe. I think this is approaching the problem from the wrong angle. > Encryption is more a means of achieving something. OK, for compliance > purposes it's useful to be able to tick "encryption" checkbox. But other > than that, people really care about threat models and how encryption > improves them (or does not). Yes, that is what I am trying to do with this email thread. > So I think it'd be valuable to improve the "thread models" section on > that wiki page, with more detailed cases. We need to explain what > capabilities the attacker has (can he read files?can he interact with > the database? can he read memory? ..) and then explain how that works > with encrypted cluster. > > > > * perhaps easier to implement than file system encryption > > > > Not sure. IMO filesystem encryption is fairly simple to use, to the > extent that it's hard to beat. The problem is usually people can't use > it for various reasons - lack of support on their OS, no access to the > block device, problems with obtaining the privileges etc. Right, that's the "perhaps easier" part of my text above. > Having it built into the database menas you can sidestep most of those > issue (e.g. you can deploy it as a DBA, on arbitrary OS, ...). > > Plus it allows features you can't easily achieve with fs encryption, > because the filesystem only sees opaque data files. So having keys per > database/user/... is easier from within the database. Yes, but we will not be doing that for the first release because of the complexity related to handling this in WAL and requiring crash recovery to be able to unlock all the keys for replay. I personally think that database/user/... keys are best done at the SQL level, with proper locking. pgcryptokey (http://momjian.us/download/pgcryptokey/) is an example of that. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Oct 1, 2019 at 11:54:26AM -0400, Bruce Momjian wrote: > On Tue, Oct 1, 2019 at 03:43:05PM +0200, Tomas Vondra wrote: > > Plus it allows features you can't easily achieve with fs encryption, > > because the filesystem only sees opaque data files. So having keys per > > database/user/... is easier from within the database. > > Yes, but we will not be doing that for the first release because of the > complexity related to handling this in WAL and requiring crash recovery > to be able to unlock all the keys for replay. I personally think that > database/user/... keys are best done at the SQL level, with proper > locking. pgcryptokey (http://momjian.us/download/pgcryptokey/) is an > example of that. Just to give more detail. Initially, there was a desire to store keys in only one place, either in the file system or in database tables. However, it became clear that the needs of booting the server and crash recovery required file system keys, and per-user/db keys were best done at the SQL level, so that indexing can be used, and logical dumps contain the locked keys. SQL-level storage allows databases to be completely independent of other databases in terms of key storage and usage. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Oct 1, 2019 at 12:19 PM Bruce Momjian <bruce@momjian.us> wrote: > Just to give more detail. Initially, there was a desire to store keys > in only one place, either in the file system or in database tables. > However, it became clear that the needs of booting the server and crash > recovery required file system keys, and per-user/db keys were best done > at the SQL level, so that indexing can be used, and logical dumps > contain the locked keys. SQL-level storage allows databases to be > completely independent of other databases in terms of key storage and > usage. Wait, we're going to store the encryption keys with the database? It seems like you're debating whether to store your front door keys under the doormat or in a fake rock by the side of the path, when what you really ought to be doing is keeping them physically separated from the house, like in your pocket or your purse. It seems to me that the right design is that there's a configurable mechanism for PostgreSQL to request keys from someplace outside the database, and that other place is responsible for storing the keys securely and not losing them. Probably, it's a key-server of some kind running on another machine, but if you really want you can do something insecure instead, like getting them from the local filesystem. I admit I haven't been following the threads on this topic, but this just seems like a really strange idea. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Greetings, * Robert Haas (robertmhaas@gmail.com) wrote: > On Tue, Oct 1, 2019 at 12:19 PM Bruce Momjian <bruce@momjian.us> wrote: > > Just to give more detail. Initially, there was a desire to store keys > > in only one place, either in the file system or in database tables. > > However, it became clear that the needs of booting the server and crash > > recovery required file system keys, and per-user/db keys were best done > > at the SQL level, so that indexing can be used, and logical dumps > > contain the locked keys. SQL-level storage allows databases to be > > completely independent of other databases in terms of key storage and > > usage. > > Wait, we're going to store the encryption keys with the database? It > seems like you're debating whether to store your front door keys under > the doormat or in a fake rock by the side of the path, when what you > really ought to be doing is keeping them physically separated from the > house, like in your pocket or your purse. This isn't news and shouldn't be shocking- databases which support TDE all have a vaulting system for managing the keys and, yes, that's stored with the database. > It seems to me that the right design is that there's a configurable > mechanism for PostgreSQL to request keys from someplace outside the > database, and that other place is responsible for storing the keys > securely and not losing them. Probably, it's a key-server of some kind > running on another machine, but if you really want you can do > something insecure instead, like getting them from the local > filesystem. I support the option to have an external vault that's used, but I don't believe that should be a requirement and I don't think that removes the need to have a vaulting system of our own, so we can have a stand-alone TDE solution. > I admit I haven't been following the threads on this topic, but this > just seems like a really strange idea. It's not new and it's how TDE works in all of the other database systems which support it. Thanks, Stephen
Attachment
On Thu, Oct 03, 2019 at 10:43:21AM -0400, Stephen Frost wrote: >Greetings, > >* Robert Haas (robertmhaas@gmail.com) wrote: >> On Tue, Oct 1, 2019 at 12:19 PM Bruce Momjian <bruce@momjian.us> wrote: >> > Just to give more detail. Initially, there was a desire to store keys >> > in only one place, either in the file system or in database tables. >> > However, it became clear that the needs of booting the server and crash >> > recovery required file system keys, and per-user/db keys were best done >> > at the SQL level, so that indexing can be used, and logical dumps >> > contain the locked keys. SQL-level storage allows databases to be >> > completely independent of other databases in terms of key storage and >> > usage. >> >> Wait, we're going to store the encryption keys with the database? It >> seems like you're debating whether to store your front door keys under >> the doormat or in a fake rock by the side of the path, when what you >> really ought to be doing is keeping them physically separated from the >> house, like in your pocket or your purse. > >This isn't news and shouldn't be shocking- databases which support TDE >all have a vaulting system for managing the keys and, yes, that's stored >with the database. > Right. The important bit here is that the vault is encrypted, and has to be unlocked using a passphrase (or something like that) when starting the database. So it's not really as silly as a key under the doormat. >> It seems to me that the right design is that there's a configurable >> mechanism for PostgreSQL to request keys from someplace outside the >> database, and that other place is responsible for storing the keys >> securely and not losing them. Probably, it's a key-server of some kind >> running on another machine, but if you really want you can do >> something insecure instead, like getting them from the local >> filesystem. > >I support the option to have an external vault that's used, but I don't >believe that should be a requirement and I don't think that removes the >need to have a vaulting system of our own, so we can have a stand-alone >TDE solution. > Right. If anything, we need a local vault that we could use for testing. In other cases it might be a simple wrapper for a vault/keyring provided by the operating system (if it's good enough for gpg keys ...). >> I admit I haven't been following the threads on this topic, but this >> just seems like a really strange idea. > >It's not new and it's how TDE works in all of the other database systems >which support it. > Yep. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Thu, Oct 03, 2019 at 10:26:15AM -0400, Robert Haas wrote: > On Tue, Oct 1, 2019 at 12:19 PM Bruce Momjian <bruce@momjian.us> wrote: > > Just to give more detail. Initially, there was a desire to store > > keys in only one place, either in the file system or in database > > tables. However, it became clear that the needs of booting the > > server and crash recovery required file system keys, and > > per-user/db keys were best done at the SQL level, so that indexing > > can be used, and logical dumps contain the locked keys. SQL-level > > storage allows databases to be completely independent of other > > databases in terms of key storage and usage. > > Wait, we're going to store the encryption keys with the database? Encryption keys are fine there so long as decryption keys are separate. Best, David. -- David Fetter <david(at)fetter(dot)org> http://fetter.org/ Phone: +1 415 235 3778 Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate