Thread: [SQL] How to index encrypted colums ?

[SQL] How to index encrypted colums ?

From
ROS Didier
Date:

Hi

               I would to create an index on an encrypted column :

 

CREATE INDEX cartedecredit_cc_idx ON cartedecredit(pgp_pub_decrypt(cc, '--------'));

 

How can I do to indicate the private key '--------'  in the sql query ?

 

Thanks in advance

 

Best Regards


Didier ROS
DSP/CSP IT-DMA/Solutions Groupe EDF/Expertise Applicative

Expertise SGBD
32 Avenue Pablo Picasso
92000 NANTERRE

Bureau : E2 565D (aile Nord-Est)
Tél. : 01.78.66.61.14
Tél. mobile : 06.49.51.11.88


Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

Re: [SQL] How to index encrypted colums ?

From
Klaus Kaisersberger
Date:
What is the point of encrypting a column in the first place, if you then want to store the index data unencrypted?
-Klaus

From: pgsql-sql-owner@postgresql.org [mailto:pgsql-sql-owner@postgresql.org] On Behalf Of ROS Didier
Sent: Monday, October 16, 2017 3:38 PM
To: pgsql-sql-owner@postgresql.org; pgsql-sql@postgresql.org
Subject: [SQL] How to index encrypted colums ?

Hi
               I would to create an index on an encrypted column :

CREATE INDEX cartedecredit_cc_idx ON cartedecredit(pgp_pub_decrypt(cc, '--------'));

How can I do to indicate the private key '--------'  in the sql query ?

Thanks in advance

Best Regards

Didier ROS 
DSP/CSP IT-DMA/Solutions Groupe EDF/Expertise Applicative
Expertise SGBD
32 Avenue Pablo Picasso
92000 NANTERRE 
Bureau : E2 565D (aile Nord-Est)
Tél. : 01.78.66.61.14
Tél. mobile : 06.49.51.11.88

Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires
etles informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa
destination,toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.
 
Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le
divulguerou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre
système,ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous
remercionségalement d'en avertir immédiatement l'expéditeur par retour du message.
 
Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont
sécuriséesou dénuées de toute erreur ou virus.
 
____________________________________________________
This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in
thisMessage is confidential. Any use of information contained in this Message not in accord with its purpose, any
disseminationor disclosure, either whole or partial, is prohibited except formal approval.
 
If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this
messagein error, please delete it and all copies from your system and notify the sender immediately by return message.
 
E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

--
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: [SQL] How to index encrypted colums ?

From
ROS Didier
Date:
Hi Klaus
We need encryption because we need to protect sensitive data. I need to create an index on encrypted columns and I
don'tknow how to insert private key.
 
Example:
CREATE INDEX cartedecredit_cc_idx ON cartedecredit(pgp_pub_decrypt(cc, '--------'));

Best Regards
Didier ROS 
DSP/CSP IT-DMA/Solutions Groupe EDF/Expertise Applicative
Expertise SGBD
32 Avenue Pablo Picasso
92000 NANTERRE 
Bureau : E2 565D (aile Nord-Est)
Tél. : 01.78.66.61.14

-----Message d'origine-----
De : pgsql-sql-owner@postgresql.org [mailto:pgsql-sql-owner@postgresql.org] 
Envoyé : lundi 16 octobre 2017 17:47
À : pgsql-sql-owner@postgresql.org; pgsql-sql@postgresql.org
Objet : Re: [SQL] How to index encrypted colums ?

What is the point of encrypting a column in the first place, if you then want to store the index data unencrypted?
-Klaus

From: pgsql-sql-owner@postgresql.org [mailto:pgsql-sql-owner@postgresql.org] On Behalf Of ROS Didier
Sent: Monday, October 16, 2017 3:38 PM
To: pgsql-sql-owner@postgresql.org; pgsql-sql@postgresql.org
Subject: [SQL] How to index encrypted colums ?

Hi
               I would to create an index on an encrypted column :

CREATE INDEX cartedecredit_cc_idx ON cartedecredit(pgp_pub_decrypt(cc, '--------'));

How can I do to indicate the private key '--------'  in the sql query ?

Thanks in advance

Best Regards

Didier ROS
DSP/CSP IT-DMA/Solutions Groupe EDF/Expertise Applicative Expertise SGBD
32 Avenue Pablo Picasso
92000 NANTERRE
Bureau : E2 565D (aile Nord-Est)
Tél. : 01.78.66.61.14
Tél. mobile : 06.49.51.11.88

Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires
etles informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa
destination,toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.
 
Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le
divulguerou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre
système,ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous
remercionségalement d'en avertir immédiatement l'expéditeur par retour du message.
 
Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont
sécuriséesou dénuées de toute erreur ou virus.
 
____________________________________________________
This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in
thisMessage is confidential. Any use of information contained in this Message not in accord with its purpose, any
disseminationor disclosure, either whole or partial, is prohibited except formal approval.
 
If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this
messagein error, please delete it and all copies from your system and notify the sender immediately by return message.
 
E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

--
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org) To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql



Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires
etles informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa
destination,toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. 

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le
divulguerou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre
système,ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous
remercionségalement d'en avertir immédiatement l'expéditeur par retour du message. 

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont
sécuriséesou dénuées de toute erreur ou virus. 
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in
thisMessage is confidential. Any use of information contained in this Message not in accord with its purpose, any
disseminationor disclosure, either whole or partial, is prohibited except formal approval. 

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this
messagein error, please delete it and all copies from your system and notify the sender immediately by return message. 

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

--
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: [SQL] How to index encrypted colums ?

From
Steve Midgley
Date:


On Oct 17, 2017 12:49 AM, "ROS Didier" <didier.ros@edf.fr> wrote:
Hi Klaus

        We need encryption because we need to protect sensitive data. I need to create an index on encrypted columns and I don't know how to insert private key.
Example:
CREATE INDEX cartedecredit_cc_idx ON cartedecredit(pgp_pub_decrypt(cc, '--------'));

I believe that Klaus is trying to tell you that if you create an index on an encrypted column, you are storing the encrypted data in plaintext within the index. An attacker can access the secure data via the index instead of the column. 

I suppose you could partially mitigate this by putting the index in a separate table space and putting that table space on a virtual disk device that is encrypted. 

But if that meets your needs why not just put the whole table on that encrypted table space and leave the column unencrypted inside Pg? 

Steve 

Re: [SQL] How to index encrypted colums ?

From
ROS Didier
Date:

Hi

               Here is my answers to your remarks :

>> 

I believe that Klaus is trying to tell you that if you create an index on an encrypted column, you are storing the encrypted data in plaintext within the index. An attacker can access the secure data via the index instead of the column.

<< 

I am not sure about this. If I index the field without deciphering it and make the query by encrypting the search string. This permits not to expose the password or the encryption key in the definition of the index.

To do this, I need to use the encrypt () and decrypt () raw encryption functions which are immutable. With identical parameters, they always return the same value and can be used to index an encrypted column.

 

>> 

I suppose you could partially mitigate this by putting the index in a separate table space and putting that table space on a virtual disk device that is encrypted.

But if that meets your needs why not just put the whole table on that encrypted table space and leave the column unencrypted inside Pg?

<< 

Interesting recommendation, but it seems too complex for us.

 

Best Regards


Didier ROS
DSP/CSP IT-DMA/Solutions Groupe EDF/Expertise Applicative

Expertise SGBD
32 Aven
ue Pablo Picasso
92000 NANTERRE

Bureau : E2 565D (aile Nord-Est)
Tél. : 01.78.66.61.14
Tél. mobile : 06.49.51.11.88
Fax : 01.78.66.93.47

Mail : didier.ros@edf.fr
Mail du support : support-oracle-niveau3@edf.fr

Mail du support : support-postgres-niveau3@edf.fr

Site du support : https://sissi.edf.fr/web/expertise-sgbd/accueil

 

De : steve.midgley.mixrun@gmail.com [mailto:steve.midgley.mixrun@gmail.com]
Envoyé : mardi 17 octobre 2017 16:15
À : ROS Didier <didier.ros@edf.fr>
Cc : pgsql-sql-owner@postgresql.org; pgsql-sql@postgresql.org
Objet : Re: [SQL] How to index encrypted colums ?

 

 

 

On Oct 17, 2017 12:49 AM, "ROS Didier" <didier.ros@edf.fr> wrote:

Hi Klaus

        We need encryption because we need to protect sensitive data. I need to create an index on encrypted columns and I don't know how to insert private key.
Example:

CREATE INDEX cartedecredit_cc_idx ON cartedecredit(pgp_pub_decrypt(cc, '--------'));

I believe that Klaus is trying to tell you that if you create an index on an encrypted column, you are storing the encrypted data in plaintext within the index. An attacker can access the secure data via the index instead of the column. 

 

I suppose you could partially mitigate this by putting the index in a separate table space and putting that table space on a virtual disk device that is encrypted. 

 

But if that meets your needs why not just put the whole table on that encrypted table space and leave the column unencrypted inside Pg? 

 

Steve 

 


Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

Re: [SQL] How to index encrypted colums ?

From
Guillaume Lelarge
Date:
2017-10-17 17:17 GMT+02:00 ROS Didier <didier.ros@edf.fr>:

Hi

               Here is my answers to your remarks :

>> 

I believe that Klaus is trying to tell you that if you create an index on an encrypted column, you are storing the encrypted data in plaintext within the index. An attacker can access the secure data via the index instead of the column.

<< 

I am not sure about this. If I index the field without deciphering it and make the query by encrypting the search string. This permits not to expose the password or the encryption key in the definition of the index.

To do this, I need to use the encrypt () and decrypt () raw encryption functions which are immutable. With identical parameters, they always return the same value and can be used to index an encrypted column.

 


No need to encrypt/decrypt anything in that case. Just index the already ciphered content of the column and be done with it.

>> 

I suppose you could partially mitigate this by putting the index in a separate table space and putting that table space on a virtual disk device that is encrypted.

But if that meets your needs why not just put the whole table on that encrypted table space and leave the column unencrypted inside Pg?

<< 

Interesting recommendation, but it seems too complex for us.

 




--
Guillaume.

Re: [SQL] How to index encrypted colums ?

From
Klaus Kaisersberger
Date:
What I meant was, that indexing an encrypted column with a plain, unencrypted index renders the whole effort of
encryptionvoid.
 
Do _not_ do that! Otherwise, there would be no sense whatsoever in encrypting the column first.

Just for completeness, though: pgp_pub_decrypt() accepts binary, which you can specify in one of many ways, e.g.
* https://www.postgresql.org/docs/9.0/static/datatype-binary.html
* dearmor(<ASCII-armored private key>)

But once again: Do _not_ do this! It would completely negate your attempts at providing the security obviously mandated
byyour client!
 
-Klaus

-----Original Message-----
From: pgsql-sql-owner@postgresql.org [mailto:pgsql-sql-owner@postgresql.org] On Behalf Of ROS Didier
Sent: Tuesday, October 17, 2017 9:49 AM
To: pgsql-sql-owner@postgresql.org; pgsql-sql@postgresql.org
Subject: Re: [SQL] How to index encrypted colums ?

Hi Klaus
We need encryption because we need to protect sensitive data. I need to create an index on encrypted columns and I
don'tknow how to insert private key.
 
Example:
CREATE INDEX cartedecredit_cc_idx ON cartedecredit(pgp_pub_decrypt(cc, '--------'));

Best Regards
Didier ROS 
DSP/CSP IT-DMA/Solutions Groupe EDF/Expertise Applicative
Expertise SGBD
32 Avenue Pablo Picasso
92000 NANTERRE 
Bureau : E2 565D (aile Nord-Est)
Tél. : 01.78.66.61.14

-----Message d'origine-----
De : pgsql-sql-owner@postgresql.org [mailto:pgsql-sql-owner@postgresql.org] 
Envoyé : lundi 16 octobre 2017 17:47
À : pgsql-sql-owner@postgresql.org; pgsql-sql@postgresql.org
Objet : Re: [SQL] How to index encrypted colums ?

What is the point of encrypting a column in the first place, if you then want to store the index data unencrypted?
-Klaus

From: pgsql-sql-owner@postgresql.org [mailto:pgsql-sql-owner@postgresql.org] On Behalf Of ROS Didier
Sent: Monday, October 16, 2017 3:38 PM
To: pgsql-sql-owner@postgresql.org; pgsql-sql@postgresql.org
Subject: [SQL] How to index encrypted colums ?

Hi
               I would to create an index on an encrypted column :

CREATE INDEX cartedecredit_cc_idx ON cartedecredit(pgp_pub_decrypt(cc, '--------'));

How can I do to indicate the private key '--------'  in the sql query ?

Thanks in advance

Best Regards

Didier ROS
DSP/CSP IT-DMA/Solutions Groupe EDF/Expertise Applicative Expertise SGBD
32 Avenue Pablo Picasso
92000 NANTERRE
Bureau : E2 565D (aile Nord-Est)
Tél. : 01.78.66.61.14
Tél. mobile : 06.49.51.11.88

Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires
etles informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa
destination,toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.
 
Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le
divulguerou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre
système,ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous
remercionségalement d'en avertir immédiatement l'expéditeur par retour du message.
 
Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont
sécuriséesou dénuées de toute erreur ou virus.
 
____________________________________________________
This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in
thisMessage is confidential. Any use of information contained in this Message not in accord with its purpose, any
disseminationor disclosure, either whole or partial, is prohibited except formal approval.
 
If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this
messagein error, please delete it and all copies from your system and notify the sender immediately by return message.
 
E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

--
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org) To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql



Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires
etles informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa
destination,toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.
 

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le
divulguerou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre
système,ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous
remercionségalement d'en avertir immédiatement l'expéditeur par retour du message.
 

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont
sécuriséesou dénuées de toute erreur ou virus.
 
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in
thisMessage is confidential. Any use of information contained in this Message not in accord with its purpose, any
disseminationor disclosure, either whole or partial, is prohibited except formal approval.
 

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this
messagein error, please delete it and all copies from your system and notify the sender immediately by return message.
 

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

-- 
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

--
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: [SQL] How to index encrypted colums ?

From
Steve Midgley
Date:
On Tue, Oct 17, 2017 at 9:26 AM, Klaus Kaisersberger <point-of-entry@outlook.com> wrote:
What I meant was, that indexing an encrypted column with a plain, unencrypted index renders the whole effort of encryption void.
Do _not_ do that! Otherwise, there would be no sense whatsoever in encrypting the column first.

Just for completeness, though: pgp_pub_decrypt() accepts binary, which you can specify in one of many ways, e.g.
* https://www.postgresql.org/docs/9.0/static/datatype-binary.html
* dearmor(<ASCII-armored private key>)

But once again: Do _not_ do this! It would completely negate your attempts at providing the security obviously mandated by your client!
-Klaus


My response was from my wrong email address so it went to Ros but not the group. Resending to reduce confusion. In short - I think Klaus is making a really strong, important point: indexes are not encrypted. You can't index an encrypted column b/c the data in the encrypted column is basically random noise, so the index can't function properly (there's no benefit to sorting data according to random noise in an index). If you decrypt the encrypted column and then index it, you are storing the unencrypted data in the index in plaintext, which means your index is defeating the encryption for an attacker.

<<my original message>>
I believe that Klaus is trying to tell you that if you create an index on an encrypted column, you are storing the encrypted data in plaintext within the index. An attacker can access the secure data via the index instead of the column. 

I suppose you could partially mitigate this by putting the index in a separate table space and putting that table space on a virtual disk device that is encrypted. 

But if that meets your needs why not just put the whole table on that encrypted table space and leave the column unencrypted inside Pg? 

Re: [SQL] How to index encrypted colums ?

From
Klaus Kaisersberger
Date:

But then again, what you asked was quite the opposite: You explicitly tried to build a *decrypted* index! Or did I just imagine the pgp_pub_decrypt() in your original question?

Huh? Confused I am.

 

From: pgsql-sql-owner@postgresql.org [mailto:pgsql-sql-owner@postgresql.org] On Behalf Of ROS Didier
Sent: Tuesday, October 17, 2017 5:18 PM
To: steve.midgley@mixrun.com
Cc: pgsql-sql-owner@postgresql.org; pgsql-sql@postgresql.org
Subject: Re: [SQL] How to index encrypted colums ?

 

Hi

               Here is my answers to your remarks :

>> 

I believe that Klaus is trying to tell you that if you create an index on an encrypted column, you are storing the encrypted data in plaintext within the index. An attacker can access the secure data via the index instead of the column.

<< 

I am not sure about this. If I index the field without deciphering it and make the query by encrypting the search string. This permits not to expose the password or the encryption key in the definition of the index.

To do this, I need to use the encrypt () and decrypt () raw encryption functions which are immutable. With identical parameters, they always return the same value and can be used to index an encrypted column.

 

>> 

I suppose you could partially mitigate this by putting the index in a separate table space and putting that table space on a virtual disk device that is encrypted.

But if that meets your needs why not just put the whole table on that encrypted table space and leave the column unencrypted inside Pg?

<< 

Interesting recommendation, but it seems too complex for us.

 

Best Regards


Didier ROS
DSP/CSP IT-DMA/Solutions Groupe EDF/Expertise Applicative

Expertise SGBD
32 Aven
ue Pablo Picasso
92000 NANTERRE

Bureau : E2 565D (aile Nord-Est)
Tél. : 01.78.66.61.14
Tél. mobile : 06.49.51.11.88
Fax : 01.78.66.93.47

Mail : didier.ros@edf.fr
Mail du support : support-oracle-niveau3@edf.fr

Mail du support : support-postgres-niveau3@edf.fr

Site du support : https://sissi.edf.fr/web/expertise-sgbd/accueil

 

De : steve.midgley.mixrun@gmail.com [mailto:steve.midgley.mixrun@gmail.com]
Envoyé : mardi 17 octobre 2017 16:15
À : ROS Didier <didier.ros@edf.fr>
Cc : pgsql-sql-owner@postgresql.org; pgsql-sql@postgresql.org
Objet : Re: [SQL] How to index encrypted colums ?

 

 

 

On Oct 17, 2017 12:49 AM, "ROS Didier" <didier.ros@edf.fr> wrote:

Hi Klaus

        We need encryption because we need to protect sensitive data. I need to create an index on encrypted columns and I don't know how to insert private key.
Example:

CREATE INDEX cartedecredit_cc_idx ON cartedecredit(pgp_pub_decrypt(cc, '--------'));

I believe that Klaus is trying to tell you that if you create an index on an encrypted column, you are storing the encrypted data in plaintext within the index. An attacker can access the secure data via the index instead of the column. 

 

I suppose you could partially mitigate this by putting the index in a separate table space and putting that table space on a virtual disk device that is encrypted. 

 

But if that meets your needs why not just put the whole table on that encrypted table space and leave the column unencrypted inside Pg? 

 

Steve 

 


Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

Re: [SQL] How to index encrypted colums ?

From
Klaus Kaisersberger
Date:
Actually, my main advice to you is:
Please try to get a firmer grasp on the use and consequences of encryption. Currently, it seems as if you might not be
awareof the consequences of actually aiming to really encrypt data (capitalization according to RFC-principles):
 
* There is no tradeoff: You encrypt something, and then there is no way to speed up searching for actual values by
indexinganymore. Any attempt to do so is flawed security-wise by principle.
 
* You MUST NOT store encrypted data unencrypted anywhere else! Especially not in indexes.
* You MUST NOT store the private key in the same database (maybe in a storage area in an encrypted partition, but that
raisesthe question on how the partition is secured etc.pp. – security is about keeping it tiny and controlled, not
aboutspreading responsibility.)
 
* You SHOULD see to it that encrypted data is salted wherever possible.
This is but _very_ basic advice regarding security.
-Klaus

From: pgsql-sql-owner@postgresql.org [mailto:pgsql-sql-owner@postgresql.org] On Behalf Of Klaus Kaisersberger
Sent: Tuesday, October 17, 2017 8:33 PM
To: ROS Didier <didier.ros@edf.fr>; steve.midgley@mixrun.com
Cc: pgsql-sql-owner@postgresql.org; pgsql-sql@postgresql.org
Subject: Re: [SQL] How to index encrypted colums ?

But then again, what you asked was quite the opposite: You explicitly tried to build a *decrypted* index! Or did I just
imaginethe pgp_pub_decrypt() in your original question?
 
Huh? Confused I am.

From: mailto:pgsql-sql-owner@postgresql.org [mailto:pgsql-sql-owner@postgresql.org] On Behalf Of ROS Didier
Sent: Tuesday, October 17, 2017 5:18 PM
To: mailto:steve.midgley@mixrun.com
Cc: mailto:pgsql-sql-owner@postgresql.org; mailto:pgsql-sql@postgresql.org
Subject: Re: [SQL] How to index encrypted colums ?

Hi
               Here is my answers to your remarks :
>>
I believe that Klaus is trying to tell you that if you create an index on an encrypted column, you are storing the
encrypteddata in plaintext within the index. An attacker can access the secure data via the index instead of the
column.
<<
I am not sure about this. If I index the field without deciphering it and make the query by encrypting the search
string.This permits not to expose the password or the encryption key in the definition of the index.
 
To do this, I need to use the encrypt () and decrypt () raw encryption functions which are immutable. With identical
parameters,they always return the same value and can be used to index an encrypted column.
 

>>
I suppose you could partially mitigate this by putting the index in a separate table space and putting that table space
ona virtual disk device that is encrypted. 
 
But if that meets your needs why not just put the whole table on that encrypted table space and leave the column
unencryptedinside Pg?
 
<<
Interesting recommendation, but it seems too complex for us.

Best Regards

Didier ROS 
DSP/CSP IT-DMA/Solutions Groupe EDF/Expertise Applicative
Expertise SGBD
32 Avenue Pablo Picasso
92000 NANTERRE 
Bureau : E2 565D (aile Nord-Est)
Tél. : 01.78.66.61.14
Tél. mobile : 06.49.51.11.88
Fax : 01.78.66.93.47
Mail : mailto:didier.ros@edf.fr
Mail du support :mailto:support-oracle-niveau3@edf.fr
Mail du support :mailto:%20support-postgres-niveau3@edf.fr
Site du support : https://sissi.edf.fr/web/expertise-sgbd/accueil 

De : mailto:steve.midgley.mixrun@gmail.com [mailto:steve.midgley.mixrun@gmail.com] 
Envoyé : mardi 17 octobre 2017 16:15
À : ROS Didier <mailto:didier.ros@edf.fr>
Cc : mailto:pgsql-sql-owner@postgresql.org; mailto:pgsql-sql@postgresql.org
Objet : Re: [SQL] How to index encrypted colums ?



On Oct 17, 2017 12:49 AM, "ROS Didier" <mailto:didier.ros@edf.fr> wrote:
Hi Klaus

        We need encryption because we need to protect sensitive data. I need to create an index on encrypted columns
andI don't know how to insert private key.
 
Example:
CREATE INDEX cartedecredit_cc_idx ON cartedecredit(pgp_pub_decrypt(cc, '--------'));
I believe that Klaus is trying to tell you that if you create an index on an encrypted column, you are storing the
encrypteddata in plaintext within the index. An attacker can access the secure data via the index instead of the
column. 

I suppose you could partially mitigate this by putting the index in a separate table space and putting that table space
ona virtual disk device that is encrypted. 
 

But if that meets your needs why not just put the whole table on that encrypted table space and leave the column
unencryptedinside Pg? 
 

Steve 


Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires
etles informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa
destination,toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.
 
Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le
divulguerou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre
système,ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous
remercionségalement d'en avertir immédiatement l'expéditeur par retour du message.
 
Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont
sécuriséesou dénuées de toute erreur ou virus.
 
____________________________________________________
This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in
thisMessage is confidential. Any use of information contained in this Message not in accord with its purpose, any
disseminationor disclosure, either whole or partial, is prohibited except formal approval.
 
If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this
messagein error, please delete it and all copies from your system and notify the sender immediately by return message.
 
E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

--
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: [SQL] How to index encrypted colums ?

From
Klaus Kaisersberger
Date:
That being said: You can do only one thing, which might satisfy your goals (performance?) without breaking security:
Encrypt the credit card number and query the table with the pre-encrypted value. To speed things up, you could create
anindex on said encrypted column - without any decryption.
 
Thereby, the comparison would be <encrypted-value>--<stored encrypted-value>, quite performant also - and no harm
done.
-Klaus

-----Original Message-----
From: pgsql-sql-owner@postgresql.org [mailto:pgsql-sql-owner@postgresql.org] On Behalf Of Klaus Kaisersberger
Sent: Tuesday, October 17, 2017 8:54 PM
To: ROS Didier <didier.ros@edf.fr>
Cc: pgsql-sql-owner@postgresql.org; pgsql-sql@postgresql.org; steve.midgley@mixrun.com
Subject: Re: [SQL] How to index encrypted colums ?

Actually, my main advice to you is:
Please try to get a firmer grasp on the use and consequences of encryption. Currently, it seems as if you might not be
awareof the consequences of actually aiming to really encrypt data (capitalization according to RFC-principles):
 
* There is no tradeoff: You encrypt something, and then there is no way to speed up searching for actual values by
indexinganymore. Any attempt to do so is flawed security-wise by principle.
 
* You MUST NOT store encrypted data unencrypted anywhere else! Especially not in indexes.
* You MUST NOT store the private key in the same database (maybe in a storage area in an encrypted partition, but that
raisesthe question on how the partition is secured etc.pp. – security is about keeping it tiny and controlled, not
aboutspreading responsibility.)
 
* You SHOULD see to it that encrypted data is salted wherever possible.
This is but _very_ basic advice regarding security.
-Klaus

From: pgsql-sql-owner@postgresql.org [mailto:pgsql-sql-owner@postgresql.org] On Behalf Of Klaus Kaisersberger
Sent: Tuesday, October 17, 2017 8:33 PM
To: ROS Didier <didier.ros@edf.fr>; steve.midgley@mixrun.com
Cc: pgsql-sql-owner@postgresql.org; pgsql-sql@postgresql.org
Subject: Re: [SQL] How to index encrypted colums ?

But then again, what you asked was quite the opposite: You explicitly tried to build a *decrypted* index! Or did I just
imaginethe pgp_pub_decrypt() in your original question?
 
Huh? Confused I am.

From: mailto:pgsql-sql-owner@postgresql.org [mailto:pgsql-sql-owner@postgresql.org] On Behalf Of ROS Didier
Sent: Tuesday, October 17, 2017 5:18 PM
To: mailto:steve.midgley@mixrun.com
Cc: mailto:pgsql-sql-owner@postgresql.org; mailto:pgsql-sql@postgresql.org
Subject: Re: [SQL] How to index encrypted colums ?

Hi
               Here is my answers to your remarks :
>>
I believe that Klaus is trying to tell you that if you create an index on an encrypted column, you are storing the
encrypteddata in plaintext within the index. An attacker can access the secure data via the index instead of the
column.
<<
I am not sure about this. If I index the field without deciphering it and make the query by encrypting the search
string.This permits not to expose the password or the encryption key in the definition of the index.
 
To do this, I need to use the encrypt () and decrypt () raw encryption functions which are immutable. With identical
parameters,they always return the same value and can be used to index an encrypted column.
 

>>
I suppose you could partially mitigate this by putting the index in a separate table space and putting that table space
ona virtual disk device that is encrypted. 
 
But if that meets your needs why not just put the whole table on that encrypted table space and leave the column
unencryptedinside Pg?
 
<<
Interesting recommendation, but it seems too complex for us.

Best Regards

Didier ROS 
DSP/CSP IT-DMA/Solutions Groupe EDF/Expertise Applicative
Expertise SGBD
32 Avenue Pablo Picasso
92000 NANTERRE 
Bureau : E2 565D (aile Nord-Est)
Tél. : 01.78.66.61.14
Tél. mobile : 06.49.51.11.88
Fax : 01.78.66.93.47
Mail : mailto:didier.ros@edf.fr
Mail du support :mailto:support-oracle-niveau3@edf.fr
Mail du support :mailto:%20support-postgres-niveau3@edf.fr
Site du support : https://sissi.edf.fr/web/expertise-sgbd/accueil 

De : mailto:steve.midgley.mixrun@gmail.com [mailto:steve.midgley.mixrun@gmail.com] 
Envoyé : mardi 17 octobre 2017 16:15
À : ROS Didier <mailto:didier.ros@edf.fr>
Cc : mailto:pgsql-sql-owner@postgresql.org; mailto:pgsql-sql@postgresql.org
Objet : Re: [SQL] How to index encrypted colums ?



On Oct 17, 2017 12:49 AM, "ROS Didier" <mailto:didier.ros@edf.fr> wrote:
Hi Klaus

        We need encryption because we need to protect sensitive data. I need to create an index on encrypted columns
andI don't know how to insert private key.
 
Example:
CREATE INDEX cartedecredit_cc_idx ON cartedecredit(pgp_pub_decrypt(cc, '--------'));
I believe that Klaus is trying to tell you that if you create an index on an encrypted column, you are storing the
encrypteddata in plaintext within the index. An attacker can access the secure data via the index instead of the
column. 

I suppose you could partially mitigate this by putting the index in a separate table space and putting that table space
ona virtual disk device that is encrypted. 
 

But if that meets your needs why not just put the whole table on that encrypted table space and leave the column
unencryptedinside Pg? 
 

Steve 


Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires
etles informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa
destination,toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.
 
Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le
divulguerou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre
système,ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous
remercionségalement d'en avertir immédiatement l'expéditeur par retour du message.
 
Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont
sécuriséesou dénuées de toute erreur ou virus.
 
____________________________________________________
This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in
thisMessage is confidential. Any use of information contained in this Message not in accord with its purpose, any
disseminationor disclosure, either whole or partial, is prohibited except formal approval.
 
If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this
messagein error, please delete it and all copies from your system and notify the sender immediately by return message.
 
E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

-- 
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

--
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql