Thread: Hiding data in postgresql
Hi guys,
does ANYONE have any tips on hiding data on a database server? This means that data is stored in places that is not necessarily picked up in the schema of the database. I am doing some research on databases and need some direction.
Any help or direction will be highly appreciated.
Kind regards
Hector
On Mon, May 24, 2010 at 2:16 PM, Hector Beyers <hqbeyers@gmail.com> wrote: > > Hi guys, > does ANYONE have any tips on hiding data on a database server? This means > that data is stored in places that is not necessarily picked up in the > schema of the database. I am doing some research on databases and need some > direction. > Any help or direction will be highly appreciated. Let me guess: an attempt at security by obscurity? I suppose you could always create a couple of columns such that some function applied over them produced the real result (. You could even actually store this in an index, so although you could never see the result directly (except in a dump) queries to get at it might perform half reasonably. -- Peter Hunsberger
On 24 May 2010 20:16, Hector Beyers <hqbeyers@gmail.com> wrote: > > Hi guys, > does ANYONE have any tips on hiding data on a database server? This means > that data is stored in places that is not necessarily picked up in the > schema of the database. I am doing some research on databases and need some > direction. > Any help or direction will be highly appreciated. > Kind regards > Hector > Why don't role permissions do what you need? Thom
On Mon, May 24, 2010 at 1:16 PM, Hector Beyers <hqbeyers@gmail.com> wrote: > > Hi guys, > does ANYONE have any tips on hiding data on a database server? This means > that data is stored in places that is not necessarily picked up in the > schema of the database. I am doing some research on databases and need some > direction. Uhhh, initdb -D /dev/null ???
In response to Scott Marlowe <scott.marlowe@gmail.com>: > On Mon, May 24, 2010 at 1:16 PM, Hector Beyers <hqbeyers@gmail.com> wrote: > > > > Hi guys, > > does ANYONE have any tips on hiding data on a database server? This means > > that data is stored in places that is not necessarily picked up in the > > schema of the database. I am doing some research on databases and need some > > direction. On a more serious note, I would think the only way to do that would be to have columns that are misnamed and don't do what you expect. For example, have an image column that stores image data with your secret data hidden via steganography. You could do something similar with some scheme to hide data in text fields or numeric fields, but I don't know of any specific technique. Assuming you mean _hide_ and not _secure_. To secure it, encrypt it (using something like pgcrypto). But in the schema? I doubt it. PG's data storage is pretty transparent, you'd probably have to hack the source to pull that off. -- Bill Moran http://www.potentialtech.com http://people.collaborativefusion.com/~wmoran/
Wow, this is really an idea I can work with.
I know this is getting really brainstorming, but do you think it is possible to apply steganography (hiding data in pictures) tactics to the columns of a database?
Regards
Hector
On Mon, May 24, 2010 at 9:30 PM, Peter Hunsberger <peter.hunsberger@gmail.com> wrote:
Let me guess: an attempt at security by obscurity?On Mon, May 24, 2010 at 2:16 PM, Hector Beyers <hqbeyers@gmail.com> wrote:
>
> Hi guys,
> does ANYONE have any tips on hiding data on a database server? This means
> that data is stored in places that is not necessarily picked up in the
> schema of the database. I am doing some research on databases and need some
> direction.
> Any help or direction will be highly appreciated.
I suppose you could always create a couple of columns such that some
function applied over them produced the real result (. You could even
actually store this in an index, so although you could never see the
result directly (except in a dump) queries to get at it might perform
half reasonably.
--
Peter Hunsberger
Yes, I mean hide. I am approaching the problem out of the perspective of a malicious user / hacker.
On Mon, May 24, 2010 at 10:08 PM, Bill Moran <wmoran@potentialtech.com> wrote:
In response to Scott Marlowe <scott.marlowe@gmail.com>:On a more serious note, I would think the only way to do that would be
> On Mon, May 24, 2010 at 1:16 PM, Hector Beyers <hqbeyers@gmail.com> wrote:
> >
> > Hi guys,
> > does ANYONE have any tips on hiding data on a database server? This means
> > that data is stored in places that is not necessarily picked up in the
> > schema of the database. I am doing some research on databases and need some
> > direction.
to have columns that are misnamed and don't do what you expect.
For example, have an image column that stores image data with your
secret data hidden via steganography. You could do something similar
with some scheme to hide data in text fields or numeric fields, but I
don't know of any specific technique.
Assuming you mean _hide_ and not _secure_. To secure it, encrypt it
(using something like pgcrypto).
But in the schema? I doubt it. PG's data storage is pretty transparent,
you'd probably have to hack the source to pull that off.
--
Bill Moran
http://www.potentialtech.com
http://people.collaborativefusion.com/~wmoran/
On Mon, May 24, 2010 at 3:10 PM, Hector Beyers <hqbeyers@gmail.com> wrote: > Wow, this is really an idea I can work with. > I know this is getting really brainstorming, but do you think it is possible > to apply steganography (hiding data in pictures) tactics to the columns of a > database? If by that, you mean a variation on my original reply, then yes, but you'd need at least one column that did not necessarily have contents that looked like normal text. This column would work as some form of offset pointer into the other columns. The result wouldn't necessarily require much in the way of a reconstruction algorithm, but if not it would likely also be fairly easy to figure out if someone stumbled across your scheme. If you want the truly secure variation on this then use some public key / private key scheme in conjunction with this (in particular, ECC could be pretty fast and have a small key size), so that some portion of this is encrypted. However, if you're doing that, you might as well just encrypt the data directly... -- Peter Hunsberger
Dear Peter,
can you elaborate on what you mean by storing 'this' in the index. Are you referring to the function that is applied over the data?
How would you be able to see the result with a dump?
Thanks, your ideas are really helping...
Regards
Hector
On Mon, May 24, 2010 at 9:30 PM, Peter Hunsberger <peter.hunsberger@gmail.com> wrote:
Let me guess: an attempt at security by obscurity?On Mon, May 24, 2010 at 2:16 PM, Hector Beyers <hqbeyers@gmail.com> wrote:
>
> Hi guys,
> does ANYONE have any tips on hiding data on a database server? This means
> that data is stored in places that is not necessarily picked up in the
> schema of the database. I am doing some research on databases and need some
> direction.
> Any help or direction will be highly appreciated.
I suppose you could always create a couple of columns such that some
function applied over them produced the real result (. You could even
actually store this in an index, so although you could never see the
result directly (except in a dump) queries to get at it might perform
half reasonably.
--
Peter Hunsberger
On Mon, May 24, 2010 at 3:24 PM, Hector Beyers <hqbeyers@gmail.com> wrote: > Dear Peter, > can you elaborate on what you mean by storing 'this' in the index. Are you > referring to the function that is applied over the data? > How would you be able to see the result with a dump? Yes, you would the function to create the index, something like: create index gunk on tbl1 ( hidefunc( a, b, c ) ) The data created in this index might be viewable in a database dump by simple inspection with something like an editor. How meaningful it might be is another question... -- Peter Hunsberger
Acutally, that's probably the best way to this. The key to effective steganography is having a large amount of data to store a small amount of data. So, if you don't mind having a db that's 10 to 100 times bigger than it has to be to store the original data it should work. On Mon, May 24, 2010 at 2:10 PM, Hector Beyers <hqbeyers@gmail.com> wrote: > Wow, this is really an idea I can work with. > I know this is getting really brainstorming, but do you think it is possible > to apply steganography (hiding data in pictures) tactics to the columns of a > database? > Regards > Hector > > > > On Mon, May 24, 2010 at 9:30 PM, Peter Hunsberger > <peter.hunsberger@gmail.com> wrote: >> >> On Mon, May 24, 2010 at 2:16 PM, Hector Beyers <hqbeyers@gmail.com> wrote: >> > >> > Hi guys, >> > does ANYONE have any tips on hiding data on a database server? This >> > means >> > that data is stored in places that is not necessarily picked up in the >> > schema of the database. I am doing some research on databases and need >> > some >> > direction. >> > Any help or direction will be highly appreciated. >> >> Let me guess: an attempt at security by obscurity? >> >> I suppose you could always create a couple of columns such that some >> function applied over them produced the real result (. You could even >> actually store this in an index, so although you could never see the >> result directly (except in a dump) queries to get at it might perform >> half reasonably. >> >> -- >> Peter Hunsberger > > -- When fascism comes to America, it will be intolerance sold as diversity.
On Mon, May 24, 2010 at 3:16 PM, Hector Beyers <hqbeyers@gmail.com> wrote: > > Hi guys, > does ANYONE have any tips on hiding data on a database server? This means > that data is stored in places that is not necessarily picked up in the > schema of the database. I am doing some research on databases and need some > direction. > Any help or direction will be highly appreciated. First question: Have you considered 1. encrypting data when you put it in the database and 2. decrypting it when you pull it out? Let me humbly state that the #1 problem that beginners face with security and encryption is focusing too much on the mechanics and not enough on the 'big picture' issues: *) What data is to remain secret? *) Who is allowed to see the secret data? *) When do they see it? *) What sacrifices are you willing to make to keep the data secret? *) Where are you going to store the key? Answers to those questions should get you more helpful answers. Postgres has a lot of features to hide data, some obvious (pgcrypto, grant/revoke) and some not so obvious (revoking permissions from pg_proc). Judging from your question you may be interested in some extra-special techniques...please be more specific! merlin
No, I have not considered encrypting or decrypting data. The reason for this is that I am trying to secure a database by thinking like a malicious user / criminal. I want to hide (for example) fraudulent data on a database where it is not easily seen by others and then build a tool to detect this hidden data.
On your questions:
*) What data is to remain secret?
*) Who is allowed to see the secret data?
*) When do they see it?
*) What sacrifices are you willing to make to keep the data secret?
*) Where are you going to store the key?
*) Who is allowed to see the secret data?
*) When do they see it?
*) What sacrifices are you willing to make to keep the data secret?
*) Where are you going to store the key?
the answers:
- fraudulent data / or data that needs to be hidden.
- only the malicious user - and hopefully later a detection mechanism that I aim to build.
- I don't really have a preference on when they can see the data, but maybe when you export a dump.
- The main purpose of hiding the data is that the normal users of the database will not easily find the hidden data. If this criteria is met, then any other sacrifices can be made.
- Still need to figure that one out.
On Mon, May 24, 2010 at 11:04 PM, Merlin Moncure <mmoncure@gmail.com> wrote:
First question: Have you considered 1. encrypting data when you put it> Hi guys,
> does ANYONE have any tips on hiding data on a database server? This means
> that data is stored in places that is not necessarily picked up in the
> schema of the database. I am doing some research on databases and need some
> direction.
> Any help or direction will be highly appreciated.
in the database and 2. decrypting it when you pull it out?
Let me humbly state that the #1 problem that beginners face with
security and encryption is focusing too much on the mechanics and not
enough on the 'big picture' issues:
*) What data is to remain secret?
*) Who is allowed to see the secret data?
*) When do they see it?
*) What sacrifices are you willing to make to keep the data secret?
*) Where are you going to store the key?
Answers to those questions should get you more helpful answers.
Postgres has a lot of features to hide data, some obvious (pgcrypto,
grant/revoke) and some not so obvious (revoking permissions from
pg_proc). Judging from your question you may be interested in some
extra-special techniques...please be more specific!
merlin
On Mon, May 24, 2010 at 05:04:10PM -0400, Merlin Moncure wrote: > Let me humbly state that the #1 problem that beginners face with > security and encryption is focusing too much on the mechanics and not > enough on the 'big picture' issues: One more that OP seems to be avoiding is why would anybody want to do this anyway? There are plenty of places that will happily host data for you--most email sites give you many gigabytes of storage these days. Seems to be a solution in search of a problem to me. I think what the OP may be asking is about the presence of "covert channels". There are plenty of these in PG, an attacker can do the obvious things like disguising data inside other data (steganography) or more subtle things like tuple order on disk, transaction orderings, or interactions between running queries (i.e. causing one to pause for a few milliseconds by reading/locking a table). Covert channels seem to be a fundamental fact of nature. As far as I know, though I'm not aware of any papers directly on the subject, it's *always* possible to design a new attack by exploiting the physical implementation of something. Hence any specific tool you design to look for any specific attack can always be avoided in an infinite number of ways, generally negating its purpose. You have to be much more specific in your requirements before useful analysis can be done. What can be done is to reduce the bandwidth of a specific covert channel, and beyond some threshold it *may* be possible to say that "no useful data can be transmitted", but that's about it. If somebody just wants to leak a password/private key a surprisingly few number of bits will go a long way. -- Sam http://samason.me.uk/
Hi guys,
thank you for your replies yesterday on this topic. I have one more question though:
Does someone have any ideas how I can hide data without the meta data noticing? To explain further, I would like to save some collection of data where the meta-data does not see it. I am trying to do some security through obscurity. It is for research purposes.
Maybe to save populate a table with 1000 rows, but the meta-data only knows of about 500 of them? Only on an export of a dump can you find the data again.
Kind regards
Hector
On Mon, May 24, 2010 at 9:16 PM, Hector Beyers <hqbeyers@gmail.com> wrote:
Hi guys,does ANYONE have any tips on hiding data on a database server? This means that data is stored in places that is not necessarily picked up in the schema of the database. I am doing some research on databases and need some direction.Any help or direction will be highly appreciated.Kind regardsHector
Hector Beyers <hqbeyers@gmail.com> wrote: > thank you for your replies yesterday on this topic. I have one more question > though: > Does someone have any ideas how I can hide data without the meta data > noticing? To explain further, I would like to save some collection of data > where the meta-data does not see it. I am trying to do some security through > obscurity. It is for research purposes. > Maybe to save populate a table with 1000 rows, but the meta-data only knows > of about 500 of them? Only on an export of a dump can you find the data > again. > [...] Before delving deeper into this, you should get your termi- nology straight: What do you mean by "meta-data"? What do you mean by "export of a dump"? What do you mean by "without the meta data noticing"? Tim
On 5/24/2010 3:18 PM, Hector Beyers wrote: > Yes, I mean hide. I am approaching the problem out of the perspective > of a malicious user / hacker. > > **snip*** First hiding data is not a solution to secure or block access to information. This only slows people down it does not stop them, never underestimate users with access to the data It would be helpful to explain the type of data that needs to be hidden/secured Example of failed attempts to hide data is to look at the numerous mistakes in securing credit card data at many Companies. In almost every case that i have read the programmers just tried to hide the data or limit access instead of doing Public Key Private Key encryption methodology . I know of several big name apps that still store credit card data where the end users can reverse the encryption meaning if the key becomes unsecured any the data is visible that is encrypted. I have seen where the data is only encrypted inside the database so the information is transmitted in the clear to the client as the database decrypted the data on the fly . What is the point?? Trying to hide information is waste of time and energy look into encryption. All legitimate Magwerks Corporation quotations are sent in a .PDF file attachment with a unique ID number generated by ourproprietary quotation system. Quotations received via any other form of communication will not be honored. CONFIDENTIALITY NOTICE: This e-mail, including attachments, may contain legally privileged, confidential or other informationproprietary to Magwerks Corporation and is intended solely for the use of the individual to whom it addresses.If the reader of this e-mail is not the intended recipient or authorized agent, the reader is hereby notified thatany unauthorized viewing, dissemination, distribution or copying of this e-mail is strictly prohibited. If you have receivedthis e-mail in error, please notify the sender by replying to this message and destroy all occurrences of this e-mailimmediately. Thank you.
Attachment
On 5/25/2010 2:58 AM, Hector Beyers wrote:
Missed this bit prior to first responds.
I think some of the assumptions here are flawed.
If hacker actually got into a database why would they do this??? what is being accomplished??? why would anyone want to do this???
Again it would make allot more sense if a hacker stored data in plain site. Create tables that look like real tables following the same naming schema or use already existing tables like logs, Modify the tables adding columns to store data. Then create/update records encrypting the contents, this would protect the contents from ever being read by anyone except by the creator.
Think this line through how long would a Hacker go unnoticed if they used the already existing tables adding in columns or take over stale records like old customers that are no longer active. Then use the text fields to store data. The hacker could create normal user account to access those records throwing up no red flags. How many people review table structures or update to already existing records.
The current crop of hackers are not hexeditor high-school wannabe's. Hackers want to go unnoticed for as long as they can so that means doing nothing out of ordinary that throws up red flags.
Just read up on the investigations on stolen credit cards. Or fake ATMS that's been installed at malls. The hackers/thieves figured out how to go unnoticed for long periods of time by appearing normal.
Second assumption is the hacker actual got a admin/root level access to be able to do these kind of things. This means security upfront was lacks which point there are far bigger problems than hidden data.
Far better way to secure is not trying think what they can do once they get access, but stop them getting in to start with. If anyone gets this high level of access protecting from or figuring out if they have hidden data is immaterial to the problems someone has.
All legitimate Magwerks Corporation quotations are sent in a .PDF file attachment with a unique ID number generated by our proprietary quotation system. Quotations received via any other form of communication will not be honored.
CONFIDENTIALITY NOTICE: This e-mail, including attachments, may contain legally privileged, confidential or other information proprietary to Magwerks Corporation and is intended solely for the use of the individual to whom it addresses. If the reader of this e-mail is not the intended recipient or authorized agent, the reader is hereby notified that any unauthorized viewing, dissemination, distribution or copying of this e-mail is strictly prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and destroy all occurrences of this e-mail immediately.
Thank you.
No, I have not considered encrypting or decrypting data. The reason for this is that I am trying to secure a database by thinking like a malicious user / criminal. I want to hide (for example) fraudulent data on a database where it is not easily seen by others and then build a tool to detect this hidden data.On your questions:*) What data is to remain secret?
*) Who is allowed to see the secret data?
*) When do they see it?
*) What sacrifices are you willing to make to keep the data secret?
*) Where are you going to store the key?the answers:
- fraudulent data / or data that needs to be hidden.
- only the malicious user - and hopefully later a detection mechanism that I aim to build.
- I don't really have a preference on when they can see the data, but maybe when you export a dump.
- The main purpose of hiding the data is that the normal users of the database will not easily find the hidden data. If this criteria is met, then any other sacrifices can be made.
- Still need to figure that one out.
Any good brainstorming ideas will help!
Missed this bit prior to first responds.
I think some of the assumptions here are flawed.
If hacker actually got into a database why would they do this??? what is being accomplished??? why would anyone want to do this???
Again it would make allot more sense if a hacker stored data in plain site. Create tables that look like real tables following the same naming schema or use already existing tables like logs, Modify the tables adding columns to store data. Then create/update records encrypting the contents, this would protect the contents from ever being read by anyone except by the creator.
Think this line through how long would a Hacker go unnoticed if they used the already existing tables adding in columns or take over stale records like old customers that are no longer active. Then use the text fields to store data. The hacker could create normal user account to access those records throwing up no red flags. How many people review table structures or update to already existing records.
The current crop of hackers are not hexeditor high-school wannabe's. Hackers want to go unnoticed for as long as they can so that means doing nothing out of ordinary that throws up red flags.
Just read up on the investigations on stolen credit cards. Or fake ATMS that's been installed at malls. The hackers/thieves figured out how to go unnoticed for long periods of time by appearing normal.
Second assumption is the hacker actual got a admin/root level access to be able to do these kind of things. This means security upfront was lacks which point there are far bigger problems than hidden data.
Far better way to secure is not trying think what they can do once they get access, but stop them getting in to start with. If anyone gets this high level of access protecting from or figuring out if they have hidden data is immaterial to the problems someone has.
All legitimate Magwerks Corporation quotations are sent in a .PDF file attachment with a unique ID number generated by our proprietary quotation system. Quotations received via any other form of communication will not be honored.
CONFIDENTIALITY NOTICE: This e-mail, including attachments, may contain legally privileged, confidential or other information proprietary to Magwerks Corporation and is intended solely for the use of the individual to whom it addresses. If the reader of this e-mail is not the intended recipient or authorized agent, the reader is hereby notified that any unauthorized viewing, dissemination, distribution or copying of this e-mail is strictly prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and destroy all occurrences of this e-mail immediately.
Thank you.
Attachment
On Tue, May 25, 2010 at 11:30 AM, Hector Beyers <hqbeyers@gmail.com> wrote: > Maybe to save populate a table with 1000 rows, but the meta-data only knows > of about 500 of them? Only on an export of a dump can you find the data > again. Well, here's your flaw: If you expect the data dumper to be able to export the rows, then the system must know that the rows/tables are there, and thus cannot be "hidden" for any definition of hidden. To find them, you just do the work that the dumper would do. Ie, you seem to want to be able to write files that postgres doesn't know are there, and that I don't believe you can do.
On Mon, May 24, 2010 at 2:16 PM, Hector Beyers <hqbeyers@gmail.com> wrote: > > Hi guys, > does ANYONE have any tips on hiding data on a database server? This means > that data is stored in places that is not necessarily picked up in the > schema of the database. I am doing some research on databases and need some > direction. > Any help or direction will be highly appreciated. Like everyone else who has responded I am unsure exactly what you mean but it might be that you want to implement something like a virtual private database. The basic idea is that every user connects to the same database but gets to see different subsets of data depending on what rights they have. You implement this using views. No-one gets access to the underlying tables, instead having access to a secured view. The secured view on table x looks like this: create view x as select * from real.x where i_can_see(x.key); The function i_can_see() determines whether you can see a particular row. Naturally access function, i_can_see(), needs to know who a particular user is and what rights they have. This involves some careful session management, particularly in today's web-centric applications. If you are interested in this technique, then my project, veil: http://veil.projects.postgresql.org/ , provides tools for building virtual private databases in Postgres. Be warned though, this is a difficult thing to do, may have unacceptable overhead, and may still leave channels open for data compromise. __ Marc