Thread: sha1, sha2 functions into core?

sha1, sha2 functions into core?

From
Peter Eisentraut
Date:
I would like to see whether there is support for adding sha1 and sha2
functions into the core.  These are obviously well-known and widely used
functions, but currently the only way to get them is either through
pgcrypto or one of the PLs.  We could say that's OK, but then we do
support md5 in core, which then encourages people to use that, when they
really shouldn't use that for new applications.  Another weirdness is
that md5() doesn't return bytea but instead the result hex-encoded in a
string, which makes it weird to use in some cases.

One thing that might be reasonable would be to move the digest()
functions
   digest(data text, type text) returns bytea   digest(data bytea, type text) returns bytea

from pgcrypto into core, so that pgcrypto is mostly restricted to
encryption, and can be kept at arm's length for those who need to do
that.

(Side note: Would the extension mechanism be able to easily cope with a
move like that?)




Re: sha1, sha2 functions into core?

From
Tom Lane
Date:
Peter Eisentraut <peter_e@gmx.net> writes:
> I would like to see whether there is support for adding sha1 and sha2
> functions into the core.

I can't get excited about that, but could put up with it as long as
there wasn't scope creep ...

> One thing that might be reasonable would be to move the digest()
> functions
>     digest(data text, type text) returns bytea
>     digest(data bytea, type text) returns bytea
> from pgcrypto into core,

... which this approach would create, because digest() isn't restricted
to just those algorithms.  I think it'd be better to just invent two
new functions, which also avoids issues for applications that currently
expect the digest functions to be installed in pgcrypto's schema.
        regards, tom lane


Re: sha1, sha2 functions into core?

From
Andrew Dunstan
Date:

On 08/10/2011 02:06 PM, Peter Eisentraut wrote:
> I would like to see whether there is support for adding sha1 and sha2
> functions into the core.  These are obviously well-known and widely used
> functions, but currently the only way to get them is either through
> pgcrypto or one of the PLs.  We could say that's OK, but then we do
> support md5 in core, which then encourages people to use that, when they
> really shouldn't use that for new applications.  Another weirdness is
> that md5() doesn't return bytea but instead the result hex-encoded in a
> string, which makes it weird to use in some cases.
>
> One thing that might be reasonable would be to move the digest()
> functions
>
>      digest(data text, type text) returns bytea
>      digest(data bytea, type text) returns bytea
>
> from pgcrypto into core, so that pgcrypto is mostly restricted to
> encryption, and can be kept at arm's length for those who need to do
> that.
>
> (Side note: Would the extension mechanism be able to easily cope with a
> move like that?)
>

It's come up before: 
<http://archives.postgresql.org/pgsql-hackers/2009-09/msg01293.php>

+1 for returning bytea though.

cheers

andrew




Re: sha1, sha2 functions into core?

From
Robert Haas
Date:
On Wed, Aug 10, 2011 at 2:24 PM, Andrew Dunstan <andrew@dunslane.net> wrote:
> It's come up before:
> <http://archives.postgresql.org/pgsql-hackers/2009-09/msg01293.php>

I was about to wonder out loud if we might be trying to hit a moving target....

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: sha1, sha2 functions into core?

From
Dave Page
Date:
On Wed, Aug 10, 2011 at 7:06 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
> I would like to see whether there is support for adding sha1 and sha2
> functions into the core.  These are obviously well-known and widely used
> functions, but currently the only way to get them is either through
> pgcrypto or one of the PLs.  We could say that's OK, but then we do
> support md5 in core, which then encourages people to use that, when they
> really shouldn't use that for new applications.

Slightly different, but related - I've seen complaints that we only
use md5 for password storage/transmission, which is apparently not
acceptable under some government security standards. In the most
recent case, they wanted to be able to use sha256 for password storage
(transmission isn't really an issue where SSL can be used of course).

If we're ready to move more hashing functions into core, then it seems
reasonable to add more options for password storage to help those who
need to meet mandated standards.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: sha1, sha2 functions into core?

From
Peter Eisentraut
Date:
On ons, 2011-08-10 at 19:29 +0100, Dave Page wrote:
> On Wed, Aug 10, 2011 at 7:06 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
> > I would like to see whether there is support for adding sha1 and sha2
> > functions into the core.  These are obviously well-known and widely used
> > functions, but currently the only way to get them is either through
> > pgcrypto or one of the PLs.  We could say that's OK, but then we do
> > support md5 in core, which then encourages people to use that, when they
> > really shouldn't use that for new applications.
> 
> Slightly different, but related - I've seen complaints that we only
> use md5 for password storage/transmission, which is apparently not
> acceptable under some government security standards. In the most
> recent case, they wanted to be able to use sha256 for password storage
> (transmission isn't really an issue where SSL can be used of course).

Yeah, that's one of those things.  These days, using md5 for anything
raises red flags, so it would be better to slowly move some alternatives
into place.

> If we're ready to move more hashing functions into core, then it seems
> reasonable to add more options for password storage to help those who
> need to meet mandated standards.

Yes, that would be good.




Re: sha1, sha2 functions into core?

From
Peter Eisentraut
Date:
On ons, 2011-08-10 at 14:26 -0400, Robert Haas wrote:
> On Wed, Aug 10, 2011 at 2:24 PM, Andrew Dunstan <andrew@dunslane.net> wrote:
> > It's come up before:
> > <http://archives.postgresql.org/pgsql-hackers/2009-09/msg01293.php>
> 
> I was about to wonder out loud if we might be trying to hit a moving target....

I think we are dealing with a lot more moving targets than adding a new
version of SHA every 12 to 15 years.



Re: sha1, sha2 functions into core?

From
Heikki Linnakangas
Date:
On 10.08.2011 21:45, Peter Eisentraut wrote:
> On ons, 2011-08-10 at 14:26 -0400, Robert Haas wrote:
>> On Wed, Aug 10, 2011 at 2:24 PM, Andrew Dunstan<andrew@dunslane.net>  wrote:
>>> It's come up before:
>>> <http://archives.postgresql.org/pgsql-hackers/2009-09/msg01293.php>
>>
>> I was about to wonder out loud if we might be trying to hit a moving target....
>
> I think we are dealing with a lot more moving targets than adding a new
> version of SHA every 12 to 15 years.

Moving to a something more modern for internal use is one thing. But 
regarding the user-visible md5() function, how about we jump off this 
treadmill and remove it altogether? And provide a backwards-compatible 
function in pgcrypto.

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com


Re: sha1, sha2 functions into core?

From
Magnus Hagander
Date:
On Wed, Aug 10, 2011 at 21:02, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:
> On 10.08.2011 21:45, Peter Eisentraut wrote:
>>
>> On ons, 2011-08-10 at 14:26 -0400, Robert Haas wrote:
>>>
>>> On Wed, Aug 10, 2011 at 2:24 PM, Andrew Dunstan<andrew@dunslane.net>
>>>  wrote:
>>>>
>>>> It's come up before:
>>>> <http://archives.postgresql.org/pgsql-hackers/2009-09/msg01293.php>
>>>
>>> I was about to wonder out loud if we might be trying to hit a moving
>>> target....
>>
>> I think we are dealing with a lot more moving targets than adding a new
>> version of SHA every 12 to 15 years.
>
> Moving to a something more modern for internal use is one thing. But
> regarding the user-visible md5() function, how about we jump off this
> treadmill and remove it altogether? And provide a backwards-compatible
> function in pgcrypto.

-1.

There are certainly a number of perfectly valid use-cases for md5, and
it would probably break a *lot* of applications to remove it.

+1 for adding the SHA functions to core as choices, of course.


--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: sha1, sha2 functions into core?

From
Peter Eisentraut
Date:
On ons, 2011-08-10 at 14:19 -0400, Tom Lane wrote:
> > One thing that might be reasonable would be to move the digest()
> > functions
> >     digest(data text, type text) returns bytea
> >     digest(data bytea, type text) returns bytea
> > from pgcrypto into core,
> 
> ... which this approach would create, because digest() isn't
> restricted
> to just those algorithms.  I think it'd be better to just invent two
> new functions, which also avoids issues for applications that
> currently
> expect the digest functions to be installed in pgcrypto's schema.

I would also prefer to simply add sha1(bytea/text) => bytea, but the
existing md5 function is md5(bytea/text) => test, so either the new
functions would be inconsistent, or we make the new functions broken
like the old one, or we invent a different naming system, such as
digest().



Re: sha1, sha2 functions into core?

From
Magnus Hagander
Date:
On Thu, Aug 11, 2011 at 09:06, Peter Eisentraut <peter_e@gmx.net> wrote:
> On ons, 2011-08-10 at 14:19 -0400, Tom Lane wrote:
>> > One thing that might be reasonable would be to move the digest()
>> > functions
>> >     digest(data text, type text) returns bytea
>> >     digest(data bytea, type text) returns bytea
>> > from pgcrypto into core,
>>
>> ... which this approach would create, because digest() isn't
>> restricted
>> to just those algorithms.  I think it'd be better to just invent two
>> new functions, which also avoids issues for applications that
>> currently
>> expect the digest functions to be installed in pgcrypto's schema.
>
> I would also prefer to simply add sha1(bytea/text) => bytea, but the
> existing md5 function is md5(bytea/text) => test, so either the new
> functions would be inconsistent, or we make the new functions broken
> like the old one, or we invent a different naming system, such as
> digest().

You could always combine them and create digest_sha1(bytea/text) =>
bytea, etc. That still won't have the "open ended" problem of just
digest().


--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: sha1, sha2 functions into core?

From
Marko Kreen
Date:
On Wed, Aug 10, 2011 at 9:19 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Peter Eisentraut <peter_e@gmx.net> writes:
>> I would like to see whether there is support for adding sha1 and sha2
>> functions into the core.
>
> I can't get excited about that, but could put up with it as long as
> there wasn't scope creep ...
>
>> One thing that might be reasonable would be to move the digest()
>> functions
>>     digest(data text, type text) returns bytea
>>     digest(data bytea, type text) returns bytea
>> from pgcrypto into core,
>
> ... which this approach would create, because digest() isn't restricted
> to just those algorithms.  I think it'd be better to just invent two
> new functions, which also avoids issues for applications that currently
> expect the digest functions to be installed in pgcrypto's schema.

I would suggest digest() with fixed list of algorithms: md5, sha1, sha2.

The uncommon/obsolete algorithms that can be used
from digest() if compiled with openssl, are not something we
need to worry over.  In fact we have never "supported" them,
as no testing has been done.

Then we could also add hexdigest() which would fix whole bytea/hex
confusion without bloating pg_proc.

--
marko


Re: sha1, sha2 functions into core?

From
Tom Lane
Date:
Marko Kreen <markokr@gmail.com> writes:
> On Wed, Aug 10, 2011 at 9:19 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> ... which this approach would create, because digest() isn't restricted
>> to just those algorithms. �I think it'd be better to just invent two
>> new functions, which also avoids issues for applications that currently
>> expect the digest functions to be installed in pgcrypto's schema.

> I would suggest digest() with fixed list of algorithms: md5, sha1, sha2.

> The uncommon/obsolete algorithms that can be used
> from digest() if compiled with openssl, are not something we
> need to worry over.  In fact we have never "supported" them,
> as no testing has been done.

Hmm ... they may be untested by us, but I feel sure that if we remove
that functionality from pgcrypto, *somebody* is gonna complain.

I don't see anything much wrong with sha1(bytea/text) -> bytea.
There's no law that says it has to work exactly like md5() does.
        regards, tom lane


Re: sha1, sha2 functions into core?

From
Andrew Dunstan
Date:

On 08/11/2011 10:46 AM, Tom Lane wrote:
> Marko Kreen<markokr@gmail.com>  writes:
>> On Wed, Aug 10, 2011 at 9:19 PM, Tom Lane<tgl@sss.pgh.pa.us>  wrote:
>>> ... which this approach would create, because digest() isn't restricted
>>> to just those algorithms.  I think it'd be better to just invent two
>>> new functions, which also avoids issues for applications that currently
>>> expect the digest functions to be installed in pgcrypto's schema.
>> I would suggest digest() with fixed list of algorithms: md5, sha1, sha2.
>> The uncommon/obsolete algorithms that can be used
>> from digest() if compiled with openssl, are not something we
>> need to worry over.  In fact we have never "supported" them,
>> as no testing has been done.
> Hmm ... they may be untested by us, but I feel sure that if we remove
> that functionality from pgcrypto, *somebody* is gonna complain.

Yeah. Maybe we should add a test or two.

> I don't see anything much wrong with sha1(bytea/text) ->  bytea.
> There's no law that says it has to work exactly like md5() does.
>
>             

I agree. We could provide an md5_b(text/bytea) -> bytea if people are 
really concerned about orthogonality.

cheers

andrew


Re: sha1, sha2 functions into core?

From
Marko Kreen
Date:
On Thu, Aug 11, 2011 at 5:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Marko Kreen <markokr@gmail.com> writes:
>> On Wed, Aug 10, 2011 at 9:19 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> ... which this approach would create, because digest() isn't restricted
>>> to just those algorithms.  I think it'd be better to just invent two
>>> new functions, which also avoids issues for applications that currently
>>> expect the digest functions to be installed in pgcrypto's schema.
>
>> I would suggest digest() with fixed list of algorithms: md5, sha1, sha2.
>
>> The uncommon/obsolete algorithms that can be used
>> from digest() if compiled with openssl, are not something we
>> need to worry over.  In fact we have never "supported" them,
>> as no testing has been done.
>
> Hmm ... they may be untested by us, but I feel sure that if we remove
> that functionality from pgcrypto, *somebody* is gonna complain.

Well, if you are worried about that, you can duplicate current
pgcrypto behaviour - if postgres is compiled against openssl,
you get whatever algorithms are available in that particular
version of openssl.

My point was that giving such open-ended list of algorithms
was bad idea, but there is no problem keeping old behaviour.

> I don't see anything much wrong with sha1(bytea/text) -> bytea.
> There's no law that says it has to work exactly like md5() does.

The problem is that list of must-have algorithms is getting
quite long: md5, sha1, sha224, sha256, sha384, sha512,
+ at least 4 from upcoming sha3.

Another problem is that generic hashes are bad way for storing passwords
- identical passwords will look identical, and its easy to brute-force
passwords as the algorithms are very fast.

So the question is: is there actual *good* reason add each algorithm separately,
without uniform API to core functions?

If the user requests are about storing passwords, and we want to make
that easier, then we should import crypt() also, as that is the secure
way for password storage.  Then the md5(), md5_b() plus bunch of
sha-s will look silly.

--
marko


Re: sha1, sha2 functions into core?

From
"David E. Wheeler"
Date:
On Aug 12, 2011, at 5:02 AM, Marko Kreen wrote:

> My point was that giving such open-ended list of algorithms
> was bad idea, but there is no problem keeping old behaviour.
>
>> I don't see anything much wrong with sha1(bytea/text) -> bytea.
>> There's no law that says it has to work exactly like md5() does.
>
> The problem is that list of must-have algorithms is getting
> quite long: md5, sha1, sha224, sha256, sha384, sha512,
> + at least 4 from upcoming sha3.

+1

I think some sort of digest() function that takes a parameter naming the algorithm would be the way to go. That's not
tosay that the existing named functions could continue to exist -- md5() in core and sha1() in pg_crypto. But it sure
seemsto me like we ought to have just one function for digests (or 2, if we also have hexdigest()). 

Best,

David



Re: sha1, sha2 functions into core?

From
Marko Kreen
Date:
On Thu, Aug 11, 2011 at 5:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Marko Kreen <markokr@gmail.com> writes:
>> On Wed, Aug 10, 2011 at 9:19 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> ... which this approach would create, because digest() isn't restricted
>>> to just those algorithms.  I think it'd be better to just invent two
>>> new functions, which also avoids issues for applications that currently
>>> expect the digest functions to be installed in pgcrypto's schema.
>
>> I would suggest digest() with fixed list of algorithms: md5, sha1, sha2.
>
>> The uncommon/obsolete algorithms that can be used
>> from digest() if compiled with openssl, are not something we
>> need to worry over.  In fact we have never "supported" them,
>> as no testing has been done.
>
> Hmm ... they may be untested by us, but I feel sure that if we remove
> that functionality from pgcrypto, *somebody* is gonna complain.

If you dont want to break digest() but do not want such behaviour in core,
we could go with hash(data, algo) that has fixed number of digests,
but also couple non-cryptographic hashes like crc32, lookup2/3.
This would also fix the problem of people using hashtext() in user code.

--
marko


Re: sha1, sha2 functions into core?

From
"Ross J. Reedstrom"
Date:
On Fri, Aug 12, 2011 at 10:14:58PM +0300, Marko Kreen wrote:
> On Thu, Aug 11, 2011 at 5:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Marko Kreen <markokr@gmail.com> writes:
> >> On Wed, Aug 10, 2011 at 9:19 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >>> ... which this approach would create, because digest() isn't restricted
> >>> to just those algorithms.  I think it'd be better to just invent two
> >>> new functions, which also avoids issues for applications that currently
> >>> expect the digest functions to be installed in pgcrypto's schema.
> >
> >> I would suggest digest() with fixed list of algorithms: md5, sha1, sha2.
> >
> >> The uncommon/obsolete algorithms that can be used
> >> from digest() if compiled with openssl, are not something we
> >> need to worry over.  In fact we have never "supported" them,
> >> as no testing has been done.
> >
> > Hmm ... they may be untested by us, but I feel sure that if we remove
> > that functionality from pgcrypto, *somebody* is gonna complain.
> 
> If you dont want to break digest() but do not want such behaviour in core,
> we could go with hash(data, algo) that has fixed number of digests,
> but also couple non-cryptographic hashes like crc32, lookup2/3.
> This would also fix the problem of people using hashtext() in user code.
Hmm, this thread seems to have petered out without a conclusion. Just
wanted to comment that there _are_ non-password storage uses for these
digests: I use them in a context of storing large files in a bytea
column, as a means to doing data deduplication, and avoiding pushing
files from clients to server and back.

Ross
-- 
Ross Reedstrom, Ph.D.                                 reedstrm@rice.edu
Systems Engineer & Admin, Research Scientist        phone: 713-348-6166
Connexions                  http://cnx.org            fax: 713-348-3665
Rice University MS-375, Houston, TX 77005
GPG Key fingerprint = F023 82C8 9B0E 2CC6 0D8E  F888 D3AE 810E 88F0 BEDE


Re: sha1, sha2 functions into core?

From
Daniel Farina
Date:
On Wed, Aug 31, 2011 at 11:12 AM, Ross J. Reedstrom <reedstrm@rice.edu> wrote:
> Hmm, this thread seems to have petered out without a conclusion. Just
> wanted to comment that there _are_ non-password storage uses for these
> digests: I use them in a context of storing large files in a bytea
> column, as a means to doing data deduplication, and avoiding pushing
> files from clients to server and back.

Yes, agreed: there is no decent content-addressing type in PostgreSQL,
so one rolls their own using shas and joins; I've seen this more than
once. It's a useful way to get non-bloated index on a series of
(larger than sha1) values where one only cares about the equality
operator (hash indexes, as unattractive as they were before in
PostgreSQL's implementation are even less so now with streaming
replication).

When that content to be addressed can be submitted from another
source, anything with md5 is correctly met with suspicion. We have
gone to the trouble of using pgcrypto to get sha1 access, but I know
of other applications that would have preferred to use sha but settle
for md5 simply because it's known to be bundled in core everywhere.

CREATE EXTENSION -- particularly if there is *any* way (is there? even
with ugliness like utility statement hooks) to configure it on the
provider end to not require superuser for common extensions like
'pgcrypto' -- could ablate this issue and one could get off the hash
"treadmill", including md5 -- but I think that would be a mistake.
Applications need a high quality digest to enable any kind of
principled content addressing use case, and I think making that any
harder than a builtin is going to negatively impact the state of
things at large.  As a compromise, I'd also be happy with making
CREATE EXTENSION so trivial that everyone who has that use case can
get pgcrypto on any hosting provider.

-- 
fdr


Re: sha1, sha2 functions into core?

From
Peter Eisentraut
Date:
On ons, 2011-08-31 at 13:12 -0500, Ross J. Reedstrom wrote:
> Hmm, this thread seems to have petered out without a conclusion. Just
> wanted to comment that there _are_ non-password storage uses for these
> digests: I use them in a context of storing large files in a bytea
> column, as a means to doing data deduplication, and avoiding pushing
> files from clients to server and back.

But I suppose you don't need the hash function in the database system
for that.



Re: sha1, sha2 functions into core?

From
"ktm@rice.edu"
Date:
On Fri, Sep 02, 2011 at 09:54:07PM +0300, Peter Eisentraut wrote:
> On ons, 2011-08-31 at 13:12 -0500, Ross J. Reedstrom wrote:
> > Hmm, this thread seems to have petered out without a conclusion. Just
> > wanted to comment that there _are_ non-password storage uses for these
> > digests: I use them in a context of storing large files in a bytea
> > column, as a means to doing data deduplication, and avoiding pushing
> > files from clients to server and back.
> 
> But I suppose you don't need the hash function in the database system
> for that.
> 

It is very useful to have the same hash function used internally by
PostgreSQL exposed externally. I know you can get the code and add an
equivalent one of your own...

Regards,
Ken


Re: sha1, sha2 functions into core?

From
"Ross J. Reedstrom"
Date:
On Fri, Sep 02, 2011 at 02:05:45PM -0500, ktm@rice.edu wrote:
> On Fri, Sep 02, 2011 at 09:54:07PM +0300, Peter Eisentraut wrote:
> > On ons, 2011-08-31 at 13:12 -0500, Ross J. Reedstrom wrote:
> > > Hmm, this thread seems to have petered out without a conclusion. Just
> > > wanted to comment that there _are_ non-password storage uses for these
> > > digests: I use them in a context of storing large files in a bytea
> > > column, as a means to doing data deduplication, and avoiding pushing
> > > files from clients to server and back.
> > 
> > But I suppose you don't need the hash function in the database system
> > for that.
> > 
> 
> It is very useful to have the same hash function used internally by
> PostgreSQL exposed externally. I know you can get the code and add an
> equivalent one of your own...
> 
Thanks for the support Ken, but Peter's right: the only backend use in
my particular case is to let the backend do the hash calc during bulk
loads: in the production code path, having the hash in two places
doesn't save any work, since the client code has to calculate the hash
in order to test for its existence in the backend. I suppose if the
network cost was negligable, I could just push the files anyway, and
have a before-insert trigger calculate the hash and do the dedup: then
it'd be hidden in the backend completely. But as is, I can do all the
work in the client.

Ross
-- 
Ross Reedstrom, Ph.D.                                 reedstrm@rice.edu
Systems Engineer & Admin, Research Scientist        phone: 713-348-6166
Connexions                  http://cnx.org            fax: 713-348-3665
Rice University MS-375, Houston, TX 77005
GPG Key fingerprint = F023 82C8 9B0E 2CC6 0D8E  F888 D3AE 810E 88F0 BEDE


Re: sha1, sha2 functions into core?

From
"ktm@rice.edu"
Date:
On Fri, Sep 02, 2011 at 04:27:46PM -0500, Ross J. Reedstrom wrote:
> On Fri, Sep 02, 2011 at 02:05:45PM -0500, ktm@rice.edu wrote:
> > On Fri, Sep 02, 2011 at 09:54:07PM +0300, Peter Eisentraut wrote:
> > > On ons, 2011-08-31 at 13:12 -0500, Ross J. Reedstrom wrote:
> > > > Hmm, this thread seems to have petered out without a conclusion. Just
> > > > wanted to comment that there _are_ non-password storage uses for these
> > > > digests: I use them in a context of storing large files in a bytea
> > > > column, as a means to doing data deduplication, and avoiding pushing
> > > > files from clients to server and back.
> > > 
> > > But I suppose you don't need the hash function in the database system
> > > for that.
> > > 
> > 
> > It is very useful to have the same hash function used internally by
> > PostgreSQL exposed externally. I know you can get the code and add an
> > equivalent one of your own...
> > 
> Thanks for the support Ken, but Peter's right: the only backend use in
> my particular case is to let the backend do the hash calc during bulk
> loads: in the production code path, having the hash in two places
> doesn't save any work, since the client code has to calculate the hash
> in order to test for its existence in the backend. I suppose if the
> network cost was negligable, I could just push the files anyway, and
> have a before-insert trigger calculate the hash and do the dedup: then
> it'd be hidden in the backend completely. But as is, I can do all the
> work in the client.
> 

While it is true that it doesn't save any work. My motivation for having
it exposed is that "good" hash functions are non-trivial to find. I have
dealt with computational artifacts produced by hash functions that seemed
at first to be good. We use a very well behaved function within the data-
base and exposing it will help prevent bad user hash function
implementations.

Regards,
Ken


Re: sha1, sha2 functions into core?

From
Bruce Momjian
Date:
Is there a TODO here?

---------------------------------------------------------------------------

On Wed, Aug 10, 2011 at 09:43:18PM +0300, Peter Eisentraut wrote:
> On ons, 2011-08-10 at 19:29 +0100, Dave Page wrote:
> > On Wed, Aug 10, 2011 at 7:06 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
> > > I would like to see whether there is support for adding sha1 and sha2
> > > functions into the core.  These are obviously well-known and widely used
> > > functions, but currently the only way to get them is either through
> > > pgcrypto or one of the PLs.  We could say that's OK, but then we do
> > > support md5 in core, which then encourages people to use that, when they
> > > really shouldn't use that for new applications.
> > 
> > Slightly different, but related - I've seen complaints that we only
> > use md5 for password storage/transmission, which is apparently not
> > acceptable under some government security standards. In the most
> > recent case, they wanted to be able to use sha256 for password storage
> > (transmission isn't really an issue where SSL can be used of course).
> 
> Yeah, that's one of those things.  These days, using md5 for anything
> raises red flags, so it would be better to slowly move some alternatives
> into place.
> 
> > If we're ready to move more hashing functions into core, then it seems
> > reasonable to add more options for password storage to help those who
> > need to meet mandated standards.
> 
> Yes, that would be good.
> 
> 
> 
> -- 
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + It's impossible for everything to be true. +



Re: sha1, sha2 functions into core?

From
Marko Kreen
Date:
On Wed, Aug 15, 2012 at 6:11 AM, Bruce Momjian <bruce@momjian.us> wrote:
> Is there a TODO here?

There is still open ToDecide here:

1) Status quo - md5() in core, everything else in pgcrypto

2) Start moving other hashes as separate functions into core: eg. sha1()
Problems:
- sha1 will be obsolete soon, like md5
- many newer hashes: sha2 family, upcoming sha3 family
- hex vs. binary api issue - hex-by-default in not good when you actually need cryptographic hash (eg. indexes)
- does not solve the password storage problem.

3) Move digest() from pgcrypto into core, with same api.
Problems:
- does not solve the password storage problem.

4) Move both digest() and crypt() into core, with same api.


Password problem - the cryptographic hashes are meant
for universal usage, thus they need to be usable even
on megabytes of data.  This means they are easily
bruteforceable, when the amount of data is microscopic
(1..16 chars).  Also they are unsalted, thus making
cracking even easier.  crypt() is better api for passwords.

So when the main need to have hashes is password
storage, then 2) is bad choice.

My vote: 4), 1)

-- 
marko



Re: sha1, sha2 functions into core?

From
Tom Lane
Date:
Marko Kreen <markokr@gmail.com> writes:
> On Wed, Aug 15, 2012 at 6:11 AM, Bruce Momjian <bruce@momjian.us> wrote:
>> Is there a TODO here?

> There is still open ToDecide here: [snip]

The argument against moving crypto code into core remains the same as it
was, ie export regulations.  I don't see that that situation has changed
at all.  Thus, I think we should leave all the pgcrypto code where it
is, in an extension that's easily separated out by anybody who's
concerned about legal restrictions.  The recent improvements in the ease
of installing extensions have made it even less interesting than it used
to be to merge extension-supported code into core --- if anything, we
ought to be trying to move functionality the other way.

If anybody's concerned about the security of our password storage,
they'd be much better off working on improving the length and randomness
of the salt string than replacing the md5 hash per se.
        regards, tom lane



Re: sha1, sha2 functions into core?

From
Joe Conway
Date:
On 08/15/2012 06:48 AM, Tom Lane wrote:
>> On Wed, Aug 15, 2012 at 6:11 AM, Bruce Momjian <bruce@momjian.us> wrote:
>>> Is there a TODO here?
> 
> If anybody's concerned about the security of our password storage,
> they'd be much better off working on improving the length and randomness
> of the salt string than replacing the md5 hash per se.

Or change to an md5 HMAC rather than straight md5 with salt. Last I
checked (which admittedly was a while ago) there were still no known
cryptographic weaknesses associated with an HMAC based on md5.

Joe

-- 
Joe Conway
credativ LLC: http://www.credativ.us
Linux, PostgreSQL, and general Open Source
Training, Service, Consulting, & 24x7 Support



Re: sha1, sha2 functions into core?

From
Andrew Dunstan
Date:
On 08/15/2012 11:22 AM, Joe Conway wrote:
> On 08/15/2012 06:48 AM, Tom Lane wrote:
>>> On Wed, Aug 15, 2012 at 6:11 AM, Bruce Momjian <bruce@momjian.us> wrote:
>>>> Is there a TODO here?
>> If anybody's concerned about the security of our password storage,
>> they'd be much better off working on improving the length and randomness
>> of the salt string than replacing the md5 hash per se.
> Or change to an md5 HMAC rather than straight md5 with salt. Last I
> checked (which admittedly was a while ago) there were still no known
> cryptographic weaknesses associated with an HMAC based on md5.
>



Possibly. I still think the right time to revisit this whole area will 
be when the NIST Hash Function competition ends supposedly later this 
year. See <http://csrc.nist.gov/groups/ST/hash/timeline.html>. At that 
time we should probably consider moving our password handling to use the 
new standard function.

cheers

andrew





Re: sha1, sha2 functions into core?

From
Bruce Momjian
Date:
On Wed, Aug 15, 2012 at 11:37:04AM -0400, Andrew Dunstan wrote:
> 
> On 08/15/2012 11:22 AM, Joe Conway wrote:
> >On 08/15/2012 06:48 AM, Tom Lane wrote:
> >>>On Wed, Aug 15, 2012 at 6:11 AM, Bruce Momjian <bruce@momjian.us> wrote:
> >>>>Is there a TODO here?
> >>If anybody's concerned about the security of our password storage,
> >>they'd be much better off working on improving the length and randomness
> >>of the salt string than replacing the md5 hash per se.
> >Or change to an md5 HMAC rather than straight md5 with salt. Last I
> >checked (which admittedly was a while ago) there were still no known
> >cryptographic weaknesses associated with an HMAC based on md5.
> >
> 
> 
> 
> Possibly. I still think the right time to revisit this whole area
> will be when the NIST Hash Function competition ends supposedly
> later this year. See
> <http://csrc.nist.gov/groups/ST/hash/timeline.html>. At that time we
> should probably consider moving our password handling to use the new
> standard function.

Are we really going to be comforable with a algorithm that is new?

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + It's impossible for everything to be true. +



Re: sha1, sha2 functions into core?

From
Merlin Moncure
Date:
On Wed, Aug 15, 2012 at 10:22 AM, Joe Conway <mail@joeconway.com> wrote:
> On 08/15/2012 06:48 AM, Tom Lane wrote:
>>> On Wed, Aug 15, 2012 at 6:11 AM, Bruce Momjian <bruce@momjian.us> wrote:
>>>> Is there a TODO here?
>>
>> If anybody's concerned about the security of our password storage,
>> they'd be much better off working on improving the length and randomness
>> of the salt string than replacing the md5 hash per se.
>
> Or change to an md5 HMAC rather than straight md5 with salt. Last I
> checked (which admittedly was a while ago) there were still no known
> cryptographic weaknesses associated with an HMAC based on md5.

There is no cryptographic with md5 either really.  The best known
attack IIRC is 2^123 (well outside of any practical brute force
search) and the algorithm is very well studied.   The main issue with
md5 is that it's fast enough that you can search low entropy passwords
(rainbow tables etc) which does not depend on the strength of the
hashing algorithm.  If the hacker has access to the salt, then it will
only slow him/her down somewhat because the search will be have to be
restarted for each password.

The sha family are engineered to be fast and are therefore not
meaningfully safer IMO.  Ditto NIST hash function (upcoming sha-3)
that Andrew is mentioning downthread (that might be a good idea for
other reasons but I don't really think it's better in terms of
securing user password).  If you want to give the user good password
security, I think you have only two choices:

1) allow use hmac as you suggest (but this forces user to maintain
additional password or some token)
2) force or at least strongly encourage user to choose high entropy password

A lot of people argue for
3) use a purposefully slow hashing function like bcrypt.

but I disagree: I don't like any scheme that encourages use of low
entropy passwords.

merlin



Re: sha1, sha2 functions into core?

From
Andrew Dunstan
Date:
On 08/15/2012 11:48 AM, Bruce Momjian wrote:
> On Wed, Aug 15, 2012 at 11:37:04AM -0400, Andrew Dunstan wrote:
>> On 08/15/2012 11:22 AM, Joe Conway wrote:
>>> On 08/15/2012 06:48 AM, Tom Lane wrote:
>>>>> On Wed, Aug 15, 2012 at 6:11 AM, Bruce Momjian <bruce@momjian.us> wrote:
>>>>>> Is there a TODO here?
>>>> If anybody's concerned about the security of our password storage,
>>>> they'd be much better off working on improving the length and randomness
>>>> of the salt string than replacing the md5 hash per se.
>>> Or change to an md5 HMAC rather than straight md5 with salt. Last I
>>> checked (which admittedly was a while ago) there were still no known
>>> cryptographic weaknesses associated with an HMAC based on md5.
>>>
>>
>>
>> Possibly. I still think the right time to revisit this whole area
>> will be when the NIST Hash Function competition ends supposedly
>> later this year. See
>> <http://csrc.nist.gov/groups/ST/hash/timeline.html>. At that time we
>> should probably consider moving our password handling to use the new
>> standard function.
> Are we really going to be comforable with a algorithm that is new?
>


The only thing that will be new about it will be that it's the new 
standard. There is a reason these crypto function competitions runs for 
quite a few years.

cheers

andrew




Re: sha1, sha2 functions into core?

From
Joe Conway
Date:
On 08/15/2012 08:49 AM, Merlin Moncure wrote:
> 1) allow use hmac as you suggest (but this forces user to maintain
> additional password or some token)

Not really. You would store a token and the HMAC of the token using the
user password on the server side. You would need access to the hash
function on the client side as well. On authentication the server sends
the token to the client, and the client calculates the HMAC using the
user provided password. The result is sent back to the server for
comparison. This way the user's password is never actually sent over the
wire.

Now this is still susceptible to a replay attack, but you can fix that
by adding another layer. On authentication the server generates a new
nonce (random token) and sends it to the client along with the stored
token, as well as calculating the HMAC of the nonce using the stored
user HMAC as the key. On the client side the the process is repeated --
HMAC(nonce,HMAC(token,password)). This provides a one time calculation
preventing replay and does not expose the user's password or token-HMAC
over the wire.

The final problem as you stated is weak passwords and some kind of
dictionary attack against a stolen set of tokens and HMACs. Didn't we
add a hook some time ago for user provided password checker?

Joe


-- 
Joe Conway
credativ LLC: http://www.credativ.us
Linux, PostgreSQL, and general Open Source
Training, Service, Consulting, & 24x7 Support



Re: sha1, sha2 functions into core?

From
Bruce Momjian
Date:
On Wed, Aug 15, 2012 at 09:18:48AM -0700, Joe Conway wrote:
> The final problem as you stated is weak passwords and some kind of
> dictionary attack against a stolen set of tokens and HMACs. Didn't we
> add a hook some time ago for user provided password checker?

Yes, contrib/passwordcheck:
http://www.postgresql.org/docs/9.2/static/passwordcheck.html

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + It's impossible for everything to be true. +



Re: sha1, sha2 functions into core?

From
Marko Kreen
Date:
On Wed, Aug 15, 2012 at 4:48 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Marko Kreen <markokr@gmail.com> writes:
>> On Wed, Aug 15, 2012 at 6:11 AM, Bruce Momjian <bruce@momjian.us> wrote:
>>> Is there a TODO here?
>
>> There is still open ToDecide here: [snip]
>
> The argument against moving crypto code into core remains the same as it
> was, ie export regulations.  I don't see that that situation has changed
> at all.  Thus, I think we should leave all the pgcrypto code where it
> is, in an extension that's easily separated out by anybody who's
> concerned about legal restrictions.  The recent improvements in the ease
> of installing extensions have made it even less interesting than it used
> to be to merge extension-supported code into core --- if anything, we
> ought to be trying to move functionality the other way.

Ok.

> If anybody's concerned about the security of our password storage,
> they'd be much better off working on improving the length and randomness
> of the salt string than replacing the md5 hash per se.

Sorry, I was not clear enough - by "password storage" I meant password
storage for application-specific passwords, not postgres users.

Postgres own usage of md5 is kind of fine, as:
- only superusers can see the hashes (pg_shadow).
- if somebody sees contents of pg_shadow, they don't need to crack them, they can use hashes to log in immediately.

But for storage of application passwords, the hash needs
to be one-way and hard to crack, to decrease any damage
caused by leaks.

-- 
marko



Re: sha1, sha2 functions into core?

From
Josh Berkus
Date:
On 8/15/12 6:48 AM, Tom Lane wrote:
> The argument against moving crypto code into core remains the same as it
> was, ie export regulations.  I don't see that that situation has changed
> at all.

Actually, I believe that it has, based on my experience getting an
export certificate for Sun Postgres back in 2008.

The US Federal government lifted restrictions on shipping well-known
cryptographic algorithms to most countries several years ago, except to
specific countries with embargoes (Iran, Burma, etc.).  However, *all*
exports of software to those embargoed countries are restricted,
cryptographic or not.

The USA does require an export certificate for any
cryptographic-supporting software which is shipped from the USA.  For
that, however, MD5 and our support for SSL authentication already
requires a certificate, whether we include SHA or not.  So, my personal
non-lawyer experience is that including SHA in core or not would make no
difference whatsoever to our export status.

The above is all secondhand legal knowledge, so if it really matters to
our decisions on what algorithms we include in Core, we should ask SFLC
for a real opinion.  We certainly shouldn't make one based on assumptions.

I think it's more significant, though, that nobody has been able to
demonstrate that SHA hashing of passwords actually makes Postgres more
secure.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com



Re: sha1, sha2 functions into core?

From
Andrew Dunstan
Date:
On 08/20/2012 03:10 PM, Josh Berkus wrote:
> On 8/15/12 6:48 AM, Tom Lane wrote:
>> The argument against moving crypto code into core remains the same as it
>> was, ie export regulations.  I don't see that that situation has changed
>> at all.
> Actually, I believe that it has, based on my experience getting an
> export certificate for Sun Postgres back in 2008.
>
> The US Federal government lifted restrictions on shipping well-known
> cryptographic algorithms to most countries several years ago, except to
> specific countries with embargoes (Iran, Burma, etc.).  However, *all*
> exports of software to those embargoed countries are restricted,
> cryptographic or not.
>
> The USA does require an export certificate for any
> cryptographic-supporting software which is shipped from the USA.  For
> that, however, MD5 and our support for SSL authentication already
> requires a certificate, whether we include SHA or not.  So, my personal
> non-lawyer experience is that including SHA in core or not would make no
> difference whatsoever to our export status.
>
> The above is all secondhand legal knowledge, so if it really matters to
> our decisions on what algorithms we include in Core, we should ask SFLC
> for a real opinion.  We certainly shouldn't make one based on assumptions.
>
> I think it's more significant, though, that nobody has been able to
> demonstrate that SHA hashing of passwords actually makes Postgres more
> secure.
>



I don't think US export regulations are the only issue. Some other 
countries (mostly the usual suspects) forbid the use of crypto software. 
If we build more crypto functions into the core we make it harder to use 
Postgres legally in those places.

cheers

andrew



Re: sha1, sha2 functions into core?

From
Josh Berkus
Date:
> I don't think US export regulations are the only issue. Some other
> countries (mostly the usual suspects) forbid the use of crypto software.
> If we build more crypto functions into the core we make it harder to use
> Postgres legally in those places.

Again, that sounds like we need an actual legal opinion if we're going
to make a decision on that basis.  So let's make the decision on whether
we even *want* SHA in core, and if we do, we can ask our
attorneys/community if it's a legal problem.


-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com



Re: sha1, sha2 functions into core?

From
"Joshua D. Drake"
Date:
On 08/20/2012 01:21 PM, Josh Berkus wrote:
>
>
>> I don't think US export regulations are the only issue. Some other
>> countries (mostly the usual suspects) forbid the use of crypto software.
>> If we build more crypto functions into the core we make it harder to use
>> Postgres legally in those places.

I fail to see how that is our problem. We shouldn't make the software 
less useful because of those places.

JD




-- 
Command Prompt, Inc. - http://www.commandprompt.com/
PostgreSQL Support, Training, Professional Services and Development
High Availability, Oracle Conversion, Postgres-XC
@cmdpromptinc - 509-416-6579



Re: sha1, sha2 functions into core?

From
Andrew Dunstan
Date:
On 08/20/2012 04:26 PM, Joshua D. Drake wrote:
>
> On 08/20/2012 01:21 PM, Josh Berkus wrote:
>>
>>
>>> I don't think US export regulations are the only issue. Some other
>>> countries (mostly the usual suspects) forbid the use of crypto 
>>> software.
>>> If we build more crypto functions into the core we make it harder to 
>>> use
>>> Postgres legally in those places.
>
> I fail to see how that is our problem. We shouldn't make the software 
> less useful because of those places.
>
>

But there is absolutely no evidence that we are making it less useful. 
Postgres is designed top be extensible and we've just enhanced that. 
pgcrypto makes use of that. If we cen leverage that to make Postgres 
available to more people then why would we not do so?

cheers

andrew





Re: sha1, sha2 functions into core?

From
"Joshua D. Drake"
Date:
On 08/20/2012 01:33 PM, Andrew Dunstan wrote:

> But there is absolutely no evidence that we are making it less useful.
> Postgres is designed top be extensible and we've just enhanced that.
> pgcrypto makes use of that. If we cen leverage that to make Postgres
> available to more people then why would we not do so?

O.k. that is valid a valid argument. Let me counter.

Everybody else does it, why don't we? PostgreSQL is extensible, modular 
and programmable, why are we limiting those features by not including 
them in core? Contrib, whether we like it or not, is not core.

For some things it makes absolute sense to keep them in contrib or pgxn 
but cryptography is pretty much a basic core feature set at this point.

MySQL, MSSQL, Oracle (not sure if integrated or as a pack) and not to 
mention Java and Python all have them integrated.

Sincerely,

Joshua D. Drake



-- 
Command Prompt, Inc. - http://www.commandprompt.com/
PostgreSQL Support, Training, Professional Services and Development
High Availability, Oracle Conversion, Postgres-XC
@cmdpromptinc - 509-416-6579



Re: sha1, sha2 functions into core?

From
Peter Geoghegan
Date:
On 20 August 2012 21:26, Joshua D. Drake <jd@commandprompt.com> wrote:
>
> On 08/20/2012 01:21 PM, Josh Berkus wrote:
>>
>>
>>
>>> I don't think US export regulations are the only issue. Some other
>>> countries (mostly the usual suspects) forbid the use of crypto software.
>>> If we build more crypto functions into the core we make it harder to use
>>> Postgres legally in those places.
>
>
> I fail to see how that is our problem. We shouldn't make the software less
> useful because of those places.

Agreed.

I find the idea of some secret policeman urging the use of MySQL
because it doesn't have a built-in SHA-1 cryptographic hash function
seems extremely far-fetched. The BitTorrent protocol uses SHA-1 to
validate chunks, and it has been variously estimated that 10% - 50% of
all internet traffic is BitTorrent traffic. SHA-1 is also integral to
the way that git makes content effectively tamper-proof:

http://www.youtube.com/watch?v=4XpnKHJAok8#t=56m

-- 
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services



Re: sha1, sha2 functions into core?

From
"Greg Sabino Mullane"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160


> If the hacker has access to the salt, then it will only slow 
> him/her down somewhat because the search will be have to be
> restarted for each password.

This. Further, anyone using MD5 or SHA* or any hash function for 
any serious storage of passwords is nuts, in this day and age. 
GPUs and rentable cloud computers means the ability to test billions 
of passwords per second is easy for anyone, salted or not.

The issue is not Postgres' internal use of MD5 for passwords - that's 
a red herring, as it is basically no more relatively secure/insecure 
versus any other hashing algorithm that is not designed to be 
slow (e.g. bcrypt, scrypt, PBKDF2). The issue is simply exposing a 
more useful day to day algorithm by default. Much of the world uses 
SHA instead of MD5 these days for all sorts of purposes.

So I am torn on this. On the one hand, having a few more things in core 
would be very nice, as it seems silly we have md5() as a builtin but 
sha256() requires a special module. But once you add sha* in, why not 
AES? Blowfish? Why not go the whole way and include some extremely 
useful ones such as bcrypt? At that point, we've deprecated pg_crypto 
and moved everything to core. Why I personally would love to see that 
someday (then we can boast "built-in crypto" :), I recognize that will 
be a very tough sell. So I will take the addition of whatever we can, 
including just a sha() as this thread asked for.

> 3) use a purposefully slow hashing function like bcrypt.
>
> but I disagree: I don't like any scheme that encourages use of low
> entropy passwords.

Perhaps off-topic, but how to do you figure that?

- -- 
Greg Sabino Mullane greg@turnstep.com
End Point Corporation http://www.endpoint.com/
PGP Key: 0x14964AC8 201208201849
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAlAywBwACgkQvJuQZxSWSsiS4QCbBC7X9MyQgVKC3DTKgjv0aj7D
ik0AoNh1YBmhuaMXEKOP7z/GEBUR+EHe
=54A2
-----END PGP SIGNATURE-----





Re: sha1, sha2 functions into core?

From
Tom Lane
Date:
"Joshua D. Drake" <jd@commandprompt.com> writes:
> On 08/20/2012 01:33 PM, Andrew Dunstan wrote:
>> But there is absolutely no evidence that we are making it less useful.
>> Postgres is designed top be extensible and we've just enhanced that.
>> pgcrypto makes use of that. If we cen leverage that to make Postgres
>> available to more people then why would we not do so?

> O.k. that is valid a valid argument. Let me counter.

> Everybody else does it, why don't we? PostgreSQL is extensible, modular 
> and programmable, why are we limiting those features by not including 
> them in core? Contrib, whether we like it or not, is not core.

Nonsense.  By that argument, all the sweat we've expended on
extensibility was wasted effort, and everything should be in core.

pg_crypto's functionality is perfectly fine where it is.  The fact that
there might be some contexts where people actively don't want the
functionality in core is just a small extra reason not to be in a hurry
to merge it --- but even without that, I'd vote against this on overall
project management grounds.  We should be looking to push decouplable
bits of functionality *out* of core, not bring them back in.

The only reason I can see for pushing more crypto into core is
if we needed to stop using MD5 for the core password authentication
functionality.  While that might come to pass eventually, I am aware of
no evidence whatever that SHAn, per se, is an improvement over MD5 for
password auth purposes.  Moreover, as Josh just mentioned, anybody who
thinks it might be insufficiently secure for their purposes has got
plenty of alternatives available today (SSL certificates, PAM backed
by whatever-you-want, etc).

TBH, I think if we do anything at all about this in the near future,
it'll be window dressing to shut up the people who heard once that MD5
was insecure and know nothing about it beyond that --- but if Postgres
uses MD5 for passwords, it must be insecure.  So I tend to agree with
Andrew that we should wait till the NIST competition dust settles; but
what I'll be looking for afterwards is which algorithm has the most
street cred with the average slashdotter.

Also, as I mentioned upthread, we need to do more than just drop in
a new hashing algorithm.  MD5 is far from being the weakest link
in what we're doing today.
        regards, tom lane



Re: sha1, sha2 functions into core?

From
Andrew Dunstan
Date:
On 08/20/2012 07:08 PM, Tom Lane wrote:


> Moreover, as Josh just mentioned, anybody who
> thinks it might be insufficiently secure for their purposes has got
> plenty of alternatives available today (SSL certificates, PAM backed
> by whatever-you-want, etc).
>

Yeah, I think we need to emphasize this lots more. Anyone who wants 
really secure authentication needs to be getting away from password 
based auth altogether. Another hash function will make very little 
difference.

cheers

andrew





Re: sha1, sha2 functions into core?

From
"Joshua D. Drake"
Date:
On 08/20/2012 05:12 PM, Andrew Dunstan wrote:
>
>
> On 08/20/2012 07:08 PM, Tom Lane wrote:
>
>
>> Moreover, as Josh just mentioned, anybody who
>> thinks it might be insufficiently secure for their purposes has got
>> plenty of alternatives available today (SSL certificates, PAM backed
>> by whatever-you-want, etc).
>>
>
> Yeah, I think we need to emphasize this lots more. Anyone who wants
> really secure authentication needs to be getting away from password
> based auth altogether. Another hash function will make very little
> difference.

Actually, I concede here. If we were pushing our other abilities more 
visibly, I don't know that this argument would ever come up.

Sincerely,

Joshua D. Drake



>
> cheers
>
> andrew
>
>
>
>


-- 
Command Prompt, Inc. - http://www.commandprompt.com/
PostgreSQL Support, Training, Professional Services and Development
High Availability, Oracle Conversion, Postgres-XC
@cmdpromptinc - 509-416-6579



Re: sha1, sha2 functions into core?

From
Merlin Moncure
Date:
On Mon, Aug 20, 2012 at 5:54 PM, Greg Sabino Mullane <greg@turnstep.com> wrote:
>> 3) use a purposefully slow hashing function like bcrypt.
>>
>> but I disagree: I don't like any scheme that encourages use of low
>> entropy passwords.
>
> Perhaps off-topic, but how to do you figure that?

Yeah -- bcrypt's main claim to fame is that it's slow.  I *lot* of
people argue your'e better off with a slow hash and that's reasonable
but I just don't like the speed/convenience tradeoff.   I suppose I'm
impatient.  My take on this is that relying on hash speed to protect
you if the attacker has the hash, the salt, and knows the algorithm is
pretty weak sauce.  At best it lowers the entropy requirements
somewhat:  an 80 bit entropy password is not brute forcible no matter
how many server farmed GPUs you have.  The mechanics of how the hash
is calculated (see Joe C's excellent comments upthread) are much more
important considerations than algorithm choice.

If you have high security requirements and your users refuse to use
high entropy passwords, I think you're better off going 2-factor then
hoisting slowness on everything that needs to authenticate.

merlin



Re: sha1, sha2 functions into core?

From
Bruce Momjian
Date:
On Mon, Aug 20, 2012 at 07:08:12PM -0400, Tom Lane wrote:
> The only reason I can see for pushing more crypto into core is
> if we needed to stop using MD5 for the core password authentication
> functionality.  While that might come to pass eventually, I am aware of
> no evidence whatever that SHAn, per se, is an improvement over MD5 for
> password auth purposes.  Moreover, as Josh just mentioned, anybody who
> thinks it might be insufficiently secure for their purposes has got
> plenty of alternatives available today (SSL certificates, PAM backed
> by whatever-you-want, etc).
> 
> TBH, I think if we do anything at all about this in the near future,
> it'll be window dressing to shut up the people who heard once that MD5
> was insecure and know nothing about it beyond that --- but if Postgres
> uses MD5 for passwords, it must be insecure.  So I tend to agree with
> Andrew that we should wait till the NIST competition dust settles; but
> what I'll be looking for afterwards is which algorithm has the most
> street cred with the average slashdotter.
> 
> Also, as I mentioned upthread, we need to do more than just drop in
> a new hashing algorithm.  MD5 is far from being the weakest link
> in what we're doing today.

If anyone believe Tom is inaccurate in "MD5 is far from being the
weakest link", see this 2004 email from Greg Stark explaining the odds
of salt reuse and password packet replay:
http://archives.postgresql.org/pgsql-hackers/2004-08/msg01540.php

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + It's impossible for everything to be true. +