Thread: SCRAM inplementation

Some of you may be aware that postgres 10 will have an additional authentication method called SCRAM

Alvaro has graciously offered to implement this for JDBC. I mention this because if anyone is in the process of implementing it perhaps we should discuss it and break it up into parts. 

Also this means we are going to be adding some dependencies. Alvaro can you talk about which dependencies you are contemplating ?

Regards,

Dave Cramer

    Hi Dave, everybody.

    Yes, I happily offered to work on this :) I already started working a bit, but can't make hard promises on a deadline. If anyone's interested in helping, please let me know.

    Here's my initial ideas:

* Write both a client and server implementation. pgjdbc will only require the former, of course, but having both will be great for testing.
* Do so as an independent library, also as an independent repository on Github. This will help its reuse and testing by independent projects. I presume it will have at least three different artifacts, a scram-common, scram-server and scram-client. Only the latter will be directly imported as a direct dependency by pgjdbc.
* The implementation will *not* provide support for message sending and/or serialization. Only message generation and parsing (after all, messages are only strings, so it's easy). This is meant to be easily reused, but will of course require some glue code on the pgjdbc side.
* Channel biding will not be on the first version. It is not used in PostgreSQL 10 either (as of today).
* Both SHA-1 and SHA-256 will be implemented (yeah, I know about SHA-1 collision, but it's still an RFC and adding it is a one-liner so... I leave the decision to the users). PG 10 will only ship with SHA-256, though, AFAIK.
* First version will not implement SaslPrep (neither PG10 does). When it will do.... I will probably do a separate repository for StringPrep/SaslPrep code, as it is again of a very reusable nature outside of the SCRAM (and of cours pgjdbc) projects.
* I have tried to lay out the code so that no further external dependencies are required. It might be difficult. There are some libraries out there that could help (for example on the StringPrep/SaslPrep side). There could be more dependencies used, but only for testing.
* I want to make a consideration about writing Java6 vs Java8 code. Unless there would be general agreement on moving pgjdbc to Java8 (and this could be a topic on itself), I will develop it for Java6. However, there are some downsides:
    - There's no "official" API for Base64 in JDK6, and I won't do it by hand. Either apache-commons-codec is used or JDK 6's javax.xml.bind.DatatypeConverter methods are used. I will use the latter, but it is not an ideal approach. Java8 has a new Base64 class for this.
    - Both SHA-1 and SHA-256 and their corresponding HMACs are supported in both versions. But the "Hi" function defined in the RFC (aka PBKDF2) is not supported in Java6 but is implemented in Java8: SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256"). It shouldn't be hard to implement it "by hand", having the hmac functions available, but this is clearly my limit into how much crypto code I would write by hand (not that I wouldn't like to, it's fun, but I favor a secure implementation here). The alternative is to bring another library, of course.
    - The client API will contain some FunctionalInterfaces I will not be able to annotate as such if in Java6.
So... I don't want to spur a heavy debate on Java6 vs Java8, and maybe that should go on a separate thread, but... thinking of it why not? There are indeed many versions (up to and including 42) of pgjdbc which work for Java6, are fully featured, and mature. While they are developed for a platform EOLed for several years. Does it make real sense to continue developing new versions with this platform in mind? Are there very strong reasons to remain Java6-compatible for future versions?


    Please let me know your thoughts.


    Regards,

    Álvaro

-- 

Álvaro Hernández Tortosa


-----------
<8K>data

> On Apr 2, 2017, at 9:05 AM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
> On 31/03/17 14:31, Dave Cramer wrote:
>>
>> Also this means we are going to be adding some dependencies. Alvaro can you talk about which dependencies you are
contemplating? 
>
>
> So... I don't want to spur a heavy debate on Java6 vs Java8, and maybe that should go on a separate thread, but...
thinkingof it why not? There are indeed many versions (up to and including 42) of pgjdbc which work for Java6, are
fullyfeatured, and mature. While they are developed for a platform EOLed for several years. Does it make real sense to
continuedeveloping new versions with this platform in mind? Are there very strong reasons to remain Java6-compatible
forfuture versions? 

One random $USER's opinion, Java 7 is already EOL.  Why do we even talk about 6 anymore?

If better or less code can be written, or reliability improved, by moving to Java 8 - do it!  We should live in the
future,not the past :) 

On 2 April 2017 at 17:05, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

> * Write both a client and server implementation. pgjdbc will only require
> the former, of course, but having both will be great for testing.
> * Do so as an independent library, also as an independent repository on
> Github. This will help its reuse and testing by independent projects. I
> presume it will have at least three different artifacts, a scram-common,
> scram-server and scram-client. Only the latter will be directly imported as
> a direct dependency by pgjdbc.

Don't see any need for the external stuff. We just want a patch on the
driver so any external dependencies are minimized, so we retain full
control over what the driver is doing. That can be reused easily
enough in other projects, if they wish.

Thanks for offering to write a patch.

--
Simon Riggs                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 03/04/17 00:18, Simon Riggs wrote:
> On 2 April 2017 at 17:05, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
>
>> * Write both a client and server implementation. pgjdbc will only require
>> the former, of course, but having both will be great for testing.
>> * Do so as an independent library, also as an independent repository on
>> Github. This will help its reuse and testing by independent projects. I
>> presume it will have at least three different artifacts, a scram-common,
>> scram-server and scram-client. Only the latter will be directly imported as
>> a direct dependency by pgjdbc.
> Don't see any need for the external stuff. We just want a patch on the
> driver so any external dependencies are minimized, so we retain full
> control over what the driver is doing.

     Hi Simon.

     It's not that is a need. Its that it is what should be. Code
re-utilization and sharing across different projects is mutually
beneficial. C might not make this easy (it doesn't) but Java does and it
should be leveraged. A specific external SCRAM project encourages other
to use it, improve it, test it, outside our limited capabilities.

     I don't buy how this would mean less control over what the driver
is doing. Code is open source, BSD license, either patches or forks
could be created.... and it will be designed initially for pgjdbc needs.
That's all we need, I believe.
> That can be reused easily
> enough in other projects, if they wish.

     How? Copy and paste the code somewhere else? This is exactly what I
want to avoid. What if others find this functionality interesting? I
think this method of being an external project favors sharing and
collaboration among open source communities.

>
> Thanks for offering to write a patch.
>

     Thanks for all the patches you have written so far ^_^


     Álvaro

--

Álvaro Hernández Tortosa


-----------
<8K>data

On Mon, Apr 3, 2017 at 1:05 AM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
> * Write both a client and server implementation. pgjdbc will only require
> the former, of course, but having both will be great for testing.

Doesn't JDBC require already a Postgres instance when testing? You
could just rely on that.

> * Do so as an independent library, also as an independent repository on
> Github. This will help its reuse and testing by independent projects. I
> presume it will have at least three different artifacts, a scram-common,
> scram-server and scram-client. Only the latter will be directly imported as
> a direct dependency by pgjdbc.
> * The implementation will *not* provide support for message sending and/or
> serialization. Only message generation and parsing (after all, messages are
> only strings, so it's easy). This is meant to be easily reused, but will of
> course require some glue code on the pgjdbc side.
> * Channel biding will not be on the first version. It is not used in
> PostgreSQL 10 either (as of today).

Yep, this has been deferred for later versions. The protocol name with
channel binding uses -PLUS as suffix, so that's no big deal from an
implementation point of view to get that later on.

> * Both SHA-1 and SHA-256 will be implemented (yeah, I know about SHA-1
> collision, but it's still an RFC and adding it is a one-liner so... I leave
> the decision to the users). PG 10 will only ship with SHA-256, though,
> AFAIK.

We are not going to ship with SCRAM-SHA-1 anyway, so I would advise to
just have no trace of it.

> * First version will not implement SaslPrep (neither PG10 does). When it
> will do.... I will probably do a separate repository for StringPrep/SaslPrep
> code, as it is again of a very reusable nature outside of the SCRAM (and of
> cours pgjdbc) projects.

We'll see about that. I have a patch able to address the problem...
--
Michael

On 03/04/17 02:28, Michael Paquier wrote:
> On Mon, Apr 3, 2017 at 1:05 AM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
>> * Write both a client and server implementation. pgjdbc will only require
>> the former, of course, but having both will be great for testing.
> Doesn't JDBC require already a Postgres instance when testing? You
> could just rely on that.

     Needless to say, it will be tested against Postgres 10. But since I
plan to develop it as a separate library external to pgjdbc, meant for
general purpose use, it will have both client and server
implementations. It has the added benefit of unit tests that do not have
external and non-Java dependencies (like PostgreSQL).

>
>> * Do so as an independent library, also as an independent repository on
>> Github. This will help its reuse and testing by independent projects. I
>> presume it will have at least three different artifacts, a scram-common,
>> scram-server and scram-client. Only the latter will be directly imported as
>> a direct dependency by pgjdbc.
>> * The implementation will *not* provide support for message sending and/or
>> serialization. Only message generation and parsing (after all, messages are
>> only strings, so it's easy). This is meant to be easily reused, but will of
>> course require some glue code on the pgjdbc side.
>> * Channel biding will not be on the first version. It is not used in
>> PostgreSQL 10 either (as of today).
> Yep, this has been deferred for later versions. The protocol name with
> channel binding uses -PLUS as suffix, so that's no big deal from an
> implementation point of view to get that later on.

     Right.

>
>> * Both SHA-1 and SHA-256 will be implemented (yeah, I know about SHA-1
>> collision, but it's still an RFC and adding it is a one-liner so... I leave
>> the decision to the users). PG 10 will only ship with SHA-256, though,
>> AFAIK.
> We are not going to ship with SCRAM-SHA-1 anyway, so I would advise to
> just have no trace of it.

     Same as above: the idea is to have it as a general-purpose library.
Maybe others want SHA-1. It is very simple to implement it, it adds no
extra effort.

>
>> * First version will not implement SaslPrep (neither PG10 does). When it
>> will do.... I will probably do a separate repository for StringPrep/SaslPrep
>> code, as it is again of a very reusable nature outside of the SCRAM (and of
>> cours pgjdbc) projects.
> We'll see about that. I have a patch able to address the problem...

     Please let me know any further info :)

     Thanks,

     Álvaro


--

Álvaro Hernández Tortosa


-----------
<8K>data

On Mon, Apr 3, 2017 at 9:34 AM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
>     Same as above: the idea is to have it as a general-purpose library.
> Maybe others want SHA-1. It is very simple to implement it, it adds no extra
> effort.

You may be interested in something like this one:
https://github.com/ogrebgr/scram-sasl
--
Michael

On 03/04/17 02:50, Michael Paquier wrote:
> On Mon, Apr 3, 2017 at 9:34 AM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
>>      Same as above: the idea is to have it as a general-purpose library.
>> Maybe others want SHA-1. It is very simple to implement it, it adds no extra
>> effort.
> You may be interested in something like this one:
> https://github.com/ogrebgr/scram-sasl

     Thanks. I already did have a look at it.

     But this library comes with 0 tests. And has some code copied over
other projects, which becomes (probably) unmaintained. I'm writing a
clean-room implementation, with proper tests and no code copied
(dependencies if required).

     There are some nice ideas in the library, though.

     Álvaro


--

Álvaro Hernández Tortosa


-----------
<8K>data