WIP: SCRAM authentication - Mailing list pgsql-hackers

There have been numerous threads on replacing our MD5 authentication
method, so I started hacking on that to see what it might look like.
Just to be clear, this is 9.6 material. Attached is a WIP patch series
that adds support for SCRAM. There's no need to look at the details yet,
but it demonstrates what the protocol changes and the code structure
would be like.

I'm not wedded to SCRAM - SRP or JPAKE or something else might be
better. But replacing the algorithm, or adding more of them, should be
straightforward with this.

There is no negotiation of the authentication mechanism. SCRAM is just
added as a new one, alongside all the existing ones. If the server
requests SCRAM authentication, but the client doesn't support it, the
attempt will fail. We might want to do something about that, to make the
transition easier, but it's an orthogonal feature and not absolutely
required.

There are four patches in the series. The first two are just
refactoring: moving the SHA-1 implementation from pgcrypto to
src/common, and some refactoring in src/backend/auth.c that IMHO would
make sense anyway.

Patches three and four are the interesting ones:

3. Allow storing multiple verifiers in pg_authid
------------------------------------------------

Replace the pg_authid.rolpassword text field with an array, and rename
it to 'rolverifiers'. This allows storing multiple password hashes: an
MD5 hash for MD5 authentication, and a SCRAM salt and stored key for
SCRAM authentication, etc. Each element in the array is a string that
begins with the method's name. For example "md5:<MD5 hash>", or
"password:<plaintext>".

For dump/reload, and for clients that wish to create the hashes in the
client-side, there is a new option to CREATE/ALTER USER commands:
PASSWORD VERIFIERS '{ ... }', that allows replacing the array.

The old "ENCRYPTED/UNENCRYPTED PASSWORD 'foo'" options are still
supported for backwards-compatibility, but it's not clear what it should
mean now.

TODO:

* Password-checking hook needs to be redesigned, to allow for more kinds
of hashes.

* With "CREATE USER PASSWORD 'foo'", which hashes/verifiers should be
generated by default? We currently have a boolean password_encryption
setting for that. Needs to be a list.

4. Implement SCRAM
------------------

The protocol and the code is structured so that it would be fairly easy
to add more built-in SASL mechanisms, or to use a SASL library to
provide more. But for now I'm focusing on adding exactly one new
built-in mechanism, to replace MD5 in the long term.

In the protocol, there is a new AuthenticationSASL message, alongside
the existing AuthenticationMD5, AuthenticationSSPI etc. The
AuthenticationSASL message contains the name of the SASL mechanism used
("SCRAM-SHA-1"). Just like in the GSSAPI/SSPI authentication, a number
of PasswordMessage and AuthenticationSASLContinue messages are exchanged
after that, carrying the data specified by the SCRAM spec, until the
authentication succeeds (or not).

TODO:

* Per the SCRAM specification, the client sends the username in the
handshake. But in the FE/BE protocol, we've already sent it in the
startup packet. In the patch, libpq always sends an empty username in
the SCRAM exchange, and the username from the startup packet is what
matters. We could also require it to be the same, but in SCRAM the
username to be UTF-8 encoded, while in PostgreSQL the username can be in
any encoding. That is a source of annoyance in itself, as it's not
well-defined in PostgreSQL which encoding to use when sending a username
to the server. But I don't want to try fixing that in this patch, so it
seems easiest to just require the username to be empty.

* Need a source of randomness in client, to generate random nonces used
in the handshake. The SCRAM specification is not explicit about it, but
I believe it doesn't need to be unpredictable, as long as a different
nonce is used for each authentication.

* The client does not authenticate the server, even though the SCRAM
protocol allows that. The client does verify the proof the server sends,
but nothing stops a malicious server that's impersonating the real
server from not requesting SCRAM authentication in the first place. It
could just send AuthenticationOK without any authentication at all. To
take advantage of the server authentication, we'll need to add something
similar to the "sslmode=verify-ca" option in the client. In that mode,
the client should refuse the connection if the server doesn't request
SCRAM authentication (or some other future authentication mechanism that
authenticates the server to the client).

* Channel binding is not implemented. Not essential, but would be nice
to have. Together with the new client option mentioned in the previous
point, it would allow the client to know that there is no
man-in-the-middle, without having to verify the server's SSL certificate.

- Heikki