Re: BUG #17760: SCRAM authentication fails with "modern" (rsassaPss signature) server certificate - Mailing list pgsql-bugs

On 25/01/2023 13:32, PG Bug reporting form wrote:
> The following bug has been logged on the website:
> 
> Bug reference:      17760
> Logged by:          Gunnar "Nick" Bluth
> Email address:      gunnar.bluth@pro-open.de
> PostgreSQL version: 13.8
> Operating system:   Ubuntu 20.04
> Description:
> 
> My client recently started rolling out new server certificates, which, when
> dumped with "openssl x509 [...]", show slightly different information
> regarding signature, hash etc.
> 
> Old:
> Signature Algorithm: sha256WithRSAEncryption
> New:
> Signature Algorithm: rsassaPss
>           Hash Algorithm: sha512
>           Mask Algorithm: mgf1 with sha512
>            Salt Length: 0x40
>           Trailer Field: 0xBC (default)
> 
> When trying to authenticate on a server using such a certificate using
> scram-sha-256, we receive an error message: "could not find digest for NID
> UNDEF"

I was able to create a certificate like that with these commands:

openssl genpkey -algorithm rsa-pss -pkeyopt rsa_keygen_bits:2048 
-pkeyopt rsa_keygen_pubexp:65537 -out CA.key

openssl req -new -x509 -days 365 -nodes -text -out server.crt -key 
CA.key -keyout server.key -subj "/CN=rsapss.localtest.me"

And was able to reproduce the problem with that.

As a workaround, you can disable channel binding with the 
"channel_binding=disable" libpq option.

> I tried to boil this down a bit. The error comes from
> src/interfaces/libpq/fe-secure-openssl.c, pgtls_get_peer_certificate_hash(),
> line 440+.
> The comment there says "If something else is used, the same hash as the
> signature algorithm is used."
> And obviously, "EVP_get_digestbynid(NID_rsassaPss)" doesn't return a
> result.
> 
> Now, openssl's "crypto/objects/obj_xref.txt" contains this:
> # OID cross reference table.
> # Links signatures OIDs to their corresponding public key algorithms
> # and digests.
> <snip>
> sha256WithRSAEncryption sha256  rsaEncryption
> <snip>
> # For PSS the digest algorithm can vary and depends on the included
> # AlgorithmIdentifier. The digest "undef" indicates the public key
> # method should handle this explicitly.
> rsassaPss               undef   rsassaPss
> 
> 
> That explains the "UNDEF" in the error message (i.e., I *think* that's where
> it's coming from).
> 
> I have to say that I'm not all too deep into crypto stuff. I couldn't even
> tell how to create one such certificate, let alone what that PSS stuff is
> all about ;-/
> 
> Maybe this is even fixed with recent OpenSSL versions (client has 1.1.1f,
> Ubuntu 20.04)? Though that line was introduced in 2010...

Thanks for the investigation!

> I do think however that this is an oversight on our side and has to be
> addressed. If not in code, the docs should point out that certain server
> certificate types (PSS) may not work with SCRAM auth (or libpq needs to be
> compiled against a minimum version of OpenSSL, if that's the root cause).

There are a few things we change in PostgreSQL for this:

1. If the server cannot compute a hash of the certificate, because it 
cannot unambiguously determine the hash algorithm to use, it should not 
offer the SCRAM-SHA-256-PLUS authentication method to the client. In 
other words, if the server cannot do channel binding with it's 
certificate, it should say so, instead of erroring out later.

2. Similarly in the client: if libpq cannot determinate the hash 
algorithm to use with the server's certificate, it should not attempt 
channel binding.

I believe this is OK from a security point of view. If the server is 
using a certificate that cannot be used with channel binding, and the 
client doesn't require channel binding, it's OK to not do it. A man in 
the middle can present a certificate to the client that cannot be used 
with channel binding, but if they can do that they could also just not 
offer the SCRAM-SHA-256-PLUS option to the client. So I don't see risk 
of downgrade attacks here.

3. Add support for channel binding with RSA-PSS. The problem is that 
be_tls_get_certificate_hash() doesn't know which digest algorithm to 
use. As you noted, OBJ_find_sigid_algs() returns "undef" (NID_undef) for 
rsassaPss certificates. I did some googling: when certificate uses 
RSASSA-PSS as the signature algorithm, there is a separate field, 
RSASSA-PSS-params that specifies the hash algorithm 
(https://www.rfc-editor.org/rfc/rfc4055#section-3.1). If you ask me, 
OBJ_find_sigid_algs() should look into RSASSA-PSS-params and figure it 
out, but I'm no expert in the OpenSSL codebase so maybe that would be 
the wrong place to do it.

OpenSSL 3.0 actually has a function called X509_digest_sig(), which 
parses the RSASSA-PSS-params field. It also has a fallback mechanism to 
use SHA512 for ED25519, SHAKE256 for ED448, and SHA256 for anything 
else. We could use that function, perhaps without accepting the 
fallbacks, though. Or call lower level OpenSSL functions to get the hash 
algorithm from the RSASSA-PSS-params field ourselves.

I'll write a patch for 1 and 2, at least, and maybe 3 if it's not too 
complicated. Please poke me if you don't hear from me in a few days.

- Heikki