Cryptohash OpenSSL error queue in FIPS enabled builds - Mailing list pgsql-hackers

From Daniel Gustafsson
Subject Cryptohash OpenSSL error queue in FIPS enabled builds
Date
Msg-id C89D932C-501E-4473-9750-638CFCD9095E@yesql.se
Whole thread Raw
Responses Re: Cryptohash OpenSSL error queue in FIPS enabled builds  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
In trying out an OpenSSL 3.1 build with FIPS enabled I realized that our
cryptohash code had a small issue.  Calling a banned cipher generated two
different error messages interleaved:

  postgres=# select md5('foo');
  ERROR:  could not compute MD5 hash: unsupported
  postgres=# select md5('foo');
  ERROR:  could not compute MD5 hash: initialization error

It turns out that OpenSSL places two errors in the queue for this operation,
and we only consume one without clearing the queue in between, so we grab an
error from the previous run.

Consuming all (both) errors and creating a concatenated string seems overkill
as it would alter the API from a const error string to something that needs
freeing etc (also, very few OpenSSL consumers actually drain the queue, OpenSSL
themselves don't).  Skimming the OpenSSL code I was unable to find another
example of two errors generated.  The attached calls ERR_clear_error() as how
we do in libpq in order to avoid consuming earlier errors.

--
Daniel Gustafsson        https://vmware.com/


Attachment

pgsql-hackers by date:

Previous
From: Bharath Rupireddy
Date:
Subject: Re: How to simulate sync/async standbys being closer/farther (network distance) to primary in core postgres?
Next
From: Zhihong Yu
Date:
Subject: Re: [PATCH] Compression dictionaries for JSONB