Thread: what can go in root.crt ?

Certificates I get at $work come four layers deep:


Self-signed CA cert from "WE ISSUE TO EVERYBODY.COM"

  Intermediate from "WE ISSUE TO LOTS OF FOLKS.COM"

    Intermediate from "WE ISSUE TO ORGS LIKE YOURS.COM"

      End-entity cert for my server.


Until today, we had the topmost, self-signed cert in root.crt
and stuff worked. But I needed to renew, and it seems that
recently WE ISSUE TO ORGS LIKE YOURS has chosen somebody else
to sign their certs, so I have new certs for the issuers above them,
so I have to go deal with root.crt.

And that got me thinking: do I really want WE ISSUE TO EVERYBODY
to be what I'm calling trusted in root.crt?

I considered just putting the end-entity cert for my server in there,
but it's only good for a couple years, and I'd rather not have to
fuss with editing and distributing root.crt that often.

As a compromise, I tried putting the WE ISSUE TO ORGS LIKE YOURS cert there.
I think I'm willing to accept that much risk. But psql says:

psql: SSL error: certificate verify failed

I would be happy if it gave a little more detail. Is it failing
verification because the cert I put in root.crt is *not* self-signed,
and I didn't include the two issuers above it?

Does that mean it also would fail if I directly put the server's
end-entity cert there?

Would I have to put all three of WE ISSUE TO ORGS LIKE YOURS,
WE ISSUE TO LOTS, and WE ISSUE TO EVERYBODY in the root.crt file
in order for verification to succeed?

If I did that, would the effect be any different from simply putting
WE ISSUE TO EVERYBODY there, as before? Would it then happily accept
a cert with a chain that ended at WE ISSUE TO EVERYBODY via some other
path? Is there a way I can accomplish trusting only certs issued by
WE ISSUE TO ORGS LIKE YOURS?

I never noticed how thin the docs or verify-failure messages were
on this topic until just now. Are there any options, openssl
environment variables, or the like, to get it to be a little more
forthcoming about what it expects?

Regards,
-Chap

On 05/25/20 15:15, Chapman Flack wrote:
> Does that mean it also would fail if I directly put the server's
> end-entity cert there?
> 
> Would I have to put all three of WE ISSUE TO ORGS LIKE YOURS,
> WE ISSUE TO LOTS, and WE ISSUE TO EVERYBODY in the root.crt file
> in order for verification to succeed?
> 
> If I did that, would the effect be any different from simply putting
> WE ISSUE TO EVERYBODY there, as before? Would it then happily accept
> a cert with a chain that ended at WE ISSUE TO EVERYBODY via some other
> path? Is there a way I can accomplish trusting only certs issued by
> WE ISSUE TO ORGS LIKE YOURS?

The client library is the PG 10 one that comes with Ubuntu 18.04
in case it matters.

I think I have just verified that I can't make it work by putting
the end entity cert there either. It is back working again with only
the WE ISSUE TO EVERYBODY cert there, but if there is a workable way
to narrow that grant of trust a teensy little bit, I would be happy
to do that.

Regards,
-Chap

On Mon, May 25, 2020 at 03:32:52PM -0400, Chapman Flack wrote:
> On 05/25/20 15:15, Chapman Flack wrote:
> > Does that mean it also would fail if I directly put the server's
> > end-entity cert there?
> > 
> > Would I have to put all three of WE ISSUE TO ORGS LIKE YOURS,
> > WE ISSUE TO LOTS, and WE ISSUE TO EVERYBODY in the root.crt file
> > in order for verification to succeed?
> > 
> > If I did that, would the effect be any different from simply putting
> > WE ISSUE TO EVERYBODY there, as before? Would it then happily accept
> > a cert with a chain that ended at WE ISSUE TO EVERYBODY via some other
> > path? Is there a way I can accomplish trusting only certs issued by
> > WE ISSUE TO ORGS LIKE YOURS?
> 
> The client library is the PG 10 one that comes with Ubuntu 18.04
> in case it matters.
> 
> I think I have just verified that I can't make it work by putting
> the end entity cert there either. It is back working again with only
> the WE ISSUE TO EVERYBODY cert there, but if there is a workable way
> to narrow that grant of trust a teensy little bit, I would be happy
> to do that.

Did you review the PG documentation about intermediate certificates?

    https://www.postgresql.org/docs/13/ssl-tcp.html#SSL-CERTIFICATE-CREATION

Is there a specific question you have?  I don't know how to improve the
error reporting.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

On 05/25/20 22:03, Bruce Momjian wrote:
> Did you review the PG documentation about intermediate certificates?
> 
>     https://www.postgresql.org/docs/13/ssl-tcp.html#SSL-CERTIFICATE-CREATION

AFAICT, there isn't much in that section to apply to my question.

> Is there a specific question you have?

I'm pretty sure there is. :)  Let me try to clarify it.

The doc section you linked has an example showing how to be my own CA,
and generate a chain of certs including a self-signed one to serve as
the root. It suggests putting that self-signed one in root.crt on the
client.

That means the client will happily connect to any server wearing a
certificate signed by that root (or by intermediates that can be followed
up to that root). For the example that's fine, because that root signer
is me, and there aren't a lot of other certs around that chain back to it.

At $work we have Ways Of Doing Things. Generating our own self-signed certs
generally isn't among those. If I want a certificate so I can stand up
a server, I generate a key and a CSR, I send the CSR to our Bureau of
Making Certificates Happen, and they send me back a signed cert with
a chain of external authorities, ending in the self-signed certificate
of a prominent commercial root CA.

Sure, I can put that self-signed root cert into root.crt on the client,
and my client will happily connect to my server.

But in this case the world is teeming with other certificates and even
other whole sub-CAs that chain back to that prominent root issuer.
Granted, you might have to be a bit enterprising to find a sub-CA out
there that will sign a cert for you with the name of my server in it,
but if you can, my client will follow the chain back to that same root,
and therefore trust it.

So I would like to be able to do one of two things:

1. I would like to put my server's end-entity (leaf) certificate
   in the root.crt file, and have my client only accept a server
   with that exact cert.

Or,

2. I would like to put one of the lower intermediates from the chain
   into the root.crt file, to at least limit my client to trusting
   only certs signed by that particular sub-CA.


What seems to be happening (for the libpq and libssl versions in 18.04
anyway) is that the certificate that I put in root.crt is found, but
because it isn't a literal "root", as in signer-of-itself, the library
declares a verification failure because it hasn't been able to continue
climbing the chain to find a "root" cert.

Whereas I would like it to say "but I don't have to do that, because
I have already verified as far as this certificate that the administrator
deliberately placed in this file here to tell me to trust it."

In Java, for example, the analogous file is called trustStore, which
may be a better name. You populate the trustStore with certificates you
consider trusted. They can be root certs, intermediate certs, or flat-out
leaf certs of individual servers. Whenever Java is verifying a connection,
as soon as its chain-following brings it to a cert that you placed in
the trustStore, it stops and says "yes, I trust this, because you have
told me to."

I have also encountered web browsers that work in both of these ways.
The last time I was standing up a temporary web service to test something,
I did make a self-signed cert and then use it to sign a leaf cert for
the service. I was testing with Chrome and Firefox and they both have
spiffy UIs for managing a list of trusted certs, but one of them (I have
forgotten which) allowed me to simply load the leaf cert that I wanted
to trust, while the other insisted I give it the self-signed root that
I had signed the leaf cert with.

I think the former behavior, which is like Java's, is strictly more useful.

What puzzled me today, and why I began this thread, is that I hadn't
(and still haven't) found a clear discussion in the doc of these two
approaches and which one libpq is intended to supply. I know that my
attempts to use root.crt like a trustStore have so far been met with
failure, but between the terse error message and the sparse doc, it is
hard to know whether that's a "you can't do that, dummy!" or a "you
just haven't guessed the right way yet."

If there is a way to get a trustStore-like behavior and have the client
trust an intermediate or leaf cert that I explicitly tell it to, but I
just haven't pronounced the magic words right, this email may be read
as "oh good, how do I do it?"

If the current implementation really is stuck accepting only self-signed
certs in that file and therefore can't offer trustStore-like behavior,
this email may be read as "it could be made more useful by changing that."

And in either case, there seems to be room in the docs for some
discussion of the difference between those two models and which one
libpq is meant to offer.

I would not be unwilling to try my hand at such a doc patch one day,
but for now I'm still hoping to learn the answers myself.

Regards,
-Chap

On Mon, 2020-05-25 at 15:15 -0400, Chapman Flack wrote:
> Certificates I get at $work come four layers deep:
> 
> 
> Self-signed CA cert from "WE ISSUE TO EVERYBODY.COM"
> 
>   Intermediate from "WE ISSUE TO LOTS OF FOLKS.COM"
> 
>     Intermediate from "WE ISSUE TO ORGS LIKE YOURS.COM"
> 
>       End-entity cert for my server.
> 
> 
> And that got me thinking: do I really want WE ISSUE TO EVERYBODY
> to be what I'm calling trusted in root.crt?

I don't know if there is a way to get this to work, but the
fundamental problem seems that you have got the system wrong.

If you don't trust WE ISSUE TO EVERYBODY, then you shouldn't use
it as a certification authority.

Yours,
Laurenz Albe

On Tue, May 26, 2020 at 05:22:13AM +0200, Laurenz Albe wrote:
> On Mon, 2020-05-25 at 15:15 -0400, Chapman Flack wrote:
> > Certificates I get at $work come four layers deep:
> > 
> > 
> > Self-signed CA cert from "WE ISSUE TO EVERYBODY.COM"
> > 
> >   Intermediate from "WE ISSUE TO LOTS OF FOLKS.COM"
> > 
> >     Intermediate from "WE ISSUE TO ORGS LIKE YOURS.COM"
> > 
> >       End-entity cert for my server.
> > 
> > 
> > And that got me thinking: do I really want WE ISSUE TO EVERYBODY
> > to be what I'm calling trusted in root.crt?
> 
> I don't know if there is a way to get this to work, but the
> fundamental problem seems that you have got the system wrong.
> 
> If you don't trust WE ISSUE TO EVERYBODY, then you shouldn't use
> it as a certification authority.

It is true that WE ISSUE TO EVERYBODY can create a new intermediate with
the same intemediate name anytime they want.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

On 05/25/20 23:22, Laurenz Albe wrote:
> I don't know if there is a way to get this to work, but the
> fundamental problem seems that you have got the system wrong.
> 
> If you don't trust WE ISSUE TO EVERYBODY, then you shouldn't use
> it as a certification authority.

That's a reasonable viewpoint.

I've worked in organizations from smallish to largish, and in the
largish ones, sometimes there are Ways Of Doing Things, that were
laid down by Other People.

There the challenge becomes how to piece together practices that
maximize my comfort level, within the ways of doing things that
come down from others.

If the libpq root.crt file can be made to work similarly to a
Java trustStore, that expands the possible solution space.

Regards,
-Chap

On 2020-May-25, Chapman Flack wrote:

> If the libpq root.crt file can be made to work similarly to a
> Java trustStore, that expands the possible solution space.

If I understand you correctly, you want a file in which you drop any of
these intermediate CA's cert in, causing the server to trust a cert
emitted by that CA -- regardless of that CA being actually root.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 05/26/20 00:07, Alvaro Herrera wrote:
>> If the libpq root.crt file can be made to work similarly to a
>> Java trustStore, that expands the possible solution space.
> 
> If I understand you correctly, you want a file in which you drop any of
> these intermediate CA's cert in, causing the server to trust a cert
> emitted by that CA -- regardless of that CA being actually root.

Right: an intermediate cert, or a self-signed root cert, or even the
end-entity (leaf) cert for a specific machine. You name it, if I put
in in the trust store, and a connection verification starts with or leads
to a cert that I put there, success.

Regards,
-Chap

On 05/26/20 00:07, Isaac Morland wrote:
> What about the SSH model? In the Postgres context, this would basically be
> a table containing authorized certificates for each user. Upon receiving a
> connection attempt, look up the user and the presented certificate and see
> if it is one of the authorized ones. If so, do the usual verification that
> the client really does have the corresponding private key and if so,
> authenticate the connection.

I like the SSH model, but just in case it wasn't clear, I wasn't thinking
about client-cert authentication here, just about conventional verification
by the client of a certificate for the server.

By the same token, there's no reason not to ask the same questions about
the other direction.

Regards,
-Chap

On 05/26/20 02:05, Craig Ringer wrote:
> The main reason to put intermediate certificates in the root.crt is that it
> allows PostgreSQL to supply the whole certificate chain to a client during

Hold on a sec; you're not talking about what I'm talking about, yet.

Yes, you have make the chain available to the server to serve out with
its own cert so clients can verify. root.crt isn't where you put that,
though. You put that in server.crt (or wherever the ssl_cert_file GUC
points).

> That frees the clients from needing to have local copies
> of the intermediate certificates; they only have to know about WE ISSUE TO
> EVERYBODY.

Bingo. Put WE ISSUE TO EVERYBODY in the root.crt (client-side, libpq) file,
and the clients happily connect to the server. It is easy and convenient.

But if WE STEAL YOUR STUFF gets their certs signed by WE SIGN ANYTHING
FOR A PRICE and their CA is WE'RE SOMETIMES LESS ATTENTIVE THAN YOU HOPE
and /their/ CA is WE ISSUE TO EVERYBODY, then the clients would just as
happily connect to a server of the same name run by WE STEAL YOUR STUFF.

Which brings us around to what I was talking about.

> If you wanted to require that your certs are signed by WE ISSUE TO ORGS
> LIKE YOURS.COM, you must configure your CLIENTS with a restricted root of
> trust that accepts only the intermediate certificate of WE ISSUE TO ORGS
> LIKE YOURS.COM .

Precisely. And the place to configure that restricted root of trust would
have to be ~/.postgresql/root.crt on the client, and the question is,
does that work?

> Assuming the client will accept it; not all clients allow
> you to configure "certificates I trust to sign peers" separately to
> "certificates that sign my trusted roots". Because really, in security
> terms that's nonsensical.

And that's the key question: there are clients that grok that and clients
that don't; so now, libpq is which kind of client?

Could you expand on your "sign _peers_" notion, and on what exactly
you are calling nonsensical?

Each of those intermediate CAs really is a CA; the WE ISSUE TO ORGS
LIKE yours cert does contain these extensions:

        X509v3 extensions:
            ...
            X509v3 Key Usage: critical
                Digital Signature, Certificate Sign, CRL Sign
            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:0
            ...

My server's end-entity certificate does not have the Certificate Sign,
CRL Sign or CA:TRUE bits, or a URL to a revocation-checking service.
An end entity and a CA are not 'peers' in those respects.

Regards,
-Chap

On 5/25/20 3:32 PM, Chapman Flack wrote:
> On 05/25/20 15:15, Chapman Flack wrote:
>> Does that mean it also would fail if I directly put the server's
>> end-entity cert there?
>>
>> Would I have to put all three of WE ISSUE TO ORGS LIKE YOURS,
>> WE ISSUE TO LOTS, and WE ISSUE TO EVERYBODY in the root.crt file
>> in order for verification to succeed?
>>
>> If I did that, would the effect be any different from simply putting
>> WE ISSUE TO EVERYBODY there, as before? Would it then happily accept
>> a cert with a chain that ended at WE ISSUE TO EVERYBODY via some other
>> path? Is there a way I can accomplish trusting only certs issued by
>> WE ISSUE TO ORGS LIKE YOURS?
> The client library is the PG 10 one that comes with Ubuntu 18.04
> in case it matters.
>
> I think I have just verified that I can't make it work by putting
> the end entity cert there either. It is back working again with only
> the WE ISSUE TO EVERYBODY cert there, but if there is a workable way
> to narrow that grant of trust a teensy little bit, I would be happy
> to do that.
>


The trouble is I think you have it the wrong way round. It makes sense
to give less trust to a non-root CA than to one of its up-chain
authorities, e.g. only trust it for certain domains, or for a lesser
period of time. But it doesn't seem to make much sense to trust the
up-chain CA less, since it is what you should base your trust of the
lower CA on.



cheers


andrew


-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 05/26/20 09:35, Andrew Dunstan wrote:

> The trouble is I think you have it the wrong way round. It makes sense
> to give less trust to a non-root CA than to one of its up-chain
> authorities, e.g. only trust it for certain domains, or for a lesser
> period of time. But it doesn't seem to make much sense to trust the
> up-chain CA less, since it is what you should base your trust of the
> lower CA on.

I wonder if there might be different meanings of 'trust' in play here
complicating the conversation.

At $work, when I make a certificate request and send it off to our
own in-house bureau of making certificates happen, what you might
expect is that they would be running the first level of CA right
in house (and IIRC that was the case in my early years here).
So I would get back some chain like this:

  WE ARE A PROMINENT GLOBAL ISSUER FOUND IN WEB BROWSER TRUST STORES
    WE ISSUE TO LOTS OF FOLKS
      WE ISSUE TO ORGS LIKE YOURS
        WE ARE YOUR ORG
          my server cert

In that picture, the question of whether I give more or less trust to
PROMINENT GLOBAL ISSUER because they have larger market cap and their
name in the news, or to WE ARE YOUR ORG because they are my org, seems
to turn on different understandings of trust. There might be a lot of
reasons in general to trust PROMINENT GLOBAL in the sense of putting
their cert in widely distributed web browser trust stores. But there
are excellent reasons to trust WE ARE YOUR ORG as authoritative on
what's a server for my org.

Now in these later days when there is no longer an in-house CA at the
bottom of this chain, the situation's not as clear-cut. WE ISSUE TO ORGS
LIKE YOURS isn't quite authoritative on what's a server for my org.
But there are inked signatures on paper between their honcho and my org's
honcho that don't exist between my org and PROMINENT GLOBAL. And you would
have to work harder to get a spoof cert for one of my servers signed by
them. You would have to talk /them/ into it.

If I have PROMINENT GLOBAL in there, you just have to make offers to
their umpty sub-CAs and their umpty-squared sub-sub-CAs and find just
one that will make a deal.

> to give less trust to a non-root CA than to one of its up-chain
> authorities, e.g. only trust it for certain domains, or for a lesser

That's certainly appropriate, and I'd be delighted if the root.crt file
supported syntax like this:

 *.myorg.org: WE ARE YOUR ORG.crt
 *: PROMINENT GLOBAL ISSUER.crt { show exfiltration/HIPAA/FERPA banner }


Doing the same thing (or some of it) in certificate style, you would
want WE ARE YOUR ORG.crt to be signed with a Name Constraints extension
limiting it to be a signer for .myorg.org certificates. That is indeed
a thing. The history in [1] shows it was at first of limited value
because client libraries didn't all grok it, or would accept certificates
without Subject Alt Name extensions and verify by CN instead, without the
constraint. But I have noticed more recently that mainstream web browsers,
anyway, are no longer tolerant of certs without SAN, and that seems to be
part of a road map to giving the Name Constraints more teeth.

Regards,
-Chap

[1]

https://security.stackexchange.com/questions/31376/can-i-restrict-a-certification-authority-to-signing-certain-domains-only

On Tue, May 26, 2020 at 10:13:56AM -0400, Chapman Flack wrote:
> At $work, when I make a certificate request and send it off to our
> own in-house bureau of making certificates happen, what you might
> expect is that they would be running the first level of CA right
> in house (and IIRC that was the case in my early years here).
> So I would get back some chain like this:
> 
>   WE ARE A PROMINENT GLOBAL ISSUER FOUND IN WEB BROWSER TRUST STORES
>     WE ISSUE TO LOTS OF FOLKS
>       WE ISSUE TO ORGS LIKE YOURS
>         WE ARE YOUR ORG
>           my server cert
> 
> In that picture, the question of whether I give more or less trust to
> PROMINENT GLOBAL ISSUER because they have larger market cap and their
> name in the news, or to WE ARE YOUR ORG because they are my org, seems
> to turn on different understandings of trust. There might be a lot of
> reasons in general to trust PROMINENT GLOBAL in the sense of putting
> their cert in widely distributed web browser trust stores. But there
> are excellent reasons to trust WE ARE YOUR ORG as authoritative on
> what's a server for my org.

I think it gets down to an issue I blogged about in 2017:

    https://momjian.us/main/blogs/pgblog/2017.html#January_9_2017

    The use of public certificate authorities doesn't make sense for most
    databases because it allows third parties to create trusted
    certificates. Their only reasonable use is if you wish to allow public
    certificate authorities to independently issue certificates that you
    wish to trust. This is necessary for browsers because they often connect
    to unaffiliated websites where trust must be established by a third
    party. (Browsers include a list of public certificate authorities who
    can issue website certificates it trusts.) 

The server certificate should be issued by a certificate authority root
outside of your organization only if you want people outside of your
organization to trust your server certificate, but you are then asking
for the client to only trust an intermediate inside your organization. 
The big question is why bother having the server certificate chain to a
root certificat you don't trust when you have no intention of having
clients outside of your organization trust the server certificate. 
Postgres could be made to handle such cases, but is is really a valid
configuration we should support?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee

On Wed, Jun  3, 2020 at 03:07:30PM +0300, Ants Aasma wrote:
> On Tue, 2 Jun 2020 at 20:14, Bruce Momjian <bruce@momjian.us> wrote:
> 
>     The server certificate should be issued by a certificate authority root
>     outside of your organization only if you want people outside of your
>     organization to trust your server certificate, but you are then asking
>     for the client to only trust an intermediate inside your organization.
>     The big question is why bother having the server certificate chain to a
>     root certificat you don't trust when you have no intention of having
>     clients outside of your organization trust the server certificate.
>     Postgres could be made to handle such cases, but is is really a valid
>     configuration we should support?
> 
> 
> I think the "why" the org cert is not root was already made clear, that is the
> copmany policy. I don't think postgres should take a stance whether the
> certificate designated as the root of trust is self-signed or claims to get its
> power from somewhere else.

Uh, we sure can.  We disallow many configurations that we consider
unsafe.  openssl allowed a lot of things, and their flexibility make
them less secure.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee

On 06/03/20 08:07, Ants Aasma wrote:
> I think the "why" the org cert is not root was already made clear, that is
> the copmany policy.

Thank you, yes, that was what I had intended to convey, and you have saved
me finishing a weedsier follow-up message hoping to convey it better.

> I don't think postgres should take a stance ...

> whether the
> certificate designated as the root of trust is self-signed or claims to get
> its power from somewhere else.

On 06/03/20 16:34, Bruce Momjian wrote:
> Uh, we sure can.  We disallow many configurations that we consider
> unsafe.

Ok, so a person in the situation described here, who is not in a position
to demand changes in an organizational policy (whether or not it seems
ill-conceived to you or even to him/her), is facing this question:

What are the "safest" things I /can/ do, under the existing constraints,
and /which of those will work in PostgreSQL/?

For example, we might agree that it is safe to trust nothing but the
end-entity cert of my server itself. I made a server, here is its cert,
here is a root.crt file for libpq containing only this exact cert, I
want libpq to connect only ever to this server with this cert and nothing
else. It's a pain because I have to roll out new root.crt files to everybody
whenever the cert changes, but it would be hard to call it unsafe.

Great! Can I do that? I think the answer is no. I haven't found it
documented, but I think libpq will fail such a connection, because
the cert it has found in root.crt is not self-signed.

Or, vary the scenario just enough that my organization, or even my
department in my organization, now has its own CA, as the first
intermediate, the issuer of the end-entity cert.

It might be entirely reasonable to put that CA cert into root.crt,
so libpq would only connect to things whose certs were issued in
my department, or at least my org. I trust the person who would be
issuing my department's certs (in all likelihood, me). I would
more-or-less trust my org to issue certs for my org.

Great! Can I do that? I think that answer is also no, for the same
reason.

My department's or org's CA cert isn't going to be self-signed, it's
going to be vouched for by a chain of more certs leading to a globally
recognized one. Why? Short answer, our org also has web sites. We like
for people's browsers to be able to see those. We have one group that
makes server certs and they follow one procedure, and the certs they make
come out the same way. That shouldn't be a problem for PostgreSQL, so it's
hard to argue they should have to use a different procedure just for my
cert.

> I don't think postgres should take a stance whether the
> certificate designated as the root of trust is self-signed or claims
> to get its power from somewhere else.

I'm inclined to agree but I would change the wording a bit for clarity.
*Any* certificate we are going to trust gets its power from somewhere
else. Being self-signed is not an exception to that rule (if it were,
every Snake Oil, Ltd. self-signed cert generated by every student in
every network security class ever would be a root of trust).

For us to trust a cert, it must be vouched for in some way. The most
important vouching that happens, as far as libpq is concerned, is
vouching by the administrator who controls the file system where root.crt
is found and the contents of that file.

If libpq is looking at a cert and finds it in that file, I have vouched
for it. That's why it's there.

If it is self-signed, then I'm the only person vouching for it, and that's
ok.

If it is not self-signed, that just means somebody else also has vouched
for it. Maybe for the same use, maybe for some other use. In any event,
the fact that somebody else has also vouched for it does not in any way
negate that I vouched for it, by putting it there in that file I control.

> It's pretty easy to conceive of certificate management procedures that make
> use of this chain to implement certificate replacement securely. For
> example one might trust the global issuer to verify that a CSR is coming
> from the O= value that it's claiming to come from to automate replacement
> of intermediate certificates, but not trust that every other sub-CA signed
> by root and their sub-sub-CA-s are completely honest and secure.

That's an example of the kind of policy design I think ought to be possible,
but a first step to getting there would be to just better document what
does and doesn't work in libpq now. There seem to be some possible
configurations that aren't available, not because of principled arguments
for disallowing them, but because they fail unstated assumptions.

In an ideal world, I think libpq would be using this algorithm:

  I'm looking at the server's certificate, s.
  Is s unexpired and in the trust file? If so, SUCCEED.

  otherwise, loop:
    get issuer certificate i from s (if s is self-signed, FAIL).
    does i have CA:TRUE and Certificate Sign bits? If not, FAIL.
    does i's Domain Constraint allow it to sign s? If not, FAIL.
    is i unexpired, or has s a Signed Certificate Timestamp made
     while i was unexpired? If not, FAIL.
    is i in the trust file? If so, SUCCEED.
    s := i, continue.

(I left out steps like verify signature, check revocation, etc.)

What it seems to be doing, though, is just:

  I'm looking at s
  Follow chain all the way to a self-signed cert
  is that in the file?

which seems too simplistic.

Regards,
-Chap

On Wed, 2020-06-03 at 19:57 -0400, Chapman Flack wrote:
> Ok, so a person in the situation described here, who is not in a position
> to demand changes in an organizational policy (whether or not it seems
> ill-conceived to you or even to him/her), is facing this question:
> 
> What are the "safest" things I /can/ do, under the existing constraints,
> and /which of those will work in PostgreSQL/?

I feel bad about bending the basic idea of certificates and trust to suit
some misbegotten bureaucratic constraints on good security.

If you are working for a company that has a bad idea of security
and cannot be dissuaded from it, you point that out loudly and then
keep going.  Trying to subvert the principles of an architecture
very often leads to pain in my experience.

Yours,
Laurenz Albe

On 06/04/20 02:07, Laurenz Albe wrote:
> I feel bad about bending the basic idea of certificates and trust to suit
> some misbegotten bureaucratic constraints on good security.

Can you elaborate on what, in the email message you replied to here,
represented a bending of the basic idea of certificates and trust?

I didn't notice any.

Regards,
-Chap

On Thu, 2020-06-04 at 08:25 -0400, Chapman Flack wrote:
> > I feel bad about bending the basic idea of certificates and trust to suit
> > some misbegotten bureaucratic constraints on good security.
> 
> Can you elaborate on what, in the email message you replied to here,
> represented a bending of the basic idea of certificates and trust?
> 
> I didn't notice any.

I was referring to the wish to *not* use a self-signed CA certificate,
but an intermediate certificate as the ultimate authority, based on
a distrust of the certification authority that your organization says
you should trust.

Yours,
Laurenz Albe

On 06/04/20 11:04, Laurenz Albe wrote:
> I was referring to the wish to *not* use a self-signed CA certificate,
> but an intermediate certificate as the ultimate authority, based on
> a distrust of the certification authority that your organization says
> you should trust.

Are you aware of any principled reason it should be impossible to
include an end-entity certificate in the trust store used by a client?

Are you aware of any principled reason it should be impossible to
include a certificate that has the CA:TRUE and Certificate Sign bits
in the trust store used by a client, whether it is its own signer
or has been signed by another CA?

Regards,
-Chap

On 6/3/20 7:57 PM, Chapman Flack wrote:
>
> In an ideal world, I think libpq would be using this algorithm:
>
>   I'm looking at the server's certificate, s.
>   Is s unexpired and in the trust file? If so, SUCCEED.
>
>   otherwise, loop:
>     get issuer certificate i from s (if s is self-signed, FAIL).
>     does i have CA:TRUE and Certificate Sign bits? If not, FAIL.
>     does i's Domain Constraint allow it to sign s? If not, FAIL.
>     is i unexpired, or has s a Signed Certificate Timestamp made
>      while i was unexpired? If not, FAIL.
>     is i in the trust file? If so, SUCCEED.
>     s := i, continue.
>
> (I left out steps like verify signature, check revocation, etc.)
>
> What it seems to be doing, though, is just:
>
>   I'm looking at s
>   Follow chain all the way to a self-signed cert
>   is that in the file?
>
> which seems too simplistic.
>


Do we actually do any of this sort of thing? I confess my impression was
this is all handled by the openssl libraries, we just hand over the
certs and let openssl do its thing. Am I misinformed about that?


cheers


andrew


-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 06/04/20 17:31, Andrew Dunstan wrote:
> Do we actually do any of this sort of thing? I confess my impression was
> this is all handled by the openssl libraries, we just hand over the
> certs and let openssl do its thing. Am I misinformed about that?

I haven't delved very far into the code yet (my initial aim with this
thread was not to pose a rhetorical question, but an ordinary one, and
somebody would know the answer).

By analogy to other SSL libraries I have worked with, my guess would
be that there are certain settings and callbacks available that would
determine some of what it is doing.

In the javax.net.ssl package [1], for example, there are HostnameVerifier
and TrustManager interfaces; client code can supply implementations of these
that embody its desired policies.

Regards,
-Chap

Chapman Flack <chap@anastigmatix.net> writes:
> On 06/04/20 17:31, Andrew Dunstan wrote:
>> Do we actually do any of this sort of thing? I confess my impression was
>> this is all handled by the openssl libraries, we just hand over the
>> certs and let openssl do its thing. Am I misinformed about that?

> By analogy to other SSL libraries I have worked with, my guess would
> be that there are certain settings and callbacks available that would
> determine some of what it is doing.

It's possible that we could force openssl to validate cases it doesn't
accept now.  Whether we *should* deviate from its standard behavior is
a fairly debatable question though.  I would not be inclined to do so
unless we find that many other consumers of the library also do that.
Overriding a library in its specific area of expertise seems like a
good way to get your fingers burnt.

            regards, tom lane

On 06/04/20 18:03, Tom Lane wrote:
> It's possible that we could force openssl to validate cases it doesn't
> accept now.  Whether we *should* deviate from its standard behavior is
> a fairly debatable question though.  I would not be inclined to do so
> unless we find that many other consumers of the library also do that.
> Overriding a library in its specific area of expertise seems like a
> good way to get your fingers burnt.

Sure. It seems sensible to me to start by documenting /what/ it is doing
now, and to what extent that should be called "its standard behavior"
versus "the way libpq is calling it", because even if nothing is to be
changed, there will be people who need to be able to find that information
to understand what will and won't work.

Regards,
-Chap

Chapman Flack <chap@anastigmatix.net> writes:
> Sure. It seems sensible to me to start by documenting /what/ it is doing
> now, and to what extent that should be called "its standard behavior"
> versus "the way libpq is calling it", because even if nothing is to be
> changed, there will be people who need to be able to find that information
> to understand what will and won't work.

Fair enough.  I'm certainly prepared to believe that there might be things
we're doing with that API that are not (anymore?) considered best
practice.  But I'd want to approach any changes as "what is considered
best practice", not "how can we get this predetermined behavior".

            regards, tom lane

On Wed, Jun  3, 2020 at 07:57:16PM -0400, Chapman Flack wrote:
> For example, we might agree that it is safe to trust nothing but the
> end-entity cert of my server itself. I made a server, here is its cert,
> here is a root.crt file for libpq containing only this exact cert, I
> want libpq to connect only ever to this server with this cert and nothing
> else. It's a pain because I have to roll out new root.crt files to everybody
> whenever the cert changes, but it would be hard to call it unsafe.

I think you have hit on the reason CAs are used.  By putting a valid
root certificate on the client, the server certificate can be changed
without modifying the certificate on the client.

Without that ability, every client would need be changed as soon as the
server certificate was changed.  Allowing intermediate certificates to
function as root certificates would fix that problem.  When the
non-trusted CA changes your certificate, you are going to have the same
problem updating everything at once.  This is why a root certificate,
which never changes, is helpful.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee

On 06/12/20 15:13, Bruce Momjian wrote:
> On Wed, Jun  3, 2020 at 07:57:16PM -0400, Chapman Flack wrote:
>> here is a root.crt file for libpq containing only this exact cert, I
>> want libpq to connect only ever to this server with this cert and nothing
>> else. It's a pain because I have to roll out new root.crt files to everybody
>> whenever the cert changes, but it would be hard to call it unsafe.
> 
> I think you have hit on the reason CAs are used.  By putting a valid
> root certificate on the client, the server certificate can be changed
> without modifying the certificate on the client.
> 
> Without that ability, every client would need be changed as soon as the
> server certificate was changed.  Allowing intermediate certificates to
> function as root certificates would fix that problem.  When the
> non-trusted CA changes your certificate, you are going to have the same
> problem updating everything at once.

There seems to be a use of language here that works to make the picture
muddier rather than clearer.

I mean the use of "trusted"/"non-trusted" as if they somehow mapped onto
"self-signed"/"not self-signed" (unless you had some other mapping in mind
there).

That's downright ironic, as a certificate that is self-signed is one that
carries with it the absolute minimum grounds for trusting it: precisely
zero. There can't be any certificate you have less reason to trust than
a self-signed one.

(Ok, I take it back: a certificate you find on a revocation list /might/
be one you have less reason to trust.)

If a certificate, signed only by itself, ends up being relied on by
a TLS validator, that can only be because it is trusted for some other
reason. Typically that reason is that it has been placed in a file that
can only be edited by the admin who decides what certs to trust. By
editing it into that file, that responsible person has vouched for it,
and /that/ is why the TLS client should trust it. The fact that it is
self-signed, meaning only that nobody else ever vouched for it anywhere,
has nothing to do with why the TLS client should trust it.

Now, suppose that same responsible person edits that same file, but this
time places in it a cert that has been signed by some other authority.
That is a cert that has been vouched for in two ways: by the admin
placing it in this file, and by some other PKI authority.

As far as the TLS client is concerned, the endorsement that counts is
still the local one, that it has been placed in the local file by the
admin responsible for deciding what this client should trust. The fact
that somebody else vouched for it too is no reason for this client
to trust it, but is also no reason for this client not to trust it.
It is certainly in no way less to be trusted than a cert signed only
by itself.

The key point is that as soon as you find the cert you are looking at
in the local file curated by your admin, you know you've been cleared
to trust what you're looking at.

If the cert you're looking at is not in that file, and it has no signer
but itself, you must at that point fail. Dead end. There can be no reason
to trust it.

On the other hand, if you are looking a cert that has a signer, you have
not hit a dead end yet; you can climb that link and hope to find the signer
in your curated file, and so on.

You need to climb until you find something that's in that curated file.
Every step that you climbed needs to have had a valid signature made
while the signer was valid and not revoked, the signer needed to be allowed
to sign certs, and to sign certs for the subject's domain. Those things
needed to be checked at every step.

But once you have followed those steps and arrived at a cert that
was placed in your trust store by the admin, it's unnecessary and
limiting to insist arbitrarily on other properties of the cert you
found there.

> This is why a root certificate, which never changes, is helpful.

But who says it never changes?

As I mentioned earlier, my org has not always had its current procedures
on the issuance of certs, and not so many years ago did have its own
in-house CA. I ran across a copy of that CA cert recently. It was generated
in July of 2010 and is still good through next month. (I have not checked
to see whether the former in-house CA made a revocation entry somewhere
for it before turning the lights out.)

If we were still using that CA cert, I would still have to roll out
new root.crt files next month. I'm sure at the time it was generated,
ten years seemed like 'almost never', and like a reasonable time in which
to hope no adversary would crack a 2048 bit RSA key.

One certainly wouldn't plan on giving an important cert a lifetime
much longer than that.

So the benefit of putting a signing cert in root.crt is not so much
that it will never expire and need updating, but that you can keep
using it to sign other certs for new services you stand up or update,
and so you don't have to distribute new root.crt files every time you
do those things.

For that purpose, it matters not whether the signing cert you put there
is self-signed or not.

Regards,
-Chap

On 06/12/20 16:17, Chapman Flack wrote:
> reason. Typically that reason is that it has been placed in a file that
> can only be edited by the admin who decides what certs to trust. By
> editing it into that file, that responsible person has vouched for it,
> and /that/ is why the TLS client should trust it.

In order to wave my hands less, and map more easily onto the RFCs,
I ought to start saying these things:

Relying Party       when I mean libpq in its role of validating a server

Trust Anchor Store  when I mean libpq's root.crt file

Trust Anchor Manager  when I mean me, putting a thing into root.crt

Trust Anchor        when I mean a thing I put into root.crt

Target Certificate  the one the Relying Party wants to validate; in this
                    case, the end-entity cert assigned to the pgsql server

Certification Path Validation   the algorithm in RFC 5280 sec. 6

Certification Path Building     the task described in RFC 4158



RFC 5280 expresses the Path Validation algorithm as starting from
a Trust Anchor and proceeding toward the Target Certificate. In this
thread so far I've been waving my hands in the other direction, but
that properly falls under Path Building.

If your Trust Anchor Store contains only one Trust Anchor, then the
Path Validation algorithm is all you need. If there may be multiple
Trust Anchors there, Path Building is the process of enumerating
possible paths with a Trust Anchor at one end and the Target Certificate
at the other, in the hope that Path Validation will succeed for at least
one of them.

RFC 4158 isn't prescriptive: it doesn't give one way to build paths, but
a smörgåsbord of approaches. Amusingly, what it calls "forward path
building" is when you start from the Target Certificate and search toward
a Trust Anchor (same way I've been waving my hands, but reversed w.r.t.
Path Validation), and what it calls "reverse path building" is when you
start with your Trust Anchors and search toward the Target Certificate
(the same direction as Path Validation).

RFC 4158 has an extensive gallery of ASCII art showing what the PKI
can end up looking like in some large enterprises.  :O



Initial inputs to Path Validation include a distinguished name and
a public key, optionally with some constraints, and those things come
from a Trust Anchor. There is no requirement that a Trust Anchor be
a cert, signed, self-signed, or otherwise. The certificate format has
often been used as a Trust Anchor container format because, hey, it
holds a distinguished name and a public key and constraints, and if
you're writing a path validator, you already have a parser for it.

Other things that happen to be present in a certificate-as-Trust-Anchor,
such as an issuer name, key, and signature, are non-inputs to the path
validation algorithm and have no effect on it.

Disappointingly, common implementations have tended also to ignore
constraints held in a certificate-as-Trust-Anchor, even though they
correctly apply constraints in other certs encountered along the path,
and initial constraints are supposed to be inputs to the algorithm.
It is the point of RFC 5937 to fix that.

'Constraints' in this context are limits such as "this cert is for
identifying servers" or "this is a CA cert but only allowed to sign
certs for *.example.com". The RFCs plainly anticipate that I might
want to put new constraints on a Trust Anchor, say to use the cert
of a CA that has other customers, without implying a trust relationship
with their other customers.

For that purpose, the historical 'convenience' of using certificates
as Trust Anchor containers is a genuine hindrance, because of course
certificates are cryptographically signed objects so editing their
constraints can't be easily done.[1]

The Trust Anchor Format (cited in [1] as "in progress" but since published
as RFC 5914) therefore proposes a couple alternatives to the use of a
certificate as an ersatz Trust Anchor container.

RFC 6024, Trust Anchor Management Requirements, sets out the considerations
RFC 5914 and RFC 5934 were to address. (In classic see-I-did-it-perfectly
fashion, it was published after they were.)


So, if libpq had a Trust Anchor Store that worked as described in these
RFCs, my use case would be trivial to set up.

I guess the next question is to what extent recent OpenSSL groks those,
or how far back was the first version that did (these RFCs are from 2010),
and what would be entailed in taking advantage of that support if it's
present.

Regards,
-Chap


[1] https://dl.acm.org/doi/10.1145/1750389.1750403

On Fri, Jun 12, 2020 at 04:17:56PM -0400, Chapman Flack wrote:
> On 06/12/20 15:13, Bruce Momjian wrote:
> > Without that ability, every client would need be changed as soon as the
> > server certificate was changed.  Allowing intermediate certificates to
> > function as root certificates would fix that problem.  When the
> > non-trusted CA changes your certificate, you are going to have the same
> > problem updating everything at once.
> 
> There seems to be a use of language here that works to make the picture
> muddier rather than clearer.
> 
> I mean the use of "trusted"/"non-trusted" as if they somehow mapped onto
> "self-signed"/"not self-signed" (unless you had some other mapping in mind
> there).

I meant you trust your local/intermediate CA, but not the root one. 

> That's downright ironic, as a certificate that is self-signed is one that
> carries with it the absolute minimum grounds for trusting it: precisely
> zero. There can't be any certificate you have less reason to trust than
> a self-signed one.

Self-signed certs can certainly be trusted by the creator. Organizations
often create self-signed certs that are trusted inside the organization.

> As far as the TLS client is concerned, the endorsement that counts is
> still the local one, that it has been placed in the local file by the
> admin responsible for deciding what this client should trust. The fact
> that somebody else vouched for it too is no reason for this client
> to trust it, but is also no reason for this client not to trust it.
> It is certainly in no way less to be trusted than a cert signed only
> by itself.

Yes, I see your point in that the intermediate has more validity than a
self-signed certificate, though that extra validity is useless in the
use-case we are describing.

> But once you have followed those steps and arrived at a cert that
> was placed in your trust store by the admin, it's unnecessary and
> limiting to insist arbitrarily on other properties of the cert you
> found there.

Well, I can see the use-case for what you are saying, but I also think
it could lead to misconfiguration.  Right now, Postgres uses the client
root.cert, which can contain intermediates certs, and the
server-provided cert, which can also contain intermediates shipped to
the client, to try to check for a common root:

    https://www.postgresql.org/docs/13/ssl-tcp.html

What you are suggesting is that we take the server chain and client
chain and claim success when _any_ cert matches between the two, not
just the root one.

I can see that working but I can also imagine people putting only
intermediate certs in their root.cert and not realizing that they are
misconfigured since they might want to expire the intermediate someday
or might want to trust a different intermediate from the same root. 
Frankly, we really didn't even documention how to handle intermediate
certificates until 2018, which shows how obscure this security stuff can
be:

    https://git.postgresql.org/gitweb/?p=postgresql.git&a=commitdiff&h=815f84aa16

Do we want to allow such cases, or is the risk of misconfiguration too
high?  I am thinking it is the later.  I think we could have a libpq
parameter that allowed it, but is there enough demand to add it since it
would be a user-visible API?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee