Thread: A successor for PQgetssl
There was some discussion about the issues relating to using other SSL
libraries. In a nutshell, it came down to that we couldn't return
anything other than an OpenSSL pointer from PQgetssl because existing
programs simply wouldn't know what to do with it.
So, I was pondering what we might want from an alternative. What I've
come up with is the following:
PGresult *PQgettlsinfo(PGconn *conn);
What it does instead of returning a single pointer is return a PGresult
that has various info depending on the library involved. For example,
if you connected using a libpq compiled with GnuTLS it would contain
the following:
key | value
---------------------+---------------------------------------tls_library | GnuTLStls_library_version |
1.0.16tls_sslmode | prefertls_active | yestls_verify_server | yestls_peerdn |
C=AU,ST=NSW,L=Sydney,O=Home,CN=Myselftls_peercn | Myselftls_protocol | TLS 1.0tls_cipher | AES
256CBCtls_keysize | 256 bitstls_kx | DHE RSAtls_mac | SHAtls_compression |
NULLtls_certtype | X.509
(14 rows)
And when you connected with OpenSSL you would get something like:
key | value
---------------------+----------------------------------------tls_library | OpenSSLtls_library_version |
OpenSSL0.9.7e 25 Oct 2004tls_sslmode | prefertls_active | yestls_peerdn |
/C=AU/ST=NSW/L=Sydney/O=Home/CN=Myselftls_peercn | Myselftls_cipher | DHE-RSA-AES256-SHAtls_protocol
| TLSv1/SSLv3tls_keysize | 256 bits
(9 rows)
Now, other than for the first time giving users access to the
information like peer DN and CN, it also provides some other
information they might want. And it's done in a way that's extensible.
Do people like this idea?
Note, I don't return a pointer to the GnuTLS session anywhere. I think
that's a bad idea all round and we need to provide another way for
programs to acheive the same effect.
The thing is, it could be extended to include almost anything. One
example would be if the user authenticated using kerberos, we could add
a few rows indicating that. I suppose you would call it
PQgetconninfo().
Thoughts?
*** PostgreSQL with GnuTLS
I've got it almost completely working and have tested interoperability.
You can find it here:
http://svana.org/kleptog/temp/gnutls.patch
The patch does the following:
- Adds configure stuff for gnutls so it checks for the libraries when
you specify --with-gnutls. You may need to run autoconf and autoheader
after patching.
- Both fe-secure.c and be-secure.c have been made TLS library agnostic.
They only refer to functions that implement TLS specific stuff which
are implemented in the files:
src/interfaces/libpq/fe-secure-openssl.c
src/interfaces/libpq/fe-secure-gnutls.c
src/backend/libpq/be-secure-openssl.c
src/backend/libpq/be-secure-gnutls.c
The makefile determines which (if either) is linked in.
- Implements the PQgettlsinfo() as described above and alters psql to
use it. Hence psql is now also TLS library agnostic.
Differences between the two implementations are:
- GnuTLS generates the DH key on the fly on server start, which takes a
few seconds. The OpenSSL versions use hardcoded keys which can be
overridden by the user. Not sure which is best here.
- This breaks psqlODBC when it uses libpq because it wants to use OpenSSL
and when libpq is compiled with GnuTLS that obviously won't work.
Recent thread on -hackers found no resolution for this problem.
- Both support authentication of the server and authentication of the
client, though more testing is need to test all the different
combinations of keys and certificates that are allowed.
- Different output for PQgettlsinfo()
That about it. There's no real difference from the users point of view,
it Just Works either way. In the future we may be able to use the PGP
support in GnuTLS. In other words, provide the server with a pgp
keyring and it accepts any user which a matching key in the keyring.
I hope to post of -patches sometime soon, once some of the kinks have
been ironed out.
Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.
Martijn van Oosterhout <kleptog@svana.org> writes:
> Do people like this idea?
Not really ...
> Note, I don't return a pointer to the GnuTLS session anywhere. I think
> that's a bad idea all round and we need to provide another way for
> programs to acheive the same effect.
No, failing to provide that is the bad idea, because then you're buying
into the notion that libpq will provide a universal API that will
incorporate anything anyone could possibly want to do with the
underlying SSL library. The above is *not* that, prima facie because
it is read-only access. Even if this was a feasible goal, it would absorb
a lot of time on our part that could be better spent elsewhere, plus a
lot of time on the part of app programmers rewriting existing
OpenSSL-aware or GnuTLS-aware code to instead use whatever random API we
tell them they ought to use. They have better things to do with their
time, too.
> I've got it almost completely working and have tested interoperability.
> ...
> - This breaks psqlODBC when it uses libpq because it wants to use OpenSSL
> and when libpq is compiled with GnuTLS that obviously won't work.
That alone is sufficient reason why we're not going down that path.
If we expose a GnuTLS-handle-fetching API then it's up to the ODBC
guys to extend their code to handle that SSL library when they feel
like it. But telling them that we're simply going to break their
code and not provide them a path to fix it is not happening.
regards, tom lane
On Sun, Apr 16, 2006 at 05:29:04PM -0400, Tom Lane wrote: > No, failing to provide that is the bad idea, because then you're buying > into the notion that libpq will provide a universal API that will > incorporate anything anyone could possibly want to do with the > underlying SSL library. The above is *not* that, prima facie because > it is read-only access. Even if this was a feasible goal, it would absorb The intention is not to provide access to everything. If people want to know more about the certificate, we simply export the certificate to them and they can do with it what they like, including sending it to another program. I wasn't expecting that list to grow much because there's not much to export. Besides, what's wrong with read-only access? What parameters were you expecting them to want to change? After a session is setup there are no parameters to change anymore. All you need is read and write. > a lot of time on our part that could be better spent elsewhere, plus a > lot of time on the part of app programmers rewriting existing > OpenSSL-aware or GnuTLS-aware code to instead use whatever random API we > tell them they ought to use. They have better things to do with their > time, too. The whole point is that app writers should not be aware at all which library we're using. At the moment psqlODBC requires openssl because we force them to. They only use three OpenSSL functions, SSL_read, SSL_write, and SSL_get_error. If we provided a hook to allow people read/write directly, they wouldn't need to know about the SSL connection at all. I think that's a much better way to go than adding a new library specific function for every little feature we add. You objected to this on the grounds of a problem with the COPY functions, except I can't see any problem that's relevent. The problem with copy was that data didn't have a length. Given the user is sending their own packets, we always have a length. > > - This breaks psqlODBC when it uses libpq because it wants to use OpenSSL > > and when libpq is compiled with GnuTLS that obviously won't work. > > That alone is sufficient reason why we're not going down that path. > If we expose a GnuTLS-handle-fetching API then it's up to the ODBC > guys to extend their code to handle that SSL library when they feel > like it. But telling them that we're simply going to break their > code and not provide them a path to fix it is not happening. By going down this path you're saying that psql will never be able display the cipher of an SSL connection if the libpq was compiled with a different library. If we provide a read/write than psqlODBC can remove code and it will work with GnuTLS. Isn't that much better? Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do the other 95% so you can sue them.
Martijn van Oosterhout <kleptog@svana.org> writes:
> On Sun, Apr 16, 2006 at 05:29:04PM -0400, Tom Lane wrote:
>> No, failing to provide that is the bad idea, because then you're buying
>> into the notion that libpq will provide a universal API that will
>> incorporate anything anyone could possibly want to do with the
>> underlying SSL library. ...
> [ snip ]
> Besides, what's wrong with read-only access?
Well, psqlODBC seems a sufficient counterexample. But the problem
with this is that you're asking a bunch of non-SSL-experts to design,
evaluate, and then maintain an API for an SSL library. The real
answer to the above is "I don't know, and I doubt you do either."
This is the sort of problem that we should be avoiding, rather than
going out of our way to get involved in. PQgetssl made it possible
for us to stay out of the way for SSL-using applications, and I think
we should continue to follow that philosophy for other SSL libraries.
regards, tom lane
* Martijn van Oosterhout (kleptog@svana.org) wrote:
> The intention is not to provide access to everything. If people want to
> know more about the certificate, we simply export the certificate to
> them and they can do with it what they like, including sending it to
> another program. I wasn't expecting that list to grow much because
> there's not much to export.
>
> Besides, what's wrong with read-only access? What parameters were you
> expecting them to want to change? After a session is setup there are no
> parameters to change anymore. All you need is read and write.
I have to agree with this... Certificate handling isn't growing new
things in leaps and bounds these days and as long as the certificates
are available to the user then the other things are really just
nicities for people who don't want to decode the certs themselves.
> > a lot of time on our part that could be better spent elsewhere, plus a
> > lot of time on the part of app programmers rewriting existing
> > OpenSSL-aware or GnuTLS-aware code to instead use whatever random API we
> > tell them they ought to use. They have better things to do with their
> > time, too.
>
> The whole point is that app writers should not be aware at all which
> library we're using. At the moment psqlODBC requires openssl because we
> force them to. They only use three OpenSSL functions, SSL_read,
> SSL_write, and SSL_get_error.
>
> If we provided a hook to allow people read/write directly, they
> wouldn't need to know about the SSL connection at all. I think that's a
> much better way to go than adding a new library specific function for
> every little feature we add.
I have to agree with Martijn here too. It's not all that expensive to
provide read/write calls to abstract away the specific library being
used (since psqlODBC, at least, couldn't care less which library is
being used, really) rather than ask for application developers to write
their apps to support multiple SSL libraries to handle the case where
libpq is compiled with one library vs. compiled with another...
> > > - This breaks psqlODBC when it uses libpq because it wants to use OpenSSL
> > > and when libpq is compiled with GnuTLS that obviously won't work.
> >
> > That alone is sufficient reason why we're not going down that path.
> > If we expose a GnuTLS-handle-fetching API then it's up to the ODBC
> > guys to extend their code to handle that SSL library when they feel
> > like it. But telling them that we're simply going to break their
> > code and not provide them a path to fix it is not happening.
>
> By going down this path you're saying that psql will never be able
> display the cipher of an SSL connection if the libpq was compiled with
> a different library. If we provide a read/write than psqlODBC can
> remove code and it will work with GnuTLS. Isn't that much better?
I didn't really understand Tom's point here either... In order to not
break psqlODBC when libpq is compiled with GnuTLS we'd have to code up
support for returning an SSL* from the GnuTLS library that would work
for all the OpenSSL functions which just isn't going to happen... I
agree that we probably shouldn't go out of our way to break psqlODBC
when libpq is compiled with OpenSSL (and so we shouldn't really change
PQgetssl but mark it obsolete and ask that people don't use it and
provide some other way for psqlODBC to do what it wants, or not, perhaps
return NULL when using GnuTLS or '0x1', or not define it when libpq is
compiled with GnuTLS...).
Thanks,
Stephen
Stephen Frost <sfrost@snowman.net> writes:
> I have to agree with Martijn here too. It's not all that expensive to
> provide read/write calls to abstract away the specific library being
> used (since psqlODBC, at least, couldn't care less which library is
> being used, really)
You're failing to consider async applications. AFAICS, the *minimum*
API would bereadwriteread ready?write ready?get socket so I can use it in select()
(very possibly there's some stuff I missed, considering I haven't
consumed any caffeine yet today...). And that's just considering
the data transport aspect of it. I'm still concerned that SSL-using
apps may wish to twiddle the SSL library in ways we don't even know
about.
regards, tom lane
On Mon, Apr 17, 2006 at 11:07:26AM -0400, Stephen Frost wrote: > I didn't really understand Tom's point here either... In order to not > break psqlODBC when libpq is compiled with GnuTLS we'd have to code up > support for returning an SSL* from the GnuTLS library that would work > for all the OpenSSL functions which just isn't going to happen... I > agree that we probably shouldn't go out of our way to break psqlODBC > when libpq is compiled with OpenSSL (and so we shouldn't really change > PQgetssl but mark it obsolete and ask that people don't use it and > provide some other way for psqlODBC to do what it wants, or not, perhaps > return NULL when using GnuTLS or '0x1', or not define it when libpq is > compiled with GnuTLS...). Just mark PQgetssl() obsolete. We have to return a NULL or a valid pointer, otherwise existing programs will just crash. The major reason for this PQgettlsinfo() function is so people can actually *know* which library is active. Consider stuff like this: PQgetssl() not NULL -> using openssl PQgetgnutls not NULL -> using gnutls else not using ssl OR using some unknown library That's not a way to design an interface. This PQgettlsinfo() would tell you, one way or the other, if there is a TLS library in use. Give the user the certificates and the cipher and if the user wants more info, they can use the SSL library of their choice to get the information they want. Note, some of the info there is not stuff you can actually get from the PQgetssl() function currently anyway. You can't ask the SSL library easily if we provided a certificate during authentication. libpq knows this easily. Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do the other 95% so you can sue them.
* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Stephen Frost <sfrost@snowman.net> writes:
> > I have to agree with Martijn here too. It's not all that expensive to
> > provide read/write calls to abstract away the specific library being
> > used (since psqlODBC, at least, couldn't care less which library is
> > being used, really)
>
> You're failing to consider async applications. AFAICS, the *minimum*
You're assuming I intended just 'read()' and 'write()', I suppose I
should have been more explicit but I certainly understand there's more
than just read() and write() involved, though not by much. I'm not sure
I see a reason we wouldn't just use the existing PQsocket() for the same
socket back to the user for select()-based systems (don't we do this for
SSL-enabled connections too anyway?)... There's not a whole lot special
when it comes to handling data transport, or much that has changed
lately or is likely to change anytime soon.
> consumed any caffeine yet today...). And that's just considering
> the data transport aspect of it. I'm still concerned that SSL-using
> apps may wish to twiddle the SSL library in ways we don't even know
> about.
SSL is all about the certificates, really. As long as we provide
that back to the user in a library-agnostic way they'll be able to
do whatever they like with it, such as compare it to a CRL or use OCSP
to check the current status. Of course, these are usually things you're
more concerned about on the *server* side, which we don't currently
support anyway. With supporting multiple SSL libraries we'll have to
deal with these issues on the *server* side, should we decide to
implement them someday (which would be nice...) anyway so trying to
play like we don't want to or don't care to know about them doesn't
really work. Just saying we won't do CRLs or OCSP ever would just mean
we don't get run in some environments.
Of course, in either case we can play the "if you want it, write the
code" card but I'd really expect to get complaints from admins who want
to use X.509 but need CRL/OCSP support before getting complaints from
application developers that they need libpq to provide something more
for them...
Thanks,
Stephen
On Mon, Apr 17, 2006 at 11:25:26AM -0400, Tom Lane wrote: > You're failing to consider async applications. AFAICS, the *minimum* > API would be > read > write > read ready? > write ready? > get socket so I can use it in select() Actually, you only need two functions: read write The "get socket" already exists as PQsocket(). Both SSL libraries work perfectly fine if the user sets the connection to non-blocking via PQsetnonblocking(). They just return -EAGAIN. I was actually thinking of the two functions as follows: typedef PostgresPollingStatusType pq_read_func( PGconn *conn, void *buf, int *len); typedef PostgresPollingStatusType pq_write_func( PGconn *conn, const void *buf, int *len); The existing PostgresPollingStatusType seems to handle both blocking and non-blocking states just fine. > (very possibly there's some stuff I missed, considering I haven't > consumed any caffeine yet today...). And that's just considering > the data transport aspect of it. I'm still concerned that SSL-using > apps may wish to twiddle the SSL library in ways we don't even know > about. Well, I checked a few libs like libcurl and libldap. They generally allow you to configure the files containing the certificates but that's about it. Seriously, if people want to do really sophisticated things with the SSL library, they should setup s_tunnel instead. If we wanted to let users control everything we'd allow Anonymous DH key transfers. The README.SSL lists the situations were SSL makes sense and if you fall outside of that you shouldn't be using SSL. All I'm asking for is that libpq be made SSL-library *agnostic* so that users like psqlODBC can just *use* the connection with having to jump through hoops. Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do the other 95% so you can sue them.
* Martijn van Oosterhout (kleptog@svana.org) wrote:
> Seriously, if people want to do really sophisticated things with the
> SSL library, they should setup s_tunnel instead. If we wanted to let
I certainly agree with all the rest but I'm just not sure I can agree
with you here. While s_tunnel is nice it's not always an option and I
think it *would* be nice to have Postgres support things like CRLs and
OCSP but more from the server-side of things than the client-side.
Thanks,
Stephen
On Mon, Apr 17, 2006 at 12:24:40PM -0400, Stephen Frost wrote: > * Martijn van Oosterhout (kleptog@svana.org) wrote: > > Seriously, if people want to do really sophisticated things with the > > SSL library, they should setup s_tunnel instead. If we wanted to let > > I certainly agree with all the rest but I'm just not sure I can agree > with you here. While s_tunnel is nice it's not always an option and I > think it *would* be nice to have Postgres support things like CRLs and > OCSP but more from the server-side of things than the client-side. CRLs are easy, almost a one line change. I was actually surprised it wasn't done but I didn't add it because I figured someone had left it out for a reason. OCSP is something else. And in any case, you don't need a result of PQgetssl() to use it since it's a completely seperate part of the library. But neither of these are what I considered "sophisticated". I don't think either of these require any API changes either. Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do the other 95% so you can sue them.
* Martijn van Oosterhout (kleptog@svana.org) wrote:
> On Mon, Apr 17, 2006 at 12:24:40PM -0400, Stephen Frost wrote:
> > I certainly agree with all the rest but I'm just not sure I can agree
> > with you here. While s_tunnel is nice it's not always an option and I
> > think it *would* be nice to have Postgres support things like CRLs and
> > OCSP but more from the server-side of things than the client-side.
>
> CRLs are easy, almost a one line change. I was actually surprised it
> wasn't done but I didn't add it because I figured someone had left it
> out for a reason.
I doubt there was a reason it was left out...
> OCSP is something else. And in any case, you don't need a result of
> PQgetssl() to use it since it's a completely seperate part of the
> library.
Right, I mentioned you'd just need the certificate, and that I was
talking about it on the server-side...
> But neither of these are what I considered "sophisticated". I don't
> think either of these require any API changes either.
Right, they don't require API changes (at least, not libpq) and wouldn't
provided the certificate is available. What I was getting at is that
they're more complicated SSL issues that the server *should* be able to
deal with (imv) and would require additional configuration parameters
and more of the SSL libraries than we're currently using.
Mainly I was trying to point out that trying to claim that "we don't do
anything with SSL, it's all up to the application to decide what they
want to do, and we give the application the SSL pointer so we don't have
to worry about it" doesn't really fly- we have an SSL-using application
ourselves which is the postmaster and we really should be able to
properly support it and be at least somewhat knowledgable about how it
works because SSL is a Good Thing for us to have.
Thanks,
Stephen