Patch to "configure" to enable PostgreSQL build with Kerberos 5 on Solaris 11 - Mailing list pgsql-patches

Included below are extracts from an earlier email thread (on
pgsql-ports) discussing the problem.

Attached are the context diffs for configure.in.

This change has no impact unless the "--with-krb5" option is used with
"configure". If the option *is* used, configure will now only search for
function krb5_sendauth(), instead of looking for both krb5_encrypt() and
krb5_sendauth().

I've tested (i.e. built using --with-krb5) with version 8.1.4 on Solaris
11 only. This change should have no negative impact for builds on other
platforms since:

a) The check for krb5_sendauth() remains, which is sufficient to
determine the presence of Kerberos 5

and

b) None of the PostgreSQL code uses krb5_encrypt() anyway


James Gates wrote:
 > Prior to Solaris 11 (Nevada), the full Kerberos 5 API was never exposed
 > (only the gss interface), so building PostgreSQL with the "--with-krb5"
 > option is a problem.
 >
 > In Nevada, Sun has exposed the full MIT Kerberos 5 API (v1.4.0). So
 > building PostgreSQL with Kerberos should be possible/easy. If I try to
 > build 8.1.4 though, it fails with the following error:
 >
 > $ ./configure --with-krb5 --without-readline
 > checking build system type... sparc-sun-solaris2.11
 > checking host system type... sparc-sun-solaris2.11
 > ... snip ...
 > checking for library containing com_err... -lkrb5
 > checking for library containing krb5_encrypt... no
 > configure: error: could not find function 'krb5_encrypt' required for
 > Kerberos 5
 >
 > This is because in krb5 v1.4.0, the krb5_encrypt() function is
 > deprecated/removed, so doesn't exist anywhere in the Solaris libraries.
 > It is replaced by krb5_c_encrypt() (I think this change occurred
 > sometime between krb5 v1.2.1 and v1.4.0)
 >
 > But looking more closely at the PostgreSQL 8.1.4 code, I see that it
 > never even uses the krb5_encrypt() function anyway! So although it's
 > presence might be a useful method for detecting the presence of Kerberos
 > 5 (pre v1.4.0), it seems unnecessary for the successful operation of
 > PostgreSQL.
 >
 > By simply removing the check for krb5_encrypt() from the configure
 > script, I can successfully build PostgreSQL with krb5 on Nevada.
 >
 > Does anyone know why the check for krb5_encrypt() exists in configure
 > when the code doesn't use it? And would absence of a good reason
 > indicate this is a bug (and the check should be removed)?

Tom Lane wrote:
 > James Gates <James.Gates@Sun.COM> writes:
 >> Does anyone know why the check for krb5_encrypt() exists in configure
 >> when the code doesn't use it?
 >
 > At the time it was chosen, it was probably a reasonable choice of
 > function to probe for to make sure Kerberos libraries are present.
 > Do you have a better suggestion?
 >
 >             regards, tom lane

James Gates wrote:
 > The configure script already checks for krb5_sendauth() as well as
 > krb5_encrypt(). The libpq code *does* use krb5_sendauth(), which is not
 > deprecated (and I know of no plans to make it so).
 >
 > I discussed this problem last night with Magnus Hagander, and we're both
 > of the opinion that the search for krb5_sendauth() alone is sufficient
 > to determine if krb5 is present on your system.
 >
 > Magus suspects that at some point in the past, PostgreSQL did use
 > krb5_encrypt(), and it was changed (maybe in anticipation of the
 > function becoming deprecated?). Whoever made the change,
 > forgot/neglected to remove the check from the configure script.
 >
 > I propose that we remove the check for krb5_encrypt() from the configure
 > script, leaving just the check for krb5_sendauth().
 >
 > Note - krb5_encrypt() is replaced by krb5_c_encrypt() in the latest
 > implementation of krb5. We could change the configure script to check
 > for this as well, but as mentioned above, I think this is unnecessary.

$ runsocks cvs diff -c configure.in
Index: configure.in
===================================================================
RCS file: /projects/cvsroot/pgsql/configure.in,v
retrieving revision 1.467
diff -c -r1.467 configure.in
*** configure.in        18 Jun 2006 18:30:20 -0000      1.467
--- configure.in        10 Jul 2006 20:56:44 -0000
***************
*** 671,678 ****
    if test "$PORTNAME" != "win32"; then
       AC_SEARCH_LIBS(com_err, [krb5 'krb5 -ldes -lasn1 -lroken' com_err], [],
                      [AC_MSG_ERROR([could not find function 'com_err' required for Kerberos 5])])
-      AC_SEARCH_LIBS(krb5_encrypt, [krb5 'krb5 -ldes -lasn1 -lroken' crypto k5crypto], [],
-                     [AC_MSG_ERROR([could not find function 'krb5_encrypt' required for Kerberos 5])])
       AC_SEARCH_LIBS(krb5_sendauth, [krb5 'krb5 -ldes -lasn1 -lroken'], [],
                      [AC_MSG_ERROR([could not find function 'krb5_sendauth' required for Kerberos 5])])
    else
--- 671,676 ----