Thread: Compile psql 9.6 with SSL Version 1.1.0

On 10/12/18 7:15 AM, Ádám Maracska wrote:
> Hi,
> 
> Currently I am trying to compile my pqlib with SSL version 1.1.0, but I 
> get the following errors:
> /
> /
> /fe-secure-openssl.c(1582): error C2037: left of 'ptr' specifies 
> undefined struct/union 'bio_st'/
> /fe-secure-openssl.c(1582): error C2198: 'pqsecure_raw_read': too few 
> arguments for call/
> /fe-secure-openssl.c(1612): error C2037: left of 'ptr' specifies 
> undefined struct/union 'bio_st'/
> /fe-secure-openssl.c(1612): error C2198: 'pqsecure_raw_write': too few 
> arguments for call/
> /fe-secure-openssl.c(1670): error C2027: use of undefined type 
> 'bio_method_st'/
> /fe-secure-openssl.c(1673): error C2027: use of undefined type 
> 'bio_method_st'/
> /fe-secure-openssl.c(1674): error C2037: left of 'bread' specifies 
> undefined struct/union 'bio_method_st'/
> /fe-secure-openssl.c(1675): error C2037: left of 'bwrite' specifies 
> undefined struct/union 'bio_method_st'/
> /fe-secure-openssl.c(1701): error C2037: left of 'ptr' specifies 
> undefined struct/union 'bio_st'/
> 
> As I was checking the issue on google I found the following two message 
> thread about it:
> 
> - From 2016: Based on this the ssl version 1.1 is supported, but the 
> message thread had not so promising end: 
> https://www.postgresql.org/message-id/flat/E1bkVG3-0000L8-Ln%40gemulon.postgresql.org
> - From 2018: It had even more worse end. The questioner went back to 
> 1.0.2 ssl version, which is not an option for me, because that version 
> will not be supported shortly by openssl itself. 
> https://www.postgresql.org/message-id/CAAw-Mseg9JYpp%3DA%3D51HR3rKiQtbvT0MWw%2BaYFwNeJEbdNr%3DCDA%40mail.gmail.com
> 
> Also I checked the source files and I saw these kind of comments:
> / They are also not needed with OpenSSL 1.1.0 anymore. 
> (fe-secure-openssl.c)/
> /
> /
> I suppose it is possible to compile the pqlib with ssl 1.1, but I can 
> not wrap my head around it. Maybe I should set some ifdefs for the 
> precompiler beforehand?
> 
> I appreciate any help.

What OS and version of same are you compiling in?

Do you have the SSL dev libraries installed?

> 
> Best Regards,
> Adam


-- 
Adrian Klaver
adrian.klaver@aklaver.com

=?UTF-8?B?w4Fkw6FtIE1hcmFjc2th?= <csusza@gmail.com> writes:
> Currently I am trying to compile my pqlib with SSL version 1.1.0, but I get
> the following errors:
> *fe-secure-openssl.c(1582): error C2037: left of 'ptr' specifies undefined
> struct/union 'bio_st'*
> [etc]

That certainly ought to work, and does work for me and for several of
our buildfarm machines.  I suspect you messed up the build somehow;
perhaps you changed OpenSSL versions and didn't re-run configure?  Or
the OpenSSL headers you have installed don't match the library version?
Are the errors you showed the *only* ones you get?

            regards, tom lane

Pratik Parikh <pratik.p.parikh@gmail.com> writes:
> I have seen the same failure on windows build.  I can only use
> openssl 1.02(x).

Hmm ... looking at it closer, I realize that part of the problem is
that with 1.1.0 you need HAVE_BIO_METH_NEW to be defined, and there's
no mechanism for doing that in the MSVC build code.  Worse, there's
half a dozen other OpenSSL-version-dependent symbols that get set by
configure in non-Windows builds, but we have nothing equivalent to
that for Windows.  Michael P. laid out the situation in a recent commit
message:


Author: Michael Paquier <michael@paquier.xyz>
Branch: master Release: REL_11_BR [bde64eb61] 2018-06-19 09:00:33 +0900

    Track new configure flags introduced for version 11 in pg_config.h.win32
    
    The following set of flags mainly matter when building Postgres code
    with MSVC and those have been forgotten with latest developments:
    - HAVE_LDAP_INITIALIZE, added by 35c0754f, and marked as disabled.
    ldap_initialize() is a non-standard extension that provides a way to use
    "ldaps" with OpenLDAP, but it is not supported on Windows, and instead
    the non-standard ldap_sslinit() is used if WIN32 is defined.  Per input
    from Thomas Munro.
    - HAVE_X509_GET_SIGNATURE_NID, added by 054e8c6c, which is used by
    SCRAM's channel binding tls-server-end-point.  Having this flag disabled
    would cause this channel binding type to be unsupported for Windows
    builds.
    - HAVE_SSL_CLEAR_OPTIONS, added recently as of a364dfa4 to disable SSL
    compression.
    - HAVE_ASN1_STRING_GET0_DATA, added by 5c6df67, which is used to track
    a new compatibility with OpenSSL 1.1.0.  This was missing from
    pg_config.win32.h and is not enabled by default.  HAVE_BIO_GET_DATA,
    HAVE_OPENSSL_INIT_SSL and HAVE_BIO_METH_NEW gain the same treatment.
    
    The second and third flags are enabled with this commit, which raises
    the bar of OpenSSL support to 1.0.2 on Windows as a minimum.  As this
    is the LTS (long-time support) version of OpenSSL community and knowing
    that all recent installers referred by OpenSSL upstream don't have
    anymore 1.0.1 or older, we could live with that requirement.  In order
    to allow the code to compile with OpenSSL 1.1.0, all the flags mentioned
    above need to be enabled in pg_config.h.win32.
    
    Author: Michael Paquier
    Reviewed-by: Andrew Dunstan
    Discussion: https://postgr.es/m/20180529211559.GF6632@paquier.xyz


So the immediate fix for building on Windows is you need to manually
define all of those in pg_config.h.  We probably ought to try to make
that less painful, though.  At the very least it'd be good if there
were just one symbol you had to set manually ...

            regards, tom lane

On 10/12/18 8:06 AM, Ádám Maracska wrote:

Please reply to list also.
Ccing list.
> Hi,
> 
> Thanks for the short answer and thanks for reminding me for the most 
> essential details that I missed.:)
> 
>  >What OS and version of same are you compiling in?
> So I would like to compile it for windows 10 x64 and with the postgresql 
> 9.6.5 sources.
> (Later I would like to do this for debian also.)
> 
>  >Do you have the SSL dev libraries installed?
> No, I do not unfortunately.
> 
> Thanks,
> Adam
> 
> Adrian Klaver <adrian.klaver@aklaver.com 
> <mailto:adrian.klaver@aklaver.com>> ezt írta (időpont: 2018. okt. 12., 
> P, 16:51):
> 
>     On 10/12/18 7:15 AM, Ádám Maracska wrote:
>      > Hi,
>      >
>      > Currently I am trying to compile my pqlib with SSL version 1.1.0,
>     but I
>      > get the following errors:
>      > /
>      > /
>      > /fe-secure-openssl.c(1582): error C2037: left of 'ptr' specifies
>      > undefined struct/union 'bio_st'/
>      > /fe-secure-openssl.c(1582): error C2198: 'pqsecure_raw_read': too
>     few
>      > arguments for call/
>      > /fe-secure-openssl.c(1612): error C2037: left of 'ptr' specifies
>      > undefined struct/union 'bio_st'/
>      > /fe-secure-openssl.c(1612): error C2198: 'pqsecure_raw_write':
>     too few
>      > arguments for call/
>      > /fe-secure-openssl.c(1670): error C2027: use of undefined type
>      > 'bio_method_st'/
>      > /fe-secure-openssl.c(1673): error C2027: use of undefined type
>      > 'bio_method_st'/
>      > /fe-secure-openssl.c(1674): error C2037: left of 'bread' specifies
>      > undefined struct/union 'bio_method_st'/
>      > /fe-secure-openssl.c(1675): error C2037: left of 'bwrite' specifies
>      > undefined struct/union 'bio_method_st'/
>      > /fe-secure-openssl.c(1701): error C2037: left of 'ptr' specifies
>      > undefined struct/union 'bio_st'/
>      >
>      > As I was checking the issue on google I found the following two
>     message
>      > thread about it:
>      >
>      > - From 2016: Based on this the ssl version 1.1 is supported, but the
>      > message thread had not so promising end:
>      >
>     https://www.postgresql.org/message-id/flat/E1bkVG3-0000L8-Ln%40gemulon.postgresql.org
>      > - From 2018: It had even more worse end. The questioner went back to
>      > 1.0.2 ssl version, which is not an option for me, because that
>     version
>      > will not be supported shortly by openssl itself.
>      >
>
https://www.postgresql.org/message-id/CAAw-Mseg9JYpp%3DA%3D51HR3rKiQtbvT0MWw%2BaYFwNeJEbdNr%3DCDA%40mail.gmail.com
>      >
>      > Also I checked the source files and I saw these kind of comments:
>      > / They are also not needed with OpenSSL 1.1.0 anymore.
>      > (fe-secure-openssl.c)/
>      > /
>      > /
>      > I suppose it is possible to compile the pqlib with ssl 1.1, but I
>     can
>      > not wrap my head around it. Maybe I should set some ifdefs for the
>      > precompiler beforehand?
>      >
>      > I appreciate any help.
> 
>     What OS and version of same are you compiling in?
> 
>     Do you have the SSL dev libraries installed?
> 
>      >
>      > Best Regards,
>      > Adam
> 
> 
>     -- 
>     Adrian Klaver
>     adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com

On Fri, Oct 12, 2018 at 12:17:40PM -0400, Tom Lane wrote:
> So the immediate fix for building on Windows is you need to manually
> define all of those in pg_config.h.  We probably ought to try to make
> that less painful, though.  At the very least it'd be good if there
> were just one symbol you had to set manually ...

The root of the issue is that we have no way to change dynamically the
set of flags used in pg_config.h.win32 with the MSVC scripts depending
on the version of OpenSSL compiled.  We can tweak the scripts to do that
automatically....  Patches welcome.
--
Michael

Hi,

Thanks for the replies and the help, but unfortunately it did not help my
case to just simply add and enable the missing flags as it is stated here:

> In order to allow the code to compile with
> OpenSSL 1.1.0, all the flags mentioned above need to be enabled in
> pg_config.h.win32  

https://www.postgresql.org/message-id/20180624221521.GA7755@fetter.org

I suppose it is only relevant for version 11, but since I am using 9.6 it
does not help.

Since my code is not out for production yet and AWS is also supporting
Postgres 11 I could switch to version 11, but it is important for me to be
sure that it will solve my problem and I could compile Postgresql 11 for
windows and use it as an On-Premise and also in AWS.


What do you think?
Should I just switch to version 11 and leave the not working 9.6 version
behind and then I could ?

Thanks,
Best,
Adam



--
Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

On 10/16/18 2:27 AM, csusza wrote:
> Hi,
> 
> Thanks for the replies and the help, but unfortunately it did not help my
> case to just simply add and enable the missing flags as it is stated here:

Did you get the same errors?

> 
>> In order to allow the code to compile with
>> OpenSSL 1.1.0, all the flags mentioned above need to be enabled in
>> pg_config.h.win32
> 
> https://www.postgresql.org/message-id/20180624221521.GA7755@fetter.org
> 
> I suppose it is only relevant for version 11, but since I am using 9.6 it
> does not help.
> 
> Since my code is not out for production yet and AWS is also supporting
> Postgres 11 I could switch to version 11, but it is important for me to be
> sure that it will solve my problem and I could compile Postgresql 11 for
> windows and use it as an On-Premise and also in AWS.
> 
> 
> What do you think?
> Should I just switch to version 11 and leave the not working 9.6 version
> behind and then I could ?
> 
> Thanks,
> Best,
> Adam
> 
> 
> 
> --
> Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html
> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com