Thread: Issues cross-compiling libpq 14.x to MacOS armv8

Issues cross-compiling libpq 14.x to MacOS armv8

From
Vincas Dargis
Date:
Hi list,

I wanted to update [0] Conan package for building libpq 14.1. Usually it's enough to add new tarball and declare it's 
hash, but it seems that since 14.0 cross-compiling to armv8 MacOS now fails, and I *guess* it's due to removed 
`--disable-strong-random` option.

Here's some snippets from build log: [1]

```
...
Cross-build from 'Macos:x86_64' to 'Macos:armv8'
...
checking which random number source to use... /dev/urandom
checking for /dev/urandom... libpq/14.1:
libpq/14.1: WARN: Build folder is dirty, removing it: 
/Users/jenkins/w/BuildSingleReference@2/.conan/data/libpq/14.1/_/_/build/30acef53c04f36d5f9412c84a1b3a7434a1f10fb
configure: WARNING: unrecognized options: --disable-strong-random
configure: WARNING: using cross tools not prefixed with host triplet
configure: error: cannot check for file existence when cross compiling
libpq/14.1: ERROR: Package '30acef53c04f36d5f9412c84a1b3a7434a1f10fb' build failed
```

Could this mean that building on armv8 Macos cannot work with "strong random", or at least in the way PostgreSQL 
configure script expect that to be detected to work?

Thanks!

P.S. there was earlier attempt by another contributor to update Conan package to 14.0, which also failed in the same 
manner [2].

[0] https://github.com/conan-io/conan-center-index/pull/8109
[1] 

https://c3i.jfrog.io/c3i/misc/logs/pr/8109/2-configs/macos-m1-clang/libpq/14.1//30acef53c04f36d5f9412c84a1b3a7434a1f10fb-build.txt
[2] https://github.com/conan-io/conan-center-index/pull/7676



Re: Issues cross-compiling libpq 14.x to MacOS armv8

From
Tom Lane
Date:
Vincas Dargis <vindrg@gmail.com> writes:
> checking which random number source to use... /dev/urandom
> checking for /dev/urandom... 
> configure: error: cannot check for file existence when cross compiling

Hmm ... this evidently stems from 16f96c74d.

AFAICS this is the only test in our configure script that is a hard
fail when cross-compiling, and I don't see a reason for it to be that.
We could just assume that /dev/urandom will be available --- that's no
worse than a lot of the other optimistic assumptions that configure
makes in that mode.

            regards, tom lane



Re: Issues cross-compiling libpq 14.x to MacOS armv8

From
Daniel Gustafsson
Date:
> On 30 Nov 2021, at 20:59, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Vincas Dargis <vindrg@gmail.com> writes:
>> checking which random number source to use... /dev/urandom
>> checking for /dev/urandom...
>> configure: error: cannot check for file existence when cross compiling
>
> Hmm ... this evidently stems from 16f96c74d.
>
> AFAICS this is the only test in our configure script that is a hard
> fail when cross-compiling, and I don't see a reason for it to be that.
> We could just assume that /dev/urandom will be available --- that's no
> worse than a lot of the other optimistic assumptions that configure
> makes in that mode.

Agreed, I don't see a problem with that.  I'm not terribly familiar with
supporting cross compilation in autoconf, would a reasonable approach be to
just remove the check altogether like the below sketch?

diff --git a/configure.ac b/configure.ac
index a5c10b8d56..80fe0de38d 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2289,13 +2289,6 @@ elif test x"$PORTNAME" = x"win32" ; then
   AC_MSG_RESULT([Windows native])
 else
   AC_MSG_RESULT([/dev/urandom])
-  AC_CHECK_FILE([/dev/urandom], [], [])
-
-  if test x"$ac_cv_file__dev_urandom" = x"no" ; then
-    AC_MSG_ERROR([
-no source of strong random numbers was found
-PostgreSQL can use OpenSSL, native Windows API or /dev/urandom as a source of random numbers.])
-  fi
 fi

--
Daniel Gustafsson        https://vmware.com/




Re: Issues cross-compiling libpq 14.x to MacOS armv8

From
Tom Lane
Date:
Daniel Gustafsson <daniel@yesql.se> writes:
>> On 30 Nov 2021, at 20:59, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> AFAICS this is the only test in our configure script that is a hard
>> fail when cross-compiling, and I don't see a reason for it to be that.
>> We could just assume that /dev/urandom will be available --- that's no
>> worse than a lot of the other optimistic assumptions that configure
>> makes in that mode.

> Agreed, I don't see a problem with that.  I'm not terribly familiar with
> supporting cross compilation in autoconf, would a reasonable approach be to
> just remove the check altogether like the below sketch?

It seems like a useful test when *not* cross compiling, which is most
of the time.  I'd just wrap that bit in

    if test "$cross_compiling" = no; then
    ...
    fi

(I'm a bit surprised that the AC_CHECK_FILE macro doesn't provide
an action-if-cross-compiling option, but it apparently doesn't.)

            regards, tom lane



Re: Issues cross-compiling libpq 14.x to MacOS armv8

From
Tom Lane
Date:
I wrote:
> It seems like a useful test when *not* cross compiling, which is most
> of the time.  I'd just wrap that bit in
>     if test "$cross_compiling" = no; then

Or actually, since we should print something, it looks like this will do:

diff --git a/configure.ac b/configure.ac
index a5c10b8d56..7257afda20 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2287,6 +2287,8 @@ if test x"$with_ssl" = x"openssl" ; then
   AC_MSG_RESULT([OpenSSL])
 elif test x"$PORTNAME" = x"win32" ; then
   AC_MSG_RESULT([Windows native])
+elif test x"$cross_compiling" = x"yes"; then
+  AC_MSG_RESULT([assuming /dev/urandom])
 else
   AC_MSG_RESULT([/dev/urandom])
   AC_CHECK_FILE([/dev/urandom], [], [])

Off to see if I can verify that before pushing.

            regards, tom lane



Re: Issues cross-compiling libpq 14.x to MacOS armv8

From
Daniel Gustafsson
Date:
> On 30 Nov 2021, at 22:33, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> I wrote:
>> It seems like a useful test when *not* cross compiling, which is most
>> of the time.  I'd just wrap that bit in
>>    if test "$cross_compiling" = no; then
> 
> Or actually, since we should print something, it looks like this will do:

+1, looks reasonable.

> +elif test x"$cross_compiling" = x"yes"; then

I noticed that we test without the x"foo" = x"yes" construction for zic (line
1135), should we change that while at it and be consistent for all
$cross_compiling uses?

--
Daniel Gustafsson        https://vmware.com/




Re: Issues cross-compiling libpq 14.x to MacOS armv8

From
Tom Lane
Date:
Daniel Gustafsson <daniel@yesql.se> writes:
> I noticed that we test without the x"foo" = x"yes" construction for zic (line
> 1135), should we change that while at it and be consistent for all
> $cross_compiling uses?

Probably.  $cross_compiling should theoretically always be set, but
there's no harm in being bulletproof.

I had thought I could crank up a cross-compile test painlessly, but
at least on my available systems it doesn't seem to be painless :-(.
So I'm just going to push the change without having really tested
that end of it.  I wonder if it is at all feasible to maintain a
cross-compiling buildfarm member ...

            regards, tom lane



Re: Issues cross-compiling libpq 14.x to MacOS armv8

From
Peter Eisentraut
Date:
On 30.11.21 22:04, Tom Lane wrote:
> Daniel Gustafsson <daniel@yesql.se> writes:
>>> On 30 Nov 2021, at 20:59, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> AFAICS this is the only test in our configure script that is a hard
>>> fail when cross-compiling, and I don't see a reason for it to be that.
>>> We could just assume that /dev/urandom will be available --- that's no
>>> worse than a lot of the other optimistic assumptions that configure
>>> makes in that mode.
> 
>> Agreed, I don't see a problem with that.  I'm not terribly familiar with
>> supporting cross compilation in autoconf, would a reasonable approach be to
>> just remove the check altogether like the below sketch?
> 
> It seems like a useful test when *not* cross compiling, which is most
> of the time.

You still don't know whether the file will exist on the system you are 
running this on.

> (I'm a bit surprised that the AC_CHECK_FILE macro doesn't provide
> an action-if-cross-compiling option, but it apparently doesn't.)

Because you are only supposed to look for files that you need during the 
build.



Re: Issues cross-compiling libpq 14.x to MacOS armv8

From
Daniel Gustafsson
Date:
> On 1 Dec 2021, at 07:11, Peter Eisentraut <peter.eisentraut@enterprisedb.com> wrote:
> On 30.11.21 22:04, Tom Lane wrote:

>> (I'm a bit surprised that the AC_CHECK_FILE macro doesn't provide
>> an action-if-cross-compiling option, but it apparently doesn't.)
>
> Because you are only supposed to look for files that you need during the build.

So by that logic, do you think the AC_CHECK_FILE call should be removed?

--
Daniel Gustafsson        https://vmware.com/




Re: Issues cross-compiling libpq 14.x to MacOS armv8

From
Tom Lane
Date:
Daniel Gustafsson <daniel@yesql.se> writes:
>> On 1 Dec 2021, at 07:11, Peter Eisentraut <peter.eisentraut@enterprisedb.com> wrote:
>> Because you are only supposed to look for files that you need during the build.

> So by that logic, do you think the AC_CHECK_FILE call should be removed?

I don't buy that.  The test is useful on net, and I don't particularly
believe that /dev/urandom will be there on one instance of a platform
and not another.

            regards, tom lane



Re: Issues cross-compiling libpq 14.x to MacOS armv8

From
Vincas Dargis
Date:
Thanks Tom!

Should we expect this fix in the next 14 patch release, or only in 15.x?

If latter, I would add this patch into Conan package itself, to make it work earlier.

On 2021-11-30 23:33, Tom Lane wrote:
> I wrote:
>> It seems like a useful test when *not* cross compiling, which is most
>> of the time.  I'd just wrap that bit in
>>      if test "$cross_compiling" = no; then
> 
> Or actually, since we should print something, it looks like this will do:
> 
> diff --git a/configure.ac b/configure.ac
> index a5c10b8d56..7257afda20 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -2287,6 +2287,8 @@ if test x"$with_ssl" = x"openssl" ; then
>     AC_MSG_RESULT([OpenSSL])
>   elif test x"$PORTNAME" = x"win32" ; then
>     AC_MSG_RESULT([Windows native])
> +elif test x"$cross_compiling" = x"yes"; then
> +  AC_MSG_RESULT([assuming /dev/urandom])
>   else
>     AC_MSG_RESULT([/dev/urandom])
>     AC_CHECK_FILE([/dev/urandom], [], [])
> 
> Off to see if I can verify that before pushing.
> 
>             regards, tom lane
> 



Re: Issues cross-compiling libpq 14.x to MacOS armv8

From
Tom Lane
Date:
Vincas Dargis <vindrg@gmail.com> writes:
> Should we expect this fix in the next 14 patch release, or only in 15.x?

I did push it into v14:

https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=175edafd1f30a78643359b56c5545b5e7aabfb50

            regards, tom lane