Thread: zlib detection in Meson on Windows broken?
Hi
I'm working on updating the build of PostgreSQL that pgAdmin uses in its Windows installers to use Meson ready for the v17 release. I'm using Visual Studio 2022, on Windows Server 2022.
I've been unable to persuade Meson to detect zlib, whilst OpenSSL seems to be fine.
The dependencies have been built and installed as follows:
mkdir c:\build64
wget https://zlib.net/zlib-1.3.2.tar.gz
tar -zxvf zlib-1.3.2.tar.gz
cd zlib-1.3.2
cmake -DCMAKE_INSTALL_PREFIX=C:/build64/zlib -G "Visual Studio 17 2022" .
msbuild ALL_BUILD.vcxproj /p:Configuration=Release
msbuild RUN_TESTS.vcxproj /p:Configuration=Release
msbuild INSTALL.vcxproj /p:Configuration=Release
cd ..
wget https://www.openssl.org/source/openssl-3.0.13.tar.gz
tar -zxvf openssl-3.0.13.tar.gz
cd openssl-3.0.013
perl Configure VC-WIN64A no-asm --prefix=C:\build64\openssl no-ssl3 no-comp
nmake
nmake test
nmake install
cd ..
This results in the following headers and libraries being installed for zlib:
C:\Users\dpage\git\postgresql>dir C:\build64\zlib\include
Volume in drive C has no label.
Volume Serial Number is 3AAD-5864
Directory of C:\build64\zlib\include
17/05/2024 15:56 <DIR> .
17/05/2024 15:56 <DIR> ..
17/05/2024 15:54 17,096 zconf.h
22/01/2024 19:32 96,829 zlib.h
2 File(s) 113,925 bytes
2 Dir(s) 98,842,726,400 bytes free
C:\Users\dpage\git\postgresql>dir C:\build64\zlib\lib
Volume in drive C has no label.
Volume Serial Number is 3AAD-5864
Directory of C:\build64\zlib\lib
17/05/2024 17:01 <DIR> .
17/05/2024 15:56 <DIR> ..
17/05/2024 15:55 16,638 zlib.lib
17/05/2024 15:55 184,458 zlibstatic.lib
2 File(s) 201,096 bytes
2 Dir(s) 98,842,726,400 bytes free
I then attempt to build PostgreSQL:
meson setup build -Dextra_include_dirs=C:/build64/openssl/include,C:/build64/zlib/include -Dextra_lib_dirs=C:/build64/openssl/lib,C:/build64/zlib/lib -Dssl=openssl -Dzlib=enabled --prefix=c:/build64/pgsql
Which results in the output in output.txt, indicating that OpenSSL was correctly found, but zlib was not. I've also attached the meson log.
I have very little experience with Meson, and even less interpreting it's logs, but it seems to me that it's not including the extra lib and include directories when it runs the test compile, given the command line it's reporting:
cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
Bug, or am I doing something silly?
--
I'm working on updating the build of PostgreSQL that pgAdmin uses in its Windows installers to use Meson ready for the v17 release. I'm using Visual Studio 2022, on Windows Server 2022.
I've been unable to persuade Meson to detect zlib, whilst OpenSSL seems to be fine.
The dependencies have been built and installed as follows:
mkdir c:\build64
wget https://zlib.net/zlib-1.3.2.tar.gz
tar -zxvf zlib-1.3.2.tar.gz
cd zlib-1.3.2
cmake -DCMAKE_INSTALL_PREFIX=C:/build64/zlib -G "Visual Studio 17 2022" .
msbuild ALL_BUILD.vcxproj /p:Configuration=Release
msbuild RUN_TESTS.vcxproj /p:Configuration=Release
msbuild INSTALL.vcxproj /p:Configuration=Release
cd ..
wget https://www.openssl.org/source/openssl-3.0.13.tar.gz
tar -zxvf openssl-3.0.13.tar.gz
cd openssl-3.0.013
perl Configure VC-WIN64A no-asm --prefix=C:\build64\openssl no-ssl3 no-comp
nmake
nmake test
nmake install
cd ..
This results in the following headers and libraries being installed for zlib:
C:\Users\dpage\git\postgresql>dir C:\build64\zlib\include
Volume in drive C has no label.
Volume Serial Number is 3AAD-5864
Directory of C:\build64\zlib\include
17/05/2024 15:56 <DIR> .
17/05/2024 15:56 <DIR> ..
17/05/2024 15:54 17,096 zconf.h
22/01/2024 19:32 96,829 zlib.h
2 File(s) 113,925 bytes
2 Dir(s) 98,842,726,400 bytes free
C:\Users\dpage\git\postgresql>dir C:\build64\zlib\lib
Volume in drive C has no label.
Volume Serial Number is 3AAD-5864
Directory of C:\build64\zlib\lib
17/05/2024 17:01 <DIR> .
17/05/2024 15:56 <DIR> ..
17/05/2024 15:55 16,638 zlib.lib
17/05/2024 15:55 184,458 zlibstatic.lib
2 File(s) 201,096 bytes
2 Dir(s) 98,842,726,400 bytes free
I then attempt to build PostgreSQL:
meson setup build -Dextra_include_dirs=C:/build64/openssl/include,C:/build64/zlib/include -Dextra_lib_dirs=C:/build64/openssl/lib,C:/build64/zlib/lib -Dssl=openssl -Dzlib=enabled --prefix=c:/build64/pgsql
Which results in the output in output.txt, indicating that OpenSSL was correctly found, but zlib was not. I've also attached the meson log.
I have very little experience with Meson, and even less interpreting it's logs, but it seems to me that it's not including the extra lib and include directories when it runs the test compile, given the command line it's reporting:
cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
Bug, or am I doing something silly?
Dave Page
pgAdmin: https://www.pgadmin.org
Attachment
On 2024-05-20 Mo 06:58, Dave Page wrote:
Hi
I'm working on updating the build of PostgreSQL that pgAdmin uses in its Windows installers to use Meson ready for the v17 release. I'm using Visual Studio 2022, on Windows Server 2022.
I've been unable to persuade Meson to detect zlib, whilst OpenSSL seems to be fine.
The dependencies have been built and installed as follows:
mkdir c:\build64
wget https://zlib.net/zlib-1.3.2.tar.gz
tar -zxvf zlib-1.3.2.tar.gz
cd zlib-1.3.2
cmake -DCMAKE_INSTALL_PREFIX=C:/build64/zlib -G "Visual Studio 17 2022" .
msbuild ALL_BUILD.vcxproj /p:Configuration=Release
msbuild RUN_TESTS.vcxproj /p:Configuration=Release
msbuild INSTALL.vcxproj /p:Configuration=Release
cd ..
wget https://www.openssl.org/source/openssl-3.0.13.tar.gz
tar -zxvf openssl-3.0.13.tar.gz
cd openssl-3.0.013
perl Configure VC-WIN64A no-asm --prefix=C:\build64\openssl no-ssl3 no-comp
nmake
nmake test
nmake install
cd ..
This results in the following headers and libraries being installed for zlib:
C:\Users\dpage\git\postgresql>dir C:\build64\zlib\include
Volume in drive C has no label.
Volume Serial Number is 3AAD-5864
Directory of C:\build64\zlib\include
17/05/2024 15:56 <DIR> .
17/05/2024 15:56 <DIR> ..
17/05/2024 15:54 17,096 zconf.h
22/01/2024 19:32 96,829 zlib.h
2 File(s) 113,925 bytes
2 Dir(s) 98,842,726,400 bytes free
C:\Users\dpage\git\postgresql>dir C:\build64\zlib\lib
Volume in drive C has no label.
Volume Serial Number is 3AAD-5864
Directory of C:\build64\zlib\lib
17/05/2024 17:01 <DIR> .
17/05/2024 15:56 <DIR> ..
17/05/2024 15:55 16,638 zlib.lib
17/05/2024 15:55 184,458 zlibstatic.lib
2 File(s) 201,096 bytes
2 Dir(s) 98,842,726,400 bytes free
I then attempt to build PostgreSQL:
meson setup build -Dextra_include_dirs=C:/build64/openssl/include,C:/build64/zlib/include -Dextra_lib_dirs=C:/build64/openssl/lib,C:/build64/zlib/lib -Dssl=openssl -Dzlib=enabled --prefix=c:/build64/pgsql
Which results in the output in output.txt, indicating that OpenSSL was correctly found, but zlib was not. I've also attached the meson log.
I have very little experience with Meson, and even less interpreting it's logs, but it seems to me that it's not including the extra lib and include directories when it runs the test compile, given the command line it's reporting:
cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
Bug, or am I doing something silly?
Hi Dave!
Not sure ;-) But this works for the buildfarm animal drongo, so we should be able to make it work for you. I'll contact you offlist and see if I can help.
cheers
andrew
-- Andrew Dunstan EDB: https://www.enterprisedb.com
Hi Dave,
Is the .pc file generated after the successful build of zlib? If yes, then meson should be able to detect the installation ideally
Is the .pc file generated after the successful build of zlib? If yes, then meson should be able to detect the installation ideally
On Mon, May 20, 2024 at 4:28 PM Dave Page <dpage@pgadmin.org> wrote:
Hi
I'm working on updating the build of PostgreSQL that pgAdmin uses in its Windows installers to use Meson ready for the v17 release. I'm using Visual Studio 2022, on Windows Server 2022.
I've been unable to persuade Meson to detect zlib, whilst OpenSSL seems to be fine.
The dependencies have been built and installed as follows:
mkdir c:\build64
wget https://zlib.net/zlib-1.3.2.tar.gz
tar -zxvf zlib-1.3.2.tar.gz
cd zlib-1.3.2
cmake -DCMAKE_INSTALL_PREFIX=C:/build64/zlib -G "Visual Studio 17 2022" .
msbuild ALL_BUILD.vcxproj /p:Configuration=Release
msbuild RUN_TESTS.vcxproj /p:Configuration=Release
msbuild INSTALL.vcxproj /p:Configuration=Release
cd ..
wget https://www.openssl.org/source/openssl-3.0.13.tar.gz
tar -zxvf openssl-3.0.13.tar.gz
cd openssl-3.0.013
perl Configure VC-WIN64A no-asm --prefix=C:\build64\openssl no-ssl3 no-comp
nmake
nmake test
nmake install
cd ..
This results in the following headers and libraries being installed for zlib:
C:\Users\dpage\git\postgresql>dir C:\build64\zlib\include
Volume in drive C has no label.
Volume Serial Number is 3AAD-5864
Directory of C:\build64\zlib\include
17/05/2024 15:56 <DIR> .
17/05/2024 15:56 <DIR> ..
17/05/2024 15:54 17,096 zconf.h
22/01/2024 19:32 96,829 zlib.h
2 File(s) 113,925 bytes
2 Dir(s) 98,842,726,400 bytes free
C:\Users\dpage\git\postgresql>dir C:\build64\zlib\lib
Volume in drive C has no label.
Volume Serial Number is 3AAD-5864
Directory of C:\build64\zlib\lib
17/05/2024 17:01 <DIR> .
17/05/2024 15:56 <DIR> ..
17/05/2024 15:55 16,638 zlib.lib
17/05/2024 15:55 184,458 zlibstatic.lib
2 File(s) 201,096 bytes
2 Dir(s) 98,842,726,400 bytes free
I then attempt to build PostgreSQL:
meson setup build -Dextra_include_dirs=C:/build64/openssl/include,C:/build64/zlib/include -Dextra_lib_dirs=C:/build64/openssl/lib,C:/build64/zlib/lib -Dssl=openssl -Dzlib=enabled --prefix=c:/build64/pgsql
Which results in the output in output.txt, indicating that OpenSSL was correctly found, but zlib was not. I've also attached the meson log.
I have very little experience with Meson, and even less interpreting it's logs, but it seems to me that it's not including the extra lib and include directories when it runs the test compile, given the command line it's reporting:
cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
Bug, or am I doing something silly?--Dave PagepgAdmin: https://www.pgadmin.org
Sandeep Thakkar
Hi, On Tue, 21 May 2024 at 10:20, Sandeep Thakkar <sandeep.thakkar@enterprisedb.com> wrote: > > Hi Dave, > > Is the .pc file generated after the successful build of zlib? If yes, then meson should be able to detect the installationideally If meson is not able to find the .pc file automatically, using 'meson setup ... --pkg-config-path $ZLIB_PC_PATH' might help. -- Regards, Nazir Bilal Yavuz Microsoft
Hi Sandeep, Nazir,
On Tue, 21 May 2024 at 10:14, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
Hi,
On Tue, 21 May 2024 at 10:20, Sandeep Thakkar
<sandeep.thakkar@enterprisedb.com> wrote:
>
> Hi Dave,
>
> Is the .pc file generated after the successful build of zlib? If yes, then meson should be able to detect the installation ideally
If meson is not able to find the .pc file automatically, using 'meson
setup ... --pkg-config-path $ZLIB_PC_PATH' might help.
The problem is that on Windows there are no standard locations for a Unix-style development library installation such as this, so the chances are that the .pc file will point to entirely the wrong location.
For example, please see https://github.com/dpage/winpgbuild/actions/runs/9172187335 which is a Github action that builds a completely vanilla zlib using VC++. If you look at the uploaded artefact containing the build output and example the .pc file, you'll see it references /zlib as the location, which is simply where I built it in that action. On a developer's machine that's almost certainly not going to be where it actually ends up. For example, on the pgAdmin build farm, the dependencies all end up in C:\build64\[whatever]. On the similar Github action I'm building for PostgreSQL, that artefact will be unpacked into /build/zlib.
Of course, for my own builds I can easily make everything use consistent directories, however most people who are likely to want to build PostgreSQL may not want to also build all the dependencies themselves as well, as some are a lot more difficult than zlib. So what tends to happen is people find third party builds or upstream official builds.
I would therefore argue that if the .pc file that's found doesn't provide correct paths for us, then Meson should fall back to searching in the paths specified on its command line for the appropriate libraries/headers (which is what it does for OpenSSL for example, as that doesn't include a .pc file). This is also what happens with PG16 and earlier.
One other thing I will note is that PG16 and earlier try to use the wrong filename for the import library. For years, it's been a requirement to do something like this: "copy \zlib\lib\zlib.lib \zlib\lib\zdll.lib" to make a build succeed against a "vanilla" zlib build. I haven't got as far as figuring out if the same is true with Meson yet.
Dave Page
pgAdmin: https://www.pgadmin.org
Hi Dave,
On Tue, May 21, 2024 at 3:12 PM Dave Page <dpage@pgadmin.org> wrote:
Hi Sandeep, Nazir,On Tue, 21 May 2024 at 10:14, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:Hi,
On Tue, 21 May 2024 at 10:20, Sandeep Thakkar
<sandeep.thakkar@enterprisedb.com> wrote:
>
> Hi Dave,
>
> Is the .pc file generated after the successful build of zlib? If yes, then meson should be able to detect the installation ideally
If meson is not able to find the .pc file automatically, using 'meson
setup ... --pkg-config-path $ZLIB_PC_PATH' might help.The problem is that on Windows there are no standard locations for a Unix-style development library installation such as this, so the chances are that the .pc file will point to entirely the wrong location.For example, please see https://github.com/dpage/winpgbuild/actions/runs/9172187335 which is a Github action that builds a completely vanilla zlib using VC++. If you look at the uploaded artefact containing the build output and example the .pc file, you'll see it references /zlib as the location, which is simply where I built it in that action. On a developer's machine that's almost certainly not going to be where it actually ends up. For example, on the pgAdmin build farm, the dependencies all end up in C:\build64\[whatever]. On the similar Github action I'm building for PostgreSQL, that artefact will be unpacked into /build/zlib.
The above link returned 404. But I found a successful build at https://github.com/dpage/winpgbuild/actions/runs/9175426807. I downloaded the artifact but didn't find .pc file as I wanted to look into the content of that file.
I had a word with Murali who mentioned he encountered a similar issue while building PG17 on windows. He worked-around is by using a template .pc file that includes these lines:
--
prefix=${pcfiledir}/../..exec_prefix=${prefix}
libdir=${prefix}/lib
sharedlibdir=${prefix}/lib
includedir=${prefix}/include
--
But in general I agree with you on the issue of Meson's dependency on pkgconfig files to detect the third party libraries.
--Of course, for my own builds I can easily make everything use consistent directories, however most people who are likely to want to build PostgreSQL may not want to also build all the dependencies themselves as well, as some are a lot more difficult than zlib. So what tends to happen is people find third party builds or upstream official builds.I would therefore argue that if the .pc file that's found doesn't provide correct paths for us, then Meson should fall back to searching in the paths specified on its command line for the appropriate libraries/headers (which is what it does for OpenSSL for example, as that doesn't include a .pc file). This is also what happens with PG16 and earlier.One other thing I will note is that PG16 and earlier try to use the wrong filename for the import library. For years, it's been a requirement to do something like this: "copy \zlib\lib\zlib.lib \zlib\lib\zdll.lib" to make a build succeed against a "vanilla" zlib build. I haven't got as far as figuring out if the same is true with Meson yet.Dave PagepgAdmin: https://www.pgadmin.org
Sandeep Thakkar
Hi
On Tue, 21 May 2024 at 15:12, Sandeep Thakkar <sandeep.thakkar@enterprisedb.com> wrote:
Hi Dave,On Tue, May 21, 2024 at 3:12 PM Dave Page <dpage@pgadmin.org> wrote:Hi Sandeep, Nazir,On Tue, 21 May 2024 at 10:14, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:Hi,
On Tue, 21 May 2024 at 10:20, Sandeep Thakkar
<sandeep.thakkar@enterprisedb.com> wrote:
>
> Hi Dave,
>
> Is the .pc file generated after the successful build of zlib? If yes, then meson should be able to detect the installation ideally
If meson is not able to find the .pc file automatically, using 'meson
setup ... --pkg-config-path $ZLIB_PC_PATH' might help.The problem is that on Windows there are no standard locations for a Unix-style development library installation such as this, so the chances are that the .pc file will point to entirely the wrong location.For example, please see https://github.com/dpage/winpgbuild/actions/runs/9172187335 which is a Github action that builds a completely vanilla zlib using VC++. If you look at the uploaded artefact containing the build output and example the .pc file, you'll see it references /zlib as the location, which is simply where I built it in that action. On a developer's machine that's almost certainly not going to be where it actually ends up. For example, on the pgAdmin build farm, the dependencies all end up in C:\build64\[whatever]. On the similar Github action I'm building for PostgreSQL, that artefact will be unpacked into /build/zlib.
The above link returned 404. But I found a successful build at https://github.com/dpage/winpgbuild/actions/runs/9175426807. I downloaded the artifact but didn't find .pc file as I wanted to look into the content of that file.
Yeah, sorry - that was an old one.
Following some offline discussion with Andrew I realised I was building zlib incorrectly - using cmake/msbuild instead of nmake and some manual copying. Whilst the former does create a working library and a sane looking installation, it's not the recommended method, which is documented in a very non-obvious way - far less obvious than "oh look, there's a cmake config - let's use that".
The new build is done using the recommended method, which works with PG16 and below out of the box with no need to rename any files.
I had a word with Murali who mentioned he encountered a similar issue while building PG17 on windows. He worked-around is by using a template .pc file that includes these lines:--prefix=${pcfiledir}/../..
exec_prefix=${prefix}
libdir=${prefix}/lib
sharedlibdir=${prefix}/libincludedir=${prefix}/include--
The issue here is that there is no .pc file created with the correct way of building zlib, and even if there were (or I created a dummy one), pkg-config isn't really a thing on Windows.
I'd also note that from what Andrew has shown me of the zlib installation on the buildfarm member drongo, there is no .pc file there either, and yet seems to work fine (and my zlib installation now has the exact same set of files as his does).
But in general I agree with you on the issue of Meson's dependency on pkgconfig files to detect the third party libraries.--Of course, for my own builds I can easily make everything use consistent directories, however most people who are likely to want to build PostgreSQL may not want to also build all the dependencies themselves as well, as some are a lot more difficult than zlib. So what tends to happen is people find third party builds or upstream official builds.I would therefore argue that if the .pc file that's found doesn't provide correct paths for us, then Meson should fall back to searching in the paths specified on its command line for the appropriate libraries/headers (which is what it does for OpenSSL for example, as that doesn't include a .pc file). This is also what happens with PG16 and earlier.One other thing I will note is that PG16 and earlier try to use the wrong filename for the import library. For years, it's been a requirement to do something like this: "copy \zlib\lib\zlib.lib \zlib\lib\zdll.lib" to make a build succeed against a "vanilla" zlib build. I haven't got as far as figuring out if the same is true with Meson yet.Dave PagepgAdmin: https://www.pgadmin.org--Sandeep Thakkar
Dave Page
pgAdmin: https://www.pgadmin.org
Hi, On 2024-05-20 11:58:05 +0100, Dave Page wrote: > I have very little experience with Meson, and even less interpreting it's > logs, but it seems to me that it's not including the extra lib and include > directories when it runs the test compile, given the command line it's > reporting: > > cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi- > > Bug, or am I doing something silly? It's a buglet. We rely on meson's internal fallback detection of zlib, if it's not provided via pkg-config or cmake. But it doesn't know about our extra_include_dirs parameter. We should probably fix that... Greetings, Andres Freund
On Tue, 21 May 2024 at 16:04, Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2024-05-20 11:58:05 +0100, Dave Page wrote:
> I have very little experience with Meson, and even less interpreting it's
> logs, but it seems to me that it's not including the extra lib and include
> directories when it runs the test compile, given the command line it's
> reporting:
>
> cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c
> /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
>
> Bug, or am I doing something silly?
It's a buglet. We rely on meson's internal fallback detection of zlib, if it's
not provided via pkg-config or cmake. But it doesn't know about our
extra_include_dirs parameter. We should probably fix that...
Oh good, then I'm not going bonkers. I'm still curious about how it works for Andrew but not me, however fixing that buglet should solve my issue, and would be sensible behaviour.
Thanks!
Dave Page
pgAdmin: https://www.pgadmin.org
Hi, On 2024-05-20 11:58:05 +0100, Dave Page wrote: > I then attempt to build PostgreSQL: > > meson setup build > -Dextra_include_dirs=C:/build64/openssl/include,C:/build64/zlib/include > -Dextra_lib_dirs=C:/build64/openssl/lib,C:/build64/zlib/lib -Dssl=openssl > -Dzlib=enabled --prefix=c:/build64/pgsql > > Which results in the output in output.txt, indicating that OpenSSL was > correctly found, but zlib was not. I've also attached the meson log. I forgot to mention that earlier: Assuming you're building something to be distributed, I'd recommend --auto-features=enabled/disabled and specifying specifically which dependencies you want to be used. Greetings, Andres Freund
On 2024-05-21 Tu 11:04, Andres Freund wrote: > Hi, > > On 2024-05-20 11:58:05 +0100, Dave Page wrote: >> I have very little experience with Meson, and even less interpreting it's >> logs, but it seems to me that it's not including the extra lib and include >> directories when it runs the test compile, given the command line it's >> reporting: >> >> cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c >> /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi- >> >> Bug, or am I doing something silly? > It's a buglet. We rely on meson's internal fallback detection of zlib, if it's > not provided via pkg-config or cmake. But it doesn't know about our > extra_include_dirs parameter. We should probably fix that... > Yeah. Meanwhile, what I got working on a totally fresh Windows + VS install was instead of using extra_include_dirs etc to add the relevant directories to the environment LIB and INCLUDE settings before calling `"meson setup". cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
On Tue May 21, 2024 at 10:04 AM CDT, Andres Freund wrote:
> Hi,
>
> On 2024-05-20 11:58:05 +0100, Dave Page wrote:
> > I have very little experience with Meson, and even less interpreting it's
> > logs, but it seems to me that it's not including the extra lib and include
> > directories when it runs the test compile, given the command line it's
> > reporting:
> >
> > cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c
> > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
> >
> > Bug, or am I doing something silly?
>
> It's a buglet. We rely on meson's internal fallback detection of zlib, if it's
> not provided via pkg-config or cmake. But it doesn't know about our
> extra_include_dirs parameter. We should probably fix that...
Here is the relevant Meson code for finding zlib in the Postgres tree:
postgres_inc_d = ['src/include']
postgres_inc_d += get_option('extra_include_dirs')
...
postgres_inc = [include_directories(postgres_inc_d)]
...
zlibopt = get_option('zlib')
zlib = not_found_dep
if not zlibopt.disabled()
zlib_t = dependency('zlib', required: zlibopt)
if zlib_t.type_name() == 'internal'
# if fallback was used, we don't need to test if headers are present (they
# aren't built yet, so we can't test)
zlib = zlib_t
elif not zlib_t.found()
warning('did not find zlib')
elif not cc.has_header('zlib.h',
args: test_c_args, include_directories: postgres_inc,
dependencies: [zlib_t], required: zlibopt)
warning('zlib header not found')
elif not cc.has_type('z_streamp',
dependencies: [zlib_t], prefix: '#include <zlib.h>',
args: test_c_args, include_directories: postgres_inc)
if zlibopt.enabled()
error('zlib version is too old')
else
warning('zlib version is too old')
endif
else
zlib = zlib_t
endif
if zlib.found()
cdata.set('HAVE_LIBZ', 1)
endif
endif
You can see that we do pass the include dirs to the has_header check.
Something seems to be going wrong here since your extra_include_dirs
isn't being properly translated to include arguments.
--
Tristan Partin
https://tristan.partin.io
On Tue, 21 May 2024 at 18:00, Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2024-05-20 11:58:05 +0100, Dave Page wrote:
> I then attempt to build PostgreSQL:
>
> meson setup build
> -Dextra_include_dirs=C:/build64/openssl/include,C:/build64/zlib/include
> -Dextra_lib_dirs=C:/build64/openssl/lib,C:/build64/zlib/lib -Dssl=openssl
> -Dzlib=enabled --prefix=c:/build64/pgsql
>
> Which results in the output in output.txt, indicating that OpenSSL was
> correctly found, but zlib was not. I've also attached the meson log.
I forgot to mention that earlier: Assuming you're building something to be
distributed, I'd recommend --auto-features=enabled/disabled and specifying
specifically which dependencies you want to be used.
Good idea - thanks.
Dave Page
pgAdmin: https://www.pgadmin.org
On Tue, 21 May 2024 at 20:54, Andrew Dunstan <andrew@dunslane.net> wrote:
On 2024-05-21 Tu 11:04, Andres Freund wrote:
> Hi,
>
> On 2024-05-20 11:58:05 +0100, Dave Page wrote:
>> I have very little experience with Meson, and even less interpreting it's
>> logs, but it seems to me that it's not including the extra lib and include
>> directories when it runs the test compile, given the command line it's
>> reporting:
>>
>> cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c
>> /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
>>
>> Bug, or am I doing something silly?
> It's a buglet. We rely on meson's internal fallback detection of zlib, if it's
> not provided via pkg-config or cmake. But it doesn't know about our
> extra_include_dirs parameter. We should probably fix that...
>
Yeah. Meanwhile, what I got working on a totally fresh Windows + VS
install was instead of using extra_include_dirs etc to add the relevant
directories to the environment LIB and INCLUDE settings before calling
`"meson setup".
Yes, that works for me too.
Dave Page
pgAdmin: https://www.pgadmin.org
Hi, On Tue, 21 May 2024 at 18:24, Dave Page <dpage@pgadmin.org> wrote: > > > > On Tue, 21 May 2024 at 16:04, Andres Freund <andres@anarazel.de> wrote: >> >> Hi, >> >> On 2024-05-20 11:58:05 +0100, Dave Page wrote: >> > I have very little experience with Meson, and even less interpreting it's >> > logs, but it seems to me that it's not including the extra lib and include >> > directories when it runs the test compile, given the command line it's >> > reporting: >> > >> > cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c >> > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi- >> > >> > Bug, or am I doing something silly? >> >> It's a buglet. We rely on meson's internal fallback detection of zlib, if it's >> not provided via pkg-config or cmake. But it doesn't know about our >> extra_include_dirs parameter. We should probably fix that... > > > Oh good, then I'm not going bonkers. I'm still curious about how it works for Andrew but not me, however fixing that bugletshould solve my issue, and would be sensible behaviour. > > Thanks! I tried to install your latest zlib artifact (nmake one) to the Windows CI images (not the official ones) [1]. Then, I used the default meson.build file to build but meson could not find the zlib. After that, I modified it like you suggested before; I used a 'cc.find_library()' to find zlib as a fallback method and it seems it worked [2]. Please see meson setup logs below [3], does something similar to the attached solve your problem? The interesting thing is, I also tried this 'cc.find_library' method with your old artifact (cmake one). It was able to find zlib but all tests failed [4]. Experimental zlib meson.build diff is attached. [1] https://cirrus-ci.com/task/6736867247259648 [2] https://cirrus-ci.com/build/5286228755480576 [3] Run-time dependency zlib found: NO (tried pkgconfig, cmake and system) Has header "zlib.h" : YES Library zlib found: YES ... External libraries ... zlib : YES ... [4] https://cirrus-ci.com/task/5208433811521536 -- Regards, Nazir Bilal Yavuz Microsoft
Attachment
Hi
On Wed, 22 May 2024 at 14:11, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
Hi,
On Tue, 21 May 2024 at 18:24, Dave Page <dpage@pgadmin.org> wrote:
>
>
>
> On Tue, 21 May 2024 at 16:04, Andres Freund <andres@anarazel.de> wrote:
>>
>> Hi,
>>
>> On 2024-05-20 11:58:05 +0100, Dave Page wrote:
>> > I have very little experience with Meson, and even less interpreting it's
>> > logs, but it seems to me that it's not including the extra lib and include
>> > directories when it runs the test compile, given the command line it's
>> > reporting:
>> >
>> > cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c
>> > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
>> >
>> > Bug, or am I doing something silly?
>>
>> It's a buglet. We rely on meson's internal fallback detection of zlib, if it's
>> not provided via pkg-config or cmake. But it doesn't know about our
>> extra_include_dirs parameter. We should probably fix that...
>
>
> Oh good, then I'm not going bonkers. I'm still curious about how it works for Andrew but not me, however fixing that buglet should solve my issue, and would be sensible behaviour.
>
> Thanks!
I tried to install your latest zlib artifact (nmake one) to the
Windows CI images (not the official ones) [1]. Then, I used the
default meson.build file to build but meson could not find the zlib.
After that, I modified it like you suggested before; I used a
'cc.find_library()' to find zlib as a fallback method and it seems it
worked [2]. Please see meson setup logs below [3], does something
similar to the attached solve your problem?
That patch does solve my problem - thank you!
The interesting thing is, I also tried this 'cc.find_library' method
with your old artifact (cmake one). It was able to find zlib but all
tests failed [4].
Very odd. Whilst I haven't used that particular build elsewhere, we've been building PostgreSQL and shipping client utilities with pgAdmin using cmake-built zlib for years.
Experimental zlib meson.build diff is attached.
[1] https://cirrus-ci.com/task/6736867247259648
[2] https://cirrus-ci.com/build/5286228755480576
[3]
Run-time dependency zlib found: NO (tried pkgconfig, cmake and system)
Has header "zlib.h" : YES
Library zlib found: YES
...
External libraries
...
zlib : YES
...
[4] https://cirrus-ci.com/task/5208433811521536
--
Regards,
Nazir Bilal Yavuz
Microsoft
Dave Page
pgAdmin: https://www.pgadmin.org
Hi, On Wed, 22 May 2024 at 17:21, Dave Page <dpage@pgadmin.org> wrote: > > Hi > > On Wed, 22 May 2024 at 14:11, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote: >> >> >> I tried to install your latest zlib artifact (nmake one) to the >> Windows CI images (not the official ones) [1]. Then, I used the >> default meson.build file to build but meson could not find the zlib. >> After that, I modified it like you suggested before; I used a >> 'cc.find_library()' to find zlib as a fallback method and it seems it >> worked [2]. Please see meson setup logs below [3], does something >> similar to the attached solve your problem? > > > That patch does solve my problem - thank you! I am glad that it worked! Do you think that we need to have this patch in the upstream Postgres? I am not sure because: - There is a case that meson is able to find zlib but tests fail. - This might be a band-aid fix rather than a permanent fix. -- Regards, Nazir Bilal Yavuz Microsoft
On Wed, 22 May 2024 at 17:50, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
Hi,
On Wed, 22 May 2024 at 17:21, Dave Page <dpage@pgadmin.org> wrote:
>
> Hi
>
> On Wed, 22 May 2024 at 14:11, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
>>
>>
>> I tried to install your latest zlib artifact (nmake one) to the
>> Windows CI images (not the official ones) [1]. Then, I used the
>> default meson.build file to build but meson could not find the zlib.
>> After that, I modified it like you suggested before; I used a
>> 'cc.find_library()' to find zlib as a fallback method and it seems it
>> worked [2]. Please see meson setup logs below [3], does something
>> similar to the attached solve your problem?
>
>
> That patch does solve my problem - thank you!
I am glad that it worked!
Do you think that we need to have this patch in the upstream Postgres?
I am not sure because:
- There is a case that meson is able to find zlib but tests fail.
- This might be a band-aid fix rather than a permanent fix.
Yes I do:
- This is the documented way to build/install zlib on Windows.
- The behaviour with the patch matches <= PG16
- The behaviour with the patch is consistent with OpenSSL detection, and (from a quick, unrelated test), libxml2 detection.
Thanks!