Re: Meson far from ready on Windows - Mailing list pgsql-hackers

From Dave Page
Subject Re: Meson far from ready on Windows
Date
Msg-id CA+OCxozsg3NOkTfK4KCb0gmxwnaNv4P=yuMZe_CgeXTM=ZGW7g@mail.gmail.com
Whole thread Raw
In response to Re: Meson far from ready on Windows  (Andres Freund <andres@anarazel.de>)
Responses Re: Meson far from ready on Windows
List pgsql-hackers
Hi

On Tue, 18 Jun 2024 at 17:08, Andres Freund <andres@anarazel.de> wrote:
> None of the dependencies include cmake files for distribution on Windows,
> so there are no additional files to tell meson to search for. The same
> applies to pkgconfig files, which is why the EDB team had to manually craft
> them.

Many of them do include at least cmake files on windows if you build them
though?

The only one that does is libxml2 as far as I can see. And that doesn't seem to work even if I use --cmake-prefix-path= as you suggested:

C:\Users\dpage\git\postgresql>meson setup --auto-features=disabled --wipe -Dlibxml=enabled --cmake-prefix-path=C:\build64\lib\cmake\libxml2-2.11.8 build-libxml
The Meson build system
Version: 1.4.0
Source dir: C:\Users\dpage\git\postgresql
Build dir: C:\Users\dpage\git\postgresql\build-libxml
Build type: native build
Project name: postgresql
Project version: 17beta1
C compiler for the host machine: cl (msvc 19.39.33523 "Microsoft (R) C/C++ Optimizing Compiler Version 19.39.33523 for x64")
C linker for the host machine: link link 14.39.33523.0
Host machine cpu family: x86_64
Host machine cpu: x86_64
Run-time dependency threads found: YES
Library ws2_32 found: YES
Library secur32 found: YES
Program perl found: YES (C:\Strawberry\perl\bin\perl.EXE)
Program python found: YES (C:\Python312\python.EXE)
Program win_flex found: YES 2.6.4 2.6.4 (C:\ProgramData\chocolatey\bin\win_flex.EXE)
Program win_bison found: YES 3.7.4 3.7.4 (C:\ProgramData\chocolatey\bin\win_bison.EXE)
Program sed found: YES (C:\ProgramData\chocolatey\bin\sed.EXE)
Program prove found: YES (C:\Strawberry\perl\bin\prove.BAT)
Program tar found: YES (C:\Windows\system32\tar.EXE)
Program gzip found: YES (C:\ProgramData\chocolatey\bin\gzip.EXE)
Program lz4 found: NO
Program openssl found: YES (C:\build64\bin\openssl.EXE)
Program zstd found: NO
Program dtrace skipped: feature dtrace disabled
Program config/missing found: YES (sh C:\Users\dpage\git\postgresql\config/missing)
Program cp found: YES (C:\Program Files (x86)\GnuWin32\bin\cp.EXE)
Program xmllint found: YES (C:\build64\bin\xmllint.EXE)
Program xsltproc found: YES (C:\build64\bin\xsltproc.EXE)
Program wget found: YES (C:\ProgramData\chocolatey\bin\wget.EXE)
Program C:\Python312\Scripts\meson found: YES (C:\Python312\Scripts\meson.exe)
Check usable header "bsd_auth.h" skipped: feature bsd_auth disabled
Check usable header "dns_sd.h" skipped: feature bonjour disabled
Compiler for language cpp skipped: feature llvm disabled
Found pkg-config: YES (C:\ProgramData\chocolatey\bin\pkg-config.EXE) 0.28
Found CMake: C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.EXE (3.28.0)
Run-time dependency libxml-2.0 found: NO (tried pkgconfig and cmake)

meson.build:796:11: ERROR: Dependency "libxml-2.0" not found, tried pkgconfig and cmake

A full log can be found at C:\Users\dpage\git\postgresql\build-libxml\meson-logs\meson-log.txt
 


Btw, I've been working with Bilal to add a few of the dependencies to the CI
images so we can test those automatically. Using vcpkg. We got that nearly
working, but he's on vacation this week...  That does ensure both cmake and
.pc files are generated, fwiw.

Currently builds gettext, icu, libxml2, libxslt, lz4, openssl, pkgconf,
python3, tcl, zlib, zstd. 

That appears to be using Mingw/Msys, which is quite different from a VC++ build, in part because it's a full environment with its own package manager and packages that people have put a lot of effort into making work as they do on unix. 
 
I'm *NOT* sure that vcpkg is the way to go, fwiw. It does seem advantageous to
use one of the toolkits thats commonly built for building dependencies on
windows, which seems to mean vcpkg or conan.

I don't think requiring or expecting vcpkg or conan is reasonable at all, for a number of reasons:

- Neither supports all the dependencies at present.
- There are real supply chain verification concerns for vendors.
- That would be a huge change from what we've required in the last 19 years, with no deprecation notices or warnings for packagers etc.
 
> And that's why we really need to be able to locate headers and libraries
> easily by passing paths to meson, as we can't rely on pkgconfig, cmake, or
> things being in some standard directory on Windows.

Except that that often causes hard to diagnose breakages, because that doesn't
allow including the necessary compiler/linker flags [2].  It's a bad model, we shouldn't
perpetuate it.  If we want to forever make windows a complicated annoying
stepchild, that's the way to go.

That is a good point, though I suspect it wouldn't solve your second example of the Kerberos libraries, as you'll get both 32 and 64 bit libs if you follow their standard process for building on Windows so you still need to have code to pick the right ones.
 

FWIW, at least libzstd, libxml [3], lz4, zlib can generate cmake dependency
files on windows in their upstream code.

In the case of zstd, it does not if you build with VC++, the Makefile, or Meson, at least in my testing. It looks like it would if you built it with cmake, but I couldn't get that to work in 10 minutes or so of messing around. And that's a perfect example of what I'm bleating about - there are often many ways of building things on Windows and there are definitely many ways of getting things on Windows, and they're not all equal. We've either got to be extremely prescriptive in our docs, telling people precisely what they need to download for each dependency, or how to build it themselves in the way that will work with PostgreSQL, or the build system needs to be flexible enough to handle different dependency variations, as the old VC++ build system was.
 

I'm *not* against adding "hardcoded" dependency lookup stuff for libraries
where other approaches aren't feasible, I just don't think it's a good idea to
add fragile stuff that will barely be tested, when not necessary.

Greetings,

Andres Freund


[1] Here's a build of PG with the dependencies installed, builds
    https://cirrus-ci.com/task/4953968097361920

[2] E.g.
    https://github.com/postgres/postgres/blob/REL_16_STABLE/src/tools/msvc/Mkvcbuild.pm#L600
    https://github.com/postgres/postgres/blob/REL_16_STABLE/src/tools/msvc/Solution.pm#L1039

[3] Actually, at least your libxml build actually *did* include both .pc and
    cmake files. So just pointing to the relevant path would do the trick.


--

pgsql-hackers by date:

Previous
From: Bruce Momjian
Date:
Subject: Re: Proposal for Updating CRC32C with AVX-512 Algorithm.
Next
From: Ranier Vilela
Date:
Subject: Re: Add pg_get_acl() function get the ACL for a database object