meson vs mingw, plpython, readline and other fun - Mailing list pgsql-hackers

Hi,

Melih mentioned on IM that while he could build postgres with meson on windows
w/ mingw, the tests didn't run.

Issues:

- The bit computing PATH to the temporary installation for running tests only
  dealt with backward slashes in paths on windows, because that's what
  python.org python uses by default. But a msys ucrt python returns forward
  slashes. Trivial fix.

  I didn't encounter this because I'd used a meson from git, which thus didn't
  have msys's patch to return a different prefix.

  This made pg_regress/isolationtester tests other than the main regression
  tests pass.


- I'd only passed in a fake HOST_TUPLE when building pg_regress, oops.

  I don't think it makes sense to come up with a config.guess compatible name
  - they're quite random. And we can't rely on shell to work when targetting
  msvc. The attached patch does:

# Need make up something roughly like x86_64-pc-mingw64. resultmap matches on
# patterns like ".*-.*-mingw.*". We probably can do better, but for now just
# replace 'gcc' with 'mingw' on windows.
host_tuple_cc = cc.get_id()
if host_system == 'windows' and host_tuple_cc == 'gcc'
  host_tuple_cc = 'mingw'
endif
host_tuple = '@0@-@1@-@2@'.format(host_cpu, host_system, host_tuple_cc)

  which I don't perfectly like (e.g. clang also kind of works on windows), but
  it seems ok enough for now. I suspect we'd need a bunch of other changes to
  make clang on windows work.

  This made the main pg_regress tests pass.


- I had not added the logic to not use existing getopt on mingw, causing tap
  tests to fail. Fixing that didn't immediately work because of duplicate
  symbols - because I hadn't copied over -Wl,--allow-multiple-definition.

  "Contrary" to the comment in src/template/win32 it doesn't appear to be
  needed for libpq and pgport - likely because for the meson build an export
  file is generated (needed for msvc anyway, I didn't think to disable it for
  mingw).  But since we need to be able to override getopt(), we obviously
  need --allow-multiple-definition anyway.


- This lead me to try to also add -Wl,--disable-auto-import. However, that
  caused two problems.

  1) plpython tests started to fail, due to not finding Pg_magic_func in
     plpython3.dll. This confounded me for quite a while. It worked for every
     other extension .dll? A lot of looking around lead me to define
#define PGDLLEXPORT __declspec (dllexport)
     for mingw as well. For mingw we otherwise end up with
#define PGDLLEXPORT __attribute__((visibility("default")))
     which works.

     As far as I can tell __attribute__((visibility("default"))) works as long as
     as there's no declspec(dllexport) symbol in the same dll. If there is, it
     stops working. Ugh.

     I don't see a reason not to define PGDLLEXPORT as __declspec(dllexport)
     for mingw as well?

     I suspect this is an issue for autoconf mingw build as well, but that
     fails to even configure - there's no coverage on the BF for it I think.

     This made plpython's test pass (again).


  2) psql failed to build due to readline. I hadn't implemented disabling it
     automatically. Somewhat luckily - turns out it actually works (as long as
     --disable-auto-import isn't used), including autocomplete!

     The issue afaict is that while readline has an import library, functions
     aren't "annotated" with __declspec(dllimport), thus without
     --enable-auto-import the references are assumed to be local, and thus
     linking fails.

     It's possible we could "fix" this by declaring the relevant symbols
     ourselves or such.  But for now just adding --enable-auto-import to the
     flags used to link to readline seems saner?

     I think it'd be very cool to finally have a working readline on windows.

     Unfortunately IO::Pty isn't installable on windows, it'd have been
     interesting to see how well that readline works.


- Before I updated mingw, interactive psql didn't show a prompt, making me
  think something was broken. That turned out to be the same in an autoconf
  build. When inside the msys terminal (mintty) isatty returned 0, because of
  some detail of how it emulates ttys. After updating mingw that problem is
  gone.

  I included this partially so I have something to find in email next time I
  search for mintty and isatty :)


With these things fixed, postgres built and ran tests successfully! With
nearly all "relevant" dependencies:
icu, libxml, libslt, lz4, nls, plperl, plpython, pltcl, readline, ssl, zlib,
zstd

Missing are gss and uuid. Both apparently work on windows, but they're not in
the msys repository, and I don't feel like compiling them myself.

Only 5 tests skipped:
- recovery/017_shm - probably not applicable
- recovery/022_crash_temp_files - I don't think it's ok that this test skips,
  but that's for another thread
- psql/010_tab_completion - IO::Pty can't be installed
- psql/010_cancel  - IO::Pty can't be installed
- ldap/001_auth - doesn't know how to find slapd on windows


Stared too long at the screen to figure all of this out. Food next. I'll clean
the patches up later tonight or tomorrow morning.


Greetings,

Andres Freund