Thread: JIT on Windows with Postgres 12.1

JIT on Windows with Postgres 12.1

From
Catch All
Date:
I am using the EDB Windows PG 12.1 installer and it seems not to have JIT enabled:
C:\Program Files\PostgreSQL\12\bin>pg_config --configure
--enable-thread-safety --enable-nls --with-ldap --with-openssl --with-ossp-uuid
--with-libxml --with-libxslt --with-icu --with-tcl --with-perl --with-python

Is this expected?  Is there a Windows installer available for PG 12 with JIT enabled?

Re: JIT on Windows with Postgres 12.1

From
Josef Šimánek
Date:
Hello.

If I understand it well, JIT is enabled only for builds with "--with-llvm" flag. Seems EDB Windows PG 12.1 installer is not build with llvm (and jit) support.

I'm not aware of any PG 12 Windows installer with JIT/LLVM enabled.

út 11. 2. 2020 v 22:41 odesílatel Catch All <catchall@lightn.org> napsal:
I am using the EDB Windows PG 12.1 installer and it seems not to have JIT enabled:
C:\Program Files\PostgreSQL\12\bin>pg_config --configure
--enable-thread-safety --enable-nls --with-ldap --with-openssl --with-ossp-uuid
--with-libxml --with-libxslt --with-icu --with-tcl --with-perl --with-python

Is this expected?  Is there a Windows installer available for PG 12 with JIT enabled?

Re: JIT on Windows with Postgres 12.1

From
Andres Freund
Date:
Hi,

On 2020-02-12 09:39:19 +0100, Josef Šimánek wrote:
> I'm not aware of any PG 12 Windows installer with JIT/LLVM enabled.

It's not currently supported by the windows buildsystem. I don't
personally have the expertise to change that (I've not worked on windows
after ~2003). So somebody would have to step in to make that work,
first.

Greetings,

Andres Freund



Re: JIT on Windows with Postgres 12.1

From
Tom Lane
Date:
Andres Freund <andres@anarazel.de> writes:
> On 2020-02-12 09:39:19 +0100, Josef Šimánek wrote:
>> I'm not aware of any PG 12 Windows installer with JIT/LLVM enabled.

> It's not currently supported by the windows buildsystem. I don't
> personally have the expertise to change that (I've not worked on windows
> after ~2003). So somebody would have to step in to make that work,
> first.

First you would need either a Windows-native LLVM, or a JIT provider
based on Microsoft's compiler.  I don't know if the former exists
(it would not be our job to create it).  I'm not sure if the latter
is possible, but for sure it'd be a large task if it is possible.

            regards, tom lane



Re: JIT on Windows with Postgres 12.1

From
Justin
Date:
Here is a link to build  LLVM on windows


On Wed, Feb 12, 2020 at 2:55 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Andres Freund <andres@anarazel.de> writes:
> On 2020-02-12 09:39:19 +0100, Josef Šimánek wrote:
>> I'm not aware of any PG 12 Windows installer with JIT/LLVM enabled.

> It's not currently supported by the windows buildsystem. I don't
> personally have the expertise to change that (I've not worked on windows
> after ~2003). So somebody would have to step in to make that work,
> first.

First you would need either a Windows-native LLVM, or a JIT provider
based on Microsoft's compiler.  I don't know if the former exists
(it would not be our job to create it).  I'm not sure if the latter
is possible, but for sure it'd be a large task if it is possible.

                        regards, tom lane


Re: JIT on Windows with Postgres 12.1

From
Andres Freund
Date:
Hi,

On 2020-02-12 14:54:56 -0500, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2020-02-12 09:39:19 +0100, Josef Šimánek wrote:
> >> I'm not aware of any PG 12 Windows installer with JIT/LLVM enabled.
> 
> > It's not currently supported by the windows buildsystem. I don't
> > personally have the expertise to change that (I've not worked on windows
> > after ~2003). So somebody would have to step in to make that work,
> > first.
> 
> First you would need either a Windows-native LLVM, or a JIT provider
> based on Microsoft's compiler.  I don't know if the former exists
> (it would not be our job to create it).

Yes, LLVM would work on windows. I'd not even be surprised if one could
make it work on windows already when using a mingw based build, instead
of msvc.


> I'm not sure if the latter is possible, but for sure it'd be a large
> task if it is possible.

Yea, it'd be a lot of work to add a second provider. No iea if there's
any support for it around visual studio - but I also see no point going
there.

Greetings,

Andres Freund



Re: JIT on Windows with Postgres 12.1

From
Michael Paquier
Date:
On Wed, Feb 12, 2020 at 12:05:11PM -0800, Andres Freund wrote:
> Yes, LLVM would work on windows. I'd not even be surprised if one could
> make it work on windows already when using a mingw based build, instead
> of msvc.

For MSVC, assuming that you have the proper dependencies for JIT
compilation and LLVM available, and just for the reference if anybody
is interesting in making that happen (Tom and Andres are aware of that
already), one would need to do roughly an update of src/tools/msvc/ to
add a new option path to LLVM, and update install-windows.sgml to add
the new optional dependency.  Note that including LLVM in the builds
means adding conditionally the files part of src/backend/jit/llvm/,
but one can take for example OpenSSL as a model of how it happens.
That would take care of the build.  There may be some
Windows-dependent stuff needed to make JIT work properly on Windows
though.
--
Michael

Attachment

Re: JIT on Windows with Postgres 12.1

From
Andres Freund
Date:
Hi,

On 2020-02-13 12:25:40 +0900, Michael Paquier wrote:
> On Wed, Feb 12, 2020 at 12:05:11PM -0800, Andres Freund wrote:
> > Yes, LLVM would work on windows. I'd not even be surprised if one could
> > make it work on windows already when using a mingw based build, instead
> > of msvc.
> 
> For MSVC, assuming that you have the proper dependencies for JIT
> compilation and LLVM available, and just for the reference if anybody
> is interesting in making that happen (Tom and Andres are aware of that
> already), one would need to do roughly an update of src/tools/msvc/ to
> add a new option path to LLVM, and update install-windows.sgml to add
> the new optional dependency.  Note that including LLVM in the builds
> means adding conditionally the files part of src/backend/jit/llvm/,
> but one can take for example OpenSSL as a model of how it happens.
> That would take care of the build.

That's not really the hard part. That's integrating the generation of
LLVM bitcode files into the buildsystem. As the absolute minimum
llvmjit_types.bc needs to be generated, but to be meaningfully supported
we'd need to generate all the .bc files we're generating on make based
builds.

Greetings,

Andres Freund



Re: JIT on Windows with Postgres 12.1

From
Michael Paquier
Date:
On Wed, Feb 12, 2020 at 07:32:08PM -0800, Andres Freund wrote:
> That's not really the hard part. That's integrating the generation of
> LLVM bitcode files into the buildsystem. As the absolute minimum
> llvmjit_types.bc needs to be generated, but to be meaningfully supported
> we'd need to generate all the .bc files we're generating on make based
> builds.

Oh, I see.  That's indeed...  Not straight-forward.
--
Michael

Attachment

Re: JIT on Windows with Postgres 12.1

From
109 ktke
Date:
On Wed, Feb 12, 2020 at 12:05:11PM -0800, Andres Freund wrote:
> Yes, LLVM would work on windows. I'd not even be surprised if one could
> make it work on windows already when using a mingw based build, instead
> of msvc.
In my mingw environment, postgresql build with option "--with-llvm"
failed with the follwing message. Is it difficult to make PostgreSQL
JIT work on windows using mingw?

--Message1
/mingw64/bin/clang -Wno-ignored-attributes -fno-strict-aliasing
-fwrapv -O2  -I../../../../src/include  -I./src/include/port/win32
"-I../../../../src/include/port/win32" -DWIN32_STACK_RLIMIT=4194304
-DBUILDING_DLL -flto=thin -emit-llvm -c -o latch.bc latch.c
latch.c:680:19: error: static_assert expression is not an integral
constant expression
        StaticAssertStmt(WSA_INVALID_EVENT == NULL, "");
        ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--

--Message2(When commenting latch.c:680 out)
llvmjit_inline.cpp:32:10: fatal error: sys/mman.h: No such file or directory
   32 | #include <sys/mman.h>
      |          ^~~~~~~~~~~~
compilation terminated.
--

--Environment
PostgreSQL:postgresql13.0
Windows:Windows10
MSYS2 packages:mingw-w64-x86_64-gcc base-devel
mingw-w64-x86_64-llvm(llvm11 installed) mingw-w64-x86_64-clang(clang11
installed)
--

--
Yuuki Fujii