Thread: windows build slow due to windows.h includes
Hi, For the AIO stuff I needed to build postgres for windows. And I was a bit horrified by the long compile times. At first I was ready to blame the MS compiler for being slow, until I noticed that using mingw gcc from linux to cross compile to windows is also a *lot* slower than building for linux. I found some blog-post-documented-only compiler flags [1], most importantly /d1reportTime. Which shows that the include processing of postgres.h takes 0.6s [2] Basically all the time in a debug windows build is spent parsing windows.h and related headers. Argh. The amount of stuff we include in win32_port.h and declare is pretty absurd imo. There's really no need to expose the whole backend to all of it. Most of it should just be needed in a few port/ files and a few select users. But that's too much work for my taste. As it turns out there's a partial solution to windows.h being just so damn big, the delightfully named WIN32_LEAN_AND_MEAN. This reduces the non-incremental buildtime in my 8 core windows VM from 187s to 140s. Cross compiling from linux it's master: real 0m53.807s user 22m16.930s sys 2m50.264s WIN32_LEAN_AND_MEAN real 0m32.956s user 12m17.773s sys 1m52.313s Still far from !windows compile times, but still not a bad improvement. Most of the compile time after this is still spent doing parsing / preprocessing. I sidetracked myself into looking at precompiled headers, but it's not trivial to do that right unfortunately. I think it'd be good if win32_port.h were slimmed down, and more of its contents were moved into fake "port/win32/$name-of-unix-header" style headers or such. Greetings, Andres Freund [1] https://aras-p.info/blog/2019/01/21/Another-cool-MSVC-flag-d1reportTime/ [2] postgres.c Include Headers: Count: 483 c:\Users\anfreund\src\postgres\src\include\postgres.h: 0.561795s c:\Users\anfreund\src\postgres\src\include\c.h: 0.556991s c:\Users\anfreund\src\postgres\src\include\postgres_ext.h: 0.000488s c:\Users\anfreund\src\postgres\src\include\pg_config_ext.h: 0.000151s c:\Users\anfreund\src\postgres\src\include\pg_config.h: 0.000551s c:\Users\anfreund\src\postgres\src\include\pg_config_manual.h: 0.000286s c:\Users\anfreund\src\postgres\src\include\pg_config_os.h: 0.014283s C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\crtdefs.h:0.009727s ... c:\Users\anfreund\src\postgres\src\include\port\win32_port.h: 0.487469s C:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\um\winsock2.h:0.449373s ... C:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\um\windows.h:0.439666s
Attachment
On 9/21/21 3:30 PM, Andres Freund wrote: > Hi, > > For the AIO stuff I needed to build postgres for windows. And I was a bit > horrified by the long compile times. At first I was ready to blame the MS > compiler for being slow, until I noticed that using mingw gcc from linux to > cross compile to windows is also a *lot* slower than building for linux. > > I found some blog-post-documented-only compiler flags [1], most importantly > /d1reportTime. Which shows that the include processing of postgres.h takes > 0.6s [2] > > Basically all the time in a debug windows build is spent parsing windows.h and > related headers. Argh. > > The amount of stuff we include in win32_port.h and declare is pretty absurd > imo. There's really no need to expose the whole backend to all of it. Most of > it should just be needed in a few port/ files and a few select users. > > But that's too much work for my taste. As it turns out there's a partial > solution to windows.h being just so damn big, the delightfully named > WIN32_LEAN_AND_MEAN. > > This reduces the non-incremental buildtime in my 8 core windows VM from 187s to > 140s. Cross compiling from linux it's > master: > real 0m53.807s > user 22m16.930s > sys 2m50.264s > WIN32_LEAN_AND_MEAN > real 0m32.956s > user 12m17.773s > sys 1m52.313s Nice! I also see references to VC_EXTRALEAN which defines this and some other stuff that might make things even faster. Worth investigating. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
Hi, On 2021-09-21 16:13:55 -0400, Andrew Dunstan wrote: > I also see references to VC_EXTRALEAN which defines this and some other > stuff that might make things even faster. I don't think that's relevant to "us", just mfc apps (which we gladly aren't). From what I can see we'd have to actually clean up our includes to not have windows.h everywhere or use precompiled headers to benefit further. Greetings, Andres Freund
Em ter., 21 de set. de 2021 às 16:30, Andres Freund <andres@anarazel.de> escreveu:
Hi,
For the AIO stuff I needed to build postgres for windows. And I was a bit
horrified by the long compile times. At first I was ready to blame the MS
compiler for being slow, until I noticed that using mingw gcc from linux to
cross compile to windows is also a *lot* slower than building for linux.
I found some blog-post-documented-only compiler flags [1], most importantly
/d1reportTime. Which shows that the include processing of postgres.h takes
0.6s [2]
Basically all the time in a debug windows build is spent parsing windows.h and
related headers. Argh.
The amount of stuff we include in win32_port.h and declare is pretty absurd
imo. There's really no need to expose the whole backend to all of it. Most of
it should just be needed in a few port/ files and a few select users.
But that's too much work for my taste. As it turns out there's a partial
solution to windows.h being just so damn big, the delightfully named
WIN32_LEAN_AND_MEAN.
+1
But I did a quick dirty test here, and removed windows.h in win32_port.h,
and compiled normally with msvc 2019 (64 bit), would it work with mingw cross compile?
regards,
Ranier Vilela
Hi, On 2021-09-21 20:26:36 -0300, Ranier Vilela wrote: > Em ter., 21 de set. de 2021 às 16:30, Andres Freund <andres@anarazel.de> > escreveu: > > But that's too much work for my taste. As it turns out there's a partial > > solution to windows.h being just so damn big, the delightfully named > > WIN32_LEAN_AND_MEAN. > > > +1 > But I did a quick dirty test here, and removed windows.h in win32_port.h, > and compiled normally with msvc 2019 (64 bit), would it work with mingw > cross compile? That's likely only because winsock indirectly includes windows.h - because of that it won't actually reduce compile time. And you can't remove the other headers that indirectly include windows.h without causing compilation errors. Greetings, Andres Freund
On Tue, Sep 21, 2021 at 12:30:35PM -0700, Andres Freund wrote: > solution to windows.h being just so damn big, the delightfully named > WIN32_LEAN_AND_MEAN. > > This reduces the non-incremental buildtime in my 8 core windows VM from 187s to > 140s. Cross compiling from linux it's > master: > real 0m53.807s > user 22m16.930s > sys 2m50.264s > WIN32_LEAN_AND_MEAN > real 0m32.956s > user 12m17.773s > sys 1m52.313s +1, great win for a one-liner.
On Wed, Sep 22, 2021 at 1:56 AM Andres Freund <andres@anarazel.de> wrote:
On 2021-09-21 20:26:36 -0300, Ranier Vilela wrote:
> Em ter., 21 de set. de 2021 às 16:30, Andres Freund <andres@anarazel.de>
> escreveu:
> > But that's too much work for my taste. As it turns out there's a partial
> > solution to windows.h being just so damn big, the delightfully named
> > WIN32_LEAN_AND_MEAN.
> >
> +1
> But I did a quick dirty test here, and removed windows.h in win32_port.h,
> and compiled normally with msvc 2019 (64 bit), would it work with mingw
> cross compile?
That's likely only because winsock indirectly includes windows.h - because of
that it won't actually reduce compile time. And you can't remove the other
headers that indirectly include windows.h without causing compilation errors.
You are right about winsock2.h including some parts of windows.h, please see note in [1]. You could move the windows.h inclusion for clarity:
+ #define WIN32_LEAN_AND_MEAN
+ #endif
+
+ #include <windows.h>
#include <winsock2.h>
#include <ws2tcpip.h>
- #include <windows.h>
Regards,
Juan José Santamaría Flecha
On Wed, Sep 22, 2021 at 11:14 AM Noah Misch <noah@leadboat.com> wrote: > > On Tue, Sep 21, 2021 at 12:30:35PM -0700, Andres Freund wrote: > > solution to windows.h being just so damn big, the delightfully named > > WIN32_LEAN_AND_MEAN. > > > > This reduces the non-incremental buildtime in my 8 core windows VM from 187s to > > 140s. Cross compiling from linux it's > > master: > > real 0m53.807s > > user 22m16.930s > > sys 2m50.264s > > WIN32_LEAN_AND_MEAN > > real 0m32.956s > > user 12m17.773s > > sys 1m52.313s > > +1, great win for a one-liner. > +1. It reduced the build time of Postgres from "Time Elapsed 00:01:57.60" to "Time Elapsed 00:01:38.11" in my Windows env. (Win 10, MSVC 2019). -- With Regards, Amit Kapila.