Hi,
On 2022-08-06 20:05:14 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2022-08-06 16:09:24 -0700, Noah Misch wrote:
> >> From the earliest days of wrasse, the compiler used too much RAM to build
> >> preproc.o with --enable-debug. As of 2021-04, the compiler's "acomp" phase
> >> needed 10G in one process, and later phases needed 11.6G across two processes.
> >> Compilation wrote 3.7G into TMPDIR. Since /tmp consumes RAM+swap, overriding
> >> TMPDIR relieved 3.7G of RAM pressure. Even with those protections, wrasse
> >> intermittently reaches the 14G limit I impose (via "ulimit -v 14680064"). I
> >> had experimented with different optimization levels, but that didn't help.
>
> > Yikes. And it's not like newer compiler versions are likely to be forthcoming
> > (12.6 is newest and is from 2017...). Wonder if we should just require gcc on
> > solaris... There's a decent amount of stuff we could rip out in that case.
>
> Seems like it's only a matter of time before we add enough stuff to
> the grammar that the build fails, period.
Yea, it doesn't look too far off.
> However, I wonder why exactly it's so large, and why the backend's gram.o
> isn't an even bigger problem. Maybe an effort to cut preproc.o's code
> size could yield dividends?
gram.c also compiles slowly and uses a lot of memory. Roughly ~8GB memory at
the peak (just watching top) and 1m40s (with debugging disabled, temp files on
disk etc).
I don't entirely know what parse.pl actually tries to achieve. The generated
output looks more different from gram.y than I'd have imagined.
It's certainly interesting that it ends up rougly 30% larger .c bison
output. Which roughly matches the difference in memory usage.
> FWIW, my late and unlamented animal gaur was also showing unhappiness with
> the size of preproc.o, manifested as a boatload of warnings like
> /var/tmp//cc0MHZPD.s:11594: Warning: .stabn: description field '109d3' too big, try a different debug format
> which did not happen with gram.o.
I suspect we're going to have to do something about the gram.c size on its
own. It's already the slowest compilation step by a lot, even on modern
compilers.
Greetings,
Andres Freund
