On Wed, 29 Jul 2020 at 09:28, Andres Freund <andres@anarazel.de> wrote:
> FWIW, I created a demo workload for this, and repro'ed the issue with
> that. Those improvements does make a very significant difference:
> Before:
> Timing: Generation 335.345 ms, Inlining 51.025 ms, Optimization 11967.776 ms, Emission 9201.499 ms, Total
21555.645ms
> IR size: unoptimized: 9022868 bytes, optimized: 6206368 bytes
>
> After:
> Timing: Generation 261.283 ms, Inlining 30.875 ms, Optimization 1671.969 ms, Emission 18.557 ms, Total 1982.683
ms
> IR size: unoptimized 8776100 bytes, optimized 115868 bytes
That's a really impressive speedup. However, no matter how fast we
make the compilation, it's still most likely to be a waste of time
doing it for plan nodes that are just not that costly.
I just wrote a patch to consider JIT on a per-plan-node basis instead
of globally over the entire plan. I'll post it to -hackers.
With a 1000 partition table where all of the cost is on just 1
partition, running a query that hits all partitions, I see:
Master jit=on:
JIT:
Functions: 3002
Options: Inlining true, Optimization true, Expressions true, Deforming true
Timing: Generation 141.587 ms, Inlining 11.760 ms, Optimization
6518.664 ms, Emission 3152.266 ms, Total 9824.277 ms
Execution Time: 12588.292 ms
Master jit=off:
Execution Time: 3672.391 ms
Patched jit=on:
JIT:
Functions: 5
Options: Inlining true, Optimization true, Expressions true, Deforming true
Timing: Generation 0.675 ms, Inlining 3.322 ms, Optimization 10.766
ms, Emission 5.892 ms, Total 20.655 ms
Execution Time: 2754.160 ms
Most likely the EXPLAIN output will need to do something more than
show true/false for the options here, but I didn't want to go to too
much trouble unless this is seen as a good direction to go in.
> That obviously needs to be improved further, but it's already a lot
> better. In particular after these changes the generated code could be
> cached.
That would be a game-changer.
David
