Thread: jit and explain nontext
/* don't print information if no JITing happened */
if (!ji || ji->created_functions == 0)
return;
This applies even when (es->format != EXPLAIN_FORMAT_TEXT), which I think is
wrong. Jit use can be determined by cost, so I think jit details should be
shown in non-text format whenever ji!=NULL, even if it's zeros. Arguably, bits
could be omitted if jit_expressions=off or jit_tuple_deforming=off, but I don't
see the point.
--
Justin
On Thu, 15 Oct 2020 at 08:39, Justin Pryzby <pryzby@telsasoft.com> wrote: > > /* don't print information if no JITing happened */ > if (!ji || ji->created_functions == 0) > return; > > This applies even when (es->format != EXPLAIN_FORMAT_TEXT), which I think is > wrong. Jit use can be determined by cost, so I think jit details should be > shown in non-text format whenever ji!=NULL, even if it's zeros. Arguably, bits > could be omitted if jit_expressions=off or jit_tuple_deforming=off, but I don't > see the point. Just for some reference. Some wisdom was shared in [1], which made a lot of sense to me. If we apply that, then we just need to decide if displaying any jit related fields without any jitted expressions is relevant. I'm a bit undecided. [1] https://www.postgresql.org/message-id/2276865.1593102811%40sss.pgh.pa.us
David Rowley <dgrowleyml@gmail.com> writes:
> Just for some reference. Some wisdom was shared in [1], which made a
> lot of sense to me.
> If we apply that, then we just need to decide if displaying any jit
> related fields without any jitted expressions is relevant.
Hmm, I dunno if my opinion counts as "wisdom", but what I was arguing for
there was that we should print stuff if it's potentially invoked by a
run-time decision, but not if it was excluded at plan time. I'm not
totally clear on whether jitting decisions are fixed by the plan tree
(including its cost values) or if the executor can make different
decisions in different executions of the identical plan tree.
If the latter, then I agree with Justin that this is a bug.
regards, tom lane
On Thu, 15 Oct 2020 at 14:15, Tom Lane <tgl@sss.pgh.pa.us> wrote: > > David Rowley <dgrowleyml@gmail.com> writes: > > Just for some reference. Some wisdom was shared in [1], which made a > > lot of sense to me. > > If we apply that, then we just need to decide if displaying any jit > > related fields without any jitted expressions is relevant. > > Hmm, I dunno if my opinion counts as "wisdom", but what I was arguing for > there was that we should print stuff if it's potentially invoked by a > run-time decision, but not if it was excluded at plan time. I'm not > totally clear on whether jitting decisions are fixed by the plan tree > (including its cost values) or if the executor can make different > decisions in different executions of the identical plan tree. > If the latter, then I agree with Justin that this is a bug. As far as I know, the only exception where the executor overwrites the planner's decision is in nodeValuesscan.c where it turns jit off because each VALUES will get evaluated just once, which would be a waste of effort to JIT. Apart from that the choice is baked in by the planner and set in PlannedStmt.jitfFlags. David
On Thu, Oct 15, 2020 at 02:23:01PM +1300, David Rowley wrote: > On Thu, 15 Oct 2020 at 14:15, Tom Lane <tgl@sss.pgh.pa.us> wrote: > > David Rowley <dgrowleyml@gmail.com> writes: > > > Just for some reference. Some wisdom was shared in [1], which made a > > > lot of sense to me. > > > If we apply that, then we just need to decide if displaying any jit > > > related fields without any jitted expressions is relevant. > > > > Hmm, I dunno if my opinion counts as "wisdom", but what I was arguing for > > there was that we should print stuff if it's potentially invoked by a > > run-time decision, but not if it was excluded at plan time. I'm not > > totally clear on whether jitting decisions are fixed by the plan tree > > (including its cost values) or if the executor can make different > > decisions in different executions of the identical plan tree. > > If the latter, then I agree with Justin that this is a bug. > > As far as I know, the only exception where the executor overwrites the > planner's decision is in nodeValuesscan.c where it turns jit off > because each VALUES will get evaluated just once, which would be a > waste of effort to JIT. > > Apart from that the choice is baked in by the planner and set in > PlannedStmt.jitfFlags. What about the GUCs themselves ? They can change after planning, which means a given execution of a plan might or might not use jit. -- Justin
On Thu, 15 Oct 2020 at 14:43, Justin Pryzby <pryzby@telsasoft.com> wrote: > > On Thu, Oct 15, 2020 at 02:23:01PM +1300, David Rowley wrote: > > On Thu, 15 Oct 2020 at 14:15, Tom Lane <tgl@sss.pgh.pa.us> wrote: > > > Hmm, I dunno if my opinion counts as "wisdom", but what I was arguing for > > > there was that we should print stuff if it's potentially invoked by a > > > run-time decision, but not if it was excluded at plan time. I'm not > > > totally clear on whether jitting decisions are fixed by the plan tree > > > (including its cost values) or if the executor can make different > > > decisions in different executions of the identical plan tree. > > > If the latter, then I agree with Justin that this is a bug. > > > > As far as I know, the only exception where the executor overwrites the > > planner's decision is in nodeValuesscan.c where it turns jit off > > because each VALUES will get evaluated just once, which would be a > > waste of effort to JIT. > > > > Apart from that the choice is baked in by the planner and set in > > PlannedStmt.jitfFlags. > > What about the GUCs themselves ? > > They can change after planning, which means a given execution of a plan might > or might not use jit. That's a pretty good point. If we do SET enable_sort TO off; then cached plans are unaffected. That's not the case when someone does SET jit TO off; as we'll check that in provider_init() during execution. Although, switching jit back on again works differently. If the planner saw it was off then switching it on again won't have existing plans use it. That's slightly weird, but perhaps it was done that way to ensure there was a hard off switch. You might want to ensure that to ensure queries don't break if there was some problem with LLVM libraries. David
Hi, On 2020-10-15 14:51:38 +1300, David Rowley wrote: > That's a pretty good point. If we do SET enable_sort TO off; then > cached plans are unaffected. In contrast to that we do however use the current work_mem for execution, I believe. Which could be fairly nasty, if a plan was made for a huge work_mem, for example. > That's not the case when someone does SET jit TO off; as we'll check > that in provider_init() during execution. Although, switching jit > back on again works differently. If the planner saw it was off then > switching it on again won't have existing plans use it. That's > slightly weird, but perhaps it was done that way to ensure there was a > hard off switch. It was motivated by not wanting to just enable JIT for queries that were prepared within something like SET LOCAL jit=off;PREPARE; RESET jit;. I'm open to revising it, but that's where it's coming from. Greetings, Andres Freund
On Thu, Oct 15, 2020 at 02:51:38PM +1300, David Rowley wrote: > On Thu, 15 Oct 2020 at 14:43, Justin Pryzby <pryzby@telsasoft.com> wrote: > > On Thu, Oct 15, 2020 at 02:23:01PM +1300, David Rowley wrote: > > > On Thu, 15 Oct 2020 at 14:15, Tom Lane <tgl@sss.pgh.pa.us> wrote: > > > > Hmm, I dunno if my opinion counts as "wisdom", but what I was arguing for > > > > there was that we should print stuff if it's potentially invoked by a > > > > run-time decision, but not if it was excluded at plan time. I'm not > > > > totally clear on whether jitting decisions are fixed by the plan tree > > > > (including its cost values) or if the executor can make different > > > > decisions in different executions of the identical plan tree. > > > > If the latter, then I agree with Justin that this is a bug. > > > > > > As far as I know, the only exception where the executor overwrites the > > > planner's decision is in nodeValuesscan.c where it turns jit off > > > because each VALUES will get evaluated just once, which would be a > > > waste of effort to JIT. > > > > > > Apart from that the choice is baked in by the planner and set in > > > PlannedStmt.jitfFlags. > > > > What about the GUCs themselves ? > > > > They can change after planning, which means a given execution of a plan might > > or might not use jit. > > That's a pretty good point. Added at: https://commitfest.postgresql.org/30/2766/ diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c index 41317f1837..7345971507 100644 --- a/src/backend/commands/explain.c +++ b/src/backend/commands/explain.c @@ -839,7 +839,8 @@ ExplainPrintJIT(ExplainState *es, int jit_flags, JitInstrumentation *ji) instr_time total_time; /* don't print information if no JITing happened */ - if (!ji || ji->created_functions == 0) + if (!ji || (ji->created_functions == 0 && + es->format == EXPLAIN_FORMAT_TEXT)) return; /* calculate total time */ -- 2.17.0
Attachment
On Sun, 18 Oct 2020 at 08:21, Justin Pryzby <pryzby@telsasoft.com> wrote: > /* don't print information if no JITing happened */ > - if (!ji || ji->created_functions == 0) > + if (!ji || (ji->created_functions == 0 && > + es->format == EXPLAIN_FORMAT_TEXT)) > return; Isn't that comment now outdated? I imagine something more like; /* Only show JIT details when we jitted something or when in non-text mode */ might be better after making that code change. David
On 2020-10-17 21:21, Justin Pryzby wrote: > Added at:https://commitfest.postgresql.org/30/2766/ > > diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c > index 41317f1837..7345971507 100644 > --- a/src/backend/commands/explain.c > +++ b/src/backend/commands/explain.c > @@ -839,7 +839,8 @@ ExplainPrintJIT(ExplainState *es, int jit_flags, JitInstrumentation *ji) > instr_time total_time; > > /* don't print information if no JITing happened */ > - if (!ji || ji->created_functions == 0) > + if (!ji || (ji->created_functions == 0 && > + es->format == EXPLAIN_FORMAT_TEXT)) > return; > > /* calculate total time */ Can you show an output example of where this patch makes a difference? Just from reading the description, I would expect some kind of additional JIT-related output from something like EXPLAIN (FORMAT YAML) SELECT 1; but I don't see anything.
On 2020-11-20 17:16, Justin Pryzby wrote: > It matters if it was planned with jit but executed without jit. > > postgres=# DEALLOCATE p; SET jit=on; SET jit_above_cost=0; prepare p as select from generate_series(1,9); explain(formatyaml) execute p; SET jit=off; explain(format yaml) execute p; > > Patched shows this for both explains: > JIT: + > Functions: 3 + > > Unpatched shows only in the first case. In this context, I don't see the point of this change. If you set jit=off explicitly, then there is no need to clutter the EXPLAIN output with a bunch of zeroes about JIT.
Justin Pryzby <pryzby@telsasoft.com> writes:
> On Sat, Nov 21, 2020 at 10:26:00AM -0600, Justin Pryzby wrote:
>> On Sat, Nov 21, 2020 at 08:39:11AM +0100, Peter Eisentraut wrote:
>>> In this context, I don't see the point of this change. If you set jit=off
>>> explicitly, then there is no need to clutter the EXPLAIN output with a bunch
>>> of zeroes about JIT.
> If there's no interest or agreement in it, we should just close it.
> I have no personal need for it, but noticed it in passing.
I dug around a bit and saw that essentially all of the JIT control
GUCs are consulted only at plan time (cf standard_planner, which
fills PlannedStmt.jitFlags based on the then-active settings).
So the only thing that really counts as a "run time decision"
here is that if you set jit = off between planning and execution,
or if we fail to load the JIT provider at all, then you'll get
no JITting even though the planner expected it to happen.
On balance I agree with Peter's opinion that this isn't worth
changing. I would be for the patch if the executor had a little
more freedom of action, but as things stand there's not much
freedom there.
regards, tom lane
On Fri, Jan 15, 2021 at 02:53:49PM -0500, Tom Lane wrote: > On balance I agree with Peter's opinion that this isn't worth > changing. I would be for the patch if the executor had a little > more freedom of action, but as things stand there's not much > freedom there. Thanks for looking CF: withdrawn. -- Justin