Re: Lambda expressions (was Re: BUG #15471) - Mailing list pgsql-hackers

Dear Tom, Andres, Andrew, all

Any update on this interesting topic?

--Stefan

P.S. TIL Databricks SQL lambda functions: "A parameterized expression
that can be passed to a function to control its behavior. For example,
array_sort function (Databricks SQL) accepts a lambda function as an
argument to define a custom sort order."
https://docs.databricks.com/sql/language-manual/sql-ref-lambda-functions.html

Am Di., 30. Okt. 2018 um 23:20 Uhr schrieb Andres Freund <andres@anarazel.de>:
>
> Hi,
>
> On 2018-10-30 16:54:45 -0400, Tom Lane wrote:
> > Andres Freund <andres@anarazel.de> writes:
> > > On 2018-10-30 16:23:37 -0400, Tom Lane wrote:
> > >> Well, a Lambda expression is not something that can be optimized away
> > >> (unless perhaps you can get rid of the need for any of its output Params)
> > >> so I don't see how any of its subexpressions would ever wind up split out
> > >> from inside it.
> >
> > > Isn't that a pretty fundamental requirement for the postgis (and I
> > > presume lots of other) usecases?  What they have is wrapper function
> > > functions like ST_DWithin(geometry, geometry, float) that roughly expand
> > > to something (simplified) like
> >
> > > SELECT $1 && ST_Expand($2, $3) AND _ST_DWithin($1, $2, $3);
> >
> > > where && is the overlaps operator, and then _ST_DWithin is the accurate
> > > computation. That allows a quick bounding-box (or similar) search via
> > > index, and then an accurate re-check.   But $2 might be the result of a
> > > function (with constant input), and that'll currently prevent the SQL
> > > function from being inlined when the function is expensive. And the
> > > postgis folks *want* its functions be marked expensive, because they
> > > really are - but getting index searches is also important.
> >
> > Hm.  But if we're trying to avoid evaluating $2 twice, how would you
> > expect to allow the ST_Expand and _ST_DWithin calls to get separated?
> > They can't be allowed to get very far apart in the tree, ISTM.
>
> I think postgis would be OK just tacking a 'force-inline this' on
> functions like ST_DWithin, and live with the redundant expansion. Right
> now they can either have accurate costs (and thus trigger
> e.g. parallelism / avoid plans with more costs) but no index scans, or
> the reverse.  While not optimal, it'd be better than what's there now -
> but it's still awfully crummy.
>
> And yes, they can't get allowed to get too far apart. That's the problem
> I was trying - and obviously failing - to describe in the lounge, btw
> :).  If we allowed the LET() to be split at the baserel level (around
> the match_restriction_clauses_to_index() in create_index_paths()), it
> would probably be ok, but that'd be architecturally somewhat
> invasive.
>
>
> > The patch as I had it also dealt with another limitation on function
> > inlining, which was the restrictions on what you can do with strict
> > functions, by means of allowing a LambdaExpr to be "strict"; that
> > is short-circuit to a NULL result if any of the Param values turn
> > out null.
>
> Right, and similar with volatile.
>
>
> > It doesn't seem possible to do what you're talking about in that case
> > either.  Maybe the PostGIS guys don't care, since they're probably OK
> > with not marking their wrapper functions strict, but I'm not sure that
> > that's the whole universe we should be worried about.
>
> I agree, but we also should take care not to regress cases of inlining
> like postgis', which currently be separated.  If we'd only create the
> new LET node in cases we previously couldn't inline that's obviously not
> a problem.
>
> Greetings,
>
> Andres Freund
>