Re: Lambda expressions (was Re: BUG #15471) - Mailing list pgsql-hackers

Andres Freund <andres@anarazel.de> writes:
> On 2018-10-30 15:04:55 -0400, Tom Lane wrote:
>> The core idea that I'm working on is to invent a new node type
>> LambdaExpr that evaluates an expression and substitutes it as a
>> Param into another expression, notationally sort of like
>> LET($n := expression1 IN ... result expression using $n ...)

> Right, that's what I was thinking about as well (except I called it a
> WithValues node).

> How did you deal with the fact that we might extract subset of the LET()
> into e.g. a RestrictionInfo (and then e.g. an IndexPath), but another
> part would be e.g. evaluated as part of a qual?

Well, a Lambda expression is not something that can be optimized away
(unless perhaps you can get rid of the need for any of its output Params)
so I don't see how any of its subexpressions would ever wind up split out
from inside it.

> I wonder a bit if we should have Param nodes of the LET kind refer to
> their "parent" LET node, and have something paramlevelsup or letlevelsup
> (similar to varlevelsup) type logic, where each LET node has a link to
> its parent. The numbering for each level would start at 0, solving the
> issue of breaking equal() in most cases. But that seems awfully
> complicated.

We can't have up-links in an expression tree without creating all sorts of
fun circularity problems.  However, maybe we do not need to.  It might be
sufficient if each Lambda has a nesting depth number that's guaranteed
greater than the nesting depth number of any parent Lambda --- not
necessarily exactly parent + 1, just greater.  Then if its referencing
Param(s) also have that nesting depth number, we can tell what's what.
If there are other Lambdas elsewhere with the same nesting depth, it's
not a problem, as long as they aren't parent or child of one with a
conflicting depth.

The trick here would be to preserve that relationship when doing tree
manipulations --- subquery flattening, in particular, would likely find
that to be a nightmare when pulling up Lambdas.  Maybe it could be made
to work, but I'm not sure.

Also, this doesn't seem like it fixes the equal() problem.  We could fix
equal() if we numbered from the bottom up (that is, a Lambda has depth
exactly 0 if it contains no Lambdas, exactly 1 if it contains only Lambdas
with depth 0, etc) but I'm afraid that definition makes the tree
manipulation problem even worse.

The approach I was using was just to assign unique output Param IDs to
each Lambda, generated from a query-wide counter.  That seems to make
the tree manipulations a lot more robust than I think these other ways
could hope to be, and I'm not sure that losing equal() matching is a huge
problem as long as we manage not to have it happen on parser output trees
(which is what DDL works on).

> Alternatively I wonder if there's really a need to have a Param number
> for this new kind of Param. All it really needs is a way to 1) refer to
> the expression that determines it value 2) a way to avoid repeatedly
> evaluate the expression 3) a way to have the cached values reset between
> expression evaluations.

Hm, isn't (1) going to end up being isomorphic to a Param number?
Also, I don't think we need (3) as long as we use a Lambda node, because
it's descending through the Lambda node that triggers evaluation of the
associated Param(s).

I'd been envisioning that a Lambda could set more than one Param, so you'd
need both a depth number and an index number within the referenced Lambda.
That's not utterly necessary to allow, but stacking several Lambdas for a
single inlined function seems kinda inefficient.

            regards, tom lane