Amit Kapila <amit.kapila16@gmail.com> writes:
> On Tue, Aug 13, 2019 at 3:18 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> To clarify my position --- I think it's definitely possible to improve
>> the situation a great deal. We "just" have to pass down more information
>> about whether rescans are possible.
> Right, you have speculated above that it is possible via adding some
> eflag bits. Can you please describe a bit more about that idea, so
> that somebody else can try to write a patch?
Well, there are two components to solving this problem:
1. What are we going to do about the executor's external API?
Right now, callers of ExecutorStart don't have to say whether they
might call ExecutorRewind. We need some way for callers to make a
binding promise that they won't do any such thing. Perhaps we just
want to invent another flag that's like EXEC_FLAG_BACKWARD, but it's
not clear how it should interact with the existing "soft" REWIND
flag. Nor do I know how far up the call stack will we have to make
changes to make it possible to promise as much as we can -- for
instance, will we have to adapt the SPI interfaces?
2. What happens inside ExecutorStart in response to such promises?
I imagine that we translate them into additional eflags bits that
get passed down to node init functions, possibly with modification
(e.g., nodeNestloop.c would have to revoke the no-rescans promise
to its inner input). You'd need to work out what is the most
convenient set of conventions (positive or negative sense of the
flag bits, etc), and go through all the non-leaf node types to
determine what they can pass down.
(BTW, unless I'm missing something, there's not currently any
enforcement of EXEC_FLAG_BACKWARD, ie a caller can fail to pass
that and then try to back up anyway. We probably want to improve
that situation, and also enforce this new flag about
ExecutorRewind.)
The reason I'm dubious about back-patching this is that each
of these things seems likely to affect external code. Point 1
could affect external callers of the executor, and point 2 is
likely to have consequences for FDWs and custom-scan providers.
Maybe we can set things up so that everything defaults in a
safe direction for unchanged code, but I don't want to contort
the design just to do that.
regards, tom lane
