Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size - Mailing list pgsql-hackers

On 8/9/22 11:03 AM, Andrew Dunstan wrote:
> 
> On 2022-08-09 Tu 09:59, Jonathan S. Katz wrote:

>> The RMT met today to discuss the state of this open item surrounding
>> the SQL/JSON feature set. We discussed the specific concerns raised
>> about the code and debated four different options:
>>
>>    1. Do nothing and continue with the community process of stabilizing
>> the code without significant redesign
>>
>>    2. Recommend holding up the v15 release to allow for the code to be
>> redesigned and fixed (as based on Andres' estimates, this would push
>> the release out several months).
>>
>>    3. Revert the problematic parts of the code but try to include some
>> of the features in the v15 release (e.g. JSON_TABLE)
>>
>>    4. Revert the feature set and redesign and try to include for v16
>>
>> Based on the concerns raised, the RMT is recommending option #4, to
>> revert the SQL/JSON changes for v15, and come back with a redesign for
>> v16.
>>
>> If folks think there are some bits we can include in v15, we can
>> consider option #3. (Personally, I would like to see if we can keep
>> JSON_TABLE, but if there is too much overlap with the problematic
>> portions of the code I am fine with waiting for v16).
>>
>> At this stage in the release process coupled with the concerns, we're
>> a bit worried about introducing changes that are unpredictable in
>> terms of stability and maintenance. We also do not want to hold up the
>> release while this feature set is goes through a redesign without
>> agreement on what such a design would look like as well as a timeline.
>>
>> Note that the above are the RMT's recommendations; while the RMT can
>> explicitly call for a revert, we want to first guide the discussion on
>> the best path forward knowing the challenges for including these
>> features in v15.
>>
>>
> 
> I very much doubt option 3 is feasible. The parts that are controversial
> go back at least in part to the first patches of the series. Trying to
> salvage anything would almost certainly be more disruptive than trying
> to fix it.
> 
> I'm not sure what the danger is to stability, performance or correctness
> in applying the changes Amit has proposed for release 15. But if that
> danger is judged to be too great then I agree we should revert.

Speaking personally, I would like to see what we could do to include 
support for this batch of the SQL/JSON features in v15. What is included 
looks like it closes most of the gap on what we've been missing 
syntactically since the standard was adopted, and the JSON_TABLE work is 
very convenient for converting JSON data into a relational format. I 
believe having this feature set is important for maintaining standards 
compliance, interoperability, tooling support, and general usability. 
Plus, JSON still seems to be pretty popular :)

Rereading the thread for the umpteenth time, I have seen Amit working 
through Andres' concerns. From my read, the ones that seem pressing are:

* Lack of design documentation, which may be leading to some of the 
general design concerns
* The use of substransactions within the executor, though docs 
explaining the decisions on that could alleviate it (I realize this is a 
big topic and any summary I give won't capture the full nuance)
* Debate on how to handle the type coercions

(Please correct me if I've missed anything).

I hope that these can be addressed satisfactorily in a reasonable (read: 
now a much shorter) timeframe so we can include the SQL/JSON work in v15.

With my RMT hat on, the issue is that we're now at beta 3 and we still 
have not not reached a resolution on this open item. Even if it were 
committed tomorrow, we would definitely need a beta 4, and we would want 
to let the code bake a bit more to ensure we get adequate test coverage 
on it. This would likely put the release date into October, presuming we 
have not found any other issues that could cause a release delay.

With my advocacy hat on, we're at the point in the release cycle where I 
kick off the GA release process (e.g. announcement drafting). Not 
knowing the final status of a feature that's likely to be highlighted 
makes it difficult to write said release as well as kick off the other 
machinery (e.g. translations). If there is at least a decision on next 
steps, I can adjust the GA release process timeline.

> I should add that I'm very grateful to Amit for his work, and I'm sure
> it isn't wasted effort, whatever the decision.

+1. While I've been quiet on this thread to date, I have definitely seen 
Amit working hard on addressing the concerns.

Thanks,

Jonathan