Thread: Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part

Florents Tselai <florents.tselai@gmail.com> writes:
> This patch is a follow-up and generalization to [0].
> It adds the following jsonpath methods:  lower, upper, initcap, l/r/btrim,
> replace, split_part.

How are you going to deal with the fact that this makes jsonpath
operations not guaranteed immutable?  (See commit cb599b9dd
for some context.)  Those are all going to have behavior that's
dependent on the underlying locale.

We have the kluge of having separate "_tz" functions to support
non-immutable datetime operations, but that way doesn't seem like
it's going to scale well to multiple sources of mutability.

            regards, tom lane

On Thu, Sep 26, 2024 at 12:04 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Florents Tselai <florents.tselai@gmail.com> writes:
> > This patch is a follow-up and generalization to [0].
> > It adds the following jsonpath methods:  lower, upper, initcap, l/r/btrim,
> > replace, split_part.
>
> How are you going to deal with the fact that this makes jsonpath
> operations not guaranteed immutable?  (See commit cb599b9dd
> for some context.)  Those are all going to have behavior that's
> dependent on the underlying locale.
>
> We have the kluge of having separate "_tz" functions to support
> non-immutable datetime operations, but that way doesn't seem like
> it's going to scale well to multiple sources of mutability.

While inventing "_tz" functions I was thinking about jsonpath methods
and operators defined in standard then.  Now I see huge interest on
extending that.  I wonder if we can introduce a notion of flexible
mutability?  Imagine that jsonb_path_query() function (and others) has
another function which analyzes arguments and reports mutability.  If
jsonpath argument is constant and all methods inside are safe then
jsonb_path_query() is immutable otherwise it is stable.  I was
thinking about that back working on jsonpath, but that time problem
seemed too limited for this kind of solution.  Now, it's possibly time
to shake off the dust from this idea.  What do you think?

------
Regards,
Alexander Korotkov
Supabase

On Sep 26, 2024, at 13:59, Florents Tselai <florents.tselai@gmail.com> wrote:

> Speaking of extensible: the jsonpath standard does mention function extensions [1] ,
> so it looks like we're covered by the standard, and the mutability aspect is an implementation detail. No?

That’s not the standard used for Postgres jsonpath. Postgres follows the SQL/JSON standard in the SQL standard, which
isnot publicly available, but a few people on the list have copies they’ve purchased and so could provide some context. 

In a previous post I wondered if the SQL standard had some facility for function extensions, but I suspect not. Maybe
inthe next iteration? 

> And having said that, the whole jsonb/jsonpath parser/executor infrastructure is extremely powerful
> and kinda under-utilized if we use it "only" for jsonpath.
> Tbh, I can see it supporting more specific DSLs and even offering hooks for extensions.
> And I know for certain I'm not the only one thinking about this.
> See [2] for example where they've lifted, shifted and renamed the jsonb/jsonpath infra to build a separate language
forgraphs 

I’m all for extensibility, though jsonpath does need to continue to comply with the SQL standard. Do you have some idea
ofthe sorts of hooks that would allow extension authors to use some of that underlying capability? 

Best,

David

> On 27 Sep 2024, at 12:45 PM, David E. Wheeler <david@justatheory.com> wrote:
>
> On Sep 26, 2024, at 13:59, Florents Tselai <florents.tselai@gmail.com> wrote:
>
>> Speaking of extensible: the jsonpath standard does mention function extensions [1] ,
>> so it looks like we're covered by the standard, and the mutability aspect is an implementation detail. No?
>
> That’s not the standard used for Postgres jsonpath. Postgres follows the SQL/JSON standard in the SQL standard, which
isnot publicly available, but a few people on the list have copies they’ve purchased and so could provide some context. 
>
> In a previous post I wondered if the SQL standard had some facility for function extensions, but I suspect not. Maybe
inthe next iteration? 
>
>> And having said that, the whole jsonb/jsonpath parser/executor infrastructure is extremely powerful
>> and kinda under-utilized if we use it "only" for jsonpath.
>> Tbh, I can see it supporting more specific DSLs and even offering hooks for extensions.
>> And I know for certain I'm not the only one thinking about this.
>> See [2] for example where they've lifted, shifted and renamed the jsonb/jsonpath infra to build a separate language
forgraphs 
>
> I’m all for extensibility, though jsonpath does need to continue to comply with the SQL standard. Do you have some
ideaof the sorts of hooks that would allow extension authors to use some of that underlying capability? 

Re-tracing what I had to do

1. Define a new JsonPathItemType jpiMyExtType and map it to a JsonPathKeyword
2. Add a new JsonPathKeyword and make the lexer and parser aware of that,
3. Tell the main executor executeItemOptUnwrapTarget what to do when the new type is matched.

I think 1, 2 are the trickiest because they require hooks to  jsonpath_scan.l and parser jsonpath_gram.y

3. is the meat of a potential hook, which would be something like
extern JsonPathExecResult executeOnMyJsonpathItem(JsonPathExecContext *cxt, JsonbValue *jb, JsonValueList *found);
This should be called by the main executor executeItemOptUnwrapTarget when it encounters case jpiMyExtType

It looks like quite an endeavor, to be honest.