Re: WIP Incremental JSON Parser - Mailing list pgsql-hackers

From Andrew Dunstan
Subject Re: WIP Incremental JSON Parser
Date
Msg-id CAD5tBcK-RzoWpwb2UBHFOoMxrn1LHaKGt_XJd9wKiQg=rwBjNA@mail.gmail.com
Whole thread Raw
In response to Re: WIP Incremental JSON Parser  (Jacob Champion <jacob.champion@enterprisedb.com>)
Responses Re: WIP Incremental JSON Parser
Re: WIP Incremental JSON Parser
List pgsql-hackers


On Mon, Mar 25, 2024 at 6:15 PM Jacob Champion <jacob.champion@enterprisedb.com> wrote:
On Wed, Mar 20, 2024 at 11:56 PM Andrew Dunstan <andrew@dunslane.net> wrote:
> Thanks, included that and attended to the other issues we discussed. I think this is pretty close now.

Okay, looking over the thread, there are the following open items:
- extend the incremental test in order to exercise the semantic callbacks [1]


Yeah, I'm on a super long plane trip later this week, so I might get it done then :-)
 
- add Assert calls in impossible error cases [2]

ok, will do
 
- error out if the non-incremental lex doesn't consume the entire token [2]

ok, will do
 
- double-check that out of memory is an appropriate failure mode for
the frontend [3]


Well, what's the alternative? The current parser doesn't check stack depth in frontend code. Presumably it too will eventually just run out of memory, possibly rather sooner as the stack frames could  be more expensive than the incremental parser stack extensions.

 

Just as a general style nit:

> +   if (lex->incremental)
> +   {
> +       lex->input = lex->token_terminator = lex->line_start = json;
> +       lex->input_length = len;
> +       lex->inc_state->is_last_chunk = is_last;
> +   }
> +   else
> +       return JSON_INVALID_LEXER_TYPE;

I think flipping this around would probably make it more readable;
something like:

    if (!lex->incremental)
        return JSON_INVALID_LEXER_TYPE;

    lex->input = ...



Noted. will do, Thanks.

cheers

andrew

 
[1] https://www.postgresql.org/message-id/CAOYmi%2BnHV55Uhz%2Bo-HKq0GNiWn2L5gMcuyRQEz_fqpGY%3DpFxKA%40mail.gmail.com
[2] https://www.postgresql.org/message-id/CAD5tBcLi2ffZkktV2qrsKSBykE-N8CiYgrfbv0vZ-F7%3DxLFeqw%40mail.gmail.com
[3] https://www.postgresql.org/message-id/CAOYmi%2BnY%3DrF6dJCzaOuA3d-3FbwXCcecOs_S1NutexFA3dRXAw%40mail.gmail.com

pgsql-hackers by date:

Previous
From: Jacob Champion
Date:
Subject: Re: WIP Incremental JSON Parser
Next
From: Jacob Champion
Date:
Subject: Re: WIP Incremental JSON Parser