This new return path...
> + if (!tok_done)
> + {
> + if (lex->inc_state->is_last_chunk)
> + return JSON_INVALID_TOKEN;
...also needs to set the token pointers. See one approach in the
attached diff, which additionally asserts that we've consumed the
entire chunk in this case, along with a handful of new tests.
--Jacob