Thread: printing JsonbPair values of input JSONB on server side?

>>>>> "T" == T L <tinlyx@gmail.com> writes:

 T>     //Problem line!!!
 T>     //            elog(NOTICE, "print_kv_pair(): k = %s, v = %s",
 T>      ptr-> key.val.string.val, numeric_out(ptr->value.val.numeric));

string.val isn't a C string (notice the "not null terminated" comment in
the structure definition), and you can't call numeric_out like that.
Either of those would crash it.

You could use pnstrdup to get a valid C string, and use
DatumGetCString(DirectFunctionCall1(
                  numeric_out,
                  NumericGetDatum(ptr->value.val.numeric)))

to get the numeric value as a C string.

--
Andrew (irc:RhodiumToad)

>>>>> "T" == T L <tinlyx@gmail.com> writes:

 T>         //problem lines!!! //either elog crashes pg server
 T>         char *buf = pnstrdup(ptr->key.val.string.val,
 T>                              ptr-> key.val.string.len);
 T>         elog(NOTICE, "print_kv_pair(): k = %s",
 T> (ptr->key).val.string.val);  //debug

It doesn't help to make a null-terminated copy of the string if you're
then just going to try and print the original.

    elog(NOTICE, "print_kv_pair(): k = %s", buf);

 T>         elog(NOTICE, "print_kv_pair(): v = %s",
 T> DatumGetCString(DirectFunctionCall1(numeric_out,
 T>                 NumericGetDatum(ptr->value.val.numeric))) ); //debug

That should work, _provided_ that value.type == jbvNumeric - did you
consider checking that first?

--
Andrew (irc:RhodiumToad)

>>>>> "T" == T L <tinlyx@gmail.com> writes:

 T> Below is my test. It prints a strange character instead of "a"; and
 T> says that the value isn't numeric.

Yeah, there's plenty else wrong with your code.

Did you look at how JsonbToCStringWorker does it? that looks like the
best example I can find on a quick scan.

--
Andrew (irc:RhodiumToad)

Yeah I'm not sure why you're looping using pointer math, the iterators are there to provide that service.  Another function to check out 'jsonb_each', other than the set returning function parts, it does what it looks like your are trying to do.

-Michel

On Mon, Mar 18, 2019 at 4:12 PM Andrew Gierth <andrew@tao11.riddles.org.uk> wrote:

>>>>> "T" == T L <tinlyx@gmail.com> writes:

 T> Below is my test. It prints a strange character instead of "a"; and
 T> says that the value isn't numeric.

Yeah, there's plenty else wrong with your code.

Did you look at how JsonbToCStringWorker does it? that looks like the
best example I can find on a quick scan.

--
Andrew (irc:RhodiumToad)

I need this in my C code on the server side. Any link to the `jsonb_each` for this? Examples I found in a quick search are on the client side in SQL.

I am just confused about the various jsonb types and how to effectively extract values and convert between them:

There are Jsonb, JsonbValue (plus the associated JsonbPair ) to begin with. The ` JsonbToCStringWorker ` example that Andrew pointed out uses still another "JsonbContainer" type.

But the type I get from "PG_GETARG_JSONB_P" is Jsonb. And it doesn't fit into " JsonbContainer" or the pointer math about "JsonPair" that I found online.

What I am struggling with adapting some of the iterator code I saw is how to delete irrelevant code without breaking it. My use case is very restricted and handles hstore-like jsonb's.

I don't need or want the code to have the ability to descend into nested objects or handle arrays etc., as they are invalid input in my case.

I thought the pointer math example I found is easier to adapt, but I couldn't get a valid "JsonbPair" from the input parameter to feed into the pointer math.

On Tue, Mar 19, 2019 at 9:50 AM Michel Pelletier <pelletier.michel@gmail.com> wrote:

Yeah I'm not sure why you're looping using pointer math, the iterators are there to provide that service.  Another function to check out 'jsonb_each', other than the set returning function parts, it does what it looks like your are trying to do.

-Michel

On Mon, Mar 18, 2019 at 4:12 PM Andrew Gierth <andrew@tao11.riddles.org.uk> wrote:

>>>>> "T" == T L <tinlyx@gmail.com> writes:

 T> Below is my test. It prints a strange character instead of "a"; and
 T> says that the value isn't numeric.

Yeah, there's plenty else wrong with your code.

Did you look at how JsonbToCStringWorker does it? that looks like the
best example I can find on a quick scan.

--
Andrew (irc:RhodiumToad)

jsonb_each is a wrapper around each_worker_jsonb.  It produces a row for every key/value pair in an object.

https://doxygen.postgresql.org/jsonfuncs_8c.html#a7511a3aa3918eb956f3f4211d07bdbb0

the iteration is:

  while ((r = JsonbIteratorNext(&it, &v, skipNested)) != WJB_DONE)

On Tue, Mar 19, 2019 at 11:20 AM T L <tinlyx@gmail.com> wrote:

I need this in my C code on the server side. Any link to the `jsonb_each` for this? Examples I found in a quick search are on the client side in SQL.

I am just confused about the various jsonb types and how to effectively extract values and convert between them:

There are Jsonb, JsonbValue (plus the associated JsonbPair ) to begin with. The ` JsonbToCStringWorker ` example that Andrew pointed out uses still another "JsonbContainer" type.

But the type I get from "PG_GETARG_JSONB_P" is Jsonb. And it doesn't fit into " JsonbContainer" or the pointer math about "JsonPair" that I found online.

What I am struggling with adapting some of the iterator code I saw is how to delete irrelevant code without breaking it. My use case is very restricted and handles hstore-like jsonb's.

I don't need or want the code to have the ability to descend into nested objects or handle arrays etc., as they are invalid input in my case.

I thought the pointer math example I found is easier to adapt, but I couldn't get a valid "JsonbPair" from the input parameter to feed into the pointer math.

On Tue, Mar 19, 2019 at 9:50 AM Michel Pelletier <pelletier.michel@gmail.com> wrote:

Yeah I'm not sure why you're looping using pointer math, the iterators are there to provide that service.  Another function to check out 'jsonb_each', other than the set returning function parts, it does what it looks like your are trying to do.

-Michel

On Mon, Mar 18, 2019 at 4:12 PM Andrew Gierth <andrew@tao11.riddles.org.uk> wrote:

>>>>> "T" == T L <tinlyx@gmail.com> writes:

 T> Below is my test. It prints a strange character instead of "a"; and
 T> says that the value isn't numeric.

Yeah, there's plenty else wrong with your code.

Did you look at how JsonbToCStringWorker does it? that looks like the
best example I can find on a quick scan.

--
Andrew (irc:RhodiumToad)

Thanks a lot for the hint. I've used the iteration style and cleaned up the code as far as I can.

It now correctly prints the keys and values, but the server crashes near function return.

Any suggestions?

-- function code --

PG_FUNCTION_INFO_V1(print_kv_pair);
Datum
print_kv_pair(PG_FUNCTION_ARGS)
{
    //1. extracting JsonbValue
    Jsonb *jb = PG_GETARG_JSONB_P(0);
    JsonbIterator *it;
    JsonbValue    v;
    JsonbIteratorToken r;
    JsonbParseState *state = NULL;

    if (jb == NULL)
        PG_RETURN_BOOL(false);

    if (!JB_ROOT_IS_OBJECT(jb))
        ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("Can only take objects")));

    it = JsonbIteratorInit(&jb->root);
    r = JsonbIteratorNext(&it, &v, false);
    if (r != WJB_BEGIN_OBJECT)
        ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("Iterator was not an object")));

    //2. iterating through key-value pairs
    char *buf;
    while ((r = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
    {
        switch (r) {
        case WJB_KEY:
            buf = pnstrdup(v.val.string.val, v.val.string.len);
            elog(NOTICE, "print_kv_pair(): k = %s", buf);  //debug
            break;
        case WJB_VALUE:
            if (v.type != jbvNumeric) {
                ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("value must be numeric")));
            }
            elog(NOTICE, "print_kv_pair(): v = %s", DatumGetCString(DirectFunctionCall1(numeric_out,
                    NumericGetDatum(v.val.numeric))) ); //debug
            break;
        case WJB_END_OBJECT:
            break;
        default:
            elog(ERROR, "invalid JsonbIteratorNext rc: %d", (int ) r);
        }
    }
    elog(NOTICE, "print_kv_pair(): ok4");

    PG_RETURN_BOOL(true);
}

-- output --

=> select print_kv_pair('{"a":1, "b": 2}');
NOTICE:  print_kv_pair(): k = a
NOTICE:  print_kv_pair(): v = 1
NOTICE:  print_kv_pair(): k = b
NOTICE:  print_kv_pair(): v = 2
NOTICE:  print_kv_pair(): ok4
server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!> 

On Tue, Mar 19, 2019 at 2:22 PM Michel Pelletier <pelletier.michel@gmail.com> wrote:

jsonb_each is a wrapper around each_worker_jsonb.  It produces a row for every key/value pair in an object.

https://doxygen.postgresql.org/jsonfuncs_8c.html#a7511a3aa3918eb956f3f4211d07bdbb0

the iteration is:

  while ((r = JsonbIteratorNext(&it, &v, skipNested)) != WJB_DONE)

On Tue, Mar 19, 2019 at 11:20 AM T L <tinlyx@gmail.com> wrote:

I need this in my C code on the server side. Any link to the `jsonb_each` for this? Examples I found in a quick search are on the client side in SQL.

I am just confused about the various jsonb types and how to effectively extract values and convert between them:

There are Jsonb, JsonbValue (plus the associated JsonbPair ) to begin with. The ` JsonbToCStringWorker ` example that Andrew pointed out uses still another "JsonbContainer" type.

But the type I get from "PG_GETARG_JSONB_P" is Jsonb. And it doesn't fit into " JsonbContainer" or the pointer math about "JsonPair" that I found online.

What I am struggling with adapting some of the iterator code I saw is how to delete irrelevant code without breaking it. My use case is very restricted and handles hstore-like jsonb's.

I don't need or want the code to have the ability to descend into nested objects or handle arrays etc., as they are invalid input in my case.

I thought the pointer math example I found is easier to adapt, but I couldn't get a valid "JsonbPair" from the input parameter to feed into the pointer math.

On Tue, Mar 19, 2019 at 9:50 AM Michel Pelletier <pelletier.michel@gmail.com> wrote:

Yeah I'm not sure why you're looping using pointer math, the iterators are there to provide that service.  Another function to check out 'jsonb_each', other than the set returning function parts, it does what it looks like your are trying to do.

-Michel

On Mon, Mar 18, 2019 at 4:12 PM Andrew Gierth <andrew@tao11.riddles.org.uk> wrote:

>>>>> "T" == T L <tinlyx@gmail.com> writes:

 T> Below is my test. It prints a strange character instead of "a"; and
 T> says that the value isn't numeric.

Yeah, there's plenty else wrong with your code.

Did you look at how JsonbToCStringWorker does it? that looks like the
best example I can find on a quick scan.

--
Andrew (irc:RhodiumToad)

For server crashes you can't beat gdb in my opinion.  It's a challenge but worth it in the long run to have gdb skills if you're coding in C (or Python, since pdb shares many of gdb's keybindings).

But just looking at the function I don't see what's immediately wrong, what's your CREATE FUNCTION statement look like?

On Tue, Mar 19, 2019 at 1:08 PM T L <tinlyx@gmail.com> wrote:

Thanks a lot for the hint. I've used the iteration style and cleaned up the code as far as I can.

It now correctly prints the keys and values, but the server crashes near function return.

Any suggestions?

-- function code --

PG_FUNCTION_INFO_V1(print_kv_pair);
Datum
print_kv_pair(PG_FUNCTION_ARGS)
{
    //1. extracting JsonbValue
    Jsonb *jb = PG_GETARG_JSONB_P(0);
    JsonbIterator *it;
    JsonbValue    v;
    JsonbIteratorToken r;
    JsonbParseState *state = NULL;

    if (jb == NULL)
        PG_RETURN_BOOL(false);

    if (!JB_ROOT_IS_OBJECT(jb))
        ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("Can only take objects")));

    it = JsonbIteratorInit(&jb->root);
    r = JsonbIteratorNext(&it, &v, false);
    if (r != WJB_BEGIN_OBJECT)
        ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("Iterator was not an object")));

    //2. iterating through key-value pairs
    char *buf;
    while ((r = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
    {
        switch (r) {
        case WJB_KEY:
            buf = pnstrdup(v.val.string.val, v.val.string.len);
            elog(NOTICE, "print_kv_pair(): k = %s", buf);  //debug
            break;
        case WJB_VALUE:
            if (v.type != jbvNumeric) {
                ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("value must be numeric")));
            }
            elog(NOTICE, "print_kv_pair(): v = %s", DatumGetCString(DirectFunctionCall1(numeric_out,
                    NumericGetDatum(v.val.numeric))) ); //debug
            break;
        case WJB_END_OBJECT:
            break;
        default:
            elog(ERROR, "invalid JsonbIteratorNext rc: %d", (int ) r);
        }
    }
    elog(NOTICE, "print_kv_pair(): ok4");

    PG_RETURN_BOOL(true);
}

-- output --

=> select print_kv_pair('{"a":1, "b": 2}');
NOTICE:  print_kv_pair(): k = a
NOTICE:  print_kv_pair(): v = 1
NOTICE:  print_kv_pair(): k = b
NOTICE:  print_kv_pair(): v = 2
NOTICE:  print_kv_pair(): ok4
server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!> 

On Tue, Mar 19, 2019 at 2:22 PM Michel Pelletier <pelletier.michel@gmail.com> wrote:

jsonb_each is a wrapper around each_worker_jsonb.  It produces a row for every key/value pair in an object.

https://doxygen.postgresql.org/jsonfuncs_8c.html#a7511a3aa3918eb956f3f4211d07bdbb0

the iteration is:

  while ((r = JsonbIteratorNext(&it, &v, skipNested)) != WJB_DONE)

On Tue, Mar 19, 2019 at 11:20 AM T L <tinlyx@gmail.com> wrote:

I need this in my C code on the server side. Any link to the `jsonb_each` for this? Examples I found in a quick search are on the client side in SQL.

I am just confused about the various jsonb types and how to effectively extract values and convert between them:

There are Jsonb, JsonbValue (plus the associated JsonbPair ) to begin with. The ` JsonbToCStringWorker ` example that Andrew pointed out uses still another "JsonbContainer" type.

But the type I get from "PG_GETARG_JSONB_P" is Jsonb. And it doesn't fit into " JsonbContainer" or the pointer math about "JsonPair" that I found online.

What I am struggling with adapting some of the iterator code I saw is how to delete irrelevant code without breaking it. My use case is very restricted and handles hstore-like jsonb's.

I don't need or want the code to have the ability to descend into nested objects or handle arrays etc., as they are invalid input in my case.

I thought the pointer math example I found is easier to adapt, but I couldn't get a valid "JsonbPair" from the input parameter to feed into the pointer math.

On Tue, Mar 19, 2019 at 9:50 AM Michel Pelletier <pelletier.michel@gmail.com> wrote:

Yeah I'm not sure why you're looping using pointer math, the iterators are there to provide that service.  Another function to check out 'jsonb_each', other than the set returning function parts, it does what it looks like your are trying to do.

-Michel

On Mon, Mar 18, 2019 at 4:12 PM Andrew Gierth <andrew@tao11.riddles.org.uk> wrote:

>>>>> "T" == T L <tinlyx@gmail.com> writes:

 T> Below is my test. It prints a strange character instead of "a"; and
 T> says that the value isn't numeric.

Yeah, there's plenty else wrong with your code.

Did you look at how JsonbToCStringWorker does it? that looks like the
best example I can find on a quick scan.

--
Andrew (irc:RhodiumToad)

Yeah I'm not sure why you're looping using pointer math, the iterators are there to provide that service.  Another function to check out 'jsonb_each', other than the set returning function parts, it does what it looks like your are trying to do.

-Michel

On Mon, Mar 18, 2019 at 4:12 PM Andrew Gierth <andrew@tao11.riddles.org.uk> wrote:

>>>>> "T" == T L <tinlyx@gmail.com> writes:

 T> Below is my test. It prints a strange character instead of "a"; and
 T> says that the value isn't numeric.

Yeah, there's plenty else wrong with your code.

Did you look at how JsonbToCStringWorker does it? that looks like the
best example I can find on a quick scan.

--
Andrew (irc:RhodiumToad)