Re: Some encoding trouble via libpq - Mailing list pgsql-general

Ahhh, I get it!  Thanks for pointing this out.  It ended up not being the problem I was running into, but eventually it would have snagged me (solution to original problem below).  I spent some time going over that bit of code and what you said with a coworker, and we ended up taking care of it.  So many little gotchas when doing this stuff!  Here's the snippet as of now for anyone interested:

    int size = STDIN_BLOCK + 1;
    char *buffer = (char *) xmalloc(size);
    int offset = 0;
    int read = 1;
   
    while ( (read > 0) && (offset <= STDIN_MAX) )
    {
        if (offset + STDIN_BLOCK >= size)
        {
            size += STDIN_BLOCK;
            buffer = xrealloc (buffer, size);
        }
        syslog (LOG_DEBUG, "Reading a block...");
        read = fread (buffer + offset, 1, STDIN_BLOCK, stdin);
        offset += read;
    } // while
   
    // null terminate the string...
    memset(buffer + offset + 1, '\0', 1);

Also, an update on the original problem:  It turned out with some experimenting that the problem I experienced had nothing to do with the standard input reading routine, or postgresql itself, but rather how I was re-inserting into one query data I had gotten from a previous query.  And this is maybe useful for others on the list:

I was doing a select on one table to get a foreign key id that I would use subsequently in an insert statement, like thus:

result = PQexecParams(conn,
                            "SELECT id FROM events WHERE ping_id = $1 AND serial = $2",
                            2,
                            NULL, // backend figures out type itself
                            paramValues,
                            NULL, // apparently we don't need param lengths
                            NULL, // all text params
                            0 // we don't want binary results, no
                        );
... a bunch of tuples checking, then...
event_id = PQgetvalue(result, 0, 0);  // <-- BAD GUY!
PQclear(result);  // <-- ACCOMPLICE!
... then the insert....
    paramValues[0] = ping_id;
    paramValues[1] = event_id;
    paramValues[2] = message; // get from std input! how do we do that again???
    result = PQexecParams(conn,
                            "INSERT INTO event_changes (ping_id, event_id, created_at, message) VALUES ($1, $2, NOW(), $3)",
                            3,
                            NULL, // backend figures out type itself
                            paramValues,
                            NULL, // apparently we don't need param lengths
                            NULL, // all text params
                            0 // we don't want binary results, no
                        );

This is what was causing our woes.  event_id is merely a pointer to data in result.  When you PQclear the result, it's gone!  Sort of...  Mac OS X was forgiving on some level in that the data we were pointing to was still there.  But since different operating systems manage their memory differently, Linux wasn't having any of it, it was basically pointing to garbage!

The solution was to not clear result until later, or to copy the data in allocated memory, then clear the result, which we did thusly:

    tmp_str = PQgetvalue(result, 0, 0);
    tmp_str_len = strlen(tmp_str);
    event_id = xmalloc(tmp_str_len + 1);
    strncpy(event_id, tmp_str, tmp_str_len);

Until next time,
Billy