Thread: PostgreSQL 9.3 Segfault in "hstore_to_json" function
Good afternoon!
Have found an occurrence of a segfault within "vanilla" PostgreSQL 9.3. Fresh install, only change is having the hstore extension enabled.
Issue occurs when using the `hstore_to_json` function. Data was inserted using SQLAlchemy but can be reproduced when using a COPY statement (see "reproduce.sh"). Was not able to reproduce when running an INSERT statement from the psql client (as it escaped the data differently and then did not throw the segfault). If I copy / paste the data into a Python console and print it it shows up correctly so I'm pretty sure this is not an encoding issue.
There are two strange things about this bug. First, it only seems to happen when `hstore_to_json` is called twice during the same session (compare the last two test cases in the included "reproduce.sh" script).
Second, it only occurs when there is another key / value pair present in the hstore column, and this key must be of a certain length to trigger the issue (from what I can tell, 8 characters). The second test case illustrates that it works with shorter length keys.
To reproduce, adjust the `USER` and `DATABASE` variables in "reproduce.sh" to account for your local username / database. I needed to do it like this due to the aforementioned issues with reproducing it using an INSERT statement.
Included files:
"reproduce.sh" - Shell script for reproducing the issue. Runs through multiple test cases to illustrate different scenarios that work before providing a breaking example.
"log_output.txt" - Relevant log lines when running with DEBUG5 logging level (note line 11)
Version / encoding information is contained in "reproduce.sh"
Cheers!
Attachment
Saul Costa <saul@mixrank.com> writes:
> Have found an occurrence of a segfault within "vanilla" PostgreSQL 9.3.
> Fresh install, only change is having the hstore extension enabled.
> Issue occurs when using the `hstore_to_json` function. Data was inserted
> using SQLAlchemy but can be reproduced when using a COPY statement (see
> "reproduce.sh"). Was not able to reproduce when running an INSERT statement
> from the psql client (as it escaped the data differently and then did not
> throw the segfault). If I copy / paste the data into a Python console and
> print it it shows up correctly so I'm pretty sure this is not an encoding
> issue.
It looks like the problem is that 9.3's version of hstore_to_json() embeds
an assumption that escape_json() can't increase the length of a string
more than 2x. This is false for ASCII control characters, which get
converted to "\uNNNN", so your test data leads to a buffer overrun that
might or might not chance to stomp on anything critical.
I see that's been fixed in 9.4 and HEAD, but it should've been
back-patched to 9.3.
regards, tom lane