Thread: Bizarre problem: Python stored procedure using protocol buffers not working

I have a table containing a protocol buffer (pb) attribute, and a
stored procedure foo that tries to parse it and return the parsed
value as a human-friendly string (not actual application code, just a
minimal test case to demonstrate the problem). But running foo gives
me nothing back.

yang=# \d qapb
     Table "public.qapb"
 Column |  Type   | Modifiers
--------+---------+-----------
 id     | integer |
 pb     | bytea   |

yang=# select * from qapb;
 id |                                   pb
----+------------------------------------------------------------------------
  0 | \012\006hello?\020\000\030\000 \000*\014\012\006hello!\020\000\030\000
(1 row)

yang=# create or replace function foo() returns text as $$
import sys, plpy
if '/home/yang/work/pod/' not in sys.path:
sys.path.append('/home/yang/work/pod/')
from my_pb2 import * # this is a protobuf generated module
rv = plpy.execute('select * from qapb')
q = Q()
q.ParseFromString(str(rv[0]['pb']))
return str(q);
$$ language plpythonu;
CREATE FUNCTION
yang=# select foo();
 foo
-----

(1 row)

From a Python prompt, this works fine. I also verified that parsing
works fine from a Python client that first pulls the blob out and
parses client-side.

>>> from my_pb2 import *
>>> q=Q()
>>> q.ParseFromString('\012\006hello?\020\000\030\000 \000*\014\012\006hello!\020\000\030\000')
>>> print q
a: "hello?"
b: 0
c: 0
d: 0
e {
  a: "hello!"
  b: 0
  c: 0
}

Any hints on how I can get to the bottom of this? Thanks in advance.
--
Yang Zhang
http://yz.mit.edu/

On lör, 2010-05-15 at 15:40 -0700, Yang Zhang wrote:
> yang=# select * from qapb;
>  id |                                   pb
> ----+------------------------------------------------------------------------
>   0 | \012\006hello?\020\000\030\000 \000*\014\012\006hello!\020\000\030\000
> (1 row)
>
> yang=# create or replace function foo() returns text as $$
> import sys, plpy
> if '/home/yang/work/pod/' not in sys.path:
> sys.path.append('/home/yang/work/pod/')
> from my_pb2 import * # this is a protobuf generated module
> rv = plpy.execute('select * from qapb')
> q = Q()
> q.ParseFromString(str(rv[0]['pb']))
> return str(q);
> $$ language plpythonu;
> CREATE FUNCTION
> yang=# select foo();
>  foo
> -----
>
> (1 row)

You have null bytes in the data value, which is not supported very well
in PL/Python.  Try the 9.0 beta version; it should be fixed there.

On Sat, May 15, 2010 at 10:20 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
> You have null bytes in the data value, which is not supported very well
> in PL/Python.  Try the 9.0 beta version; it should be fixed there.

Thanks. Out of curiosity, is this an issue just with PL/Python or with
other stored procedure languages as well?
--
Yang Zhang
http://yz.mit.edu/

On lör, 2010-05-15 at 22:50 -0700, Yang Zhang wrote:
> On Sat, May 15, 2010 at 10:20 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
> > You have null bytes in the data value, which is not supported very well
> > in PL/Python.  Try the 9.0 beta version; it should be fixed there.
>
> Thanks. Out of curiosity, is this an issue just with PL/Python or with
> other stored procedure languages as well?

It's an artifact of the PL/Python implementation.