Thread: adding import in pl/python function
Hi,
I'm wondering if it would be OK to change the procedure code before execution. I'm thinking about adding magically an import at the beginning of a function.
Currently numeric arguments passed to the procedure are converted into floats. This is not good, as it causes loss of information.
The proposed solution in code comment is "maybe use a string?".
I'm thinking about something else. We could convert it into Decimal (http://docs.python.org/2/library/decimal.html) class in Python. Unfortunately this class requires import like `from decimal import Decimal` from a standard Python library.
I'm wondering if it would be a good idea to do it like this. It shouldn't fail even with the trusted version of pl/python, as I'd rather see the trusted version to allow importing packages from standard library.
regards,
Szymon
On Fri, May 24, 2013 at 4:10 PM, Szymon Guz <mabewlun@gmail.com> wrote: > > I'm thinking about something else. We could convert it into Decimal > (http://docs.python.org/2/library/decimal.html) class in Python. > Unfortunately this class requires import like `from decimal import Decimal` > from a standard Python library. > > I'm wondering if it would be a good idea to do it like this. It shouldn't > fail even with the trusted version of pl/python, as I'd rather see the > trusted version to allow importing packages from standard library. Why would passing a decimal require an import? The extension itself needs a reference to Decimal, to build them, but the procedure's context doesn't need to have it.
<div dir="ltr"><br /><div class="gmail_extra"><div class="gmail_quote">On 24 May 2013 21:14, Claudio Freire <span dir="ltr"><<ahref="mailto:klaussfreire@gmail.com" target="_blank">klaussfreire@gmail.com</a>></span> wrote:<br /><blockquoteclass="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">OnFri, May 24, 2013 at 4:10 PM, Szymon Guz <<a href="mailto:mabewlun@gmail.com">mabewlun@gmail.com</a>>wrote:<br /> ><br /> > I'm thinking about something else.We could convert it into Decimal<br /> > (<a href="http://docs.python.org/2/library/decimal.html" target="_blank">http://docs.python.org/2/library/decimal.html</a>)class in Python.<br /> > Unfortunately this class requiresimport like `from decimal import Decimal`<br /> > from a standard Python library.<br /> ><br /> > I'm wonderingif it would be a good idea to do it like this. It shouldn't<br /> > fail even with the trusted version of pl/python,as I'd rather see the<br /> > trusted version to allow importing packages from standard library.<br /><br /><br/></div>Why would passing a decimal require an import?<br /><br /> The extension itself needs a reference to Decimal,to build them, but<br /> the procedure's context doesn't need to have it.<br /></blockquote></div><br /></div><divclass="gmail_extra" style="style">Hm... maybe you're right. I think I don't understand fully how the proceduresare executed, and I need to read more to get it.</div><div class="gmail_extra" style="style"><br /></div><div class="gmail_extra"style="style"><br /></div><div class="gmail_extra" style="style">thanks,</div><div class="gmail_extra"style="style">Szymon</div></div>
On Fri, May 24, 2013 at 4:22 PM, Szymon Guz <mabewlun@gmail.com> wrote:
> Hm... maybe you're right. I think I don't understand fully how the
> procedures are executed, and I need to read more to get it.
Well, it's easy.
Instead of PLyFloat_FromNumeric[0], you can make a
PLyDecimal_FromNumeric. There, you'd do with the Python/C[1]:
PyObject *decimal = PyImport_Import("decimal");
PyObject *decimal_dict = PyModule_GetDict(decimal);
PyObject *decimal_ctor = PyDict_GetItemString(decimal_dict, "Decimal");
And invoke it with a string rep of your Numeric:
PyObject *value = PyObject_CallFunction(decimal_ctor, "S", string_value);
Add of course all kinds of error checking and reference count boiler
plate, and you'd have a very dumb version of it.
To make it more "pro", you'd want to do all that stuff to get
decimal_ctor only at initialization time. Especially since you don't
want to fumble with the import lock right there in _FromNumeric.
And to make it totally "pro", you can even freeze Decimal (using
pyfreeze) if you'd like. I would only do this in contexts where you
don't have a stdlib of course. Not sure whether windows falls into
that category. Linux doesn't.
[0] http://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/pl/plpython/plpy_typeio.c#l518
[1] http://docs.python.org/2/c-api/import.html
<div dir="ltr"><br /><div class="gmail_extra"><div class="gmail_quote">On 24 May 2013 21:46, Claudio Freire <span
dir="ltr"><<ahref="mailto:klaussfreire@gmail.com" target="_blank">klaussfreire@gmail.com</a>></span> wrote:<br
/><blockquoteclass="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div
class="im">OnFri, May 24, 2013 at 4:22 PM, Szymon Guz <<a
href="mailto:mabewlun@gmail.com">mabewlun@gmail.com</a>>wrote:<br /> > Hm... maybe you're right. I think I don't
understandfully how the<br /> > procedures are executed, and I need to read more to get it.<br /><br /><br
/></div>Well,it's easy.<br /><br /> Instead of PLyFloat_FromNumeric[0], you can make a<br /> PLyDecimal_FromNumeric.
There,you'd do with the Python/C[1]:<br /><br /> PyObject *decimal = PyImport_Import("decimal");<br /> PyObject
*decimal_dict= PyModule_GetDict(decimal);<br /> PyObject *decimal_ctor = PyDict_GetItemString(decimal_dict,
"Decimal");<br/><br /> And invoke it with a string rep of your Numeric:<br /><br /> PyObject *value =
PyObject_CallFunction(decimal_ctor,"S", string_value);<br /><br /> Add of course all kinds of error checking and
referencecount boiler<br /> plate, and you'd have a very dumb version of it.<br /><br /> To make it more "pro", you'd
wantto do all that stuff to get<br /> decimal_ctor only at initialization time. Especially since you don't<br /> want
tofumble with the import lock right there in _FromNumeric.<br /><br /> And to make it totally "pro", you can even
freezeDecimal (using<br /> pyfreeze) if you'd like. I would only do this in contexts where you<br /> don't have a
stdlibof course. Not sure whether windows falls into<br /> that category. Linux doesn't.<br /><br /><br /> [0] <a
href="http://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/pl/plpython/plpy_typeio.c#l518"
target="_blank">http://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/pl/plpython/plpy_typeio.c#l518</a><br/>
[1]<a href="http://docs.python.org/2/c-api/import.html"
target="_blank">http://docs.python.org/2/c-api/import.html</a><br/></blockquote></div><br /></div><div
class="gmail_extra"style="style"><br /></div><div class="gmail_extra" style="style">Thanks, I will take a look at this,
lookspretty easy. However testing on Windows will be pretty funny :)</div><div class="gmail_extra" style="style"><br
/></div><divclass="gmail_extra" style="style">thanks,</div><div class="gmail_extra" style="style">Szymon</div></div>
On Fri, 2013-05-24 at 16:46 -0300, Claudio Freire wrote: > Well, it's easy. > > Instead of PLyFloat_FromNumeric[0], you can make a > PLyDecimal_FromNumeric. Please send a patch. This would be a welcome addition.
On Mon, May 27, 2013 at 8:13 PM, Peter Eisentraut <peter_e@gmx.net> wrote: > On Fri, 2013-05-24 at 16:46 -0300, Claudio Freire wrote: >> Well, it's easy. >> >> Instead of PLyFloat_FromNumeric[0], you can make a >> PLyDecimal_FromNumeric. > > Please send a patch. This would be a welcome addition. I can write it blind as I have more than enough experience with CPython, but I don't use PLPython so I can't perform extensive testing. If someone's willing to do the testing, by all means.
On Mon, 2013-05-27 at 20:43 -0300, Claudio Freire wrote: > On Mon, May 27, 2013 at 8:13 PM, Peter Eisentraut <peter_e@gmx.net> wrote: > > On Fri, 2013-05-24 at 16:46 -0300, Claudio Freire wrote: > >> Well, it's easy. > >> > >> Instead of PLyFloat_FromNumeric[0], you can make a > >> PLyDecimal_FromNumeric. > > > > Please send a patch. This would be a welcome addition. > > > I can write it blind as I have more than enough experience with > CPython, but I don't use PLPython so I can't perform extensive > testing. > If someone's willing to do the testing, by all means. Yes please.
<div dir="ltr">On 28 May 2013 01:55, Peter Eisentraut <span dir="ltr"><<a href="mailto:peter_e@gmx.net" target="_blank">peter_e@gmx.net</a>></span>wrote:<br /><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote"style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">OnMon, 2013-05-27 at 20:43 -0300, Claudio Freire wrote:<br /> > On Mon, May 27, 2013 at 8:13 PM, Peter Eisentraut<<a href="mailto:peter_e@gmx.net">peter_e@gmx.net</a>> wrote:<br /> > > On Fri, 2013-05-24 at 16:46-0300, Claudio Freire wrote:<br /> > >> Well, it's easy.<br /> > >><br /> > >> Instead ofPLyFloat_FromNumeric[0], you can make a<br /> > >> PLyDecimal_FromNumeric.<br /> > ><br /> > > Pleasesend a patch. This would be a welcome addition.<br /> ><br /> ><br /> > I can write it blind as I have morethan enough experience with<br /> > CPython, but I don't use PLPython so I can't perform extensive<br /> > testing.<br/> > If someone's willing to do the testing, by all means.<br /><br /></div></div>Yes please.<br /><br /></blockquote></div><br/></div><div class="gmail_extra" style="style">I'm working on that.</div><div class="gmail_extra"style="style"><br /></div><div class="gmail_extra" style="style">- Szymon</div></div>