Thread: Re: [COMMITTERS] pgsql: PL/Python: Convert numeric to Decimal

Re: [COMMITTERS] pgsql: PL/Python: Convert numeric to Decimal

From
Tom Lane
Date:
Peter Eisentraut <peter_e@gmx.net> writes:
> PL/Python: Convert numeric to Decimal

Assorted buildfarm members don't like this patch.
        regards, tom lane



Re: [COMMITTERS] pgsql: PL/Python: Convert numeric to Decimal

From
Claudio Freire
Date:
On Sat, Jul 6, 2013 at 2:39 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Peter Eisentraut <peter_e@gmx.net> writes:
>> PL/Python: Convert numeric to Decimal
>
> Assorted buildfarm members don't like this patch.


Do you have failure details?

This is probably an attempt to operate decimals vs floats.

Ie: Decimal('3.0') > 0 works, but Decimal('3.0') > 1.3 doesn't
(decimal is explicitly forbidden from operating on floats, some design
decision that can only be disabled in 3.3).



Re: [COMMITTERS] pgsql: PL/Python: Convert numeric to Decimal

From
Andrew Dunstan
Date:
On 07/06/2013 01:52 AM, Claudio Freire wrote:
> On Sat, Jul 6, 2013 at 2:39 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Peter Eisentraut <peter_e@gmx.net> writes:
>>> PL/Python: Convert numeric to Decimal
>> Assorted buildfarm members don't like this patch.
>
> Do you have failure details?
>
> This is probably an attempt to operate decimals vs floats.
>
> Ie: Decimal('3.0') > 0 works, but Decimal('3.0') > 1.3 doesn't
> (decimal is explicitly forbidden from operating on floats, some design
> decision that can only be disabled in 3.3).
>
>


Instead of speculating, you can actually see for yourself. The dashboard 
is at <http://www.pgbuildfarm.org/cgi-bin/show_status.pl> Pick one of 
the machines failing at PLCheck-C and click its 'Details' link. Then 
scroll down a bit and you'll see what is failing.

cheers

andrew




Re: [COMMITTERS] pgsql: PL/Python: Convert numeric to Decimal

From
Claudio Freire
Date:
Look at that:
 return x $$ LANGUAGE plpythonu; SELECT * FROM test_type_conversion_numeric(100);
! INFO:  (Decimal('100'), 'Decimal') CONTEXT:  PL/Python function "test_type_conversion_numeric"
test_type_conversion_numeric------------------------------
 
--- 219,225 ---- return x $$ LANGUAGE plpythonu; SELECT * FROM test_type_conversion_numeric(100);
! INFO:  (Decimal("100"), 'Decimal') CONTEXT:  PL/Python function "test_type_conversion_numeric"
test_type_conversion_numeric------------------------------
 

" instead of '

All the more reason to use as_tuple



On Sat, Jul 6, 2013 at 9:16 AM, Andrew Dunstan <andrew@dunslane.net> wrote:
>
> On 07/06/2013 01:52 AM, Claudio Freire wrote:
>>
>> On Sat, Jul 6, 2013 at 2:39 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>>
>>> Peter Eisentraut <peter_e@gmx.net> writes:
>>>>
>>>> PL/Python: Convert numeric to Decimal
>>>
>>> Assorted buildfarm members don't like this patch.
>>
>>
>> Do you have failure details?
>>
>> This is probably an attempt to operate decimals vs floats.
>>
>> Ie: Decimal('3.0') > 0 works, but Decimal('3.0') > 1.3 doesn't
>> (decimal is explicitly forbidden from operating on floats, some design
>> decision that can only be disabled in 3.3).
>>
>>
>
>
> Instead of speculating, you can actually see for yourself. The dashboard is
> at <http://www.pgbuildfarm.org/cgi-bin/show_status.pl> Pick one of the
> machines failing at PLCheck-C and click its 'Details' link. Then scroll down
> a bit and you'll see what is failing.
>
> cheers
>
> andrew
>



Re: [COMMITTERS] pgsql: PL/Python: Convert numeric to Decimal

From
Szymon Guz
Date:



On 6 July 2013 17:58, Claudio Freire <klaussfreire@gmail.com> wrote:
Look at that:

  return x
  $$ LANGUAGE plpythonu;
  SELECT * FROM test_type_conversion_numeric(100);
! INFO:  (Decimal('100'), 'Decimal')
  CONTEXT:  PL/Python function "test_type_conversion_numeric"
   test_type_conversion_numeric
  ------------------------------
--- 219,225 ----
  return x
  $$ LANGUAGE plpythonu;
  SELECT * FROM test_type_conversion_numeric(100);
! INFO:  (Decimal("100"), 'Decimal')
  CONTEXT:  PL/Python function "test_type_conversion_numeric"
   test_type_conversion_numeric
  ------------------------------

" instead of '

All the more reason to use as_tuple



On Sat, Jul 6, 2013 at 9:16 AM, Andrew Dunstan <andrew@dunslane.net> wrote:
>
> On 07/06/2013 01:52 AM, Claudio Freire wrote:
>>
>> On Sat, Jul 6, 2013 at 2:39 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>>
>>> Peter Eisentraut <peter_e@gmx.net> writes:
>>>>
>>>> PL/Python: Convert numeric to Decimal
>>>
>>> Assorted buildfarm members don't like this patch.
>>
>>
>> Do you have failure details?
>>
>> This is probably an attempt to operate decimals vs floats.
>>
>> Ie: Decimal('3.0') > 0 works, but Decimal('3.0') > 1.3 doesn't
>> (decimal is explicitly forbidden from operating on floats, some design
>> decision that can only be disabled in 3.3).
>>
>>
>
>
> Instead of speculating, you can actually see for yourself. The dashboard is
> at <http://www.pgbuildfarm.org/cgi-bin/show_status.pl> Pick one of the
> machines failing at PLCheck-C and click its 'Details' link. Then scroll down
> a bit and you'll see what is failing.
>
> cheers
>
> andrew
>


Hi,
I've modifled the tests to check the numeric->decimal conversion some other way. They check now conversion to float/int and to string, and also tuple values.

I've checked that on decimal and cdecimal on python 2.7 and 3.3. The outputs are the same regardles the Python and decimal versions.

thanks,
Szymon
Attachment

Re: [COMMITTERS] pgsql: PL/Python: Convert numeric to Decimal

From
Peter Eisentraut
Date:
On Sun, 2013-07-07 at 02:01 -0300, Claudio Freire wrote:
> You really want to test more than just the str. The patch contains
> casts to int and float, which is something existing PLPython code will
> be doing, so it's good to test it still can be done with decimal.

Let's remember that we are regression testing PL/Python here, not
Python.  The functionality of PL/Python is to convert a numeric to
Decimal, and that's what we are testing.  What Python can or cannot do
with that is not our job to test.

> Existing python code will also expect the number to be a float, and
> will try to operate against other floats. That's not going to work
> anymore, that has to go into release notes.

Right, this will be listed in the release notes under upgrade caveats.






Re: [COMMITTERS] pgsql: PL/Python: Convert numeric to Decimal

From
Peter Eisentraut
Date:
On Sun, 2013-07-07 at 17:21 +0200, Szymon Guz wrote:
> I think that these tests are much better, so they should go into
> trunk.
> As for Python 2.5 I think we could modify the code and makefile (with
> additional documentation info) so the decimal code wouldn't be
> compiled
> with python 2.5. 

I'd welcome updated tests, if you want to work on that.  But they would
need to work uniformly for Python 2.4 through 3.3.





Re: [COMMITTERS] pgsql: PL/Python: Convert numeric to Decimal

From
Szymon Guz
Date:
On 7 July 2013 21:35, Peter Eisentraut <peter_e@gmx.net> wrote:
On Sun, 2013-07-07 at 17:21 +0200, Szymon Guz wrote:
> I think that these tests are much better, so they should go into
> trunk.
> As for Python 2.5 I think we could modify the code and makefile (with
> additional documentation info) so the decimal code wouldn't be
> compiled
> with python 2.5.

I'd welcome updated tests, if you want to work on that.  But they would
need to work uniformly for Python 2.4 through 3.3.



Well... I don't know what to do and which solution is better.

This patch works, but the tests are not working on some old machines.

This patch works, but changes the plpython functions, so I assume that it will provide errors to some existing functions. I've noticed yesterday that you cannot run code like `Decimal(10) - float(10)`. So if a function accepts a numeric parameter 'x', which currently is converted to float, then the code like `x - float(10)` currently works, and will not work after this change.

Introducing decimal.Decimal also breaks python earlier than 2.4, as the decimal module has been introduced in 2.4. We could use the old conversion for versions before 2.4, and the new for 2.4 and newer. Do we want it to work like this? Do we want to have different behaviour for different python versions? I'm not sure if anyone still uses Python 2.3, but I've already realised that the patch breaks all the functions for 2.3 which use numeric argument.

I assume that the patch will be rolled back, if it the tests don't work on some machines, right?

szymon

Re: [COMMITTERS] pgsql: PL/Python: Convert numeric to Decimal

From
Claudio Freire
Date:
On Sun, Jul 7, 2013 at 4:28 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
> On Sun, 2013-07-07 at 02:01 -0300, Claudio Freire wrote:
>> You really want to test more than just the str. The patch contains
>> casts to int and float, which is something existing PLPython code will
>> be doing, so it's good to test it still can be done with decimal.
>
> Let's remember that we are regression testing PL/Python here, not
> Python.  The functionality of PL/Python is to convert a numeric to
> Decimal, and that's what we are testing.  What Python can or cannot do
> with that is not our job to test.

I was just proposing to test behavior likely to be expected by
PL/Python functions, but you probably know better than I.