Lew wrote:
> Postgres User wrote:
>> The problem turned out to be related to my function..
>>
>> Given this table:
>>
>> CREATE TABLE "table2" (
>> "s_val" numeric(6,2),
>> "e_val" numeric(6,2)
>> ) WITH OIDS;
>>
>> The following functions of code will set retval = NULL;
>>
>> declare
>> retval numeric(6,2);
>> rec record;
>> begin
>> SELECT * INTO rec FROM table2 LIMIT 0;
>> rec.s_val = 100;
>> rec.e_val = 101;
>> retval = (rec.s_val - rec.e_val) / rec.s_val;
>>
>> return retval;
>> end
>>
>> However, if I explicitly typecast, then it returns the proper value:
>> retval = (rec.s_val::numeric(6,2) - rec.e_val::numeric(6,2)) /
>> rec.s_val::numeric(6,2);
>
> Sure, because the first way you're doing integer division, and the
> second way you're doing floating point division. In integer division,
> -1/100 yields zero.
The more I look at this, the more I think I'm wrong.
I'm researching the semantics of the idioms that you used. I don't know what
type rec.s_val and rec.e_val end up being after the integer assignments.
--
Lew
