Thread: Purpose of rscale/dscale in NUMERIC?

What is the reason for maintaining separate rscale and dscale values in
numeric variables?

I am finding that this arrangement leads to some odd results, for
example this:

regression=# select (exp(ln(2.0)) - 2.0);     ?column?
----------------------0.0000000000000000
(1 row)

regression=# select (exp(ln(2.0)) - 2.0) * 100000;     ?column?
----------------------0.0000000000000010
(1 row)

The difference between rscale and dscale allows some "hidden" digits to
be carried along in an expression result, and then possibly exposed
later.  This seems pretty nonintuitive for an allegedly exact
calculational datatype.  ISTM the policy should be "what you see is what
you get" - no hidden digits.  That would mean there's no need for
separating rscale and dscale, so I'm wondering why they were put in
to begin with.
        regards, tom lane

Tom Lane wrote:
> 
> What is the reason for maintaining separate rscale and dscale values in
> numeric variables?
> 
> I am finding that this arrangement leads to some odd results, for
> example this:
> 
> regression=# select (exp(ln(2.0)) - 2.0);
>       ?column?
> ---------------------
>  -0.0000000000000000
> (1 row)
> 
> regression=# select (exp(ln(2.0)) - 2.0) * 100000;
>       ?column?
> ---------------------
>  -0.0000000000000010
> (1 row)
> 
> The difference between rscale and dscale allows some "hidden" digits to
> be carried along in an expression result, and then possibly exposed
> later.  This seems pretty nonintuitive for an allegedly exact
> calculational datatype.  ISTM the policy should be "what you see is what
> you get" - no hidden digits.  That would mean there's no need for
> separating rscale and dscale, so I'm wondering why they were put in
> to begin with.

You need to carry around a decent number of digits when you divide
already. Exposing them in a manner that numericcol(15,2) / 3.0 all of
the sudden displays 16 or more digits isn't much more intuitive. But
carrying around only 2 here leads to nonintuitively fuzzy results on the
other hand.

It only applies to division and higher functions, and these are not
"exact" if you calculate the result and represent it decimal. They never
have been.

So to answer your question, they are there to make the NUMERIC datatype
useful for non-exact stuff too. You can expect an exact result where an
exact representation in decimal can be expected. Where this is not the
case, you get a good approximation.


Jan

-- 

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #

Jan Wieck <JanWieck@Yahoo.com> writes:
> Tom Lane wrote:
>> What is the reason for maintaining separate rscale and dscale values in
>> numeric variables?

> You need to carry around a decent number of digits when you divide
> already. Exposing them in a manner that numericcol(15,2) / 3.0 all of
> the sudden displays 16 or more digits isn't much more intuitive. But
> carrying around only 2 here leads to nonintuitively fuzzy results on the
> other hand.

Certainly you need extra guard digits while you do the calculation.
What I'm wondering is why the delivered result would have hidden digits
in it.  If they're accurate, why not show them?  If they're not accurate
(which they're not, at least in the case I showed) why is it a good idea
to let them escape?
        regards, tom lane

Tom Lane wrote:
> 
> Jan Wieck <JanWieck@Yahoo.com> writes:
> > Tom Lane wrote:
> >> What is the reason for maintaining separate rscale and dscale values in
> >> numeric variables?
> 
> > You need to carry around a decent number of digits when you divide
> > already. Exposing them in a manner that numericcol(15,2) / 3.0 all of
> > the sudden displays 16 or more digits isn't much more intuitive. But
> > carrying around only 2 here leads to nonintuitively fuzzy results on the
> > other hand.
> 
> Certainly you need extra guard digits while you do the calculation.
> What I'm wondering is why the delivered result would have hidden digits
> in it.  If they're accurate, why not show them?  If they're not accurate
> (which they're not, at least in the case I showed) why is it a good idea
> to let them escape?

So we need them in the calculation, and if it's a nested tree of
function calls, they have to travel around too. What do you think is a
good place to kill these critters then?


Jan

-- 

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #