Re: How does Numeric division determine precision? - Mailing list pgsql-sql

From Tom Lane
Subject Re: How does Numeric division determine precision?
Date
Msg-id 29730.1342114219@sss.pgh.pa.us
Whole thread Raw
In response to How does Numeric division determine precision?  (Will Pugh <willpugh@gmail.com>)
List pgsql-sql
Will Pugh <willpugh@gmail.com> writes:
> It seems that is 9.1, numerics that don't have a specified precision
> and scale are arbitrary scale/precision.
> For many operations this is straightforward.  However, when doing a
> division operation that does not terminate, I'm curious about how the
> number of digits is determined.

According to select_div_scale() in src/backend/utils/adt/numeric.c,
   /*    * The result scale of a division isn't specified in any SQL standard. For    * PostgreSQL we select a result
scalethat will give at least    * NUMERIC_MIN_SIG_DIGITS significant digits, so that numeric gives a    * result no
lessaccurate than float8; but use a scale not less than    * either input's display scale.    */
 

I wouldn't necessarily claim that that couldn't be improved on,
but that's what it does now.
        regards, tom lane


pgsql-sql by date:

Previous
From: Will Pugh
Date:
Subject: How does Numeric division determine precision?
Next
From: Jasen Betts
Date:
Subject: Re: Simple Upgrade from PostgreSQL version 8.1.11 (With schemas)