Re: WIP: rewrite numeric division - Mailing list pgsql-patches

From Tom Lane
Subject Re: WIP: rewrite numeric division
Date
Msg-id 27542.1182263309@sss.pgh.pa.us
Whole thread Raw
In response to Re: WIP: rewrite numeric division  (Gregory Stark <stark@enterprisedb.com>)
List pgsql-patches
Gregory Stark <stark@enterprisedb.com> writes:
> "Tom Lane" <tgl@sss.pgh.pa.us> writes:
>> Yeah, my proposed patch is schoolbook division.  I don't think I'd trust
>> Newton's method to give anything but a close approximation, which is
>> what we have in HEAD already.

> Well unless we start storing rational numbers it'll always be a limited to a
> finite number of decimals. The key is whether we can guarantee a lower bound
> on the number of accurate decimal places. As long as we can then we can keep
> going until the decimal places we're going to return are all accurate.

This is nonsense.  Consider an approximate division that gives

    ...123.999999999999999999...

You can keep generating 9's forever, but you'll never know whether the
accurate result of trunc() should be 123 or 124.  Unless you use a
method that gives you trustworthy digits to start with.

> It looks like multiplication can also generate incorrect results.

It can, but only if told to produce fewer digits than would be in the
exact result, so I'm not worried about that.

            regards, tom lane

pgsql-patches by date:

Previous
From: Magnus Hagander
Date:
Subject: Re: Preliminary GSSAPI Patches
Next
From: Tom Lane
Date:
Subject: Re: [HACKERS] 'Waiting on lock'