Re: Inconsistent results for division and multiplication operations - Mailing list pgsql-sql

From Tom Lane
Subject Re: Inconsistent results for division and multiplication operations
Date
Msg-id 1352240.1732552478@sss.pgh.pa.us
Whole thread Raw
In response to Re: Inconsistent results for division and multiplication operations  (Erik Brandsberg <erik@heimdalldata.com>)
List pgsql-sql
Erik Brandsberg <erik@heimdalldata.com> writes:
> This is a common issue with using floating point math.  You will see the
> same issue with many systems.  Basically, the order of operations can
> trigger very minor differences in results, but if you round the first
> result to the same number of significant digits as the input, it would be
> identical.
> https://learn.microsoft.com/en-us/office/troubleshoot/access/floating-calculations-info

Yeah.  The OP is actually working with PG's "numeric" type, not
floating-point, but the principle is the same.  Some division
results can't be represented exactly in any finite number of
digits, so you get roundoff error.

            regards, tom lane



pgsql-sql by date:

Previous
From: Philip Semanchuk
Date:
Subject: Re: Inconsistent results for division and multiplication operations
Next
From: Martin Norbäck Olivers
Date:
Subject: Re: Inconsistent results for division and multiplication operations