Re: avg() with floating-point types - Mailing list pgsql-sql

From Tom Lane
Subject Re: avg() with floating-point types
Date
Msg-id 26697.1136165150@sss.pgh.pa.us
Whole thread Raw
In response to avg() with floating-point types  ("George Pavlov" <gpavlov@mynewplace.com>)
List pgsql-sql
"George Pavlov" <gpavlov@mynewplace.com> writes:
> The datatype of both city.latitude and postal_code.latitude is
> number(16,12).
> This works, but I would like to understand why there is sometimes a
> discrepancy between avg(pc.latitude) and what actually gets inserted
> into the city table -- is it the usual floating-point discrepancy or is
> there something I can do about it? E.g. after the above update:

You're forcing the result of the avg() calculation to be rounded to 12
digits when you store it into city.latitude.  Your example simply shows
that the avg() calculation is being done to more precision than that.
        regards, tom lane


pgsql-sql by date:

Previous
From: "George Pavlov"
Date:
Subject: avg() with floating-point types
Next
From: Michael Fuhr
Date:
Subject: Re: avg() with floating-point types