Re: floating point representation - Mailing list pgsql-hackers

From Tom Lane
Subject Re: floating point representation
Date
Msg-id 25005.982598312@sss.pgh.pa.us
Whole thread Raw
In response to Re: floating point representation  (Bruce Momjian <pgman@candle.pha.pa.us>)
List pgsql-hackers
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Added to TODO:
>     * Add SET FLOAT4_PRECISION and SET FLOAT8_PRECISION using printf args

foo_PRECISION is not the right name if these variables will contain
printf format strings.  Perhaps foo_FORMAT?  Anyone have a better idea?

After further thought I think that we ought to standardize on %.6g and
%.15g even if the local <float.h> offers slightly different values of
FLT_DIG and DBL_DIG.  IEEE or near-IEEE float math is so close to
universal that I don't think it's worth worrying about the possibility
that different precisions would be more appropriate for some platforms.
Furthermore, having cross-platform consistency of display format seems
more useful than not.

Something else we should perhaps think about, though we are very late
in beta: once these variables exist, we could have the geometry regress
test set them to suppress a couple of digits, and eliminate most if not
all of the need for platform-specific geometry results.  Doing this
would be a no-brainer at any other time in the development cycle, but
right now I am worried about whether we'd be able to reconfirm regress
results on all the currently-supported platforms before release.
Comments?
        regards, tom lane


pgsql-hackers by date:

Previous
From: Zeugswetter Andreas SB
Date:
Subject: AW: AW: Re: beta5 ...
Next
From: Tom Lane
Date:
Subject: Re: AW: Re: beta5 ...