Re: float output precision questions - Mailing list pgsql-hackers

From Tom Lane
Subject Re: float output precision questions
Date
Msg-id 8166.1036087101@sss.pgh.pa.us
Whole thread Raw
In response to Re: float output precision questions  ("Pedro M. Ferreira" <pfrazao@ualg.pt>)
Responses Re: float output precision questions  (Bruno Wolff III <bruno@wolff.to>)
Re: float output precision questions  (Bruce Momjian <pgman@candle.pha.pa.us>)
List pgsql-hackers
"Pedro M. Ferreira" <pfrazao@ualg.pt> writes:
> Have two parameters, say DOUBLE_OUTPUT and EXTRA_DIGITS. DOUBLE_OUTPUT 
> would select from decimal output or normalized output. EXTRA_DIGITS 
> would add the required extra digits, from 0 (default) to 3,  when output 
> is decimal.

I'm not happy with adding the hex-output option, since it's not
very portable and doesn't seem necessary to solve the problem anyway.

But I think an EXTRA_DIGITS setting might be interesting.  In
particular, suppose we allowed EXTRA_DIGITS to be negative?  Setting
it to -1 or -2 would go a long way towards eliminating our problems
with platform variations in the geometry regression test.

Perhaps something like

extra_float_digits    int    range -2 to 2, default 0

extra_float_digits adjusts the number of digits displayed for float4 and
float8 output; the base value of 0 means we output FLT_DIG or DBL_DIG
digits respectively.

Per discussion, there's no reason to allow a value greater than 2, but
I'm not as sure what the lower limit should be --- maybe there's some
use in setting it less than -2?
        regards, tom lane


pgsql-hackers by date:

Previous
From: Rod Taylor
Date:
Subject: Re: 7.2.3 vacuum bug
Next
From: "Pedro M. Ferreira"
Date:
Subject: Re: float output precision questions