Re: patch for getXXX methods - Mailing list pgsql-jdbc

Oliver,

Yes, and this is why I needed to do it for getLong, but I don't think
it's necessary for getInt, getByte, as it is really just to test to see
if it is greater than the max allowed value.

Dave
On Sun, 2004-07-11 at 19:37, Oliver Jowett wrote:
> Dave Cramer wrote:
> > Oliver,
> >
> > I don't believe you will lose precision if the number is below MAX_LONG
> > ? When I tested it on my system, I was able to retrieve a double that
> > was equal to MAX_LONG without losing precision.
>
> The attached testcase says otherwise. It produces this output:
>
> > Mismatch: 9223372036854775806 => 9.223372036854776E18 => 9223372036854775807
> > Mismatch: 9223372036854775805 => 9.223372036854776E18 => 9223372036854775807
> > Mismatch: 9223372036854775804 => 9.223372036854776E18 => 9223372036854775807
> > Mismatch: 9223372036854775803 => 9.223372036854776E18 => 9223372036854775807
> > Mismatch: 9223372036854775802 => 9.223372036854776E18 => 9223372036854775807
> > Mismatch: 9223372036854775801 => 9.223372036854776E18 => 9223372036854775807
> > Mismatch: 9223372036854775800 => 9.223372036854776E18 => 9223372036854775807
> > Mismatch: 9223372036854775799 => 9.223372036854776E18 => 9223372036854775807
> [...]
> > Mismatch: 9223372036854775296 => 9.223372036854776E18 => 9223372036854775807
> > Mismatch: 9223372036854775295 => 9.2233720368547748E18 => 9223372036854774784
> > Mismatch: 9223372036854775294 => 9.2233720368547748E18 => 9223372036854774784
>
> and so on.
>
> The problem is that near MAX_LONG you need almost 64 bits of mantissa to
> exactly represent the value -- but a double is only a 64-bit value
> including space for the exponent. I can't remember the exact split but
> from the above it looks like there is around 10 bits of exponent so you
> only have ~54 bits for the mantissa -- so you only get a precision of
> about +/- 512 when you're dealing with numbers of a magnitude around
> MAX_LONG.
>
> -O
>
>
> !DSPAM:40f1cf8080747915514021!
>
> ______________________________________________________________________
> public class TestDoublePrecision {
>     public static void main(String[] args) {
>     for (long l = Long.MAX_VALUE; l != Long.MIN_VALUE; --l) {
>         double d = (double)l;
>         long check = (long)d;
>
>         if (check != l)
>         System.out.println("Mismatch: " + l + " => " + d + " => " + check);
>     }
>     }
> }
>
>
> !DSPAM:40f1cf8080747915514021!
--
Dave Cramer
519 939 0336
ICQ # 14675561