Re: Numeric x^y for negative x - Mailing list pgsql-hackers

From Dean Rasheed
Subject Re: Numeric x^y for negative x
Date
Msg-id CAEZATCX-giGez-yYFUbBAkKhUOOG+rAQuTbnXkemJma8gouWSQ@mail.gmail.com
Whole thread Raw
In response to Re: Numeric x^y for negative x  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Numeric x^y for negative x  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
On Thu, 5 Aug 2021 at 17:04, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> It looks like castoroides is not happy with this patch:
>
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=castoroides&dt=2021-08-01%2008%3A52%3A43
>
> Maybe there's some hidden C99 dependency in what you did?
> Although pademelon, which is one of our other pre-C99
> dinosaurs, doesn't seem to be unhappy.
>

Hmm, there's something very weird going on there. The 0.9999999999 ^
70000000000000 test, for example, is one that would have thrown an
overflow error before, but it's not doing that.

So somehow, when it hits the overflow/underflow test, Abs(val) is not
greater than NUMERIC_MAX_RESULT_SCALE * 3.01, which is 6020. The thing
is, when I step through it, I get val = -7000, which should trigger
that comfortably. Even if I play with the return value from
estimate_ln_dweight(), which relies on some double precision
arithmetic, making it -11 or -9 instead of -10, I still get val =
-7000. And even if I force val to be -6000, or even val = 0, so that
it doesn't trigger the overflow/underflow test, it still returns zero
in the end. The end result in this case just isn't very sensitive to
changes in these values.

So I'm wondering if it's somehow not even getting that far. Maybe if
the earlier test to see if exp can be represented as an integer is
failing, it might be going through power_var_int() instead, which
would explain it returning a non-zero value. That hypothesis would be
easy to test, by changing the test to 0.9999999999 ^ 70000000000000.5.

In any case, it would be interesting to see what castoroides returns for

select 0.9999999999 ^ 70000000000000;

and

select 0.9999999999 ^ 70000000000000.5;

Regards,
Dean



pgsql-hackers by date:

Previous
From: Pavel Stehule
Date:
Subject: Re: very long record lines in expanded psql output
Next
From: Platon Pronko
Date:
Subject: Re: very long record lines in expanded psql output