Re: Infinities in type numeric - Mailing list pgsql-hackers

From Dean Rasheed
Subject Re: Infinities in type numeric
Date
Msg-id CAEZATCULz=LNg+jvmEhasLc2stvMeKX4izm9JC=z0xVhvHrA2w@mail.gmail.com
Whole thread Raw
In response to Infinities in type numeric  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Infinities in type numeric
List pgsql-hackers
On Fri, 12 Jun 2020 at 02:16, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> * I had to invent some semantics for non-standardized functions,
> particularly numeric_mod, numeric_gcd, numeric_lcm.  This area
> could use review to be sure that I chose desirable behaviors.
>

I think the semantics you've chosen for numeric_mod() are reasonable,
and I think they're consistent with POSIX fmod().

However, it looks like you've chosen gcd('Inf', x) = x, whereas I'd
say that the result should be 'NaN'.

One way to look at it is that the GCD result should exactly divide
both inputs with no remainder, but the remainder when you divide 'Inf'
by x is undefined, so you can't say that x exactly divides 'Inf'.

Another way to look at it is that gcd('Inf', x) is limit(n -> 'Inf',
gcd(n, x)), but that limit isn't well-defined. For example, suppose
x=10, then gcd('Inf', 10) = limit(n -> 'Inf', gcd(n, 10)), but gcd(n,
10) is either 1,2,5 or 10 depending on n, and it does not converge to
any particular value in the limit n -> 'Inf'.

A third way to look at it would be to apply one round of Euclid's
algorithm to it: gcd('Inf', x) = gcd(x, mod('Inf', x)) = gcd(x, 'NaN')
= 'NaN'.

Now you could argue that x=0 is a special case, and gcd('Inf', 0) =
'Inf' on the grounds that gcd(a, 0) = a for all finite 'a'. However, I
don't think that's particularly useful, and it fails the first test
that the result exactly divides both inputs because mod('Inf', 'Inf')
is undefined ('NaN').

Similar arguments apply to lcm(), so I'd say that both gcd() and lcm()
should return 'NaN' if either input is 'Inf' or 'NaN'.

Regards,
Dean



pgsql-hackers by date:

Previous
From: Daniel Gustafsson
Date:
Subject: Re: [bug?] Is the format of tables in the documentation broken in PG13?
Next
From: Peter Eisentraut
Date:
Subject: Re: SEARCH and CYCLE clauses