Re: Greatest Common Divisor - Mailing list pgsql-hackers

From Vik Fearing
Subject Re: Greatest Common Divisor
Date
Msg-id 127babac-d457-07c6-2251-f607ea9442ee@2ndquadrant.com
Whole thread Raw
In response to Re: Greatest Common Divisor  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Greatest Common Divisor  (Tom Lane <tgl@sss.pgh.pa.us>)
Re: Greatest Common Divisor  (Vik Fearing <vik.fearing@2ndquadrant.com>)
List pgsql-hackers
On 04/01/2020 01:21, Tom Lane wrote:
> Vik Fearing <vik.fearing@2ndquadrant.com> writes:
>> On 03/01/2020 20:14, Fabien COELHO wrote:
>>> I'm unsure about gcd(INT_MIN, 0) should error. Possibly 0 would be nicer?
>> What justification for that do you have?
> Zero is the "correct" answer for that, isn't it, independently of overflow
> considerations?  


I would say not.  The correct answer is INT_MIN but we've decided a
negative result is not desirable.


> We should strive to give the correct answer if it's known
> and representable, rather than have arbitrary failure conditions.


On that we fully agree.


> (IOW, we should throw errors only when the *result* is out of range
> or undefined, not just because the input is an edge case.)


That's what I do with the rest of it.  INT_MIN is only an error if the
result of the calculation is also INT_MIN.

-- 

Vik Fearing




pgsql-hackers by date:

Previous
From: Vik Fearing
Date:
Subject: Re: Greatest Common Divisor
Next
From: Tom Lane
Date:
Subject: Re: Greatest Common Divisor