Re: BUG #19340: Wrong result from CORR() function - Mailing list pgsql-bugs

From Tom Lane
Subject Re: BUG #19340: Wrong result from CORR() function
Date
Msg-id 513345.1764717860@sss.pgh.pa.us
Whole thread Raw
In response to Re: BUG #19340: Wrong result from CORR() function  (Dean Rasheed <dean.a.rasheed@gmail.com>)
Responses Re: BUG #19340: Wrong result from CORR() function
Re: BUG #19340: Wrong result from CORR() function
List pgsql-bugs
Dean Rasheed <dean.a.rasheed@gmail.com> writes:
> I played around with having just 2 extra array elements, constX and
> constY equal to the common value if all the values are the same, and
> NaN otherwise.

Hmm.

> Doing it that way does lead to one difference though: all-NaN inputs
> leads to a NaN result, whereas your patch produces NULL for that case.

Yeah, I did it as I did precisely because I wanted all-NaN-input to be
seen as a constant.  But you could make an argument that NaN is not
really a fixed value but has more kinship to the "we don't know what
the value is" interpretation of SQL NULL.  In that case your proposal
is semantically reasonable on the grounds that maybe the NaNs aren't
really all equal, and I agree it ought to be a little faster than
mine.

            regards, tom lane



pgsql-bugs by date:

Previous
From: Dean Rasheed
Date:
Subject: Re: BUG #19340: Wrong result from CORR() function
Next
From: Dean Rasheed
Date:
Subject: Re: BUG #19340: Wrong result from CORR() function