Re: BUG #19340: Wrong result from CORR() function - Mailing list pgsql-bugs

From Dean Rasheed
Subject Re: BUG #19340: Wrong result from CORR() function
Date
Msg-id CAEZATCUo6skdTfRP1XM1sbMveRo1PgpjzOMZEps69Bbz6-OnKA@mail.gmail.com
Whole thread Raw
In response to Re: BUG #19340: Wrong result from CORR() function  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: BUG #19340: Wrong result from CORR() function
List pgsql-bugs
On Tue, 2 Dec 2025 at 20:26, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> I experimented with the attached patch, which is very incomplete;
> I just carried it far enough to be able to run performance checks on
> the modified code, and so all the binary statistics aggregates except
> corr() are broken.  I observe about 2% slowdown on this test case:

I played around with having just 2 extra array elements, constX and
constY equal to the common value if all the values are the same, and
NaN otherwise. For me, that was slightly faster, which I put down to
floating point comparison being faster than converting back and forth
between floating point and integer. Either way, it seems like a
difference that no one is likely to notice.

Doing it that way does lead to one difference though: all-NaN inputs
leads to a NaN result, whereas your patch produces NULL for that case.

Regards,
Dean



pgsql-bugs by date:

Previous
From: Laurenz Albe
Date:
Subject: Re: BUG #19341: REPLACE() fails to match final character when using nondeterministic ICU collation
Next
From: Tom Lane
Date:
Subject: Re: BUG #19340: Wrong result from CORR() function