Re: Aggregate Function corr does not always return the correct value - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Aggregate Function corr does not always return the correct value
Date
Msg-id 1243664.1756229674@sss.pgh.pa.us
Whole thread Raw
In response to Aggregate Function corr does not always return the correct value  (Maxim Orlov <orlovmg@gmail.com>)
Responses Re: Aggregate Function corr does not always return the correct value
Re: Aggregate Function corr does not always return the correct value
List pgsql-hackers
Maxim Orlov <orlovmg@gmail.com> writes:
> One of the clients complained as to why the query for calculating the
> correlation coefficient with the CORR function yielded such weird
> results. After a little analysis, it was discovered that they were
> calculating the correlation coefficient for two sets, one of which is
> more or less random and the other of which is simply a set of constant
> values (0.09 if that matters). As a result, they were attaining
> unexpected results. However, as far as I am aware, they should have
> received NULL because it is impossible to calculate the standard
> deviation for such a set.

[ shrug... ]  Calculations with float8 are inherently inexact, so
it's unsurprising that we sometimes fail to detect that the input
is exactly a horizontal or vertical line.  I don't think there is
anything to be done here that wouldn't end in making things worse.

            regards, tom lane



pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: [BUG] Remove self joins causes 'variable not found in subplan target lists' error
Next
From: "DINESH NAIR"
Date:
Subject: Re: Aggregate Function corr does not always return the correct value