Thread: Re: pgsql: Protect GIST logic that assumes penalty values can't be negative
On Tue, May 31, 2011 at 2:54 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Prevent problems by clamping negative penalty values to > zero. (Just to be really sure, I also made it force NaNs to zero.) Do gistchoose et al expect the triangle function to obey the triangle inequality? If so isn't it possible treating NaNs as zero would fail that? I'm not sure there's any safe assumption for NaN -- greg
Greg Stark <gsstark@mit.edu> writes:
> On Tue, May 31, 2011 at 2:54 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> �Prevent problems by clamping negative penalty values to
>> zero. �(Just to be really sure, I also made it force NaNs to zero.)
> Do gistchoose et al expect the triangle function to obey the triangle
> inequality?
Don't think so.
> If so isn't it possible treating NaNs as zero would fail
> that? I'm not sure there's any safe assumption for NaN
Well, leaving it as NaN is almost certain to not work desirably.
regards, tom lane
On Tue, May 31, 2011 at 4:11 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> Do gistchoose et al expect the triangle function to obey the triangle >> inequality? > > Don't think so. > I guess it was obvious but that was "expect the *penalty* function to obey the triangle inequality" -- greg
Re: pgsql: Protect GIST logic that assumes penalty values can't be negative
From
Alexander Korotkov
Date:
On Wed, Jun 1, 2011 at 3:57 AM, Greg Stark <gsstark@mit.edu> wrote:
------I guess it was obvious but that was "expect the *penalty* function toobey the triangle inequality"
Actually, penalty function for boxes is even not commutative. Fox example:
A = ((0,0)-(1,1))
B = ((0,0)-(2,2))
penalty(A,B) = 3
penalty(B,A) = 0
With best regards,
Alexander Korotkov.