makesign() broken in tsearch2 - Mailing list pgsql-hackers

From Tom Lane
Subject makesign() broken in tsearch2
Date
Msg-id 27381.1137794614@sss.pgh.pa.us
Whole thread Raw
Responses Re: makesign() broken in tsearch2  (Teodor Sigaev <teodor@sigaev.ru>)
List pgsql-hackers
I noticed the following code in tsearch2:

typedef uint64 TPQTGist;

static TPQTGist
makesign(QUERYTYPE * a)
{   int         i;   ITEM       *ptr = GETQUERY(a);   TPQTGist    sign = 0;
   for (i = 0; i < a->size; i++)   {       if (ptr->type == VAL)           sign |= 1 << (ptr->val % SIGLEN);
ptr++;  }
 
   return sign;
}

This is wrong because "1" is an int constant, not an int64, and
therefore the shift will be done in int width.  Correct would be
           sign |= ((TPQTGist) 1) << (ptr->val % SIGLEN);

The effect of this is at least that the high-order half of sign
remains always zero.  Depending on what the machine does with shifts
exceeding the word width (which is undefined according to ANSI C),
the bottom half might be messed up too.  So we are failing to exploit
the full intended "sign" space, which presumably is costing something
in index efficiency.

It looks to me like the values calculated by this routine end up on
disk, and therefore we can't fix it without forcing an initdb, or
at least REINDEX of all affected indexes.  Is that correct?
        regards, tom lane


pgsql-hackers by date:

Previous
From: Mike Rylander
Date:
Subject: Re: Surrogate keys (Was: enums)
Next
From: Tom Lane
Date:
Subject: Re: panic on 7.3