Thread: Re: Full Text Indexing Using Tsearch2-Module

Re: Full Text Indexing Using Tsearch2-Module

From
Michael Fuhr
Date:
On Mon, Jan 23, 2006 at 03:05:07PM +0530, Praveen Kumar (TUV) wrote:
> I have installed Tsearch-Module for full text indexing .But when
> I search text using gist(idxFTI) index on table I also found all
> data which have same accent.Example
> 1.If I try search for MANI word it also search for MANY word.
> 2.If I try search for ANDY word it also search for ANDI word.
> Please can you tell me how to avoid this problem ? If i want to
> search text MANI it should search only for MANI not MANY.

Your tsearch2 configuration is turning the words "many" and "andy"
into the lexemes "mani" and "andi", like this:

test=> SELECT * FROM ts_debug('many andy');
 ts_name | tok_type | description | token | dict_name | tsvector
---------+----------+-------------+-------+-----------+----------
 default | lword    | Latin word  | many  | {en_stem} | 'mani'
 default | lword    | Latin word  | andy  | {en_stem} | 'andi'
(2 rows)

To learn how to change that see "Parsing and Lexing" and "Configurations"
in the tsearch2 documentation:

http://www.sai.msu.su/~megera/postgres/gist/tsearch/V2/docs/tsearch2-guide.html#parsing_lexing
http://www.sai.msu.su/~megera/postgres/gist/tsearch/V2/docs/tsearch2-ref.html#configurations

Another approach is to think of tsearch2 as returning "might match"
rows and add another restriction to find the definite matches:

test=> SELECT * FROM foo
test-> WHERE idxcontent @@ to_tsquery('many');
 id | content | idxcontent
----+---------+------------
  1 | many    | 'mani':1
  2 | mani    | 'mani':1
(2 rows)

test=> SELECT * FROM foo
test-> WHERE idxcontent @@ to_tsquery('many') AND content ~* 'many';
 id | content | idxcontent
----+---------+------------
  1 | many    | 'mani':1
(1 row)

--
Michael Fuhr

Re: Full Text Indexing Using Tsearch2-Module

From
Oleg Bartunov
Date:
The simplest way is to use custom dictionary like 'synonym' !
many:many
and let it first dictionary !

     Oleg
On Mon, 23 Jan 2006, Michael Fuhr wrote:

> On Mon, Jan 23, 2006 at 03:05:07PM +0530, Praveen Kumar (TUV) wrote:
>> I have installed Tsearch-Module for full text indexing .But when
>> I search text using gist(idxFTI) index on table I also found all
>> data which have same accent.Example
>> 1.If I try search for MANI word it also search for MANY word.
>> 2.If I try search for ANDY word it also search for ANDI word.
>> Please can you tell me how to avoid this problem ? If i want to
>> search text MANI it should search only for MANI not MANY.
>
> Your tsearch2 configuration is turning the words "many" and "andy"
> into the lexemes "mani" and "andi", like this:
>
> test=> SELECT * FROM ts_debug('many andy');
> ts_name | tok_type | description | token | dict_name | tsvector
> ---------+----------+-------------+-------+-----------+----------
> default | lword    | Latin word  | many  | {en_stem} | 'mani'
> default | lword    | Latin word  | andy  | {en_stem} | 'andi'
> (2 rows)
>
> To learn how to change that see "Parsing and Lexing" and "Configurations"
> in the tsearch2 documentation:
>
> http://www.sai.msu.su/~megera/postgres/gist/tsearch/V2/docs/tsearch2-guide.html#parsing_lexing
> http://www.sai.msu.su/~megera/postgres/gist/tsearch/V2/docs/tsearch2-ref.html#configurations
>
> Another approach is to think of tsearch2 as returning "might match"
> rows and add another restriction to find the definite matches:
>
> test=> SELECT * FROM foo
> test-> WHERE idxcontent @@ to_tsquery('many');
> id | content | idxcontent
> ----+---------+------------
>  1 | many    | 'mani':1
>  2 | mani    | 'mani':1
> (2 rows)
>
> test=> SELECT * FROM foo
> test-> WHERE idxcontent @@ to_tsquery('many') AND content ~* 'many';
> id | content | idxcontent
> ----+---------+------------
>  1 | many    | 'mani':1
> (1 row)
>
>

     Regards,
         Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83