> I know. My guess is the parser does not read the stop word file at
> least with default configuration.
Parser should not read stopword file: its deal for dictionaries.
>
> So if a character is not ASCII, it returns 0 even if p_isalpha returns
> 1. Is this what you expect?
No, p_islatin should return true only for latin characters, not for national ones.
>
> In our case, we added JAPANESE_STOP_WORD into english.stop then:
> select to_tsvector(JAPANESE_STOP_WORD)
> which returns words even they are in JAPANESE_STOP_WORD.
> And with the patches the problem was solved.
Pls, show your configuration for lexemes/dictionaries. I suspect that you have
en_stem dictionary on for lword lexemes type. Better way is to use 'simple'
distionary (it's support stopword the same way as en_stem does) and set it for
nlword, word, part_hword, nlpart_hword, hword, nlhword lexeme's types. Note,
leave unchanged en_stem for any latin word.
--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/
