processing urls with tsearch2 - Mailing list pgsql-general

Hey guys,

maybe anyone using tsearch2 could advise on this. With the default installation, url, host and some other tokens are processed with the simple dictionary. Thus term like mywebsite.com gets stored as 'mywebsite.com'. The parser correctly assigns token id of type host to the term, but then the dictionary the terms gets routed through is simple and what gets stored is mywebsite.com

The questions are:

1) is there a dictionary available that I could utilize that will remove .com, .net, .org, etc? I could write one myself, but after seeing some sample dictionary implementations and C code I try to avoid, I got scared a bit.

2) has anyone else dealt with this maybe in a different way?


Thanks for any suggestions and help,

Laimis