Oleg, the data I have right now was generated using a random paragraph
generator. The words are real words, but there are only 508 distinct
keywords in the 3,000,000 records that tsearch2 will pick up, using
default settings. I was using this data set for the purpose of testing
tsearch2's capabilities, so it's not real world data. If you still want
it, let me know where to send it and I will send you a dump of the DB.
Kris
Oleg Bartunov wrote:
>Kris,
>
>we're working on prototype of tsearchd - full text search daemon, which
>maintain static inverted index outside of postgresql using the same
>parser, dictionary tsearch2 does. This approach could scale up
>fts capability preserving access to metadata, so yo may have
>"archive" part of your collection (tsearchd) and "online", which could be
>searchable with tsearch2.
>
>Here is what we have right now:
>
>pages ( tid integer, fts_index tsvector)
>
>1. Create index
>select count(tdindex(tid,fts_index)) from pages;
>2. Flush index
>select tdflush();
>3. Search
>select pages.tid, rank(fts_index,to_tsquery('supernovae & magellan')) as rank
>from pages, tdsearch(to_tsquery('supernovae & magellan')) as idx where
>tid=idx order by rank desc;
>
>If it's possible, you could share your data, so we could test our
>prototype on real data.
>
>
> Oleg
>
>
