By the way, what is the largest TSearch2 database that you know of
and how fast does it return results? Maybe my expectations are
unrealistic.
Benjamin
On Mar 21, 2007, at 8:42 AM, Oleg Bartunov wrote:
> Benjamin,
>
> as one of the author of tsearch2 I'd like to know more about your
> setup.
> tsearch2 in 8.2 has GIN index support, which scales much better
> than old
> GiST index.
>
> Oleg
>
> On Wed, 21 Mar 2007, Benjamin Arai wrote:
>
>> Hi,
>>
>> I have been struggling with getting fulltext searching for very
>> large databases. I can fulltext index 10s if gigs without any
>> problem but when I start geting to hundreds of gigs it becomes
>> slow. My current system is a quad core with 8GB of memory. I
>> have the resource to throw more hardware at it but realistically
>> it is not cost effective to buy a system with 128GB of memory. Is
>> there any solutions that people have come up with for indexing
>> very large text databases?
>>
>> Essentially I have several terabytes of text that I need to
>> index. Each record is about 5 paragraphs of text. I am currently
>> using TSearch2 (stemming and etc) and getting sub-optimal
>> results. Queries take more than a second to execute. Has anybody
>> implemented such a database using multiple systems or some special
>> add-on to TSearch2 to make things faster? I want to do something
>> like partitioning the data into multiple systems and merging the
>> ranked results at some master node. Is something like this
>> possible for PostgreSQL or must it be a software solution?
>>
>> Benjamin
>>
>> ---------------------------(end of
>> broadcast)---------------------------
>> TIP 9: In versions below 8.0, the planner will ignore your desire to
>> choose an index scan if your joining column's datatypes do not
>> match
>
> Regards,
> Oleg
> _____________________________________________________________
> Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
> Sternberg Astronomical Institute, Moscow University, Russia
> Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
> phone: +007(495)939-16-83, +007(495)939-23-83
>
