Re: multi terabyte fulltext searching - Mailing list pgsql-general

From Benjamin Arai
Subject Re: multi terabyte fulltext searching
Date
Msg-id 4AE0FCAC-AFDC-4131-AF8E-3D45A4E50B0B@araisoft.com
Whole thread Raw
In response to Re: multi terabyte fulltext searching  (Oleg Bartunov <oleg@sai.msu.su>)
List pgsql-general
Are there any examples of dblink being used in commercial
environments.  I am curious to understand how it deals with node
failures and etc.

Benjamin

On Mar 21, 2007, at 9:35 AM, Oleg Bartunov wrote:

> On Wed, 21 Mar 2007, Benjamin Arai wrote:
>
>> Can't you implement something similar to google by aggregating
>> results for TSearch2 over many machines?
>
> tsearch2 doesn't use any global statistics, so, in principle, you
> should be able to run fts on several machines and combine them using
> dblink (contrib/dblink).
>
>>
>> Benjamin
>> On Mar 21, 2007, at 8:59 AM, Teodor Sigaev wrote:
>>
>>> I'm afraid that fulltext search on multiterabytes set of
>>> documents can not be implemented on any RDBMS, at least on single
>>> box. Specialized fulltext search engines (with exact matching and
>>> time to search about one second) has practical limit near 20
>>> millions of docs, cluster - near 100 millions. Bigger collections
>>> require engines like a google.
>>> --
>>> Teodor Sigaev                                   E-mail:
>>> teodor@sigaev.ru
>>>                                                   WWW: http://
>>> www.sigaev.ru/
>>
>>
>> ---------------------------(end of
>> broadcast)---------------------------
>> TIP 9: In versions below 8.0, the planner will ignore your desire to
>>     choose an index scan if your joining column's datatypes do not
>>     match
>
>     Regards,
>         Oleg
> _____________________________________________________________
> Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
> Sternberg Astronomical Institute, Moscow University, Russia
> Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
> phone: +007(495)939-16-83, +007(495)939-23-83
>


pgsql-general by date:

Previous
From: "Candy Gutierrez"
Date:
Subject: PostgreSQL 7.3.4 using Cygwin on a Windows 2003 Server R2 SP2
Next
From: Ottavio Campana
Date:
Subject: questions about query design