Thread: search engines

How can search engine search through millions of records in just a few
seconds?

All data is kept in memory, and a search engine is actually a cluster of
hundreds of computers. Every web page is heavily indexed, and the indexes
may be divided up across many computers. Thus, your search hits several
computers at a single time. Because each machine has gigabytes of web-page
indexes, it can quickly gather all results. They are then sorted and return
to the user.

David
----- Original Message -----
From: "Bruce Schreiber" <bruces@real-info.com>
To: <pgsql-general@postgresql.org>
Sent: Thursday, February 14, 2002 10:56 AM
Subject: [GENERAL] search engines


> How can search engine search through millions of records in just a few
> seconds?
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

On 14 Feb 2002, Bruce Schreiber wrote:

> How can search engine search through millions of records in just a few
> seconds?

They scan documents when they are inserted into the database. Every word
found is added to an index. So if the search is for a particular word
then the search engine just looks in the index for that word and finds a
list with corresponding documents.