Re: full text searching - Mailing list pgsql-general

From Mitch Vincent
Subject Re: full text searching
Date
Msg-id 004101c09058$18200f90$0200000a@windows
Whole thread Raw
In response to full text searching  (Culley Harrelson <culleyharrelson@yahoo.com>)
List pgsql-general
> Well you wouldn't want to start  building these kind of rules in your
> application - better to have them in the search engine. The fulltextindex
> in the contrib package does of course not offer anything like this, it
> would be nice to see a third party addon provide fulltext capabilities for
> Postgresql.

Well, the search engine isn't the database, IMHO. The search "engine" is
your application... The database will go get anything you tell it to, you
just have to know how to tell it and make sure that your application tells
it in the correct way.

Teaching an application or database the English language is going to be a
hell of a project, good luck!

 Personally, I think the FTI trigger and function that's in contrib is
pretty bad. It's not usable in a lot of situations, I re-wrote it to remove
duplicates and index whole words but it still didn't work out for me...
Namely when you have fairly large chunks of text (30k or so), one for each
record in a row (and you have 10,000 rows).. Well, ripping out and indexing
30k*10k text chunks is a lot by itself but then when you search it you have
to join the two tables... It becomes a mess and was actually slower than
when I used LIKE to search the big text fields in my single table. It only
take a few seconds for the seq scan and the index scan on the FTI table but
with FTI updating became a 30 second job (of course there were like 4
million rows and each app did have 30k of text or so).. I don't have too
many small databases, so maybe this works for a lot of people :-)

Anyway. Moral of the story.. I'd like to see native PostgreSQL full text
indexing before we go adding on to the contrib'd trigger/function
implementation...

-Mitch


pgsql-general by date:

Previous
From: The Hermit Hacker
Date:
Subject: Re: 7.1 release date?
Next
From: Fred Yankowski
Date:
Subject: Re: startup Postgres on NT