Re: String Similarity - Mailing list pgsql-hackers

From Mark Woodward
Subject Re: String Similarity
Date
Msg-id 18465.24.91.171.78.1148086851.squirrel@mail.mohawksoft.com
Whole thread Raw
In response to Re: String Similarity  ("Greg Sabino Mullane" <greg@turnstep.com>)
List pgsql-hackers
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
>> I have a side project that needs to "intelligently" know if two strings
>> are contextually similar.
>
> The examples you gave seem heavy on word order and whitespace
> consideration,
> before applying any algorithms. Here's a quick perl version that does the
> job:

[SNIP]

This is a case where the example was too simple to explain the problem,
sorry. I have an implementation of Oracle's "contains" function for
PostgreSQL, and it does basically what you are doing, and, in fact, also
has Mohawk Software Extensions (LOL) that provide metaphone. The problem
is that parsing white space realy isn't reliable. Sometimes it is
pinkfloyd-darksideofthemoon.

Also, I have been thinking of other applications.

I have a piece of code that does this:

apps$ ./stratest "pink foyd dark side of the moon money" "money dark side
of the moon pink floyd"
Match:  dark side of the moon
Match: pink f
Match: money
Match: oyd

apps$ ./stratest "pinkfoyddarksideofthemoonmoney"
"moneydarksideofthemoonpinkfloyd"
Match: darksideofthemoon
Match: pinkf
Match: money
Match: oyd

I need to come up with a numerically sane way of taking this information
and understanding overall "similarity."


pgsql-hackers by date:

Previous
From: Bruce Momjian
Date:
Subject: Re: patch review, please: Autovacuum/Vacuum times via stats.
Next
From: Oleg Bartunov
Date:
Subject: Re: String Similarity