Re: Using database to find file doublettes in my computer - Mailing list pgsql-general

On Mon, Nov 17, 2008 at 11:22:47AM -0800, Lothar Behrens wrote:
> I have a problem to find as fast as possible files that are double or
> in other words, identical.
> Also identifying those files that are not identical.

I'd probably just take a simple Unix command line approach, something
like:

  find /base/dir -type f -exec md5sum {} \; | sort | uniq -Dw 32

this will give you a list of files whose contents are identical
(according to an MD5 hash).  An alternative would be to put the hashes
into a database and run the matching up there.


  Sam