Re: Slow query needs a kick in the pants. - Mailing list pgsql-general

From Dann Corbit
Subject Re: Slow query needs a kick in the pants.
Date
Msg-id D90A5A6C612A39408103E6ECDD77B829408AB7@voyager.corporate.connx.com
Whole thread Raw
In response to Slow query needs a kick in the pants.  ("Dann Corbit" <DCorbit@connx.com>)
Responses Re: Slow query needs a kick in the pants.  (Dennis Gearon <gearond@cvc.net>)
Re: Slow query needs a kick in the pants.  (Greg Stark <gsstark@mit.edu>)
List pgsql-general
The other query attempts given to me do not produce the desired results.  Thanks for taking a look anyway.
 
Let me explain the problem set, and what I want to accomplish...
 
I have two tables in all cases.  Each table pair consists of the following columns:
1.  A primary key of one or more columns {with a unique index}
2.  An Oid column {with a unique index}
3.  A 64 bit CRC
 
For both tables, the primary key information will "mostly" match.  I need to know which primary keys are found in the first table but not in the second.  Also, which primary keys are found in the second table but not in the first.
 
One possibility would be to add a tag column and set it where the two tables agree on primary key.  Unfortunately, we are likely to have many millions of rows (and only a few thousand disagreements) and so I would end up rewriting the entire table for both tables (less the tiny difference set) and hence that would be inefficient.   My outer join also stinks up the place.  (Several minutes for a million rows -- unacceptable).
 
Probably, I am going to have to write my own piece of custom software that manipulates the data outside of any database.  The database query approach is just too slow.
 
 

pgsql-general by date:

Previous
From: "Dann Corbit"
Date:
Subject: Re: About OIDs
Next
From: Jeff Davis
Date:
Subject: Re: Python Error