NOT IN clause performing badly - Mailing list pgsql-sql

From Christian Rudow
Subject NOT IN clause performing badly
Date
Msg-id 379C25B3.A86C31F8@thinx.ch
Whole thread Raw
Responses Re: [SQL] NOT IN clause performing badly  (Herouth Maoz <herouth@oumail.openu.ac.il>)
List pgsql-sql
I have a problem with NOT IN queries
------------------------------------
Has anyone experinced the same problem, or even found a solution to it ?

-- simple NOT IN clause does not perform
-- create a table with an integer primary key and any number and type  of attributes.
-- load approx. 10'000 records and vacuum;
-- then select a subset of say 8000 id's into a seperate table.
-- now try to find out the Id's of the remaining 2000 records

-- I would use a NOT IN clause
-- but this is what happens :

drop table tempx1;

select mytable.id      into table tempx1      from mytable, othertable      where mytable.id = othertable.id      and
othertable.someattribute= "Somevalue";
 

-- this will take only a few seconds
-- on platform : PG6.4 SusE 6.0 PII/300mhz
-- rdbms startup : su - postgres -c with parameters :
-- "/sbin/startproc -l $LOGFILE $H -B 256 -i -o -F -D$DATADIR"

-- select count(*) from tempx1 : now returns 8000 rows

select id from mytable      where id NOT IN (      select id from tempx1      );

-- this will take some 100 minutes at least !!!
-- but it does finish ! no crash or hangup.
-- NOTICE:  QUERY PLAN:
-- Seq Scan on mytable  (cost=958.89 size=10000 width=4)
--   SubPlan
--     ->  Seq Scan on tempx1  (cost=0.00 size=0 width=4)
--
-- an index on mytbale(id) exists.

Questions
---------
Is this a known problem ?
What is causing the problem ?
Is it a problem of RDBMS parameters ?
Is there an alternative to the NOT IN clause ?


I'd appreciate any help in this matter.
ThanX
Chris
-- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Christian Rudow                 E-Mail: Christian.Rudow@thinx.ch
ThinX networked business services    Stahlrain 10, CH-5200 Brugg
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


pgsql-sql by date:

Previous
From: Oleg Bartunov
Date:
Subject: Re: [SQL] inserts/updates problem under stressing !
Next
From: wieck@debis.com (Jan Wieck)
Date:
Subject: Re: [SQL] ERROR: DefineQueryRewrite: rule plan string too big.