Thread: Query never returns ...
The following query: SELECT tblSIDEDrugLink.DrugID, tblSIDEDrugLink.MedCondID, tblMedCond.PatientName AS MedCondPatientName, tblMedCond.ProfessionalName AS MedCondProfessionalName, tblSIDEDrugLink.Frequency, tblSIDEDrugLink.SeverityLevel FROM tblSIDEDrugLink, tblMedCond WHERE (tblSIDEDrugLink.DrugID IN ('DG-18698')) AND (tblSIDEDrugLink.MedCondID = tblMedCond.MedCondID) ORDER BY tblSIDEDrugLink.DrugID, tblSIDEDrugLink.Frequency, tblSIDEDrugLink.SeverityLevel, tblSIDEDrugLink.MedCondID; seems to not be liked by PostgreSQL. Table 'tblSIDEDrugLink' has the following structure: CREATE TABLE TBLSIDEDRUGLINK ( DRUGID VARCHAR(10) NOT NULL, MEDCONDID VARCHAR(10) NOT NULL, FREQUENCY INT2, SEVERITYLEVEL INT2, CONSTRAINT PK_TBLSIDEDRUGLINK PRIMARY KEY (DRUGID, MEDCONDID) ); with the following index: CREATE INDEX IX_TBLSIDEDRUGLINK1 ON TBLSIDEDRUGLINK (MEDCONDID); This table has 153,288 rows. Table 'tblMedCond' has the following structure: CREATE TABLE TBLMEDCOND ( MEDCONDID VARCHAR(10) NOT NULL, PROFESSIONALNAME VARCHAR(58), PATIENTNAME VARCHAR(58), CONSTRAINT PK_TBLMEDCOND PRIMARY KEY (MEDCONDID) ); This table has 1,730 rows. The query above is made by a third-party API that I don't have the source for, so I can't modify the query in the API, though the third-party has been quite willing to help out - they may even ship me a 'special' version of the API if there's something in this query that PostgreSQL for some reason doesn't implement efficiently enough. If it would help anyone to see the query plan or such - I can modify the logs to show that, just let me know. Btw - I've let this query run for a while & I haven't seen it complete ... soooo ... I don't know if it would ever complete or not. Any help at all is as always, appreciated. Sincerest regards, -- Brice Ruth WebProjkt, Inc. VP, Director of Internet Technology http://www.webprojkt.com/
FYI - I let the query run for 11.5 minutes before killing it off. It had consumed 11 minutes, 18 seconds of CPU time (reported by ps). The following messages are from the server log, I'm pasting in all messages directly following the query: 010208.10:04:29.473 [24041] ProcessQuery 010208.10:15:59.212 [24041] FATAL 1: The system is shutting down 010208.10:15:59.213 [24041] AbortCurrentTransaction FATAL: s_lock(4001600c) at spin.c:111, stuck spinlock. Aborting. FATAL: s_lock(4001600c) at spin.c:111, stuck spinlock. Aborting. Server process (pid 24041) exited with status 134 at Thu Feb 8 10:17:09 2001 Terminating any active server processes... Server processes were terminated at Thu Feb 8 10:17:09 2001 Regards, Brice Ruth Brice Ruth wrote: > > The following query: > > SELECT > tblSIDEDrugLink.DrugID, > tblSIDEDrugLink.MedCondID, > tblMedCond.PatientName AS MedCondPatientName, > tblMedCond.ProfessionalName AS MedCondProfessionalName, > tblSIDEDrugLink.Frequency, > tblSIDEDrugLink.SeverityLevel > FROM > tblSIDEDrugLink, > tblMedCond > WHERE > (tblSIDEDrugLink.DrugID IN ('DG-18698')) AND > (tblSIDEDrugLink.MedCondID = tblMedCond.MedCondID) > ORDER BY > tblSIDEDrugLink.DrugID, > tblSIDEDrugLink.Frequency, > tblSIDEDrugLink.SeverityLevel, > tblSIDEDrugLink.MedCondID; > > seems to not be liked by PostgreSQL. Table 'tblSIDEDrugLink' has the > following structure: > > CREATE TABLE TBLSIDEDRUGLINK > ( > DRUGID VARCHAR(10) NOT NULL, > MEDCONDID VARCHAR(10) NOT NULL, > FREQUENCY INT2, > SEVERITYLEVEL INT2, > CONSTRAINT PK_TBLSIDEDRUGLINK PRIMARY KEY (DRUGID, MEDCONDID) > ); > > with the following index: > CREATE INDEX IX_TBLSIDEDRUGLINK1 ON TBLSIDEDRUGLINK (MEDCONDID); > > This table has 153,288 rows. > > Table 'tblMedCond' has the following structure: > > CREATE TABLE TBLMEDCOND > ( > MEDCONDID VARCHAR(10) NOT NULL, > PROFESSIONALNAME VARCHAR(58), > PATIENTNAME VARCHAR(58), > CONSTRAINT PK_TBLMEDCOND PRIMARY KEY (MEDCONDID) > ); > > This table has 1,730 rows. > > The query above is made by a third-party API that I don't have the > source for, so I can't modify the query in the API, though the > third-party has been quite willing to help out - they may even ship me a > 'special' version of the API if there's something in this query that > PostgreSQL for some reason doesn't implement efficiently enough. > > If it would help anyone to see the query plan or such - I can modify the > logs to show that, just let me know. > > Btw - I've let this query run for a while & I haven't seen it complete > ... soooo ... I don't know if it would ever complete or not. > > Any help at all is as always, appreciated. > > Sincerest regards, > -- > Brice Ruth > WebProjkt, Inc. > VP, Director of Internet Technology > http://www.webprojkt.com/ -- Brice Ruth WebProjkt, Inc. VP, Director of Internet Technology http://www.webprojkt.com/
What does explain show for the query and have you run vacuum analyze recently on the tables? On Thu, 8 Feb 2001, Brice Ruth wrote: > The following query: > > SELECT > tblSIDEDrugLink.DrugID, > tblSIDEDrugLink.MedCondID, > tblMedCond.PatientName AS MedCondPatientName, > tblMedCond.ProfessionalName AS MedCondProfessionalName, > tblSIDEDrugLink.Frequency, > tblSIDEDrugLink.SeverityLevel > FROM > tblSIDEDrugLink, > tblMedCond > WHERE > (tblSIDEDrugLink.DrugID IN ('DG-18698')) AND > (tblSIDEDrugLink.MedCondID = tblMedCond.MedCondID) > ORDER BY > tblSIDEDrugLink.DrugID, > tblSIDEDrugLink.Frequency, > tblSIDEDrugLink.SeverityLevel, > tblSIDEDrugLink.MedCondID; > > seems to not be liked by PostgreSQL. Table 'tblSIDEDrugLink' has the > following structure: > > CREATE TABLE TBLSIDEDRUGLINK > ( > DRUGID VARCHAR(10) NOT NULL, > MEDCONDID VARCHAR(10) NOT NULL, > FREQUENCY INT2, > SEVERITYLEVEL INT2, > CONSTRAINT PK_TBLSIDEDRUGLINK PRIMARY KEY (DRUGID, MEDCONDID) > ); > > with the following index: > CREATE INDEX IX_TBLSIDEDRUGLINK1 ON TBLSIDEDRUGLINK (MEDCONDID); > > This table has 153,288 rows. > > Table 'tblMedCond' has the following structure: > > CREATE TABLE TBLMEDCOND > ( > MEDCONDID VARCHAR(10) NOT NULL, > PROFESSIONALNAME VARCHAR(58), > PATIENTNAME VARCHAR(58), > CONSTRAINT PK_TBLMEDCOND PRIMARY KEY (MEDCONDID) > ); > > This table has 1,730 rows. > > The query above is made by a third-party API that I don't have the > source for, so I can't modify the query in the API, though the > third-party has been quite willing to help out - they may even ship me a > 'special' version of the API if there's something in this query that > PostgreSQL for some reason doesn't implement efficiently enough. > > If it would help anyone to see the query plan or such - I can modify the > logs to show that, just let me know. > > Btw - I've let this query run for a while & I haven't seen it complete > ... soooo ... I don't know if it would ever complete or not. > > Any help at all is as always, appreciated. > > Sincerest regards, > -- > Brice Ruth > WebProjkt, Inc. > VP, Director of Internet Technology > http://www.webprojkt.com/ >
Followup: This query, run against the same dataset in a MS Access 2000 database, returns immediately with the resultset. (I truly mean immediately) The computer running the Access db is a laptop, PII 266 w/ 128MB RAM (an old Dell Latitude). The server running PostgreSQL is a dual PIII 450 w/ 256MB RAM. This is just meant as an FYI. -Brice Brice Ruth wrote: > > The following query: > > SELECT > tblSIDEDrugLink.DrugID, > tblSIDEDrugLink.MedCondID, > tblMedCond.PatientName AS MedCondPatientName, > tblMedCond.ProfessionalName AS MedCondProfessionalName, > tblSIDEDrugLink.Frequency, > tblSIDEDrugLink.SeverityLevel > FROM > tblSIDEDrugLink, > tblMedCond > WHERE > (tblSIDEDrugLink.DrugID IN ('DG-18698')) AND > (tblSIDEDrugLink.MedCondID = tblMedCond.MedCondID) > ORDER BY > tblSIDEDrugLink.DrugID, > tblSIDEDrugLink.Frequency, > tblSIDEDrugLink.SeverityLevel, > tblSIDEDrugLink.MedCondID; > > seems to not be liked by PostgreSQL. Table 'tblSIDEDrugLink' has the > following structure: > > CREATE TABLE TBLSIDEDRUGLINK > ( > DRUGID VARCHAR(10) NOT NULL, > MEDCONDID VARCHAR(10) NOT NULL, > FREQUENCY INT2, > SEVERITYLEVEL INT2, > CONSTRAINT PK_TBLSIDEDRUGLINK PRIMARY KEY (DRUGID, MEDCONDID) > ); > > with the following index: > CREATE INDEX IX_TBLSIDEDRUGLINK1 ON TBLSIDEDRUGLINK (MEDCONDID); > > This table has 153,288 rows. > > Table 'tblMedCond' has the following structure: > > CREATE TABLE TBLMEDCOND > ( > MEDCONDID VARCHAR(10) NOT NULL, > PROFESSIONALNAME VARCHAR(58), > PATIENTNAME VARCHAR(58), > CONSTRAINT PK_TBLMEDCOND PRIMARY KEY (MEDCONDID) > ); > > This table has 1,730 rows. > > The query above is made by a third-party API that I don't have the > source for, so I can't modify the query in the API, though the > third-party has been quite willing to help out - they may even ship me a > 'special' version of the API if there's something in this query that > PostgreSQL for some reason doesn't implement efficiently enough. > > If it would help anyone to see the query plan or such - I can modify the > logs to show that, just let me know. > > Btw - I've let this query run for a while & I haven't seen it complete > ... soooo ... I don't know if it would ever complete or not. > > Any help at all is as always, appreciated. > > Sincerest regards, > -- > Brice Ruth > WebProjkt, Inc. > VP, Director of Internet Technology > http://www.webprojkt.com/ -- Brice Ruth WebProjkt, Inc. VP, Director of Internet Technology http://www.webprojkt.com/
All right ... after reading up on the documentation for vacuum, I understand why that's necessary. I've run vacuum analyze on all the tables, now. Here are the more realistic results from explain: NOTICE: QUERY PLAN: Sort (cost=62.46..62.46 rows=14 width=64) -> Nested Loop (cost=0.00..62.19 rows=14 width=64) -> Index Scan using pk_tblsidedruglink on tblsidedruglink (cost=0.00..33.82 rows=14 width=28) -> Index Scan using pk_tblmedcond on tblmedcond (cost=0.00..2.01 rows=1 width=36) The query runs lightning fast now ... THANK YOU!!! :) -Brice Stephan Szabo wrote: > > After you load the data, you need to run vacuum analzye. That'll > get statistics on the current data in the table. Of course, I'm > not sure that'll help in this case. > > On Thu, 8 Feb 2001, Brice Ruth wrote: > > > Stephan, > > > > Here is what EXPLAIN shows: > > > > NOTICE: QUERY PLAN: > > > > Sort (cost=0.02..0.02 rows=1 width=64) > > -> Nested Loop (cost=0.00..0.01 rows=1 width=64) > > -> Seq Scan on tblmedcond (cost=0.00..0.00 rows=1 width=36) > > -> Seq Scan on tblsidedruglink (cost=0.00..0.00 rows=1 width=28) > > > > As for vacuum analyze - prior to running into these problems, I deleted > > all data from the database (using delete from <tblname>) and then ran > > vacuumdb -a, after which I loaded the data into the tables using 'copy > > ... from' - there have been no updates to the database since then - > > merely selects. -- Brice Ruth WebProjkt, Inc. VP, Director of Internet Technology http://www.webprojkt.com/
After you load the data, you need to run vacuum analzye. That'll get statistics on the current data in the table. Of course, I'm not sure that'll help in this case. On Thu, 8 Feb 2001, Brice Ruth wrote: > Stephan, > > Here is what EXPLAIN shows: > > NOTICE: QUERY PLAN: > > Sort (cost=0.02..0.02 rows=1 width=64) > -> Nested Loop (cost=0.00..0.01 rows=1 width=64) > -> Seq Scan on tblmedcond (cost=0.00..0.00 rows=1 width=36) > -> Seq Scan on tblsidedruglink (cost=0.00..0.00 rows=1 width=28) > > As for vacuum analyze - prior to running into these problems, I deleted > all data from the database (using delete from <tblname>) and then ran > vacuumdb -a, after which I loaded the data into the tables using 'copy > ... from' - there have been no updates to the database since then - > merely selects.
Brice - What does EXPLAIN say for that query? With empty tables, I get two index scans, a merge join, and two sorts. I'm guessing wildly that you've got a non-optimal sort strategy happening somehow, given the four fold ORDER BY clause. Ross Here's the empty version: NOTICE: QUERY PLAN: Sort (cost=84.25..84.25 rows=100 width=64) -> Merge Join (cost=8.30..80.93 rows=100 width=64) -> Index Scan usingpk_tblmedcond on tblmedcond (cost=0.00..60.00 rows=1000 width=36) -> Sort (cost=8.30..8.30 rows=10 width=28) -> Index Scan using pk_tblsidedruglink on tblsidedruglink (cost=0.00..8.14 rows=10 width=28) EXPLAIN On Thu, Feb 08, 2001 at 10:19:43AM -0600, Brice Ruth wrote: > FYI - I let the query run for 11.5 minutes before killing it off. It > had consumed 11 minutes, 18 seconds of CPU time (reported by ps). The > following messages are from the server log, I'm pasting in all messages > directly following the query: >
Brice, This sounds like a problem with you postgresql install or your HDD rather than a problem with your query ... which appears to be fine. I would reccomend taking it up with PGSQL Inc. (or Great Bridge) pay-for support if this is a commercial project. -Josh Berkus -- ______AGLIO DATABASE SOLUTIONS___________________________ Josh Berkus Complete informationtechnology josh@agliodbs.com and data management solutions (415) 565-7293 for law firms, small businesses fax 621-2533 and non-profit organizations. San Francisco
Stephan, Here is what EXPLAIN shows: NOTICE: QUERY PLAN: Sort (cost=0.02..0.02 rows=1 width=64) -> Nested Loop (cost=0.00..0.01 rows=1 width=64) -> Seq Scan on tblmedcond (cost=0.00..0.00 rows=1 width=36) -> Seq Scan on tblsidedruglink (cost=0.00..0.00 rows=1 width=28) As for vacuum analyze - prior to running into these problems, I deleted all data from the database (using delete from <tblname>) and then ran vacuumdb -a, after which I loaded the data into the tables using 'copy ... from' - there have been no updates to the database since then - merely selects. -Brice Stephan Szabo wrote: > > What does explain show for the query and have you run > vacuum analyze recently on the tables? > > On Thu, 8 Feb 2001, Brice Ruth wrote: > > > The following query: > > > > SELECT > > tblSIDEDrugLink.DrugID, > > tblSIDEDrugLink.MedCondID, > > tblMedCond.PatientName AS MedCondPatientName, > > tblMedCond.ProfessionalName AS MedCondProfessionalName, > > tblSIDEDrugLink.Frequency, > > tblSIDEDrugLink.SeverityLevel > > FROM > > tblSIDEDrugLink, > > tblMedCond > > WHERE > > (tblSIDEDrugLink.DrugID IN ('DG-18698')) AND > > (tblSIDEDrugLink.MedCondID = tblMedCond.MedCondID) > > ORDER BY > > tblSIDEDrugLink.DrugID, > > tblSIDEDrugLink.Frequency, > > tblSIDEDrugLink.SeverityLevel, > > tblSIDEDrugLink.MedCondID; > > > > seems to not be liked by PostgreSQL. Table 'tblSIDEDrugLink' has the > > following structure: > > > > CREATE TABLE TBLSIDEDRUGLINK > > ( > > DRUGID VARCHAR(10) NOT NULL, > > MEDCONDID VARCHAR(10) NOT NULL, > > FREQUENCY INT2, > > SEVERITYLEVEL INT2, > > CONSTRAINT PK_TBLSIDEDRUGLINK PRIMARY KEY (DRUGID, MEDCONDID) > > ); > > > > with the following index: > > CREATE INDEX IX_TBLSIDEDRUGLINK1 ON TBLSIDEDRUGLINK (MEDCONDID); > > > > This table has 153,288 rows. > > > > Table 'tblMedCond' has the following structure: > > > > CREATE TABLE TBLMEDCOND > > ( > > MEDCONDID VARCHAR(10) NOT NULL, > > PROFESSIONALNAME VARCHAR(58), > > PATIENTNAME VARCHAR(58), > > CONSTRAINT PK_TBLMEDCOND PRIMARY KEY (MEDCONDID) > > ); > > > > This table has 1,730 rows. > > > > The query above is made by a third-party API that I don't have the > > source for, so I can't modify the query in the API, though the > > third-party has been quite willing to help out - they may even ship me a > > 'special' version of the API if there's something in this query that > > PostgreSQL for some reason doesn't implement efficiently enough. > > > > If it would help anyone to see the query plan or such - I can modify the > > logs to show that, just let me know. > > > > Btw - I've let this query run for a while & I haven't seen it complete > > ... soooo ... I don't know if it would ever complete or not. > > > > Any help at all is as always, appreciated. > > > > Sincerest regards, > > -- > > Brice Ruth > > WebProjkt, Inc. > > VP, Director of Internet Technology > > http://www.webprojkt.com/ > > -- Brice Ruth WebProjkt, Inc. VP, Director of Internet Technology http://www.webprojkt.com/
Ross, Thanx to Stephan's help, I found out that after loading the tables w/ data, I had to run vacuum analyze to inform the optimizer of the amount of data in the table (amongst other things, I imagine). After running that on all the tables, the query performs fine. -Brice "Ross J. Reedstrom" wrote: > > Brice - > What does EXPLAIN say for that query? With empty tables, I get two index scans, > a merge join, and two sorts. I'm guessing wildly that you've got a non-optimal > sort strategy happening somehow, given the four fold ORDER BY clause. > > Ross > > Here's the empty version: > > NOTICE: QUERY PLAN: > > Sort (cost=84.25..84.25 rows=100 width=64) > -> Merge Join (cost=8.30..80.93 rows=100 width=64) > -> Index Scan using pk_tblmedcond on tblmedcond (cost=0.00..60.00 rows=1000 width=36) > -> Sort (cost=8.30..8.30 rows=10 width=28) > -> Index Scan using pk_tblsidedruglink on tblsidedruglink (cost=0.00..8.14 rows=10 width=28) > > EXPLAIN > > On Thu, Feb 08, 2001 at 10:19:43AM -0600, Brice Ruth wrote: > > FYI - I let the query run for 11.5 minutes before killing it off. It > > had consumed 11 minutes, 18 seconds of CPU time (reported by ps). The > > following messages are from the server log, I'm pasting in all messages > > directly following the query: > > -- Brice Ruth WebProjkt, Inc. VP, Director of Internet Technology http://www.webprojkt.com/
Brice Ruth <brice@webprojkt.com> writes: > As for vacuum analyze - prior to running into these problems, I deleted > all data from the database (using delete from <tblname>) and then ran > vacuumdb -a, after which I loaded the data into the tables using 'copy > ... from' - there have been no updates to the database since then - > merely selects. That was the wrong order to do things in :-(. The VACUUM ANALYZE posted statistics showing all your tables as empty, and the planner is now faithfully choosing plans that are good for tiny tables --- ie, minimal startup cost and don't worry about per-tuple cost. There has been some talk of having stats automatically updated by COPY, but right now it doesn't happen. So the correct procedure is to do a VACUUM ANALYZE on a table *after* you do any sizable data additions. BTW, people have occasionally stated on the list that you need to redo VACUUM ANALYZE after adding/dropping indexes, but that's not true, at least not in the present state of the world. VACUUM ANALYZE only posts stats about the data in the table(s). The planner always looks at the current set of indices for a table, together with the last-posted data statistics, to choose a plan. regards, tom lane
Um, no. You should run vacuum analyze AFTER you loaded up the data, otherwise, your table statistics will be all wrong (it'll contain 'empty table' statistics). -alex On Thu, 8 Feb 2001, Brice Ruth wrote: > Stephan, > > Here is what EXPLAIN shows: > > NOTICE: QUERY PLAN: > > Sort (cost=0.02..0.02 rows=1 width=64) > -> Nested Loop (cost=0.00..0.01 rows=1 width=64) > -> Seq Scan on tblmedcond (cost=0.00..0.00 rows=1 width=36) > -> Seq Scan on tblsidedruglink (cost=0.00..0.00 rows=1 width=28) > > As for vacuum analyze - prior to running into these problems, I deleted > all data from the database (using delete from <tblname>) and then ran > vacuumdb -a, after which I loaded the data into the tables using 'copy > ... from' - there have been no updates to the database since then - > merely selects. > > -Brice > > Stephan Szabo wrote: > > > > What does explain show for the query and have you run > > vacuum analyze recently on the tables? > > > > On Thu, 8 Feb 2001, Brice Ruth wrote: > > > > > The following query: > > > > > > SELECT > > > tblSIDEDrugLink.DrugID, > > > tblSIDEDrugLink.MedCondID, > > > tblMedCond.PatientName AS MedCondPatientName, > > > tblMedCond.ProfessionalName AS MedCondProfessionalName, > > > tblSIDEDrugLink.Frequency, > > > tblSIDEDrugLink.SeverityLevel > > > FROM > > > tblSIDEDrugLink, > > > tblMedCond > > > WHERE > > > (tblSIDEDrugLink.DrugID IN ('DG-18698')) AND > > > (tblSIDEDrugLink.MedCondID = tblMedCond.MedCondID) > > > ORDER BY > > > tblSIDEDrugLink.DrugID, > > > tblSIDEDrugLink.Frequency, > > > tblSIDEDrugLink.SeverityLevel, > > > tblSIDEDrugLink.MedCondID; > > > > > > seems to not be liked by PostgreSQL. Table 'tblSIDEDrugLink' has the > > > following structure: > > > > > > CREATE TABLE TBLSIDEDRUGLINK > > > ( > > > DRUGID VARCHAR(10) NOT NULL, > > > MEDCONDID VARCHAR(10) NOT NULL, > > > FREQUENCY INT2, > > > SEVERITYLEVEL INT2, > > > CONSTRAINT PK_TBLSIDEDRUGLINK PRIMARY KEY (DRUGID, MEDCONDID) > > > ); > > > > > > with the following index: > > > CREATE INDEX IX_TBLSIDEDRUGLINK1 ON TBLSIDEDRUGLINK (MEDCONDID); > > > > > > This table has 153,288 rows. > > > > > > Table 'tblMedCond' has the following structure: > > > > > > CREATE TABLE TBLMEDCOND > > > ( > > > MEDCONDID VARCHAR(10) NOT NULL, > > > PROFESSIONALNAME VARCHAR(58), > > > PATIENTNAME VARCHAR(58), > > > CONSTRAINT PK_TBLMEDCOND PRIMARY KEY (MEDCONDID) > > > ); > > > > > > This table has 1,730 rows. > > > > > > The query above is made by a third-party API that I don't have the > > > source for, so I can't modify the query in the API, though the > > > third-party has been quite willing to help out - they may even ship me a > > > 'special' version of the API if there's something in this query that > > > PostgreSQL for some reason doesn't implement efficiently enough. > > > > > > If it would help anyone to see the query plan or such - I can modify the > > > logs to show that, just let me know. > > > > > > Btw - I've let this query run for a while & I haven't seen it complete > > > ... soooo ... I don't know if it would ever complete or not. > > > > > > Any help at all is as always, appreciated. > > > > > > Sincerest regards, > > > -- > > > Brice Ruth > > > WebProjkt, Inc. > > > VP, Director of Internet Technology > > > http://www.webprojkt.com/ > > > > >