Re: Slow query with self-join, group by, 100m rows - Mailing list pgsql-performance

From Tom Lane
Subject Re: Slow query with self-join, group by, 100m rows
Date
Msg-id 23039.1316617362@sss.pgh.pa.us
Whole thread Raw
In response to Slow query with self-join, group by, 100m rows  (Thomas Kappler <tkappler@googlemail.com>)
List pgsql-performance
Thomas Kappler <tkappler@googlemail.com> writes:
> The query we want to do is (with example values):

> select t.cid, count(distinct t1.subject_id)
> from termindex t1, termindex t2
> where t1.cid=20642 and t1.indextype=636 and t2.indextype=636 and
> t2.subject_id=t1.subject_id
> group by t2.cid;

The EXPLAIN output you provided doesn't appear to match this query (in
particular, I don't see the indextype restrictions being checked
anyplace in the plan).

One quick-and-dirty thing that might help is to raise work_mem enough
so that (1) you get a hash aggregation not a sort/group one, and (2)
if there are still any sorts being done, they don't spill to disk.
That will probably be a higher number than would be prudent to install
as a global setting, but you could SET it locally in the current
session before issuing the expensive query.

            regards, tom lane

pgsql-performance by date:

Previous
From: Shaun Thomas
Date:
Subject: Re: REINDEX not working for wastedspace
Next
From: Greg Smith
Date:
Subject: Re: Show_database_bloat reliability? [was: Re: REINDEX not working for wastedspace]