Postgres Pro
Downloads
Home   >   mailing lists

Thread: JOINS and non use of indexes

JOINS and non use of indexes

From
"Ian Cass"
Date:
Hi,

Hope someone can spot where I'm going wrong here.

I'm transferring a database & reporting scripts across from Oracle & a few
of the SQL statements behave a little differently with regards to indexes.
The one that's got me stumped at the moment is this...

select * from messages, statusinds
WHERE statusinds.gateway_id = messages.gateway_id
AND (messages.client_id = '7' AND messages.user_name in ('U66515'))
limit 5;

It's using the index on the messages table. On Oracle, it would do a
sequential index scan on the messages table & lookup the appropriate entry
in the statusinds table using the index. However, on Postgres, I can't get
it to use the statusinds index - it does a sequential scan through the
entire table each time! As you can imagine, it's taking ages to do this
where it used to take a few seconds on Oracle.

I've tried explicitly specifying the JOIN type & I can't seem to find the
right combination. I've tried doing a simple select on statusinds where
gateway_id = 'xxx' and the explain tells me it's doing index lookups.

Indexes are as follows...

-- Index: statusinds_200204_ix1
CREATE UNIQUE INDEX statusinds_ix1 ON statusinds USING btree (gateway_id,
status, logtime);
-- Index: messages_200204_ix1
CREATE UNIQUE INDEX messages_200204_ix1 ON messages_200204 USING btree
(host, qos_id);

Explain plan....
Limit (cost=0.00..35.06 rows=5 width=620)
-> Nested Loop (cost=0.00..30986063.51 rows=4418898 width=620)
-> Append (cost=0.00..441.93 rows=111 width=496)
-> Seq Scan on messages (cost=0.00..0.00 rows=1 width=496)
-> Index Scan using messages_200203_ix2 on messages_200203 messages
(cost=0.00..272.61 rows=68 width=383)
-> Index Scan using messages_200204_ix2 on messages_200204 messages
(cost=0.00..169.32 rows=42 width=384)
-> Append (cost=0.00..180413.11 rows=7996912 width=124)
-> Seq Scan on statusinds (cost=0.00..0.00 rows=1 width=124)
-> Seq Scan on statusinds_200203 statusinds (cost=0.00..142835.73
rows=6292073 width=71)
-> Seq Scan on statusinds_200204 statusinds (cost=0.00..37577.38
rows=1704838 width=65)

(tables_YYYYMM are inherited)

--
Ian Cass

Re: JOINS and non use of indexes

From
"Ian Cass"
Date:
Of course, the messages index looks like this...

CREATE INDEX messages_200204_ix2 ON messages_200204 USING btree (client_id,
user_name);

Duh!

> -- Index: messages_200204_ix1
> CREATE UNIQUE INDEX messages_200204_ix1 ON messages_200204 USING btree

Re: JOINS and non use of indexes

From
"Christopher Kings-Lynne"
Date:
I haven't had a good look, but just remember that indexes on a table in
postgres are NOT inherited by its children.  You cannot define a unique
index over a column that is inherited - it will be unique for the table you
define it on only.  Hence, you may not actually have indexes on those
inherited tables, and therefore they cannot be used...

Chris

> Explain plan....
> Limit (cost=0.00..35.06 rows=5 width=620)
> -> Nested Loop (cost=0.00..30986063.51 rows=4418898 width=620)
> -> Append (cost=0.00..441.93 rows=111 width=496)
> -> Seq Scan on messages (cost=0.00..0.00 rows=1 width=496)
> -> Index Scan using messages_200203_ix2 on messages_200203 messages
> (cost=0.00..272.61 rows=68 width=383)
> -> Index Scan using messages_200204_ix2 on messages_200204 messages
> (cost=0.00..169.32 rows=42 width=384)
> -> Append (cost=0.00..180413.11 rows=7996912 width=124)
> -> Seq Scan on statusinds (cost=0.00..0.00 rows=1 width=124)
> -> Seq Scan on statusinds_200203 statusinds (cost=0.00..142835.73
> rows=6292073 width=71)
> -> Seq Scan on statusinds_200204 statusinds (cost=0.00..37577.38
> rows=1704838 width=65)
>
> (tables_YYYYMM are inherited)
>
> --
> Ian Cass
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>

Re: JOINS and non use of indexes

From
"Ian Cass"
Date:
You're right. But I do this....

my $sql = "create table messages_$month () inherits (messages)";
...
my $sql = "create unique index messages_" . $month . "_ix1 on
messages_$month using btree (host, qos_id)";
...
my $sql = "create index messages_" . $month . "_ix2 on messages_$month using
btree (client_id, user_name)";
...

Similar thing for my statusinds tables too.

--
Ian Cass

----- Original Message -----
From: "Christopher Kings-Lynne" <chriskl@familyhealth.com.au>
To: "Ian Cass" <ian.cass@mblox.com>; <pgsql-sql@postgresql.org>
Sent: Monday, April 08, 2002 3:33 PM
Subject: Re: [SQL] JOINS and non use of indexes


> I haven't had a good look, but just remember that indexes on a table in
> postgres are NOT inherited by its children.  You cannot define a unique
> index over a column that is inherited - it will be unique for the table
you
> define it on only.  Hence, you may not actually have indexes on those
> inherited tables, and therefore they cannot be used...
>
> Chris
>
> > Explain plan....
> > Limit (cost=0.00..35.06 rows=5 width=620)
> > -> Nested Loop (cost=0.00..30986063.51 rows=4418898 width=620)
> > -> Append (cost=0.00..441.93 rows=111 width=496)
> > -> Seq Scan on messages (cost=0.00..0.00 rows=1 width=496)
> > -> Index Scan using messages_200203_ix2 on messages_200203 messages
> > (cost=0.00..272.61 rows=68 width=383)
> > -> Index Scan using messages_200204_ix2 on messages_200204 messages
> > (cost=0.00..169.32 rows=42 width=384)
> > -> Append (cost=0.00..180413.11 rows=7996912 width=124)
> > -> Seq Scan on statusinds (cost=0.00..0.00 rows=1 width=124)
> > -> Seq Scan on statusinds_200203 statusinds (cost=0.00..142835.73
> > rows=6292073 width=71)
> > -> Seq Scan on statusinds_200204 statusinds (cost=0.00..37577.38
> > rows=1704838 width=65)
> >
> > (tables_YYYYMM are inherited)
> >
> > --
> > Ian Cass
> >
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 4: Don't 'kill -9' the postmaster
> >
>
>

Re: JOINS and non use of indexes

From
Tom Lane
Date:
"Ian Cass" <ian.cass@mblox.com> writes:
> I'm transferring a database & reporting scripts across from Oracle & a few
> of the SQL statements behave a little differently with regards to indexes.
> The one that's got me stumped at the moment is this...

> select * from messages, statusinds
> WHERE statusinds.gateway_id = messages.gateway_id
> AND (messages.client_id = '7' AND messages.user_name in ('U66515'))
> limit 5;

> (tables_YYYYMM are inherited)

There's your problem (and don't tell me Oracle gets this right; they
don't do inheritance, do they?).  The planner isn't smart about
indexscan-based joins for inheritance trees.  This could possibly be
improved with some work, but I haven't thought about the details.
        regards, tom lane

Re: JOINS and non use of indexes

From
"Ian Cass"
Date:
> There's your problem (and don't tell me Oracle gets this right; they
> don't do inheritance, do they?).

No, Oracle doesn't do inheritance.

> The planner isn't smart about
> indexscan-based joins for inheritance trees.  This could possibly be
> improved with some work, but I haven't thought about the details.

Is there any way I could 'educate' it by using more explicit sql, or do I
have to abandon the idea of using inheritance altogether?

--
Ian Cass

Re: JOINS and non use of indexes

From
"Ian Cass"
Date:
> There's your problem (and don't tell me Oracle gets this right; they
> don't do inheritance, do they?).  The planner isn't smart about
> indexscan-based joins for inheritance trees.  This could possibly be
> improved with some work, but I haven't thought about the details.

It's true. When I use the monthly tables explicitly and not it's parent, it
uses the index properly. Unfortunately this isn't the solution I was looking
for :/

select * from messages_200203 as messages, statusinds_200203 as statusinds
WHERE (messages.client_id = '7' AND messages.user_name in ('U66515'))
AND statusinds.gateway_id = messages.gateway_id
limit 5;

Limit (cost=0.00..20.69 rows=5 width=455)
-> Nested Loop (cost=0.00..13495.79 rows=3261 width=455)
-> Index Scan using messages_200203_ix2 on messages_200203 messages
(cost=0.00..272.61 rows=68 width=383)
-> Index Scan using statusinds_200203_ix1 on statusinds_200203 statusinds
(cost=0.00..194.87 rows=48 width=72)

--
Ian Cass

Re: JOINS and non use of indexes

From
Tom Lane
Date:
"Ian Cass" <ian.cass@mblox.com> writes:
>> The planner isn't smart about
>> indexscan-based joins for inheritance trees.  This could possibly be
>> improved with some work, but I haven't thought about the details.

> Is there any way I could 'educate' it by using more explicit sql, or do I
> have to abandon the idea of using inheritance altogether?

You'd have to spell out the join for each member of the inheritance
tree:
SELECT ... FROM outside, inside_1 WHERE ...UNION ALLSELECT ... FROM outside, inside_2 WHERE ...UNION ALLSELECT ... FROM
outside,inside_3 WHERE ...
 

which is just about as bad as not using inheritance :-(
        regards, tom lane