Thread: Index not used without explicit typecast

Index not used without explicit typecast

From
Jan Kort
Date:

Hi,

 

When I have a large table, the index is not used if the type I use in the comparison does not match exactly. The result is the same, but it takes longer.

 

Below is an example that illustrates this.

 

Is it possible to fix this?

 

Regards,

 

Jan

 

-- Create 4M records

drop table if exists table1;

create table table1 (

    id integer NOT NULL,

    CONSTRAINT table1p PRIMARY KEY (id),

    date1 timestamp without time zone);

insert into table1 (id) values (1);

insert into table1 (id) select id + 1 from table1;

insert into table1 (id) select id + 2 from table1;

insert into table1 (id) select id + 4 from table1;

insert into table1 (id) select id + 8 from table1;

insert into table1 (id) select id + 16 from table1;

insert into table1 (id) select id + 32 from table1;

insert into table1 (id) select id + 64 from table1;

insert into table1 (id) select id + 128 from table1;

insert into table1 (id) select id + 256 from table1;

insert into table1 (id) select id + 512 from table1;

insert into table1 (id) select id + 1024 from table1;

insert into table1 (id) select id + 2048 from table1;

insert into table1 (id) select id + 4096 from table1;

insert into table1 (id) select id + 8192 from table1;

insert into table1 (id) select id + 16384 from table1;

insert into table1 (id) select id + 32768 from table1;

insert into table1 (id) select id + 65536 from table1;

insert into table1 (id) select id + 65536 * 2 from table1;

insert into table1 (id) select id + 65536 * 4 from table1;

insert into table1 (id) select id + 65536 * 8 from table1;

insert into table1 (id) select id + 65536 * 16 from table1;

insert into table1 (id) select id + 65536 * 32 from table1;

vacuum analyze table1;

 

select count(*) from table1

UPDATE TABLE1 SET date1 = current_timestamp WHERE ID = 1000000;

-- 45ms

 

UPDATE TABLE1 SET date1 = current_timestamp WHERE ID = 1000000::numeric;

-- 490ms

 

 

Re: Index not used without explicit typecast

From
Tom Lane
Date:
Jan Kort <jan.kort@genetics.nl> writes:
> UPDATE TABLE1 SET date1 = current_timestamp WHERE ID = 1000000;
> [ uses index on integer column ID ]
> UPDATE TABLE1 SET date1 = current_timestamp WHERE ID = 1000000::numeric;
> [ doesn't use index ]

Yeah.  This is the price we pay for extensibility.  The only available
"=" operator that can match the second query is "numeric = numeric",
so the parser effectively converts it to "ID::numeric = 1000000::numeric",
and then "ID::numeric" does not match the index, any more than say
"abs(ID)" would.

In principle one could invent an "integer = numeric" operator and then
make it a member of the appropriate btree operator class, but there are
assorted pitfalls and gotchas in that.  The biggest risk is that the
extra operator would result in "ambiguous operator" failures for queries
that work fine today.

If you're desperate for a workaround that doesn't involve fixing the
query, you could build an additional index on "ID::numeric".  This'd
be kind of expensive from an index-maintenance standpoint, of course.

            regards, tom lane



Re: Index not used without explicit typecast

From
Jan Kort
Date:
Thanks for the quick reply and suggestions. I will change all references to integer type then.

Adding more indexes could be a good fallback  if the other solution has unforeseen problems at the application side.

Regards,
Jan


From: Tom Lane <tgl@sss.pgh.pa.us>
Sent: Wednesday, August 5, 2020 4:32:22 PM
To: Jan Kort <jan.kort@genetics.nl>
Cc: pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Subject: Re: Index not used without explicit typecast
 
Jan Kort <jan.kort@genetics.nl> writes:
> UPDATE TABLE1 SET date1 = current_timestamp WHERE ID = 1000000;
> [ uses index on integer column ID ]
> UPDATE TABLE1 SET date1 = current_timestamp WHERE ID = 1000000::numeric;
> [ doesn't use index ]

Yeah.  This is the price we pay for extensibility.  The only available
"=" operator that can match the second query is "numeric = numeric",
so the parser effectively converts it to "ID::numeric = 1000000::numeric",
and then "ID::numeric" does not match the index, any more than say
"abs(ID)" would.

In principle one could invent an "integer = numeric" operator and then
make it a member of the appropriate btree operator class, but there are
assorted pitfalls and gotchas in that.  The biggest risk is that the
extra operator would result in "ambiguous operator" failures for queries
that work fine today.

If you're desperate for a workaround that doesn't involve fixing the
query, you could build an additional index on "ID::numeric".  This'd
be kind of expensive from an index-maintenance standpoint, of course.

                        regards, tom lane

Re: Index not used without explicit typecast

From
Pavel Stehule
Date:
Hi

st 5. 8. 2020 v 18:47 odesílatel Jan Kort <jan.kort@genetics.nl> napsal:
Thanks for the quick reply and suggestions. I will change all references to integer type then.

Adding more indexes could be a good fallback  if the other solution has unforeseen problems at the application side.

Any other index increases the cost of INSERT, UPDATE or VACUUM.

Regards

Pavel


Regards,
Jan


From: Tom Lane <tgl@sss.pgh.pa.us>
Sent: Wednesday, August 5, 2020 4:32:22 PM
To: Jan Kort <jan.kort@genetics.nl>
Cc: pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Subject: Re: Index not used without explicit typecast
 
Jan Kort <jan.kort@genetics.nl> writes:
> UPDATE TABLE1 SET date1 = current_timestamp WHERE ID = 1000000;
> [ uses index on integer column ID ]
> UPDATE TABLE1 SET date1 = current_timestamp WHERE ID = 1000000::numeric;
> [ doesn't use index ]

Yeah.  This is the price we pay for extensibility.  The only available
"=" operator that can match the second query is "numeric = numeric",
so the parser effectively converts it to "ID::numeric = 1000000::numeric",
and then "ID::numeric" does not match the index, any more than say
"abs(ID)" would.

In principle one could invent an "integer = numeric" operator and then
make it a member of the appropriate btree operator class, but there are
assorted pitfalls and gotchas in that.  The biggest risk is that the
extra operator would result in "ambiguous operator" failures for queries
that work fine today.

If you're desperate for a workaround that doesn't involve fixing the
query, you could build an additional index on "ID::numeric".  This'd
be kind of expensive from an index-maintenance standpoint, of course.

                        regards, tom lane