Thread: Index scan / Index cond limitation or ?
Hello people, I'm having trouble to persuade index scan to check all of the conditions I specify _inside_ index cond. That is, _some_ condition always get pushed out of index cond and applied later (which will often result, for my real table contents, in too many unwanted rows initially hit by index scan and hence randomly slow queries) An index with all relevant columns does exist of course. Here goes an example. create table foo ( id serial primary key, rec_time timestamp with time zone DEFAULT now(), some_value integer, some_data text ); CREATE INDEX foo_test ON foo (id, rec_time, some_value); set enable_seqscan = false; set enable_bitmapscan = true; explain select id from foo where true and rec_time > '2010-01-01 22:00:06' --and rec_time < '2010-10-14 23:59' and some_value in (1, 2) and id > 123 This one works perfectly as I want it (and note "and rec_time < ... " condition is commented out): Bitmap Heap Scan on foo (cost=13.18..17.19 rows=1 width=4) Recheck Cond: ((id > 123) AND (rec_time > '2010-01-01 22:00:06+03'::timestamp with time zone) AND (some_value = ANY ('{1,2}'::integer[]))) -> Bitmap Index Scan on foo_test (cost=0.00..13.18 rows=1 width=0) Index Cond: ((id > 123) AND (rec_time > '2010-01-01 22:00:06+03'::timestamp with time zone) AND (some_value = ANY ('{1,2}'::integer[])))" Now, as soon as I enable "and rec_time < ... " condition, I get the following: explain select id from foo where true and rec_time > '2010-01-01 22:00:06' and rec_time < '2010-10-14 23:59' and some_value in (1, 2) and id > 123 Bitmap Heap Scan on foo (cost=8.59..13.94 rows=1 width=4) Recheck Cond: ((id > 123) AND (rec_time > '2010-01-01 22:00:06+03'::timestamp with time zone) AND (rec_time < '2010-10-14 23:59:00+04'::timestamp with time zone)) Filter: (some_value = ANY ('{1,2}'::integer[])) -> Bitmap Index Scan on foo_test (cost=0.00..8.59 rows=2 width=0) Index Cond: ((id > 123) AND (rec_time > '2010-01-01 22:00:06+03'::timestamp with time zone) AND (rec_time < '2010-10-14 23:59:00+04'::timestamp with time zone)) So, "in (1, 2)" condition is not in Index Cond anymore! Why is that? How can I push it back? SELECT version(); PostgreSQL 8.3.1, compiled by Visual C++ build 1400 but the behaviour seems exactly the same in 9.0 (just checked it briefly). Thank you! Please CC me, I'm not on the list. Nikolai
Nikolai Zhubr <n-a-zhubr@yandex.ru> writes: > So, "in (1, 2)" condition is not in Index Cond anymore! Why is that? How > can I push it back? It thinks the indexscan condition is sufficiently selective already. An = ANY condition like that will force multiple index searches, one for each of the OR'd possibilities, so it's far from "free" to add it to the index condition. The planner doesn't think it's worth it. Perhaps on your real query it is, but there's not much point in debating about the behavior on this toy table; without realistic table sizes and up-to-date stats it's impossible to say whether that choice is correct or not. > SELECT version(); > PostgreSQL 8.3.1, compiled by Visual C++ build 1400 You really, really, really ought to be running 8.3.something-newer. We didn't put out the last 11 8.3.x bugfix updates just because we didn't have anything better to do. regards, tom lane
15.10.2010 22:29, Tom Lane: > Nikolai Zhubr<n-a-zhubr@yandex.ru> writes: >> So, "in (1, 2)" condition is not in Index Cond anymore! Why is that? How >> can I push it back? > > It thinks the indexscan condition is sufficiently selective already. > An = ANY condition like that will force multiple index searches, > one for each of the OR'd possibilities, so it's far from "free" to add > it to the index condition. The planner doesn't think it's worth it. Aha, ok. It makes sense then. Is this specific case (=ANY in index cond) described somewhere with reasonable detail? I always try to RTFM first and most of the time I can find pretty good hints in the regular manual already (sufficient as a starting point at least) but this specific topic seems to be somewhat mysterious. > Perhaps on your real query it is, but there's not much point in > debating about the behavior on this toy table; without realistic > table sizes and up-to-date stats it's impossible to say whether that > choice is correct or not. > >> SELECT version(); >> PostgreSQL 8.3.1, compiled by Visual C++ build 1400 > > You really, really, really ought to be running 8.3.something-newer. > We didn't put out the last 11 8.3.x bugfix updates just because > we didn't have anything better to do. Yes, I know, and I do appreciate the efforts of postgresql devels to create updates for older versions too. This server is internal-only (so it does not see any real world yet). Anyway, I hope to update everything to 9.0.1 soon. Thank you! Nikolai > > regards, tom lane > >