Re: Streaming relation data out of order - Mailing list pgsql-hackers

From Thomas Munro
Subject Re: Streaming relation data out of order
Date
Msg-id CA+hUKG+53X=26HwQrykdzwtt-j+jWGU3WRa8VEM7AFyyQK-Atg@mail.gmail.com
Whole thread Raw
In response to Re: Streaming relation data out of order  (Robert Haas <robertmhaas@gmail.com>)
Responses Re: Streaming relation data out of order
List pgsql-hackers
Here is a more fleshed out version of this concept patch, now that we
have lots of streams wired up to try it with.  Bitmap Heap Scan seemed
to be a good candidate.

postgres=# create table t (a int unique, b int unique);
CREATE TABLE
postgres=# insert into t select generate_series(1, 100000),
generate_series(1, 100000);
INSERT 0 100000
postgres=# select ctid from t where (a between 1 and 8000 or b between
1 and 8000) and ctid::text like '%,1)';
  ctid
--------
 (0,1)
 (1,1)
 (2,1)
... pages in order ...
 (33,1)
 (34,1)
 (35,1)
(36 rows)
postgres=# select pg_buffercache_evict(bufferid) from pg_buffercache
where relfilenode = pg_relation_filenode('t'::regclass) and
relblocknumber % 3 != 2;
... some pages being kicked out to create a mixed hit/miss scan ...
postgres=# select ctid from t where (a between 1 and 8000 or b between
1 and 8000) and ctid::text like '%,1)';
  ctid
--------
 (0,1)
 (1,1)
 (2,1)
... order during early distance ramp-up ...
 (12,1)
 (20,1) <-- chaos reigns
 (23,1)
 (17,1)
 (26,1)
 (13,1)
 (29,1)
...
 (34,1)
(36 rows)

One weird issue, not just with reordering, is that read_stream.c's
distance cools off too easily with some simple test patterns.  Start
at 1, double on miss, subtract one on hit, repeat, and you can be
trapped alternating between 1 and 2 when you'd certainly benefit from
IO concurrency and also reordering.  It may need a longer memory.
That seemed like too artificial a problem to worry about for v18, but
it's why I used relblocknumber % 3 != 2 and not eg relblocknumber % 2
!= 1 above.

Attachment

pgsql-hackers by date:

Previous
From: John Naylor
Date:
Subject: Re: Feature freeze
Next
From: Luc Vlaming Hummel
Date:
Subject: Re: Assertion with aborted UPDATE in subtransaction