Thread: pgsql: When creating a large hash index, pre-sort the index entries by
pgsql: When creating a large hash index, pre-sort the index entries by
From
tgl@postgresql.org (Tom Lane)
Date:
Log Message:
-----------
When creating a large hash index, pre-sort the index entries by estimated
bucket number, so as to ensure locality of access to the index during the
insertion step. Without this, building an index significantly larger than
available RAM takes a very long time because of thrashing. On the other
hand, sorting is just useless overhead when the index does fit in RAM.
We choose to sort when the initial index size exceeds effective_cache_size.
This is a revised version of work by Tom Raney and Shreya Bhargava.
Modified Files:
--------------
pgsql/src/backend/access/hash:
Makefile (r1.14 -> r1.15)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/hash/Makefile?r1=1.14&r2=1.15)
hash.c (r1.99 -> r1.100)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/hash/hash.c?r1=1.99&r2=1.100)
hashpage.c (r1.73 -> r1.74)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/hash/hashpage.c?r1=1.73&r2=1.74)
pgsql/src/backend/access/nbtree:
nbtsort.c (r1.114 -> r1.115)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/nbtree/nbtsort.c?r1=1.114&r2=1.115)
pgsql/src/backend/utils/sort:
tuplesort.c (r1.81 -> r1.82)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/sort/tuplesort.c?r1=1.81&r2=1.82)
pgsql/src/include/access:
hash.h (r1.85 -> r1.86)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/access/hash.h?r1=1.85&r2=1.86)
pgsql/src/include/utils:
tuplesort.h (r1.28 -> r1.29)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/utils/tuplesort.h?r1=1.28&r2=1.29)
Added Files:
-----------
pgsql/src/backend/access/hash:
hashsort.c (r1.1)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/hash/hashsort.c?rev=1.1&content-type=text/x-cvsweb-markup)