Thread: Latest Bitmap Index patch
Hi all, It seems that my previous email to pgsql-hackers about the latest bitmap index patch did not go through. Please find the latest patch in the attachment. Any suggestions and comments are appreciated. This patch is generated against the PostgreSQL CVS HEAD. This patch includes: (Gavin, if I miss anything, please let me know.) (1) Code style changes. (2) Fixed bugs on the multi-column support, the race condition problem if two backends try to insert the same new value (Thank Heikki for reporting this), and some others. (3) Added the stream bitmap implementation. (4) Added performance improvements for creating a bitmap index. (5) Added documentation. There are several TODO items: (1) Currently, the stream bitmap implementation does not support ArrayOpExpr properly. That is, if you have a query like 'select * from T where T.a in (1,2);' and there is a bitmap index on T.a. If the planner chooses to use the plan using the bitmap scan, the current stream bitmap implementation will return an incorrect result. I have been thinking about the following two options. Either one requires quite some changes. I would like to get some comments first before I make these changes. The options are (a) Disable the ArrayOpExpr support while creating an execution plan for the stream bitmap case -- use BitmapOr instead. (b) Add a new OR logic into nodeBitmapIndexscan.c. The current OR logic supporting hash bitmaps does not work for stream bitmaps, which handles keys in an ArrayOpExpr one by one, which stream bitmaps require to handle keys at the same time. If you have any comments on this, or any suggestions on other options, please let me know. Note: the regression test on 'create_index' will fail because I added a check on this for the bitmap index. (2) Add the support for range queries, such as 'where a<10'. (3) Add a better implementation for vacuuming. Thanks, Jie
Attachment
Hi all, It seems that my previous email to pgsql-hackers about the latest bitmap index patch did not go through. Please find the latest patch in the attachment. Any suggestions and comments are appreciated. This patch is generated against the PostgreSQL CVS HEAD. This patch includes: (Gavin, if I miss anything, please let me know.) (1) Code style changes. (2) Fixed bugs on the multi-column support, the race condition problem if two backends try to insert the same new value (Thank Heikki for reporting this), and some others. (3) Added the stream bitmap implementation. (4) Added performance improvements for creating a bitmap index. (5) Added documentation. There are several TODO items: (1) Currently, the stream bitmap implementation does not support ArrayOpExpr properly. That is, if you have a query like 'select * from T where T.a in (1,2);' and there is a bitmap index on T.a. If the planner chooses to use the plan using the bitmap scan, the current stream bitmap implementation will return an incorrect result. I have been thinking about the following two options. Either one requires quite some changes. I would like to get some comments first before I make these changes. The options are (a) Disable the ArrayOpExpr support while creating an execution plan for the stream bitmap case -- use BitmapOr instead. (b) Add a new OR logic into nodeBitmapIndexscan.c. The current OR logic supporting hash bitmaps does not work for stream bitmaps, which handles keys in an ArrayOpExpr one by one, which stream bitmaps require to handle keys at the same time. If you have any comments on this, or any suggestions on other options, please let me know. Note: the regression test on 'create_index' will fail because I added a check on this for the bitmap index. (2) Add the support for range queries, such as 'where a<10'. (3) Add a better implementation for vacuuming. Thanks, Jie
Attachment
On Mon, 18 Sep 2006, Jie Zhang wrote: > Hi all, > > It seems that my previous email to pgsql-hackers about the latest bitmap > index patch did not go through. Please find the latest patch in the > attachment. Any suggestions and comments are appreciated. > > This patch is generated against the PostgreSQL CVS HEAD. This patch > includes: (Gavin, if I miss anything, please let me know.) > > (1) Code style changes. > > (2) Fixed bugs on the multi-column support, the race condition problem if > two backends try to insert the same new value (Thank Heikki for reporting > this), and some others. > > (3) Added the stream bitmap implementation. > > (4) Added performance improvements for creating a bitmap index. > > (5) Added documentation. Other bugs to do with page layout and WAL recovery. Other TODOs are to add a bitmap specific cost estimate and to take another pass over the code to push things into the usual backend style. Gavin