Thread: Re: Parallel index build for BRIN
Hi everyone, Thisthread doesn'tseem to have attractedattention, so let me try again. Two documentation pages claim that B-tree is the only access method that supports parallel building, which is no longer true. I propose to fix it in a way like this: diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index d54f9049569..b5b1580dee7 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -2835,7 +2835,7 @@ include_dir 'conf.d' Sets the maximum number of parallel workers that can be started by a single utility command. Currently, the parallel utility commands that support the use of parallel workers are - <command>CREATE INDEX</command> only when building a B-tree index, + <command>CREATE INDEX</command> when building a B-tree or BRIN index, and <command>VACUUM</command> without <literal>FULL</literal> option. Parallel workers are taken from the pool of processes established by <xref linkend="guc-max-worker-processes"/>, limited diff --git a/doc/src/sgml/ref/create_index.sgml b/doc/src/sgml/ref/create_index.sgml index 621bc0e253c..208389e8006 100644 --- a/doc/src/sgml/ref/create_index.sgml +++ b/doc/src/sgml/ref/create_index.sgml @@ -808,7 +808,7 @@ Indexes: leveraging multiple CPUs in order to process the table rows faster. This feature is known as <firstterm>parallel index build</firstterm>. For index methods that support building indexes - in parallel (currently, only B-tree), + in parallel (currently, B-tree and BRIN), <varname>maintenance_work_mem</varname> specifies the maximum amount of memory that can be used by each index build operation as a whole, regardless of how many worker processes were started. Thanks, Egor On 05.11.2024 12:12, Egor Rogov wrote: > Hi, > > Commit b4375717 introduced parallel CREATE INDEX for BRIN. I've > noticed that a couple of documentation pages need to be updated > accordingly. A small patch is attached. > > Thanks, > Egor
On 12/8/24 16:00, Egor Rogov wrote: > Hi, > > > On 17.11.2024 11:28, Egor Rogov wrote: >> Hi everyone, >> >> This thread doesn't seem to have attracted attention, so let me try >> again. Two documentation pages claim that B-tree is the only access >> method that supports parallel building, which is no longer true. I >> propose to fix it in a way like this: >> >> diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml >> index d54f9049569..b5b1580dee7 100644 >> --- a/doc/src/sgml/config.sgml >> +++ b/doc/src/sgml/config.sgml >> @@ -2835,7 +2835,7 @@ include_dir 'conf.d' >> Sets the maximum number of parallel workers that can be >> started by a single utility command. Currently, the parallel >> utility commands that support the use of parallel workers are >> - <command>CREATE INDEX</command> only when building a B-tree >> index, >> + <command>CREATE INDEX</command> when building a B-tree or >> BRIN index, >> and <command>VACUUM</command> without <literal>FULL</literal> >> option. Parallel workers are taken from the pool of processes >> established by <xref linkend="guc-max-worker-processes"/>, >> limited >> diff --git a/doc/src/sgml/ref/create_index.sgml b/doc/src/sgml/ref/ >> create_index.sgml >> index 621bc0e253c..208389e8006 100644 >> --- a/doc/src/sgml/ref/create_index.sgml >> +++ b/doc/src/sgml/ref/create_index.sgml >> @@ -808,7 +808,7 @@ Indexes: >> leveraging multiple CPUs in order to process the table rows faster. >> This feature is known as <firstterm>parallel index >> build</firstterm>. For index methods that support building indexes >> - in parallel (currently, only B-tree), >> + in parallel (currently, B-tree and BRIN), >> <varname>maintenance_work_mem</varname> specifies the maximum >> amount of memory that can be used by each index build operation as >> a whole, regardless of how many worker processes were started. > > > I've spotted another mention of B-tree being the only AM that supports > parallel builds: comment in src/backend/catalog/index.c. As this mention > is not visible to the users, I'd propose removing it altogether rather > than fixing it. Updated patch is attached. > Thanks for noticing this and the patches. You're right, this should have been updated with the BRIN parallel builds. I'll get this committed sometime the week. regards -- Tomas Vondra
On 12/9/24 19:54, Tomas Vondra wrote: > On 12/8/24 16:00, Egor Rogov wrote: >> Hi, >> >> ... >> >> I've spotted another mention of B-tree being the only AM that supports >> parallel builds: comment in src/backend/catalog/index.c. As this mention >> is not visible to the users, I'd propose removing it altogether rather >> than fixing it. Updated patch is attached. >> > > Thanks for noticing this and the patches. You're right, this should have > been updated with the BRIN parallel builds. I'll get this committed > sometime the week. > I've pushed the doc fix, and backpatched it to PG 17. Thanks for the patience! -- Tomas Vondra
On 16.12.2024 21:24, Tomas Vondra wrote: > On 12/9/24 19:54, Tomas Vondra wrote: >> On 12/8/24 16:00, Egor Rogov wrote: >>> Hi, >>> >>> ... >>> >>> I've spotted another mention of B-tree being the only AM that supports >>> parallel builds: comment in src/backend/catalog/index.c. As this mention >>> is not visible to the users, I'd propose removing it altogether rather >>> than fixing it. Updated patch is attached. >>> >> Thanks for noticing this and the patches. You're right, this should have >> been updated with the BRIN parallel builds. I'll get this committed >> sometime the week. >> > I've pushed the doc fix, and backpatched it to PG 17. Thanks so much, Tomas! Please note that the comment in src/backend/catalog/index.c remains unchanged: --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -2988,8 +2988,7 @@ index_build(Relation heapRelation, Assert(PointerIsValid(indexRelation->rd_indam->ambuildempty)); /* - * Determine worker process details for parallel CREATE INDEX. Currently, - * only btree has support for parallel builds. + * Determine worker process details for parallel CREATE INDEX. * * Note that planner considers parallel safety for us. */
On 12/17/24 07:40, Egor Rogov wrote: > On 16.12.2024 21:24, Tomas Vondra wrote: > >> On 12/9/24 19:54, Tomas Vondra wrote: >>> On 12/8/24 16:00, Egor Rogov wrote: >>>> Hi, >>>> >>>> ... >>>> >>>> I've spotted another mention of B-tree being the only AM that supports >>>> parallel builds: comment in src/backend/catalog/index.c. As this >>>> mention >>>> is not visible to the users, I'd propose removing it altogether rather >>>> than fixing it. Updated patch is attached. >>>> >>> Thanks for noticing this and the patches. You're right, this should have >>> been updated with the BRIN parallel builds. I'll get this committed >>> sometime the week. >>> >> I've pushed the doc fix, and backpatched it to PG 17. > > > Thanks so much, Tomas! > > Please note that the comment in src/backend/catalog/index.c remains > unchanged: > > > --- a/src/backend/catalog/index.c > +++ b/src/backend/catalog/index.c > @@ -2988,8 +2988,7 @@ index_build(Relation heapRelation, > Assert(PointerIsValid(indexRelation->rd_indam->ambuildempty)); > > /* > - * Determine worker process details for parallel CREATE INDEX. > Currently, > - * only btree has support for parallel builds. > + * Determine worker process details for parallel CREATE INDEX. > * > * Note that planner considers parallel safety for us. > */ > Oh, thanks for noticing that. I'm not sure why I missed that, I must have used the first patch by mistake. Anyway, I found another obsolete comment in planner.c, so I fixed that too and pushed. Thanks -- Tomas Vondra