Re: [WIP] speeding up GIN build with parallel workers - Mailing list pgsql-hackers

From Dmitry Ivanov
Subject Re: [WIP] speeding up GIN build with parallel workers
Date
Msg-id 2503839.W1o5kfb8lG@abook
Whole thread Raw
In response to Re: [WIP] speeding up GIN build with parallel workers  ("Constantin S. Pan" <kvapen@gmail.com>)
Responses Re: [WIP] speeding up GIN build with parallel workers
Re: [WIP] speeding up GIN build with parallel workers
List pgsql-hackers
Hi Constantin,

I did a quick review of your patch, and here are my comments:

- This patch applies cleanly to the current HEAD (61d2ebdbf91).

- Code compiles without warnings.

- Currently there's no documentation regarding parallel gin build feature and 
provided GUC variables.

- Built indexes work seem to work normally.


Performance
-----------

I've made a few runs on my laptop (i7-4710HQ, default postgresql.conf), here 
are the results:
workers    avg. time (s)0            4124            1338            81

Looks like 8 workers & a backend do the job 5x times faster than a sole 
backend, which is good!


Code style
----------

There are some things that you've probably overlooked, such as:

> task->heap_oid = heap->rd_id;
> task->index_oid = index->rd_id;

You could replace direct access to 'rd_id' field with the RelationGetRelid 
macro.

> static void ginDumpEntry(GinState *ginstate,
>                          volatile WorkerResult *r

Parameter 'r' is unused, you could remove it.

Some of the functions and pieces of code that you've added do not comply to 
the formatting conventions, e. g.

> static int claimSomeBlocks(...
> static GinEntryStack *pushEntry(

>> // The message consists of 2 or 3 parts. iovec allows us to send them as

etc.

Keep up the good work!



pgsql-hackers by date:

Previous
From: Teodor Sigaev
Date:
Subject: Re: [PATCH] we have added support for box type in SP-GiST index
Next
From: Andres Freund
Date:
Subject: Re: Performance degradation in commit ac1d794