Re: [WIP] speeding up GIN build with parallel workers - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: [WIP] speeding up GIN build with parallel workers
Date
Msg-id CAA4eK1+9S2vPvFPtyd7D3cKyYwEZuymNt2b-EEo4Y=dKqdrGyw@mail.gmail.com
Whole thread Raw
In response to Re: [WIP] speeding up GIN build with parallel workers  ("Constantin S. Pan" <kvapen@gmail.com>)
Responses Re: [WIP] speeding up GIN build with parallel workers
List pgsql-hackers
On Wed, Mar 16, 2016 at 5:41 AM, Constantin S. Pan <kvapen@gmail.com> wrote:
On Mon, 14 Mar 2016 08:42:26 -0400
David Steele <david@pgmasters.net> wrote:

> On 2/18/16 10:10 AM, Constantin S. Pan wrote:
> > On Wed, 17 Feb 2016 23:01:47 +0300
> > Oleg Bartunov <obartunov@gmail.com> wrote:
> >
> >> My feedback is (Mac OS X 10.11.3)
> >>
> >> set gin_parallel_workers=2;
> >> create index message_body_idx on messages using gin(body_tsvector);
> >> LOG:  worker process: parallel worker for PID 5689 (PID 6906) was
> >> terminated by signal 11: Segmentation fault
> >
> > Fixed this, try the new patch. The bug was in incorrect handling
> > of some GIN categories.
>
> Oleg, it looks like Constantin has updated to patch to address the
> issue you were seeing.  Do you have time to retest and review?
>
> Thanks,

Actually, there was some progress since. The patch is
attached.

1. Added another GUC parameter for changing the amount of
shared memory for parallel GIN workers.

2. Changed the way results are merged. It uses shared memory
message queue now.

3. Tested on some real data (GIN index on email message body
tsvectors). Here are the timings for different values of
'gin_shared_mem' and 'gin_parallel_workers' on a 4-CPU
machine. Seems 'gin_shared_mem' has no visible effect.

wnum mem(MB) time(s)
   0      16     247
   1      16     256


It seems from you data that with 1 worker, you are always seeing slowdown, have you investigated the reason of same?

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

pgsql-hackers by date:

Previous
From:
Date:
Subject: Re: [PROPOSAL] VACUUM Progress Checker.
Next
From: Amit Kapila
Date:
Subject: Pushdown target list below gather node (WAS Re: WIP: Upper planner pathification)