Re: [WIP] speeding up GIN build with parallel workers - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: [WIP] speeding up GIN build with parallel workers
Date
Msg-id CAA4eK1KizkcMVdJLgg4EQDJRTbbhXcnxfmOMFaHutbw7wmsXbQ@mail.gmail.com
Whole thread Raw
In response to Re: [WIP] speeding up GIN build with parallel workers  ("Constantin S. Pan" <kvapen@gmail.com>)
Responses Re: [WIP] speeding up GIN build with parallel workers
List pgsql-hackers
On Wed, Mar 16, 2016 at 7:50 PM, Constantin S. Pan <kvapen@gmail.com> wrote:
>
> On Wed, 16 Mar 2016 18:08:38 +0530
> Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> >
> > Why backend just waits, why can't it does the same work as any worker
> > does?  In general, for other parallelism features the backend also
> > behaves the same way as worker in producing the results if the
> > results from workers is not available.
>
> We can make backend do the same work as any worker, but that
> will complicate the code for less than 2 % perfomance boost.

Why do you think it will be just 2%?  I think for single worker case, it should be much more as the master backend will be less busy in consuming tuples from tuple queue.  I can't say much about code-complexity, as I haven't yet looked carefully at the logic of patch, but we didn't find much difficulty while doing it for parallel scans.  One of the commit which might help you in understanding how currently heap scans are parallelised is ee7ca559fcf404f9a3bd99da85c8f4ea9fbc2e92, you can see if that can help you in anyway for writing a generic API for Gin parallel builds. 


With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

pgsql-hackers by date:

Previous
From: Oskari Saarenmaa
Date:
Subject: Re: Show dropped users' backends in pg_stat_activity
Next
From: Petr Jelinek
Date:
Subject: Re: WIP: Access method extendability