Re: Parallel tuplesort (for parallel B-Tree index creation) - Mailing list pgsql-hackers

From Robert Haas
Subject Re: Parallel tuplesort (for parallel B-Tree index creation)
Date
Msg-id CA+TgmoZHRwBm5WtdSy8KzsFdwJtYFXxBSLJgE8dQ-DSPuF8WeA@mail.gmail.com
Whole thread Raw
In response to Re: Parallel tuplesort (for parallel B-Tree index creation)  (Heikki Linnakangas <hlinnaka@iki.fi>)
Responses Re: Parallel tuplesort (for parallel B-Tree index creation)
List pgsql-hackers
On Thu, Sep 22, 2016 at 3:51 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
> It'd be good if you could overlap the final merges in the workers with the
> merge in the leader. ISTM it would be quite straightforward to replace the
> final tape of each worker with a shared memory queue, so that the leader
> could start merging and returning tuples as soon as it gets the first tuple
> from each worker. Instead of having to wait for all the workers to complete
> first.

If you do that, make sure to have the leader read multiple tuples at a
time from each worker whenever possible.  It makes a huge difference
to performance.  See bc7fcab5e36b9597857fa7e3fa6d9ba54aaea167.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



pgsql-hackers by date:

Previous
From: Heikki Linnakangas
Date:
Subject: Re: Parallel tuplesort (for parallel B-Tree index creation)
Next
From: Andrew Dunstan
Date:
Subject: pg_upgrade vs user created range type extension