Re: Threaded Sorting - Mailing list pgsql-hackers

From Hans-Jürgen Schönig
Subject Re: Threaded Sorting
Date
Msg-id 3D9DA865.6060400@cybertec.at
Whole thread Raw
In response to Threaded Sorting  (Hans-Jürgen Schönig <postgres@cybertec.at>)
List pgsql-hackers
Greg Copeland wrote:

>I wouldn't hold your breath for any form of threading.  Since PostgreSQL
>is process based, you might consider having a pool of sort processes
>which address this but I doubt you'll get anywhere talking about threads
>here.
>
>Greg
>
>  
>

I came across the problem yesterday. We thought about SMP and did some 
tests on huge tables. The postmaster was running full speed to get the 
stuff sorted - even on an IDE system.
I asked my friends who are doing a lot of work with Oracle on huge SMP 
machines. I was told that Oracle has a mechanism which can run efficient 
sorts on SMP machines. It seems to speed up sorting a lot.

If we could reduce the time needed to build up an index by 25% it would 
be a wonderful thing. Just think of a scenario:
1 thread: 24 hours
many threads: 18 hours

We  could gain 6 hours which is a LOT.
We have many people running PostgreSQL on systems having wonderful IO 
systems - in this case IO is not the bottleneck anymore.

I had a brief look at the code used for sorting. It is very well 
documented so maybe it is worth thinking about a parallel algorithm.

When talking about threads: A pool of processes for sorting? Maybe this 
could be useful but I doubt if it the best solution to avoid overhead.
Somewhere in the TODO it says that there will be experiments with a 
threaded backend. This make me think that threads are not a big no no.
   Hans

-- 
*Cybertec Geschwinde u Schoenig*
Ludo-Hartmannplatz 1/14, A-1160 Vienna, Austria
Tel: +43/1/913 68 09; +43/664/233 90 75
www.postgresql.at <http://www.postgresql.at>, cluster.postgresql.at 
<http://cluster.postgresql.at>, www.cybertec.at 
<http://www.cybertec.at>, kernel.cybertec.at <http://kernel.cybertec.at>



pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Analysis of ganged WAL writes
Next
From: Hans-Jürgen Schönig
Date:
Subject: Re: Threaded Sorting