Re: Using quicksort and a merge step to significantly improve on tuplesort's single run "external sort" - Mailing list pgsql-hackers

From Peter Geoghegan
Subject Re: Using quicksort and a merge step to significantly improve on tuplesort's single run "external sort"
Date
Msg-id CAM3SWZT4zVfo864x9oaJudquotGYMccr3F3VBNmSyYQY11oRqg@mail.gmail.com
Whole thread Raw
In response to Re: Using quicksort and a merge step to significantly improve on tuplesort's single run "external sort"  (Greg Stark <stark@mit.edu>)
List pgsql-hackers
On Thu, Jul 30, 2015 at 4:26 AM, Greg Stark <stark@mit.edu> wrote:
> I'm a bit confused where the big win comes from though. Is what's going on
> that the external sort only exceeded memory by a small amount  so nearly all
> the tuples are still in memory?

Yes, that's why this can be much faster just as the work_mem threshold
is crossed. You get an "almost internal" sort, which means you can
mostly quicksort, and you can avoid dumping most tuples. It's still a
pretty nice win when less than half of tuples fit in memory, though --
just not as nice. Below that, the optimization isn't used.

-- 
Peter Geoghegan



pgsql-hackers by date:

Previous
From: Peter Geoghegan
Date:
Subject: Re: Using quicksort and a merge step to significantly improve on tuplesort's single run "external sort"
Next
From: Peter Geoghegan
Date:
Subject: Re: Using quicksort and a merge step to significantly improve on tuplesort's single run "external sort"