Re: [PATCH] Incremental sort (was: PoC: Partial sort) - Mailing list pgsql-hackers

From James Coleman
Subject Re: [PATCH] Incremental sort (was: PoC: Partial sort)
Date
Msg-id CAAaqYe9mifZ9wv-Jm=+J3FWEOrXM-6SgY7z_YJhObfiG7dcxnA@mail.gmail.com
Whole thread Raw
In response to Re: [PATCH] Incremental sort (was: PoC: Partial sort)  (Tomas Vondra <tomas.vondra@2ndquadrant.com>)
Responses Re: [PATCH] Incremental sort (was: PoC: Partial sort)
List pgsql-hackers
On Sat, Apr 7, 2018 at 4:56 PM, Alexander Korotkov <
a(dot)korotkov(at)postgrespro(dot)ru> wrote:
> I agree with that. For bounded sort, attached patch now selects minimal
> group
> size as Min(DEFAULT_MIN_GROUP_SIZE, bound). That should improve
> "LIMIT small_number" case.

As I was working on some benchmarking I noticed that incremental sort
never seemed to switch into the top-n heapsort mode, which meant for
very large groups it significantly underperformed a regular sort since
it would have to spill to disk every time. Perhaps this indicates some
missing tests also.

I tracked that down to a missing case for IncrementalSortState in
ExecSetTupleBound and have updated the patched to correct the issue
(and confirmed it now properly switches sort modes).

That also means the optimization of choosing the min group size based
on bounds (if available) wasn't previously working.

I also haven't seen incremental sort used in any parallel plans,
though there seems to be some code intended to support it. I haven't
dug into really at all yet though, so can't comment further.

I'm attaching updated patch, and will reply separately with more
detailed comments on my current benchmarking work.

James Coleman

Attachment

pgsql-hackers by date:

Previous
From: Andrew Dunstan
Date:
Subject: Re: Plugging some testing holes
Next
From: Tomas Vondra
Date:
Subject: Re: Index Skip Scan