Thread: [HACKERS] [PATCH] Incremental sort (was: PoC: Partial sort)

On Sat, Feb 18, 2017 at 4:01 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:
> I decided to start new thread for this patch for following two reasons.
>  * It's renamed from "Partial sort" to "Incremental sort" per suggestion by
> Robert Haas [1].  New name much better characterizes the essence of
> algorithm.
>  * I think it's not PoC anymore.  Patch received several rounds of review
> and now it's in the pretty good shape.
>
> Attached revision of patch has following changes.
>  * According to review [1], two new path and plan nodes are responsible for
> incremental sort: IncSortPath and IncSort which are inherited from SortPath
> and Sort correspondingly.  That allowed to get rid of set of hacks with
> minimal code changes.
>  * According to review [1] and comment [2], previous tuple is stored in
> standalone tuple slot of SortState rather than just HeapTuple.
>  * New GUC parameter enable_incsort is introduced to control planner ability
> to choose incremental sort.
>  * Test of postgres_fdw with not pushed down cross join is corrected.  It
> appeared that with incremental sort such query is profitable to push down.
> I changed ORDER BY columns so that index couldn't be used.  I think this
> solution is more elegant than setting enable_incsort = off.

I usually advocate for spelling things out instead of abbreviating, so
I guess I'll stay true to form here and suggest that abbreviating
incremental to inc doesn't seem like a great idea.  Is that sort
incrementing, incremental, incredible, incautious, or incorporated?

The first hunk in the patch, a change in the postgres_fdw regression
test output, looks an awful lot like a bug: now the query that
formerly returned various different numbers is returning all zeroes.
It might not actually be a bug, because you've also changed the test
query (not sure why), but anyway the new regression test output that
is all zeroes seems less useful for catching bugs in, say, the
ordering of the results than the old output where the different rows
were different.

I don't know of any existing cases where the same executor file is
responsible for executing more than 1 different type of executor node.
I was imagining a more-complete separation of the new executor node.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Feb 27, 2017 at 8:29 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:

This patch needs to be rebased.

1. It fails while applying as below

patching file src/test/regress/expected/sysviews.out
Hunk #1 FAILED at 70.
1 out of 1 hunk FAILED -- saving rejects to file
src/test/regress/expected/sysviews.out.rej
patching file src/test/regress/sql/inherit.sql

2. Also, there are compilation errors due to new commits.

-fwrapv -fexcess-precision=standard -O2 -I../../../../src/include
-D_GNU_SOURCE   -c -o createplan.o createplan.c
createplan.c: In function ‘create_gather_merge_plan’:
createplan.c:1510:11: warning: passing argument 3 of ‘make_sort’ makes
integer from pointer without a cast [enabled by default]          gm_plan->nullsFirst);          ^
createplan.c:235:14: note: expected ‘int’ but argument is of type ‘AttrNumber *’static Sort *make_sort(Plan *lefttree,
intnumCols, int skipCols,             ^ 
createplan.c:1510:11: warning: passing argument 4 of ‘make_sort’ from
incompatible pointer type [enabled by default]          gm_plan->nullsFirst);

--
Thanks and Regards
Mithun C Y
EnterpriseDB: http://www.enterprisedb.com

On 03/20/2017 11:33 AM, Alexander Korotkov wrote:
> Please, find rebased patch in the attachment.

I had a quick look at this.

* I'd love to have an explanation of what an Incremental Sort is, in the 
file header comment for nodeIncrementalSort.c.

* I didn't understand the maxMem stuff in tuplesort.c. The comments 
there use the phrase "on-disk memory", which seems like an oxymoron. 
Also, "maximum status" seems weird, as it assumes that there's a natural 
order to the states.

* In the below example, the incremental sort is significantly slower 
than the Seq Scan + Sort you get otherwise:

create table foo (a int4, b int4, c int4);
insert into sorttest select g, g, g from generate_series(1, 1000000) g;
vacuum foo;
create index i_sorttest on sorttest (a, b, c);
set work_mem='100MB';


postgres=# explain select count(*) from (select * from sorttest order by 
a, c) as t;                                              QUERY PLAN 

------------------------------------------------------------------------------------------------------- Aggregate
(cost=138655.68..138655.69rows=1 width=8)   ->  Incremental Sort  (cost=610.99..124870.38 rows=1102824 width=12)
Sort Key: sorttest.a, sorttest.c         Presorted Key: sorttest.a         ->  Index Only Scan using i_sorttest on
sorttest
 
(cost=0.43..53578.79 rows=1102824 width=12)
(5 rows)

Time: 0.409 ms
postgres=# select count(*) from (select * from sorttest order by a, c) as t;  count
--------- 1000000
(1 row)

Time: 387.091 ms


postgres=# explain select count(*) from (select * from sorttest order by 
a, c) as t;                                  QUERY PLAN 

------------------------------------------------------------------------------- Aggregate  (cost=130063.84..130063.85
rows=1width=8)   ->  Sort  (cost=115063.84..117563.84 rows=1000000 width=12)         Sort Key: sorttest.a, sorttest.c
     ->  Seq Scan on sorttest  (cost=0.00..15406.00 rows=1000000 
 
width=12)
(4 rows)

Time: 0.345 ms
postgres=# select count(*) from (select * from sorttest order by a, c) as t;  count
--------- 1000000
(1 row)

Time: 231.668 ms

According to 'perf', 85% of the CPU time is spent in ExecCopySlot(). To 
alleviate that, it might be worthwhile to add a special case for when 
the group contains exactly one group, and not put the tuple to the 
tuplesort in that case. Or if we cannot ensure that the Incremental Sort 
is actually faster, the cost model should probably be smarter, to avoid 
picking an incremental sort when it's not a win.

- Heikki

Hi Alexander,

On 3/20/17 10:19 AM, Heikki Linnakangas wrote:
> On 03/20/2017 11:33 AM, Alexander Korotkov wrote:
>> Please, find rebased patch in the attachment.
>
> I had a quick look at this.

<...>

> According to 'perf', 85% of the CPU time is spent in ExecCopySlot(). To
> alleviate that, it might be worthwhile to add a special case for when
> the group contains exactly one group, and not put the tuple to the
> tuplesort in that case. Or if we cannot ensure that the Incremental Sort
> is actually faster, the cost model should probably be smarter, to avoid
> picking an incremental sort when it's not a win.

This thread has been idle for over a week.  Please respond with a new 
patch by 2017-03-30 00:00 AoE (UTC-12) or this submission will be marked 
"Returned with Feedback".

-- 
-David
david@pgmasters.net

On 2017-03-29 00:17:02 +0300, Alexander Korotkov wrote:
> On Tue, Mar 28, 2017 at 5:27 PM, David Steele <david@pgmasters.net> wrote:
> 
> > Hi Alexander,
> >
> > On 3/20/17 10:19 AM, Heikki Linnakangas wrote:
> >
> >> On 03/20/2017 11:33 AM, Alexander Korotkov wrote:
> >>
> >>> Please, find rebased patch in the attachment.
> >>>
> >>
> >> I had a quick look at this.
> >>
> >
> > <...>
> >
> > According to 'perf', 85% of the CPU time is spent in ExecCopySlot(). To
> >> alleviate that, it might be worthwhile to add a special case for when
> >> the group contains exactly one group, and not put the tuple to the
> >> tuplesort in that case. Or if we cannot ensure that the Incremental Sort
> >> is actually faster, the cost model should probably be smarter, to avoid
> >> picking an incremental sort when it's not a win.
> >>
> >
> > This thread has been idle for over a week.  Please respond with a new
> > patch by 2017-03-30 00:00 AoE (UTC-12) or this submission will be marked
> > "Returned with Feedback".

> Thank you for reminder!

I've just done so.  Please resubmit once updated, it's a cool feature.

- Andres

On April 3, 2017 12:03:56 PM PDT, Alexander Korotkov <a.korotkov@postgrespro.ru> wrote:
>On Mon, Apr 3, 2017 at 9:34 PM, Andres Freund <andres@anarazel.de>
>wrote:
>
>> On 2017-03-29 00:17:02 +0300, Alexander Korotkov wrote:
>> > On Tue, Mar 28, 2017 at 5:27 PM, David Steele <david@pgmasters.net>
>> wrote:
>> > > On 3/20/17 10:19 AM, Heikki Linnakangas wrote:
>> > >
>> > >> On 03/20/2017 11:33 AM, Alexander Korotkov wrote:
>> > >>
>> > >>> Please, find rebased patch in the attachment.
>> > >>>
>> > >>
>> > >> I had a quick look at this.
>> > >>
>> > >
>> > > <...>
>> > >
>> > > According to 'perf', 85% of the CPU time is spent in
>ExecCopySlot(). To
>> > >> alleviate that, it might be worthwhile to add a special case for
>when
>> > >> the group contains exactly one group, and not put the tuple to
>the
>> > >> tuplesort in that case. Or if we cannot ensure that the
>Incremental
>> Sort
>> > >> is actually faster, the cost model should probably be smarter,
>to
>> avoid
>> > >> picking an incremental sort when it's not a win.
>> > >>
>> > >
>> > > This thread has been idle for over a week.  Please respond with a
>new
>> > > patch by 2017-03-30 00:00 AoE (UTC-12) or this submission will be
>> marked
>> > > "Returned with Feedback".
>>
>> > Thank you for reminder!
>>
>> I've just done so.  Please resubmit once updated, it's a cool
>feature.
>>
>
>Thank you!
>I already sent version of patch after David's reminder.
>Please find rebased patch in the attachment.

Cool. I think that's still a bit late for v10?

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Hi,

On 2017-04-04 00:04:09 +0300, Alexander Korotkov wrote:
> > >Thank you!
> > >I already sent version of patch after David's reminder.
> > >Please find rebased patch in the attachment.
> >
> > Cool. I think that's still a bit late for v10?
> >
> 
> I don't know.  ISTM, that I addressed all the issues raised by reviewers.
> Also, this patch is pending since late 2013.  It would be very nice to
> finally get it in...

To me this hasn't gotten even remotely enough performance evaluation.
And I don't think it's fair to characterize it as pending since 2013,
given it was essentially "waiting on author" for most of that.

Greetings,

Andres Freund

On Mon, Apr 3, 2017 at 5:09 PM, Andres Freund <andres@anarazel.de> wrote:
> To me this hasn't gotten even remotely enough performance evaluation.
> And I don't think it's fair to characterize it as pending since 2013,
> given it was essentially "waiting on author" for most of that.

This is undeniably a patch which has been kicking around for a lot of
time without getting a lot of attention, and if it just keeps getting
punted down the road, it's never going to become committable.
Alexander's questions upthread about what decisions the committer who
took an interest (Heikki) would prefer never really got an answer, for
example.  I don't deny that there may be some work left to do here,
but I think blaming the author for a week's delay when this has been
ignored so often for so long is unfair.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 2017-04-03 22:18:21 -0400, Robert Haas wrote:
> On Mon, Apr 3, 2017 at 5:09 PM, Andres Freund <andres@anarazel.de> wrote:
> > To me this hasn't gotten even remotely enough performance evaluation.
> > And I don't think it's fair to characterize it as pending since 2013,
> > given it was essentially "waiting on author" for most of that.
> 
> This is undeniably a patch which has been kicking around for a lot of
> time without getting a lot of attention, and if it just keeps getting
> punted down the road, it's never going to become committable.

Indeed, it's old.  And it hasn't gotten enough timely feedback.

But I don't think the wait time can meaningfully be measured by
subtracting two dates:
The first version of the patch, as a PoC, has been posted 2013-12-14,
which then got a good amount of feedback & revisions, and then stalled
till 2014-07-12.  There a few back-and forths yielded a new version.
From 2014-09-15 till 2015-10-16 the patch stalled, waiting on its
author.  That version had open todos ([1]), as had the version from
2016-03-13 [2], which weren't addressed 2016-03-30 - unfortunately that
was pretty much when the tree was frozen.  2016-09-13 a rebased patch
was sent, some minor points were raised 2016-10-02 (unaddressed), a
larger review was done 2016-12-01 ([5]), unaddressed till 2017-02-18.
At that point we're in this thread.

There's obviously some long waiting-on-author periods in there.  And
some long needs-review periods.


> Alexander's questions upthread about what decisions the committer who
> took an interest (Heikki) would prefer never really got an answer, for
> example.  I don't deny that there may be some work left to do here,
> but I think blaming the author for a week's delay when this has been
> ignored so often for so long is unfair.

I'm not trying to blame Alexander for a week's worth of delay, at all.
It's just that, well, we're past the original code-freeze date, three
days before the "final" code freeze. I don't think fairness is something
we can achieve at this point :(.  Given the risk of regressions -
demonstrated in this thread although partially adressed - and the very
limited amount of benchmarking done, it seems unlikely that this is
going to be merged.

Regards,

Andres


[1] http://archives.postgresql.org/message-id/CAPpHfdvhwMsG69exCRUGK3ms-ng0PSPcucH5FU6tAaM-qL-1%2Bw%40mail.gmail.com
[2] http://archives.postgresql.org/message-id/CAPpHfdvzjYGLTyA-8ib8UYnKLPrewd9Z%3DT4YJNCRWiHWHHweWw%40mail.gmail.com
[3] http://archives.postgresql.org/message-id/CAPpHfdtCcHZ-mLWzsFrRCvHpV1LPSaOGooMZ3sa40AkwR=7ouQ@mail.gmail.com
[4] http://archives.postgresql.org/message-id/CAPpHfdvj1Tdi2WA64ZbBp5-yG-uzaRXzk3K7J7zt-cRX6YSd0A@mail.gmail.com
[5] http://archives.postgresql.org/message-id/CA+TgmoZapyHRm7NVyuyZ+yAV=U1a070BOgRe7PkgyrAegR4JDA@mail.gmail.com
[6] http://archives.postgresql.org/message-id/CAPpHfds1waRZ=NOmueYq0sx1ZSCnt+5QJvizT8ndT2=etZEeAQ@mail.gmail.com

On Wed, Apr 26, 2017 at 8:39 AM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:
> That appears to be wrong.  I intended to make cost_sort prefer plain sort
> over incremental sort for this dataset size.  But, that appears to be not
> always right solution.  Quick sort is so fast only on presorted data.

As you may know, I've often said that the precheck for sorted input
added to our quicksort implementation by a3f0b3d is misguided. It
sometimes throws away a ton of work if the presorted input isn't
*perfectly* presorted. This happens when the first out of order tuple
is towards the end of the presorted input.

I think that it isn't fair to credit our qsort with doing so well on a
100% presorted case, because it doesn't do the necessary bookkeeping
to not throw that work away completely in certain important cases.

-- 
Peter Geoghegan

VMware vCenter Server
https://www.vmware.com/

On Wed, Apr 26, 2017 at 10:10 AM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:
> OK, I get it.  Our qsort is so fast not only on 100% presorted case.
> However, that doesn't change many things in context of incremental sort.

The important point is to make any presorted test case only ~99%
presorted, so as to not give too much credit to the "high risk"
presort check optimization.

The switch to insertion sort that we left in (not the bad one removed
by a3f0b3d -- the insertion sort that actually comes from the B&M
paper) does "legitimately" make sorting faster with presorted cases.

-- 
Peter Geoghegan

VMware vCenter Server
https://www.vmware.com/

On Wed, Apr 26, 2017 at 11:39 AM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:
> But I'd like to make incremental sort not slower than quicksort in case of
> presorted data.  New idea about it comes to my mind.  Since cause of
> incremental sort slowness in this case is too frequent reset of tuplesort,
> then what if we would artificially put data in larger groups.  Attached
> revision of patch implements this: it doesn't stop to accumulate tuples to
> tuplesort until we have MIN_GROUP_SIZE tuples.
>
> Now, incremental sort is not slower than quicksort.  And this seems to be
> cool.
> However, in the LIMIT case we will pay the price of fetching some extra
> tuples from outer node.  But, that doesn't seem to hurt us too much.
>
> Any thoughts?

Nice idea.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, May 5, 2017 at 11:13 AM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:
> Incremental sort is faster in vast majority of cases.  It appears to be
> slower only when whose dataset is one sort group.  In this case incremental
> sort is useless, and it should be considered as misuse of incremental sort.
> Slowdown is related to the fact that we anyway have to do extra comparisons,
> unless we somehow push our comparison result into qsort itself and save some
> cpu cycles (but that would be unreasonable break of encapsulation).  Thus,
> in such cases regression seems to be inevitable anyway.  I think we could
> evade this regression during query planning.  If we see that there would be
> only few groups, we should choose plain sort instead of incremental sort.

I'm sorry that I don't have time to review this in detail right now,
but it sounds like you are doing good work to file down cases where
this might cause regressions, which is great.  Regarding the point in
the paragraph above, I'd say that it's OK for the planner to be
responsible for picking between Sort and Incremental Sort in some way.
It is, after all, the planner's job to decide between different
strategies for executing the same query and, of course, sometimes it
will be wrong, but that's OK as long as it's not wrong too often (or
by too much, hopefully).  It may be a little difficult to get this
right, though, because I'm not sure that the information you need
actually exists (or is reliable).  For example, consider the case
where we need to sort 100m rows and there are 2 groups.  If 1 group
contains 1 row and the other group contains all of the rest, there is
really no point in an incremental sort.  On the other hand, if each
group contains 50m rows and we can get the data presorted by the
grouping column, there might be a lot of point to an incremental sort,
because two 50m-row sorts might be a lot cheaper than one 100m sort.
More generally, it's quite easy to imagine situations where the
individual groups can be quicksorted but sorting all of the rows
requires I/O, even when the number of groups isn't that big.  On the
other hand, the real sweet spot for this is probably the case where
the number of groups is very large, with many single-row groups or
many groups with just a few rows each, so if we can at least get this
to work in those cases that may be good enough.  On the third hand,
when costing aggregation, I think we often underestimate the number of
groups and there might well be similar problems here.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Oct 2, 2017 at 12:37 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:
> I've applied patch on top of c12d570f and rerun the same benchmarks.
> CSV-file with results is attached.  There is no dramatical changes.  There
> is still minority of performance regression cases while majority of cases
> has improvement.

Yes, I think these results look pretty good.  But are these times in
seconds?  You might need to do some testing with bigger sorts.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Alexander Korotkov <a.korotkov@postgrespro.ru> wrote:

> Patch rebased to current master is attached. I'm going to improve my testing script and post new results.

I wanted to review this patch but incremental-sort-8.patch fails to apply. Can
you please rebase it again?

--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at

Antonin Houska <ah@cybertec.at> wrote:

> Alexander Korotkov <a.korotkov@postgrespro.ru> wrote:
>
> > Patch rebased to current master is attached. I'm going to improve my testing script and post new results.
>
> I wanted to review this patch but incremental-sort-8.patch fails to apply. Can
> you please rebase it again?

I could find the matching HEAD quite easily (9b6cb46), so following are my comments:

* cost_sort()

** "presorted_keys" missing in the prologue

** when called from label_sort_with_costsize(), 0 is passed for  "presorted_keys". However label_sort_with_costsize()
cansometimes be  called on an IncrementalSort, in which case there are some "presorted  keys". See
create_mergejoin_plan()for example. (IIUC this should only  make EXPLAIN inaccurate, but should not cause incorrect
decisions.)


** instead of

if (!enable_incrementalsort)  presorted_keys = false;

you probably meant

if (!enable_incrementalsort)  presorted_keys = 0;


** instead of

/* Extract presorted keys as list of expressions */
foreach(l, pathkeys)
{PathKey *key = (PathKey *)lfirst(l);EquivalenceMember *member = (EquivalenceMember *)
lfirst(list_head(key->pk_eclass->ec_members));

you can use linitial():

/* Extract presorted keys as list of expressions */
foreach(l, pathkeys)
{PathKey *key = (PathKey *)lfirst(l);EquivalenceMember *member = (EquivalenceMember *)
linitial(key->pk_eclass->ec_members);


* get_cheapest_fractional_path_for_pathkeys()

The prologue says "... at least partially satisfies the given pathkeys ..."
but I see no change in the function code. In particular the use of
pathkeys_contained_in() does not allow for any kind of partial sorting.


* pathkeys_useful_for_ordering()

Extra whitespace following the comment opening string "/*":

/* * When incremental sort is disabled, pathkeys are useful only when they


* make_sort_from_pathkeys() - the "skipCols" argument should be mentioned in the prologue.


* create_sort_plan()

I noticed that pathkeys_common() is called, but the value of n_common_pathkeys
should already be in the path's "skipCols" field if the underlying path is
actually IncrementalSortPath.


* create_unique_plan() does not seem to make use of the incremental sort. Shouldn't it do?


* nodeIncrementalSort.c

** These comments seem to contain typos:

"Incremental sort algorithm would sort by xfollowing groups, which have ..."

"Interate while skip cols are same as in saved tuple"

** (This is rather a pedantic comment) I think prepareSkipCols() should be  defined in front of cmpSortSkipCols().

** the MIN_GROUP_SIZE constant deserves a comment.


* ExecIncrementalSort()

** if (node->tuplesortstate == NULL)

If both branches contain the expression
    node->groupsCount++;

I suggest it to be moved outside the "if" construct.

--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at

On Wed, Nov 15, 2017 at 7:42 AM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:
> Sure, please find rebased patch attached.

+ /*
+  * Check if first "skipCols" sort values are equal.
+  */
+ static bool
+ cmpSortSkipCols(IncrementalSortState *node, TupleTableSlot *a,
+                                                             TupleTableSlot *b)
+ {
+     int n, i;
+
+     Assert(IsA(node->ss.ps.plan, IncrementalSort));
+
+     n = ((IncrementalSort *) node->ss.ps.plan)->skipCols;
+
+     for (i = 0; i < n; i++)
+     {
+         Datum datumA, datumB, result;
+         bool isnullA, isnullB;
+         AttrNumber attno = node->skipKeys[i].attno;
+         SkipKeyData *key;
+
+         datumA = slot_getattr(a, attno, &isnullA);
+         datumB = slot_getattr(b, attno, &isnullB);
+
+         /* Special case for NULL-vs-NULL, else use standard comparison */
+         if (isnullA || isnullB)
+         {
+             if (isnullA == isnullB)
+                 continue;
+             else
+                 return false;
+         }
+
+         key = &node->skipKeys[i];
+
+         key->fcinfo.arg[0] = datumA;
+         key->fcinfo.arg[1] = datumB;
+
+         /* just for paranoia's sake, we reset isnull each time */
+         key->fcinfo.isnull = false;
+
+         result = FunctionCallInvoke(&key->fcinfo);
+
+         /* Check for null result, since caller is clearly not expecting one */
+         if (key->fcinfo.isnull)
+             elog(ERROR, "function %u returned NULL", key->flinfo.fn_oid);
+
+         if (!DatumGetBool(result))
+             return false;
+     }
+     return true;
+ }

Is there some reason not to use ApplySortComparator for this?  I think
you're missing out on lower-overhead comparators, and in any case it's
probably good code reuse, no?

Embarrassingly, I was unaware of this patch and started prototyping
exactly the same thing independently[1].  I hadn't got very far and
will now abandon that, but that's one thing I did differently.  Two
other things that may be different: I had a special case for groups of
size 1 that skipped the sorting, and I only sorted on the suffix
because I didn't put tuples with different prefixes into the sorter (I
was assuming that tuplesort_reset was going to be super efficient,
though I hadn't got around to writing that).  I gather that you have
determined empirically that it's better to be able to sort groups of
at least MIN_GROUP_SIZE than to be able to skip the comparisons on the
leading attributes, but why is that the case?

[1] https://github.com/macdice/postgres/commit/ab0f8aff9c4b25d5598aa6b3c630df864302f572

-- 
Thomas Munro
http://www.enterprisedb.com

On Mon, Nov 20, 2017 at 3:34 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:
> Thank you very much for review.  I really appreciate this topic gets
> attention.  Please, find next revision of patch in the attachment.

I would really like to see this get into v11. This is an important
patch, that has fallen through the cracks far too many times.

-- 
Peter Geoghegan

On Tue, Nov 21, 2017 at 1:00 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:
> On Mon, Nov 20, 2017 at 12:24 AM, Thomas Munro
> <thomas.munro@enterprisedb.com> wrote:
>> Is there some reason not to use ApplySortComparator for this?  I think
>> you're missing out on lower-overhead comparators, and in any case it's
>> probably good code reuse, no?
>
> However, for incremental sort case we don't need to know here whether A > B
> or B > A.  It's enough for us to know if A = B or A != B.  In some cases
> it's way cheaper.  For instance, for texts equality check is basically
> memcmp while comparison may use collation.

Ah, right, of course.

>> I gather that you have
>> determined empirically that it's better to be able to sort groups of
>> at least MIN_GROUP_SIZE than to be able to skip the comparisons on the
>> leading attributes, but why is that the case?
>
> Right.  The issue that not only case of one tuple per group could cause
> overhead, but few tuples (like 2 or 3) is also case of overhead.  Also,
> overhead is related not only to sorting.  While investigate of regression
> case provided by Heikki [1], I've seen extra time spent mostly in extra
> copying of sample tuple and comparison with that.  In order to cope this
> overhead I've introduced MIN_GROUP_SIZE which allows to skip copying sample
> tuples too frequently.

I see.  I wonder if there could ever be a function like
ExecMoveTuple(dst, src).  Given the polymorphism involved it'd be
slightly complicated and you'd probably have a general case that just
copies the tuple to dst and clears src, but there might be a bunch of
cases where you can do something more efficient like moving a pointer
and pin ownership.  I haven't really thought that through and
there may be fundamental problems with it...

If you're going to push the tuples into the sorter every time, then I
guess there are some special cases that could allow future
optimisations: (1) if you noticed that every prefix was different, you
can skip the sort operation (that is, you can use the sorter as a dumb
tuplestore and just get the tuples out in the same order you put them
in; not sure if Tuplesort supports that but it presumably could), (2)
if you noticed that every prefix was the same (that is, you have only
one prefix/group in the sorter) then you could sort only on the suffix
(that is, you could somehow tell Tuplesort to ignore the leading
columns), (3) as a more complicated optimisation for intermediate
group sizes 1 < n < MIN_GROUP_SIZE, you could somehow number the
groups with an integer that increments whenever you see the prefix
change, and somehow tell tuplesort.c to use that instead of the
leading columns.  Ok, that last one is probably hard but the first two
might be easier...

-- 
Thomas Munro
http://www.enterprisedb.com

Alexander Korotkov <a.korotkov@postgrespro.ru> wrote:

> Antonin Houska <ah@cybertec.at> wrote:

> >  * ExecIncrementalSort()
> >
> >  ** if (node->tuplesortstate == NULL)
> >
> >  If both branches contain the expression
> >
> >  node->groupsCount++;
> >
> >  I suggest it to be moved outside the "if" construct.
>
> Done.

One more comment on this: I wonder if the field isn't incremented too
early. It seems to me that the value can end up non-zero if the input set is
to be empty (not sure if it can happen in practice).

And finally one question about regression tests: what's the purpose of the
changes in contrib/postgres_fdw/sql/postgres_fdw.sql ? I see no
IncrementalSort node in the output.

--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at

On Mon, Mar 20, 2017 at 6:33 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:
> Thank you for the report.
> Please, find rebased patch in the attachment.

This patch cannot be applied. Please provide a rebased version. I am
moving it to next CF with waiting on author as status.
-- 
Michael

Hello Alexander,

On Thu, January 4, 2018 4:36 pm, Alexander Korotkov wrote:
> On Fri, Dec 8, 2017 at 4:06 PM, Alexander Korotkov <
> a.korotkov@postgrespro.ru> wrote:
>
>> Thank you for pointing that.  Sure, both cases are better.  I've added
>> second case as well as comments.  Patch is attached.

I had a quick look, this isn't a full review, but a few things struck me
on a read through the diff:

There are quite a few places where lines are broken like so:

+            ExecIncrementalSortInitializeWorker((IncrementalSortState *) planstate,
+                                                pwcxt);
Or like this:

+            result = (PlanState *) ExecInitIncrementalSort(
+                                    (IncrementalSort *) node, estate, eflags);

e.g. a param is on the next line, but aligned to the very same place where
it would be w/o the linebreak. Or is this just some sort of artefact
because I viewed the diff with tabspacing = 8?

I'd fix the grammar here:

+ *        Incremental sort is specially optimized kind of multikey sort when
+ *        input is already presorted by prefix of required keys list.

Like so:

"Incremental sort is a specially optimized kind of multikey sort used when
the input is already presorted by a prefix of the required keys list."

+ *        Consider following example.  We have input tuples consisting from

"Consider the following example: We have ..."

+         * In incremental sort case we also have to cost to detect sort groups.

"we also have to cost the detection of sort groups."

"+         * It turns out into extra copy and comparison for each tuple."

"This turns out to be one extra copy and comparison per tuple."

+ "Portions Copyright (c) 1996-2017"

Should probably be 2018 now - time flies fast :)

         return_value = _readMaterial();
     else if (MATCH("SORT", 4))
         return_value = _readSort();
+    else if (MATCH("INCREMENTALSORT", 7))
+        return_value = _readIncrementalSort();
     else if (MATCH("GROUP", 5))
         return_value = _readGroup();

I think the ", 7" here is left-over from when it was named "INCSORT", and
it should be MATCH("INCREMENTALSORT", 15)), shouldn't it?

+                                   space, fase when it's value for in-memory

typo: "space, false when ..."

+            bool    cmp;
+            cmp = cmpSortSkipCols(node, node->sampleSlot, slot);
+
+            if (cmp)

In the above, the variable cmp could be optimized away with:

+            if (cmpSortSkipCols(node, node->sampleSlot, slot))

(not sure if modern compilers won't do this, anway, though)

+typedef struct IncrementalSortState
+{
+    ScanState    ss;                /* its first field is NodeTag */
+    bool        bounded;        /* is the result set
bounded? */
+    int64        bound;            /* if bounded, how many
tuples are needed */

If I'm not wrong, the layout of the struct will include quite a bit of
padding on 64 bit due to the mixing of bool and int64, maybe it would be
better to sort the fields differently, e.g. pack 4 or 8 bools together?
Not sure if that makes much of a difference, though.

That's all for now :)

Thank you for your work,

Tels

Alexander Korotkov <a.korotkov@postgrespro.ru> wrote:

> Antonin Houska <ah@cybertec.at> wrote:

> >  Shouldn't the test contain *both* cases?

> Thank you for pointing that. Sure, both cases are better. I've added second case as well as comments. Patch is
attached.

I'm fine with the tests now but have a minor comment on this comment:

-- CROSS JOIN, not pushed down, because we don't push down LIMIT and remote side
-- can't perform top-N sort like local side can.

I think the note on LIMIT push-down makes the comment less clear because
there's no difference in processing the LIMIT: EXPLAIN shows that both

SELECT t1.c1, t2.c1 FROM ft1 t1 CROSS JOIN ft2 t2 ORDER BY t1.c1, t2.c1 OFFSET
100 LIMIT 10;

and

SELECT t1.c3, t2.c3 FROM ft1 t1 CROSS JOIN ft2 t2 ORDER BY t1.c3, t2.c3 OFFSET
100 LIMIT 10;

evaluate the LIMIT clause only locally.

What I consider the important difference is that the 2nd case does not
generate the appropriate input for remote incremental sort (while incremental
sort tends to be very cheap). Therefore it's cheaper to do no remote sort at
all and perform the top-N sort locally than to do a regular (non-incremental)
remote sort.

I have no other questions about this patch. I expect the CFM to set the status
to "ready for committer" as soon as the other reviewers confirm they're happy
about the patch status.

--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at

Hi,

I have started reviewing the patch and doing some testing, and I have
pretty quickly ran into a segfault. Attached is a simple reproducer and
an backtrace. AFAICS the bug seems to be somewhere in the tuplesort
changes, likely resetting a memory context too soon or something like
that. I haven't investigated it further, but it matches my hunch that
tuplesort is likely where the bugs will be.

Otherwise the patch seems fairly complete. A couple of minor things that
I noticed while eyeballing the changes in a diff editor.


1) On a couple of places the new code has this comment

    /* even when not parallel-aware */

while all the immediately preceding blocks use

    /* even when not parallel-aware, for EXPLAIN ANALYZE */

I suggest using the same comment, otherwise it kinda suggests it's not
because of EXPLAIN ANALYZE.


2) I think the purpose of sampleSlot should be explicitly documented
(and I'm not sure "sample" is a good term here, as is suggest some sort
of sampling (for example nodeAgg uses grp_firstTuple).


3) skipCols/SkipKeyData seems a bit strange too, I think. I'd use
PresortedKeyData or something like that.


4) In cmpSortSkipCols, when checking if the columns changed, the patch
does this:

    n = ((IncrementalSort *) node->ss.ps.plan)->skipCols;

    for (i = 0; i < n; i++)
    {
        ... check i-th key ...
    }

My hunch is that checking the keys from the last one, i.e.

    for (i = (n-1); i >= 0; i--)
    {
        ....
    }

would be faster. The reasoning is that with "ORDER BY a,b" the column
"b" changes more often. But I've been unable to test this because of the
segfault crashes.


5) The changes from

    if (pathkeys_contained_in(...))

to

    n = pathkeys_common(pathkeys, subpath->pathkeys);


    if (n == 0)

seem rather inconvenient to me, as it makes the code unnecessarily
verbose. I wonder if there's a better way to deal with this.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 03/05/2018 11:07 PM, Alexander Korotkov wrote:
> Hi!
> 
> Thank you for reviewing this patch!
> Revised version is attached.
> 

OK, the revised patch works fine - I've done a lot of testing and
benchmarking, and not a single segfault or any other crash.

Regarding the benchmarks, I generally used queries of the form

    SELECT * FROM (SELECT * FROM t ORDER BY a) foo ORDER BY a,b

with the first sort done in various ways:

    * regular Sort node
    * indexes with Index Scan
    * indexes with Index Only Scan

and all these three options with and without LIMIT (the limit was set to
1% of the source table).

I've also varied parallelism (max_parallel_workers_per_gather was set to
either 0 or 2), work_mem (from 4MB to 256MB) and data set size (tables
from 1000 rows to 10M rows).

All of this may seem like an overkill, but I've found a couple of
regressions thanks to that.

The full scripts and results are available here:

    https://github.com/tvondra/incremental-sort-tests

The queries actually executed are a bit more complicated, to eliminate
overhead due to data transfer to client etc. The same approach was used
in the other sorting benchmarks we've done in the past.

I'm attaching results for two scales - 10k and 10M rows, preprocessed
into .ods format. I haven't looked at the other scales yet, but I don't
expect any surprises there.

Each .ods file contains raw data for one of the tests (matching the .sh
script filename), pivot table, and comparison of durations with and
without the incremental sort.

In general, I think the results look pretty impressive. Almost all the
comparisons are green, which means "faster than master" - usually by
tens of percent (without limit), or by up to ~95% (with LIMIT).

There are a couple of regressions in two cases sort-indexes and
sort-indexes-ios.

Oh the small dataset this seems to be related to the number of groups
(essentially, number of distinct values in a column). My assumption is
that there is some additional overhead when "switching" between the
groups, and with many groups it's significant enough to affect results
on these tiny tables (where master only takes ~3ms to do the sort). The
slowdown seems to be

On the large data set it seems to be somehow related to both work_mem
and number of groups, but I didn't have time to investigate that yet
(there are explain analyze plans in the results, so feel free to look).

In general, I think this looks really nice. It's certainly awesome with
the LIMIT case, as it allows us to leverage indexes on a subset of the
ORDER BY columns.

Now, there's a caveat in those tests - the data set is synthetic and
perfectly random, i.e. all groups equally likely, no correlations or
anything like that.

I wonder what is the "worst case" scenario, i.e. how to construct a data
set with particularly bad behavior of the incremental sort.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 03/10/2018 06:05 PM, Alexander Korotkov wrote:
> On Sat, Mar 10, 2018 at 6:42 PM, Alexander Korotkov
> <a.korotkov@postgrespro.ru <mailto:a.korotkov@postgrespro.ru>> wrote:
>
> ...
>
> After some investigation of benchmark results, I found 2 sources of
> regressions of incremental sort.
> 
> *Case 1: Underlying node scan lose is bigger than incremental sort win*
> 
> ===== 33 [Wed Mar  7 10:14:14 CET 2018] scale:10000000 groups:10
> work_mem:64MB incremental:on max_workers:0 =====
> SELECT * FROM s_1 ORDER BY a, b
>                                                                    QUERY
> PLAN                                                                    
>
-------------------------------------------------------------------------------------------------------------------------------------------------
>  Limit  (cost=1588080.84..1588080.84 rows=1 width=20) (actual
> time=5874.527..5874.527 rows=0 loops=1)
>    ->  Incremental Sort  (cost=119371.51..1488081.45 rows=9999939
> width=20) (actual time=202.842..5653.224 rows=10000000 loops=1)
>          Sort Key: s_1.a, s_1.b
>          Presorted Key: s_1.a
>          Sort Method: external merge  Disk: 29408kB
>          Sort Groups: 11
>          ->  Index Scan using s_1_a_idx on s_1  (cost=0.43..323385.52
> rows=9999939 width=20) (actual time=0.051..1494.105 rows=10000000 loops=1)
>  Planning time: 0.269 ms
>  Execution time: 5877.367 ms
> (9 rows)
> 
> ===== 37 [Wed Mar  7 10:15:51 CET 2018] scale:10000000 groups:10
> work_mem:64MB incremental:off max_workers:0 =====
> SELECT * FROM s_1 ORDER BY a, b
>                                                           QUERY PLAN   
>                                                       
>
------------------------------------------------------------------------------------------------------------------------------
>  Limit  (cost=1656439.93..1656439.93 rows=1 width=20) (actual
> time=4741.716..4741.716 rows=0 loops=1)
>    ->  Sort  (cost=1531440.69..1556440.54 rows=9999939 width=20) (actual
> time=3522.156..4519.278 rows=10000000 loops=1)
>          Sort Key: s_1.a, s_1.b
>          Sort Method: external merge  Disk: 293648kB
>          ->  Seq Scan on s_1  (cost=0.00..163694.39 rows=9999939
> width=20) (actual time=0.021..650.322 rows=10000000 loops=1)
>  Planning time: 0.249 ms
>  Execution time: 4777.088 ms
> (7 rows)
> 
> In this case optimizer have decided that "Index Scan + Incremental
> Sort" would be cheaper than "Seq Scan + Sort".  But it appears that
> the amount of time we loose by selecting Index Scan over Seq Scan is
> bigger than amount of time we win by selecting Incremental Sort over
> Sort.  I would note that regular Sort consumes about 10X more disk
> space.  I bet that all this space has fit to OS cache of test 
> machine.  But optimizer did expect actual IO to take place in this 
> case.  This has lead actual time to be inadequate the costing.
> 

Yes, you're right the temporary file(s) likely fit into RAM in this test
(and even if they did not, the storage system is pretty good).

> *Case 2: Underlying node is not parallelyzed*
> 
> ===== 178 [Wed Mar  7 11:18:53 CET 2018] scale:10000000 groups:100
> work_mem:8MB incremental:on max_workers:2 =====
> SELECT * FROM s_2 ORDER BY a, b, c
>                                                                    
>  QUERY PLAN                                                             
>        
>
----------------------------------------------------------------------------------------------------------------------------------------------------
>  Limit  (cost=1179047.88..1179047.88 rows=1 width=20) (actual
> time=4819.999..4819.999 rows=0 loops=1)
>    ->  Incremental Sort  (cost=89.04..1079047.34 rows=10000054 width=20)
> (actual time=0.203..4603.197 rows=10000000 loops=1)
>          Sort Key: s_2.a, s_2.b, s_2.c
>          Presorted Key: s_2.a, s_2.b
>          Sort Method: quicksort  Memory: 135kB
>          Sort Groups: 10201
>          ->  Index Scan using s_2_a_b_idx on s_2  (cost=0.43..406985.62
> rows=10000054 width=20) (actual time=0.052..1461.177 rows=10000000 loops=1)
>  Planning time: 0.313 ms
>  Execution time: 4820.037 ms
> (9 rows)
> 
> ===== 182 [Wed Mar  7 11:20:11 CET 2018] scale:10000000 groups:100
> work_mem:8MB incremental:off max_workers:2 =====
> SELECT * FROM s_2 ORDER BY a, b, c
>                                                                  QUERY
> PLAN                                                                 
>
--------------------------------------------------------------------------------------------------------------------------------------------
>  Limit  (cost=1705580.76..1705580.76 rows=1 width=20) (actual
> time=3985.818..3985.818 rows=0 loops=1)
>    ->  Gather Merge  (cost=649951.66..1622246.98 rows=8333378 width=20)
> (actual time=1782.354..3750.868 rows=10000000 loops=1)
>          Workers Planned: 2
>          Workers Launched: 2
>          ->  Sort  (cost=648951.64..659368.36 rows=4166689 width=20)
> (actual time=1778.362..2091.253 rows=3333333 loops=3)
>                Sort Key: s_2.a, s_2.b, s_2.c
>                Sort Method: external merge  Disk: 99136kB
>                Worker 0:  Sort Method: external merge  Disk: 96984kB
>                Worker 1:  Sort Method: external merge  Disk: 97496kB
>                ->  Parallel Seq Scan on s_2  (cost=0.00..105361.89
> rows=4166689 width=20) (actual time=0.022..233.640 rows=3333333 loops=3)
>  Planning time: 0.265 ms
>  Execution time: 4007.591 ms
> (12 rows)
> 
> The situation is similar to case #1 except that in the pair "Seq Scan
> + Sort" Sort also gets paralellyzed.  In the same way as in previous
> case, disk writes/reads during external sort are overestimated,
> because they actually use OS cache. I would also say that it's not
> necessary wrong decision of optimizer, because doing this work in
> single backend may consume less resources despite being overall
> slower.
> 

Yes, that seems like a likely explanation too.

I agree those don't seem like an issue in the Incremental Sort patch,
but like a more generic costing problems.


Thanks for looking into the benchmark results.

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 03/16/2018 09:47 AM, Alexander Korotkov wrote:
> On Fri, Mar 16, 2018 at 5:12 AM, Tomas Vondra
> <tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:
> 
>     I agree those don't seem like an issue in the Incremental Sort patch,
>     but like a more generic costing problems.
> 
> 
> Yes, I think so too.

I wonder if we could make the costing a bit more pessimistic, to make
these loses less likely, while still keeping the main wins (particularly
for the LIMIT queries). But that seems a bit like a lost case, I guess.

> Do you think we can mark this patch RFC assuming that it have
> already got pretty much of review previously.
> 

Actually, I was going to propose to switch it to RFC, so I've just done
that. I think the patch is clearly ready for a committer to take a
closer look. I really like this improvement.

I'm going to rerun the tests, but that's mostly because I'm interested
if the change from i++ to i-- in cmpSortPresortedCols makes a measurable
difference. I don't expect to find any issues, so why wait with the RFC?

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

> BTW, patch had conflicts with master.  Please, find rebased version attached.

Sorry, but patch conflicts with master again.


-- 
Teodor Sigaev                                   E-mail: teodor@sigaev.ru
                                                    WWW: http://www.sigaev.ru/

> BTW, patch had conflicts with master.  Please, find rebased version attached.

Despite by patch conflist patch looks commitable, has anybody objections to 
commit it?

Patch recieved several rounds of review during 2 years, and seems to me, keeping 
it out from sources may cause a lost it. Although it suggests performance 
improvement in rather wide usecases.

-- 
Teodor Sigaev                                   E-mail: teodor@sigaev.ru
                                                    WWW: http://www.sigaev.ru/

On 03/28/2018 03:28 PM, Teodor Sigaev wrote:
>> BTW, patch had conflicts with master.  Please, find rebased version
>> attached.
> 
> Despite by patch conflist patch looks commitable, has anybody objections
> to commit it?
> 
> Patch recieved several rounds of review during 2 years, and seems to me,
> keeping it out from sources may cause a lost it. Although it suggests
> performance improvement in rather wide usecases.
> 

No objections from me - if you want me to do one final round of review
after the rebase (not sure how invasive it'll turn out), let me know.

BTW one detail I'd change is name of the GUC variable. enable_incsort
seems unnecessarily terse - let's go for enable_incremental_sort or
something like that.

regards


-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 03/28/2018 05:12 PM, Alexander Korotkov wrote:
> On Wed, Mar 28, 2018 at 4:44 PM, Tomas Vondra
> <tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:
> 
>     On 03/28/2018 03:28 PM, Teodor Sigaev wrote:
>     >> BTW, patch had conflicts with master.  Please, find rebased version
>     >> attached.
>     >
>     > Despite by patch conflist patch looks commitable, has anybody objections
>     > to commit it?
>     >
>     > Patch recieved several rounds of review during 2 years, and seems to me,
>     > keeping it out from sources may cause a lost it. Although it suggests
>     > performance improvement in rather wide usecases.
>     >
> 
>     No objections from me - if you want me to do one final round of review
>     after the rebase (not sure how invasive it'll turn out), let me know.
> 
> 
> Rebased patch is attached.  Incremental sort get used in multiple places
> of partition_aggregate regression test.  I've checked those cases, and
> it seems that incremental sort was selected right.
> 

OK, I'll take a look.

>     BTW one detail I'd change is name of the GUC variable. enable_incsort
>     seems unnecessarily terse - let's go for enable_incremental_sort or
>     something like that.
> 
> 
> Already enable_incsort was already renamed to enable_incrementalsort
> since [1].  
> 
> 1.
> https://www.postgresql.org/message-id/CAPpHfduAVmiGDZC%2BdfNL1rEGu0mt45Rd_mxwjY57uqwWhrvQzg%40mail.gmail.com 
> 

Ah, apologies. I've been looking at the wrong version.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Teodor Sigaev wrote:
> > BTW, patch had conflicts with master.  Please, find rebased version attached.
> 
> Despite by patch conflist patch looks commitable, has anybody objections to
> commit it?
> 
> Patch recieved several rounds of review during 2 years, and seems to me,
> keeping it out from sources may cause a lost it. Although it suggests
> performance improvement in rather wide usecases.

Can we have a recap on what the patch *does*?  I see there's a
description in Alexander's first email
https://postgr.es/m/CAPpHfdscOX5an71nHd8WSUH6GNOCf=V7wgDaTXdDd9=goN-gfA@mail.gmail.com
but that was a long time ago, and the patch has likely changed in the
meantime ...

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Hi,

On 2018-03-28 16:28:01 +0300, Teodor Sigaev wrote:
> > BTW, patch had conflicts with master.  Please, find rebased version attached.
> 
> Despite by patch conflist patch looks commitable, has anybody objections to
> commit it?

> Patch recieved several rounds of review during 2 years, and seems to me,
> keeping it out from sources may cause a lost it. Although it suggests
> performance improvement in rather wide usecases.

My impression it has *NOT* received enough review to be RFC. Not saying
it's impossible to get there this release, but that just committing it
doesn't seem wise.

Greetings,

Andres Freund

Hi Alexander,

I took a quick look at the patch. Some things I fixed myself in the 
attached patch v.21. Here is the summary:

Typo in compare_fractional_path_costs() should be fixed as a separate patch.
Remove unused function estimate_pathkeys_groups.
Extra MemoryContextReset before tuplesort_end() shouldn't be a big deal, 
so we don't have to add a parameter to tuplesoft_free().
Add comment to maincontext declaration.
Fix typo in INITIAL_MEMTUPSIZE.
Remove trailing whitespace.

Some other things I found:

In tuplesort_reset:
if (state->memtupsize < INITIAL_MEMTUPSIZE)
     <reallocate memtuples to INITIAL_MEMTUPSIZE>
I'd add a comment explaining when and why we have to do this. Also maybe 
a comment to other allocations of memtuples in tuplesort_begin() and 
mergeruns(), explaining why it is reallocated and why in maincontext.


In tuplesort_updatemax:
     /* XXX */
     if (spaceUsedOnDisk > state->maxSpaceOnDisk ||
         (spaceUsedOnDisk == state->maxSpaceOnDisk && spaceUsed > 
state->maxSpace))
XXX. Also comparing bools with '>' looks confusing to me.


We should add a comment on top of tuplesort.c, explaining that we now 
have a faster way to sort multiple batches of data using the same sort 
conditions.


The name 'main context' sounds somewhat vague. Maybe 'top context'? Not 
sure.


In ExecSupportBackwardsScan:
         case T_IncrementalSort:
             return false;
This separate case looks useless, I'd either add a comment explaining 
why it can't scan backwards, or just return false by default.



That's all I have for today; tomorrow I'll continue with reviewing the 
planner part of the patch.

-- 
Alexander Kuzmenkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Hi,

I've been doing a bit more review of the patch today, focusing on the
planner part, and I'm starting to have some doubts regarding the way
incremental sort paths are created. I do have some question about the
executor and other parts too.

I'll mark this as 'waiting on author' to make it clear the patch is
still being discussed, RFC is not appropriate status for that.

Attached is a patch that highlights some of the interesting places, and
also suggests some minor changes to comments and other tweaks.


1) planning/costing of incremental vs. non-incremental sorts
------------------------------------------------------------

In sort, all the various places that create/cost sorts:

 * createplan.c (make_sort)
 * planner.c (create_sort_path)
 * pathnode.c (cost_sort)

seem to prefer incremental sorts whenever available. Consider for
example this code from create_merge_append_plan():

    if (!pathkeys_common_contained_in(pathkeys, subpath->pathkeys,
                                      &n_common_pathkeys))
    {
        Sort       *sort = make_sort(subplan, numsortkeys,
                                     n_common_pathkeys,
                                     sortColIdx, sortOperators,
                                     collations, nullsFirst);

        label_sort_with_costsize(root, sort, best_path->limit_tuples);
        subplan = (Plan *) sort;
    }

This essentially says that when (n_common_pathkeys > 0), the sort is
going to be incremental.

That however seems to rely on an important assumption - when the input
is presorted, the incremental sort is expected to be cheaper than
regular sort.

This assumption however seems to be proven invalid by cost_sort, which
does the common part for both sort modes (incremental/non-incremental)
first, and then does this:

    /* Extra costs of incremental sort */
    if (presorted_keys > 0)
    {
        ... add something to the costs ...
    }

That is, the incremental cost seems to be pretty much guaranteed to be
more expensive than regular Sort (with the exception of LIMIT queries,
where it's guaranteed to win thanks to lower startup cost).

I don't know how significant the cost difference may be (perhaps not
much), or if it may lead to inefficient plans. For example what if the
cheapest total path is partially sorted by chance, and only has a single
prefix group? Then all the comparisons with pivotSlot are unnecessary.

But I'm pretty sure it may lead to surprising behavior - for example if
you disable incremental sorts (enable_incrementalsort=off), the plan
will switch to plain sort without the additional costs. So you'll get a
cheaper plan by disabling some operation. That's surprising.

So I think it would be more appropriate if those places actually did a
costing of incremental vs. non-incremental sorts, and then constructed
the cheaper option. Essentially we should consider both plain and
incremental sort for each partially sorted input path, and then pick the
right one.

Of course, this is going to be tricky in createplan.c which builds the
plans directly - in that case it might be integrated into make_sort() or
something like that.

Also, I wonder if we could run into problems, due to incremental sort
not supporting things the regular sort does (rewind, backwards scans and
mark/restore).


2) nodeIncrementalsort.c
------------------------

There's a couple of obsolete comments, that came from nodeSort.c and did
not get tweaked (and so talk about first-time through when incremental
sort needs to do that for each group, etc.). The attached diff tweaks
those, and clarifies a couple of others. I've also added some comments
explaining what the pivotSlot is about etc. There's also a couple of XXX
comments asking additional questions/clarifications.

I'm wondering if a static MIN_GROUP_SIZE is good idea. For example, what
if the subplan is expected to return only very few tuples (say, 33), but
the query includes LIMIT 1. Now, let's assume the startup/total cost of
the subplan is 1 and 1000000. With MIN_GROUP_SIZE 32 we're bound to
execute it pretty much till the end, while we could terminate after the
first tuple (if the prefix changes).

So I think we should use a Min(limit,MIN_GROUP_SIZE) here, and perhaps
this should depend on average group size too.

The other questionable thing seems to be this claim:

 * We unlikely will have huge groups with incremental sort.  Therefore
 * usage of abbreviated keys would be likely a waste of time.

followed by disabling abbreviated keys in the tuplesort_begin_heap call.
I find this rather dubious and unsupported by any arguments (I certainly
don't see any in the comments).

If would be more acceptable if the estimated number of groups was used
when deciding whether to use incremental sort or not, but that's not the
case - as explained in the first part, we simply prefer incremental
sorts whenever there is a prefix. In those cases we have very little
idea (or even guarantees) regarding the average group size.

Furthermore, cost_sort is estimating the number of groups, so it does
know the average group size. I don't see why we couldn't consider it
here too, and disable/enable abbreviated keys depending on that.


3) pathkeys.c
-------------

The new function pathkeys_useful_for_ordering() does actually change
behavior depending on enable_incrementalsort. That seems like a rather
bad idea, for a couple or reasons.

AFAICS pathkeys.c is supposed to provide generic utils for work with
pathkeys, and no one would expect the functions to change behavior
depending on enable_* GUCs. I certainly would not.

In short, this does not seem like the right place to enable/disable
incremental sorts, that should be done when costing the plan (i.e. in
costsize.c) or creating the plan.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 03/31/2018 10:43 PM, Tomas Vondra wrote:
> ...
> But I'm pretty sure it may lead to surprising behavior - for example if
> you disable incremental sorts (enable_incrementalsort=off), the plan
> will switch to plain sort without the additional costs. So you'll get a
> cheaper plan by disabling some operation. That's surprising.
> 

To illustrate this is a valid issue, consider this trivial example:

create table t (a int, b int, c int);

insert into t select 10*random(), 10*random(), 10*random()
  from generate_series(1,1000000) s(i);

analyze t;

explain select * from (select * from t order by a,b) foo order by a,b,c;

                               QUERY PLAN
------------------------------------------------------------------------
 Incremental Sort  (cost=133100.48..264139.27 rows=1000000 width=12)
   Sort Key: t.a, t.b, t.c
   Presorted Key: t.a, t.b
   ->  Sort  (cost=132154.34..134654.34 rows=1000000 width=12)
         Sort Key: t.a, t.b
         ->  Seq Scan on t  (cost=0.00..15406.00 rows=1000000 width=12)
(6 rows)

set enable_incrementalsort = off;

explain select * from (select * from t order by a,b) foo order by a,b,c;
                               QUERY PLAN
------------------------------------------------------------------------
 Sort  (cost=261402.69..263902.69 rows=1000000 width=12)
   Sort Key: t.a, t.b, t.c
   ->  Sort  (cost=132154.34..134654.34 rows=1000000 width=12)
         Sort Key: t.a, t.b
         ->  Seq Scan on t  (cost=0.00..15406.00 rows=1000000 width=12)
(5 rows)

So the cost with incremental sort was 264139, and after disabling the
incremental cost it dropped to 263902. Granted, the difference is
negligible in this case, but it's still surprising.

Also, it can be made much more significant by reducing the number of
prefix groups in the data:

truncate t;

insert into t select 1,1,1 from generate_series(1,1000000) s(i);

analyze t;

set enable_incrementalsort = on;

explain select * from (select * from t order by a,b) foo order by a,b,c;

                               QUERY PLAN
------------------------------------------------------------------------
 Incremental Sort  (cost=324165.83..341665.85 rows=1000000 width=12)
   Sort Key: t.a, t.b, t.c
   Presorted Key: t.a, t.b
   ->  Sort  (cost=132154.34..134654.34 rows=1000000 width=12)
         Sort Key: t.a, t.b
         ->  Seq Scan on t  (cost=0.00..15406.00 rows=1000000 width=12)
(6 rows)

So that's 263902 vs. 341665, yet we still prefer the incremental mode.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 04/03/2018 11:09 AM, Alexander Korotkov wrote:
> On Sun, Apr 1, 2018 at 12:06 AM, Tomas Vondra
> <tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:
> 
>     On 03/31/2018 10:43 PM, Tomas Vondra wrote:
>     > ...
>     > But I'm pretty sure it may lead to surprising behavior - for example if
>     > you disable incremental sorts (enable_incrementalsort=off), the plan
>     > will switch to plain sort without the additional costs. So you'll get a
>     > cheaper plan by disabling some operation. That's surprising.
>     >
> 
>     To illustrate this is a valid issue, consider this trivial example:
> 
>     create table t (a int, b int, c int);
> 
>     insert into t select 10*random(), 10*random(), 10*random()
>       from generate_series(1,1000000) s(i);
> 
>     analyze t;
> 
>     explain select * from (select * from t order by a,b) foo order by a,b,c;
> 
>                                    QUERY PLAN
>     ------------------------------------------------------------------------
>      Incremental Sort  (cost=133100.48..264139.27 rows=1000000 width=12)
>        Sort Key: t.a, t.b, t.c
>        Presorted Key: t.a, t.b
>        ->  Sort  (cost=132154.34..134654.34 rows=1000000 width=12)
>              Sort Key: t.a, t.b
>              ->  Seq Scan on t  (cost=0.00..15406.00 rows=1000000 width=12)
>     (6 rows)
> 
>     set enable_incrementalsort = off;
> 
>     explain select * from (select * from t order by a,b) foo order by a,b,c;
>                                    QUERY PLAN
>     ------------------------------------------------------------------------
>      Sort  (cost=261402.69..263902.69 rows=1000000 width=12)
>        Sort Key: t.a, t.b, t.c
>        ->  Sort  (cost=132154.34..134654.34 rows=1000000 width=12)
>              Sort Key: t.a, t.b
>              ->  Seq Scan on t  (cost=0.00..15406.00 rows=1000000 width=12)
>     (5 rows)
> 
>     So the cost with incremental sort was 264139, and after disabling the
>     incremental cost it dropped to 263902. Granted, the difference is
>     negligible in this case, but it's still surprising.
> 
>     Also, it can be made much more significant by reducing the number of
>     prefix groups in the data:
> 
>     truncate t;
> 
>     insert into t select 1,1,1 from generate_series(1,1000000) s(i);
> 
>     analyze t;
> 
>     set enable_incrementalsort = on;
> 
>     explain select * from (select * from t order by a,b) foo order by a,b,c;
> 
>                                    QUERY PLAN
>     ------------------------------------------------------------------------
>      Incremental Sort  (cost=324165.83..341665.85 rows=1000000 width=12)
>        Sort Key: t.a, t.b, t.c
>        Presorted Key: t.a, t.b
>        ->  Sort  (cost=132154.34..134654.34 rows=1000000 width=12)
>              Sort Key: t.a, t.b
>              ->  Seq Scan on t  (cost=0.00..15406.00 rows=1000000 width=12)
>     (6 rows)
> 
>     So that's 263902 vs. 341665, yet we still prefer the incremental mode.
> 
> 
> Problem is well-defined, thank you.
> I'll check what can be done in this field today.
> 

I think solving this may be fairly straight-forward. Essentially, until
now we only had one way to do the sort, so it was OK to make the sort
implicit by checking if the path is sorted

    if (input not sorted)
    {
        ... add a Sort node ...
    }

But now we have multiple possible ways to do the sort, with different
startup/total costs. So the places that create the sorts need to
actually generate the Sort paths for each sort alternative, and store
the information in the Sort node (instead of relying on pathkeys).

Ultimately, this should simplify the createplan.c places making all the
make_sort calls unnecessary (i.e. the input should be already sorted
when needed). Otherwise it'd mean the decision needs to be done locally,
but I don't think that should be needed.

But it's surely a fairly invasive change to the patch ...

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 04/06/2018 01:43 AM, Alexander Korotkov wrote:
> Hi!
> 
> On Tue, Apr 3, 2018 at 2:10 PM, Tomas Vondra
> <tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:
> 
>     I think solving this may be fairly straight-forward. Essentially, until
>     now we only had one way to do the sort, so it was OK to make the sort
>     implicit by checking if the path is sorted
> 
>         if (input not sorted)
>         {
>             ... add a Sort node ...
>         }
> 
>     But now we have multiple possible ways to do the sort, with different
>     startup/total costs. So the places that create the sorts need to
>     actually generate the Sort paths for each sort alternative, and store
>     the information in the Sort node (instead of relying on pathkeys).
> 
>     Ultimately, this should simplify the createplan.c places making all the
>     make_sort calls unnecessary (i.e. the input should be already sorted
>     when needed). Otherwise it'd mean the decision needs to be done locally,
>     but I don't think that should be needed.
> 
>     But it's surely a fairly invasive change to the patch ...
> 
> 
> Right, there are situation when incremental sort has lower startup cost,
> but higher total cost.  In order to find lower cost, we ideally should
> generate
> paths for both full sort and incremental sort.  However, that would increase
> number of total pathes, and could slowdown planning time.  Another issue
> that we don't always generate pathes for sort.  And yes, it would be rather
> invasive.  So, that doesn't look feasible to get into 11.
> 

I agree that's probably not feasible for PG11, considering the freeze is
about 48h from now. Not necessarily because of amount of code needed to
do that (it might be fairly small, actually) but because of the risk of
regressions in other types of plans and lack of time for review/testing.

I do not think this would cause a significant increase in path number.
We already do have the (partially sorted) paths in pathlist, otherwise
v21 wouldn't be able to build the incremental sort path anyway. And the
plans that did the decision in createplan.c could fairly easily do the
decision when constructing the path, I believe.

Looking v21, this affects three different places:

1) create_merge_append_plan

For merge_append the issue is that generate_mergeappend_paths() calls
get_cheapest_path_for_pathkeys(), which however only looks at cheapest
startup/total path, and then simply falls-back to cheapest_total_path if
there are no suitably sorted paths. IMHO if you modify this to also
consider partially-sorted paths, it should work. You'll have to account
for the extra cost of incremental cost, and it needs to be fairly cheap
(perhaps even by first quickly computing some initial cost estimate -
see e.g. initial_cost_nestloop/final_cost_nestloop).

2) create_mergejoin_plan

For mergejoin, the issue is that sort_inner_and_outer() only looks at
cheapest_total_path for both sides, even before computing the merge
pathkeys. So some of the code would have to move into the foreach loop,
and the paths would be picked by get_cheapest_path_for_pathkeys(). This
time only using total cost, per the comment in sort_inner_and_outer().

3) create_gather_merge_plan

This seems fairly simple - the issue is that gather_grouping_paths()
only looks at cheapest_partial_path. Should be a matter of simply
calling the improved get_cheapest_path_for_pathkeys().

Of course, this is all fairly hand-wavy and it may turn out to be fairly
expensive in some cases. But we can use another trick - we don't need to
search through all partially sorted paths, because for each pathkey
prefix there can be just one "best" path for startup cost and one for
total cost. So we could maintain a much shorter list of partially sorted
paths, I think.

>
> Intead, I decided to cut usage of incremental sort.  Now, incremental sort
> is generated only in create_sort_path().  Cheaper path selected between
> incremental sort and full sort with taking limit_tuples into account.
> That limits usage of incremental sort, however risk of regression by this
> patch is also minimal.  In fact, incremental sort will be used only when
> sort is explicitly specified and simultaneously LIMIT is specified or
> dataset to be sorted is large and incremental sort saves disk IO.
> 

I personally am OK with reducing the scope of the patch like this. It's
still beneficial for the common ORDER BY + LIMIT case, which is good. I
don't think it may negatively affect other cases (at least I can't think
of any).

It's pretty obvious it may be extremely useful for the other cases too
(particularly for mergejoin on large tables, where it can allow
in-memory sort with quick startup).

But even if you managed to make the necessary code changes, it's
unlikely any experienced committer will look into such significant
change this close to the cutoff. Either they are going to be away for a
weekend, or they are already looking at other patches :-(

> Attached patch also incorporates following commits made by Alexander
> Kuzmenkov:
> * Rename fields of IncrementalSortState to snake_case for the sake of
> consistency.
> * Rename group test function to isCurrentGroup.
> * Comments from Tomas Vondra about nodeIncrementalSort.c
> * Add a test for incremental sort.
> * Add a separate function to calculate costs of incremental sort.
> 

Those changes seem fine, but are still a couple of issues remaining:

1) pathkeys_useful_for_ordering() still uses enable_incrementalsort,
which I think is a bad idea. I've complained about it in my review on
31/3, and I don't see any explanation why this is a good idea.

2) Likewise, I've suggested that the claim about abbreviated keys in
nodeIncrementalsort.c is dubious. No response, and the XXX comment was
instead merged into the patch:

 * XXX The claim about abbreviated keys seems rather dubious, IMHO.

3) There is a comment at cost_merge_append, despite there being no
relevant changes in that function. Misplaced comment?

4) It's not clear to me why INITIAL_MEMTUPSIZE is defined the way it is.
There needs to be a comment - the intent seems to be making it large
enough to exceed ALLOCSET_SEPARATE_THRESHOLD, but it's not quite clear
why that's a good idea.

5) I do get this warning when building the code:

costsize.c: In function ‘cost_incremental_sort’:
costsize.c:1812:2: warning: ISO C90 forbids mixed declarations and code
[-Wdeclaration-after-statement]
  List    *presortedExprs = NIL;
  ^~~~

6) The comment at cost_incremental_sort talks about cost_sort_internal,
but it's cost_sort_tuplesort I guess.

7) The new code in create_sort_path is somewhat ugly, I guess. It's
correct, but it really needs to be made easier to comprehend. I might
have time to look into that tomorrow, but I can't promise that.

Attached is a diff highlighting some of those places, and couple of
minor code formatting fixes.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Hi all,

This is the other Alexander K. speaking.


On 06.04.2018 20:26, Tomas Vondra wrote:
> I personally am OK with reducing the scope of the patch like this. It's
> still beneficial for the common ORDER BY + LIMIT case, which is good. I
> don't think it may negatively affect other cases (at least I can't think
> of any).

I think we can reduce it even further. Just try incremental sort along 
with full sort over the cheapest path in create_ordered_paths, and don't 
touch anything else. This is a very minimal and a probably safe start, 
and then we can continue working on other, more complex cases. In the 
attached patch I tried to do this. We probably should also remove 
changes in make_sort() and create a separate function 
make_incremental_sort() for it, but I'm done for today.


> 1) pathkeys_useful_for_ordering() still uses enable_incrementalsort,
> which I think is a bad idea. I've complained about it in my review on
> 31/3, and I don't see any explanation why this is a good idea.

Removed.

> 2) Likewise, I've suggested that the claim about abbreviated keys in
> nodeIncrementalsort.c is dubious. No response, and the XXX comment was
> instead merged into the patch:
>
>   * XXX The claim about abbreviated keys seems rather dubious, IMHO.

Not sure about that, maybe just use abbreviated keys for the first 
version? Later we can research this more closely and maybe start 
deciding whether to use abbrev on planning stage.

> 3) There is a comment at cost_merge_append, despite there being no
> relevant changes in that function. Misplaced comment?

Removed.

> 4) It's not clear to me why INITIAL_MEMTUPSIZE is defined the way it is.
> There needs to be a comment - the intent seems to be making it large
> enough to exceed ALLOCSET_SEPARATE_THRESHOLD, but it's not quite clear
> why that's a good idea.

Not sure myself, let's ask the other Alexander.


> 5) I do get this warning when building the code:
>
> costsize.c: In function ‘cost_incremental_sort’:
> costsize.c:1812:2: warning: ISO C90 forbids mixed declarations and code
> [-Wdeclaration-after-statement]
>    List    *presortedExprs = NIL;
>    ^~~~
>
> 6) The comment at cost_incremental_sort talks about cost_sort_internal,
> but it's cost_sort_tuplesort I guess.

Fixed.

> 7) The new code in create_sort_path is somewhat ugly, I guess. It's
> correct, but it really needs to be made easier to comprehend. I might
> have time to look into that tomorrow, but I can't promise that.

Removed this code altogether, now the costs are compared by add_path as 
usual.

> Attached is a diff highlighting some of those places, and couple of
> minor code formatting fixes.

Applied.

Also some other changes from me:

Remove extra blank lines
label_sort_with_costsize shouldn't have to deal with IncrementalSort 
plans, because they are only created from corresponding Path nodes.
Reword a comment in ExecSupportsBackwardsScan.
Clarify cost calculations.
enable_incrementalsort is checked at path level, we don't have to check 
it again at plan level.
enable_sort should act as a cost-based soft disable for both incremental 
and normal sort.

-- 
Alexander Kuzmenkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

> by this patch.  Revised version is attached.

Fine, patch got several rounds of review in all its parts. Is any places 
which should be improved before commit?

-- 
Teodor Sigaev                      E-mail: teodor@sigaev.ru
                                       WWW: http://www.sigaev.ru/

On 04/07/2018 04:37 PM, Teodor Sigaev wrote:
>> by this patch.  Revised version is attached.
> 
> Fine, patch got several rounds of review in all its parts. Is any
> places which should be improved before commit?
> 

I personally feel rather uneasy about committing it, TBH.

While I don't see any obvious issues in the patch at the moment, the
recent changes were rather significant so I might easily miss some
unexpected consequences. (OTOH it's true it was mostly about reduction
of scope, to limit the risks.)

I don't have time to do more review and testing on the latest patch
version, unfortunately, certainly not before the CF end.


So I guess the ultimate review / decision is up to you ...

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Alexander Korotkov <a.korotkov@postgrespro.ru> writes:
> On Wed, Mar 28, 2018 at 6:38 PM, Alvaro Herrera <alvherre@alvh.no-ip.org>
> wrote:
>> Can we have a recap on what the patch *does*?

> Ggeneral idea hasn't been changed much since first email.
> Incremental sort gives benefit when you need to sort your dataset
> by some list of columns while you alredy have input presorted
> by some prefix of that list of columns.  Then you don't do full sort
> of dataset, but rather sort groups where values of prefix columns
> are equal (see header comment in nodeIncremenalSort.c).

I dunno, how would you estimate whether this is actually a win or not?
I don't think our model of sort costs is anywhere near refined enough
or accurate enough to reliably predict whether this is better than
just doing it in one step.  Even if the cost model is good, it's not
going to be better than our statistics about the number/size of the
groups in the first column(s), and that's a notoriously unreliable stat.

Given that we already have more than enough dubious patches that have
been shoved in in the last few days, I'd rather not pile on stuff that
there's any question about.

            regards, tom lane

On 2018-04-07 12:06:52 -0400, Tom Lane wrote:
> Alexander Korotkov <a.korotkov@postgrespro.ru> writes:
> > On Wed, Mar 28, 2018 at 6:38 PM, Alvaro Herrera <alvherre@alvh.no-ip.org>
> > wrote:
> >> Can we have a recap on what the patch *does*?
> 
> > Ggeneral idea hasn't been changed much since first email.
> > Incremental sort gives benefit when you need to sort your dataset
> > by some list of columns while you alredy have input presorted
> > by some prefix of that list of columns.  Then you don't do full sort
> > of dataset, but rather sort groups where values of prefix columns
> > are equal (see header comment in nodeIncremenalSort.c).
> 
> I dunno, how would you estimate whether this is actually a win or not?
> I don't think our model of sort costs is anywhere near refined enough
> or accurate enough to reliably predict whether this is better than
> just doing it in one step.  Even if the cost model is good, it's not
> going to be better than our statistics about the number/size of the
> groups in the first column(s), and that's a notoriously unreliable stat.
> 
> Given that we already have more than enough dubious patches that have
> been shoved in in the last few days, I'd rather not pile on stuff that
> there's any question about.

I don't disagree with any of that. Just wanted to pipe up to say that
there's a fair argument to be made that this patch, which has lingered
for years, "deserves" more to mature in tree than some of the rest.

Greetings,

Andres Freund

> I dunno, how would you estimate whether this is actually a win or not?
> I don't think our model of sort costs is anywhere near refined enough
> or accurate enough to reliably predict whether this is better than
> just doing it in one step.  Even if the cost model is good, it's not
> going to be better than our statistics about the number/size of the
> groups in the first column(s), and that's a notoriously unreliable stat.
I think that improvement in cost calculation of sort should be a 
separate patch, not directly connected to this one. Postpone patches 
till other part will be ready to get max improvement for postponed ones 
doesn't seem to me very good, especially if it suggests some improvement 
right now.


-- 
Teodor Sigaev                      E-mail: teodor@sigaev.ru
                                       WWW: http://www.sigaev.ru/

Teodor Sigaev <teodor@sigaev.ru> writes:
>> I dunno, how would you estimate whether this is actually a win or not?
>> I don't think our model of sort costs is anywhere near refined enough
>> or accurate enough to reliably predict whether this is better than
>> just doing it in one step.  Even if the cost model is good, it's not
>> going to be better than our statistics about the number/size of the
>> groups in the first column(s), and that's a notoriously unreliable stat.

> I think that improvement in cost calculation of sort should be a 
> separate patch, not directly connected to this one. Postpone patches 
> till other part will be ready to get max improvement for postponed ones 
> doesn't seem to me very good, especially if it suggests some improvement 
> right now.

No, you misunderstand the point of my argument.  Without a reasonably
reliable cost model, this patch could easily make performance *worse*
not better for many people, due to choosing incremental-sort plans
where they were really a loss.

If we were at the start of a development cycle and work were being
promised to be done later in the cycle to improve the planning aspect,
I'd be more charitable about it.  But this isn't merely the end of a
cycle, it's the *last day*.  Now is not the time to commit stuff that
needs, or even just might need, follow-on work.

            regards, tom lane

On 04/07/2018 06:23 PM, Tom Lane wrote:
> Teodor Sigaev <teodor@sigaev.ru> writes:
>>> I dunno, how would you estimate whether this is actually a win or not?
>>> I don't think our model of sort costs is anywhere near refined enough
>>> or accurate enough to reliably predict whether this is better than
>>> just doing it in one step.  Even if the cost model is good, it's not
>>> going to be better than our statistics about the number/size of the
>>> groups in the first column(s), and that's a notoriously unreliable stat.
> 
>> I think that improvement in cost calculation of sort should be a 
>> separate patch, not directly connected to this one. Postpone patches 
>> till other part will be ready to get max improvement for postponed ones 
>> doesn't seem to me very good, especially if it suggests some improvement 
>> right now.
> 
> No, you misunderstand the point of my argument.  Without a reasonably
> reliable cost model, this patch could easily make performance *worse*
> not better for many people, due to choosing incremental-sort plans
> where they were really a loss.
> 

Yeah. Essentially the patch could push the planner to pick a path that
has low startup cost (and very high total cost), assuming it'll only
need to read small part of the input. But if the number of groups in the
input is low (perhaps just one huge group), that would be a regression.

> If we were at the start of a development cycle and work were being
> promised to be done later in the cycle to improve the planning aspect,
> I'd be more charitable about it.  But this isn't merely the end of a
> cycle, it's the *last day*.  Now is not the time to commit stuff that
> needs, or even just might need, follow-on work.
> 

+1 to that

FWIW I'm willing to spend some time on the patch for PG12, particularly
on the planner / costing part. The potential gains are too interesting.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 4/9/18 11:56 AM, Alexander Korotkov wrote:
> 
> Thank you very much for your efforts on reviewing this patch.
> I'm looking forward work with you on this feature for PG12.

Since there's a new patch I have changed the status to Needs Review and
moved the entry to the next CF.

Still, it seems to me that discussion and new patches will be required
to address Tom's concerns.

Regards,
-- 
-David
david@pgmasters.net

I've attached an updated copy of the patch that applies cleanly to 
current master.

The following review has been posted through the commitfest application:
make installcheck-world:  tested, failed
Implements feature:       tested, passed
Spec compliant:           not tested
Documentation:            not tested

A fairly common planning problem for us is what we call "most recent first" queries; i.e., "the 50 most recent <table>
rowsfor a <foreign key>".
 

Here's a basic setup:

-- created_at has very high cardinality
create table foo(pk serial primary key, owner_fk integer, created_at timestamp);
create index idx_foo_on_owner_and_created_at on foo(owner_fk, created_at);

-- technically this data guarantees unique created_at values,
-- but there's no reason it couldn't be modified to have a few
-- random non-unique values to prove the point
insert into foo(owner_fk, created_at)
  select i % 100, now() - (i::text || ' minutes')::interval
  from generate_series(1, 1000000) t(i);


And here's the naive query to get the results we want:

select *
from foo
where owner_fk = 23
-- pk is only here to disambiguate/guarantee a stable sort
-- in the rare case that there are collisions in the other
-- sort field(s)
order by created_at desc, pk desc
limit 50;


On stock Postgres this ends up being pretty terrible for cases where the fk filter represents a large number of rows,
becausethe planner generates a sort node under the limit node and therefore fetches all matches, sorts them, and then
appliesthe limit. Here's the plan:
 

 Limit  (cost=61386.12..61391.95 rows=50 width=16) (actual time=187.814..191.653 rows=50 loops=1)
   ->  Gather Merge  (cost=61386.12..70979.59 rows=82224 width=16) (actual time=187.813..191.647 rows=50 loops=1)
         Workers Planned: 2
         Workers Launched: 2
         ->  Sort  (cost=60386.10..60488.88 rows=41112 width=16) (actual time=185.639..185.642 rows=42 loops=3)
               Sort Key: created_at DESC, pk DESC
               Sort Method: top-N heapsort  Memory: 27kB
               Worker 0:  Sort Method: top-N heapsort  Memory: 27kB
               Worker 1:  Sort Method: top-N heapsort  Memory: 27kB
               ->  Parallel Bitmap Heap Scan on foo  (cost=3345.24..59020.38 rows=41112 width=16) (actual
time=25.150..181.804rows=33333 loops=3)
 
                     Recheck Cond: (owner_fk = 23)
                     Heap Blocks: exact=18014
                     ->  Bitmap Index Scan on idx_foo_on_owner_and_created_at  (cost=0.00..3320.57 rows=98668 width=0)
(actualtime=16.992..16.992 rows=100000 loops=1)
 
                           Index Cond: (owner_fk = 23)
 Planning Time: 0.384 ms
 Execution Time: 191.704 ms


I have a recursive CTE that implements the algorithm:
- Find first n+1 results
- If result at n+1’s created_at value differs from the n’s value, return first n values.
- If those equal, gather more results until a new created_at value is encountered.
- Sort all results by created_at and a tie-breaker (e.g., pk) and return the first n values.
But nobody wants to use/write that normally (it's quite complex).

This patch solves the problem presented; here's the plan:

 Limit  (cost=2.70..2.76 rows=50 width=16) (actual time=0.233..0.367 rows=50 loops=1)
   ->  Incremental Sort  (cost=2.70..111.72 rows=98668 width=16) (actual time=0.232..0.362 rows=50 loops=1)
         Sort Key: created_at DESC, pk DESC
         Presorted Key: created_at
         Sort Method: quicksort  Memory: 26kB
         Sort Groups: 2
         ->  Index Scan Backward using idx_foo_on_owner_and_created_at on foo  (cost=0.56..210640.79 rows=98668
width=16)(actual time=0.054..0.299 rows=65 loops=1)
 
               Index Cond: (owner_fk = 23)
 Planning Time: 0.428 ms
 Execution Time: 0.393 ms


While check world fails, the only failure appears to be a plan output change in
test/isolation/expected/drop-index-concurrently-1.outthat just needs to be updated (incremental sort is now used in
thisplan); I don't see any functionality breakage. 

James Coleman <jtc331@gmail.com> writes:
> A fairly common planning problem for us is what we call "most recent first" queries; i.e., "the 50 most recent
<table>rows for a <foreign key>". 

> Here's a basic setup:

> -- created_at has very high cardinality
> create table foo(pk serial primary key, owner_fk integer, created_at timestamp);
> create index idx_foo_on_owner_and_created_at on foo(owner_fk, created_at);

> -- technically this data guarantees unique created_at values,
> -- but there's no reason it couldn't be modified to have a few
> -- random non-unique values to prove the point
> insert into foo(owner_fk, created_at)
>   select i % 100, now() - (i::text || ' minutes')::interval
>   from generate_series(1, 1000000) t(i);

> And here's the naive query to get the results we want:

> select *
> from foo
> where owner_fk = 23
> -- pk is only here to disambiguate/guarantee a stable sort
> -- in the rare case that there are collisions in the other
> -- sort field(s)
> order by created_at desc, pk desc
> limit 50;

If you're concerned about the performance of this case, why don't you make
an index that actually matches the query?

regression=# create index on foo (owner_fk, created_at, pk);
CREATE INDEX
regression=# explain analyze select * from foo where owner_fk = 23 order by created_at desc, pk desc limit 50;
                                                                           QUERY PLAN
                                         

-----------------------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=0.42..110.92 rows=50 width=16) (actual time=0.151..0.280 rows=50 loops=1)
   ->  Index Only Scan Backward using foo_owner_fk_created_at_pk_idx on foo  (cost=0.42..20110.94 rows=9100 width=16)
(actualtime=0.146..0.255 rows=50 loops=1) 
         Index Cond: (owner_fk = 23)
         Heap Fetches: 50
 Planning Time: 0.290 ms
 Execution Time: 0.361 ms
(6 rows)

There may be use-cases for Alexander's patch, but I don't find this
one to be terribly convincing.

            regards, tom lane

On 09/06/2018 08:04 PM, James Coleman wrote:
> I disagree because it's not ideal to basically have to append pk to
> every index in the system just to get the ability to tie-break when
> there's actually very low likelihood of ties anyway.
> 
> A similar use case is trying to batch through a result set, in which
> case you need a stable sort as well.
> 
> The benefit is retaining the generality of indexes (and saving space in
> them etc.) while still allowing using them for more specific sorts. Any
> time you paginate or batch this way you benefit from this, which in many
> applications applies to a very high percentage of indexes.
> 

I 100% with this.

I see incremental sort as a way to run queries with fewer indexes that
are less query-specific, while still benefiting from them. Which means
lower overhead when writing data, lower disk space usage, and so on.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Hi Alexander,

On 06/01/2018 04:22 PM, Alexander Korotkov wrote:
> Hi, James!
> 
> On Thu, May 31, 2018 at 11:10 PM James Coleman <jtc331@gmail.com
> <mailto:jtc331@gmail.com>> wrote:
> 
>     I've attached an updated copy of the patch that applies cleanly to
>     current master.
> 
> 
> Thank you for rebasing this patch.  Next time sending a patch,
> please make sure you've bumped its version, if even you made no
> changes there besides to pure rebase. Otherwise, it would be hard to
> distinguish patch versions, because patch files have exactly same
> names.
> 
> I'd like to note, that I'm going to provide revised version of this 
> patch to the next commitfest. After some conversations at PGCon
> regarding this patch, I got more confident that providing separate
> paths for incremental sorts is right.  In the next revision of this
> patch, incremental sort paths would be provided in more cases.
> 

Do you plan to submit an updated patch version for the November CF? It
would be good to look at the costing model soon, otherwise it might end
up missing PG12, and that would be unfortunate.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

> On Mon, Mar 20, 2017 at 10:34 AM Alexander Korotkov <a.korotkov@postgrespro.ru> wrote:
>
> Please, find rebased patch in the attachment.

It's been a while since this patch was posted. There is already some amount of
feedback in this thread, and the patch itself unfortunately has some conflicts
with the current master. Alexander, do you have any plans about this feature?
For now probably I'll mark it as "Returned with feedback".

Another ping on this Incremental Sort patch.

Alexander, you'd noted that you would try to get it into subsequent
Commit Fests with improvements you've been considering, but I don't
see it in anything but 2018-11. Have you abandoned this as a
maintainer? If so, it would be nice to know so someone else can pick
it up.

-- 
Shaun M Thomas - 2ndQuadrant
PostgreSQL Training, Services and Support
shaun.thomas@2ndquadrant.com | www.2ndQuadrant.com

On 2018-06-01 14:22:26, Alexander Korotkov wrote:
> I'd like to note, that I'm going to provide revised version of this patch
> to the next commitfest.
> After some conversations at PGCon regarding this patch, I got more
> confident that providing separate paths for incremental sorts is right.
> In the next revision of this patch, incremental sort paths would be
> provided in more cases.

Alexander,

I'm currently rebasing the patch, and if I get some time I'm going to
look into working on it.

Would you be able to provide a bit more detail on the changes you were
hoping to make after conversations you'd had with others? I'm hoping
for any pointers/context you have from those conversations as to what
you felt was necessary to get this change committed.

Thanks,
James Coleman

I've rebased the patch on master and confirmed make check world passes.

Hi James,

On Fri, May 31, 2019 at 03:51:57PM -0400, James Coleman wrote:
>I've rebased the patch on master and confirmed make check world passes.

Thanks for the rebase! I think the patch is in pretty good shape - I'm
sure we'll find ways to make it more efficient etc. but IMO that's fine
and should not prevent getting it committed.

The planning/costing logic may need further discussion and improvements,
though. IIRC this was the main reason why the patch missed PG11, because
at that point it simply used incremental sort whenever the input was
already presorted by a pathkey prefix, but that may be slower than regular
sort in some cases (unexpectedly large groups, etc.).

I see the current patch partially improves this by removing the creating
both paths (full and and incremental sort). That'd good, because it means
the decision is cost-based (as it should be). The question however is how
accurate the costing model is - and per discussion in the thread, it may
need some improvements, to handle skewed distributions better.

Currently, the costing logic (cost_incremental_sort) assumes all groups
have the same size, which is fine for uniform distributions. But for
skewed distributions, that may be an issue.

Consider for example a table with 1M rows, two columns, 100 groups in each
column, and index on the first one.

    CREATE table t (a INT, b INT);

    INSERT INTO t SELECT 100*random(), 100*random()
      FROM generate_series(1,1000000);

Now, let's do a simple limit query to find the first row:

    SELECT * FROM t ORDER BU a, b LIMIT 1;

In this case the current costing logic is fine - the groups are close to
average size, and we only need to sort the first group, i.e. 1% of data.

Now, let's say the first group is much larger:


    INSERT INTO t SELECT 0, 100*random()
      FROM generate_series(1,900000) s(i);

    INSERT INTO t SELECT 100*random(), 100*random()
      FROM generate_series(1,100000);

That is, the first group is roughly 90% of data, but the number of groups
is still the same. But we need to scan 90% of data. But the average group
size is still the same, so the current cost model is oblivious to this.

But I think we can improve this by looking at the MCV lists (either
per-column or multi-column) and see if some groups are much larger, and
consider that when computing the costs.

In particular, I think we should estimate the size of the first group,
because that's important for startup cost - we need to process the whole
first group before producing the first tuple, and that matters for LIMIT
queries etc.

For example, let's say the first (already sorted) column has a MCV. Then
we can see how large the first group (by valule, not frequency) is, and
use that instead of the average group size. E.g. in the above example we'd
know the first group is ~90%.

And we could do the same for multiple columns, either by looking at
multi-column MCV lists (if there's one), or by using minimum from each
per-column MCV lists.

Of course, these are only the groups that made it to the MCV list, and
there may be other (smaller) groups before this large one. For example
there could be a group with "-1" value and a single row.

For a moment I thought we could/should look at the histogram, becase that
could tell us if there are groups "before" the first MCV one, but I don't
think we should do that, for two reasons. Firstly, rare values may not get
to the histogram anyway, which makes this rather unreliable and might
introduce sudden plan changes, because the cost would vary wildly
depending on whether we happened to sample the rare row or not. And
secondly, the rare row may be easily filtered out by a WHERE condition or
something, at which point we'll have to deal with the large group anyway.

So I think we should look at the MCV list, and use that information when
computing the startup/total cost. I think using the first/largest group to
compute the startup cost, and the average group size for total cost would
do the trick.

I don't think we can do much better than this during planning. There will
inevitably be cases where the costing model will push us to do the wrong
thing, in either direction. The question is how serious issue that is, and
whether we could remedy that during execution somehow.

When we "incorrectly" pick full sort (when the incremental sort would be
faster), that's obviously sad but I think it's mostly fine because it's
not a regression.

The opposite direction (picking incremental sort, while full sort being
faster) is probably more serious, because it's a regression between
releases.

I don't think we can fully fix that by refining the cost model. We have
two basic options:

1) Argue/show that this is not an issue in practice, because (a) such
cases are very rare, and/or (b) the regression is limited. In short, the
benefits of the patch outweight the losses.

2) Provide some fallback at execution time. For example, we might watch
the size of the group, and if we run into an unexpectedly large one we
might abandon the incremental sort and switch to a "full sort" mode.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Sun, Jun 2, 2019 at 5:18 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
> Thanks for the rebase! I think the patch is in pretty good shape - I'm
> sure we'll find ways to make it more efficient etc. but IMO that's fine
> and should not prevent getting it committed.

Thank you for the in-depth review!

> Currently, the costing logic (cost_incremental_sort) assumes all groups
> have the same size, which is fine for uniform distributions. But for
> skewed distributions, that may be an issue.
>
> Consider for example a table with 1M rows, two columns, 100 groups in each
> column, and index on the first one.
>
>     CREATE table t (a INT, b INT);
>
>     INSERT INTO t SELECT 100*random(), 100*random()
>       FROM generate_series(1,1000000);
>
> Now, let's do a simple limit query to find the first row:
>
>     SELECT * FROM t ORDER BU a, b LIMIT 1;
>
> In this case the current costing logic is fine - the groups are close to
> average size, and we only need to sort the first group, i.e. 1% of data.
>
> Now, let's say the first group is much larger:
>
>
>     INSERT INTO t SELECT 0, 100*random()
>       FROM generate_series(1,900000) s(i);
>
>     INSERT INTO t SELECT 100*random(), 100*random()
>       FROM generate_series(1,100000);
>
> That is, the first group is roughly 90% of data, but the number of groups
> is still the same. But we need to scan 90% of data. But the average group
> size is still the same, so the current cost model is oblivious to this.

Thinking out loud here: the current implementation doesn't guarantee
that sort groups always have the same prefix column values because
(from code comments) "Sorting many small groups with tuplesort is
inefficient). While this seems a reasonable optimization, I think it's
possible that thinking steered away from an optimization in the
opposite direction. Perhaps we should always track whether or not all
prefix tuples are the same (currently we only do that after reaching
DEFAULT_MIN_GROUP_SIZE tuples) and use that information to be able to
have tuplesort only care about the non-prefix columns (where currently
it has to sort on all pathkey columns even though for a large group
the prefix columns are guaranteed to be equal).

Essentially I'm trying to think of ways that would get us to
comparable performance with regular sort in the case of large batch
sizes.

One other thing about the DEFAULT_MIN_GROUP_SIZE logic is that in the
case where you have a very small group and then a very large batch,
we'd lose the ability to optimize in the above way. That makes me
wonder if we shouldn't intentionally optimize for the possibility of
large batch sizes, since a little extra expense per group/tuple is
more likely to be a non-concern with small groups anyway when there
are large numbers of input tuples but a relatively small limit.

Thoughts?

> So I think we should look at the MCV list, and use that information when
> computing the startup/total cost. I think using the first/largest group to
> compute the startup cost, and the average group size for total cost would
> do the trick.

I think this sounds very reasonable.

> I don't think we can do much better than this during planning. There will
> inevitably be cases where the costing model will push us to do the wrong
> thing, in either direction. The question is how serious issue that is, and
> whether we could remedy that during execution somehow.
>
> When we "incorrectly" pick full sort (when the incremental sort would be
> faster), that's obviously sad but I think it's mostly fine because it's
> not a regression.
>
> The opposite direction (picking incremental sort, while full sort being
> faster) is probably more serious, because it's a regression between
> releases.
>
> I don't think we can fully fix that by refining the cost model. We have
> two basic options:
>
> 1) Argue/show that this is not an issue in practice, because (a) such
> cases are very rare, and/or (b) the regression is limited. In short, the
> benefits of the patch outweight the losses.

My comments above go in this direction. If we can improve performance
in the worst case, I think it's plausible this concern becomes a
non-issue.

> 2) Provide some fallback at execution time. For example, we might watch
> the size of the group, and if we run into an unexpectedly large one we
> might abandon the incremental sort and switch to a "full sort" mode.

Are there good examples of our doing this in other types of nodes
(whether the fallback is an entirely different algorithm/node type)? I
like this idea in theory, but I also think it's likely it would add a
very significant amount of complexity. The other problem is knowing
where to draw the line: you end up creating these kinds of cliffs
where pulling one more tuple through the incremental sort would give
you your batch and result in not having to pull many more tuples in a
regular sort node, but the fallback logic kicks in anyway.

Unrelated to all of the above: if I read the patch properly it
intentionally excludes backwards scanning. I don't see any particular
reason why that ought to be the case, and it seems like an odd
limitation for the feature should it be merged. Should that be a
blocker to merging?

James Coleman

On Mon, 3 Jun 2019 at 15:39, James Coleman <jtc331@gmail.com> wrote:
>
> On Sun, Jun 2, 2019 at 5:18 PM Tomas Vondra
> <tomas.vondra@2ndquadrant.com> wrote:
> > Currently, the costing logic (cost_incremental_sort) assumes all groups
> > have the same size, which is fine for uniform distributions. But for
> > skewed distributions, that may be an issue.
> >
> > Consider for example a table with 1M rows, two columns, 100 groups in each
> > column, and index on the first one.
> >
> >     CREATE table t (a INT, b INT);
> >
> >     INSERT INTO t SELECT 100*random(), 100*random()
> >       FROM generate_series(1,1000000);
> >
> > Now, let's do a simple limit query to find the first row:
> >
> >     SELECT * FROM t ORDER BU a, b LIMIT 1;
> >
> > In this case the current costing logic is fine - the groups are close to
> > average size, and we only need to sort the first group, i.e. 1% of data.
> >
> > Now, let's say the first group is much larger:
> >
> >
> >     INSERT INTO t SELECT 0, 100*random()
> >       FROM generate_series(1,900000) s(i);
> >
> >     INSERT INTO t SELECT 100*random(), 100*random()
> >       FROM generate_series(1,100000);
> >
> > That is, the first group is roughly 90% of data, but the number of groups
> > is still the same. But we need to scan 90% of data. But the average group
> > size is still the same, so the current cost model is oblivious to this.
>
> Thinking out loud here: the current implementation doesn't guarantee
> that sort groups always have the same prefix column values because
> (from code comments) "Sorting many small groups with tuplesort is
> inefficient). While this seems a reasonable optimization, I think it's
> possible that thinking steered away from an optimization in the
> opposite direction. Perhaps we should always track whether or not all
> prefix tuples are the same (currently we only do that after reaching
> DEFAULT_MIN_GROUP_SIZE tuples) and use that information to be able to
> have tuplesort only care about the non-prefix columns (where currently
> it has to sort on all pathkey columns even though for a large group
> the prefix columns are guaranteed to be equal).
>
+1 for passing only the non-prefix columns to tuplesort.
> Essentially I'm trying to think of ways that would get us to
> comparable performance with regular sort in the case of large batch
> sizes.
>
> One other thing about the DEFAULT_MIN_GROUP_SIZE logic is that in the
> case where you have a very small group and then a very large batch,
> we'd lose the ability to optimize in the above way. That makes me
> wonder if we shouldn't intentionally optimize for the possibility of
> large batch sizes, since a little extra expense per group/tuple is
> more likely to be a non-concern with small groups anyway when there
> are large numbers of input tuples but a relatively small limit.
>
What about using the knowledge of MCV here, if we know the next value
is in MCV list then take the overhead of sorting this small group
alone and then leverage the optimization for the larger group, by
passing only the non-prefix columns.
> Thoughts?
>
> > So I think we should look at the MCV list, and use that information when
> > computing the startup/total cost. I think using the first/largest group to
> > compute the startup cost, and the average group size for total cost would
> > do the trick.
>
+1
> I think this sounds very reasonable.
>
> > I don't think we can do much better than this during planning. There will
> > inevitably be cases where the costing model will push us to do the wrong
> > thing, in either direction. The question is how serious issue that is, and
> > whether we could remedy that during execution somehow.
> >
> > When we "incorrectly" pick full sort (when the incremental sort would be
> > faster), that's obviously sad but I think it's mostly fine because it's
> > not a regression.
> >
> > The opposite direction (picking incremental sort, while full sort being
> > faster) is probably more serious, because it's a regression between
> > releases.
> >
> > I don't think we can fully fix that by refining the cost model. We have
> > two basic options:
> >
> > 1) Argue/show that this is not an issue in practice, because (a) such
> > cases are very rare, and/or (b) the regression is limited. In short, the
> > benefits of the patch outweight the losses.
>
> My comments above go in this direction. If we can improve performance
> in the worst case, I think it's plausible this concern becomes a
> non-issue.
>
> > 2) Provide some fallback at execution time. For example, we might watch
> > the size of the group, and if we run into an unexpectedly large one we
> > might abandon the incremental sort and switch to a "full sort" mode.
>
> Are there good examples of our doing this in other types of nodes
> (whether the fallback is an entirely different algorithm/node type)? I
> like this idea in theory, but I also think it's likely it would add a
> very significant amount of complexity. The other problem is knowing
> where to draw the line: you end up creating these kinds of cliffs
> where pulling one more tuple through the incremental sort would give
> you your batch and result in not having to pull many more tuples in a
> regular sort node, but the fallback logic kicks in anyway.
>
What about having some simple mechanism for this, like if we encounter
the group with more tuples than the one estimated then simply switch
to normal sort for the remaining tuples, as the estimates does not
hold true anyway. Atleast this will not give issues of having
regressions of incremental sort being too bad than the normal sort.
I mean having something like this, populate the tuplesortstate and
keep on counting the number of tuples in a group, if they are within
the budget call tuplesort_performsort, otherwise put all the tuples in
the tuplesort and then call tuplesort_performsort. We may have an
additional field in IncrementalSortState to save the estimated size of
each group. I am assuming that we use MCV lists to approximate better
the group sizes as suggested above by Tomas.

> Unrelated to all of the above: if I read the patch properly it
> intentionally excludes backwards scanning. I don't see any particular
> reason why that ought to be the case, and it seems like an odd
> limitation for the feature should it be merged. Should that be a
> blocker to merging?

Regarding this, I came across this,
/*
  * Incremental sort can't be used with either EXEC_FLAG_REWIND,
  * EXEC_FLAG_BACKWARD or EXEC_FLAG_MARK, because we hold only current
  * bucket in tuplesortstate.
  */
I think that is quite understandable. How are you planning to support
backwards scan for this? In other words, when will incremental sort be
useful for backward scan.

On a different note, I can't stop imagining this operator on the lines
similar to parallel-append, wherein multiple workers can sort the
different groups independently at the same time.

-- 
Regards,
Rafia Sabih

On Wed, Jun 5, 2019 at 12:14 PM Rafia Sabih <rafia.pghackers@gmail.com> wrote:
> > > 2) Provide some fallback at execution time. For example, we might watch
> > > the size of the group, and if we run into an unexpectedly large one we
> > > might abandon the incremental sort and switch to a "full sort" mode.
> >
> > Are there good examples of our doing this in other types of nodes
> > (whether the fallback is an entirely different algorithm/node type)? I
> > like this idea in theory, but I also think it's likely it would add a
> > very significant amount of complexity. The other problem is knowing
> > where to draw the line: you end up creating these kinds of cliffs
> > where pulling one more tuple through the incremental sort would give
> > you your batch and result in not having to pull many more tuples in a
> > regular sort node, but the fallback logic kicks in anyway.
> >
> What about having some simple mechanism for this, like if we encounter
> the group with more tuples than the one estimated then simply switch
> to normal sort for the remaining tuples, as the estimates does not
> hold true anyway. Atleast this will not give issues of having
> regressions of incremental sort being too bad than the normal sort.
> I mean having something like this, populate the tuplesortstate and
> keep on counting the number of tuples in a group, if they are within
> the budget call tuplesort_performsort, otherwise put all the tuples in
> the tuplesort and then call tuplesort_performsort. We may have an
> additional field in IncrementalSortState to save the estimated size of
> each group. I am assuming that we use MCV lists to approximate better
> the group sizes as suggested above by Tomas.

I think the first thing to do is get some concrete numbers on performance if we:

1. Only sort one group at a time.
2. Update the costing to prefer traditional sort unless we have very
high confidence we'll win with incremental sort.

It'd be nice not to have to add additional complexity if at all possible.

> > Unrelated to all of the above: if I read the patch properly it
> > intentionally excludes backwards scanning. I don't see any particular
> > reason why that ought to be the case, and it seems like an odd
> > limitation for the feature should it be merged. Should that be a
> > blocker to merging?
>
> Regarding this, I came across this,
> /*
>   * Incremental sort can't be used with either EXEC_FLAG_REWIND,
>   * EXEC_FLAG_BACKWARD or EXEC_FLAG_MARK, because we hold only current
>   * bucket in tuplesortstate.
>   */
> I think that is quite understandable. How are you planning to support
> backwards scan for this? In other words, when will incremental sort be
> useful for backward scan.

For some reason I was thinking we'd need it to support backwards scans
to be able to handle DESC sort on the index, but I've tested and
confirmed that already works. I suppose that's because the index scan
provides that ordering and the sort node doesn't need to reverse the
direction of what's provided to it. That's not particularly obvious to
someone newer to the codebase; I'm not sure if that's documented
anywhere.

> On a different note, I can't stop imagining this operator on the lines
> similar to parallel-append, wherein multiple workers can sort the
> different groups independently at the same time.

That is an interesting idea. I suppose it'd be particularly valuable
if somehow there a node that was generating each batch in parallel
already, though I'm not sure at first though what kind of query or
node would result in that. I also wonder if (assuming that weren't the
case) it would be much of an improvement since a single thread would
have to generate each batch anyway; I'm not sure if the overhead of
farming each batch out to a worker would actually be useful or if the
real blocker is the base scan.

At the very least it's an interesting idea.

---

I've been writing down notes here, and I realized that my test case
from far upthread is actually a useful setup to see how much overhead
is involved in sorting each batch individually, since it sets up data
with each batch only containing 1 tuple. That particular case is one
we could easily optimize anyway in the code and skip sorting
altogether -- might be a useful enhancement.

I hope to do some more testing and then report back with the results.

James Coleman

On Thu, Jun 13, 2019 at 11:38:12PM -0400, James Coleman wrote:
>On Wed, Jun 5, 2019 at 12:14 PM Rafia Sabih <rafia.pghackers@gmail.com> wrote:
>> > > 2) Provide some fallback at execution time. For example, we might watch
>> > > the size of the group, and if we run into an unexpectedly large one we
>> > > might abandon the incremental sort and switch to a "full sort" mode.
>> >
>> > Are there good examples of our doing this in other types of nodes
>> > (whether the fallback is an entirely different algorithm/node type)? I
>> > like this idea in theory, but I also think it's likely it would add a
>> > very significant amount of complexity. The other problem is knowing
>> > where to draw the line: you end up creating these kinds of cliffs
>> > where pulling one more tuple through the incremental sort would give
>> > you your batch and result in not having to pull many more tuples in a
>> > regular sort node, but the fallback logic kicks in anyway.
>> >
>> What about having some simple mechanism for this, like if we encounter
>> the group with more tuples than the one estimated then simply switch
>> to normal sort for the remaining tuples, as the estimates does not
>> hold true anyway. Atleast this will not give issues of having
>> regressions of incremental sort being too bad than the normal sort.
>> I mean having something like this, populate the tuplesortstate and
>> keep on counting the number of tuples in a group, if they are within
>> the budget call tuplesort_performsort, otherwise put all the tuples in
>> the tuplesort and then call tuplesort_performsort. We may have an
>> additional field in IncrementalSortState to save the estimated size of
>> each group. I am assuming that we use MCV lists to approximate better
>> the group sizes as suggested above by Tomas.
>
>I think the first thing to do is get some concrete numbers on performance if we:
>
>1. Only sort one group at a time.
>2. Update the costing to prefer traditional sort unless we have very
>high confidence we'll win with incremental sort.
>
>It'd be nice not to have to add additional complexity if at all possible.
>

+1 to that

>> > Unrelated to all of the above: if I read the patch properly it
>> > intentionally excludes backwards scanning. I don't see any particular
>> > reason why that ought to be the case, and it seems like an odd
>> > limitation for the feature should it be merged. Should that be a
>> > blocker to merging?
>>
>> Regarding this, I came across this,
>> /*
>>   * Incremental sort can't be used with either EXEC_FLAG_REWIND,
>>   * EXEC_FLAG_BACKWARD or EXEC_FLAG_MARK, because we hold only current
>>   * bucket in tuplesortstate.
>>   */
>> I think that is quite understandable. How are you planning to support
>> backwards scan for this? In other words, when will incremental sort be
>> useful for backward scan.
>
>For some reason I was thinking we'd need it to support backwards scans
>to be able to handle DESC sort on the index, but I've tested and
>confirmed that already works. I suppose that's because the index scan
>provides that ordering and the sort node doesn't need to reverse the
>direction of what's provided to it. That's not particularly obvious to
>someone newer to the codebase; I'm not sure if that's documented
>anywhere.
>

Yeah, backward scans are not about ASC/DESC, it's about being able to walk
back through the result. And we can't do that with incremental sorts
without materialization.

>> On a different note, I can't stop imagining this operator on the lines
>> similar to parallel-append, wherein multiple workers can sort the
>> different groups independently at the same time.
>
>That is an interesting idea. I suppose it'd be particularly valuable
>if somehow there a node that was generating each batch in parallel
>already, though I'm not sure at first though what kind of query or
>node would result in that. I also wonder if (assuming that weren't the
>case) it would be much of an improvement since a single thread would
>have to generate each batch anyway; I'm not sure if the overhead of
>farming each batch out to a worker would actually be useful or if the
>real blocker is the base scan.
>
>At the very least it's an interesting idea.
>

I kinda doubt that'd be very valuable. Or more precisely, we kinda already
have that capability because we can do things like this:

   -> Gather Merge
      -> Sort
         -> ... scan ...

so I imagine we'd just do an Incremental Sort here. Granted, it's not
distributed by prefix groups (I assume that's what you mean by batches
here), but I don't think that's a big problem.

>---
>
>I've been writing down notes here, and I realized that my test case
>from far upthread is actually a useful setup to see how much overhead
>is involved in sorting each batch individually, since it sets up data
>with each batch only containing 1 tuple. That particular case is one
>we could easily optimize anyway in the code and skip sorting
>altogether -- might be a useful enhancement.
>
>I hope to do some more testing and then report back with the results.
>
>James Coleman

OK.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Wed, Jun 05, 2019 at 06:14:14PM +0200, Rafia Sabih wrote:
>On Mon, 3 Jun 2019 at 15:39, James Coleman <jtc331@gmail.com> wrote:
>>
>> On Sun, Jun 2, 2019 at 5:18 PM Tomas Vondra
>> <tomas.vondra@2ndquadrant.com> wrote:
>> > Currently, the costing logic (cost_incremental_sort) assumes all groups
>> > have the same size, which is fine for uniform distributions. But for
>> > skewed distributions, that may be an issue.
>> >
>> > Consider for example a table with 1M rows, two columns, 100 groups in each
>> > column, and index on the first one.
>> >
>> >     CREATE table t (a INT, b INT);
>> >
>> >     INSERT INTO t SELECT 100*random(), 100*random()
>> >       FROM generate_series(1,1000000);
>> >
>> > Now, let's do a simple limit query to find the first row:
>> >
>> >     SELECT * FROM t ORDER BU a, b LIMIT 1;
>> >
>> > In this case the current costing logic is fine - the groups are close to
>> > average size, and we only need to sort the first group, i.e. 1% of data.
>> >
>> > Now, let's say the first group is much larger:
>> >
>> >
>> >     INSERT INTO t SELECT 0, 100*random()
>> >       FROM generate_series(1,900000) s(i);
>> >
>> >     INSERT INTO t SELECT 100*random(), 100*random()
>> >       FROM generate_series(1,100000);
>> >
>> > That is, the first group is roughly 90% of data, but the number of groups
>> > is still the same. But we need to scan 90% of data. But the average group
>> > size is still the same, so the current cost model is oblivious to this.
>>
>> Thinking out loud here: the current implementation doesn't guarantee
>> that sort groups always have the same prefix column values because
>> (from code comments) "Sorting many small groups with tuplesort is
>> inefficient). While this seems a reasonable optimization, I think it's
>> possible that thinking steered away from an optimization in the
>> opposite direction. Perhaps we should always track whether or not all
>> prefix tuples are the same (currently we only do that after reaching
>> DEFAULT_MIN_GROUP_SIZE tuples) and use that information to be able to
>> have tuplesort only care about the non-prefix columns (where currently
>> it has to sort on all pathkey columns even though for a large group
>> the prefix columns are guaranteed to be equal).
>>
>+1 for passing only the non-prefix columns to tuplesort.
>> Essentially I'm trying to think of ways that would get us to
>> comparable performance with regular sort in the case of large batch
>> sizes.
>>
>> One other thing about the DEFAULT_MIN_GROUP_SIZE logic is that in the
>> case where you have a very small group and then a very large batch,
>> we'd lose the ability to optimize in the above way. That makes me
>> wonder if we shouldn't intentionally optimize for the possibility of
>> large batch sizes, since a little extra expense per group/tuple is
>> more likely to be a non-concern with small groups anyway when there
>> are large numbers of input tuples but a relatively small limit.
>>
>What about using the knowledge of MCV here, if we know the next value
>is in MCV list then take the overhead of sorting this small group
>alone and then leverage the optimization for the larger group, by
>passing only the non-prefix columns.

Not sure. It very much depends on how expensive the comparisons are for
that particular data type. If the comparisons are cheap, then I'm not sure
it matters very much whether the group is small or large. For expensive
comparison it may not be a win either, because we need to search the MCV
lists whenever the group changes.

I guess we'll need to make some benchmarks and see if it's a win or not.

>> Thoughts?
>>
>> > So I think we should look at the MCV list, and use that information when
>> > computing the startup/total cost. I think using the first/largest group to
>> > compute the startup cost, and the average group size for total cost would
>> > do the trick.
>>
>+1
>> I think this sounds very reasonable.
>>
>> > I don't think we can do much better than this during planning. There will
>> > inevitably be cases where the costing model will push us to do the wrong
>> > thing, in either direction. The question is how serious issue that is, and
>> > whether we could remedy that during execution somehow.
>> >
>> > When we "incorrectly" pick full sort (when the incremental sort would be
>> > faster), that's obviously sad but I think it's mostly fine because it's
>> > not a regression.
>> >
>> > The opposite direction (picking incremental sort, while full sort being
>> > faster) is probably more serious, because it's a regression between
>> > releases.
>> >
>> > I don't think we can fully fix that by refining the cost model. We have
>> > two basic options:
>> >
>> > 1) Argue/show that this is not an issue in practice, because (a) such
>> > cases are very rare, and/or (b) the regression is limited. In short, the
>> > benefits of the patch outweight the losses.
>>
>> My comments above go in this direction. If we can improve performance
>> in the worst case, I think it's plausible this concern becomes a
>> non-issue.
>>
>> > 2) Provide some fallback at execution time. For example, we might watch
>> > the size of the group, and if we run into an unexpectedly large one we
>> > might abandon the incremental sort and switch to a "full sort" mode.
>>
>> Are there good examples of our doing this in other types of nodes
>> (whether the fallback is an entirely different algorithm/node type)? I
>> like this idea in theory, but I also think it's likely it would add a
>> very significant amount of complexity. The other problem is knowing
>> where to draw the line: you end up creating these kinds of cliffs
>> where pulling one more tuple through the incremental sort would give
>> you your batch and result in not having to pull many more tuples in a
>> regular sort node, but the fallback logic kicks in anyway.
>>

I don't think we have nodes where we'd switch to an entirely different
algorithm - say from hash-join to nested-loop join (as is often proposed
as a solution for excessive memory consumption). That's obviously very
complex thing to implement.

But I don't think that's the type of fallback we'd need here - IMO it's
more similar to switching from in-memory to on-disk sort. Essentially,
we'd need to disable the extra logic (detecting the prefix grouping), and
just stash all the remaining tuples into tuplesort and do regular sort.

Of course, I haven't actually implemented this, so maybe it's trickier.

>What about having some simple mechanism for this, like if we encounter
>the group with more tuples than the one estimated then simply switch
>to normal sort for the remaining tuples, as the estimates does not
>hold true anyway. Atleast this will not give issues of having
>regressions of incremental sort being too bad than the normal sort.
>I mean having something like this, populate the tuplesortstate and
>keep on counting the number of tuples in a group, if they are within
>the budget call tuplesort_performsort, otherwise put all the tuples in
>the tuplesort and then call tuplesort_performsort. We may have an
>additional field in IncrementalSortState to save the estimated size of
>each group. I am assuming that we use MCV lists to approximate better
>the group sizes as suggested above by Tomas.
>

Maybe. I suggest we try to implement the simplest solution first, trying
to do as much as possible during planning, and then try to be smart at
execution time only if necessary.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Sat, Apr 7, 2018 at 4:56 PM, Alexander Korotkov <
a(dot)korotkov(at)postgrespro(dot)ru> wrote:
> I agree with that. For bounded sort, attached patch now selects minimal
> group
> size as Min(DEFAULT_MIN_GROUP_SIZE, bound). That should improve
> "LIMIT small_number" case.

As I was working on some benchmarking I noticed that incremental sort
never seemed to switch into the top-n heapsort mode, which meant for
very large groups it significantly underperformed a regular sort since
it would have to spill to disk every time. Perhaps this indicates some
missing tests also.

I tracked that down to a missing case for IncrementalSortState in
ExecSetTupleBound and have updated the patched to correct the issue
(and confirmed it now properly switches sort modes).

That also means the optimization of choosing the min group size based
on bounds (if available) wasn't previously working.

I also haven't seen incremental sort used in any parallel plans,
though there seems to be some code intended to support it. I haven't
dug into really at all yet though, so can't comment further.

I'm attaching updated patch, and will reply separately with more
detailed comments on my current benchmarking work.

James Coleman

On Thu, Jun 13, 2019 at 11:38:12PM -0400, James Coleman wrote:
>I think the first thing to do is get some concrete numbers on performance if we:
>
>1. Only sort one group at a time.
>2. Update the costing to prefer traditional sort unless we have very
>high confidence we'll win with incremental sort.
>
>It'd be nice not to have to add additional complexity if at all possible.

I've been focusing my efforts so far on seeing how much we can
eliminate performance penalties (relative to traditional sort). It
seems that if we can improve things enough there that we'd limit the
amount of adjustment needed to costing -- we'd still need to consider
cases where the lower startup cost results in picking significantly
different plans in a broad sense (presumably due to lower startup cost
and the ability to short circuit on a limit). But I'm hopeful then we
might be able to avoid having to consult MCV lists (and we wouldn't
have that available in all cases anyway)

As I see it the two most significant concerning cases right now are:
1. Very large batches (in particular where the batch is effectively
all of the matching rows such that we're really just doing a standard
sort).
2. Many very small batches.

(From reading the whole thread through again, there's a third case:
skewed group sizes, but I'm ignoring that for now because my intuition
is that if the sort can reasonably handle the above two cases *in
execution not just planning* the skew problem will become a
non-issue-)

For (1), I'd really like to be able to find a way to still benefit
from knowing we have prefix columns already sorted. Currently the
patch doesn't do that because of the optimization for small batches
precludes knowing that all prefixes in a batch are indeed equal. As
such I've temporarily removed that optimization in my testing to see
how much of a gain we get from not needing to sort prefix columns.
Intuitively it seems plausible that this would always beat a standard
sort, since a.) equality is sometimes much cheaper than ordering, and
b.) it reduces the likelihood of spilling to disk. In addition my fix
to make top-n heapsort improves things. The one thing I'm not sure
about here is the fact that I haven't seen parallel incremental sort
happen at all, and a parallel regular sort can sometimes beat single
process incremental sort (though at the cost of higher system
resources of course). As noted upthread though I haven't really
investigated the parallel component yet at all.

I did confirm a measurable speedup with suffix-only sorting (roughly
5%, relative to the current patch).

For (2), I'd love to have an optimization for the "batches are a
single tuple" case (indeed in my most common real-world use case,
that's virtually guaranteed). But that optimization is also
incompatible with the current minimum batch size optimization. There
was previously reference to the cost of copying the tuple slot being
the problem factor with small batches, and I've confirmed there is a
performance penalty for small batches if you handle every prefix as a
separate batch (relative to the current patch), but what I plan to
investigate next is whether or not that penalty is primarily due to
the bookkeeping of frequent copying of a pivot tuple or whether it's
due to running the sort algorithm itself so frequently. If it's just
the overhead of copying the pivot tuple, then I'm wondering if adding
functionality to tuplesort to allow peeking at the first inserted
tuple might be worth it.

Alternatively I've been thinking about ways to achieve a hybrid
approach: using single-batch suffix-only sort for large batches and
multi-batch full sort for very small batches. For example, I could
imagine maintaining two different tuplesorts (one for full sort and
one for suffix-only sort) and inserting tuples into the full sorter
until the minimum batch size, and then (like now) checking every tuple
to see when we've finished the batch. If we finish the batch by, say,
2 * minimum batch size, then we perform the full sort. If we don't
find a new batch by that point, we'd go back and check to see if the
tuples we've accumulated so far are actually all the same batch. If
so, we'd move all of them to the suffix-only sorter and continue
adding rows until we hit a new batch. If not, we'd perform the full
sort, but only return tuples out of the full sorter until we encounter
the current batch, and then we'd move the remainder into the
suffix-only sorter.

Using this kind of approach you can also imagine further optimizations
like checking the current tuple against the pivot tuple only every 10
processed tuples or so. That adds another level of complication,
obviously, but could be interesting.

Unrelated: There was some discussion upthread about whether or not
abbreviated column support could/should be supported. The comments
around tuplesort_set_bound in tuplesort.c claim that for bounded sort
abbreviated keys are not useful, and, in fact, the code there disables
abbreviated sort. Given that the single largest use case for this
patch is when we have a LIMIT, I think we can conclude not supporting
abbreviated keys is reasonable.

One other (meta) question: Do we care mostly about "real" queries on
synthetic data (and comparing them to master)? Or do we care more
about comparing solely the difference in sorting cost between the two
approaches? I think some of both is necessary, but I'm curious what
you all think.

James Coleman

On Mon, Jun 24, 2019 at 12:56 PM Simon Riggs <simon@2ndquadrant.com> wrote:
>
> On Mon, 24 Jun 2019 at 16:10, James Coleman <jtc331@gmail.com> wrote:
>>
>> On Thu, Jun 13, 2019 at 11:38:12PM -0400, James Coleman wrote:
>> >I think the first thing to do is get some concrete numbers on performance if we:
>> >
>> >1. Only sort one group at a time.
>> >2. Update the costing to prefer traditional sort unless we have very
>> >high confidence we'll win with incremental sort.
>> >
>> >It'd be nice not to have to add additional complexity if at all possible.
>>
>> I've been focusing my efforts so far on seeing how much we can
>> eliminate performance penalties (relative to traditional sort). It
>> seems that if we can improve things enough there that we'd limit the
>> amount of adjustment needed to costing -- we'd still need to consider
>> cases where the lower startup cost results in picking significantly
>> different plans in a broad sense (presumably due to lower startup cost
>> and the ability to short circuit on a limit). But I'm hopeful then we
>> might be able to avoid having to consult MCV lists (and we wouldn't
>> have that available in all cases anyway)
>>
>> As I see it the two most significant concerning cases right now are:
>> 1. Very large batches (in particular where the batch is effectively
>> all of the matching rows such that we're really just doing a standard
>> sort).
>> 2. Many very small batches.
>
>
> What is the specific use case for this? This sounds quite general case.

They are both general cases in some sense, but the concerns lie mostly
with what happens when they're unexpectedly encountered. For example,
if the expected row count or group size is off by a good bit and we
effectively have to perform a sort of all (or most) possible rows.

If we can get the performance to a point where that misestimated row
count or group size doesn't much matter, then ISTM including the patch
becomes a much more obvious total win.

> Do we know something about the nearly-sorted rows that could help us? Or could we introduce some information
elsewherethat would help with the sort?
 
>
> Could we for-example, pre-sort the rows block by block, or filter out the rows that are clearly out of order, so we
canre-merge them later?
 

I'm not sure what you mean by "block by block"?

James Coleman

On Mon, Jun 24, 2019 at 01:00:50PM -0400, James Coleman wrote:
>On Mon, Jun 24, 2019 at 12:56 PM Simon Riggs <simon@2ndquadrant.com> wrote:
>>
>> On Mon, 24 Jun 2019 at 16:10, James Coleman <jtc331@gmail.com> wrote:
>>>
>>> On Thu, Jun 13, 2019 at 11:38:12PM -0400, James Coleman wrote:
>>> >I think the first thing to do is get some concrete numbers on performance if we:
>>> >
>>> >1. Only sort one group at a time.
>>> >2. Update the costing to prefer traditional sort unless we have very
>>> >high confidence we'll win with incremental sort.
>>> >
>>> >It'd be nice not to have to add additional complexity if at all possible.
>>>
>>> I've been focusing my efforts so far on seeing how much we can
>>> eliminate performance penalties (relative to traditional sort). It
>>> seems that if we can improve things enough there that we'd limit the
>>> amount of adjustment needed to costing -- we'd still need to consider
>>> cases where the lower startup cost results in picking significantly
>>> different plans in a broad sense (presumably due to lower startup cost
>>> and the ability to short circuit on a limit). But I'm hopeful then we
>>> might be able to avoid having to consult MCV lists (and we wouldn't
>>> have that available in all cases anyway)
>>>
>>> As I see it the two most significant concerning cases right now are:
>>> 1. Very large batches (in particular where the batch is effectively
>>> all of the matching rows such that we're really just doing a standard
>>> sort).
>>> 2. Many very small batches.
>>
>>
>> What is the specific use case for this? This sounds quite general case.
>
>They are both general cases in some sense, but the concerns lie mostly
>with what happens when they're unexpectedly encountered. For example,
>if the expected row count or group size is off by a good bit and we
>effectively have to perform a sort of all (or most) possible rows.
>
>If we can get the performance to a point where that misestimated row
>count or group size doesn't much matter, then ISTM including the patch
>becomes a much more obvious total win.
>

Yes, that seems like a reasonable approach. Essentially, we're trying to
construct plausible worst case examples, and then minimize the overhead
compared to regular sort. If we get sufficiently close, then it's fine
to rely on somewhat shaky stats - like group size estimates.


>> Do we know something about the nearly-sorted rows that could help us?
>> Or could we introduce some information elsewhere that would help with
>> the sort?
>>
>> Could we for-example, pre-sort the rows block by block, or filter out
>> the rows that are clearly out of order, so we can re-merge them
>> later?
>
>I'm not sure what you mean by "block by block"?
>

I'm not sure what "block by block" means either.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 

On Mon, Jun 24, 2019 at 07:05:24PM +0100, Simon Riggs wrote:
>On Mon, 24 Jun 2019 at 18:01, James Coleman <jtc331@gmail.com> wrote:
>
>> On Mon, Jun 24, 2019 at 12:56 PM Simon Riggs <simon@2ndquadrant.com>
>> wrote:
>>
>> > What is the specific use case for this? This sounds quite general case.
>>
>> They are both general cases in some sense, but the concerns lie mostly
>> with what happens when they're unexpectedly encountered. For example,
>> if the expected row count or group size is off by a good bit and we
>> effectively have to perform a sort of all (or most) possible rows.
>>
>> If we can get the performance to a point where that misestimated row
>> count or group size doesn't much matter, then ISTM including the patch
>> becomes a much more obvious total win.
>>
>
>I was trying to think of ways of using external information/circumstance to
>knowingly avoid negative use cases. i.e. don't treat sort as a black box,
>use its context.
>

Like what, for example? I'm not saying there's no such additional
information, but I can't think of anything at the moment.

One of the issues is that while the decision whether to use incremental
sort is done during planning, most of the additional (and reliable)
infomation about the data is only available at execution time. And it's
likely not available explicitly - we need to deduce it. Which I think is
exactly what the min group size heuristics is about, no?


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 

On Mon, Jun 24, 2019 at 4:16 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Mon, Jun 24, 2019 at 01:00:50PM -0400, James Coleman wrote:
> >On Mon, Jun 24, 2019 at 12:56 PM Simon Riggs <simon@2ndquadrant.com> wrote:
> >>
> >> On Mon, 24 Jun 2019 at 16:10, James Coleman <jtc331@gmail.com> wrote:
> >>>
> >>> On Thu, Jun 13, 2019 at 11:38:12PM -0400, James Coleman wrote:
...
> >>> As I see it the two most significant concerning cases right now are:
> >>> 1. Very large batches (in particular where the batch is effectively
> >>> all of the matching rows such that we're really just doing a standard
> >>> sort).
> >>> 2. Many very small batches.
> >>
> >>
> >> What is the specific use case for this? This sounds quite general case.
> >
> >They are both general cases in some sense, but the concerns lie mostly
> >with what happens when they're unexpectedly encountered. For example,
> >if the expected row count or group size is off by a good bit and we
> >effectively have to perform a sort of all (or most) possible rows.
> >
> >If we can get the performance to a point where that misestimated row
> >count or group size doesn't much matter, then ISTM including the patch
> >becomes a much more obvious total win.
> >
>
> Yes, that seems like a reasonable approach. Essentially, we're trying to
> construct plausible worst case examples, and then minimize the overhead
> compared to regular sort. If we get sufficiently close, then it's fine
> to rely on somewhat shaky stats - like group size estimates.

I have a bit of a mystery in my performance testing. I've been setting
up a table like so:

create table foo(pk serial primary key, owner_fk integer, created_at timestamp);
insert into foo(owner_fk, created_at)
select fk_t.i, now() - (time_t.i::text || ' minutes')::interval
from generate_series(1, 10000) time_t(i)
cross join generate_series(1, 1000) fk_t(i);
-- double up on one set to guarantee matching prefixes
insert into foo (owner_fk, created_at) select owner_fk, created_at
from foo where owner_fk = 23;
create index idx_foo_on_owner_and_created_at on foo(owner_fk, created_at);
analyze foo;

and then I have the following query:

select *
from foo
where owner_fk = 23
order by created_at desc, pk desc
limit 20000;

The idea here is to force a bit of a worst case for small groups: we
have 10,000 batches (i.e., equal prefix groups) of 2 tuples each and
then query with a limit matching the actual number of rows we know
will match the query -- so even though there's a limit we're forcing a
total sort (and also guaranteeing both plans have to touch the same
number of rows). Note: I know that batches of size is actually the
worst case, but I chose batches of two because I've also been testing
a change that would skip the sort entirely for single tuple batches.

On master (really the commit right before the current revision of the
patch), I get:
latency average = 14.271 ms
tps = 70.075243 (excluding connections establishing)

With the patch (and incremental sort enabled):
latency average = 11.975 ms
tps = 83.512090 (excluding connections establishing)

With the patch (but incremental sort disabled):
latency average = 11.884 ms
tps = 84.149834 (excluding connections establishing)

All of those are 60 seconds runs on pgbench with a single thread.

So we have a very substantial speedup with patch *even if the new
feature isn't enabled*. I've confirmed the plan looks the same on
patched with incremental sort disabled and master. The only changes
that would seem to really effect execution time would be the changes
to tuplesort.c, but looking through them I don't see anything I'd
expect to change things so dramatically.

Any thoughts on this?

James Coleman

On Mon, Jun 24, 2019 at 07:34:19PM -0400, James Coleman wrote:
>On Mon, Jun 24, 2019 at 4:16 PM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>>
>> On Mon, Jun 24, 2019 at 01:00:50PM -0400, James Coleman wrote:
>> >On Mon, Jun 24, 2019 at 12:56 PM Simon Riggs <simon@2ndquadrant.com> wrote:
>> >>
>> >> On Mon, 24 Jun 2019 at 16:10, James Coleman <jtc331@gmail.com> wrote:
>> >>>
>> >>> On Thu, Jun 13, 2019 at 11:38:12PM -0400, James Coleman wrote:
>...
>> >>> As I see it the two most significant concerning cases right now are:
>> >>> 1. Very large batches (in particular where the batch is effectively
>> >>> all of the matching rows such that we're really just doing a standard
>> >>> sort).
>> >>> 2. Many very small batches.
>> >>
>> >>
>> >> What is the specific use case for this? This sounds quite general case.
>> >
>> >They are both general cases in some sense, but the concerns lie mostly
>> >with what happens when they're unexpectedly encountered. For example,
>> >if the expected row count or group size is off by a good bit and we
>> >effectively have to perform a sort of all (or most) possible rows.
>> >
>> >If we can get the performance to a point where that misestimated row
>> >count or group size doesn't much matter, then ISTM including the patch
>> >becomes a much more obvious total win.
>> >
>>
>> Yes, that seems like a reasonable approach. Essentially, we're trying to
>> construct plausible worst case examples, and then minimize the overhead
>> compared to regular sort. If we get sufficiently close, then it's fine
>> to rely on somewhat shaky stats - like group size estimates.
>
>I have a bit of a mystery in my performance testing. I've been setting
>up a table like so:
>
>create table foo(pk serial primary key, owner_fk integer, created_at timestamp);
>insert into foo(owner_fk, created_at)
>select fk_t.i, now() - (time_t.i::text || ' minutes')::interval
>from generate_series(1, 10000) time_t(i)
>cross join generate_series(1, 1000) fk_t(i);
>-- double up on one set to guarantee matching prefixes
>insert into foo (owner_fk, created_at) select owner_fk, created_at
>from foo where owner_fk = 23;
>create index idx_foo_on_owner_and_created_at on foo(owner_fk, created_at);
>analyze foo;
>
>and then I have the following query:
>
>select *
>from foo
>where owner_fk = 23
>order by created_at desc, pk desc
>limit 20000;
>
>The idea here is to force a bit of a worst case for small groups: we
>have 10,000 batches (i.e., equal prefix groups) of 2 tuples each and
>then query with a limit matching the actual number of rows we know
>will match the query -- so even though there's a limit we're forcing a
>total sort (and also guaranteeing both plans have to touch the same
>number of rows). Note: I know that batches of size is actually the
>worst case, but I chose batches of two because I've also been testing
>a change that would skip the sort entirely for single tuple batches.
>
>On master (really the commit right before the current revision of the
>patch), I get:
>latency average = 14.271 ms
>tps = 70.075243 (excluding connections establishing)
>
>With the patch (and incremental sort enabled):
>latency average = 11.975 ms
>tps = 83.512090 (excluding connections establishing)
>
>With the patch (but incremental sort disabled):
>latency average = 11.884 ms
>tps = 84.149834 (excluding connections establishing)
>
>All of those are 60 seconds runs on pgbench with a single thread.
>
>So we have a very substantial speedup with patch *even if the new
>feature isn't enabled*. I've confirmed the plan looks the same on
>patched with incremental sort disabled and master. The only changes
>that would seem to really effect execution time would be the changes
>to tuplesort.c, but looking through them I don't see anything I'd
>expect to change things so dramatically.
>
>Any thoughts on this?
>

I can reproduce the same thing, so it's not just you. On my machine, I see
these tps numbers (average of 10 runs, 60 seconds each):

  master:        65.177
  patched (on):  80.368
  patched (off): 80.750

The numbers are very consistent (within 1 tps).

I've done a bit of CPU profiling, and on master I see this:

    13.84%  postgres  postgres            [.] comparetup_heap
     4.83%  postgres  postgres            [.] qsort_tuple
     3.87%  postgres  postgres            [.] pg_ltostr_zeropad
     3.55%  postgres  postgres            [.] pg_ltoa
     3.19%  postgres  postgres            [.] AllocSetAlloc
     2.68%  postgres  libc-2.28.so        [.] __GI___strlen_sse2
     2.38%  postgres  postgres            [.] LWLockRelease
     2.38%  postgres  postgres            [.] AppendSeconds.constprop.9
     2.22%  postgres  libc-2.28.so        [.] __memmove_sse2_unaligned_erms
     2.17%  postgres  postgres            [.] GetPrivateRefCountEntry
     2.03%  postgres  postgres            [.] j2date
     ...

while on patched versions I see this:

     4.60%  postgres  postgres            [.] pg_ltostr_zeropad
     4.51%  postgres  postgres            [.] pg_ltoa
     3.50%  postgres  postgres            [.] AllocSetAlloc
     3.34%  postgres  libc-2.28.so        [.] __GI___strlen_sse2
     2.99%  postgres  postgres            [.] LWLockRelease
     2.84%  postgres  postgres            [.] AppendSeconds.constprop.9
     2.65%  postgres  postgres            [.] GetPrivateRefCountEntry
     2.64%  postgres  postgres            [.] j2date
     2.60%  postgres  postgres            [.] printtup
     2.56%  postgres  postgres            [.] heap_hot_search_buffer
     ...
     1.35%  postgres  postgres            [.] comparetup_heap
     ...

So either we're calling comparetup_heap less often, or it's cheaper.

But it seems to be very dependent on the data set. If you do this:

    create table foo_2 as select * from foo order by random();
    alter table foo_2 add primary key (pk);
    create index idx_foo_2_on_owner_and_created_at on foo_2 (owner_fk, created_at);

and then run the test against this table, there's no difference.

So my guess is this particular data set triggers slightly different
behavior in tuplesort, reducing the cost of comparetup_heap. The speedup
is quite significant (~20% on my system), the question is how widely
applicable can it be.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Tue, Jun 25, 2019 at 12:02 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Mon, Jun 24, 2019 at 07:34:19PM -0400, James Coleman wrote:
> >On Mon, Jun 24, 2019 at 4:16 PM Tomas Vondra
> ><tomas.vondra@2ndquadrant.com> wrote:
> >>
> >> On Mon, Jun 24, 2019 at 01:00:50PM -0400, James Coleman wrote:
> >> >On Mon, Jun 24, 2019 at 12:56 PM Simon Riggs <simon@2ndquadrant.com> wrote:
> >> >>
> >> >> On Mon, 24 Jun 2019 at 16:10, James Coleman <jtc331@gmail.com> wrote:
> >> >>>
> >> >>> On Thu, Jun 13, 2019 at 11:38:12PM -0400, James Coleman wrote:
> >...
> >> >>> As I see it the two most significant concerning cases right now are:
> >> >>> 1. Very large batches (in particular where the batch is effectively
> >> >>> all of the matching rows such that we're really just doing a standard
> >> >>> sort).
> >> >>> 2. Many very small batches.
> >> >>
> >> >>
> >> >> What is the specific use case for this? This sounds quite general case.
> >> >
> >> >They are both general cases in some sense, but the concerns lie mostly
> >> >with what happens when they're unexpectedly encountered. For example,
> >> >if the expected row count or group size is off by a good bit and we
> >> >effectively have to perform a sort of all (or most) possible rows.
> >> >
> >> >If we can get the performance to a point where that misestimated row
> >> >count or group size doesn't much matter, then ISTM including the patch
> >> >becomes a much more obvious total win.
> >> >
> >>
> >> Yes, that seems like a reasonable approach. Essentially, we're trying to
> >> construct plausible worst case examples, and then minimize the overhead
> >> compared to regular sort. If we get sufficiently close, then it's fine
> >> to rely on somewhat shaky stats - like group size estimates.
> >
> >I have a bit of a mystery in my performance testing. I've been setting
> >up a table like so:
> >
> >create table foo(pk serial primary key, owner_fk integer, created_at timestamp);
> >insert into foo(owner_fk, created_at)
> >select fk_t.i, now() - (time_t.i::text || ' minutes')::interval
> >from generate_series(1, 10000) time_t(i)
> >cross join generate_series(1, 1000) fk_t(i);
> >-- double up on one set to guarantee matching prefixes
> >insert into foo (owner_fk, created_at) select owner_fk, created_at
> >from foo where owner_fk = 23;
> >create index idx_foo_on_owner_and_created_at on foo(owner_fk, created_at);
> >analyze foo;
> >
> >and then I have the following query:
> >
> >select *
> >from foo
> >where owner_fk = 23
> >order by created_at desc, pk desc
> >limit 20000;
> >
> >The idea here is to force a bit of a worst case for small groups: we
> >have 10,000 batches (i.e., equal prefix groups) of 2 tuples each and
> >then query with a limit matching the actual number of rows we know
> >will match the query -- so even though there's a limit we're forcing a
> >total sort (and also guaranteeing both plans have to touch the same
> >number of rows). Note: I know that batches of size is actually the
> >worst case, but I chose batches of two because I've also been testing
> >a change that would skip the sort entirely for single tuple batches.
> >
> >On master (really the commit right before the current revision of the
> >patch), I get:
> >latency average = 14.271 ms
> >tps = 70.075243 (excluding connections establishing)
> >
> >With the patch (and incremental sort enabled):
> >latency average = 11.975 ms
> >tps = 83.512090 (excluding connections establishing)
> >
> >With the patch (but incremental sort disabled):
> >latency average = 11.884 ms
> >tps = 84.149834 (excluding connections establishing)
> >
> >All of those are 60 seconds runs on pgbench with a single thread.
> >
> >So we have a very substantial speedup with patch *even if the new
> >feature isn't enabled*. I've confirmed the plan looks the same on
> >patched with incremental sort disabled and master. The only changes
> >that would seem to really effect execution time would be the changes
> >to tuplesort.c, but looking through them I don't see anything I'd
> >expect to change things so dramatically.
> >
> >Any thoughts on this?
> >
>
> I can reproduce the same thing, so it's not just you. On my machine, I see
> these tps numbers (average of 10 runs, 60 seconds each):
>
>   master:        65.177
>   patched (on):  80.368
>   patched (off): 80.750
>
> The numbers are very consistent (within 1 tps).
>
> I've done a bit of CPU profiling, and on master I see this:
>
>     13.84%  postgres  postgres            [.] comparetup_heap
>      4.83%  postgres  postgres            [.] qsort_tuple
>      3.87%  postgres  postgres            [.] pg_ltostr_zeropad
>      3.55%  postgres  postgres            [.] pg_ltoa
>      3.19%  postgres  postgres            [.] AllocSetAlloc
>      2.68%  postgres  libc-2.28.so        [.] __GI___strlen_sse2
>      2.38%  postgres  postgres            [.] LWLockRelease
>      2.38%  postgres  postgres            [.] AppendSeconds.constprop.9
>      2.22%  postgres  libc-2.28.so        [.] __memmove_sse2_unaligned_erms
>      2.17%  postgres  postgres            [.] GetPrivateRefCountEntry
>      2.03%  postgres  postgres            [.] j2date
>      ...
>
> while on patched versions I see this:
>
>      4.60%  postgres  postgres            [.] pg_ltostr_zeropad
>      4.51%  postgres  postgres            [.] pg_ltoa
>      3.50%  postgres  postgres            [.] AllocSetAlloc
>      3.34%  postgres  libc-2.28.so        [.] __GI___strlen_sse2
>      2.99%  postgres  postgres            [.] LWLockRelease
>      2.84%  postgres  postgres            [.] AppendSeconds.constprop.9
>      2.65%  postgres  postgres            [.] GetPrivateRefCountEntry
>      2.64%  postgres  postgres            [.] j2date
>      2.60%  postgres  postgres            [.] printtup
>      2.56%  postgres  postgres            [.] heap_hot_search_buffer
>      ...
>      1.35%  postgres  postgres            [.] comparetup_heap
>      ...
>
> So either we're calling comparetup_heap less often, or it's cheaper.
>
> But it seems to be very dependent on the data set. If you do this:
>
>     create table foo_2 as select * from foo order by random();
>     alter table foo_2 add primary key (pk);
>     create index idx_foo_2_on_owner_and_created_at on foo_2 (owner_fk, created_at);
>
> and then run the test against this table, there's no difference.
>
> So my guess is this particular data set triggers slightly different
> behavior in tuplesort, reducing the cost of comparetup_heap. The speedup
> is quite significant (~20% on my system), the question is how widely
> applicable can it be.

Thanks for confirming!

Given the patch contents I don't see any obviously reason why either
of those possibilities (calling comparetup_heap less often, or it's
cheaper) are likely. Is that something we should investigate further?
Or is it just a nice happy accident that we should ignore since it's
dataset specific?

Anyway, when evaluating the patch performance with oddities like this
would you compare performance of incremental sort on to off on the
same revision or to master when determining if it's a regression in
performance? I could make an argument either way I think.

James Coleman

On Tue, Jun 25, 2019 at 9:53 AM James Coleman <jtc331@gmail.com> wrote:
> Given the patch contents I don't see any obviously reason why either
> of those possibilities (calling comparetup_heap less often, or it's
> cheaper) are likely. Is that something we should investigate further?
> Or is it just a nice happy accident that we should ignore since it's
> dataset specific?

Have you actually confirmed that comparetup_heap() is called less
often, by instrumenting the number of individual calls specifically?
If there has been a reduction in calls to comparetup_heap(), then
that's weird, and needs to be explained. If it turns out that it isn't
actually called less often, then I would suggest that the speedup
might be related to memory fragmentation, which often matters a lot
within tuplesort.c. (This is why external sort merging now uses big
buffers, and double buffering.)

-- 
Peter Geoghegan

On Tue, Jun 25, 2019 at 1:13 PM Peter Geoghegan <pg@bowt.ie> wrote:
>
> On Tue, Jun 25, 2019 at 9:53 AM James Coleman <jtc331@gmail.com> wrote:
> > Given the patch contents I don't see any obviously reason why either
> > of those possibilities (calling comparetup_heap less often, or it's
> > cheaper) are likely. Is that something we should investigate further?
> > Or is it just a nice happy accident that we should ignore since it's
> > dataset specific?
>
> Have you actually confirmed that comparetup_heap() is called less
> often, by instrumenting the number of individual calls specifically?
> If there has been a reduction in calls to comparetup_heap(), then
> that's weird, and needs to be explained. If it turns out that it isn't
> actually called less often, then I would suggest that the speedup
> might be related to memory fragmentation, which often matters a lot
> within tuplesort.c. (This is why external sort merging now uses big
> buffers, and double buffering.)

No, I haven't confirmed that it's called less frequently, and I'd be
extremely surprised if it were given the diff doesn't suggest any
changes to that at all.

If you think it's important enough to do so, I can instrument it to
confirm, but I was mostly wanting to know if there were any other
plausible explanations, and I think you've provided one: there *are*
changes in the patch to memory contexts in tuplesort.c, so if memory
fragmentation is a real concern this patch could definitely notice
changes in that regard.

James Coleman

On Tue, Jun 25, 2019 at 11:03 AM James Coleman <jtc331@gmail.com> wrote:
> No, I haven't confirmed that it's called less frequently, and I'd be
> extremely surprised if it were given the diff doesn't suggest any
> changes to that at all.

I must have misunderstood, then. I thought that you were suggesting
that that might have happened.

> If you think it's important enough to do so, I can instrument it to
> confirm, but I was mostly wanting to know if there were any other
> plausible explanations, and I think you've provided one: there *are*
> changes in the patch to memory contexts in tuplesort.c, so if memory
> fragmentation is a real concern this patch could definitely notice
> changes in that regard.

Sounds like it's probably fragmentation. That's generally hard to measure.

-- 
Peter Geoghegan

On Tue, Jun 25, 2019 at 12:13:01PM -0700, Peter Geoghegan wrote:
>On Tue, Jun 25, 2019 at 11:03 AM James Coleman <jtc331@gmail.com> wrote:
>> No, I haven't confirmed that it's called less frequently, and I'd be
>> extremely surprised if it were given the diff doesn't suggest any
>> changes to that at all.
>
>I must have misunderstood, then. I thought that you were suggesting
>that that might have happened.
>
>> If you think it's important enough to do so, I can instrument it to
>> confirm, but I was mostly wanting to know if there were any other
>> plausible explanations, and I think you've provided one: there *are*
>> changes in the patch to memory contexts in tuplesort.c, so if memory
>> fragmentation is a real concern this patch could definitely notice
>> changes in that regard.
>
>Sounds like it's probably fragmentation. That's generally hard to measure.
>

I'm not sure I'm really conviced this explains the difference, because
the changes in tuplesort.c are actually fairly small - we do split the
tuplesort context into two, but vast majority of the stuff is allocated
in one of the contexts (essentially just the tuplesort state gets moved
to a new context). I wouldn't expect this to have such strong impact on
locality/fragmentation.

But maybe it does - in that case it seems it might be worthwile to do it
separately, irrespectedly of the incremental sort patch. I wonder if
perf would show that as cache hits/misses, or something?

It shouldn't be that difficult to separate this change into a separate
patch, and benchmark it on it's own, though.


FWIW while looking at the tuplesort.c changes, I've noticed some
inaccurate comments in tuplesort_free. Firstly, the top-level comment
says:

/*
 * tuplesort_free
 *
 *    Internal routine for freeing resources of tuplesort.
 */

without mentioning which resources it actually releases, so it kinda
suggests it releases everything. But that's not true - AFAICS it only
releases the per-sort resources. IMO this is a poor function name, and
people will easily keep resources longer than they think - we should
rename it to something like tuplesort_free_batch().

And then at the end tuplesort_free() does this:

    /*
     * Free the per-sort memory context, thereby releasing all working memory,
     * including the Tuplesortstate struct itself.
     */
    MemoryContextReset(state->sortcontext);

But that's clearly not true, because the tuplesortstate is allocated in
the maincontext, not sortcontext.

In general, the comments seem to be a bit confused by what 'sort' means.
Sometimes it means the whole sort operation, sometimes it means one of
the batches, etc.  And the fact that the per-batch context is called
sortcontext does not really improve the situation.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 

On Tue, Jun 25, 2019 at 4:32 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Tue, Jun 25, 2019 at 12:13:01PM -0700, Peter Geoghegan wrote:
> >On Tue, Jun 25, 2019 at 11:03 AM James Coleman <jtc331@gmail.com> wrote:
> >> No, I haven't confirmed that it's called less frequently, and I'd be
> >> extremely surprised if it were given the diff doesn't suggest any
> >> changes to that at all.
> >
> >I must have misunderstood, then. I thought that you were suggesting
> >that that might have happened.
> >
> >> If you think it's important enough to do so, I can instrument it to
> >> confirm, but I was mostly wanting to know if there were any other
> >> plausible explanations, and I think you've provided one: there *are*
> >> changes in the patch to memory contexts in tuplesort.c, so if memory
> >> fragmentation is a real concern this patch could definitely notice
> >> changes in that regard.
> >
> >Sounds like it's probably fragmentation. That's generally hard to measure.
> >
>
> I'm not sure I'm really conviced this explains the difference, because
> the changes in tuplesort.c are actually fairly small - we do split the
> tuplesort context into two, but vast majority of the stuff is allocated
> in one of the contexts (essentially just the tuplesort state gets moved
> to a new context). I wouldn't expect this to have such strong impact on
> locality/fragmentation.

OTOH it is as you noted heavily dependent on data...so it's hard to
say if it's a real win or not.

> But maybe it does - in that case it seems it might be worthwile to do it
> separately, irrespectedly of the incremental sort patch. I wonder if
> perf would show that as cache hits/misses, or something?
>
> It shouldn't be that difficult to separate this change into a separate
> patch, and benchmark it on it's own, though.

I don't know enough about perf to say, but unless this ends up being a
sticking point for the patch I'll probably avoid it for now because
there are too many other things to worry about in the patch.

> FWIW while looking at the tuplesort.c changes, I've noticed some
> inaccurate comments in tuplesort_free. Firstly, the top-level comment
> says:
>
> /*
>  * tuplesort_free
>  *
>  *      Internal routine for freeing resources of tuplesort.
>  */
>
> without mentioning which resources it actually releases, so it kinda
> suggests it releases everything. But that's not true - AFAICS it only
> releases the per-sort resources. IMO this is a poor function name, and
> people will easily keep resources longer than they think - we should
> rename it to something like tuplesort_free_batch().
>
> And then at the end tuplesort_free() does this:
>
>     /*
>      * Free the per-sort memory context, thereby releasing all working memory,
>      * including the Tuplesortstate struct itself.
>      */
>     MemoryContextReset(state->sortcontext);
>
> But that's clearly not true, because the tuplesortstate is allocated in
> the maincontext, not sortcontext.
>
> In general, the comments seem to be a bit confused by what 'sort' means.
> Sometimes it means the whole sort operation, sometimes it means one of
> the batches, etc.  And the fact that the per-batch context is called
> sortcontext does not really improve the situation.

There are also quite a few misleading or out of date comments in
nodeIncrementalSort.c as well. I'm currently working on the hybrid
approach I mentioned earlier, but once the patch proper looks like
we're coming close to addressing the performance concerns/costing I'll
look at doing a pass through the comments to clean them up.

Unrelated: if you or someone else you know that's more familiar with
the parallel code, I'd be interested in their looking at the patch at
some point, because I have a suspicion it might not be operating in
parallel ever (either that or I don't know how to trigger it), but I'm
not really familiar with that stuff at all currently. :)

James Coleman

On Tue, Jun 25, 2019 at 04:53:40PM -0400, James Coleman wrote:
>
>Unrelated: if you or someone else you know that's more familiar with
>the parallel code, I'd be interested in their looking at the patch at
>some point, because I have a suspicion it might not be operating in
>parallel ever (either that or I don't know how to trigger it), but I'm
>not really familiar with that stuff at all currently. :)
>

That's an interesting question. I don't think plans like this would be
very interesting:

    Limit
      -> Incremental Sort
         -> Gather Merge
            -> Index Scan

because most of the extra cost would be paid in the leader anyway. So
I'm not all that surprised those paths are not generated (I might be
wrong and those plans would be interesting, though).

But I think something like this would be quite beneficial:

    Limit
     -> Gather Merge
        -> Incremental Sort
           -> Index Scan

So I've looked into that, and the reason seems fairly simple - when
generating the Gather Merge paths, we only look at paths that are in
partial_pathlist. See generate_gather_paths().

And we only have sequential + index paths in partial_pathlist, not
incremental sort paths.

IMHO we can do two things:

1) modify generate_gather_paths to also consider incremental sort for
each sorted path, similarly to what create_ordered_paths does

2) modify build_index_paths to also generate an incremental sort path
for each index path

IMHO (1) is the right choice here, because it automatically does the
trick for all other types of ordered paths, not just index scans. So,
something like the attached patch, which gives me plans like this:


                                                                           QUERY PLAN

--------------------------------------------------------------------------------------------------------------------------------------------------------
     Limit  (cost=0.86..2.85 rows=100000 width=12) (actual time=3.726..233.249 rows=100000 loops=1)
       ->  Gather Merge  (cost=0.86..120.00 rows=5999991 width=12) (actual time=3.724..156.802 rows=100000 loops=1)
             Workers Planned: 2
             Workers Launched: 2
             ->  Incremental Sort  (cost=0.84..100.00 rows=2499996 width=12) (actual time=0.563..164.438 rows=33910
loops=3)
                   Sort Key: a, b
                   Presorted Key: a
                   Sort Method: quicksort  Memory: 27kB
                   Sort Groups: 389
                   Worker 0:  Sort Method: quicksort  Memory: 27kB  Groups: 1295
                   Worker 1:  Sort Method: quicksort  Memory: 27kB  Groups: 1241
                   ->  Parallel Index Scan using t_a_idx on t  (cost=0.43..250612.29 rows=2499996 width=12) (actual
time=0.027..128.518rows=33926 loops=3)
 
     Planning Time: 68559.695 ms
     Execution Time: 285.245 ms
    (14 rows)


This is not the whole story, though - there seems to be some costing
issue, because even with the parallel costs set to 0, I only get such
plans after I tweak the cost in the patch like this:

    subpath->total_cost = 100.0;
    path->path.total_cost = 120.0;

When I don't do that, the gather merge gets with total cost 1037485, and
it gets beaten by this plan:

                                                                    QUERY PLAN

------------------------------------------------------------------------------------------------------------------------------------------
     Limit  (cost=1.05..152.75 rows=100000 width=12) (actual time=0.234..374.492 rows=100000 loops=1)
       ->  Incremental Sort  (cost=1.05..9103.09 rows=5999991 width=12) (actual time=0.232..316.210 rows=100000
loops=1)
             Sort Key: a, b
             Presorted Key: a
             Sort Method: quicksort  Memory: 27kB
             Sort Groups: 2863
             ->  Index Scan using t_a_idx on t  (cost=0.43..285612.24 rows=5999991 width=12) (actual
time=0.063..240.248rows=100003 loops=1)
 
     Planning Time: 53743.858 ms
     Execution Time: 403.379 ms
    (9 rows)

I suspect it's related to the fact that for the Gather Merge plan we
don't have the information about the number of rows, while for the
incremental sort we have it.

But clearly 9103.09 is not total cost for all 6M rows the incremental
sort is expected to produce (because that has to be higher than 285612,
which is the cost of the index scan). So it seems like the total cost of
the incremental sort is ~546000, because

    (100000 / 6000000) * 546000 = 9133

so close to the total cost of the incremental sort. But then

    100000.0 / 6000000 * 9133 = 152

so it seems we actually do the linear approximation twice. That seems
pretty bogus, IMO. And indeed, if I remove this part from
cost_incremental_sort:

    if (limit_tuples > 0 && limit_tuples < input_tuples)
    {
        output_tuples = limit_tuples;
        output_groups = floor(output_tuples / group_tuples) + 1;
    }

then it behaves kinda reasonable:

    explain  select * from t order by a, b limit 100000;

                                           QUERY PLAN
    -----------------------------------------------------------------------------------------
     Limit  (cost=1.05..9103.12 rows=100000 width=12)
       ->  Incremental Sort  (cost=1.05..546124.41 rows=5999991 width=12)
             Sort Key: a, b
             Presorted Key: a
             ->  Index Scan using t_a_idx on t  (cost=0.43..285612.24 rows=5999991 width=12)
    (5 rows)

    set parallel_tuple_cost = 0;
    set parallel_setup_cost = 0;

    explain  select * from t order by a, b limit 100000;

                                                   QUERY PLAN
    --------------------------------------------------------------------------------------------------------
     Limit  (cost=0.86..6775.63 rows=100000 width=12)
       ->  Gather Merge  (cost=0.86..406486.25 rows=5999991 width=12)
             Workers Planned: 2
             ->  Incremental Sort  (cost=0.84..343937.44 rows=2499996 width=12)
                   Sort Key: a, b
                   Presorted Key: a
                   ->  Parallel Index Scan using t_a_idx on t  (cost=0.43..250612.29 rows=2499996 width=12)
    (7 rows)

But I'm not going to claim those are total fixes, it's the minimum I
needed to do to make this particular type of plan work.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Mon, Jun 24, 2019 at 11:10 AM James Coleman <jtc331@gmail.com> wrote:
> As I see it the two most significant concerning cases right now are:
> 1. Very large batches (in particular where the batch is effectively
> all of the matching rows such that we're really just doing a standard
> sort).
> 2. Many very small batches.
...
> Alternatively I've been thinking about ways to achieve a hybrid
> approach: using single-batch suffix-only sort for large batches and
> multi-batch full sort for very small batches. For example, I could
> imagine maintaining two different tuplesorts (one for full sort and
> one for suffix-only sort) and inserting tuples into the full sorter
> until the minimum batch size, and then (like now) checking every tuple
> to see when we've finished the batch. If we finish the batch by, say,
> 2 * minimum batch size, then we perform the full sort. If we don't
> find a new batch by that point, we'd go back and check to see if the
> tuples we've accumulated so far are actually all the same batch. If
> so, we'd move all of them to the suffix-only sorter and continue
> adding rows until we hit a new batch. If not, we'd perform the full
> sort, but only return tuples out of the full sorter until we encounter
> the current batch, and then we'd move the remainder into the
> suffix-only sorter.

Over the past week or two I've implemented this approach. The attached
patch implements the sort retaining the minimum group size logic
already in the patch, but then after 32 tuples only looks for a
different prefix key group for the next 32 tuples. If it finds a new
prefix key group, then everything happens exactly as before. However
if it doesn't find a new prefix key group in that number of tuples,
then it assumes it's found a large set of tuples in a single prefix
key group. To guarantee this is the case it has to transfer tuples
from the full sorting tuplesort into a presorted prefix tuplesort and
(leaving out a few details here about how it handles the possibility
of multiple prefix key groups already in the sort) continues to fill
up that optimized sort with the large group.

This approach should allow us to, broadly speaking, have our cake and
eat it too with respect to small and large batches of tuples in prefix
key groups. There is of course the cost of switching modes, but this
is much like the cost that bound sort pays to switch into top-n heap
sorting mode; there's an inflection point, and you pay some extra cost
right at it, but it's worth it in the broad case.

Initial testing shows promising performance (akin to the original
patch for small groups and similar to my variant that only ever sorted
by single prefix key groups for large batches)

A couple of thoughts:
- It'd be nice if tuplesort allowed us to pull back out tuples in FIFO
manner without sorting. That'd lower the inflection point cost of
switching modes.
- I haven't adjusted costing yet for this change in approach; I wanted
to take a more holistic look at that after getting this working.
- I don't have strong feelings about the group size inflection points.
More perf testing would be useful, and also it's plausible we should
do more adjusting of those heuristics based on the size of the bound,
if we have one.
- The guc enable_sort current also disabled incremental sort, which
makes sense, but it also means there's not a good way to tweak a plan
using a full sort into a plan that uses incremental sort. That seems
not the greatest, but I'm not sure what the best solution would be.
- I did a lot of comment work in this patch, but comments for changes
to tuplesort.c and elsewhere still need to be cleaned up.

A few style notes:
- I know some of the variable declarations don't line up well; I need
to figure out how to get Vim to do what appears to be the standard
style in PG source.
- All code and comment lines are the right length, I think with the
exception of debug printf statements. I'm not sure if long strings are
an exception.
- Lining up arguments is another thing Vim isn't setup to do, though
maybe someone has some thoughts on a good approach.
I'm planning to look at the formatting utility when I get a chance,
but it'd be nice to have the editor handle the basic setup most of the
time.

Process questions:
- Do I need to explicitly move the patch somehow to the next CF?
- Since I've basically taken over patch ownership, should I move my
name from reviewer to author in the CF app? And can there be two
authors listed there?

James Coleman

On Tue, Jun 25, 2019 at 7:22 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Tue, Jun 25, 2019 at 04:53:40PM -0400, James Coleman wrote:
> >
> >Unrelated: if you or someone else you know that's more familiar with
> >the parallel code, I'd be interested in their looking at the patch at
> >some point, because I have a suspicion it might not be operating in
...
> So I've looked into that, and the reason seems fairly simple - when
> generating the Gather Merge paths, we only look at paths that are in
> partial_pathlist. See generate_gather_paths().
>
> And we only have sequential + index paths in partial_pathlist, not
> incremental sort paths.
>
> IMHO we can do two things:
>
> 1) modify generate_gather_paths to also consider incremental sort for
> each sorted path, similarly to what create_ordered_paths does
>
> 2) modify build_index_paths to also generate an incremental sort path
> for each index path
>
> IMHO (1) is the right choice here, because it automatically does the
> trick for all other types of ordered paths, not just index scans. So,
> something like the attached patch, which gives me plans like this:
...
> But I'm not going to claim those are total fixes, it's the minimum I
> needed to do to make this particular type of plan work.

Thanks for looking into this!

I intended to apply this to my most recent version of the patch (just
sent a few minutes ago), but when I apply it I noticed that the
partition_aggregate regression tests have several of these failures:

ERROR:  could not find pathkey item to sort

I haven't had time to look into the cause yet, so I decided to wait
until the next patch revision.

James Coleman

On Thu, Jul 04, 2019 at 09:29:49AM -0400, James Coleman wrote:
>On Tue, Jun 25, 2019 at 7:22 PM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>>
>> On Tue, Jun 25, 2019 at 04:53:40PM -0400, James Coleman wrote:
>> >
>> >Unrelated: if you or someone else you know that's more familiar with
>> >the parallel code, I'd be interested in their looking at the patch at
>> >some point, because I have a suspicion it might not be operating in
>...
>> So I've looked into that, and the reason seems fairly simple - when
>> generating the Gather Merge paths, we only look at paths that are in
>> partial_pathlist. See generate_gather_paths().
>>
>> And we only have sequential + index paths in partial_pathlist, not
>> incremental sort paths.
>>
>> IMHO we can do two things:
>>
>> 1) modify generate_gather_paths to also consider incremental sort for
>> each sorted path, similarly to what create_ordered_paths does
>>
>> 2) modify build_index_paths to also generate an incremental sort path
>> for each index path
>>
>> IMHO (1) is the right choice here, because it automatically does the
>> trick for all other types of ordered paths, not just index scans. So,
>> something like the attached patch, which gives me plans like this:
>...
>> But I'm not going to claim those are total fixes, it's the minimum I
>> needed to do to make this particular type of plan work.
>
>Thanks for looking into this!
>
>I intended to apply this to my most recent version of the patch (just
>sent a few minutes ago), but when I apply it I noticed that the
>partition_aggregate regression tests have several of these failures:
>
>ERROR:  could not find pathkey item to sort
>
>I haven't had time to look into the cause yet, so I decided to wait
>until the next patch revision.
>

FWIW I don't claim the patch I shared is complete and/or 100% correct.
It was more an illustration of the issue and the smallest patch to make
a particular query work. The test failures are a consequence of that.

I'll try looking into the failures over the next couple of days, but I
can't promise anything.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 

On Thu, Jul 4, 2019 at 10:46 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Thu, Jul 04, 2019 at 09:29:49AM -0400, James Coleman wrote:
> >On Tue, Jun 25, 2019 at 7:22 PM Tomas Vondra
> ><tomas.vondra@2ndquadrant.com> wrote:
> >>
> >> On Tue, Jun 25, 2019 at 04:53:40PM -0400, James Coleman wrote:
> >> >
> >> >Unrelated: if you or someone else you know that's more familiar with
> >> >the parallel code, I'd be interested in their looking at the patch at
> >> >some point, because I have a suspicion it might not be operating in
> >...
> >> So I've looked into that, and the reason seems fairly simple - when
> >> generating the Gather Merge paths, we only look at paths that are in
> >> partial_pathlist. See generate_gather_paths().
> >>
> >> And we only have sequential + index paths in partial_pathlist, not
> >> incremental sort paths.
> >>
> >> IMHO we can do two things:
> >>
> >> 1) modify generate_gather_paths to also consider incremental sort for
> >> each sorted path, similarly to what create_ordered_paths does
> >>
> >> 2) modify build_index_paths to also generate an incremental sort path
> >> for each index path
> >>
> >> IMHO (1) is the right choice here, because it automatically does the
> >> trick for all other types of ordered paths, not just index scans. So,
> >> something like the attached patch, which gives me plans like this:
> >...
> >> But I'm not going to claim those are total fixes, it's the minimum I
> >> needed to do to make this particular type of plan work.
> >
> >Thanks for looking into this!
> >
> >I intended to apply this to my most recent version of the patch (just
> >sent a few minutes ago), but when I apply it I noticed that the
> >partition_aggregate regression tests have several of these failures:
> >
> >ERROR:  could not find pathkey item to sort
> >
> >I haven't had time to look into the cause yet, so I decided to wait
> >until the next patch revision.
> >
>
> FWIW I don't claim the patch I shared is complete and/or 100% correct.
> It was more an illustration of the issue and the smallest patch to make
> a particular query work. The test failures are a consequence of that.
>
> I'll try looking into the failures over the next couple of days, but I
> can't promise anything.

Yep, I understand, I just wanted to note that it was still an
outstanding item and give a quick update on why so.

Anything you can look at is much appreciated.

James Coleman

On Thu, Jul 04, 2019 at 09:29:49AM -0400, James Coleman wrote:
>On Tue, Jun 25, 2019 at 7:22 PM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>>
>> On Tue, Jun 25, 2019 at 04:53:40PM -0400, James Coleman wrote:
>> >
>> >Unrelated: if you or someone else you know that's more familiar with
>> >the parallel code, I'd be interested in their looking at the patch at
>> >some point, because I have a suspicion it might not be operating in
>...
>> So I've looked into that, and the reason seems fairly simple - when
>> generating the Gather Merge paths, we only look at paths that are in
>> partial_pathlist. See generate_gather_paths().
>>
>> And we only have sequential + index paths in partial_pathlist, not
>> incremental sort paths.
>>
>> IMHO we can do two things:
>>
>> 1) modify generate_gather_paths to also consider incremental sort for
>> each sorted path, similarly to what create_ordered_paths does
>>
>> 2) modify build_index_paths to also generate an incremental sort path
>> for each index path
>>
>> IMHO (1) is the right choice here, because it automatically does the
>> trick for all other types of ordered paths, not just index scans. So,
>> something like the attached patch, which gives me plans like this:
>...
>> But I'm not going to claim those are total fixes, it's the minimum I
>> needed to do to make this particular type of plan work.
>
>Thanks for looking into this!
>
>I intended to apply this to my most recent version of the patch (just
>sent a few minutes ago), but when I apply it I noticed that the
>partition_aggregate regression tests have several of these failures:
>
>ERROR:  could not find pathkey item to sort
>
>I haven't had time to look into the cause yet, so I decided to wait
>until the next patch revision.
>

I wanted to investigate this today, but I can't reprodure it. How are
you building and running the regression tests?

Attached is a patch adding the incremental sort below gather merge, and
also tweaking the costing. But that's mostly for and better planning
decisions, I don't get any pathkey errors even with the first patch.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Sun, Jul 7, 2019 at 8:34 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Thu, Jul 04, 2019 at 09:29:49AM -0400, James Coleman wrote:
> >On Tue, Jun 25, 2019 at 7:22 PM Tomas Vondra
> ><tomas.vondra@2ndquadrant.com> wrote:
> >>
> >> On Tue, Jun 25, 2019 at 04:53:40PM -0400, James Coleman wrote:
> >> >
> >> >Unrelated: if you or someone else you know that's more familiar with
> >> >the parallel code, I'd be interested in their looking at the patch at
> >> >some point, because I have a suspicion it might not be operating in
> >...
> >> So I've looked into that, and the reason seems fairly simple - when
> >> generating the Gather Merge paths, we only look at paths that are in
> >> partial_pathlist. See generate_gather_paths().
> >>
> >> And we only have sequential + index paths in partial_pathlist, not
> >> incremental sort paths.
> >>
> >> IMHO we can do two things:
> >>
> >> 1) modify generate_gather_paths to also consider incremental sort for
> >> each sorted path, similarly to what create_ordered_paths does
> >>
> >> 2) modify build_index_paths to also generate an incremental sort path
> >> for each index path
> >>
> >> IMHO (1) is the right choice here, because it automatically does the
> >> trick for all other types of ordered paths, not just index scans. So,
> >> something like the attached patch, which gives me plans like this:
> >...
> >> But I'm not going to claim those are total fixes, it's the minimum I
> >> needed to do to make this particular type of plan work.
> >
> >Thanks for looking into this!
> >
> >I intended to apply this to my most recent version of the patch (just
> >sent a few minutes ago), but when I apply it I noticed that the
> >partition_aggregate regression tests have several of these failures:
> >
> >ERROR:  could not find pathkey item to sort
> >
> >I haven't had time to look into the cause yet, so I decided to wait
> >until the next patch revision.
> >
>
> I wanted to investigate this today, but I can't reprodure it. How are
> you building and running the regression tests?
>
> Attached is a patch adding the incremental sort below gather merge, and
> also tweaking the costing. But that's mostly for and better planning
> decisions, I don't get any pathkey errors even with the first patch.

On 12be7f7f997debe4e05e84b69c03ecf7051b1d79 (the last patch I sent,
which is based on top of 5683b34956b4e8da9dccadc2e3a53b86104ebb33), I
did this:

patch -p1 < ~/Downloads/parallel-incremental-sort.patch
<rebuild> (FWIW I configure with ./configure
--prefix=$HOME/postgresql-test --enable-cassert --enable-debug
--enable-depend CFLAGS="-ggdb -Og -g3 -fno-omit-frame-pointer
-DOPTIMIZER_DEBUG")
make check-world

And I get the attached regression failures.

James Coleman

On Sun, Jul 07, 2019 at 09:01:43AM -0400, James Coleman wrote:
>On Sun, Jul 7, 2019 at 8:34 AM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>>
>> On Thu, Jul 04, 2019 at 09:29:49AM -0400, James Coleman wrote:
>> >On Tue, Jun 25, 2019 at 7:22 PM Tomas Vondra
>> ><tomas.vondra@2ndquadrant.com> wrote:
>> >>
>> >> On Tue, Jun 25, 2019 at 04:53:40PM -0400, James Coleman wrote:
>> >> >
>> >> >Unrelated: if you or someone else you know that's more familiar with
>> >> >the parallel code, I'd be interested in their looking at the patch at
>> >> >some point, because I have a suspicion it might not be operating in
>> >...
>> >> So I've looked into that, and the reason seems fairly simple - when
>> >> generating the Gather Merge paths, we only look at paths that are in
>> >> partial_pathlist. See generate_gather_paths().
>> >>
>> >> And we only have sequential + index paths in partial_pathlist, not
>> >> incremental sort paths.
>> >>
>> >> IMHO we can do two things:
>> >>
>> >> 1) modify generate_gather_paths to also consider incremental sort for
>> >> each sorted path, similarly to what create_ordered_paths does
>> >>
>> >> 2) modify build_index_paths to also generate an incremental sort path
>> >> for each index path
>> >>
>> >> IMHO (1) is the right choice here, because it automatically does the
>> >> trick for all other types of ordered paths, not just index scans. So,
>> >> something like the attached patch, which gives me plans like this:
>> >...
>> >> But I'm not going to claim those are total fixes, it's the minimum I
>> >> needed to do to make this particular type of plan work.
>> >
>> >Thanks for looking into this!
>> >
>> >I intended to apply this to my most recent version of the patch (just
>> >sent a few minutes ago), but when I apply it I noticed that the
>> >partition_aggregate regression tests have several of these failures:
>> >
>> >ERROR:  could not find pathkey item to sort
>> >
>> >I haven't had time to look into the cause yet, so I decided to wait
>> >until the next patch revision.
>> >
>>
>> I wanted to investigate this today, but I can't reprodure it. How are
>> you building and running the regression tests?
>>
>> Attached is a patch adding the incremental sort below gather merge, and
>> also tweaking the costing. But that's mostly for and better planning
>> decisions, I don't get any pathkey errors even with the first patch.
>
>On 12be7f7f997debe4e05e84b69c03ecf7051b1d79 (the last patch I sent,
>which is based on top of 5683b34956b4e8da9dccadc2e3a53b86104ebb33), I
>did this:
>
>patch -p1 < ~/Downloads/parallel-incremental-sort.patch
><rebuild> (FWIW I configure with ./configure
>--prefix=$HOME/postgresql-test --enable-cassert --enable-debug
>--enable-depend CFLAGS="-ggdb -Og -g3 -fno-omit-frame-pointer
>-DOPTIMIZER_DEBUG")
>make check-world
>
>And I get the attached regression failures.
>

OK, thanks. Apparently it's the costing changes that make it go away, if
I try just the patch that tweaks generate_gather_paths() I see the same
failures. The failure happens during plan construction, so I think the
costing changes simply mean the path with incremental sort end up not
being the cheapest one (for the problematic queries), but that's just
pure luck - it's definitely an issue that needs fixing.

That error message is triggered in two places in createplan.c, and after
changing them to Assert(false) I get a core dump with this backtrace:

#0  0x0000702b3328857f in raise () from /lib64/libc.so.6
#1  0x0000702b33272895 in abort () from /lib64/libc.so.6
#2  0x0000000000a59a9d in ExceptionalCondition (conditionName=0xc52e84 "!(0)", errorType=0xc51f96 "FailedAssertion",
fileName=0xc51fe6"createplan.c", lineNumber=5937) at assert.c:54
 
#3  0x00000000007d4ab5 in prepare_sort_from_pathkeys (lefttree=0x2bbbce0, pathkeys=0x2b7a130, relids=0x0,
reqColIdx=0x0,adjust_tlist_in_place=false, p_numsortkeys=0x7ffe1abcfd6c, p_sortColIdx=0x7ffe1abcfd60,
p_sortOperators=0x7ffe1abcfd58,p_collations=0x7ffe1abcfd50, 
 
    p_nullsFirst=0x7ffe1abcfd48) at createplan.c:5937
#4  0x00000000007d4e7f in make_incrementalsort_from_pathkeys (lefttree=0x2bbbce0, pathkeys=0x2b7a130, relids=0x0,
presortedCols=1)at createplan.c:6101
 
#5  0x00000000007cdd3f in create_incrementalsort_plan (root=0x2b787c0, best_path=0x2bb92b0, flags=1) at
createplan.c:2019
#6  0x00000000007cb7ad in create_plan_recurse (root=0x2b787c0, best_path=0x2bb92b0, flags=1) at createplan.c:469
#7  0x00000000007cd778 in create_gather_merge_plan (root=0x2b787c0, best_path=0x2bb94a0) at createplan.c:1764
#8  0x00000000007cb8fb in create_plan_recurse (root=0x2b787c0, best_path=0x2bb94a0, flags=4) at createplan.c:516
#9  0x00000000007cdf10 in create_agg_plan (root=0x2b787c0, best_path=0x2bb9b28) at createplan.c:2115
#10 0x00000000007cb834 in create_plan_recurse (root=0x2b787c0, best_path=0x2bb9b28, flags=3) at createplan.c:484
#11 0x00000000007cdc16 in create_sort_plan (root=0x2b787c0, best_path=0x2bba1e8, flags=1) at createplan.c:1986
#12 0x00000000007cb78e in create_plan_recurse (root=0x2b787c0, best_path=0x2bba1e8, flags=1) at createplan.c:464
#13 0x00000000007cb4ae in create_plan (root=0x2b787c0, best_path=0x2bba1e8) at createplan.c:330
#14 0x00000000007db63c in standard_planner (parse=0x2bf5bc8, cursorOptions=256, boundParams=0x0) at planner.c:413
#15 0x00000000007db3b4 in planner (parse=0x2bf5bc8, cursorOptions=256, boundParams=0x0) at planner.c:275
#16 0x00000000008e404f in pg_plan_query (querytree=0x2bf5bc8, cursorOptions=256, boundParams=0x0) at postgres.c:878
#17 0x0000000000657afa in ExplainOneQuery (query=0x2bf5bc8, cursorOptions=256, into=0x0, es=0x2bf5fc0,
queryString=0x2a74be8"EXPLAIN (COSTS OFF)\nSELECT a, sum(b), count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3
ORDERBY 1, 2, 3;", params=0x0, queryEnv=0x0) at explain.c:371
 
#18 0x0000000000657804 in ExplainQuery (pstate=0x2a95ff8, stmt=0x2a76628, queryString=0x2a74be8 "EXPLAIN (COSTS
OFF)\nSELECTa, sum(b), count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3 ORDER BY 1, 2, 3;", params=0x0,
queryEnv=0x0,dest=0x2a95f60) at explain.c:259
 
#19 0x00000000008ec75e in standard_ProcessUtility (pstmt=0x2b8a050, queryString=0x2a74be8 "EXPLAIN (COSTS OFF)\nSELECT
a,sum(b), count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3 ORDER BY 1, 2, 3;", context=PROCESS_UTILITY_TOPLEVEL,
params=0x0,queryEnv=0x0, dest=0x2a95f60, 
 
    completionTag=0x7ffe1abd0430 "") at utility.c:675
#20 0x00000000008ebfbe in ProcessUtility (pstmt=0x2b8a050, queryString=0x2a74be8 "EXPLAIN (COSTS OFF)\nSELECT a,
sum(b),count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3 ORDER BY 1, 2, 3;", context=PROCESS_UTILITY_TOPLEVEL,
params=0x0,queryEnv=0x0, dest=0x2a95f60, 
 
    completionTag=0x7ffe1abd0430 "") at utility.c:360
#21 0x00000000008eafab in PortalRunUtility (portal=0x2adc558, pstmt=0x2b8a050, isTopLevel=true, setHoldSnapshot=true,
dest=0x2a95f60,completionTag=0x7ffe1abd0430 "") at pquery.c:1175
 
#22 0x00000000008eacb2 in FillPortalStore (portal=0x2adc558, isTopLevel=true) at pquery.c:1035
#23 0x00000000008ea60e in PortalRun (portal=0x2adc558, count=9223372036854775807, isTopLevel=true, run_once=true,
dest=0x2b8a148,altdest=0x2b8a148, completionTag=0x7ffe1abd0620 "") at pquery.c:765
 
#24 0x00000000008e45be in exec_simple_query (query_string=0x2a74be8 "EXPLAIN (COSTS OFF)\nSELECT a, sum(b), count(*)
FROMpagg_tab_ml GROUP BY a HAVING avg(b) < 3 ORDER BY 1, 2, 3;") at postgres.c:1215
 
#25 0x00000000008e8912 in PostgresMain (argc=1, argv=0x2aa07c0, dbname=0x2aa0530 "regression", username=0x2a707e8
"user")at postgres.c:4249
 
#26 0x000000000083ed83 in BackendRun (port=0x2a99c50) at postmaster.c:4431
#27 0x000000000083e551 in BackendStartup (port=0x2a99c50) at postmaster.c:4122
#28 0x000000000083a977 in ServerLoop () at postmaster.c:1704
#29 0x000000000083a223 in PostmasterMain (argc=8, argv=0x2a6e670) at postmaster.c:1377
#30 0x000000000075a823 in main (argc=8, argv=0x2a6e670) at main.c:228

I think it's pretty obvious what's happening - I'll resist the urge
to just write "The proof is obvious and is left to the reader as a
homework." because I always despised that during math lectures ;-)

We're running query like this:

  SELECT a, sum(b), count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3 ORDER BY 1, 2, 3

but we're trying to add the incremental sort *before* the aggregation,
because the optimizer also considers group aggregate with a sorted
input. And (a) is a prefix of (a,sum(b),count(b)) so we think we
actually can do this, but clearly that's nonsense, because we can't
possibly know the aggregates yet. Hence the error.

If this is the actual issue, we need to ensure we actually can evaluate
all the pathkeys. I don't know how to do that yet. I thought that maybe
we should modify pathkeys_common_contained_in() to set presorted_keys to
0 in this case.

But then I started wondering why we don't see this issue even for
regular (non-incremental-sort) paths built in create_ordered_paths().
How come we don't see these failures there? I've modified costing to
make all incremental sort paths very cheap, and still nothing.

So presumably there's a check elsewhere (either implicit or explicit),
because create_ordered_paths() uses pathkeys_common_contained_in() and
does not have the same issue.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Sun, Jul 7, 2019 at 5:02 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
> We're running query like this:
>
>   SELECT a, sum(b), count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3 ORDER BY 1, 2, 3
>
> but we're trying to add the incremental sort *before* the aggregation,
> because the optimizer also considers group aggregate with a sorted
> input. And (a) is a prefix of (a,sum(b),count(b)) so we think we
> actually can do this, but clearly that's nonsense, because we can't
> possibly know the aggregates yet. Hence the error.
>
> If this is the actual issue, we need to ensure we actually can evaluate
> all the pathkeys. I don't know how to do that yet. I thought that maybe
> we should modify pathkeys_common_contained_in() to set presorted_keys to
> 0 in this case.
>
> But then I started wondering why we don't see this issue even for
> regular (non-incremental-sort) paths built in create_ordered_paths().
> How come we don't see these failures there? I've modified costing to
> make all incremental sort paths very cheap, and still nothing.

I assume you mean you modified costing to make regular sort paths very cheap?

> So presumably there's a check elsewhere (either implicit or explicit),
> because create_ordered_paths() uses pathkeys_common_contained_in() and
> does not have the same issue.

Given this comment in create_ordered_paths():

  generate_gather_paths() will have already generated a simple Gather
  path for the best parallel path, if any, and the loop above will have
  considered sorting it.  Similarly, generate_gather_paths() will also
  have generated order-preserving Gather Merge plans which can be used
  without sorting if they happen to match the sort_pathkeys, and the loop
  above will have handled those as well.  However, there's one more
  possibility: it may make sense to sort the cheapest partial path
  according to the required output order and then use Gather Merge.

my understanding is that generate_gather_paths() only considers paths
that already happen to be sorted (not explicit sorts), so I'm
wondering if it would make more sense for the incremental sort path
creation for this case to live alongside the explicit ordered path
creation in create_ordered_paths() rather than in
generate_gather_paths().

If I'm understanding what you're saying properly, I think you'd
expected create_ordered_paths() to be roughly similar in what it
considers as partial paths and so have the same problem, and I haven't
yet read enough of the code to understand if my proposed change
actually has any impact on the issue we're discussing, but it seems to
me that it at least fits more with what the comments imply.

I'll try to look at it a bit more later also, but at the moment other
work calls.

James Coleman

On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman wrote:
>On Sun, Jul 7, 2019 at 5:02 PM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>> We're running query like this:
>>
>>   SELECT a, sum(b), count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3 ORDER BY 1, 2, 3
>>
>> but we're trying to add the incremental sort *before* the aggregation,
>> because the optimizer also considers group aggregate with a sorted
>> input. And (a) is a prefix of (a,sum(b),count(b)) so we think we
>> actually can do this, but clearly that's nonsense, because we can't
>> possibly know the aggregates yet. Hence the error.
>>
>> If this is the actual issue, we need to ensure we actually can evaluate
>> all the pathkeys. I don't know how to do that yet. I thought that maybe
>> we should modify pathkeys_common_contained_in() to set presorted_keys to
>> 0 in this case.
>>
>> But then I started wondering why we don't see this issue even for
>> regular (non-incremental-sort) paths built in create_ordered_paths().
>> How come we don't see these failures there? I've modified costing to
>> make all incremental sort paths very cheap, and still nothing.
>
>I assume you mean you modified costing to make regular sort paths very cheap?
>

No, I mean costing of incremental sort paths, so that they end up being
the cheapest ones. If some other path is cheaper, we won't see the error
because it only happens when building plan from the cheapest path.

>> So presumably there's a check elsewhere (either implicit or explicit),
>> because create_ordered_paths() uses pathkeys_common_contained_in() and
>> does not have the same issue.
>
>Given this comment in create_ordered_paths():
>
>  generate_gather_paths() will have already generated a simple Gather
>  path for the best parallel path, if any, and the loop above will have
>  considered sorting it.  Similarly, generate_gather_paths() will also
>  have generated order-preserving Gather Merge plans which can be used
>  without sorting if they happen to match the sort_pathkeys, and the loop
>  above will have handled those as well.  However, there's one more
>  possibility: it may make sense to sort the cheapest partial path
>  according to the required output order and then use Gather Merge.
>
>my understanding is that generate_gather_paths() only considers paths
>that already happen to be sorted (not explicit sorts), so I'm
>wondering if it would make more sense for the incremental sort path
>creation for this case to live alongside the explicit ordered path
>creation in create_ordered_paths() rather than in
>generate_gather_paths().
>

How would that solve the issue? Also, we're building a gather path, so
I think generate_gather_paths() is the right place where to do it. And
we're not changing the semantics of generate_gather_paths() - the result
path should be sorted "correctly" with respect to sort_pathkeys.

>If I'm understanding what you're saying properly, I think you'd
>expected create_ordered_paths() to be roughly similar in what it
>considers as partial paths and so have the same problem, and I haven't
>yet read enough of the code to understand if my proposed change
>actually has any impact on the issue we're discussing, but it seems to
>me that it at least fits more with what the comments imply.
>

Roughly. AFAICS the problem is that we're trying to use pathkeys that are
only valid for the (aggregated) upper relation, not before it.

I have to admit I've never quite understood this pathkeys business. I
mean, I know what pathkeys are for, but clearly it's not valid to look at
root->sort_pathkeys at this place. Maybe there's some other field we
should be looking at instead, or maybe it's ensured by grouping_planner in
some implicit way ...

I.e. the question is - when you do a query like

    SELECT a, count(*) FROM t GROUP BY a ORDER BY a, count(*);

and cost the incremental sort extremely cheap, how come we don't end up
with the same issue?


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman wrote:
> >On Sun, Jul 7, 2019 at 5:02 PM Tomas Vondra
> ><tomas.vondra@2ndquadrant.com> wrote:
> >> We're running query like this:
> >>
> >>   SELECT a, sum(b), count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3 ORDER BY 1, 2, 3
> >>
> >> but we're trying to add the incremental sort *before* the aggregation,
> >> because the optimizer also considers group aggregate with a sorted
> >> input. And (a) is a prefix of (a,sum(b),count(b)) so we think we
> >> actually can do this, but clearly that's nonsense, because we can't
> >> possibly know the aggregates yet. Hence the error.
> >>
> >> If this is the actual issue, we need to ensure we actually can evaluate
> >> all the pathkeys. I don't know how to do that yet. I thought that maybe
> >> we should modify pathkeys_common_contained_in() to set presorted_keys to
> >> 0 in this case.
> >>
> >> But then I started wondering why we don't see this issue even for
> >> regular (non-incremental-sort) paths built in create_ordered_paths().
> >> How come we don't see these failures there? I've modified costing to
> >> make all incremental sort paths very cheap, and still nothing.
> >
> >I assume you mean you modified costing to make regular sort paths very cheap?
> >
>
> No, I mean costing of incremental sort paths, so that they end up being
> the cheapest ones. If some other path is cheaper, we won't see the error
> because it only happens when building plan from the cheapest path.

Ah, I misunderstood as you trying to figure out a way to try to cause
the same problem with a regular sort.

> >> So presumably there's a check elsewhere (either implicit or explicit),
> >> because create_ordered_paths() uses pathkeys_common_contained_in() and
> >> does not have the same issue.
> >
> >Given this comment in create_ordered_paths():
> >
> >  generate_gather_paths() will have already generated a simple Gather
> >  path for the best parallel path, if any, and the loop above will have
> >  considered sorting it.  Similarly, generate_gather_paths() will also
> >  have generated order-preserving Gather Merge plans which can be used
> >  without sorting if they happen to match the sort_pathkeys, and the loop
> >  above will have handled those as well.  However, there's one more
> >  possibility: it may make sense to sort the cheapest partial path
> >  according to the required output order and then use Gather Merge.
> >
> >my understanding is that generate_gather_paths() only considers paths
> >that already happen to be sorted (not explicit sorts), so I'm
> >wondering if it would make more sense for the incremental sort path
> >creation for this case to live alongside the explicit ordered path
> >creation in create_ordered_paths() rather than in
> >generate_gather_paths().
> >
>
> How would that solve the issue? Also, we're building a gather path, so
> I think generate_gather_paths() is the right place where to do it. And
> we're not changing the semantics of generate_gather_paths() - the result
> path should be sorted "correctly" with respect to sort_pathkeys.

Does that imply what the explicit sort in create_ordered_paths() is in
the wrong spot?

Or, to put it another way, do you think that both kinds of sorts
should be added in the same place? It seems confusing to me that
they'd be split between the two methods (unless I'm completely
misunderstanding how the two work).

I'm not saying it would solve the issue here; just noting that the
division of labor seemed odd to me at first read through.

James Coleman

On Mon, Jul 08, 2019 at 10:32:18AM -0400, James Coleman wrote:
>On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>>
>> On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman wrote:
>> >On Sun, Jul 7, 2019 at 5:02 PM Tomas Vondra
>> ><tomas.vondra@2ndquadrant.com> wrote:
>> >> We're running query like this:
>> >>
>> >>   SELECT a, sum(b), count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3 ORDER BY 1, 2, 3
>> >>
>> >> but we're trying to add the incremental sort *before* the aggregation,
>> >> because the optimizer also considers group aggregate with a sorted
>> >> input. And (a) is a prefix of (a,sum(b),count(b)) so we think we
>> >> actually can do this, but clearly that's nonsense, because we can't
>> >> possibly know the aggregates yet. Hence the error.
>> >>
>> >> If this is the actual issue, we need to ensure we actually can evaluate
>> >> all the pathkeys. I don't know how to do that yet. I thought that maybe
>> >> we should modify pathkeys_common_contained_in() to set presorted_keys to
>> >> 0 in this case.
>> >>
>> >> But then I started wondering why we don't see this issue even for
>> >> regular (non-incremental-sort) paths built in create_ordered_paths().
>> >> How come we don't see these failures there? I've modified costing to
>> >> make all incremental sort paths very cheap, and still nothing.
>> >
>> >I assume you mean you modified costing to make regular sort paths very cheap?
>> >
>>
>> No, I mean costing of incremental sort paths, so that they end up being
>> the cheapest ones. If some other path is cheaper, we won't see the error
>> because it only happens when building plan from the cheapest path.
>
>Ah, I misunderstood as you trying to figure out a way to try to cause
>the same problem with a regular sort.
>
>> >> So presumably there's a check elsewhere (either implicit or explicit),
>> >> because create_ordered_paths() uses pathkeys_common_contained_in() and
>> >> does not have the same issue.
>> >
>> >Given this comment in create_ordered_paths():
>> >
>> >  generate_gather_paths() will have already generated a simple Gather
>> >  path for the best parallel path, if any, and the loop above will have
>> >  considered sorting it.  Similarly, generate_gather_paths() will also
>> >  have generated order-preserving Gather Merge plans which can be used
>> >  without sorting if they happen to match the sort_pathkeys, and the loop
>> >  above will have handled those as well.  However, there's one more
>> >  possibility: it may make sense to sort the cheapest partial path
>> >  according to the required output order and then use Gather Merge.
>> >
>> >my understanding is that generate_gather_paths() only considers paths
>> >that already happen to be sorted (not explicit sorts), so I'm
>> >wondering if it would make more sense for the incremental sort path
>> >creation for this case to live alongside the explicit ordered path
>> >creation in create_ordered_paths() rather than in
>> >generate_gather_paths().
>> >
>>
>> How would that solve the issue? Also, we're building a gather path, so
>> I think generate_gather_paths() is the right place where to do it. And
>> we're not changing the semantics of generate_gather_paths() - the result
>> path should be sorted "correctly" with respect to sort_pathkeys.
>
>Does that imply what the explicit sort in create_ordered_paths() is in
>the wrong spot?
>

I think those are essentially the right places where to do this sort of
stuff. Maybe there's a better place, but I don't think those places are
somehow wrong.

>Or, to put it another way, do you think that both kinds of sorts
>should be added in the same place? It seems confusing to me that
>they'd be split between the two methods (unless I'm completely
>misunderstanding how the two work).
>

The paths built in those two places were very different in one aspect:

1) generate_gather_paths only ever looked at pathkeys for the subpath, it
never even looked at ordering expected by paths above it (or the query as
a whole). Plain Gather ignores pathkeys entirely, Gather Merge only aims
to maintain ordering of the different subpaths.

2) create_oredered_paths is meant to enforce ordering needed by upper
parts of the plan - either by using a properly sorted path, or adding an
explicit sort.


We want to extend (1) to also look at ordering expected by the upper parts
of the plan, and consider incremental sort if applicable. (2) already does
that, and it already has the correct pathkeys to enforce.

But looking at root->sort_pathkeys in (1) seems to be the wrong thing :-(

The thing is, we don't have just sort_pathkeys, there's distinct_pathkeys
and group_pathkeys too, so maybe we should be looking at those too?

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Mon, Jul 8, 2019 at 10:58 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Mon, Jul 08, 2019 at 10:32:18AM -0400, James Coleman wrote:
> >On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondra
> ><tomas.vondra@2ndquadrant.com> wrote:
> >>
> >> On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman wrote:
> >> >On Sun, Jul 7, 2019 at 5:02 PM Tomas Vondra
> >> ><tomas.vondra@2ndquadrant.com> wrote:
> >> >> We're running query like this:
> >> >>
> >> >>   SELECT a, sum(b), count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3 ORDER BY 1, 2, 3
> >> >>
> >> >> but we're trying to add the incremental sort *before* the aggregation,
> >> >> because the optimizer also considers group aggregate with a sorted
> >> >> input. And (a) is a prefix of (a,sum(b),count(b)) so we think we
> >> >> actually can do this, but clearly that's nonsense, because we can't
> >> >> possibly know the aggregates yet. Hence the error.
> >> >>
> >> >> If this is the actual issue, we need to ensure we actually can evaluate
> >> >> all the pathkeys. I don't know how to do that yet. I thought that maybe
> >> >> we should modify pathkeys_common_contained_in() to set presorted_keys to
> >> >> 0 in this case.
> >> >>
> >> >> But then I started wondering why we don't see this issue even for
> >> >> regular (non-incremental-sort) paths built in create_ordered_paths().
> >> >> How come we don't see these failures there? I've modified costing to
> >> >> make all incremental sort paths very cheap, and still nothing.
> >> >
> >> >I assume you mean you modified costing to make regular sort paths very cheap?
> >> >
> >>
> >> No, I mean costing of incremental sort paths, so that they end up being
> >> the cheapest ones. If some other path is cheaper, we won't see the error
> >> because it only happens when building plan from the cheapest path.
> >
> >Ah, I misunderstood as you trying to figure out a way to try to cause
> >the same problem with a regular sort.
> >
> >> >> So presumably there's a check elsewhere (either implicit or explicit),
> >> >> because create_ordered_paths() uses pathkeys_common_contained_in() and
> >> >> does not have the same issue.
> >> >
> >> >Given this comment in create_ordered_paths():
> >> >
> >> >  generate_gather_paths() will have already generated a simple Gather
> >> >  path for the best parallel path, if any, and the loop above will have
> >> >  considered sorting it.  Similarly, generate_gather_paths() will also
> >> >  have generated order-preserving Gather Merge plans which can be used
> >> >  without sorting if they happen to match the sort_pathkeys, and the loop
> >> >  above will have handled those as well.  However, there's one more
> >> >  possibility: it may make sense to sort the cheapest partial path
> >> >  according to the required output order and then use Gather Merge.
> >> >
> >> >my understanding is that generate_gather_paths() only considers paths
> >> >that already happen to be sorted (not explicit sorts), so I'm
> >> >wondering if it would make more sense for the incremental sort path
> >> >creation for this case to live alongside the explicit ordered path
> >> >creation in create_ordered_paths() rather than in
> >> >generate_gather_paths().
> >> >
> >>
> >> How would that solve the issue? Also, we're building a gather path, so
> >> I think generate_gather_paths() is the right place where to do it. And
> >> we're not changing the semantics of generate_gather_paths() - the result
> >> path should be sorted "correctly" with respect to sort_pathkeys.
> >
> >Does that imply what the explicit sort in create_ordered_paths() is in
> >the wrong spot?
> >
>
> I think those are essentially the right places where to do this sort of
> stuff. Maybe there's a better place, but I don't think those places are
> somehow wrong.
>
> >Or, to put it another way, do you think that both kinds of sorts
> >should be added in the same place? It seems confusing to me that
> >they'd be split between the two methods (unless I'm completely
> >misunderstanding how the two work).
> >
>
> The paths built in those two places were very different in one aspect:
>
> 1) generate_gather_paths only ever looked at pathkeys for the subpath, it
> never even looked at ordering expected by paths above it (or the query as
> a whole). Plain Gather ignores pathkeys entirely, Gather Merge only aims
> to maintain ordering of the different subpaths.
>
> 2) create_oredered_paths is meant to enforce ordering needed by upper
> parts of the plan - either by using a properly sorted path, or adding an
> explicit sort.
>
>
> We want to extend (1) to also look at ordering expected by the upper parts
> of the plan, and consider incremental sort if applicable. (2) already does
> that, and it already has the correct pathkeys to enforce.

I guess I'm still not following. If (2) is responsible (currently) for
adding an explicit sort, why wouldn't adding an incremental sort be an
example of that responsibility? The subpath that either a Sort or
IncrementalSort is being added on top of (to then feed into the
GatherMerge) is the same in both cases right?

Unless you're saying that the difference is that since we have a
partial ordering already for incremental sort then incremental sort
falls into the category of "maintaining" existing ordering of the
subpath?

> But looking at root->sort_pathkeys in (1) seems to be the wrong thing :-(
>
> The thing is, we don't have just sort_pathkeys, there's distinct_pathkeys
> and group_pathkeys too, so maybe we should be looking at those too?

I don't know enough yet to answer, but I'd like to look at (in the
debugger) the subpaths considered in each function to try to get a
better understanding of why we don't try to explicitly sort the aggs
(which we know we can't sort yet) but do for incremental sort. I
assume that means a subpath has to be present in one but not the other
since they both use the same pathkey checking function.

James Coleman

On Mon, Jun 3, 2019 at 12:18 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
> For a moment I thought we could/should look at the histogram, becase that
> could tell us if there are groups "before" the first MCV one, but I don't
> think we should do that, for two reasons. Firstly, rare values may not get
> to the histogram anyway, which makes this rather unreliable and might
> introduce sudden plan changes, because the cost would vary wildly
> depending on whether we happened to sample the rare row or not. And
> secondly, the rare row may be easily filtered out by a WHERE condition or
> something, at which point we'll have to deal with the large group anyway.

If first MCV is in the middle of first histogram bin, then it's
reasonable to think that it would fit to first group.  But if first
MCV is in the middle of histogram, such assumption would be
ridiculous.  Also, I'd like to note that with our
default_statistics_target == 100, non MCV values are not so rare.  So,
I'm +1 for taking histogram into account.

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On Thu, Jul 4, 2019 at 4:25 PM James Coleman <jtc331@gmail.com> wrote:
> Process questions:
> - Do I need to explicitly move the patch somehow to the next CF?

We didn't manage to register it on current (July) commitfest.  So,
please, register it on next (September) commitfest.

> - Since I've basically taken over patch ownership, should I move my
> name from reviewer to author in the CF app? And can there be two
> authors listed there?

Sure, you're co-author of this patch.  Two or more authors could be
listed at CF app, you can find a lot of examples on the list.

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On Mon, Jul 08, 2019 at 12:07:06PM -0400, James Coleman wrote:
>On Mon, Jul 8, 2019 at 10:58 AM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>>
>> On Mon, Jul 08, 2019 at 10:32:18AM -0400, James Coleman wrote:
>> >On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondra
>> ><tomas.vondra@2ndquadrant.com> wrote:
>> >>
>> >> On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman wrote:
>> >> >On Sun, Jul 7, 2019 at 5:02 PM Tomas Vondra
>> >> ><tomas.vondra@2ndquadrant.com> wrote:
>> >> >> We're running query like this:
>> >> >>
>> >> >>   SELECT a, sum(b), count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3 ORDER BY 1, 2, 3
>> >> >>
>> >> >> but we're trying to add the incremental sort *before* the aggregation,
>> >> >> because the optimizer also considers group aggregate with a sorted
>> >> >> input. And (a) is a prefix of (a,sum(b),count(b)) so we think we
>> >> >> actually can do this, but clearly that's nonsense, because we can't
>> >> >> possibly know the aggregates yet. Hence the error.
>> >> >>
>> >> >> If this is the actual issue, we need to ensure we actually can evaluate
>> >> >> all the pathkeys. I don't know how to do that yet. I thought that maybe
>> >> >> we should modify pathkeys_common_contained_in() to set presorted_keys to
>> >> >> 0 in this case.
>> >> >>
>> >> >> But then I started wondering why we don't see this issue even for
>> >> >> regular (non-incremental-sort) paths built in create_ordered_paths().
>> >> >> How come we don't see these failures there? I've modified costing to
>> >> >> make all incremental sort paths very cheap, and still nothing.
>> >> >
>> >> >I assume you mean you modified costing to make regular sort paths very cheap?
>> >> >
>> >>
>> >> No, I mean costing of incremental sort paths, so that they end up being
>> >> the cheapest ones. If some other path is cheaper, we won't see the error
>> >> because it only happens when building plan from the cheapest path.
>> >
>> >Ah, I misunderstood as you trying to figure out a way to try to cause
>> >the same problem with a regular sort.
>> >
>> >> >> So presumably there's a check elsewhere (either implicit or explicit),
>> >> >> because create_ordered_paths() uses pathkeys_common_contained_in() and
>> >> >> does not have the same issue.
>> >> >
>> >> >Given this comment in create_ordered_paths():
>> >> >
>> >> >  generate_gather_paths() will have already generated a simple Gather
>> >> >  path for the best parallel path, if any, and the loop above will have
>> >> >  considered sorting it.  Similarly, generate_gather_paths() will also
>> >> >  have generated order-preserving Gather Merge plans which can be used
>> >> >  without sorting if they happen to match the sort_pathkeys, and the loop
>> >> >  above will have handled those as well.  However, there's one more
>> >> >  possibility: it may make sense to sort the cheapest partial path
>> >> >  according to the required output order and then use Gather Merge.
>> >> >
>> >> >my understanding is that generate_gather_paths() only considers paths
>> >> >that already happen to be sorted (not explicit sorts), so I'm
>> >> >wondering if it would make more sense for the incremental sort path
>> >> >creation for this case to live alongside the explicit ordered path
>> >> >creation in create_ordered_paths() rather than in
>> >> >generate_gather_paths().
>> >> >
>> >>
>> >> How would that solve the issue? Also, we're building a gather path, so
>> >> I think generate_gather_paths() is the right place where to do it. And
>> >> we're not changing the semantics of generate_gather_paths() - the result
>> >> path should be sorted "correctly" with respect to sort_pathkeys.
>> >
>> >Does that imply what the explicit sort in create_ordered_paths() is in
>> >the wrong spot?
>> >
>>
>> I think those are essentially the right places where to do this sort of
>> stuff. Maybe there's a better place, but I don't think those places are
>> somehow wrong.
>>
>> >Or, to put it another way, do you think that both kinds of sorts
>> >should be added in the same place? It seems confusing to me that
>> >they'd be split between the two methods (unless I'm completely
>> >misunderstanding how the two work).
>> >
>>
>> The paths built in those two places were very different in one aspect:
>>
>> 1) generate_gather_paths only ever looked at pathkeys for the subpath, it
>> never even looked at ordering expected by paths above it (or the query as
>> a whole). Plain Gather ignores pathkeys entirely, Gather Merge only aims
>> to maintain ordering of the different subpaths.
>>
>> 2) create_oredered_paths is meant to enforce ordering needed by upper
>> parts of the plan - either by using a properly sorted path, or adding an
>> explicit sort.
>>
>>
>> We want to extend (1) to also look at ordering expected by the upper parts
>> of the plan, and consider incremental sort if applicable. (2) already does
>> that, and it already has the correct pathkeys to enforce.
>
>I guess I'm still not following. If (2) is responsible (currently) for
>adding an explicit sort, why wouldn't adding an incremental sort be an
>example of that responsibility? The subpath that either a Sort or
>IncrementalSort is being added on top of (to then feed into the
>GatherMerge) is the same in both cases right?
>
>Unless you're saying that the difference is that since we have a
>partial ordering already for incremental sort then incremental sort
>falls into the category of "maintaining" existing ordering of the
>subpath?
>

Oh, I think I understand what you're saying. Essentially, we should not
be making generate_gather_paths responsible for adding the incremental
sort. Instead, we should be looking at places than are adding explicit
sort (using create_sort_path) and also consider adding incremental sort.

I definitely agree with the second half - we should look at all places
that create explicit sorts and make them also consider incremental
sorts. That makes sense.

But I'm not sure it'll address all cases - the problem is that those
places add the explicit sort because they need sorted input. Gather
Merge does not do that, it only preserves existing ordering of paths.

So it's possible the path does not have an explicit sort on to, and
gather merge will not know to add it. And once we have the gather merge
in place, we can't push place "under" it.

In fact, we already have code dealing with this "issue" for a special
case - see gather_grouping_paths(). It generates plain gather merge
paths, but then also considers building one with explicit sort. But it
only does that for grouping paths (when it's clear we need to be looking
at grouping_pathkeys), and there are generate_gather_paths() that don't
have similar treatment.


>> But looking at root->sort_pathkeys in (1) seems to be the wrong thing :-(
>>
>> The thing is, we don't have just sort_pathkeys, there's distinct_pathkeys
>> and group_pathkeys too, so maybe we should be looking at those too?
>
>I don't know enough yet to answer, but I'd like to look at (in the
>debugger) the subpaths considered in each function to try to get a
>better understanding of why we don't try to explicitly sort the aggs
>(which we know we can't sort yet) but do for incremental sort. I
>assume that means a subpath has to be present in one but not the other
>since they both use the same pathkey checking function.
>

I've been wondering if we have some other code that needs to consider
interesting pathkeys "candidates" (instead of just getting the list
interesting in that place). Because then we could look at that code and
use it here ...

And guess what - postgres_fdw needs to do pretty much exactly that, when
building paths for remote relations. AFAIK we can't easily request all
plans from the remote node and then look at their pathkeys (like we'd do
with local node), so instead we deduce "interesting pathkeys" and then
request best plans for those. And deducing "interesing" pathkeys is
pretty much what get_useful_pathkeys_for_relation() is about.

So I've copied this function (and two more, called from it), whacked it
a bit until it removed (shakespeare-writing chimp comes to mind) and
voila, it seems to be working. The errors you reported are gone, and the
plans seems to be reasonable.

Attached is a sequence of 4 patches:


0001-fix-pathkey-processing-in-generate_gather_paths.patch
----------------------------------------------------------
This is the fixed version of my previous patch, with the stuff stolen
from postgres_fdw.


0002-fix-costing-in-cost_incremental_sort.patch
-----------------------------------------------
This is the costing fix, I mentioned before.


0003-fix-explain-in-parallel-mode.patch
---------------------------------------
Minor bug in explain, when incremental sort ends up being in the
parallel part of the plan (missing newline on per-worker line)


0004-rework-where-incremental-sort-paths-are-created.patch
----------------------------------------------------------
This undoes the generate_gather_paths() changes from 0001, and instead
modifies a bunch of places that call create_sort_path() to also consider
incremental sorts. There are a couple remaining, but those should not be
relevant to the queries I've been looking at.


Essentially, 0002 and 0003 are bugfixes. 0001 and 0004 are the two
different aproaches to building incremental sort + gather merge.

Now, consider this example:

  create table t (a int, b int, c int);
  insert into t select mod(i,100),mod(i,100),i from generate_series(1,10000000) s(i);
  create index on t (a);
  analyze t;
  explain select a,b,sum(c) from t group by 1,2 order by 1,2,3 limit 1;

With 0001+0002+0003 pathes, I get a plan like this:

                                                     QUERY PLAN                                                     
--------------------------------------------------------------------------------------------------------------------
 Limit  (cost=10375.39..10594.72 rows=1 width=16)
   ->  Incremental Sort  (cost=10375.39..2203675.71 rows=10000 width=16)
         Sort Key: a, b, (sum(c))
         Presorted Key: a, b
         ->  GroupAggregate  (cost=10156.07..2203225.71 rows=10000 width=16)
               Group Key: a, b
               ->  Gather Merge  (cost=10156.07..2128124.39 rows=10000175 width=12)
                     Workers Planned: 2
                     ->  Incremental Sort  (cost=9156.04..972856.05 rows=4166740 width=12)
                           Sort Key: a, b
                           Presorted Key: a
                           ->  Parallel Index Scan using t_a_idx on t  (cost=0.43..417690.30 rows=4166740 width=12)
(12 rows)


and with 0004, I get this:

                                              QUERY PLAN                                              
------------------------------------------------------------------------------------------------------
 Limit  (cost=20443.84..20665.32 rows=1 width=16)
   ->  Incremental Sort  (cost=20443.84..2235250.05 rows=10000 width=16)
         Sort Key: a, b, (sum(c))
         Presorted Key: a, b
         ->  GroupAggregate  (cost=20222.37..2234800.05 rows=10000 width=16)
               Group Key: a, b
               ->  Incremental Sort  (cost=20222.37..2159698.74 rows=10000175 width=12)
                     Sort Key: a, b
                     Presorted Key: a
                     ->  Index Scan using t_a_idx on t  (cost=0.43..476024.65 rows=10000175 width=12)
(10 rows)

Notice that cost of the second plan is almost double the first one. That
means 0004 does not even generate the first plan, i.e. there are cases
where we don't try to add the explicit sort before passing the path to
generate_gather_paths().

And I think I know why is that - while gather_grouping_paths() tries to
add explicit sort below the gather merge, there are other places that
call generate_gather_paths() that don't do that. In this case it's
probably apply_scanjoin_target_to_paths() which simply builds

   parallel (seq|index) scan + gather merge

and that's it. The problem is likely the same - the code does not know
which pathkeys are "interesting" at that point. We probably need to
teach planner to do this.


FWIW tweaking all the create_sort_path() places to also consider adding
incremental sort is a bit tedious and invasive, and it almost doubles
the amount of repetitive code. It's OK for experiment like this, but we
should try handling this in a nicer way (move to a separate function
that does both, or something like that).


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Note: As I was writing this, I saw a new email come in from Tomas with
a new patch series, and some similar observations. I'll look at that
patch series more, but I think it's likely far more complete, so will
end up going with that. I wanted to send this email anyway to at least
capture the debugging process for reference.

On Mon, Jul 8, 2019 at 12:07 PM James Coleman <jtc331@gmail.com> wrote:
>
> On Mon, Jul 8, 2019 at 10:58 AM Tomas Vondra
> <tomas.vondra@2ndquadrant.com> wrote:
> >
> > On Mon, Jul 08, 2019 at 10:32:18AM -0400, James Coleman wrote:
> > >On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondra
> > ><tomas.vondra@2ndquadrant.com> wrote:
> > >>
> > >> On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman wrote:
> > >> >On Sun, Jul 7, 2019 at 5:02 PM Tomas Vondra
> > >> ><tomas.vondra@2ndquadrant.com> wrote:
> > >> >> We're running query like this:
> > >> >>
> > >> >>   SELECT a, sum(b), count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3 ORDER BY 1, 2, 3
> > >> >>
> > >> >> but we're trying to add the incremental sort *before* the aggregation,
> > >> >> because the optimizer also considers group aggregate with a sorted
> > >> >> input. And (a) is a prefix of (a,sum(b),count(b)) so we think we
> > >> >> actually can do this, but clearly that's nonsense, because we can't
> > >> >> possibly know the aggregates yet. Hence the error.
> > >> >>
> > >> >> If this is the actual issue, we need to ensure we actually can evaluate
> > >> >> all the pathkeys. I don't know how to do that yet. I thought that maybe
> > >> >> we should modify pathkeys_common_contained_in() to set presorted_keys to
> > >> >> 0 in this case.
> > >> >>
> > >> >> But then I started wondering why we don't see this issue even for
> > >> >> regular (non-incremental-sort) paths built in create_ordered_paths().
> > >> >> How come we don't see these failures there? I've modified costing to
> > >> >> make all incremental sort paths very cheap, and still nothing.
> > >> >
> > >> >I assume you mean you modified costing to make regular sort paths very cheap?
> > >> >
> > >>
> > >> No, I mean costing of incremental sort paths, so that they end up being
> > >> the cheapest ones. If some other path is cheaper, we won't see the error
> > >> because it only happens when building plan from the cheapest path.
> > >
> > >Ah, I misunderstood as you trying to figure out a way to try to cause
> > >the same problem with a regular sort.
> > >
> > >> >> So presumably there's a check elsewhere (either implicit or explicit),
> > >> >> because create_ordered_paths() uses pathkeys_common_contained_in() and
> > >> >> does not have the same issue.
> > >> >
> > >> >Given this comment in create_ordered_paths():
> > >> >
> > >> >  generate_gather_paths() will have already generated a simple Gather
> > >> >  path for the best parallel path, if any, and the loop above will have
> > >> >  considered sorting it.  Similarly, generate_gather_paths() will also
> > >> >  have generated order-preserving Gather Merge plans which can be used
> > >> >  without sorting if they happen to match the sort_pathkeys, and the loop
> > >> >  above will have handled those as well.  However, there's one more
> > >> >  possibility: it may make sense to sort the cheapest partial path
> > >> >  according to the required output order and then use Gather Merge.
> > >> >
> > >> >my understanding is that generate_gather_paths() only considers paths
> > >> >that already happen to be sorted (not explicit sorts), so I'm
> > >> >wondering if it would make more sense for the incremental sort path
> > >> >creation for this case to live alongside the explicit ordered path
> > >> >creation in create_ordered_paths() rather than in
> > >> >generate_gather_paths().
> > >> >
> > >>
> > >> How would that solve the issue? Also, we're building a gather path, so
> > >> I think generate_gather_paths() is the right place where to do it. And
> > >> we're not changing the semantics of generate_gather_paths() - the result
> > >> path should be sorted "correctly" with respect to sort_pathkeys.
> > >
> > >Does that imply what the explicit sort in create_ordered_paths() is in
> > >the wrong spot?
> > >
> >
> > I think those are essentially the right places where to do this sort of
> > stuff. Maybe there's a better place, but I don't think those places are
> > somehow wrong.
> >
> > >Or, to put it another way, do you think that both kinds of sorts
> > >should be added in the same place? It seems confusing to me that
> > >they'd be split between the two methods (unless I'm completely
> > >misunderstanding how the two work).
> > >
> >
> > The paths built in those two places were very different in one aspect:
> >
> > 1) generate_gather_paths only ever looked at pathkeys for the subpath, it
> > never even looked at ordering expected by paths above it (or the query as
> > a whole). Plain Gather ignores pathkeys entirely, Gather Merge only aims
> > to maintain ordering of the different subpaths.
> >
> > 2) create_oredered_paths is meant to enforce ordering needed by upper
> > parts of the plan - either by using a properly sorted path, or adding an
> > explicit sort.
> >
> >
> > We want to extend (1) to also look at ordering expected by the upper parts
> > of the plan, and consider incremental sort if applicable. (2) already does
> > that, and it already has the correct pathkeys to enforce.
>
> I guess I'm still not following. If (2) is responsible (currently) for
> adding an explicit sort, why wouldn't adding an incremental sort be an
> example of that responsibility? The subpath that either a Sort or
> IncrementalSort is being added on top of (to then feed into the
> GatherMerge) is the same in both cases right?
>
> Unless you're saying that the difference is that since we have a
> partial ordering already for incremental sort then incremental sort
> falls into the category of "maintaining" existing ordering of the
> subpath?

To try to understand this better I looked through all usages of
generate_gather_paths(). The usage in gather_grouping_paths() also
treats explicit sorting as its responsibility rather than the
responsibility of generate_gather_paths(). Since I consider
incremental sort to be just another form of explicit sorting, I think
it's reasonable to make it the responsibility of callers, given that
currently generate_gather_paths() currently seems to be explicitly
only about fully presorted paths.

Of course that might not be ideal the more callers that have to handle
it specially. So maybe it's worth shifting responsibility.

> > But looking at root->sort_pathkeys in (1) seems to be the wrong thing :-(
> >
> > The thing is, we don't have just sort_pathkeys, there's distinct_pathkeys
> > and group_pathkeys too, so maybe we should be looking at those too?
>
> I don't know enough yet to answer, but I'd like to look at (in the
> debugger) the subpaths considered in each function to try to get a
> better understanding of why we don't try to explicitly sort the aggs
> (which we know we can't sort yet) but do for incremental sort. I
> assume that means a subpath has to be present in one but not the other
> since they both use the same pathkey checking function.

I stepped through one of the failing test cases and found that in
create_ordered_paths() the if expression fails because
input_rel->partial_pathlist == NIL. So apparently that is the guard
against incorrectly adding as-yet unsortable paths below the
aggregate. So if I move the create_incremental_sort_path() call inside
this block the error goes away.

Attached is patch revision with that changes that.

James Coleman

On Tue, Jul 09, 2019 at 03:37:03AM +0200, Tomas Vondra wrote:
>
> ...
>
>Notice that cost of the second plan is almost double the first one. That
>means 0004 does not even generate the first plan, i.e. there are cases
>where we don't try to add the explicit sort before passing the path to
>generate_gather_paths().
>
>And I think I know why is that - while gather_grouping_paths() tries to
>add explicit sort below the gather merge, there are other places that
>call generate_gather_paths() that don't do that. In this case it's
>probably apply_scanjoin_target_to_paths() which simply builds
>
>  parallel (seq|index) scan + gather merge
>
>and that's it. The problem is likely the same - the code does not know
>which pathkeys are "interesting" at that point. We probably need to
>teach planner to do this.
>

I've looked into this again, and yes - that's the reason. I've added
generate_useful_gather_paths() which is a wrapper on top of
generate_gather_paths(). It essentially does what 0001 patch did directly
in generate_gather_paths() so it's more like create_grouping_paths().

And that does the trick - we now generate the cheaper paths, and I don't
see any crashes in regression tests etc.

I still suspect we already have code doing similar checks whether pathkeys
might be useful somewhere. I've looked into pathkeys.c and elsewhere but
no luck.

Attached is a slightly modified patch series:

1) 0001 considers incremental sort paths in various places (adds the new
generate_useful_gather_paths and modifies places calling create_sort_path)

2) 0002 and 0003 are fixes I mentioned before

3) 0004 adds a new GUC force_incremental_sort that (when set to 'on')
tries to nudge the optimizer into using incremental sort by essentially
making it free (i.e. using startup/total costs of the subpath). I've found
this useful when trying to force incremental sorts into plans where it may
not be the best strategy.

I won't have time to hack on this over the next ~2 weeks, but I'll try to
respond to questions when possible.

>
>FWIW tweaking all the create_sort_path() places to also consider adding
>incremental sort is a bit tedious and invasive, and it almost doubles
>the amount of repetitive code. It's OK for experiment like this, but we
>should try handling this in a nicer way (move to a separate function
>that does both, or something like that).
>

This definitely needs more work. We need to refactor it in some way, e.g.
have a function that would consider both explicit sort (on the cheapest
path) and incremental sort (on all paths), and call it from all those
places. Haven't tried it, though.

There's also a couple more places where we do create_sort_path() and don't
consider incremental sort yet - window functions, distinct etc.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Mon, Jul 8, 2019 at 9:37 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Mon, Jul 08, 2019 at 12:07:06PM -0400, James Coleman wrote:
> >On Mon, Jul 8, 2019 at 10:58 AM Tomas Vondra
> ><tomas.vondra@2ndquadrant.com> wrote:
> >>
> >> On Mon, Jul 08, 2019 at 10:32:18AM -0400, James Coleman wrote:
> >> >On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondra
> >> ><tomas.vondra@2ndquadrant.com> wrote:
> >> >>
> >> >> On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman wrote:
> >> >> >On Sun, Jul 7, 2019 at 5:02 PM Tomas Vondra
> >> >> ><tomas.vondra@2ndquadrant.com> wrote:
> >> >> >> We're running query like this:
> >> >> >>
> >> >> >>   SELECT a, sum(b), count(*) FROM pagg_tab_ml GROUP BY a HAVING avg(b) < 3 ORDER BY 1, 2, 3
> >> >> >>
> >> >> >> but we're trying to add the incremental sort *before* the aggregation,
> >> >> >> because the optimizer also considers group aggregate with a sorted
> >> >> >> input. And (a) is a prefix of (a,sum(b),count(b)) so we think we
> >> >> >> actually can do this, but clearly that's nonsense, because we can't
> >> >> >> possibly know the aggregates yet. Hence the error.
> >> >> >>
> >> >> >> If this is the actual issue, we need to ensure we actually can evaluate
> >> >> >> all the pathkeys. I don't know how to do that yet. I thought that maybe
> >> >> >> we should modify pathkeys_common_contained_in() to set presorted_keys to
> >> >> >> 0 in this case.
> >> >> >>
> >> >> >> But then I started wondering why we don't see this issue even for
> >> >> >> regular (non-incremental-sort) paths built in create_ordered_paths().
> >> >> >> How come we don't see these failures there? I've modified costing to
> >> >> >> make all incremental sort paths very cheap, and still nothing.
> >> >> >
> >> >> >I assume you mean you modified costing to make regular sort paths very cheap?
> >> >> >
> >> >>
> >> >> No, I mean costing of incremental sort paths, so that they end up being
> >> >> the cheapest ones. If some other path is cheaper, we won't see the error
> >> >> because it only happens when building plan from the cheapest path.
> >> >
> >> >Ah, I misunderstood as you trying to figure out a way to try to cause
> >> >the same problem with a regular sort.
> >> >
> >> >> >> So presumably there's a check elsewhere (either implicit or explicit),
> >> >> >> because create_ordered_paths() uses pathkeys_common_contained_in() and
> >> >> >> does not have the same issue.
> >> >> >
> >> >> >Given this comment in create_ordered_paths():
> >> >> >
> >> >> >  generate_gather_paths() will have already generated a simple Gather
> >> >> >  path for the best parallel path, if any, and the loop above will have
> >> >> >  considered sorting it.  Similarly, generate_gather_paths() will also
> >> >> >  have generated order-preserving Gather Merge plans which can be used
> >> >> >  without sorting if they happen to match the sort_pathkeys, and the loop
> >> >> >  above will have handled those as well.  However, there's one more
> >> >> >  possibility: it may make sense to sort the cheapest partial path
> >> >> >  according to the required output order and then use Gather Merge.
> >> >> >
> >> >> >my understanding is that generate_gather_paths() only considers paths
> >> >> >that already happen to be sorted (not explicit sorts), so I'm
> >> >> >wondering if it would make more sense for the incremental sort path
> >> >> >creation for this case to live alongside the explicit ordered path
> >> >> >creation in create_ordered_paths() rather than in
> >> >> >generate_gather_paths().
> >> >> >
> >> >>
> >> >> How would that solve the issue? Also, we're building a gather path, so
> >> >> I think generate_gather_paths() is the right place where to do it. And
> >> >> we're not changing the semantics of generate_gather_paths() - the result
> >> >> path should be sorted "correctly" with respect to sort_pathkeys.
> >> >
> >> >Does that imply what the explicit sort in create_ordered_paths() is in
> >> >the wrong spot?
> >> >
> >>
> >> I think those are essentially the right places where to do this sort of
> >> stuff. Maybe there's a better place, but I don't think those places are
> >> somehow wrong.
> >>
> >> >Or, to put it another way, do you think that both kinds of sorts
> >> >should be added in the same place? It seems confusing to me that
> >> >they'd be split between the two methods (unless I'm completely
> >> >misunderstanding how the two work).
> >> >
> >>
> >> The paths built in those two places were very different in one aspect:
> >>
> >> 1) generate_gather_paths only ever looked at pathkeys for the subpath, it
> >> never even looked at ordering expected by paths above it (or the query as
> >> a whole). Plain Gather ignores pathkeys entirely, Gather Merge only aims
> >> to maintain ordering of the different subpaths.
> >>
> >> 2) create_oredered_paths is meant to enforce ordering needed by upper
> >> parts of the plan - either by using a properly sorted path, or adding an
> >> explicit sort.
> >>
> >>
> >> We want to extend (1) to also look at ordering expected by the upper parts
> >> of the plan, and consider incremental sort if applicable. (2) already does
> >> that, and it already has the correct pathkeys to enforce.
> >
> >I guess I'm still not following. If (2) is responsible (currently) for
> >adding an explicit sort, why wouldn't adding an incremental sort be an
> >example of that responsibility? The subpath that either a Sort or
> >IncrementalSort is being added on top of (to then feed into the
> >GatherMerge) is the same in both cases right?
> >
> >Unless you're saying that the difference is that since we have a
> >partial ordering already for incremental sort then incremental sort
> >falls into the category of "maintaining" existing ordering of the
> >subpath?
> >
>
> Oh, I think I understand what you're saying. Essentially, we should not
> be making generate_gather_paths responsible for adding the incremental
> sort. Instead, we should be looking at places than are adding explicit
> sort (using create_sort_path) and also consider adding incremental sort.
>
> I definitely agree with the second half - we should look at all places
> that create explicit sorts and make them also consider incremental
> sorts. That makes sense.

Yep, exactly.

> But I'm not sure it'll address all cases - the problem is that those
> places add the explicit sort because they need sorted input. Gather
> Merge does not do that, it only preserves existing ordering of paths.
>
> So it's possible the path does not have an explicit sort on to, and
> gather merge will not know to add it. And once we have the gather merge
> in place, we can't push place "under" it.

That's the explanation I was missing; and it makes sense (to restate:
sometimes sorting is useful even when not required for correctness of
the user returned data).

> In fact, we already have code dealing with this "issue" for a special
> case - see gather_grouping_paths(). It generates plain gather merge
> paths, but then also considers building one with explicit sort. But it
> only does that for grouping paths (when it's clear we need to be looking
> at grouping_pathkeys), and there are generate_gather_paths() that don't
> have similar treatment.

I just find it humorous both of us were writing separate emails
mentioning that function at the same time.

> >> But looking at root->sort_pathkeys in (1) seems to be the wrong thing :-(
> >>
> >> The thing is, we don't have just sort_pathkeys, there's distinct_pathkeys
> >> and group_pathkeys too, so maybe we should be looking at those too?
> >
> >I don't know enough yet to answer, but I'd like to look at (in the
> >debugger) the subpaths considered in each function to try to get a
> >better understanding of why we don't try to explicitly sort the aggs
> >(which we know we can't sort yet) but do for incremental sort. I
> >assume that means a subpath has to be present in one but not the other
> >since they both use the same pathkey checking function.
> >
>
> I've been wondering if we have some other code that needs to consider
> interesting pathkeys "candidates" (instead of just getting the list
> interesting in that place). Because then we could look at that code and
> use it here ...
>
> And guess what - postgres_fdw needs to do pretty much exactly that, when
> building paths for remote relations. AFAIK we can't easily request all
> plans from the remote node and then look at their pathkeys (like we'd do
> with local node), so instead we deduce "interesting pathkeys" and then
> request best plans for those. And deducing "interesing" pathkeys is
> pretty much what get_useful_pathkeys_for_relation() is about.
>
> So I've copied this function (and two more, called from it), whacked it
> a bit until it removed (shakespeare-writing chimp comes to mind) and
> voila, it seems to be working. The errors you reported are gone, and the
> plans seems to be reasonable.
>
> Attached is a sequence of 4 patches:
>
>
> 0001-fix-pathkey-processing-in-generate_gather_paths.patch
> ----------------------------------------------------------
> This is the fixed version of my previous patch, with the stuff stolen
> from postgres_fdw.
>
>
> 0002-fix-costing-in-cost_incremental_sort.patch
> -----------------------------------------------
> This is the costing fix, I mentioned before.
>
>
> 0003-fix-explain-in-parallel-mode.patch
> ---------------------------------------
> Minor bug in explain, when incremental sort ends up being in the
> parallel part of the plan (missing newline on per-worker line)
>
>
> 0004-rework-where-incremental-sort-paths-are-created.patch
> ----------------------------------------------------------
> This undoes the generate_gather_paths() changes from 0001, and instead
> modifies a bunch of places that call create_sort_path() to also consider
> incremental sorts. There are a couple remaining, but those should not be
> relevant to the queries I've been looking at.
>
>
> Essentially, 0002 and 0003 are bugfixes. 0001 and 0004 are the two
> different aproaches to building incremental sort + gather merge.
>
> Now, consider this example:
>
>   create table t (a int, b int, c int);
>   insert into t select mod(i,100),mod(i,100),i from generate_series(1,10000000) s(i);
>   create index on t (a);
>   analyze t;
>   explain select a,b,sum(c) from t group by 1,2 order by 1,2,3 limit 1;
>
> With 0001+0002+0003 pathes, I get a plan like this:
>
>                                                      QUERY PLAN
> --------------------------------------------------------------------------------------------------------------------
>  Limit  (cost=10375.39..10594.72 rows=1 width=16)
>    ->  Incremental Sort  (cost=10375.39..2203675.71 rows=10000 width=16)
>          Sort Key: a, b, (sum(c))
>          Presorted Key: a, b
>          ->  GroupAggregate  (cost=10156.07..2203225.71 rows=10000 width=16)
>                Group Key: a, b
>                ->  Gather Merge  (cost=10156.07..2128124.39 rows=10000175 width=12)
>                      Workers Planned: 2
>                      ->  Incremental Sort  (cost=9156.04..972856.05 rows=4166740 width=12)
>                            Sort Key: a, b
>                            Presorted Key: a
>                            ->  Parallel Index Scan using t_a_idx on t  (cost=0.43..417690.30 rows=4166740 width=12)
> (12 rows)
>
>
> and with 0004, I get this:
>
>                                               QUERY PLAN
> ------------------------------------------------------------------------------------------------------
>  Limit  (cost=20443.84..20665.32 rows=1 width=16)
>    ->  Incremental Sort  (cost=20443.84..2235250.05 rows=10000 width=16)
>          Sort Key: a, b, (sum(c))
>          Presorted Key: a, b
>          ->  GroupAggregate  (cost=20222.37..2234800.05 rows=10000 width=16)
>                Group Key: a, b
>                ->  Incremental Sort  (cost=20222.37..2159698.74 rows=10000175 width=12)
>                      Sort Key: a, b
>                      Presorted Key: a
>                      ->  Index Scan using t_a_idx on t  (cost=0.43..476024.65 rows=10000175 width=12)
> (10 rows)
>
> Notice that cost of the second plan is almost double the first one. That
> means 0004 does not even generate the first plan, i.e. there are cases
> where we don't try to add the explicit sort before passing the path to
> generate_gather_paths().
>
> And I think I know why is that - while gather_grouping_paths() tries to
> add explicit sort below the gather merge, there are other places that
> call generate_gather_paths() that don't do that. In this case it's
> probably apply_scanjoin_target_to_paths() which simply builds
>
>    parallel (seq|index) scan + gather merge
>
> and that's it. The problem is likely the same - the code does not know
> which pathkeys are "interesting" at that point. We probably need to
> teach planner to do this.

I had also noticed that that was an obvious place where
generate_gather_paths() was used but an explicit sort wasn't also
added separately, which makes me think the division of labor is
probably currently wrong regardless of the incremental sort patch.

Do you agree? Should we try to fix that (likely with your new
"interesting paths" version of generate_gather_paths()) first as a
prefix patch to adding incremental sort?

> FWIW tweaking all the create_sort_path() places to also consider adding
> incremental sort is a bit tedious and invasive, and it almost doubles
> the amount of repetitive code. It's OK for experiment like this, but we
> should try handling this in a nicer way (move to a separate function
> that does both, or something like that).

Agreed.

On Tue, Jul 9, 2019 at 8:11 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Tue, Jul 09, 2019 at 03:37:03AM +0200, Tomas Vondra wrote:
> >
> > ...
> >
> >Notice that cost of the second plan is almost double the first one. That
> >means 0004 does not even generate the first plan, i.e. there are cases
> >where we don't try to add the explicit sort before passing the path to
> >generate_gather_paths().
> >
> >And I think I know why is that - while gather_grouping_paths() tries to
> >add explicit sort below the gather merge, there are other places that
> >call generate_gather_paths() that don't do that. In this case it's
> >probably apply_scanjoin_target_to_paths() which simply builds
> >
> >  parallel (seq|index) scan + gather merge
> >
> >and that's it. The problem is likely the same - the code does not know
> >which pathkeys are "interesting" at that point. We probably need to
> >teach planner to do this.
> >
>
> I've looked into this again, and yes - that's the reason. I've added
> generate_useful_gather_paths() which is a wrapper on top of
> generate_gather_paths(). It essentially does what 0001 patch did directly
> in generate_gather_paths() so it's more like create_grouping_paths().
>
> And that does the trick - we now generate the cheaper paths, and I don't
> see any crashes in regression tests etc.
>
> I still suspect we already have code doing similar checks whether pathkeys
> might be useful somewhere. I've looked into pathkeys.c and elsewhere but
> no luck.
>
> Attached is a slightly modified patch series:
>
> 1) 0001 considers incremental sort paths in various places (adds the new
> generate_useful_gather_paths and modifies places calling create_sort_path)

I need to spend some decent time digesting this patch, but the concept
sounds very useful.

> 2) 0002 and 0003 are fixes I mentioned before
>
> 3) 0004 adds a new GUC force_incremental_sort that (when set to 'on')
> tries to nudge the optimizer into using incremental sort by essentially
> making it free (i.e. using startup/total costs of the subpath). I've found
> this useful when trying to force incremental sorts into plans where it may
> not be the best strategy.

That will be super helpful. I do wonder if we need to expose (in the
production patch) a GUC of some kind to adjust incremental sort
costing so that users can try to tweak when it is preferred over
regular sort.

> I won't have time to hack on this over the next ~2 weeks, but I'll try to
> respond to questions when possible.

Understood; thanks so much for your help on this.

> >
> >FWIW tweaking all the create_sort_path() places to also consider adding
> >incremental sort is a bit tedious and invasive, and it almost doubles
> >the amount of repetitive code. It's OK for experiment like this, but we
> >should try handling this in a nicer way (move to a separate function
> >that does both, or something like that).
> >
>
> This definitely needs more work. We need to refactor it in some way, e.g.
> have a function that would consider both explicit sort (on the cheapest
> path) and incremental sort (on all paths), and call it from all those
> places. Haven't tried it, though.
>
> There's also a couple more places where we do create_sort_path() and don't
> consider incremental sort yet - window functions, distinct etc.

Yep, and likely want regression tests for all of these cases also.

James Coleman

On Mon, Jul 8, 2019 at 6:37 PM Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:
>
> On Thu, Jul 4, 2019 at 4:25 PM James Coleman <jtc331@gmail.com> wrote:
> > Process questions:
> > - Do I need to explicitly move the patch somehow to the next CF?
>
> We didn't manage to register it on current (July) commitfest.  So,
> please, register it on next (September) commitfest.

I've moved it to the September cf.

> > - Since I've basically taken over patch ownership, should I move my
> > name from reviewer to author in the CF app? And can there be two
> > authors listed there?
>
> Sure, you're co-author of this patch.  Two or more authors could be
> listed at CF app, you can find a lot of examples on the list.

Done.

James Coleman

On Tue, Jul 09, 2019 at 09:28:42AM -0400, James Coleman wrote:
>On Mon, Jul 8, 2019 at 9:37 PM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>>
>> On Mon, Jul 08, 2019 at 12:07:06PM -0400, James Coleman wrote:
>> > ...
>> >
>> >I guess I'm still not following. If (2) is responsible (currently) for
>> >adding an explicit sort, why wouldn't adding an incremental sort be an
>> >example of that responsibility? The subpath that either a Sort or
>> >IncrementalSort is being added on top of (to then feed into the
>> >GatherMerge) is the same in both cases right?
>> >
>> >Unless you're saying that the difference is that since we have a
>> >partial ordering already for incremental sort then incremental sort
>> >falls into the category of "maintaining" existing ordering of the
>> >subpath?
>> >
>>
>> Oh, I think I understand what you're saying. Essentially, we should not
>> be making generate_gather_paths responsible for adding the incremental
>> sort. Instead, we should be looking at places than are adding explicit
>> sort (using create_sort_path) and also consider adding incremental sort.
>>
>> I definitely agree with the second half - we should look at all places
>> that create explicit sorts and make them also consider incremental
>> sorts. That makes sense.
>
>Yep, exactly.
>

If I remember correctly, one of the previous patch versions (in the early
2018 commitfests) actually modified many of those places, but it did that
in a somewhat "naive" way. It simply used incremental sort whenever the
path was partially sorted, or something like that. So it did not use
costing properly. There was an attempt to fix that in the last commitfest
but the costing model was deemed to be a bit too rough and unreliable
(especially the ndistinct estimates can be quite weak), so the agreement 
was to try salvaging the patch for PG11 by only considering incremental
sort in "safest" places with greatest gains.

We've significantly improved the costing model since then, and the
implementation likely handles the corner cases much better. But that does
not mean we have to introduce the incremental sort to all those places at
once - it might be wise to split that into separate patches.

It's not just about picking the right plan - we've kinda what impact these
extra paths might have on planner performance, so maybe we should look at
that too. And the impact might be different for each of those places.

I'll leave that up to you, but I certainly won't insist on doing it all in
one huge patch.

>> But I'm not sure it'll address all cases - the problem is that those
>> places add the explicit sort because they need sorted input. Gather
>> Merge does not do that, it only preserves existing ordering of paths.
>>
>> So it's possible the path does not have an explicit sort on to, and
>> gather merge will not know to add it. And once we have the gather merge
>> in place, we can't push place "under" it.
>
>That's the explanation I was missing; and it makes sense (to restate:
>sometimes sorting is useful even when not required for correctness of
>the user returned data).
>

Yes, although even when the sorting is required for correctness (because
the user specified ORDER BY) you can do it at different points in the
plan. We'd still produce correct results, but the sort might be done at
the very end without these changes.

For example we might end up with plans

   Incremental Sort (presorted: a, path keys: a,b)
    -> Gather Merge (path keys: a)
        -> Index Scan (path keys: a)

but with those changes we might push the incremental sort down into the
parallel part:

   Gather Merge (path keys: a,b)
    -> Incremental Sort (presorted: a, path keys: a,b)
        -> Index Scan (path keys: a)

which is likely better. Perhaps that's what you meant, though.


>> In fact, we already have code dealing with this "issue" for a special
>> case - see gather_grouping_paths(). It generates plain gather merge
>> paths, but then also considers building one with explicit sort. But it
>> only does that for grouping paths (when it's clear we need to be looking
>> at grouping_pathkeys), and there are generate_gather_paths() that don't
>> have similar treatment.
>
>I just find it humorous both of us were writing separate emails
>mentioning that function at the same time.
>

;-)

>> ...
>>
>> And I think I know why is that - while gather_grouping_paths() tries to
>> add explicit sort below the gather merge, there are other places that
>> call generate_gather_paths() that don't do that. In this case it's
>> probably apply_scanjoin_target_to_paths() which simply builds
>>
>>    parallel (seq|index) scan + gather merge
>>
>> and that's it. The problem is likely the same - the code does not know
>> which pathkeys are "interesting" at that point. We probably need to
>> teach planner to do this.
>
>I had also noticed that that was an obvious place where
>generate_gather_paths() was used but an explicit sort wasn't also
>added separately, which makes me think the division of labor is
>probably currently wrong regardless of the incremental sort patch.
>
>Do you agree? Should we try to fix that (likely with your new
>"interesting paths" version of generate_gather_paths()) first as a
>prefix patch to adding incremental sort?
>

I'm not sure what the generate_useful_gather_paths() should do but does
not? Or is it just the division of labor that you think is wrong? In any
case, feel free to whack it until you're happy with it.

>> ...
>>
>> Attached is a slightly modified patch series:
>>
>> 1) 0001 considers incremental sort paths in various places (adds the new
>> generate_useful_gather_paths and modifies places calling create_sort_path)
>
>I need to spend some decent time digesting this patch, but the concept
>sounds very useful.
>

OK.

>> 2) 0002 and 0003 are fixes I mentioned before
>>
>> 3) 0004 adds a new GUC force_incremental_sort that (when set to 'on')
>> tries to nudge the optimizer into using incremental sort by essentially
>> making it free (i.e. using startup/total costs of the subpath). I've found
>> this useful when trying to force incremental sorts into plans where it may
>> not be the best strategy.
>
>That will be super helpful. I do wonder if we need to expose (in the
>production patch) a GUC of some kind to adjust incremental sort
>costing so that users can try to tweak when it is preferred over
>regular sort.
>

This GUC is really meant primarily for development, to force choice of
incremental sort during regression tests (so as to use incremental sort in
as many plans as possible). I'd remove it from the final patch. I think
the general consensus on pgsql-hackers is that we should not introduce
GUCs unless absolutely necessary. But for development GUCs - sure.

FWIW I'm not sure it's a good idea to look at both enable_incremental_sort
and enable_sort in cost_incremental_sort(). Not only end up with
disable_cost twice when both GUCs are set to 'off' at the moment, but it
might be useful to be able to disable those two sort types independently.
For example you might set just enable_sort=off and we'd still generate
incremental sort paths.

>> ...
>>
>> This definitely needs more work. We need to refactor it in some way, e.g.
>> have a function that would consider both explicit sort (on the cheapest
>> path) and incremental sort (on all paths), and call it from all those
>> places. Haven't tried it, though.
>>
>> There's also a couple more places where we do create_sort_path() and don't
>> consider incremental sort yet - window functions, distinct etc.
>
>Yep, and likely want regression tests for all of these cases also.
>

Yes, that's definitely something a committable patch needs to include.

Another thing we should have is a collection of tests with data sets that
"break" the costing model in some way (skew, correlated columns,
non-uniform group sizes, ...). That's something not meant for commit,
because it'll probably require significant amounts of data, but we need it
to asses the quality of the planner/costing part. I know there are various
ad-hoc test cases in the thread history, it'd be good to consolidate that
into once place.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Tue, Jul 9, 2019 at 10:54 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Tue, Jul 09, 2019 at 09:28:42AM -0400, James Coleman wrote:
> >On Mon, Jul 8, 2019 at 9:37 PM Tomas Vondra
> ><tomas.vondra@2ndquadrant.com> wrote:
> >>
> >> On Mon, Jul 08, 2019 at 12:07:06PM -0400, James Coleman wrote:
> >> > ...
> >> >
> >> >I guess I'm still not following. If (2) is responsible (currently) for
> >> >adding an explicit sort, why wouldn't adding an incremental sort be an
> >> >example of that responsibility? The subpath that either a Sort or
> >> >IncrementalSort is being added on top of (to then feed into the
> >> >GatherMerge) is the same in both cases right?
> >> >
> >> >Unless you're saying that the difference is that since we have a
> >> >partial ordering already for incremental sort then incremental sort
> >> >falls into the category of "maintaining" existing ordering of the
> >> >subpath?
> >> >
> >>
> >> Oh, I think I understand what you're saying. Essentially, we should not
> >> be making generate_gather_paths responsible for adding the incremental
> >> sort. Instead, we should be looking at places than are adding explicit
> >> sort (using create_sort_path) and also consider adding incremental sort.
> >>
> >> I definitely agree with the second half - we should look at all places
> >> that create explicit sorts and make them also consider incremental
> >> sorts. That makes sense.
> >
> >Yep, exactly.
> >
>
> If I remember correctly, one of the previous patch versions (in the early
> 2018 commitfests) actually modified many of those places, but it did that
> in a somewhat "naive" way. It simply used incremental sort whenever the
> path was partially sorted, or something like that. So it did not use
> costing properly. There was an attempt to fix that in the last commitfest
> but the costing model was deemed to be a bit too rough and unreliable
> (especially the ndistinct estimates can be quite weak), so the agreement
> was to try salvaging the patch for PG11 by only considering incremental
> sort in "safest" places with greatest gains.
>
> We've significantly improved the costing model since then, and the
> implementation likely handles the corner cases much better. But that does
> not mean we have to introduce the incremental sort to all those places at
> once - it might be wise to split that into separate patches.

Yes, although we haven't added the MCV checking yet; that's on my
mental checklist, but another new area of the codebase for me to
understand, so I've been prioritizing other parts of the patch.

> It's not just about picking the right plan - we've kinda what impact these
> extra paths might have on planner performance, so maybe we should look at
> that too. And the impact might be different for each of those places.
>
> I'll leave that up to you, but I certainly won't insist on doing it all in
> one huge patch.

I'm not opposed to handling some of them separately, but I would like
to at least hit the places where it's most likely (for example, with
LIMIT) to improve things. I supposed I'll have to look at all of the
usages of create_sort_path() and try to rank them in terms of
perceived likely value.

> >> But I'm not sure it'll address all cases - the problem is that those
> >> places add the explicit sort because they need sorted input. Gather
> >> Merge does not do that, it only preserves existing ordering of paths.
> >>
> >> So it's possible the path does not have an explicit sort on to, and
> >> gather merge will not know to add it. And once we have the gather merge
> >> in place, we can't push place "under" it.
> >
> >That's the explanation I was missing; and it makes sense (to restate:
> >sometimes sorting is useful even when not required for correctness of
> >the user returned data).
> >
>
> Yes, although even when the sorting is required for correctness (because
> the user specified ORDER BY) you can do it at different points in the
> plan. We'd still produce correct results, but the sort might be done at
> the very end without these changes.
>
> For example we might end up with plans
>
>    Incremental Sort (presorted: a, path keys: a,b)
>     -> Gather Merge (path keys: a)
>         -> Index Scan (path keys: a)
>
> but with those changes we might push the incremental sort down into the
> parallel part:
>
>    Gather Merge (path keys: a,b)
>     -> Incremental Sort (presorted: a, path keys: a,b)
>         -> Index Scan (path keys: a)
>
> which is likely better. Perhaps that's what you meant, though.

I was thinking of ordering being useful for grouping/aggregation or
merge joins; I didn't realize the above plan wasn't possible yet, so
that explanation is helpful.

> >> And I think I know why is that - while gather_grouping_paths() tries to
> >> add explicit sort below the gather merge, there are other places that
> >> call generate_gather_paths() that don't do that. In this case it's
> >> probably apply_scanjoin_target_to_paths() which simply builds
> >>
> >>    parallel (seq|index) scan + gather merge
> >>
> >> and that's it. The problem is likely the same - the code does not know
> >> which pathkeys are "interesting" at that point. We probably need to
> >> teach planner to do this.
> >
> >I had also noticed that that was an obvious place where
> >generate_gather_paths() was used but an explicit sort wasn't also
> >added separately, which makes me think the division of labor is
> >probably currently wrong regardless of the incremental sort patch.
> >
> >Do you agree? Should we try to fix that (likely with your new
> >"interesting paths" version of generate_gather_paths()) first as a
> >prefix patch to adding incremental sort?
> >
>
> I'm not sure what the generate_useful_gather_paths() should do but does
> not? Or is it just the division of labor that you think is wrong? In any
> case, feel free to whack it until you're happy with it.

Oh, I didn't mean to imply generate_useful_gather_paths() was
deficient in some way; I was noting that I'd noticed that
apply_scanjoin_target_to_paths() didn't consider explicit sort +
gather merge on master, and that maybe that was an unintentional miss
and something that should be remedied.

And if it is a miss, then that would demonstrate in my mind that the
addition of generate_useful_gather_paths() would be a helpful refactor
as a standalone patch even without incremental sort. So "currently
wrong" above meant "on master" not "in the current version of the
patch."

> >> Attached is a slightly modified patch series:
> >>
> >> 1) 0001 considers incremental sort paths in various places (adds the new
> >> generate_useful_gather_paths and modifies places calling create_sort_path)
> >
> >I need to spend some decent time digesting this patch, but the concept
> >sounds very useful.
> >
>
> OK.

I've been reading this several times + stepping through with the
debugger to understand when this is useful, but I have a few
questions.

The first case considered in get_useful_pathkeys_for_relation (which
considers root->query_pathkeys) makes a lot of sense -- obviously if
we want sorted results then it's useful to consider both full and
incremental sort.

But I'm not sure I see yet a way to trigger the second case (which
uses get_useful_ecs_for_relation to build pathkeys potentially useful
for merge joins). In the FDW case we need to consider it since we want
to avoid local sort and so want to see if the foreign server might be
able to provide us useful presorted data, but in the local case I
don't think that's useful. From what I can tell merge join path
costing internally considers possible sorts of both the inner and
outer input paths, and merge join plan node creation is responsible
for building explicit sort nodes as necessary (i.e., there are no
explicit sort paths created for merge join paths.) That means that,
for example, a query like:

select * from
(select * from j2 order by j2.t, j2.a) j2
join (select * from j1 order by j1.t) j1
    on j1.t = j2.t and j1.a = j2.a;

don't consider incremental sort for the merge join path (I disabled
hash joins, nested loops, and full sorts testing that on an empty
table just to easily force a merge join plan). And unfortunately I
don't think there's an easy way to remedy that: from what I can tell
it'd be a pretty invasive patch requiring refactoring merge join
costing to consider both kinds of sorting (in the most simple
implementation that would mean considering up to 4x as many merge join
paths -- inner/outer sorted by: full/full, full/incremental,
incremental/full, and incremental/incremental). Given that's a
significant undertaking on its own, I think I'm going to avoid
addressing it as part of this patch.

If it's true that the get_useful_ecs_for_relation part of that logic
isn't actually exercisable currently, that that would cut down
significantly on the amount of code that needs to be added for
consideration of valid gather merge paths. But if you can think of a
counterexample, please let me know.

> >> 2) 0002 and 0003 are fixes I mentioned before

I'm incorporating those with a bit of additional cleanup.

> >> 3) 0004 adds a new GUC force_incremental_sort that (when set to 'on')
> >> tries to nudge the optimizer into using incremental sort by essentially
> >> making it free (i.e. using startup/total costs of the subpath). I've found
> >> this useful when trying to force incremental sorts into plans where it may
> >> not be the best strategy.
> >
> >That will be super helpful. I do wonder if we need to expose (in the
> >production patch) a GUC of some kind to adjust incremental sort
> >costing so that users can try to tweak when it is preferred over
> >regular sort.
> >
>
> This GUC is really meant primarily for development, to force choice of
> incremental sort during regression tests (so as to use incremental sort in
> as many plans as possible). I'd remove it from the final patch. I think
> the general consensus on pgsql-hackers is that we should not introduce
> GUCs unless absolutely necessary. But for development GUCs - sure.
>
> FWIW I'm not sure it's a good idea to look at both enable_incremental_sort
> and enable_sort in cost_incremental_sort(). Not only end up with
> disable_cost twice when both GUCs are set to 'off' at the moment, but it
> might be useful to be able to disable those two sort types independently.
> For example you might set just enable_sort=off and we'd still generate
> incremental sort paths.

That would cover the usage case I was getting at. Having enable_sort
disable incremental sort also came in without much explanation [1]:

On Fri, Apr 6, 2018 at 11:40 PM, Alexander Kuzmenkov <
a(dot)kuzmenkov(at)postgrespro(dot)ru> wrote:
> Also some other changes from me:
> ...
> enable_sort should act as a cost-based soft disable for both incremental
> and normal sort.

I wasn't sure that fully made sense to me, but was assuming the idea
was to effectively not introduce a regression for anyone already
disabling sort to force a specific plan shape. That being said, any
new execution node/planning feature can cause a regression in such
"hinted" queries, so I'm not sure that's a good reason on its own. In
any case, incremental sort is different enough in performance
qualities that I think you'd want to re-evaluate possible plans in
queries where enable_sort=off is useful, so I'm going make incremental
sort independent of enable_sort unless there's a strong objection
here.

Tangentially: I'd almost expect enable_incremental_sort to act as a
hard disable (and not even generate the paths) rather than a soft
cost-based disable, since while standard sort is the most basic
operation that needs to always be available as a last resort the same
is not true for incremental sort...

> Another thing we should have is a collection of tests with data sets that
> "break" the costing model in some way (skew, correlated columns,
> non-uniform group sizes, ...). That's something not meant for commit,
> because it'll probably require significant amounts of data, but we need it
> to asses the quality of the planner/costing part. I know there are various
> ad-hoc test cases in the thread history, it'd be good to consolidate that
> into once place.

Agreed.

I'm continuing to work on the planning side of this with the goal of
not needing to modify too many places to consider an incremental sort
path + considering which ones are most likely to be useful, but I
wanted to get my questions about get_useful_ecs_for_relation sent out
while I work on that.

If we end up wanting to limit the number of places we consider
incremental sort (whether for planning time or merely for size of the
initial patch, do you have any thoughts on what general areas we
should consider most valuable? Besides the obvious LIMIT case aother
area that might benefit was min/max, though I'm not sure yet at the
moment if that would really end up meaning considering it all over the
place.

James Coleman

[1]: https://www.postgresql.org/message-id/CAPpHfdtKHETXhf062CPvkjpG1wnjQ7rv4uLhZgYQ6VZjwqDYpg%40mail.gmail.com

On Sun, Jul 14, 2019 at 02:38:40PM -0400, James Coleman wrote:
>On Tue, Jul 9, 2019 at 10:54 AM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>>
>> On Tue, Jul 09, 2019 at 09:28:42AM -0400, James Coleman wrote:
>> >On Mon, Jul 8, 2019 at 9:37 PM Tomas Vondra
>> ><tomas.vondra@2ndquadrant.com> wrote:
>> >>
>> >> On Mon, Jul 08, 2019 at 12:07:06PM -0400, James Coleman wrote:
>> >> > ...
>> >> >
>> >> >I guess I'm still not following. If (2) is responsible (currently) for
>> >> >adding an explicit sort, why wouldn't adding an incremental sort be an
>> >> >example of that responsibility? The subpath that either a Sort or
>> >> >IncrementalSort is being added on top of (to then feed into the
>> >> >GatherMerge) is the same in both cases right?
>> >> >
>> >> >Unless you're saying that the difference is that since we have a
>> >> >partial ordering already for incremental sort then incremental sort
>> >> >falls into the category of "maintaining" existing ordering of the
>> >> >subpath?
>> >> >
>> >>
>> >> Oh, I think I understand what you're saying. Essentially, we should not
>> >> be making generate_gather_paths responsible for adding the incremental
>> >> sort. Instead, we should be looking at places than are adding explicit
>> >> sort (using create_sort_path) and also consider adding incremental sort.
>> >>
>> >> I definitely agree with the second half - we should look at all places
>> >> that create explicit sorts and make them also consider incremental
>> >> sorts. That makes sense.
>> >
>> >Yep, exactly.
>> >
>>
>> If I remember correctly, one of the previous patch versions (in the early
>> 2018 commitfests) actually modified many of those places, but it did that
>> in a somewhat "naive" way. It simply used incremental sort whenever the
>> path was partially sorted, or something like that. So it did not use
>> costing properly. There was an attempt to fix that in the last commitfest
>> but the costing model was deemed to be a bit too rough and unreliable
>> (especially the ndistinct estimates can be quite weak), so the agreement
>> was to try salvaging the patch for PG11 by only considering incremental
>> sort in "safest" places with greatest gains.
>>
>> We've significantly improved the costing model since then, and the
>> implementation likely handles the corner cases much better. But that does
>> not mean we have to introduce the incremental sort to all those places at
>> once - it might be wise to split that into separate patches.
>
>Yes, although we haven't added the MCV checking yet; that's on my
>mental checklist, but another new area of the codebase for me to
>understand, so I've been prioritizing other parts of the patch.
>

Sure, no problem.

>> It's not just about picking the right plan - we've kinda what impact these
>> extra paths might have on planner performance, so maybe we should look at
>> that too. And the impact might be different for each of those places.
>>
>> I'll leave that up to you, but I certainly won't insist on doing it all in
>> one huge patch.
>
>I'm not opposed to handling some of them separately, but I would like
>to at least hit the places where it's most likely (for example, with
>LIMIT) to improve things. I supposed I'll have to look at all of the
>usages of create_sort_path() and try to rank them in terms of
>perceived likely value.
>

Yep, makes sense.

>> >> But I'm not sure it'll address all cases - the problem is that those
>> >> places add the explicit sort because they need sorted input. Gather
>> >> Merge does not do that, it only preserves existing ordering of paths.
>> >>
>> >> So it's possible the path does not have an explicit sort on to, and
>> >> gather merge will not know to add it. And once we have the gather merge
>> >> in place, we can't push place "under" it.
>> >
>> >That's the explanation I was missing; and it makes sense (to restate:
>> >sometimes sorting is useful even when not required for correctness of
>> >the user returned data).
>> >
>>
>> Yes, although even when the sorting is required for correctness (because
>> the user specified ORDER BY) you can do it at different points in the
>> plan. We'd still produce correct results, but the sort might be done at
>> the very end without these changes.
>>
>> For example we might end up with plans
>>
>>    Incremental Sort (presorted: a, path keys: a,b)
>>     -> Gather Merge (path keys: a)
>>         -> Index Scan (path keys: a)
>>
>> but with those changes we might push the incremental sort down into the
>> parallel part:
>>
>>    Gather Merge (path keys: a,b)
>>     -> Incremental Sort (presorted: a, path keys: a,b)
>>         -> Index Scan (path keys: a)
>>
>> which is likely better. Perhaps that's what you meant, though.
>
>I was thinking of ordering being useful for grouping/aggregation or
>merge joins; I didn't realize the above plan wasn't possible yet, so
>that explanation is helpful.
>
>> >> And I think I know why is that - while gather_grouping_paths() tries to
>> >> add explicit sort below the gather merge, there are other places that
>> >> call generate_gather_paths() that don't do that. In this case it's
>> >> probably apply_scanjoin_target_to_paths() which simply builds
>> >>
>> >>    parallel (seq|index) scan + gather merge
>> >>
>> >> and that's it. The problem is likely the same - the code does not know
>> >> which pathkeys are "interesting" at that point. We probably need to
>> >> teach planner to do this.
>> >
>> >I had also noticed that that was an obvious place where
>> >generate_gather_paths() was used but an explicit sort wasn't also
>> >added separately, which makes me think the division of labor is
>> >probably currently wrong regardless of the incremental sort patch.
>> >
>> >Do you agree? Should we try to fix that (likely with your new
>> >"interesting paths" version of generate_gather_paths()) first as a
>> >prefix patch to adding incremental sort?
>> >
>>
>> I'm not sure what the generate_useful_gather_paths() should do but does
>> not? Or is it just the division of labor that you think is wrong? In any
>> case, feel free to whack it until you're happy with it.
>
>Oh, I didn't mean to imply generate_useful_gather_paths() was
>deficient in some way; I was noting that I'd noticed that
>apply_scanjoin_target_to_paths() didn't consider explicit sort +
>gather merge on master, and that maybe that was an unintentional miss
>and something that should be remedied.
>
>And if it is a miss, then that would demonstrate in my mind that the
>addition of generate_useful_gather_paths() would be a helpful refactor
>as a standalone patch even without incremental sort. So "currently
>wrong" above meant "on master" not "in the current version of the
>patch."
>

Ah, OK. I don't know if it's a miss or intentional omission - it might
be a mix of both, actually. I wouldn't be surprised if doing that was
considered and considered not worth it, but if we can come up with a
plan where adding an explicit sort here helps ...

>> >> Attached is a slightly modified patch series:
>> >>
>> >> 1) 0001 considers incremental sort paths in various places (adds the new
>> >> generate_useful_gather_paths and modifies places calling create_sort_path)
>> >
>> >I need to spend some decent time digesting this patch, but the concept
>> >sounds very useful.
>> >
>>
>> OK.
>
>I've been reading this several times + stepping through with the
>debugger to understand when this is useful, but I have a few
>questions.
>
>The first case considered in get_useful_pathkeys_for_relation (which
>considers root->query_pathkeys) makes a lot of sense -- obviously if
>we want sorted results then it's useful to consider both full and
>incremental sort.
>
>But I'm not sure I see yet a way to trigger the second case (which
>uses get_useful_ecs_for_relation to build pathkeys potentially useful
>for merge joins). In the FDW case we need to consider it since we want
>to avoid local sort and so want to see if the foreign server might be
>able to provide us useful presorted data, but in the local case I
>don't think that's useful. From what I can tell merge join path
>costing internally considers possible sorts of both the inner and
>outer input paths, and merge join plan node creation is responsible
>for building explicit sort nodes as necessary (i.e., there are no
>explicit sort paths created for merge join paths.) That means that,
>for example, a query like:
>
>select * from
>(select * from j2 order by j2.t, j2.a) j2
>join (select * from j1 order by j1.t) j1
>    on j1.t = j2.t and j1.a = j2.a;
>
>don't consider incremental sort for the merge join path (I disabled
>hash joins, nested loops, and full sorts testing that on an empty
>table just to easily force a merge join plan). And unfortunately I
>don't think there's an easy way to remedy that: from what I can tell
>it'd be a pretty invasive patch requiring refactoring merge join
>costing to consider both kinds of sorting (in the most simple
>implementation that would mean considering up to 4x as many merge join
>paths -- inner/outer sorted by: full/full, full/incremental,
>incremental/full, and incremental/incremental). Given that's a
>significant undertaking on its own, I think I'm going to avoid
>addressing it as part of this patch.
>
>If it's true that the get_useful_ecs_for_relation part of that logic
>isn't actually exercisable currently, that that would cut down
>significantly on the amount of code that needs to be added for
>consideration of valid gather merge paths. But if you can think of a
>counterexample, please let me know.
>

It's quite possible parts of the code are not needed - I've pretty much
just copied the code and did minimal changes to get it working.

That being said, it's not clear to me why plans like this would not be
useful (or why would it require changes to merge join costing):

    Merge Join
      -> Gather Merge
          -> Incremental Sort
      -> Gather Merge
          -> Incremental Sort

But I have not checked the code, so maybe it would, in which case I
think it's OK to skip it in this patch (or at least in v1 of it).


>> >> 2) 0002 and 0003 are fixes I mentioned before
>
>I'm incorporating those with a bit of additional cleanup.
>
>> >> 3) 0004 adds a new GUC force_incremental_sort that (when set to 'on')
>> >> tries to nudge the optimizer into using incremental sort by essentially
>> >> making it free (i.e. using startup/total costs of the subpath). I've found
>> >> this useful when trying to force incremental sorts into plans where it may
>> >> not be the best strategy.
>> >
>> >That will be super helpful. I do wonder if we need to expose (in the
>> >production patch) a GUC of some kind to adjust incremental sort
>> >costing so that users can try to tweak when it is preferred over
>> >regular sort.
>> >
>>
>> This GUC is really meant primarily for development, to force choice of
>> incremental sort during regression tests (so as to use incremental sort in
>> as many plans as possible). I'd remove it from the final patch. I think
>> the general consensus on pgsql-hackers is that we should not introduce
>> GUCs unless absolutely necessary. But for development GUCs - sure.
>>
>> FWIW I'm not sure it's a good idea to look at both enable_incremental_sort
>> and enable_sort in cost_incremental_sort(). Not only end up with
>> disable_cost twice when both GUCs are set to 'off' at the moment, but it
>> might be useful to be able to disable those two sort types independently.
>> For example you might set just enable_sort=off and we'd still generate
>> incremental sort paths.
>
>That would cover the usage case I was getting at. Having enable_sort
>disable incremental sort also came in without much explanation [1]:
>
>On Fri, Apr 6, 2018 at 11:40 PM, Alexander Kuzmenkov <
>a(dot)kuzmenkov(at)postgrespro(dot)ru> wrote:
>> Also some other changes from me:
>> ...
>> enable_sort should act as a cost-based soft disable for both incremental
>> and normal sort.
>
>I wasn't sure that fully made sense to me, but was assuming the idea
>was to effectively not introduce a regression for anyone already
>disabling sort to force a specific plan shape. That being said, any
>new execution node/planning feature can cause a regression in such
>"hinted" queries, so I'm not sure that's a good reason on its own. In
>any case, incremental sort is different enough in performance
>qualities that I think you'd want to re-evaluate possible plans in
>queries where enable_sort=off is useful, so I'm going make incremental
>sort independent of enable_sort unless there's a strong objection
>here.
>

OK

>Tangentially: I'd almost expect enable_incremental_sort to act as a
>hard disable (and not even generate the paths) rather than a soft
>cost-based disable, since while standard sort is the most basic
>operation that needs to always be available as a last resort the same
>is not true for incremental sort...
>

Good point. It's somewhat similar to options like enable_parallel_hash
which also are "hard" switches (i.e. not cost-based penalties).

>> Another thing we should have is a collection of tests with data sets that
>> "break" the costing model in some way (skew, correlated columns,
>> non-uniform group sizes, ...). That's something not meant for commit,
>> because it'll probably require significant amounts of data, but we need it
>> to asses the quality of the planner/costing part. I know there are various
>> ad-hoc test cases in the thread history, it'd be good to consolidate that
>> into once place.
>
>Agreed.
>
>I'm continuing to work on the planning side of this with the goal of
>not needing to modify too many places to consider an incremental sort
>path + considering which ones are most likely to be useful, but I
>wanted to get my questions about get_useful_ecs_for_relation sent out
>while I work on that.
>
>If we end up wanting to limit the number of places we consider
>incremental sort (whether for planning time or merely for size of the
>initial patch, do you have any thoughts on what general areas we
>should consider most valuable? Besides the obvious LIMIT case aother
>area that might benefit was min/max, though I'm not sure yet at the
>moment if that would really end up meaning considering it all over the
>place.
>

OK, sounds like a plan!


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 

On Sun, Jul 14, 2019 at 10:16 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
> >> >> Attached is a slightly modified patch series:
> >> >>
> >> >> 1) 0001 considers incremental sort paths in various places (adds the new
> >> >> generate_useful_gather_paths and modifies places calling create_sort_path)
> >> >
> >> >I need to spend some decent time digesting this patch, but the concept
> >> >sounds very useful.
> >> >
> >>
> >> OK.
> >
> >I've been reading this several times + stepping through with the
> >debugger to understand when this is useful, but I have a few
> >questions.
> >
> >The first case considered in get_useful_pathkeys_for_relation (which
> >considers root->query_pathkeys) makes a lot of sense -- obviously if
> >we want sorted results then it's useful to consider both full and
> >incremental sort.
> >
> >But I'm not sure I see yet a way to trigger the second case (which
> >uses get_useful_ecs_for_relation to build pathkeys potentially useful
> >for merge joins). In the FDW case we need to consider it since we want
> >to avoid local sort and so want to see if the foreign server might be
> >able to provide us useful presorted data, but in the local case I
> >don't think that's useful. From what I can tell merge join path
> >costing internally considers possible sorts of both the inner and
> >outer input paths, and merge join plan node creation is responsible
> >for building explicit sort nodes as necessary (i.e., there are no
> >explicit sort paths created for merge join paths.) That means that,
> >for example, a query like:
> >
> >select * from
> >(select * from j2 order by j2.t, j2.a) j2
> >join (select * from j1 order by j1.t) j1
> >    on j1.t = j2.t and j1.a = j2.a;
> >
> >don't consider incremental sort for the merge join path (I disabled
> >hash joins, nested loops, and full sorts testing that on an empty
> >table just to easily force a merge join plan). And unfortunately I
> >don't think there's an easy way to remedy that: from what I can tell
> >it'd be a pretty invasive patch requiring refactoring merge join
> >costing to consider both kinds of sorting (in the most simple
> >implementation that would mean considering up to 4x as many merge join
> >paths -- inner/outer sorted by: full/full, full/incremental,
> >incremental/full, and incremental/incremental). Given that's a
> >significant undertaking on its own, I think I'm going to avoid
> >addressing it as part of this patch.
> >
> >If it's true that the get_useful_ecs_for_relation part of that logic
> >isn't actually exercisable currently, that that would cut down
> >significantly on the amount of code that needs to be added for
> >consideration of valid gather merge paths. But if you can think of a
> >counterexample, please let me know.
> >
>
> It's quite possible parts of the code are not needed - I've pretty much
> just copied the code and did minimal changes to get it working.
>
> That being said, it's not clear to me why plans like this would not be
> useful (or why would it require changes to merge join costing):
>
>     Merge Join
>       -> Gather Merge
>           -> Incremental Sort
>       -> Gather Merge
>           -> Incremental Sort
>
> But I have not checked the code, so maybe it would, in which case I
> think it's OK to skip it in this patch (or at least in v1 of it).

Someone could correct me if I'm wrong, but I've noticed some comments
that seem to imply we avoid plans with multiple parallel gather
[merge]s; i.e., we try to put a single parallel node as high as
possible in the plan. I assume that's to avoid multiplying out the
number of workers we might consume. And in the sample query above that
kind of plan never got considered because there were no partial paths
to loop through (I'm not sure I fully understand why) when the new
code is called from under apply_scanjoin_target_to_paths().

Of course, we should consider non-parallel incremental sort inputs to
merge joins...but as I noted that's a lot of extra work...

> >> FWIW I'm not sure it's a good idea to look at both enable_incremental_sort
> >> and enable_sort in cost_incremental_sort(). Not only end up with
> >> disable_cost twice when both GUCs are set to 'off' at the moment, but it
> >> might be useful to be able to disable those two sort types independently.
> >> For example you might set just enable_sort=off and we'd still generate
> >> incremental sort paths.
> >
> >That would cover the usage case I was getting at. Having enable_sort
> >disable incremental sort also came in without much explanation [1]:
> >
> >On Fri, Apr 6, 2018 at 11:40 PM, Alexander Kuzmenkov <
> >a(dot)kuzmenkov(at)postgrespro(dot)ru> wrote:
> >> Also some other changes from me:
> >> ...
> >> enable_sort should act as a cost-based soft disable for both incremental
> >> and normal sort.
> >
> >I wasn't sure that fully made sense to me, but was assuming the idea
> >was to effectively not introduce a regression for anyone already
> >disabling sort to force a specific plan shape. That being said, any
> >new execution node/planning feature can cause a regression in such
> >"hinted" queries, so I'm not sure that's a good reason on its own. In
> >any case, incremental sort is different enough in performance
> >qualities that I think you'd want to re-evaluate possible plans in
> >queries where enable_sort=off is useful, so I'm going make incremental
> >sort independent of enable_sort unless there's a strong objection
> >here.
> >
>
> OK
>
> >Tangentially: I'd almost expect enable_incremental_sort to act as a
> >hard disable (and not even generate the paths) rather than a soft
> >cost-based disable, since while standard sort is the most basic
> >operation that needs to always be available as a last resort the same
> >is not true for incremental sort...
> >
>
> Good point. It's somewhat similar to options like enable_parallel_hash
> which also are "hard" switches (i.e. not cost-based penalties).

I assume the proper approach here then is to check the GUC before
building and adding the path?

> >If we end up wanting to limit the number of places we consider
> >incremental sort (whether for planning time or merely for size of the
> >initial patch, do you have any thoughts on what general areas we
> >should consider most valuable? Besides the obvious LIMIT case aother
> >area that might benefit was min/max, though I'm not sure yet at the
> >moment if that would really end up meaning considering it all over the
> >place.
> >
>
> OK, sounds like a plan!

Did you have any thoughts on the question about where this is likely
to be most valuable?

James Coleman

On Mon, Jul 15, 2019 at 09:25:32AM -0400, James Coleman wrote:
>On Sun, Jul 14, 2019 at 10:16 PM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>> >> >> Attached is a slightly modified patch series:
>> >> >>
>> >> >> 1) 0001 considers incremental sort paths in various places (adds the new
>> >> >> generate_useful_gather_paths and modifies places calling create_sort_path)
>> >> >
>> >> >I need to spend some decent time digesting this patch, but the concept
>> >> >sounds very useful.
>> >> >
>> >>
>> >> OK.
>> >
>> >I've been reading this several times + stepping through with the
>> >debugger to understand when this is useful, but I have a few
>> >questions.
>> >
>> >The first case considered in get_useful_pathkeys_for_relation (which
>> >considers root->query_pathkeys) makes a lot of sense -- obviously if
>> >we want sorted results then it's useful to consider both full and
>> >incremental sort.
>> >
>> >But I'm not sure I see yet a way to trigger the second case (which
>> >uses get_useful_ecs_for_relation to build pathkeys potentially useful
>> >for merge joins). In the FDW case we need to consider it since we want
>> >to avoid local sort and so want to see if the foreign server might be
>> >able to provide us useful presorted data, but in the local case I
>> >don't think that's useful. From what I can tell merge join path
>> >costing internally considers possible sorts of both the inner and
>> >outer input paths, and merge join plan node creation is responsible
>> >for building explicit sort nodes as necessary (i.e., there are no
>> >explicit sort paths created for merge join paths.) That means that,
>> >for example, a query like:
>> >
>> >select * from
>> >(select * from j2 order by j2.t, j2.a) j2
>> >join (select * from j1 order by j1.t) j1
>> >    on j1.t = j2.t and j1.a = j2.a;
>> >
>> >don't consider incremental sort for the merge join path (I disabled
>> >hash joins, nested loops, and full sorts testing that on an empty
>> >table just to easily force a merge join plan). And unfortunately I
>> >don't think there's an easy way to remedy that: from what I can tell
>> >it'd be a pretty invasive patch requiring refactoring merge join
>> >costing to consider both kinds of sorting (in the most simple
>> >implementation that would mean considering up to 4x as many merge join
>> >paths -- inner/outer sorted by: full/full, full/incremental,
>> >incremental/full, and incremental/incremental). Given that's a
>> >significant undertaking on its own, I think I'm going to avoid
>> >addressing it as part of this patch.
>> >
>> >If it's true that the get_useful_ecs_for_relation part of that logic
>> >isn't actually exercisable currently, that that would cut down
>> >significantly on the amount of code that needs to be added for
>> >consideration of valid gather merge paths. But if you can think of a
>> >counterexample, please let me know.
>> >
>>
>> It's quite possible parts of the code are not needed - I've pretty much
>> just copied the code and did minimal changes to get it working.
>>
>> That being said, it's not clear to me why plans like this would not be
>> useful (or why would it require changes to merge join costing):
>>
>>     Merge Join
>>       -> Gather Merge
>>           -> Incremental Sort
>>       -> Gather Merge
>>           -> Incremental Sort
>>
>> But I have not checked the code, so maybe it would, in which case I
>> think it's OK to skip it in this patch (or at least in v1 of it).
>
>Someone could correct me if I'm wrong, but I've noticed some comments
>that seem to imply we avoid plans with multiple parallel gather
>[merge]s; i.e., we try to put a single parallel node as high as
>possible in the plan. I assume that's to avoid multiplying out the
>number of workers we might consume. And in the sample query above that
>kind of plan never got considered because there were no partial paths
>to loop through (I'm not sure I fully understand why) when the new
>code is called from under apply_scanjoin_target_to_paths().
>

I think you're probably right in this case it'd be more efficient to just
do one Gather Merge on top of the merge join. And yes - we try to put the
Gather node as high as possible, to parallelize as large part of a query
as possible. Keeping the number of necessary workers low is one reason,
but it's also about the serial part in Amdahl's law.

>Of course, we should consider non-parallel incremental sort inputs to
>merge joins...but as I noted that's a lot of extra work...
>

OK, understood. I agree such plans would be useful, but if you think it'd
be a lot of extra work, then we can leave it out for now.

Although, looking at the mergejoin path construction code, it does not
seem very complex. Essentially, generate_mergejoin_paths() would need to
consider adding incremental sort on inner/outer path. It would need some
refactoring, but it's not clear to me why would this need changes to
costing? Essentially, we'd produce multiple mergejoin paths that would be
costed by the current code.

That being said, I'm perfectly fine with ignoring this for now. There are
more fundamental bits that we need to tackle first.

>> >> FWIW I'm not sure it's a good idea to look at both enable_incremental_sort
>> >> and enable_sort in cost_incremental_sort(). Not only end up with
>> >> disable_cost twice when both GUCs are set to 'off' at the moment, but it
>> >> might be useful to be able to disable those two sort types independently.
>> >> For example you might set just enable_sort=off and we'd still generate
>> >> incremental sort paths.
>> >
>> >That would cover the usage case I was getting at. Having enable_sort
>> >disable incremental sort also came in without much explanation [1]:
>> >
>> >On Fri, Apr 6, 2018 at 11:40 PM, Alexander Kuzmenkov <
>> >a(dot)kuzmenkov(at)postgrespro(dot)ru> wrote:
>> >> Also some other changes from me:
>> >> ...
>> >> enable_sort should act as a cost-based soft disable for both incremental
>> >> and normal sort.
>> >
>> >I wasn't sure that fully made sense to me, but was assuming the idea
>> >was to effectively not introduce a regression for anyone already
>> >disabling sort to force a specific plan shape. That being said, any
>> >new execution node/planning feature can cause a regression in such
>> >"hinted" queries, so I'm not sure that's a good reason on its own. In
>> >any case, incremental sort is different enough in performance
>> >qualities that I think you'd want to re-evaluate possible plans in
>> >queries where enable_sort=off is useful, so I'm going make incremental
>> >sort independent of enable_sort unless there's a strong objection
>> >here.
>> >
>>
>> OK
>>
>> >Tangentially: I'd almost expect enable_incremental_sort to act as a
>> >hard disable (and not even generate the paths) rather than a soft
>> >cost-based disable, since while standard sort is the most basic
>> >operation that needs to always be available as a last resort the same
>> >is not true for incremental sort...
>> >
>>
>> Good point. It's somewhat similar to options like enable_parallel_hash
>> which also are "hard" switches (i.e. not cost-based penalties).
>
>I assume the proper approach here then is to check the GUC before
>building and adding the path?
>

Yeah. The simplest way to do that might be setting the number of presorted
keys to 0 in pathkeys_common_contained_in(). That effectively says it
makes no sense to do incremental sort. It's a bit too deep, though - not
sure it does not affect any other plans, though.

Or you might reference the GUC in every place that considers incremental
sort, but that's going to be a lot of places.

>> >If we end up wanting to limit the number of places we consider
>> >incremental sort (whether for planning time or merely for size of the
>> >initial patch, do you have any thoughts on what general areas we
>> >should consider most valuable? Besides the obvious LIMIT case aother
>> >area that might benefit was min/max, though I'm not sure yet at the
>> >moment if that would really end up meaning considering it all over the
>> >place.
>> >
>>
>> OK, sounds like a plan!
>
>Did you have any thoughts on the question about where this is likely
>to be most valuable?
>

Good question. I can certainly list some generic cases where I'd expect
incremental sort to be most beneficial:

1) (ORDER BY + LIMIT) - in this case the main advantage is low startup
cost (compared to explicit sort, which has to fetch/sort everything)

2) ORDER BY with on-disk sort - in this case the main advantage is ability
to sort data in small chunks that fit in memory, instead of flushing large
amounts of data into temporary files

Of course, (2) helps with any operation that can leverage the ordering,
i.e. aggregation/...

But the question is how to map this to places in the source code. I don't
have a very good answer to that :-(

IMO the best thing we can do is get some realistic queries, and address
them first. And then eventually add incremental sort to some other places. 


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Mon, Jul 8, 2019 at 9:37 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
> Now, consider this example:
>
>   create table t (a int, b int, c int);
>   insert into t select mod(i,100),mod(i,100),i from generate_series(1,10000000) s(i);
>   create index on t (a);
>   analyze t;
>   explain select a,b,sum(c) from t group by 1,2 order by 1,2,3 limit 1;
>
> With 0001+0002+0003 pathes, I get a plan like this:
>
>                                                      QUERY PLAN
> --------------------------------------------------------------------------------------------------------------------
>  Limit  (cost=10375.39..10594.72 rows=1 width=16)
>    ->  Incremental Sort  (cost=10375.39..2203675.71 rows=10000 width=16)
>          Sort Key: a, b, (sum(c))
>          Presorted Key: a, b
>          ->  GroupAggregate  (cost=10156.07..2203225.71 rows=10000 width=16)
>                Group Key: a, b
>                ->  Gather Merge  (cost=10156.07..2128124.39 rows=10000175 width=12)
>                      Workers Planned: 2
>                      ->  Incremental Sort  (cost=9156.04..972856.05 rows=4166740 width=12)
>                            Sort Key: a, b
>                            Presorted Key: a
>                            ->  Parallel Index Scan using t_a_idx on t  (cost=0.43..417690.30 rows=4166740 width=12)
> (12 rows)
>
>
> and with 0004, I get this:
>
>                                               QUERY PLAN
> ------------------------------------------------------------------------------------------------------
>  Limit  (cost=20443.84..20665.32 rows=1 width=16)
>    ->  Incremental Sort  (cost=20443.84..2235250.05 rows=10000 width=16)
>          Sort Key: a, b, (sum(c))
>          Presorted Key: a, b
>          ->  GroupAggregate  (cost=20222.37..2234800.05 rows=10000 width=16)
>                Group Key: a, b
>                ->  Incremental Sort  (cost=20222.37..2159698.74 rows=10000175 width=12)
>                      Sort Key: a, b
>                      Presorted Key: a
>                      ->  Index Scan using t_a_idx on t  (cost=0.43..476024.65 rows=10000175 width=12)
> (10 rows)
>
> Notice that cost of the second plan is almost double the first one. That
> means 0004 does not even generate the first plan, i.e. there are cases
> where we don't try to add the explicit sort before passing the path to
> generate_gather_paths().
>
> And I think I know why is that - while gather_grouping_paths() tries to
> add explicit sort below the gather merge, there are other places that
> call generate_gather_paths() that don't do that. In this case it's
> probably apply_scanjoin_target_to_paths() which simply builds
>
>    parallel (seq|index) scan + gather merge
>
> and that's it. The problem is likely the same - the code does not know
> which pathkeys are "interesting" at that point. We probably need to
> teach planner to do this.

I've been working on figuring out sample queries for each of the
places we're looking at adding create_increment_sort() (starting with
the cases enabled by gather-merge nodes). The
generate_useful_gather_paths() call in
apply_scanjoin_target_to_paths() is required to generate the above
preferred plan.

But I found that if I set enable_sort=off (with our without the
_useful_ variant of generate_gather_paths()) I get a very similar plan
that's actually lower cost yet:

                                                        QUERY PLAN

--------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=10255.98..10355.77 rows=1 width=16)
   ->  Incremental Sort  (cost=10255.98..1008228.67 rows=10000 width=16)
         Sort Key: a, b, (sum(c))
         Presorted Key: a, b
         ->  Finalize GroupAggregate  (cost=10156.20..1007778.67
rows=10000 width=16)
               Group Key: a, b
               ->  Gather Merge  (cost=10156.20..1007528.67 rows=20000 width=16)
                     Workers Planned: 2
                     ->  Partial GroupAggregate
(cost=9156.18..1004220.15 rows=10000 width=16)
                           Group Key: a, b
                           ->  Incremental Sort
(cost=9156.18..972869.60 rows=4166740 width=12)
                                 Sort Key: a, b
                                 Presorted Key: a
                                 ->  Parallel Index Scan using t_a_idx
on t  (cost=0.43..417703.85 rows=4166740 width=12)

Is that something we should consider a bug at this stage? It's also
not clear to mean (costing aside) which plan is intuitively
preferable.

James

On Fri, Jul 19, 2019 at 04:59:21PM -0400, James Coleman wrote:
>On Mon, Jul 8, 2019 at 9:37 PM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>> Now, consider this example:
>>
>>   create table t (a int, b int, c int);
>>   insert into t select mod(i,100),mod(i,100),i from generate_series(1,10000000) s(i);
>>   create index on t (a);
>>   analyze t;
>>   explain select a,b,sum(c) from t group by 1,2 order by 1,2,3 limit 1;
>>
>> With 0001+0002+0003 pathes, I get a plan like this:
>>
>>                                                      QUERY PLAN
>>
--------------------------------------------------------------------------------------------------------------------
>>  Limit  (cost=10375.39..10594.72 rows=1 width=16)
>>    ->  Incremental Sort  (cost=10375.39..2203675.71 rows=10000 width=16)
>>          Sort Key: a, b, (sum(c))
>>          Presorted Key: a, b
>>          ->  GroupAggregate  (cost=10156.07..2203225.71 rows=10000 width=16)
>>                Group Key: a, b
>>                ->  Gather Merge  (cost=10156.07..2128124.39 rows=10000175 width=12)
>>                      Workers Planned: 2
>>                      ->  Incremental Sort  (cost=9156.04..972856.05 rows=4166740 width=12)
>>                            Sort Key: a, b
>>                            Presorted Key: a
>>                            ->  Parallel Index Scan using t_a_idx on t  (cost=0.43..417690.30 rows=4166740 width=12)
>> (12 rows)
>>
>>
>> and with 0004, I get this:
>>
>>                                               QUERY PLAN
>> ------------------------------------------------------------------------------------------------------
>>  Limit  (cost=20443.84..20665.32 rows=1 width=16)
>>    ->  Incremental Sort  (cost=20443.84..2235250.05 rows=10000 width=16)
>>          Sort Key: a, b, (sum(c))
>>          Presorted Key: a, b
>>          ->  GroupAggregate  (cost=20222.37..2234800.05 rows=10000 width=16)
>>                Group Key: a, b
>>                ->  Incremental Sort  (cost=20222.37..2159698.74 rows=10000175 width=12)
>>                      Sort Key: a, b
>>                      Presorted Key: a
>>                      ->  Index Scan using t_a_idx on t  (cost=0.43..476024.65 rows=10000175 width=12)
>> (10 rows)
>>
>> Notice that cost of the second plan is almost double the first one. That
>> means 0004 does not even generate the first plan, i.e. there are cases
>> where we don't try to add the explicit sort before passing the path to
>> generate_gather_paths().
>>
>> And I think I know why is that - while gather_grouping_paths() tries to
>> add explicit sort below the gather merge, there are other places that
>> call generate_gather_paths() that don't do that. In this case it's
>> probably apply_scanjoin_target_to_paths() which simply builds
>>
>>    parallel (seq|index) scan + gather merge
>>
>> and that's it. The problem is likely the same - the code does not know
>> which pathkeys are "interesting" at that point. We probably need to
>> teach planner to do this.
>
>I've been working on figuring out sample queries for each of the
>places we're looking at adding create_increment_sort() (starting with
>the cases enabled by gather-merge nodes). The
>generate_useful_gather_paths() call in
>apply_scanjoin_target_to_paths() is required to generate the above
>preferred plan.
>
>But I found that if I set enable_sort=off (with our without the
>_useful_ variant of generate_gather_paths()) I get a very similar plan
>that's actually lower cost yet:
>
>                                                        QUERY PLAN

>--------------------------------------------------------------------------------------------------------------------------
> Limit  (cost=10255.98..10355.77 rows=1 width=16)
>   ->  Incremental Sort  (cost=10255.98..1008228.67 rows=10000 width=16)
>         Sort Key: a, b, (sum(c))
>         Presorted Key: a, b
>         ->  Finalize GroupAggregate  (cost=10156.20..1007778.67
>rows=10000 width=16)
>               Group Key: a, b
>               ->  Gather Merge  (cost=10156.20..1007528.67 rows=20000 width=16)
>                     Workers Planned: 2
>                     ->  Partial GroupAggregate
>(cost=9156.18..1004220.15 rows=10000 width=16)
>                           Group Key: a, b
>                           ->  Incremental Sort
>(cost=9156.18..972869.60 rows=4166740 width=12)
>                                 Sort Key: a, b
>                                 Presorted Key: a
>                                 ->  Parallel Index Scan using t_a_idx
>on t  (cost=0.43..417703.85 rows=4166740 width=12)
>
>Is that something we should consider a bug at this stage? It's also
>not clear to mean (costing aside) which plan is intuitively
>preferable.
>

This seems like a thinko in add_partial_path() - it only looks at total
cost of the paths, and ignores the startup cost entirely. I've debugged
it a bit, and what's happening for the partially-grouped relation is
roughly this:

1) We add a partial path with startup/total costs 696263 / 738029

2) We attempt to add the "Partial GroupAggregate" path, but it loses the
fight because the total cost (1004207) is higher than the first path.
Which entirely misses that the startup cost is way lower.

3) We however use the startup cost later when computing the LIMIT cost
(because that's linear approximation between startup and total cost),
and we reject the first path too, because we happen to find something
cheaper (but more expensive than what we'd get the path rejected in 2).

Attached is a debug patch which makes this clear - it only prints info
about first step of partial aggregates, because that's what matters in
this example. You should see something like this:

WARNING:  rel 0x2aa8e00 adding partial agg path 0x2aa9448 startup 696263.839993 total 738029.919993
WARNING:  rel 0x2aa8e00 path 0x2aa9448 adding new = 1
WARNING:  rel 0x2aa8e00 adding partial agg path 0x2aa9710 startup 9156.084995 total 1004207.854495
WARNING:  A: new path 0x2aa9710 rejected because of 0x2aa9448
WARNING:  rel 0x2aa8e00 path 0x2aa9710 adding new = 0

which essentially says "path rejected because of total cost" (and you
know it's the interesting partial aggregate from the second plan). And
if you disable sort, you get this:

WARNING:  rel 0x2aa8e00 adding partial agg path 0x2aa9448 startup 10000696263.839994 total 10000738029.919996
WARNING:  rel 0x2aa8e00 path 0x2aa9448 adding new = 1
WARNING:  rel 0x2aa8e00 adding partial agg path 0x2aa9710 startup 9156.084995 total 1004207.854495
WARNING:  rel 0x2aa8e00 path 0x2aa9710 adding new = 1
...

So in this case we decided the path is interesting, thanks to the
increased cost of sort.

The comment for add_partial_path says this:

 *  Neither do we need to consider startup costs: parallelism is only
 *  used for plans that will be run to completion.  Therefore, this
 *  routine is much simpler than add_path: it needs to consider only
 *  pathkeys and total cost.

I think this may be a thinko, as this plan demonstrates - but I'm not
sure about it. I wonder if this might be penalizing some other types of
plans (essentially anything with limit + gather).

Attached is a WIP patch fixing this by considering both startup and
total cost (by calling compare_path_costs_fuzzily).


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Sat, Jul 20, 2019 at 9:22 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Fri, Jul 19, 2019 at 04:59:21PM -0400, James Coleman wrote:
> >On Mon, Jul 8, 2019 at 9:37 PM Tomas Vondra
> ><tomas.vondra@2ndquadrant.com> wrote:
> >> Now, consider this example:
> >>
> >>   create table t (a int, b int, c int);
> >>   insert into t select mod(i,100),mod(i,100),i from generate_series(1,10000000) s(i);
> >>   create index on t (a);
> >>   analyze t;
> >>   explain select a,b,sum(c) from t group by 1,2 order by 1,2,3 limit 1;
> >>
> >> With 0001+0002+0003 pathes, I get a plan like this:
> >>
> >>                                                      QUERY PLAN
> >>
--------------------------------------------------------------------------------------------------------------------
> >>  Limit  (cost=10375.39..10594.72 rows=1 width=16)
> >>    ->  Incremental Sort  (cost=10375.39..2203675.71 rows=10000 width=16)
> >>          Sort Key: a, b, (sum(c))
> >>          Presorted Key: a, b
> >>          ->  GroupAggregate  (cost=10156.07..2203225.71 rows=10000 width=16)
> >>                Group Key: a, b
> >>                ->  Gather Merge  (cost=10156.07..2128124.39 rows=10000175 width=12)
> >>                      Workers Planned: 2
> >>                      ->  Incremental Sort  (cost=9156.04..972856.05 rows=4166740 width=12)
> >>                            Sort Key: a, b
> >>                            Presorted Key: a
> >>                            ->  Parallel Index Scan using t_a_idx on t  (cost=0.43..417690.30 rows=4166740
width=12)
> >> (12 rows)
> >>
> >>
> >> and with 0004, I get this:
> >>
> >>                                               QUERY PLAN
> >> ------------------------------------------------------------------------------------------------------
> >>  Limit  (cost=20443.84..20665.32 rows=1 width=16)
> >>    ->  Incremental Sort  (cost=20443.84..2235250.05 rows=10000 width=16)
> >>          Sort Key: a, b, (sum(c))
> >>          Presorted Key: a, b
> >>          ->  GroupAggregate  (cost=20222.37..2234800.05 rows=10000 width=16)
> >>                Group Key: a, b
> >>                ->  Incremental Sort  (cost=20222.37..2159698.74 rows=10000175 width=12)
> >>                      Sort Key: a, b
> >>                      Presorted Key: a
> >>                      ->  Index Scan using t_a_idx on t  (cost=0.43..476024.65 rows=10000175 width=12)
> >> (10 rows)
> >>
> >> Notice that cost of the second plan is almost double the first one. That
> >> means 0004 does not even generate the first plan, i.e. there are cases
> >> where we don't try to add the explicit sort before passing the path to
> >> generate_gather_paths().
> >>
> >> And I think I know why is that - while gather_grouping_paths() tries to
> >> add explicit sort below the gather merge, there are other places that
> >> call generate_gather_paths() that don't do that. In this case it's
> >> probably apply_scanjoin_target_to_paths() which simply builds
> >>
> >>    parallel (seq|index) scan + gather merge
> >>
> >> and that's it. The problem is likely the same - the code does not know
> >> which pathkeys are "interesting" at that point. We probably need to
> >> teach planner to do this.
> >
> >I've been working on figuring out sample queries for each of the
> >places we're looking at adding create_increment_sort() (starting with
> >the cases enabled by gather-merge nodes). The
> >generate_useful_gather_paths() call in
> >apply_scanjoin_target_to_paths() is required to generate the above
> >preferred plan.

As I continue this, I've added a couple of test cases (notably for
generate_useful_gather_paths() in both standard_join_search() and
apply_scanjoin_target_to_paths()). Those, plus the current WIP state
of my hacking on your patch adding generate_useful_gather_paths() is
attached as 0001-parallel-and-more-paths.patch.

My current line of investigation is whether we need to do anything in
the parallel portion of create_ordered_paths(). I noticed that the
first-pass patch adding generate_useful_gather_paths() modified that
section but wasn't actually adding any new gather-merge paths (just
bare incremental sort paths). That seems pretty clearly just a
prototype miss, so I modified the prototype to build gather-merge
paths instead (as a side note that change seems to fix an oddity I was
seeing where plans would include a parallel index scan node even
though they weren't parallel plans). While the resulting plan for
something like:

explain analyze select * from t where t.a in (1,2,3,4,5,6) order by
t.a, t.b limit 50;

changes cost (to be cheaper) ever so slightly with the gather-merge
addition to create_ordered_paths(), the plan itself is otherwise
identical (including row estimates):

Limit
  -> Gather Merge
       -> Incremental Sort
          -> Parallel Index Scan

(Note: I'm forcing parallel plans here with: set
max_parallel_workers_per_gather=4; set min_parallel_table_scan_size=0;
set parallel_tuple_cost=0; set parallel_setup_cost=0; set
min_parallel_index_scan_size=0;)

I can't seem to come up with a case where adding these gather-merge
paths in create_ordered_paths() isn't entirely duplicative of paths
already created by generate_useful_gather_paths() as called from
apply_scanjoin_target_to_paths() -- which I _think_ makes sense given
that both apply_scanjoin_target_to_paths() and create_ordered_paths()
are called by grouping_planner().

Can you think of a case I'm missing here that would make it valuable
to generate new parallel plans in create_ordered_paths()?

> >But I found that if I set enable_sort=off (with our without the
> >_useful_ variant of generate_gather_paths()) I get a very similar plan
> >that's actually lower cost yet:
> >
> >                                                        QUERY PLAN
>
>--------------------------------------------------------------------------------------------------------------------------
> > Limit  (cost=10255.98..10355.77 rows=1 width=16)
> >   ->  Incremental Sort  (cost=10255.98..1008228.67 rows=10000 width=16)
> >         Sort Key: a, b, (sum(c))
> >         Presorted Key: a, b
> >         ->  Finalize GroupAggregate  (cost=10156.20..1007778.67
> >rows=10000 width=16)
> >               Group Key: a, b
> >               ->  Gather Merge  (cost=10156.20..1007528.67 rows=20000 width=16)
> >                     Workers Planned: 2
> >                     ->  Partial GroupAggregate
> >(cost=9156.18..1004220.15 rows=10000 width=16)
> >                           Group Key: a, b
> >                           ->  Incremental Sort
> >(cost=9156.18..972869.60 rows=4166740 width=12)
> >                                 Sort Key: a, b
> >                                 Presorted Key: a
> >                                 ->  Parallel Index Scan using t_a_idx
> >on t  (cost=0.43..417703.85 rows=4166740 width=12)
> >
> >Is that something we should consider a bug at this stage? It's also
> >not clear to mean (costing aside) which plan is intuitively
> >preferable.
> >
>
> This seems like a thinko in add_partial_path() - it only looks at total
> cost of the paths, and ignores the startup cost entirely. I've debugged
> it a bit, and what's happening for the partially-grouped relation is
> roughly this:
>
> 1) We add a partial path with startup/total costs 696263 / 738029
>
> 2) We attempt to add the "Partial GroupAggregate" path, but it loses the
> fight because the total cost (1004207) is higher than the first path.
> Which entirely misses that the startup cost is way lower.
>
> 3) We however use the startup cost later when computing the LIMIT cost
> (because that's linear approximation between startup and total cost),
> and we reject the first path too, because we happen to find something
> cheaper (but more expensive than what we'd get the path rejected in 2).
>
> Attached is a debug patch which makes this clear - it only prints info
> about first step of partial aggregates, because that's what matters in
> this example. You should see something like this:
>
> WARNING:  rel 0x2aa8e00 adding partial agg path 0x2aa9448 startup 696263.839993 total 738029.919993
> WARNING:  rel 0x2aa8e00 path 0x2aa9448 adding new = 1
> WARNING:  rel 0x2aa8e00 adding partial agg path 0x2aa9710 startup 9156.084995 total 1004207.854495
> WARNING:  A: new path 0x2aa9710 rejected because of 0x2aa9448
> WARNING:  rel 0x2aa8e00 path 0x2aa9710 adding new = 0
>
> which essentially says "path rejected because of total cost" (and you
> know it's the interesting partial aggregate from the second plan). And
> if you disable sort, you get this:
>
> WARNING:  rel 0x2aa8e00 adding partial agg path 0x2aa9448 startup 10000696263.839994 total 10000738029.919996
> WARNING:  rel 0x2aa8e00 path 0x2aa9448 adding new = 1
> WARNING:  rel 0x2aa8e00 adding partial agg path 0x2aa9710 startup 9156.084995 total 1004207.854495
> WARNING:  rel 0x2aa8e00 path 0x2aa9710 adding new = 1
> ...
>
> So in this case we decided the path is interesting, thanks to the
> increased cost of sort.
>
> The comment for add_partial_path says this:
>
>  *  Neither do we need to consider startup costs: parallelism is only
>  *  used for plans that will be run to completion.  Therefore, this
>  *  routine is much simpler than add_path: it needs to consider only
>  *  pathkeys and total cost.
>
> I think this may be a thinko, as this plan demonstrates - but I'm not
> sure about it. I wonder if this might be penalizing some other types of
> plans (essentially anything with limit + gather).
>
> Attached is a WIP patch fixing this by considering both startup and
> total cost (by calling compare_path_costs_fuzzily).

It seems to me that this is likely a bug, and not just a changed
needed for this. Do you think it's better addressed in a separate
thread? Or retain it as part of this patch for now (and possibly break
it out later)? On the other hand, it's entirely possible that someone
more familiar with parallel plan limitations could explain why the
above comment holds true. That makes me lean towards asking in a new
thread.

I've also attached a new base patch (incremental-sort-30.patch) which
includes some of the other obvious fixes (costing, etc.) that you'd
previously proposed.

James Coleman

On Sat, Jul 20, 2019 at 10:33:02AM -0400, James Coleman wrote:
>On Sat, Jul 20, 2019 at 9:22 AM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>>
>> On Fri, Jul 19, 2019 at 04:59:21PM -0400, James Coleman wrote:
>> >On Mon, Jul 8, 2019 at 9:37 PM Tomas Vondra
>> ><tomas.vondra@2ndquadrant.com> wrote:
>> >> Now, consider this example:
>> >>
>> >>   create table t (a int, b int, c int);
>> >>   insert into t select mod(i,100),mod(i,100),i from generate_series(1,10000000) s(i);
>> >>   create index on t (a);
>> >>   analyze t;
>> >>   explain select a,b,sum(c) from t group by 1,2 order by 1,2,3 limit 1;
>> >>
>> >> With 0001+0002+0003 pathes, I get a plan like this:
>> >>
>> >>                                                      QUERY PLAN
>> >>
--------------------------------------------------------------------------------------------------------------------
>> >>  Limit  (cost=10375.39..10594.72 rows=1 width=16)
>> >>    ->  Incremental Sort  (cost=10375.39..2203675.71 rows=10000 width=16)
>> >>          Sort Key: a, b, (sum(c))
>> >>          Presorted Key: a, b
>> >>          ->  GroupAggregate  (cost=10156.07..2203225.71 rows=10000 width=16)
>> >>                Group Key: a, b
>> >>                ->  Gather Merge  (cost=10156.07..2128124.39 rows=10000175 width=12)
>> >>                      Workers Planned: 2
>> >>                      ->  Incremental Sort  (cost=9156.04..972856.05 rows=4166740 width=12)
>> >>                            Sort Key: a, b
>> >>                            Presorted Key: a
>> >>                            ->  Parallel Index Scan using t_a_idx on t  (cost=0.43..417690.30 rows=4166740
width=12)
>> >> (12 rows)
>> >>
>> >>
>> >> and with 0004, I get this:
>> >>
>> >>                                               QUERY PLAN
>> >> ------------------------------------------------------------------------------------------------------
>> >>  Limit  (cost=20443.84..20665.32 rows=1 width=16)
>> >>    ->  Incremental Sort  (cost=20443.84..2235250.05 rows=10000 width=16)
>> >>          Sort Key: a, b, (sum(c))
>> >>          Presorted Key: a, b
>> >>          ->  GroupAggregate  (cost=20222.37..2234800.05 rows=10000 width=16)
>> >>                Group Key: a, b
>> >>                ->  Incremental Sort  (cost=20222.37..2159698.74 rows=10000175 width=12)
>> >>                      Sort Key: a, b
>> >>                      Presorted Key: a
>> >>                      ->  Index Scan using t_a_idx on t  (cost=0.43..476024.65 rows=10000175 width=12)
>> >> (10 rows)
>> >>
>> >> Notice that cost of the second plan is almost double the first one. That
>> >> means 0004 does not even generate the first plan, i.e. there are cases
>> >> where we don't try to add the explicit sort before passing the path to
>> >> generate_gather_paths().
>> >>
>> >> And I think I know why is that - while gather_grouping_paths() tries to
>> >> add explicit sort below the gather merge, there are other places that
>> >> call generate_gather_paths() that don't do that. In this case it's
>> >> probably apply_scanjoin_target_to_paths() which simply builds
>> >>
>> >>    parallel (seq|index) scan + gather merge
>> >>
>> >> and that's it. The problem is likely the same - the code does not know
>> >> which pathkeys are "interesting" at that point. We probably need to
>> >> teach planner to do this.
>> >
>> >I've been working on figuring out sample queries for each of the
>> >places we're looking at adding create_increment_sort() (starting with
>> >the cases enabled by gather-merge nodes). The
>> >generate_useful_gather_paths() call in
>> >apply_scanjoin_target_to_paths() is required to generate the above
>> >preferred plan.
>
>As I continue this, I've added a couple of test cases (notably for
>generate_useful_gather_paths() in both standard_join_search() and
>apply_scanjoin_target_to_paths()). Those, plus the current WIP state
>of my hacking on your patch adding generate_useful_gather_paths() is
>attached as 0001-parallel-and-more-paths.patch.
>
>My current line of investigation is whether we need to do anything in
>the parallel portion of create_ordered_paths(). I noticed that the
>first-pass patch adding generate_useful_gather_paths() modified that
>section but wasn't actually adding any new gather-merge paths (just
>bare incremental sort paths). That seems pretty clearly just a
>prototype miss, so I modified the prototype to build gather-merge
>paths instead (as a side note that change seems to fix an oddity I was
>seeing where plans would include a parallel index scan node even
>though they weren't parallel plans). While the resulting plan for
>something like:
>

Yes, that seems to be a bug. The block above it clealy has a gather
merge nodes, so this one should too.

>explain analyze select * from t where t.a in (1,2,3,4,5,6) order by
>t.a, t.b limit 50;
>
>changes cost (to be cheaper) ever so slightly with the gather-merge
>addition to create_ordered_paths(), the plan itself is otherwise
>identical (including row estimates):
>
>Limit
>  -> Gather Merge
>       -> Incremental Sort
>          -> Parallel Index Scan
>
>(Note: I'm forcing parallel plans here with: set
>max_parallel_workers_per_gather=4; set min_parallel_table_scan_size=0;
>set parallel_tuple_cost=0; set parallel_setup_cost=0; set
>min_parallel_index_scan_size=0;)
>
>I can't seem to come up with a case where adding these gather-merge
>paths in create_ordered_paths() isn't entirely duplicative of paths
>already created by generate_useful_gather_paths() as called from
>apply_scanjoin_target_to_paths() -- which I _think_ makes sense given
>that both apply_scanjoin_target_to_paths() and create_ordered_paths()
>are called by grouping_planner().
>
>Can you think of a case I'm missing here that would make it valuable
>to generate new parallel plans in create_ordered_paths()?
>

Good question. Not sure. I think such path would need to do something on
a relation that is neither a join nor a scan - in which case the path
should not be created by apply_scanjoin_target_to_paths().

So for example a query like this:

  SELECT
      a, b, sum(expensive_function(c))
  FROM
      t
  GROUP BY a,b
  ORDER BY a,sum(...) LIMIT 10;

should be able to produce a plan like this:

  -> Limit
      -> Gather Merge
        -> Incremental Sort (pathkeys: a, sum)
          -> Group Aggregate
             a, b, sum(expensive_function(c))
            -> Index Scan (pathkeys: a, b)

or something like that, maybe. I haven't tried this, though. The other
question is whether such queries are useful in practice ...

>> ...
>>
>> I think this may be a thinko, as this plan demonstrates - but I'm not
>> sure about it. I wonder if this might be penalizing some other types of
>> plans (essentially anything with limit + gather).
>>
>> Attached is a WIP patch fixing this by considering both startup and
>> total cost (by calling compare_path_costs_fuzzily).
>
>It seems to me that this is likely a bug, and not just a changed
>needed for this. Do you think it's better addressed in a separate
>thread? Or retain it as part of this patch for now (and possibly break
>it out later)? On the other hand, it's entirely possible that someone
>more familiar with parallel plan limitations could explain why the
>above comment holds true. That makes me lean towards asking in a new
>thread.
>

Maybe. I think creating a separate thread would be useful, provided we
manage to demonstrate the issue without an incremental sort.

>I've also attached a new base patch (incremental-sort-30.patch) which
>includes some of the other obvious fixes (costing, etc.) that you'd
>previously proposed.
>

Thanks!

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 

On Sat, Jul 20, 2019 at 11:25 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
...
> >My current line of investigation is whether we need to do anything in
> >the parallel portion of create_ordered_paths(). I noticed that the
> >first-pass patch adding generate_useful_gather_paths() modified that
> >section but wasn't actually adding any new gather-merge paths (just
> >bare incremental sort paths). That seems pretty clearly just a
> >prototype miss, so I modified the prototype to build gather-merge
> >paths instead (as a side note that change seems to fix an oddity I was
> >seeing where plans would include a parallel index scan node even
> >though they weren't parallel plans). While the resulting plan for
> >something like:
> >
>
> Yes, that seems to be a bug. The block above it clealy has a gather
> merge nodes, so this one should too.
>
> >explain analyze select * from t where t.a in (1,2,3,4,5,6) order by
> >t.a, t.b limit 50;
> >
> >changes cost (to be cheaper) ever so slightly with the gather-merge
> >addition to create_ordered_paths(), the plan itself is otherwise
> >identical (including row estimates):
> >
> >Limit
> >  -> Gather Merge
> >       -> Incremental Sort
> >          -> Parallel Index Scan
> >
> >(Note: I'm forcing parallel plans here with: set
> >max_parallel_workers_per_gather=4; set min_parallel_table_scan_size=0;
> >set parallel_tuple_cost=0; set parallel_setup_cost=0; set
> >min_parallel_index_scan_size=0;)
> >
> >I can't seem to come up with a case where adding these gather-merge
> >paths in create_ordered_paths() isn't entirely duplicative of paths
> >already created by generate_useful_gather_paths() as called from
> >apply_scanjoin_target_to_paths() -- which I _think_ makes sense given
> >that both apply_scanjoin_target_to_paths() and create_ordered_paths()
> >are called by grouping_planner().
> >
> >Can you think of a case I'm missing here that would make it valuable
> >to generate new parallel plans in create_ordered_paths()?
> >
>
> Good question. Not sure. I think such path would need to do something on
> a relation that is neither a join nor a scan - in which case the path
> should not be created by apply_scanjoin_target_to_paths().
>
> So for example a query like this:
>
>   SELECT
>       a, b, sum(expensive_function(c))
>   FROM
>       t
>   GROUP BY a,b
>   ORDER BY a,sum(...) LIMIT 10;
>
> should be able to produce a plan like this:
>
>   -> Limit
>       -> Gather Merge
>         -> Incremental Sort (pathkeys: a, sum)
>           -> Group Aggregate
>              a, b, sum(expensive_function(c))
>             -> Index Scan (pathkeys: a, b)
>
> or something like that, maybe. I haven't tried this, though. The other
> question is whether such queries are useful in practice ...

Hmm, when I step through on that example input_rel->partial_pathlist
!= NIL is false, so we don't even attempt to consider any extra
parallel paths in create_ordered_paths(). Nonetheless we get a
parallel plan, but with a different shape:

 Limit
   ->  Incremental Sort
         Sort Key: a, (sum(expensive_function(c)))
         Presorted Key: a
         ->  Finalize GroupAggregate
               Group Key: a, b
               ->  Gather Merge
                     Workers Planned: 4
                     ->  Partial GroupAggregate
                           Group Key: a, b
                           ->  Sort
                                 Sort Key: a, b
                                 ->  Parallel Seq Scan on t

(or if I disable seqscan then the sort+seq scan above becomes inc sort
+ index scan)

To be honest, I don't think I understand how you would get a plan like
that anyway since the index here only has "a" and so can't provide
(pathkeys: a, b).

Perhaps there's something I'm still missing though.

James Coleman

On Sat, Jul 20, 2019 at 12:12 PM James Coleman <jtc331@gmail.com> wrote:
>
> On Sat, Jul 20, 2019 at 11:25 AM Tomas Vondra
> <tomas.vondra@2ndquadrant.com> wrote:
> ...
> > >My current line of investigation is whether we need to do anything in
> > >the parallel portion of create_ordered_paths(). I noticed that the
> > >first-pass patch adding generate_useful_gather_paths() modified that
> > >section but wasn't actually adding any new gather-merge paths (just
> > >bare incremental sort paths). That seems pretty clearly just a
> > >prototype miss, so I modified the prototype to build gather-merge
> > >paths instead (as a side note that change seems to fix an oddity I was
> > >seeing where plans would include a parallel index scan node even
> > >though they weren't parallel plans). While the resulting plan for
> > >something like:
> > >
> >
> > Yes, that seems to be a bug. The block above it clealy has a gather
> > merge nodes, so this one should too.
> >
> > >explain analyze select * from t where t.a in (1,2,3,4,5,6) order by
> > >t.a, t.b limit 50;
> > >
> > >changes cost (to be cheaper) ever so slightly with the gather-merge
> > >addition to create_ordered_paths(), the plan itself is otherwise
> > >identical (including row estimates):
> > >
> > >Limit
> > >  -> Gather Merge
> > >       -> Incremental Sort
> > >          -> Parallel Index Scan
> > >
> > >(Note: I'm forcing parallel plans here with: set
> > >max_parallel_workers_per_gather=4; set min_parallel_table_scan_size=0;
> > >set parallel_tuple_cost=0; set parallel_setup_cost=0; set
> > >min_parallel_index_scan_size=0;)
> > >
> > >I can't seem to come up with a case where adding these gather-merge
> > >paths in create_ordered_paths() isn't entirely duplicative of paths
> > >already created by generate_useful_gather_paths() as called from
> > >apply_scanjoin_target_to_paths() -- which I _think_ makes sense given
> > >that both apply_scanjoin_target_to_paths() and create_ordered_paths()
> > >are called by grouping_planner().
> > >
> > >Can you think of a case I'm missing here that would make it valuable
> > >to generate new parallel plans in create_ordered_paths()?
> > >
> >
> > Good question. Not sure. I think such path would need to do something on
> > a relation that is neither a join nor a scan - in which case the path
> > should not be created by apply_scanjoin_target_to_paths().
> >
> > So for example a query like this:
> >
> >   SELECT
> >       a, b, sum(expensive_function(c))
> >   FROM
> >       t
> >   GROUP BY a,b
> >   ORDER BY a,sum(...) LIMIT 10;
> >
> > should be able to produce a plan like this:
> >
> >   -> Limit
> >       -> Gather Merge
> >         -> Incremental Sort (pathkeys: a, sum)
> >           -> Group Aggregate
> >              a, b, sum(expensive_function(c))
> >             -> Index Scan (pathkeys: a, b)
> >
> > or something like that, maybe. I haven't tried this, though. The other
> > question is whether such queries are useful in practice ...
>
> Hmm, when I step through on that example input_rel->partial_pathlist
> != NIL is false, so we don't even attempt to consider any extra
> parallel paths in create_ordered_paths(). Nonetheless we get a
> parallel plan, but with a different shape:
>
>  Limit
>    ->  Incremental Sort
>          Sort Key: a, (sum(expensive_function(c)))
>          Presorted Key: a
>          ->  Finalize GroupAggregate
>                Group Key: a, b
>                ->  Gather Merge
>                      Workers Planned: 4
>                      ->  Partial GroupAggregate
>                            Group Key: a, b
>                            ->  Sort
>                                  Sort Key: a, b
>                                  ->  Parallel Seq Scan on t
>
> (or if I disable seqscan then the sort+seq scan above becomes inc sort
> + index scan)
>
> To be honest, I don't think I understand how you would get a plan like
> that anyway since the index here only has "a" and so can't provide
> (pathkeys: a, b).
>
> Perhaps there's something I'm still missing though.

Also just realized I don't think (?) we can order by the sum inside a
gather-merge -- at least not without having another sort above the
parallel portion? Or is the group aggregate able to also provide
ordering on the final sum after aggregating the partial sums?

James Coleman

On Sat, Jul 20, 2019 at 12:21:01PM -0400, James Coleman wrote:
>On Sat, Jul 20, 2019 at 12:12 PM James Coleman <jtc331@gmail.com> wrote:
>>
>> On Sat, Jul 20, 2019 at 11:25 AM Tomas Vondra
>> <tomas.vondra@2ndquadrant.com> wrote:
>> ...
>> > >My current line of investigation is whether we need to do anything in
>> > >the parallel portion of create_ordered_paths(). I noticed that the
>> > >first-pass patch adding generate_useful_gather_paths() modified that
>> > >section but wasn't actually adding any new gather-merge paths (just
>> > >bare incremental sort paths). That seems pretty clearly just a
>> > >prototype miss, so I modified the prototype to build gather-merge
>> > >paths instead (as a side note that change seems to fix an oddity I was
>> > >seeing where plans would include a parallel index scan node even
>> > >though they weren't parallel plans). While the resulting plan for
>> > >something like:
>> > >
>> >
>> > Yes, that seems to be a bug. The block above it clealy has a gather
>> > merge nodes, so this one should too.
>> >
>> > >explain analyze select * from t where t.a in (1,2,3,4,5,6) order by
>> > >t.a, t.b limit 50;
>> > >
>> > >changes cost (to be cheaper) ever so slightly with the gather-merge
>> > >addition to create_ordered_paths(), the plan itself is otherwise
>> > >identical (including row estimates):
>> > >
>> > >Limit
>> > >  -> Gather Merge
>> > >       -> Incremental Sort
>> > >          -> Parallel Index Scan
>> > >
>> > >(Note: I'm forcing parallel plans here with: set
>> > >max_parallel_workers_per_gather=4; set min_parallel_table_scan_size=0;
>> > >set parallel_tuple_cost=0; set parallel_setup_cost=0; set
>> > >min_parallel_index_scan_size=0;)
>> > >
>> > >I can't seem to come up with a case where adding these gather-merge
>> > >paths in create_ordered_paths() isn't entirely duplicative of paths
>> > >already created by generate_useful_gather_paths() as called from
>> > >apply_scanjoin_target_to_paths() -- which I _think_ makes sense given
>> > >that both apply_scanjoin_target_to_paths() and create_ordered_paths()
>> > >are called by grouping_planner().
>> > >
>> > >Can you think of a case I'm missing here that would make it valuable
>> > >to generate new parallel plans in create_ordered_paths()?
>> > >
>> >
>> > Good question. Not sure. I think such path would need to do something on
>> > a relation that is neither a join nor a scan - in which case the path
>> > should not be created by apply_scanjoin_target_to_paths().
>> >
>> > So for example a query like this:
>> >
>> >   SELECT
>> >       a, b, sum(expensive_function(c))
>> >   FROM
>> >       t
>> >   GROUP BY a,b
>> >   ORDER BY a,sum(...) LIMIT 10;
>> >
>> > should be able to produce a plan like this:
>> >
>> >   -> Limit
>> >       -> Gather Merge
>> >         -> Incremental Sort (pathkeys: a, sum)
>> >           -> Group Aggregate
>> >              a, b, sum(expensive_function(c))
>> >             -> Index Scan (pathkeys: a, b)
>> >
>> > or something like that, maybe. I haven't tried this, though. The other
>> > question is whether such queries are useful in practice ...
>>
>> Hmm, when I step through on that example input_rel->partial_pathlist
>> != NIL is false, so we don't even attempt to consider any extra
>> parallel paths in create_ordered_paths(). Nonetheless we get a
>> parallel plan, but with a different shape:
>>
>>  Limit
>>    ->  Incremental Sort
>>          Sort Key: a, (sum(expensive_function(c)))
>>          Presorted Key: a
>>          ->  Finalize GroupAggregate
>>                Group Key: a, b
>>                ->  Gather Merge
>>                      Workers Planned: 4
>>                      ->  Partial GroupAggregate
>>                            Group Key: a, b
>>                            ->  Sort
>>                                  Sort Key: a, b
>>                                  ->  Parallel Seq Scan on t
>>
>> (or if I disable seqscan then the sort+seq scan above becomes inc sort
>> + index scan)
>>
>> To be honest, I don't think I understand how you would get a plan like
>> that anyway since the index here only has "a" and so can't provide
>> (pathkeys: a, b).
>>
>> Perhaps there's something I'm still missing though.

I wasn't stricly adhering to the example we used before, and I imagined
there would be an index on (a,b). Sorry if that wasn't clear.

>
>Also just realized I don't think (?) we can order by the sum inside a
>gather-merge -- at least not without having another sort above the
>parallel portion? Or is the group aggregate able to also provide
>ordering on the final sum after aggregating the partial sums?
>

Yes, you're right - an extra sort node would be necessary. But I don't
think that changes the idea behind that example. The question is whether
the extra sorts below the gather would be cheaper than doing a large sort
on top of it - but I don't see why wouldn't that be the case, and if we
only need a couple of rows from the beginning ...


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Sat, Jul 20, 2019 at 1:00 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Sat, Jul 20, 2019 at 12:21:01PM -0400, James Coleman wrote:
> >On Sat, Jul 20, 2019 at 12:12 PM James Coleman <jtc331@gmail.com> wrote:
> >>
> >> On Sat, Jul 20, 2019 at 11:25 AM Tomas Vondra
> >> <tomas.vondra@2ndquadrant.com> wrote:
> >> ...
> >> > >My current line of investigation is whether we need to do anything in
> >> > >the parallel portion of create_ordered_paths(). I noticed that the
> >> > >first-pass patch adding generate_useful_gather_paths() modified that
> >> > >section but wasn't actually adding any new gather-merge paths (just
> >> > >bare incremental sort paths). That seems pretty clearly just a
> >> > >prototype miss, so I modified the prototype to build gather-merge
> >> > >paths instead (as a side note that change seems to fix an oddity I was
> >> > >seeing where plans would include a parallel index scan node even
> >> > >though they weren't parallel plans). While the resulting plan for
> >> > >something like:
> >> > >
> >> >
> >> > Yes, that seems to be a bug. The block above it clealy has a gather
> >> > merge nodes, so this one should too.
> >> >
> >> > >explain analyze select * from t where t.a in (1,2,3,4,5,6) order by
> >> > >t.a, t.b limit 50;
> >> > >
> >> > >changes cost (to be cheaper) ever so slightly with the gather-merge
> >> > >addition to create_ordered_paths(), the plan itself is otherwise
> >> > >identical (including row estimates):
> >> > >
> >> > >Limit
> >> > >  -> Gather Merge
> >> > >       -> Incremental Sort
> >> > >          -> Parallel Index Scan
> >> > >
> >> > >(Note: I'm forcing parallel plans here with: set
> >> > >max_parallel_workers_per_gather=4; set min_parallel_table_scan_size=0;
> >> > >set parallel_tuple_cost=0; set parallel_setup_cost=0; set
> >> > >min_parallel_index_scan_size=0;)
> >> > >
> >> > >I can't seem to come up with a case where adding these gather-merge
> >> > >paths in create_ordered_paths() isn't entirely duplicative of paths
> >> > >already created by generate_useful_gather_paths() as called from
> >> > >apply_scanjoin_target_to_paths() -- which I _think_ makes sense given
> >> > >that both apply_scanjoin_target_to_paths() and create_ordered_paths()
> >> > >are called by grouping_planner().
> >> > >
> >> > >Can you think of a case I'm missing here that would make it valuable
> >> > >to generate new parallel plans in create_ordered_paths()?
> >> > >
> >> >
> >> > Good question. Not sure. I think such path would need to do something on
> >> > a relation that is neither a join nor a scan - in which case the path
> >> > should not be created by apply_scanjoin_target_to_paths().
> >> >
> >> > So for example a query like this:
> >> >
> >> >   SELECT
> >> >       a, b, sum(expensive_function(c))
> >> >   FROM
> >> >       t
> >> >   GROUP BY a,b
> >> >   ORDER BY a,sum(...) LIMIT 10;
> >> >
> >> > should be able to produce a plan like this:
> >> >
> >> >   -> Limit
> >> >       -> Gather Merge
> >> >         -> Incremental Sort (pathkeys: a, sum)
> >> >           -> Group Aggregate
> >> >              a, b, sum(expensive_function(c))
> >> >             -> Index Scan (pathkeys: a, b)
> >> >
> >> > or something like that, maybe. I haven't tried this, though. The other
> >> > question is whether such queries are useful in practice ...
> >>
> >> Hmm, when I step through on that example input_rel->partial_pathlist
> >> != NIL is false, so we don't even attempt to consider any extra
> >> parallel paths in create_ordered_paths(). Nonetheless we get a
> >> parallel plan, but with a different shape:
> >>
> >>  Limit
> >>    ->  Incremental Sort
> >>          Sort Key: a, (sum(expensive_function(c)))
> >>          Presorted Key: a
> >>          ->  Finalize GroupAggregate
> >>                Group Key: a, b
> >>                ->  Gather Merge
> >>                      Workers Planned: 4
> >>                      ->  Partial GroupAggregate
> >>                            Group Key: a, b
> >>                            ->  Sort
> >>                                  Sort Key: a, b
> >>                                  ->  Parallel Seq Scan on t
> >>
> >> (or if I disable seqscan then the sort+seq scan above becomes inc sort
> >> + index scan)
> >>
> >> To be honest, I don't think I understand how you would get a plan like
> >> that anyway since the index here only has "a" and so can't provide
> >> (pathkeys: a, b).
> >>
> >> Perhaps there's something I'm still missing though.
>
> I wasn't stricly adhering to the example we used before, and I imagined
> there would be an index on (a,b). Sorry if that wasn't clear.
>
> >
> >Also just realized I don't think (?) we can order by the sum inside a
> >gather-merge -- at least not without having another sort above the
> >parallel portion? Or is the group aggregate able to also provide
> >ordering on the final sum after aggregating the partial sums?
> >
>
> Yes, you're right - an extra sort node would be necessary. But I don't
> think that changes the idea behind that example. The question is whether
> the extra sorts below the gather would be cheaper than doing a large sort
> on top of it - but I don't see why wouldn't that be the case, and if we
> only need a couple of rows from the beginning ...

Ah, I see. Right now we get:

 Limit
   ->  Incremental Sort
         Sort Key: a, (sum(expensive_function(c)))
         Presorted Key: a
         ->  Finalize GroupAggregate
               Group Key: a, b
               ->  Gather Merge
                     Workers Planned: 2
                     ->  Partial GroupAggregate
                           Group Key: a, b
                           ->  Parallel Index Scan using t_a_b on t

even with the parallel additions to create_ordered_paths() -- that
addition doesn't actually add any new parallel paths because
input_rel->partial_pathlist != NIL is false (I'm not sure why yet), so
if we want (if I understand correctly) something more like:

I'm still struggling to understand though how another incremental sort
below the gather-merge would actually be able to help us. For one I'm
not sure it would be less expensive, but more importantly I'm not sure
how we could do that and maintain correctness. Wouldn't a per-worker
sum not actually be useful in sorting since it has no predictable
impact on the ordering of the total sum?

Describing that got me thinking of similar cases where ordering of the
partial aggregate would (in theory) be a correct partial sort for the
total ordering, and it seems like min() and max() would be. So I ran
the same experiment with that instead of sum(), but, you guessed it,
input_rel->partial_pathlist != NIL is false again, so we don't add any
parallel paths in create_ordered_paths().

I'm leaning towards thinking considering parallel incremental sorts in
create_ordered_paths() won't add value. But I also feel like this
whole project has me jumping into the deep end of the pool again (this
time the planner), so I'm still picking up a lot of pieces for how all
this fits together, and as such I don't have a great intuitive grasp
yet of how this particular part of the planning process maps to the
kind of queries and plans we consider. All that to say: if you have
further thoughts, I'm happy to look into it, but right now I'm not
seeing anything.

James Coleman

On Sat, Jul 20, 2019 at 07:37:08PM -0400, James Coleman wrote:
>> ..
>>
>> Yes, you're right - an extra sort node would be necessary. But I don't
>> think that changes the idea behind that example. The question is whether
>> the extra sorts below the gather would be cheaper than doing a large sort
>> on top of it - but I don't see why wouldn't that be the case, and if we
>> only need a couple of rows from the beginning ...
>
>Ah, I see. Right now we get:
>
> Limit
>   ->  Incremental Sort
>         Sort Key: a, (sum(expensive_function(c)))
>         Presorted Key: a
>         ->  Finalize GroupAggregate
>               Group Key: a, b
>               ->  Gather Merge
>                     Workers Planned: 2
>                     ->  Partial GroupAggregate
>                           Group Key: a, b
>                           ->  Parallel Index Scan using t_a_b on t
>
>even with the parallel additions to create_ordered_paths() -- that
>addition doesn't actually add any new parallel paths because
>input_rel->partial_pathlist != NIL is false (I'm not sure why yet), so
>if we want (if I understand correctly) something more like:
>

Well, in this particular case it's fairly simple, I think. We only call
the create_ordered_paths() from the grouping planner, on the upper
relation that represents the result of the GROUP BY. But that means it
has to see only the finalized result (after Finalize GroupAggregate). So
it can't see partial aggregation or any other partial path. So in this
case it seems guaranteed (partial_pathlist == NIL).

So maybe we should not be looking at GROUP BY queries, which probably
can't hit this particular code path at all - we need a different type of
upper relation. For example UNION ALL should hit this code, I think.

So maybe try

   select * from t union all select * from t order by a, b limit 10;

and that will hit this condition with partial_pathlist != NIL.

I don't know if such queries can benefit from incremental sort, though.
There are other upper relations too:

  typedef enum UpperRelationKind
  {
      UPPERREL_SETOP,             /* result of UNION/INTERSECT/EXCEPT, if any */
      UPPERREL_PARTIAL_GROUP_AGG, /* result of partial grouping/aggregation, if
                                   * any */
      UPPERREL_GROUP_AGG,         /* result of grouping/aggregation, if any */
      UPPERREL_WINDOW,            /* result of window functions, if any */
      UPPERREL_DISTINCT,          /* result of "SELECT DISTINCT", if any */
      UPPERREL_ORDERED,           /* result of ORDER BY, if any */
      UPPERREL_FINAL              /* result of any remaining top-level actions */
      /* NB: UPPERREL_FINAL must be last enum entry; it's used to size arrays */
  } UpperRelationKind;


>I'm still struggling to understand though how another incremental sort
>below the gather-merge would actually be able to help us. For one I'm
>not sure it would be less expensive,

That's a good question. I think in general we agree that if we can get
the gather merge to sort the data the way the operation above the gather
merge (which is the first operation that can't operate in parallel
mode), that's probably beneficial.

So this pattern seems reasonable:

    -> something
       -> non-parallel operation
          -> gather merge
             -> incremental sort
                -> something

And it's likely faster, especially when the parts above this can
leverage the lower startup cost. Say, when there's an explicit LIMIT.

I think the question is where exactly do we add the incremental sort.
It's quite possible some of the places we modified are redundant, at
least for some queries. Not sure.

>but more importantly I'm not sure how we could do that and maintain
>correctness. Wouldn't a per-worker sum not actually be useful in
>sorting since it has no predictable impact on the ordering of the total
>sum?

Yes, you're right - that wouldn't be correct. The reason why I've been
thinking about such examples because distributed databases do make such
things in some cases, and we might do that too with partitioning.

Consider a simple example

    create table t (a int, b int, c int) partition by hash (a);
    create table t0 partition of t for values with (modulus 4, remainder 0);
    create table t1 partition of t for values with (modulus 4, remainder 1);
    create table t2 partition of t for values with (modulus 4, remainder 2);
    create table t3 partition of t for values with (modulus 4, remainder 3);
    insert into t select mod(i,1000), i, i from generate_series(1,1000000) s(i);
    analyze t;
    select a, count(b) from t group by a order by a, count(b) limit 10;

In this case we might do a plan similar to what I proposed, assuming
each worker gets to execute on a different partition (because then we
know each worker will see distinct groups, thanks to the partitioning).

But AFAIK we don't do such optimizations yet, so it's probably just a
distraction.

>Describing that got me thinking of similar cases where ordering of the
>partial aggregate would (in theory) be a correct partial sort for the
>total ordering, and it seems like min() and max() would be. So I ran
>the same experiment with that instead of sum(), but, you guessed it,
>input_rel->partial_pathlist != NIL is false again, so we don't add any
>parallel paths in create_ordered_paths().
>

Right. That's because with aggregation, grouping planner only sees the
total result, not the partial paths. We need different upper rel to
exercise that code path.

>I'm leaning towards thinking considering parallel incremental sorts in
>create_ordered_paths() won't add value. But I also feel like this
>whole project has me jumping into the deep end of the pool again (this
>time the planner), so I'm still picking up a lot of pieces for how all
>this fits together, and as such I don't have a great intuitive grasp
>yet of how this particular part of the planning process maps to the
>kind of queries and plans we consider. All that to say: if you have
>further thoughts, I'm happy to look into it, but right now I'm not
>seeing anything.
>

Understood. FWIW I'm not particularly familiar with this code (or which
places are supposed to work together), so I definitely agree it may be
overwhelming. Especially when it's only a part of a larger patch.

I wonder if we're approaching this wrong. Maybe we should not reverse
engineer queries for the various places, but just start with a set of
queries that we want to optimize, and then identify which places in the
planner need to be modified.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 

On Sun, Jul 21, 2019 at 01:34:22PM +0200, Tomas Vondra wrote:
>
> ...
>
>I wonder if we're approaching this wrong. Maybe we should not reverse
>engineer queries for the various places, but just start with a set of
>queries that we want to optimize, and then identify which places in the
>planner need to be modified.
>

I've decided to do a couple of experiments, trying to make my mind about
which modified places matter to diffrent queries. But instead of trying
to reverse engineer the queries, I've taken a different approach - I've
compiled a list of queries that I think are sensible and relevant, and
then planned them with incremental sort enabled in different places.

I don't have any clear conclusions at this point - it does show some of
the places don't change plan for any of the queries, although there may
be some additional query where it'd make a difference.

But I'm posting this mostly because it might be useful. I've initially
planned to move changes that add incremental sort paths to separate
patches, and then apply/skip different subsets of those patches. But
then I realized there's a better way to do this - I've added a bunch of
GUCs, one for each such place. This allows doing this testing without
having to rebuild repeatedly.

I'm not going to post the patch(es) with extra GUCs here, because it'd
just confuse the patch tester, but it's available here:

  https://github.com/tvondra/postgres/tree/incremental-sort-20190730

There are 10 GUCs, one for each place in planner where incremental sort
paths are constructed. By default all those are set to 'false' so no
incremental sort paths are built. If you do

  SET devel_create_ordered_paths = on;

it'll start creating the paths in non-parallel in create_ordered_paths.
Then you may enable devel_create_ordered_paths_parallel to also consider
parallel paths, etc.

The list of queries (synthetic, but hopefully sufficiently realistic)
and a couple of scripts to collect the plans is in this repository:

  https://github.com/tvondra/incremental-sort-tests-2

There's also a spreadsheet with a summary of results, with a visual
representation of which GUCs affect which queries.



regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 

On Wed, Sep 04, 2019 at 11:37:48AM +0200, Rafia Sabih wrote:
>On Tue, 30 Jul 2019 at 02:17, Tomas Vondra <tomas.vondra@2ndquadrant.com>
>wrote:
>
>> On Sun, Jul 21, 2019 at 01:34:22PM +0200, Tomas Vondra wrote:
>> >
>> > ...
>> >
>> >I wonder if we're approaching this wrong. Maybe we should not reverse
>> >engineer queries for the various places, but just start with a set of
>> >queries that we want to optimize, and then identify which places in the
>> >planner need to be modified.
>> >
>>
>> I've decided to do a couple of experiments, trying to make my mind about
>> which modified places matter to diffrent queries. But instead of trying
>> to reverse engineer the queries, I've taken a different approach - I've
>> compiled a list of queries that I think are sensible and relevant, and
>> then planned them with incremental sort enabled in different places.
>>
>> I don't have any clear conclusions at this point - it does show some of
>> the places don't change plan for any of the queries, although there may
>> be some additional query where it'd make a difference.
>>
>> But I'm posting this mostly because it might be useful. I've initially
>> planned to move changes that add incremental sort paths to separate
>> patches, and then apply/skip different subsets of those patches. But
>> then I realized there's a better way to do this - I've added a bunch of
>> GUCs, one for each such place. This allows doing this testing without
>> having to rebuild repeatedly.
>>
>> I'm not going to post the patch(es) with extra GUCs here, because it'd
>> just confuse the patch tester, but it's available here:
>>
>>   https://github.com/tvondra/postgres/tree/incremental-sort-20190730
>>
>> There are 10 GUCs, one for each place in planner where incremental sort
>> paths are constructed. By default all those are set to 'false' so no
>> incremental sort paths are built. If you do
>>
>>   SET devel_create_ordered_paths = on;
>>
>> it'll start creating the paths in non-parallel in create_ordered_paths.
>> Then you may enable devel_create_ordered_paths_parallel to also consider
>> parallel paths, etc.
>>
>> The list of queries (synthetic, but hopefully sufficiently realistic)
>> and a couple of scripts to collect the plans is in this repository:
>>
>>   https://github.com/tvondra/incremental-sort-tests-2
>>
>> There's also a spreadsheet with a summary of results, with a visual
>> representation of which GUCs affect which queries.
>>
> Wow, that sounds like an elaborate experiment. But where is this
> spreadsheet you mentioned ?
>

It seems I forgot to push the commit containing the spreadsheet with
results. I'll fix that tomorrow.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Wed, Sep 04, 2019 at 09:17:10PM +0200, Tomas Vondra wrote:
>On Wed, Sep 04, 2019 at 11:37:48AM +0200, Rafia Sabih wrote:
>>On Tue, 30 Jul 2019 at 02:17, Tomas Vondra <tomas.vondra@2ndquadrant.com>
>>wrote:
>>
>>>On Sun, Jul 21, 2019 at 01:34:22PM +0200, Tomas Vondra wrote:
>>>>
>>>> ...
>>>>
>>>>I wonder if we're approaching this wrong. Maybe we should not reverse
>>>>engineer queries for the various places, but just start with a set of
>>>>queries that we want to optimize, and then identify which places in the
>>>>planner need to be modified.
>>>>
>>>
>>>I've decided to do a couple of experiments, trying to make my mind about
>>>which modified places matter to diffrent queries. But instead of trying
>>>to reverse engineer the queries, I've taken a different approach - I've
>>>compiled a list of queries that I think are sensible and relevant, and
>>>then planned them with incremental sort enabled in different places.
>>>
>>>I don't have any clear conclusions at this point - it does show some of
>>>the places don't change plan for any of the queries, although there may
>>>be some additional query where it'd make a difference.
>>>
>>>But I'm posting this mostly because it might be useful. I've initially
>>>planned to move changes that add incremental sort paths to separate
>>>patches, and then apply/skip different subsets of those patches. But
>>>then I realized there's a better way to do this - I've added a bunch of
>>>GUCs, one for each such place. This allows doing this testing without
>>>having to rebuild repeatedly.
>>>
>>>I'm not going to post the patch(es) with extra GUCs here, because it'd
>>>just confuse the patch tester, but it's available here:
>>>
>>>  https://github.com/tvondra/postgres/tree/incremental-sort-20190730
>>>
>>>There are 10 GUCs, one for each place in planner where incremental sort
>>>paths are constructed. By default all those are set to 'false' so no
>>>incremental sort paths are built. If you do
>>>
>>>  SET devel_create_ordered_paths = on;
>>>
>>>it'll start creating the paths in non-parallel in create_ordered_paths.
>>>Then you may enable devel_create_ordered_paths_parallel to also consider
>>>parallel paths, etc.
>>>
>>>The list of queries (synthetic, but hopefully sufficiently realistic)
>>>and a couple of scripts to collect the plans is in this repository:
>>>
>>>  https://github.com/tvondra/incremental-sort-tests-2
>>>
>>>There's also a spreadsheet with a summary of results, with a visual
>>>representation of which GUCs affect which queries.
>>>
>>Wow, that sounds like an elaborate experiment. But where is this
>>spreadsheet you mentioned ?
>>
>
>It seems I forgot to push the commit containing the spreadsheet with
>results. I'll fix that tomorrow.
>

OK, I've pushed the commit with the spreadsheet. The single sheet lists
the synthetic queries, and hashes of plans with different flags enables
(parallel query, force incremental sort, and the new developer GUCs
mentioned before). Only a single developer flag is set to true (or none
of them).

The columns at the end simply say whether the plan differs from the plan
generated by master (no patches). TRUE means "same as master" while
FALSE means "different plan.

The "patched" column means all developer GUCs disabled, so it's expected
to produce the same plan as master (and it is). And then there's one
column for each developer GUC. If the column is just TRUE it means the
GUC does not affect any of the synthetic queries. There are 4 of them:

- devel_add_paths_to_grouping_rel_parallel
- devel_create_partial_grouping_paths
- devel_gather_grouping_paths
- devel_standard_join_search

The places controlled by those GUCs are either useless, or the query
affected by them is not included in the list of queries.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 

On 2019-Jul-30, Tomas Vondra wrote:

> On Sun, Jul 21, 2019 at 01:34:22PM +0200, Tomas Vondra wrote:
> > 
> > I wonder if we're approaching this wrong. Maybe we should not reverse
> > engineer queries for the various places, but just start with a set of
> > queries that we want to optimize, and then identify which places in the
> > planner need to be modified.

[...]

> I've decided to do a couple of experiments, trying to make my mind about
> which modified places matter to diffrent queries. But instead of trying
> to reverse engineer the queries, I've taken a different approach - I've
> compiled a list of queries that I think are sensible and relevant, and
> then planned them with incremental sort enabled in different places.

[...]

> The list of queries (synthetic, but hopefully sufficiently realistic)
> and a couple of scripts to collect the plans is in this repository:
> 
>  https://github.com/tvondra/incremental-sort-tests-2
> 
> There's also a spreadsheet with a summary of results, with a visual
> representation of which GUCs affect which queries.

OK, so we have that now.  I suppose this spreadsheet now tells us which
places are useful and which aren't, at least for the queries that you've
tested.  Dowe that mean that we want to get the patch to consider adding
paths only the places that your spreadsheet says are useful?  I'm not
sure what the next steps are for this patch.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

> OK, so we have that now.  I suppose this spreadsheet now tells us which
> places are useful and which aren't, at least for the queries that you've
> tested.  Dowe that mean that we want to get the patch to consider adding
> paths only the places that your spreadsheet says are useful?  I'm not
> sure what the next steps are for this patch.

I wanted to note here that I haven't abandoned this patch, but ended
up needing to use my extra time for working on a conference talk. That
talk is today, so I'm hoping to be able to catch up on this again
soon.

James Coleman

On Thu, Sep 12, 2019 at 12:49:29PM -0300, Alvaro Herrera wrote:
>On 2019-Jul-30, Tomas Vondra wrote:
>
>> On Sun, Jul 21, 2019 at 01:34:22PM +0200, Tomas Vondra wrote:
>> >
>> > I wonder if we're approaching this wrong. Maybe we should not reverse
>> > engineer queries for the various places, but just start with a set of
>> > queries that we want to optimize, and then identify which places in the
>> > planner need to be modified.
>
>[...]
>
>> I've decided to do a couple of experiments, trying to make my mind about
>> which modified places matter to diffrent queries. But instead of trying
>> to reverse engineer the queries, I've taken a different approach - I've
>> compiled a list of queries that I think are sensible and relevant, and
>> then planned them with incremental sort enabled in different places.
>
>[...]
>
>> The list of queries (synthetic, but hopefully sufficiently realistic)
>> and a couple of scripts to collect the plans is in this repository:
>>
>>  https://github.com/tvondra/incremental-sort-tests-2
>>
>> There's also a spreadsheet with a summary of results, with a visual
>> representation of which GUCs affect which queries.
>
>OK, so we have that now.  I suppose this spreadsheet now tells us which
>places are useful and which aren't, at least for the queries that you've
>tested.  Dowe that mean that we want to get the patch to consider adding
>paths only the places that your spreadsheet says are useful?  I'm not
>sure what the next steps are for this patch.
>

Yes. I think the spreadsheet call help us with answering two things:

1) places actually affecting the plan (all but three do)

2) redundant places (there are some cases where two GUCs produce the
same plan in the end)

Of course, this does assume the query set makes sense and is somewhat
realistic, but I've tried to construct queries where that is true. We
may extend it over time, of course.

I think we've agreed to add incremental sort paths different places in
separate patches, to make review easier. So this may be a useful way to
decide which places to address first. I'd probably do it in this order:

- create_ordered_paths
- create_ordered_paths (parallel part)
- add_paths_to_grouping_rel
- ... not sure ...

but that's just a proposal. It'd give us most of the benefits, I think,
and we could also focus on the rest of the patch.

Also, regarding the three GUCs that don't affect any of the queries, we
can't really add them as we wouldn't be able to test them. If we manage
to construct a query that'd benefit from them, we can revisit this.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 

On Thu, Sep 12, 2019 at 11:54:06AM -0400, James Coleman wrote:
>> OK, so we have that now.  I suppose this spreadsheet now tells us which
>> places are useful and which aren't, at least for the queries that you've
>> tested.  Dowe that mean that we want to get the patch to consider adding
>> paths only the places that your spreadsheet says are useful?  I'm not
>> sure what the next steps are for this patch.
>
>I wanted to note here that I haven't abandoned this patch, but ended
>up needing to use my extra time for working on a conference talk. That
>talk is today, so I'm hoping to be able to catch up on this again
>soon.
>

Good! I'm certainly looking forward to a new patch version.

As discussed in the past, this patch is pretty sensitive (large, touches
planning, ...), so we should try getting most of it in not too late in
the cycle. For example 2019-11 would be nice.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 

On Fri, Sep 13, 2019 at 10:54 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Thu, Sep 12, 2019 at 11:54:06AM -0400, James Coleman wrote:
> >> OK, so we have that now.  I suppose this spreadsheet now tells us which
> >> places are useful and which aren't, at least for the queries that you've
> >> tested.  Dowe that mean that we want to get the patch to consider adding
> >> paths only the places that your spreadsheet says are useful?  I'm not
> >> sure what the next steps are for this patch.
> >
> >I wanted to note here that I haven't abandoned this patch, but ended
> >up needing to use my extra time for working on a conference talk. That
> >talk is today, so I'm hoping to be able to catch up on this again
> >soon.
> >
>
> Good! I'm certainly looking forward to a new patch version.
>
> As discussed in the past, this patch is pretty sensitive (large, touches
> planning, ...), so we should try getting most of it in not too late in
> the cycle. For example 2019-11 would be nice.

Completely agree; originally I'd hoped to have it in rough draft
finished form to get serious review in the September CF...but...

On Sat, Jul 20, 2019 at 11:25 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
> >> ...
> >>
> >> I think this may be a thinko, as this plan demonstrates - but I'm not
> >> sure about it. I wonder if this might be penalizing some other types of
> >> plans (essentially anything with limit + gather).
> >>
> >> Attached is a WIP patch fixing this by considering both startup and
> >> total cost (by calling compare_path_costs_fuzzily).
> >
> >It seems to me that this is likely a bug, and not just a changed
> >needed for this. Do you think it's better addressed in a separate
> >thread? Or retain it as part of this patch for now (and possibly break
> >it out later)? On the other hand, it's entirely possible that someone
> >more familiar with parallel plan limitations could explain why the
> >above comment holds true. That makes me lean towards asking in a new
> >thread.
> >
>
> Maybe. I think creating a separate thread would be useful, provided we
> manage to demonstrate the issue without an incremental sort.

I did some more thinking about this, and I can't currently come up
with a way to reproduce this issue outside of this patch. It doesn't
seem reasonable to me to assume that there's anything inherent about
this patch that means it's the only way we can end up with a partial
path with a low startup cost we'd want to prefer.

Part of me wants to pull it over to a separate thread just to get
additional feedback, but I'm not sure how useful that is given we
don't currently have an example case outside of this patch.

One thing to note though: the current patch does not also modify
add_partial_path_precheck which also does not take into account
startup cost, so we probably need to update that for completeness's
sake.

James Coleman

On Sun, Sep 15, 2019 at 09:33:33PM -0400, James Coleman wrote:
>On Sat, Jul 20, 2019 at 11:25 AM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>> >> ...
>> >>
>> >> I think this may be a thinko, as this plan demonstrates - but I'm not
>> >> sure about it. I wonder if this might be penalizing some other types of
>> >> plans (essentially anything with limit + gather).
>> >>
>> >> Attached is a WIP patch fixing this by considering both startup and
>> >> total cost (by calling compare_path_costs_fuzzily).
>> >
>> >It seems to me that this is likely a bug, and not just a changed
>> >needed for this. Do you think it's better addressed in a separate
>> >thread? Or retain it as part of this patch for now (and possibly break
>> >it out later)? On the other hand, it's entirely possible that someone
>> >more familiar with parallel plan limitations could explain why the
>> >above comment holds true. That makes me lean towards asking in a new
>> >thread.
>> >
>>
>> Maybe. I think creating a separate thread would be useful, provided we
>> manage to demonstrate the issue without an incremental sort.
>
>I did some more thinking about this, and I can't currently come up
>with a way to reproduce this issue outside of this patch. It doesn't
>seem reasonable to me to assume that there's anything inherent about
>this patch that means it's the only way we can end up with a partial
>path with a low startup cost we'd want to prefer.
>
>Part of me wants to pull it over to a separate thread just to get
>additional feedback, but I'm not sure how useful that is given we
>don't currently have an example case outside of this patch.
>

Hmm, I see.

While I initially suggested to start a separate thread only if we have
example not involving an incremental sort, that's probably not a hard
requirement. I think it's fine to start a thead briefly explaining the
issue, and pointing to incremental sort thread for actual example.

>
>One thing to note though: the current patch does not also modify
>add_partial_path_precheck which also does not take into account
>startup cost, so we probably need to update that for completeness's
>sake.
>

Good point. It does indeed seem to make the same assumption about only
comparing total cost before calling add_path_precheck.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 

On Mon, Sep 9, 2019 at 5:55 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
> The "patched" column means all developer GUCs disabled, so it's expected
> to produce the same plan as master (and it is). And then there's one
> column for each developer GUC. If the column is just TRUE it means the
> GUC does not affect any of the synthetic queries. There are 4 of them:
>
> - devel_add_paths_to_grouping_rel_parallel
> - devel_create_partial_grouping_paths
> - devel_gather_grouping_paths
> - devel_standard_join_search
>
> The places controlled by those GUCs are either useless, or the query
> affected by them is not included in the list of queries.

I'd previously found (in my reverse engineering efforts) the query:

select *
from tenk1 t1
join tenk1 t2 on t1.hundred = t2.hundred
join tenk1 t3 on t1.hundred = t3.hundred
order by t1.hundred, t1.twenty
limit 50;

can change plans to use incremental sort when
generate_useful_gather_paths() is added to standard_join_search().
Specifically, we get a merge join between t1 and t3 as the top level
(besides limit) node where the driving side of the join is a gather
merge with incremental sort. This does rely on these gucs set in the
test harness:

set local max_parallel_workers_per_gather=4;
set local min_parallel_table_scan_size=0;
set local parallel_tuple_cost=0;
set local parallel_setup_cost=0;

So I think we can reduce the number of unused gucs to 3.

James

On Mon, Sep 16, 2019 at 6:32 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Sun, Sep 15, 2019 at 09:33:33PM -0400, James Coleman wrote:
> >On Sat, Jul 20, 2019 at 11:25 AM Tomas Vondra
> ><tomas.vondra@2ndquadrant.com> wrote:
> >> >> ...
> >> >>
> >> >> I think this may be a thinko, as this plan demonstrates - but I'm not
> >> >> sure about it. I wonder if this might be penalizing some other types of
> >> >> plans (essentially anything with limit + gather).
> >> >>
> >> >> Attached is a WIP patch fixing this by considering both startup and
> >> >> total cost (by calling compare_path_costs_fuzzily).
> >> >
> >> >It seems to me that this is likely a bug, and not just a changed
> >> >needed for this. Do you think it's better addressed in a separate
> >> >thread? Or retain it as part of this patch for now (and possibly break
> >> >it out later)? On the other hand, it's entirely possible that someone
> >> >more familiar with parallel plan limitations could explain why the
> >> >above comment holds true. That makes me lean towards asking in a new
> >> >thread.
> >> >
> >>
> >> Maybe. I think creating a separate thread would be useful, provided we
> >> manage to demonstrate the issue without an incremental sort.
> >
> >I did some more thinking about this, and I can't currently come up
> >with a way to reproduce this issue outside of this patch. It doesn't
> >seem reasonable to me to assume that there's anything inherent about
> >this patch that means it's the only way we can end up with a partial
> >path with a low startup cost we'd want to prefer.
> >
> >Part of me wants to pull it over to a separate thread just to get
> >additional feedback, but I'm not sure how useful that is given we
> >don't currently have an example case outside of this patch.
> >
>
> Hmm, I see.
>
> While I initially suggested to start a separate thread only if we have
> example not involving an incremental sort, that's probably not a hard
> requirement. I think it's fine to start a thead briefly explaining the
> issue, and pointing to incremental sort thread for actual example.
>
> >
> >One thing to note though: the current patch does not also modify
> >add_partial_path_precheck which also does not take into account
> >startup cost, so we probably need to update that for completeness's
> >sake.
> >
>
> Good point. It does indeed seem to make the same assumption about only
> comparing total cost before calling add_path_precheck.

I've started a new thread to discuss:
https://www.postgresql.org/message-id/CAAaqYe-5HmM4ih6FWp2RNV9rruunfrFrLhqFXF_nrrNCPy1Zhg%40mail.gmail.com

James

On Fri, Sep 13, 2019 at 10:51 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Thu, Sep 12, 2019 at 12:49:29PM -0300, Alvaro Herrera wrote:
> >On 2019-Jul-30, Tomas Vondra wrote:
> >> I've decided to do a couple of experiments, trying to make my mind about
> >> which modified places matter to diffrent queries. But instead of trying
> >> to reverse engineer the queries, I've taken a different approach - I've
> >> compiled a list of queries that I think are sensible and relevant, and
> >> then planned them with incremental sort enabled in different places.
> >
> >[...]
> >
> >> The list of queries (synthetic, but hopefully sufficiently realistic)
> >> and a couple of scripts to collect the plans is in this repository:
> >>
> >>  https://github.com/tvondra/incremental-sort-tests-2
> >>
> >> There's also a spreadsheet with a summary of results, with a visual
> >> representation of which GUCs affect which queries.
> >
> >OK, so we have that now.  I suppose this spreadsheet now tells us which
> >places are useful and which aren't, at least for the queries that you've
> >tested.  Dowe that mean that we want to get the patch to consider adding
> >paths only the places that your spreadsheet says are useful?  I'm not
> >sure what the next steps are for this patch.
> >
>
> Yes. I think the spreadsheet call help us with answering two things:
>
> 1) places actually affecting the plan (all but three do)
>
> 2) redundant places (there are some cases where two GUCs produce the
> same plan in the end)

To expand on this further, (1) should probably help us to be able to
write test cases.

Additionally, one big thing we still need that's somewhat external to
the patch is a good way to benchmark/a set of queries that we believe
are representative enough to be good benchmarks.

I'd really appreciate some input from you all on that particular
question; I feel like it's in some sense the biggest barrier to
getting the patch merged, but also the part where long experience in
the community/exposure to other use cases will probably be quite
valuable.

> Of course, this does assume the query set makes sense and is somewhat
> realistic, but I've tried to construct queries where that is true. We
> may extend it over time, of course.
>
> I think we've agreed to add incremental sort paths different places in
> separate patches, to make review easier. So this may be a useful way to
> decide which places to address first. I'd probably do it in this order:
>
> - create_ordered_paths
> - create_ordered_paths (parallel part)
> - add_paths_to_grouping_rel
> - ... not sure ...
>
> but that's just a proposal. It'd give us most of the benefits, I think,
> and we could also focus on the rest of the patch.

Certainly the first two seem like pretty obvious most necessary base
cases. I think supporting group bys also seems like a pretty standard
case, so at first glance I'd say this seems like a reasonable course
to me.

I'm going to start breaking up the patches in this thread into a
series in support of that. Since I've started a new thread with the
add_partial_path change, I'll include that patch here as part of this
series also. Do you think it's worth moving the tuplesort changes into
a standalone patch in the series also?

Attached is a rebased v31 now broken into the following:

- 001-consider-startup-cost-in-add-partial-path_v1.patch: From the
other thread (Tomas's patch unmodified)
- 002-incremental-sort_v31.patch: Updated base incremental sort patch

Besides rebasing, I've changed the enable_incrementalsort GUC to
prevent generating paths entirely rather than being cost-based, since
incremental sort is never absolutely necessary in the way regular sort
is.

I'm hoping to add 003 soon with the initial parallel parts, but I'm
about out of time right now and wanted to get something out, so
sending this without that.

Side question: for the patch tester do I have to attach each part of
the series each time even if nothing's changed in several of them? And
does the vN number at the end need to stay the same for all of them?
My attachments to this email don't follow that... Also, since this
email changes patch naming, so I need to do anything to clear out the
old ones? (I suppose if not, then that would imply an answer to the
first question also.)

James

On Fri, Sep 27, 2019 at 08:31:30PM -0400, James Coleman wrote:
>On Fri, Sep 13, 2019 at 10:51 AM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>>
>> On Thu, Sep 12, 2019 at 12:49:29PM -0300, Alvaro Herrera wrote:
>> >On 2019-Jul-30, Tomas Vondra wrote:
>> >> I've decided to do a couple of experiments, trying to make my mind about
>> >> which modified places matter to diffrent queries. But instead of trying
>> >> to reverse engineer the queries, I've taken a different approach - I've
>> >> compiled a list of queries that I think are sensible and relevant, and
>> >> then planned them with incremental sort enabled in different places.
>> >
>> >[...]
>> >
>> >> The list of queries (synthetic, but hopefully sufficiently realistic)
>> >> and a couple of scripts to collect the plans is in this repository:
>> >>
>> >>  https://github.com/tvondra/incremental-sort-tests-2
>> >>
>> >> There's also a spreadsheet with a summary of results, with a visual
>> >> representation of which GUCs affect which queries.
>> >
>> >OK, so we have that now.  I suppose this spreadsheet now tells us which
>> >places are useful and which aren't, at least for the queries that you've
>> >tested.  Dowe that mean that we want to get the patch to consider adding
>> >paths only the places that your spreadsheet says are useful?  I'm not
>> >sure what the next steps are for this patch.
>> >
>>
>> Yes. I think the spreadsheet call help us with answering two things:
>>
>> 1) places actually affecting the plan (all but three do)
>>
>> 2) redundant places (there are some cases where two GUCs produce the
>> same plan in the end)
>
>To expand on this further, (1) should probably help us to be able to
>write test cases.
>
>Additionally, one big thing we still need that's somewhat external to
>the patch is a good way to benchmark/a set of queries that we believe
>are representative enough to be good benchmarks.
>
>I'd really appreciate some input from you all on that particular
>question; I feel like it's in some sense the biggest barrier to
>getting the patch merged, but also the part where long experience in
>the community/exposure to other use cases will probably be quite
>valuable.
>

Hmmm. I don't think anyone will hand us a set of representative queries,
so I think we have two options:

1) Generate synthetic queries covering a wide range of cases (both when
incremental sort is expected to help and not). I think the script I've
used to determine which places do matter can be the starting point.

2) Look at some established benchmarks and see if some of the queries
could benefit from the incremental sort (possibly with some changes to
indexes in the usual schema). I plan to look at TPC-H / TPC-DS, but I
wonder if some OLTP benchmarks would be relevant too.

>> Of course, this does assume the query set makes sense and is somewhat
>> realistic, but I've tried to construct queries where that is true. We
>> may extend it over time, of course.
>>
>> I think we've agreed to add incremental sort paths different places in
>> separate patches, to make review easier. So this may be a useful way to
>> decide which places to address first. I'd probably do it in this order:
>>
>> - create_ordered_paths
>> - create_ordered_paths (parallel part)
>> - add_paths_to_grouping_rel
>> - ... not sure ...
>>
>> but that's just a proposal. It'd give us most of the benefits, I think,
>> and we could also focus on the rest of the patch.
>
>Certainly the first two seem like pretty obvious most necessary base
>cases. I think supporting group bys also seems like a pretty standard
>case, so at first glance I'd say this seems like a reasonable course
>to me.
>

OK.

>I'm going to start breaking up the patches in this thread into a
>series in support of that. Since I've started a new thread with the
>add_partial_path change, I'll include that patch here as part of this
>series also. Do you think it's worth moving the tuplesort changes into
>a standalone patch in the series also?
>

Probably. I'd do that at least for the review.

>Attached is a rebased v31 now broken into the following:
>
>- 001-consider-startup-cost-in-add-partial-path_v1.patch: From the
>other thread (Tomas's patch unmodified)
>- 002-incremental-sort_v31.patch: Updated base incremental sort patch
>
>Besides rebasing, I've changed the enable_incrementalsort GUC to
>prevent generating paths entirely rather than being cost-based, since
>incremental sort is never absolutely necessary in the way regular sort
>is.
>

OK, makes sense.

>I'm hoping to add 003 soon with the initial parallel parts, but I'm
>about out of time right now and wanted to get something out, so
>sending this without that.
>
>Side question: for the patch tester do I have to attach each part of
>the series each time even if nothing's changed in several of them? And
>does the vN number at the end need to stay the same for all of them?
>My attachments to this email don't follow that... Also, since this
>email changes patch naming, so I need to do anything to clear out the
>old ones? (I suppose if not, then that would imply an answer to the
>first question also.)
>

Please always send the whole patch series. Firstly, that's the only way
how the cfbot can apply and test the patches (it can't collect patches
scattered in different messages in the thread). Secondly, it's really
annoying for the reviewers to try to pick the matching bits.

Also, it's a good ide to always mark all parts with the same version
info, not v1 for one part and v31 for another one. I'd simply do
something like

    git format-patch --suffix=-YYYYMMDD.patch master

or something like that.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Fri, Sep 27, 2019 at 01:50:30PM -0400, James Coleman wrote:
>On Mon, Sep 9, 2019 at 5:55 PM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>> The "patched" column means all developer GUCs disabled, so it's expected
>> to produce the same plan as master (and it is). And then there's one
>> column for each developer GUC. If the column is just TRUE it means the
>> GUC does not affect any of the synthetic queries. There are 4 of them:
>>
>> - devel_add_paths_to_grouping_rel_parallel
>> - devel_create_partial_grouping_paths
>> - devel_gather_grouping_paths
>> - devel_standard_join_search
>>
>> The places controlled by those GUCs are either useless, or the query
>> affected by them is not included in the list of queries.
>
>I'd previously found (in my reverse engineering efforts) the query:
>
>select *
>from tenk1 t1
>join tenk1 t2 on t1.hundred = t2.hundred
>join tenk1 t3 on t1.hundred = t3.hundred
>order by t1.hundred, t1.twenty
>limit 50;
>
>can change plans to use incremental sort when
>generate_useful_gather_paths() is added to standard_join_search().
>Specifically, we get a merge join between t1 and t3 as the top level
>(besides limit) node where the driving side of the join is a gather
>merge with incremental sort. This does rely on these gucs set in the
>test harness:
>
>set local max_parallel_workers_per_gather=4;
>set local min_parallel_table_scan_size=0;
>set local parallel_tuple_cost=0;
>set local parallel_setup_cost=0;
>
>So I think we can reduce the number of unused gucs to 3.
>

OK. I'll try extending the set of synthetic queries in [1] to also do
soemthing like this and generate similar plans.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Sun, Sep 29, 2019 at 01:00:49AM +0200, Tomas Vondra wrote:
> OK. I'll try extending the set of synthetic queries in [1] to also do
> soemthing like this and generate similar plans.

Any progress on that?

Please note that the latest patch does not apply anymore, so a rebase
is needed.  I am switching the patch as waiting on author for now.
--
Michael

On Fri, Nov 29, 2019 at 03:01:46PM +0900, Michael Paquier wrote:
>On Sun, Sep 29, 2019 at 01:00:49AM +0200, Tomas Vondra wrote:
>> OK. I'll try extending the set of synthetic queries in [1] to also do
>> soemthing like this and generate similar plans.
>
>Any progress on that?
>
>Please note that the latest patch does not apply anymore, so a rebase
>is needed.  I am switching the patch as waiting on author for now.
>--

Ah, thanks for reminding me. I've added a couple more queries with two
joins (there only were queries with two joins, I haven't expected
another joint to make such difference, but seems I was wrong).

So yes, there seem to be 6 different GUCs / places where considering
incremental sort makes a difference (the numbers say how many of the
4960 tested combinations were affected)

- create_ordered_paths_parallel (50)
- create_partial_grouping_paths_2 (228)
- standard_join_search (94)
- add_paths_to_grouping_rel (2148)
- set_rel_pathlist (156)
- apply_scanjoin_target_to_paths (286)

Clearly some of the places are more important than others, plus there
are some overlaps (two GUCs producing the same plan, etc.).

Plus there are four GUCs that did not affect any queries at all:

- create_partial_grouping_paths
- gather_grouping_paths
- create_ordered_paths
- add_paths_to_grouping_rel_parallel

Anyway, this might serve as a way to prioritize the effort. All the
test changes are in the original repo at

   https://github.com/tvondra/incremental-sort-tests-2
   
and I'm also attaching the rebased patches - the changes were pretty
minor, hopefully that helps others (all the patches with dev GUCs are in

https://github.com/tvondra/postgres/tree/incremental-sort-20191129


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Hi,

This patch has been marked as WoA since end of November, and there has
been no discussion/reviews since then :-( Based on off-list discussion
with James I don't think that's going to change in this CF, so I'll move
it to the next CF.

I plan to work on the planner part of this patch before 2020-03, with
the hope it can still make it into 13.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Tue, Jan 21, 2020 at 9:25 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> Hi,
>
> This patch has been marked as WoA since end of November, and there has
> been no discussion/reviews since then :-( Based on off-list discussion
> with James I don't think that's going to change in this CF, so I'll move
> it to the next CF.
>
> I plan to work on the planner part of this patch before 2020-03, with
> the hope it can still make it into 13.

In that off-list discussion I'd mentioned to Tomas that I would still
like to work on this, just my other responsibilities at work have left
me little time to work on the most important remaining part of this
(the planner parts) since that requires a fair amount of focus and
time.

That being said, the patch also needs some more work on improving
EXPLAIN ANALYZE output (perhaps min/max/mean or median of
memory usage number of groups in each sort mode), and I think it's far
more feasible that I can tackle that piecemeal before the next CF.

James

On Tue, Jan 21, 2020 at 09:37:01AM -0500, James Coleman wrote:
>On Tue, Jan 21, 2020 at 9:25 AM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>>
>> Hi,
>>
>> This patch has been marked as WoA since end of November, and there has
>> been no discussion/reviews since then :-( Based on off-list discussion
>> with James I don't think that's going to change in this CF, so I'll move
>> it to the next CF.
>>
>> I plan to work on the planner part of this patch before 2020-03, with
>> the hope it can still make it into 13.
>
>In that off-list discussion I'd mentioned to Tomas that I would still
>like to work on this, just my other responsibilities at work have left
>me little time to work on the most important remaining part of this
>(the planner parts) since that requires a fair amount of focus and
>time.
>
>That being said, the patch also needs some more work on improving
>EXPLAIN ANALYZE output (perhaps min/max/mean or median of
>memory usage number of groups in each sort mode), and I think it's far
>more feasible that I can tackle that piecemeal before the next CF.
>

Sure, sorry if that was not clear from my message - you're of course
more than welcome to continue working on this. My understanding is that
won't happen by the end of this CF, hence the move to 2020-03.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Tue, Jan 21, 2020 at 9:58 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> On Tue, Jan 21, 2020 at 09:37:01AM -0500, James Coleman wrote:
> >On Tue, Jan 21, 2020 at 9:25 AM Tomas Vondra
> ><tomas.vondra@2ndquadrant.com> wrote:
> >>
> >> Hi,
> >>
> >> This patch has been marked as WoA since end of November, and there has
> >> been no discussion/reviews since then :-( Based on off-list discussion
> >> with James I don't think that's going to change in this CF, so I'll move
> >> it to the next CF.
> >>
> >> I plan to work on the planner part of this patch before 2020-03, with
> >> the hope it can still make it into 13.
> >
> >In that off-list discussion I'd mentioned to Tomas that I would still
> >like to work on this, just my other responsibilities at work have left
> >me little time to work on the most important remaining part of this
> >(the planner parts) since that requires a fair amount of focus and
> >time.
> >
> >That being said, the patch also needs some more work on improving
> >EXPLAIN ANALYZE output (perhaps min/max/mean or median of
> >memory usage number of groups in each sort mode), and I think it's far
> >more feasible that I can tackle that piecemeal before the next CF.
> >
>
> Sure, sorry if that was not clear from my message - you're of course
> more than welcome to continue working on this. My understanding is that
> won't happen by the end of this CF, hence the move to 2020-03.

Oh, yeah, I probably didn't word that reply well -- I just wanted to
add some additional detail to what you had already said.

Thanks for your work on this!

James

James and Tomas,

On 1/21/20 10:03 AM, James Coleman wrote:
> On Tue, Jan 21, 2020 at 9:58 AM Tomas Vondra
> <tomas.vondra@2ndquadrant.com> wrote:
>>
>> On Tue, Jan 21, 2020 at 09:37:01AM -0500, James Coleman wrote:
>>> On Tue, Jan 21, 2020 at 9:25 AM Tomas Vondra
>>> <tomas.vondra@2ndquadrant.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> This patch has been marked as WoA since end of November, and there has
>>>> been no discussion/reviews since then :-( Based on off-list discussion
>>>> with James I don't think that's going to change in this CF, so I'll move
>>>> it to the next CF.
>>>>
>>>> I plan to work on the planner part of this patch before 2020-03, with
>>>> the hope it can still make it into 13.
>>>
>>> In that off-list discussion I'd mentioned to Tomas that I would still
>>> like to work on this, just my other responsibilities at work have left
>>> me little time to work on the most important remaining part of this
>>> (the planner parts) since that requires a fair amount of focus and
>>> time.
>>>
>>> That being said, the patch also needs some more work on improving
>>> EXPLAIN ANALYZE output (perhaps min/max/mean or median of
>>> memory usage number of groups in each sort mode), and I think it's far
>>> more feasible that I can tackle that piecemeal before the next CF.
>>>
>>
>> Sure, sorry if that was not clear from my message - you're of course
>> more than welcome to continue working on this. My understanding is that
>> won't happen by the end of this CF, hence the move to 2020-03.
> 
> Oh, yeah, I probably didn't word that reply well -- I just wanted to
> add some additional detail to what you had already said.
> 
> Thanks for your work on this!

It doesn't look there has been much movement on this patch for the last 
few CFs.  Are one or both of you planning to work on this for v13? Or 
should we mark it for v14 and/or move it to the next CF.

Regards,
-- 
-David
david@pgmasters.net

On Tue, Mar 03, 2020 at 12:17:22PM -0500, David Steele wrote:
>James and Tomas,
>
>On 1/21/20 10:03 AM, James Coleman wrote:
>>On Tue, Jan 21, 2020 at 9:58 AM Tomas Vondra
>><tomas.vondra@2ndquadrant.com> wrote:
>>>
>>>On Tue, Jan 21, 2020 at 09:37:01AM -0500, James Coleman wrote:
>>>>On Tue, Jan 21, 2020 at 9:25 AM Tomas Vondra
>>>><tomas.vondra@2ndquadrant.com> wrote:
>>>>>
>>>>>Hi,
>>>>>
>>>>>This patch has been marked as WoA since end of November, and there has
>>>>>been no discussion/reviews since then :-( Based on off-list discussion
>>>>>with James I don't think that's going to change in this CF, so I'll move
>>>>>it to the next CF.
>>>>>
>>>>>I plan to work on the planner part of this patch before 2020-03, with
>>>>>the hope it can still make it into 13.
>>>>
>>>>In that off-list discussion I'd mentioned to Tomas that I would still
>>>>like to work on this, just my other responsibilities at work have left
>>>>me little time to work on the most important remaining part of this
>>>>(the planner parts) since that requires a fair amount of focus and
>>>>time.
>>>>
>>>>That being said, the patch also needs some more work on improving
>>>>EXPLAIN ANALYZE output (perhaps min/max/mean or median of
>>>>memory usage number of groups in each sort mode), and I think it's far
>>>>more feasible that I can tackle that piecemeal before the next CF.
>>>>
>>>
>>>Sure, sorry if that was not clear from my message - you're of course
>>>more than welcome to continue working on this. My understanding is that
>>>won't happen by the end of this CF, hence the move to 2020-03.
>>
>>Oh, yeah, I probably didn't word that reply well -- I just wanted to
>>add some additional detail to what you had already said.
>>
>>Thanks for your work on this!
>
>It doesn't look there has been much movement on this patch for the 
>last few CFs.  Are one or both of you planning to work on this for 
>v13? Or should we mark it for v14 and/or move it to the next CF.
>

I'm currently working on it, I plan to submit a new patch version
shortly - hopefully by the end of this week.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 

James Coleman <jtc331@gmail.com> writes:
> I'm looking at this now, and realized that at least for parallel plans the
> current patch tracks the tuplesort instrumentation whether or not an
> EXPLAIN ANALYZE is in process.

> Is this fairly standard for executor nodes? Or is it expected to condition
> some of this tracking based on whether or not an ANALYZE is running?

No, it's entirely not standard.  Maybe you could make an argument that
it's too cheap to bother making it conditional, but without a convincing
argument for that, it needs to be conditional.

            regards, tom lane

James Coleman <jtc331@gmail.com> writes:
> That's what I figured, but as I mentioned I've having trouble figuring out
> how the fact that an analyze is in flight is determined. I assume it's
> something that lives of the EState or similar, but I'm not seeing anything
> obvious.

AFAIR, it's just whether or not the current planstate node has an
instrumentation struct attached.

            regards, tom lane