Thread: External Sort timing debug statements

The following patch implements a fairly light set of timing statements
aimed at understanding external sort performance. There is no attempt to
alter the algorithms.

Each major point in the algorithms is marked as shown in this example:

postgres=# set debug_sort=true;
SET

postgres=# explain analyze select * from test2 order by col1,col2;

NOTICE:  tuplesort begin  work_mem= 1024
NOTICE:  +0 secs  heap sort  nkeys= 2
NOTICE:  +0 secs  switching to external sort
NOTICE:  +1129 secs  starting build of next run
NOTICE:  +2229 secs  run building complete  nruns= 2
NOTICE:  +2229 secs  merging runs with 6 tapes

...

NOTICE:  +3036 secs  starting final merge

I'll add other information, as requested.

The "6 tapes" is currently hardcoded, though is included in expectation
of implementing variable numbers of tapes.

I'm not sure if I got the header file correct for full portability of
gettimeofday(). Please correct me, if this is the case.

Please post sort performance data back via this post.

Best Regards, Simon Riggs

Simon Riggs <simon@2ndquadrant.com> writes:
> The following patch implements a fairly light set of timing statements
> aimed at understanding external sort performance. There is no attempt to
> alter the algorithms.

What do people think about putting something like this into 8.1?
Strictly speaking it's a new feature, but the patch seems pretty
noninvasive, and we'd be much more likely to get data points if the
code exists in the mainline release than if people have to patch
their copies.

> postgres=# set debug_sort=true;

I'm a bit inclined to call it trace_sort instead, and to document it
under "Developer Options".  Comments?

            regards, tom lane

On Mon, Oct 03, 2005 at 10:36:57AM -0400, Tom Lane wrote:
> Simon Riggs <simon@2ndquadrant.com> writes:
> > The following patch implements a fairly light set of timing
> > statements aimed at understanding external sort performance. There
> > is no attempt to alter the algorithms.
>
> What do people think about putting something like this into 8.1?
> Strictly speaking it's a new feature, but the patch seems pretty
> noninvasive, and we'd be much more likely to get data points if the
> code exists in the mainline release than if people have to patch
> their copies.

This would be good :)

> > postgres=# set debug_sort=true;
>
> I'm a bit inclined to call it trace_sort instead, and to document it
> under "Developer Options".  Comments?

+1

Cheers,
D
--
David Fetter david@fetter.org http://fetter.org/
phone: +1 510 893 6100   mobile: +1 415 235 3778

Remember to vote!

On Mon, 2005-10-03 at 10:36 -0400, Tom Lane wrote:
> Simon Riggs <simon@2ndquadrant.com> writes:
> > The following patch implements a fairly light set of timing statements
> > aimed at understanding external sort performance. There is no attempt to
> > alter the algorithms.
>
> What do people think about putting something like this into 8.1?
> Strictly speaking it's a new feature, but the patch seems pretty
> noninvasive, and we'd be much more likely to get data points if the
> code exists in the mainline release than if people have to patch
> their copies.
>
> > postgres=# set debug_sort=true;
>
> I'm a bit inclined to call it trace_sort instead, and to document it
> under "Developer Options".  Comments?

Sounds good to me.

Any other requests for info in the trace output.... shout now

Best Regards, Simon Riggs