Query progress indication - an implementation - Mailing list pgsql-hackers

From Scara Maccai
Subject Query progress indication - an implementation
Date
Msg-id 981140.95593.qm@web24615.mail.ird.yahoo.com
Whole thread Raw
Responses Re: Query progress indication - an implementation  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
Hi all,

following the link in

http://wiki.postgresql.org/wiki/Query_progress_indication

but mostly:

http://www.postech.ac.kr/~swhwang/progress2.pdf [1]

I'm trying to write an implementation of the "dne" method in postgresql.

I added another column to the pg_stat_get_activity function to report the percentage of work done for the query (of
course,any other method could be used... the way the percentage is reported to the user can be easily changed). 

I attached a first patch (just to see if anyone is interested, the work is by no means finished).

I guess I did a lot of mistakes, since I don't know anything about postgresql code...

1) the progress indicator can be eliminated at runtime; this could be done with another runtime flag (at the moment is
alwayson) 

2) I added a new structure (Progress) to PlanState to keep all the info about execution progress

3) I needed a pointer to the root of the PlanStates, to be able to calculate the total progress of the query tree (I
betthis pointer was already available somewhere, but I couldn't find where...) 

4) sub-plans are not included yet (well, just to be honest, I don't really know what postgresql means with those... :)
)

5) the percentage is updated at most every second (can be easily changed)

6) the methods to adjust upper/lower bounds in [1] are not implemented yet (but that shouldn't be a problem)

7) the "spilled tuples" handling in [1] is not supported yet

8) only hash join, loop join, aggregate, sequence scans are implemented at the moment

9) I added another flag (EXEC_FLAG_DRIVER_BRANCH) in executor.h to signal to the sub-nodes if they are part of a branch
thatwill contain a driver node (for example, inner subtree of a Nested Loops join is not a driver branch). I guess this
couldbe done better at Plan level (instead of PlanState), but this way less code has to be changed 

10) at the moment all driver nodes have the same "work_per_tuple=1", but this could be changed (for example,
CPU-intensivedriver nodes could have a smaller work_per_tuple value) 

Well, some (very early) tests on a tpcd db showed it works as expected (well, I only did very few tests...)

Hope someone is interested



Attachment

pgsql-hackers by date:

Previous
From: Dimitri Fontaine
Date:
Subject: Re: [PATCH] backend: compare word-at-a-time in bcTruelen
Next
From: Tsutomu Yamada
Date:
Subject: Proposal: More portable way to support 64bit platforms