Thread: Thread manager
Hi, While working on Vladimir's patch, it reminded me of an idea I wanted to share. Right now, when you launch a dump, or a VACUUM, the UI kinda sucks because it puts you on another tab, still in the same dialog, and you can't be sure you can do something else while the work is ongoing. It'll worse with Vladimir's patch as there is no UI at all to show the progress on the copy (if you copy more than one table of course). I was a few weeks ago at a customer's office, and he showed me a nice UI of a tool (unfortunately, I don't remember its name) that allowed him to queue some jobs. So I wondered if we could do the same. When you're in the VACUUM dialog, and click OK, it adds a job in the jobs list, and a thread will catch it as soon as it is available for a new job. The UI could be as simple as a new pane in frmMain, with a listbox which contains the list of job, and their status. We could use if for the "copy table" patch. We could also use it for the dump/restore dialog. And there are probably more things to do with it. We could probably add buttons to pause/resume waiting jobs, and to drop jobs. IIRC, a long time ago, Peter sent a patch to show the progress of a job (VACUUM I think, but I might be wrong). Unfortunately, it has been rejected, but I guess someone (maybe Peter himself) will want to work again on this. And this would be the perfect UI to show that kind of information. Yeah, I know, that part is more a dream than a reality, but it's a nice dream :) Anyways, comments? -- Guillaume http://blog.guillaume.lelarge.info http://www.dalibo.com
On Mon, Oct 3, 2011 at 9:35 PM, Guillaume Lelarge <guillaume@lelarge.info> wrote: > Hi, > > While working on Vladimir's patch, it reminded me of an idea I wanted to > share. > > Right now, when you launch a dump, or a VACUUM, the UI kinda sucks > because it puts you on another tab, still in the same dialog, and you > can't be sure you can do something else while the work is ongoing. It'll You should be able to - the busy cursor should only be active for the busy dialogue. Switch to another window and you should get a non-busy cursor which indicates you can do stuff. > worse with Vladimir's patch as there is no UI at all to show the > progress on the copy (if you copy more than one table of course). There needs to be some status indication, even if it's just an animation (maybe busy cursor over the dialogue) and text that says "Copying table 1...", "Copying table 2..." etc. > I was a few weeks ago at a customer's office, and he showed me a nice UI > of a tool (unfortunately, I don't remember its name) that allowed him to > queue some jobs. So I wondered if we could do the same. When you're in > the VACUUM dialog, and click OK, it adds a job in the jobs list, and a > thread will catch it as soon as it is available for a new job. The UI We don't need to use a thread pool to save resources for this kind of thing, even if running on a net book - we can just launch a thread as needed and let the OS deal with resource management. I can't imagine the user will be able to start enough task threads that even a Windows 95 system couldn't cope. > could be as simple as a new pane in frmMain, with a listbox which > contains the list of job, and their status. All of these threads that we have at the moment are returning data to the UI for display (in theory, as work progresses, but I'm aware that doesn't work everywhere). Are you suggesting that we would get rid of that UI, and make the user move to another window, display the appropriate pane (something like this wouldn't be visible by default), and then double-click the appropriate row to view the output? What about purging old data? How would we decide when to remove something from the list? > We could use if for the > "copy table" patch. We could also use it for the dump/restore dialog. > And there are probably more things to do with it. We could probably add > buttons to pause/resume waiting jobs, and to drop jobs. I'm afraid I cannot see how this is a better design. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Tue, 2011-10-04 at 11:01 +0100, Dave Page wrote: > On Mon, Oct 3, 2011 at 9:35 PM, Guillaume Lelarge > <guillaume@lelarge.info> wrote: > > Hi, > > > > While working on Vladimir's patch, it reminded me of an idea I wanted to > > share. > > > > Right now, when you launch a dump, or a VACUUM, the UI kinda sucks > > because it puts you on another tab, still in the same dialog, and you > > can't be sure you can do something else while the work is ongoing. It'll > > You should be able to - the busy cursor should only be active for the > busy dialogue. Switch to another window and you should get a non-busy > cursor which indicates you can do stuff. > Yeah. It's just a cursor though. > > worse with Vladimir's patch as there is no UI at all to show the > > progress on the copy (if you copy more than one table of course). > > There needs to be some status indication, even if it's just an > animation (maybe busy cursor over the dialogue) and text that says > "Copying table 1...", "Copying table 2..." etc. > AFAICT, you only have the final status. > > I was a few weeks ago at a customer's office, and he showed me a nice UI > > of a tool (unfortunately, I don't remember its name) that allowed him to > > queue some jobs. So I wondered if we could do the same. When you're in > > the VACUUM dialog, and click OK, it adds a job in the jobs list, and a > > thread will catch it as soon as it is available for a new job. The UI > > We don't need to use a thread pool to save resources for this kind of > thing, even if running on a net book - we can just launch a thread as > needed and let the OS deal with resource management. I can't imagine > the user will be able to start enough task threads that even a Windows > 95 system couldn't cope. > I'm not talking about performance. I'm talking about UI. I'm talking about adding a single UI to handle many operations. > > could be as simple as a new pane in frmMain, with a listbox which > > contains the list of job, and their status. > > All of these threads that we have at the moment are returning data to > the UI for display (in theory, as work progresses, but I'm aware that > doesn't work everywhere). Are you suggesting that we would get rid of > that UI, and make the user move to another window, display the > appropriate pane (something like this wouldn't be visible by default), > and then double-click the appropriate row to view the output? What > about purging old data? How would we decide when to remove something > from the list? > Let's say you want to do a "VACUUM FULL" (yeah, I know, crazy idea). Now, you'll have the Maintenance window working, and you can get to the browser window to continue (if you can) your work. What I suggest is adding a pane in the browser window. So, you open the Maintenance window, do your stuff to launch a VACUUM FULL. Then, the Maintenance window is closed, the "Jobs" pane is opened (if it wasn't already), a new job appears stating that it is a VACUUM FULL job on database/table whatever, and that the job is running. We can probably add more infomations (for example, the duration, if it's blocked by another session, etc.) and more actions (kill it, pause it, resume it). > > We could use if for the > > "copy table" patch. We could also use it for the dump/restore dialog. > > And there are probably more things to do with it. We could probably add > > buttons to pause/resume waiting jobs, and to drop jobs. > > I'm afraid I cannot see how this is a better design. -- Guillaume http://blog.guillaume.lelarge.info http://www.dalibo.com
On Sun, Oct 9, 2011 at 9:22 AM, Guillaume Lelarge <guillaume@lelarge.info> wrote: > >> > worse with Vladimir's patch as there is no UI at all to show the >> > progress on the copy (if you copy more than one table of course). >> >> There needs to be some status indication, even if it's just an >> animation (maybe busy cursor over the dialogue) and text that says >> "Copying table 1...", "Copying table 2..." etc. >> > > AFAICT, you only have the final status. Some progress text would certainly be sensible - this sort of operation could take hours. >> > I was a few weeks ago at a customer's office, and he showed me a nice UI >> > of a tool (unfortunately, I don't remember its name) that allowed him to >> > queue some jobs. So I wondered if we could do the same. When you're in >> > the VACUUM dialog, and click OK, it adds a job in the jobs list, and a >> > thread will catch it as soon as it is available for a new job. The UI >> >> We don't need to use a thread pool to save resources for this kind of >> thing, even if running on a net book - we can just launch a thread as >> needed and let the OS deal with resource management. I can't imagine >> the user will be able to start enough task threads that even a Windows >> 95 system couldn't cope. >> > > I'm not talking about performance. I'm talking about UI. I'm talking > about adding a single UI to handle many operations. You were talking about a thread pool, which I think is totally unnecessary: "a thread will catch it as soon as it is available for a new job" >> > could be as simple as a new pane in frmMain, with a listbox which >> > contains the list of job, and their status. >> >> All of these threads that we have at the moment are returning data to >> the UI for display (in theory, as work progresses, but I'm aware that >> doesn't work everywhere). Are you suggesting that we would get rid of >> that UI, and make the user move to another window, display the >> appropriate pane (something like this wouldn't be visible by default), >> and then double-click the appropriate row to view the output? What >> about purging old data? How would we decide when to remove something >> from the list? >> > > Let's say you want to do a "VACUUM FULL" (yeah, I know, crazy idea). > Now, you'll have the Maintenance window working, and you can get to the > browser window to continue (if you can) your work. If you can't, then we have a bug in the current code. You should be able to minimise the Maintenance window and carry on. > What I suggest is adding a pane in the browser window. So, you open the > Maintenance window, do your stuff to launch a VACUUM FULL. Then, the > Maintenance window is closed, the "Jobs" pane is opened (if it wasn't > already), a new job appears stating that it is a VACUUM FULL job on > database/table whatever, and that the job is running. We can probably > add more infomations (for example, the duration, if it's blocked by > another session, etc.) and more actions (kill it, pause it, resume it). Yeah, I understand what you're proposing. I've seen similar things in other products in years gone by. I don't think there's any need for us to do anything like that though - I think it would take a lot of engineering and buy little. More importantly, I think it will be detrimental to the user experience, as a task that was once confined to one dialogue now is started from one, and ends on separate window. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Mon, Oct 10, 2011 at 12:06, Dave Page <dpage@pgadmin.org> wrote: > On Sun, Oct 9, 2011 at 9:22 AM, Guillaume Lelarge > <guillaume@lelarge.info> wrote: >> >>> > worse with Vladimir's patch as there is no UI at all to show the >>> > progress on the copy (if you copy more than one table of course). >>> >>> There needs to be some status indication, even if it's just an >>> animation (maybe busy cursor over the dialogue) and text that says >>> "Copying table 1...", "Copying table 2..." etc. >>> >> >> AFAICT, you only have the final status. > > Some progress text would certainly be sensible - this sort of > operation could take hours. > >>> > I was a few weeks ago at a customer's office, and he showed me a nice UI >>> > of a tool (unfortunately, I don't remember its name) that allowed him to >>> > queue some jobs. So I wondered if we could do the same. When you're in >>> > the VACUUM dialog, and click OK, it adds a job in the jobs list, and a >>> > thread will catch it as soon as it is available for a new job. The UI >>> >>> We don't need to use a thread pool to save resources for this kind of >>> thing, even if running on a net book - we can just launch a thread as >>> needed and let the OS deal with resource management. I can't imagine >>> the user will be able to start enough task threads that even a Windows >>> 95 system couldn't cope. >>> >> >> I'm not talking about performance. I'm talking about UI. I'm talking >> about adding a single UI to handle many operations. > > You were talking about a thread pool, which I think is totally > unnecessary: "a thread will catch it as soon as it is available for a > new job" Not sure that necessarily indicates a threadpool, but let's not get hung up on details :-) FWIW, I think a threadpool would be very much overkill as well - not to mention that they'd all require separate db connections anyway, so you'd have to have all sorts of ways to limit it etc.. >>> > could be as simple as a new pane in frmMain, with a listbox which >>> > contains the list of job, and their status. >>> >>> All of these threads that we have at the moment are returning data to >>> the UI for display (in theory, as work progresses, but I'm aware that >>> doesn't work everywhere). Are you suggesting that we would get rid of >>> that UI, and make the user move to another window, display the >>> appropriate pane (something like this wouldn't be visible by default), >>> and then double-click the appropriate row to view the output? What >>> about purging old data? How would we decide when to remove something >>> from the list? >>> >> >> Let's say you want to do a "VACUUM FULL" (yeah, I know, crazy idea). >> Now, you'll have the Maintenance window working, and you can get to the >> browser window to continue (if you can) your work. > > If you can't, then we have a bug in the current code. You should be > able to minimise the Maintenance window and carry on. Can you have multiple maintenance windows open at the same time, all minimized? >> What I suggest is adding a pane in the browser window. So, you open the >> Maintenance window, do your stuff to launch a VACUUM FULL. Then, the >> Maintenance window is closed, the "Jobs" pane is opened (if it wasn't >> already), a new job appears stating that it is a VACUUM FULL job on >> database/table whatever, and that the job is running. We can probably >> add more infomations (for example, the duration, if it's blocked by >> another session, etc.) and more actions (kill it, pause it, resume it). > > Yeah, I understand what you're proposing. I've seen similar things in > other products in years gone by. I don't think there's any need for us > to do anything like that though - I think it would take a lot of > engineering and buy little. More importantly, I think it will be > detrimental to the user experience, as a task that was once confined > to one dialogue now is started from one, and ends on separate window. I'd find it useful, actually. But even more so if there was one "execute now" and one "execute in the background" button, or something like that. Being able to sticka bunch of things in a queue, knowing they execute one by one over a single connection so I don't overload the server, would be useful. (Which makes it even more important that it's not a threadpool, really :D) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On Mon, 10 Oct 2011 20:45:13 +0200, Magnus Hagander <magnus@hagander.net> wrote: > On Mon, Oct 10, 2011 at 12:06, Dave Page <dpage@pgadmin.org> wrote: >> On Sun, Oct 9, 2011 at 9:22 AM, Guillaume Lelarge >> <guillaume@lelarge.info> wrote: >>> >>>> > worse with Vladimir's patch as there is no UI at all to show the >>>> > progress on the copy (if you copy more than one table of course). >>>> >>>> There needs to be some status indication, even if it's just an >>>> animation (maybe busy cursor over the dialogue) and text that says >>>> "Copying table 1...", "Copying table 2..." etc. >>>> >>> >>> AFAICT, you only have the final status. >> >> Some progress text would certainly be sensible - this sort of >> operation could take hours. >> >>>> > I was a few weeks ago at a customer's office, and he showed me a >>>> > nice UI >>>> > of a tool (unfortunately, I don't remember its name) that allowed >>>> > him to >>>> > queue some jobs. So I wondered if we could do the same. When you're >>>> > in >>>> > the VACUUM dialog, and click OK, it adds a job in the jobs list, and >>>> > a >>>> > thread will catch it as soon as it is available for a new job. The UI >>>> >>>> We don't need to use a thread pool to save resources for this kind of >>>> thing, even if running on a net book - we can just launch a thread as >>>> needed and let the OS deal with resource management. I can't imagine >>>> the user will be able to start enough task threads that even a Windows >>>> 95 system couldn't cope. >>>> >>> >>> I'm not talking about performance. I'm talking about UI. I'm talking >>> about adding a single UI to handle many operations. >> >> You were talking about a thread pool, which I think is totally >> unnecessary: "a thread will catch it as soon as it is available for a >> new job" > > Not sure that necessarily indicates a threadpool, but let's not get > hung up on details :-) FWIW, I think a threadpool would be very much > overkill as well - not to mention that they'd all require separate db > connections anyway, so you'd have to have all sorts of ways to limit > it etc.. > No, I don't want a threadpool. Maybe a timer to check if a new appeared, and maybe one thread to launch a job. >>>> > could be as simple as a new pane in frmMain, with a listbox which >>>> > contains the list of job, and their status. >>>> >>>> All of these threads that we have at the moment are returning data to >>>> the UI for display (in theory, as work progresses, but I'm aware that >>>> doesn't work everywhere). Are you suggesting that we would get rid of >>>> that UI, and make the user move to another window, display the >>>> appropriate pane (something like this wouldn't be visible by default), >>>> and then double-click the appropriate row to view the output? What >>>> about purging old data? How would we decide when to remove something >>>> from the list? >>>> >>> >>> Let's say you want to do a "VACUUM FULL" (yeah, I know, crazy idea). >>> Now, you'll have the Maintenance window working, and you can get to the >>> browser window to continue (if you can) your work. >> >> If you can't, then we have a bug in the current code. You should be >> able to minimise the Maintenance window and carry on. > > Can you have multiple maintenance windows open at the same time, all > minimized? > As many as you want... >>> What I suggest is adding a pane in the browser window. So, you open the >>> Maintenance window, do your stuff to launch a VACUUM FULL. Then, the >>> Maintenance window is closed, the "Jobs" pane is opened (if it wasn't >>> already), a new job appears stating that it is a VACUUM FULL job on >>> database/table whatever, and that the job is running. We can probably >>> add more infomations (for example, the duration, if it's blocked by >>> another session, etc.) and more actions (kill it, pause it, resume it). >> >> Yeah, I understand what you're proposing. I've seen similar things in >> other products in years gone by. I don't think there's any need for us >> to do anything like that though - I think it would take a lot of >> engineering and buy little. More importantly, I think it will be >> detrimental to the user experience, as a task that was once confined >> to one dialogue now is started from one, and ends on separate window. > > I'd find it useful, actually. But even more so if there was one > "execute now" and one "execute in the background" button, or something > like that. Being able to sticka bunch of things in a queue, knowing > they execute one by one over a single connection so I don't overload > the server, would be useful. (Which makes it even more important that > it's not a threadpool, really :D) +1 I suppose I'll work on it sometime, just to get a POC, something to show. Right now, I have something else to do :) -- Guillaume http://www.postgresql.fr http://dalibo.com