Thread: Time to fix libpgtcl for async NOTIFY
About a month ago I wrote > Yes, if anything were to be done along this line it'd also make sense > to revise libpgtcl. I think it ought to work more like this: > (a) the idle loop is invoked while waiting for a query response > (so that a pg_exec statement behaves sort of like "tkwait"); > (b) a "listen" command is sent via a new pg_listen statement that > specifies a callback command string. Subsequent notify responses > can occur whenever a callback is possible. > I suppose (a) had better be an option to pg_exec statements so that > we don't break existing Tcl code... I was a little startled to discover that pg_listen already exists (it's not documented!). But the way things are currently set up, the callbacks specified by pg_listen are executed only when the Tcl script invokes the also-undocumented pg_notifies command. pg_notifies executes an empty query (which is no longer necessary) and then looks for notify responses. A practical Tcl application would have to execute pg_notifies often, perhaps every few seconds from an "after" timer event. What I'd like to do is eliminate pg_notifies and define pg_listen callbacks as happening automatically whenever Tcl can fire event callbacks (ie, at the idle loop). There's some risk of breaking existing applications, since the apps might not be prepared for pg_listen callbacks occurring except at the specific time they execute pg_notifies. But that doesn't seem really likely to be a problem. Besides, since both these commands are undocumented, I imagine not very many libpgtcl applications use them. (A quick search of the archives turned up only a message from Massimo Dal Zotto about this topic, and he seemed to agree that getting rid of pg_notifies would be better.) Any comments? BTW, I'm not currently planning to tackle the other point about providing an asynchronous pg_exec capability in libpgtcl. I've concluded that the Tcl code I'm planning to write wouldn't use it, at least not soon; so I'll leave that project for another person or another day. regards, tom lane
> There's some risk of breaking existing applications, since the apps > might not be prepared for pg_listen callbacks occurring except at the > specific time they execute pg_notifies. But that doesn't seem really > likely to be a problem. Besides, since both these commands are > undocumented, I imagine not very many libpgtcl applications use them. > (A quick search of the archives turned up only a message from Massimo > Dal Zotto about this topic, and he seemed to agree that getting rid of > pg_notifies would be better.) > > Any comments? Yep, get rid of the old stuff. I am sure people didn't use it because of the performance problem. Your cleanup will make it use-able. -- Bruce Momjian | 830 Blythe Avenue maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026 + If your life is a hard drive, | (610) 353-9879(w) + Christ can be your backup. | (610) 853-3000(h)
> > > There's some risk of breaking existing applications, since the apps > > might not be prepared for pg_listen callbacks occurring except at the > > specific time they execute pg_notifies. But that doesn't seem really > > likely to be a problem. Besides, since both these commands are > > undocumented, I imagine not very many libpgtcl applications use them. > > (A quick search of the archives turned up only a message from Massimo > > Dal Zotto about this topic, and he seemed to agree that getting rid of > > pg_notifies would be better.) > > > > Any comments? > > Yep, get rid of the old stuff. I am sure people didn't use it because > of the performance problem. Your cleanup will make it use-able. > > -- > Bruce Momjian | 830 Blythe Avenue > maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026 > + If your life is a hard drive, | (610) 353-9879(w) > + Christ can be your backup. | (610) 853-3000(h) > The old stuff works fine, at least for me! I'm using it, and my users like it very much. It suffers of performance problems but they are caused by bottlenecks on pg_listener and not by the pg_notifies loop in tcl (wich is done every 1 second by 30+ clients). I'm probably the only one who uses this feature, partly because I never found the time to write down some documentation about it (my fault), so any change you make shouldn't break many applications. If you are making changes to the tcl listen code please consider adding the following missing feature: - the possibility to temporarily suspend callbacks for a particular listen or for all listens. This would help to avoid the above possible problem. I currently do it with ad-hoc code in the application but having it done by a single call to libpgtcl would be better. - an option to get the callback associated to a particular listen or to all listens - better support for unlistening. There is an unlisten function in my contrib code but it should be integrated into the backend with a real UNLISTEN command. -- Massimo Dal Zotto +----------------------------------------------------------------------+ | Massimo Dal Zotto e-mail: dz@cs.unitn.it | | Via Marconi, 141 phone: ++39-461-534251 | | 38057 Pergine Valsugana (TN) www: http://www.cs.unitn.it/~dz/ | | Italy pgp: finger dz@tango.cs.unitn.it | +----------------------------------------------------------------------+
Massimo Dal Zotto <dz@cs.unitn.it> writes: > If you are making changes to the tcl listen code please consider adding the > following missing feature: > - the possibility to temporarily suspend callbacks for a particular listen > or for all listens. This would help to avoid the above possible problem. That seems reasonable. I presume that if a notify comes in while the callback is suspended, we should fire the callback after it gets reenabled? What happens if multiple notifies for the same name arrive while suspended --- one callback afterwards, or many? (I think I'd vote for only one, but...) > - an option to get the callback associated to a particular listen or to all > listens Perhaps also reasonable. In the code I submitted last night, it is possible for several different Tcl interpreters to be listening within one client process. I think that an interpreter should only be able to ask about its own callbacks, however --- anything else is a security violation that defeats the point of multiple interpreters. (Your "suspend callbacks" feature would likewise have to be interpreter- local.) > - better support for unlistening. There is an unlisten function in my > contrib code but it should be integrated into the backend with a real > UNLISTEN command. I agree with that, and will do the libpq and libpgtcl changes if someone else will implement UNLISTEN on the backend side. I'm not competent to twiddle the backend however. regards, tom lane PS: I'm on vacation till next week. Don't expect any fast responses.