Thread: UPDATE is not allowed in a non-volatile function
Hi all, I missed the discussion on hacker about this, and I'd like to give my HO. The fact that a non-volatile function can not perform update is a good improvement but on the other side will limit too much if I know what I'm doing. I did a sort of Lookup framework and this is extensively used in my project. The core is a "read" function that retrieve data from a very huge table. I declared the "read" function as "STABLE" this because each call with the same argument will return always the same value. For performances reason this "read" function first look in a little table (lt) that is used as a first level cache, if the data is not found then the function retrieve the data from the big one (bt) and replace the oldest required row in the lt. As you can understand I have to update lt in order to perform my replace policy. The "read" is often used inside other "STABLE" function. Not being able to declare "read" as STABLE I have to modify the function caller as well... :-( C++ have the same "problem": a const function can not modify the internal status of an object (that is great), however C++ permit to declare some internal status as "mutable" in this way a non-const function is able to modify status marked as mutable. ( For example you need to count how many time a const function was called ). I think a clean solution is be able to declare a table as "mutable" as well. Am I completely wrong or out of mind ? Regards Gaetano Mendola
Gaetano Mendola <mendola@bigfoot.com> writes:
> The fact that a non-volatile function can not perform
> update is a good improvement but on the other side will
> limit too much if I know what I'm doing.
I've got zero sympathy for this argument. It's been documented right
along that functions with side-effects must be marked volatile. You
don't have a lot of room to complain because 8.0 started to enforce that.
In practice you can circumvent the restriction by splitting the
function in two (ie, there is no check that a nonvolatile function
doesn't call any volatile functions). So if you insist on sticking
with an unsafe application design, you can do it with relatively
localized changes.
regards, tom lane
Tom Lane wrote: > Gaetano Mendola <mendola@bigfoot.com> writes:>>> The fact that a non-volatile function can not perform>> update is a goodimprovement but on the other side will>> limit too much if I know what I'm doing.>>>> I've got zero sympathy for thisargument. It's been documented right> along that functions with side-effects must be marked volatile. You> don't havea lot of room to complain because 8.0 started to enforce that. > In practice you can circumvent the restriction by splitting the> function in two (ie, there is no check that a nonvolatilefunction> doesn't call any volatile functions). So if you insist on sticking> with an unsafe application design,you can do it with relatively> localized changes. I do not consider my design as "unsafe", this is for example how a cache works: expose a "read" without side effect but updating internal statistics. After all the read will not alter the data that it expose but other data that the user even don't know the existence. However I think that "that missing check" is "unsafe" and jeopardize the effort to avoid a wrong user design. Having say that I'm happy to know that what I did will continue to work splitting the function in two parts. Regards Gaetano Mendola
Gaetano, > > I do not consider my design as "unsafe", this is for example how a > cache works: expose a "read" without side effect but updating internal > statistics. After all the read will not alter the data that it expose > but other data that the user even don't know the existence. > > However I think that "that missing check" is "unsafe" and jeopardize the > effort to avoid a wrong user design. > > Having say that I'm happy to know that what I did will continue to work > splitting the function in two parts. > I think Gaetano has a point but I consider the solution somewhat kludgy. The Rationale for my opinion is that since there is a need to accomplish what Gaetano needs, there should be declarative power to express it and thus, prevent "unsafe" designs. We need a way to declare a function "stable with no _intrusive_ side effects". It's far better to explicitly state this then to rely on a flaw of the current function calling mechanism. The current read-only status, maintained and passed to the SPI execute functions by function developers themselves, is not good enough. There are two main reasons for this: a) Nesting is not accounted for. The correct behavior should be to block all nested volatile calls as soon as a function declared non-volatile is called. I can of course enforce this within the PL/Java domain but I have no control over functions written in other languages. b) The responsability to maintain the read-only flag should not be pushed onto the function developers. It's fully possible for the PostgreSQL calling mechanism to maintain some global structure that is updated prior to calling functions and then for the SPI functions to consult that structure. Regards, Thomas Hallgren
Thomas Hallgren <thhal@mailblocks.com> writes:
> The Rationale for my opinion is that since there is a need to accomplish
> what Gaetano needs, there should be declarative power to express it and
> thus, prevent "unsafe" designs. We need a way to declare a function
> "stable with no _intrusive_ side effects".
What you think is non-intrusive may not be so at the database's level.
regards, tom lane
Tom, >What you think is non-intrusive may not be so at the database's level. > > I know. But thats not my point. Look at this this way: I'd like to declare a function STABLE. And I'd like to trust that declaration 100%. So a stable function must *never* call a function that is VOLATILE. Not directly and not implicit through nesting. I think we agree that the current way of enforcing that protection can't be trusted. As a function developer you really need to know what you are doing and take great care not to call a volatile function from within a stable or immutable function. The system won't protect you at all. My suggestion is first and foremost an attempt to enforce the procection and make the STABLE declaration really mean something so that all users can benefit from this and be able to rely on the concept. So far, no mention of non-intrusive. I'd really like your opinion on this part as a separate issue. Now, some people, like Gaetano, might want to go further and do things that are beyond what PostgreSQL can provide 100% protection for. They *want* to take on the responsability themselves. That's where my new function characteristic with "non-intrusive" comes in. I admitt that "non-intrusive" might be a bad term for this. What I mean is a characteristic that overrides my suggested 100% reliable interpretation of STABLE. This characteristic is not intended for the everyday function developer and should be documented as such. Regards, Thomas Hallgren
On Wednesday 03 November 2004 18:06, Thomas Hallgren wrote:
> Tom,
>
> >What you think is non-intrusive may not be so at the database's level.
>
> I know. But thats not my point. Look at this this way:
>
> I'd like to declare a function STABLE. And I'd like to trust that
> declaration 100%. So a stable function must *never* call a function that
> is VOLATILE. Not directly and not implicit through nesting.
>
> I think we agree that the current way of enforcing that protection can't
> be trusted. As a function developer you really need to know what you are
> doing and take great care not to call a volatile function from within a
> stable or immutable function. The system won't protect you at all.
>
I think the guidelines are fairly clear on what types of functions should be
declared with which types. But the key is that these are guidelines, not hard
and fast rules, since there may be times when you need to ignore them.
> My suggestion is first and foremost an attempt to enforce the procection
> and make the STABLE declaration really mean something so that all users
> can benefit from this and be able to rely on the concept. So far, no
> mention of non-intrusive. I'd really like your opinion on this part as a
> separate issue.
>
"users" shouldn't care. the function developer should determine the details
and "users" shouldn't have to think about it.
> Now, some people, like Gaetano, might want to go further and do things
> that are beyond what PostgreSQL can provide 100% protection for. They
> *want* to take on the responsability themselves. That's where my new
> function characteristic with "non-intrusive" comes in. I admitt that
> "non-intrusive" might be a bad term for this. What I mean is a
> characteristic that overrides my suggested 100% reliable interpretation
> of STABLE. This characteristic is not intended for the everyday function
> developer and should be documented as such.
>
Well, personally I prefered the way thing worked in 7.4, but I'm willing to
live with the 8.x method. If you forcibly prevent the work around though,
you better provide a work around, and if that mean a fourth function type I
could live with that; it's certainly better than marking these types of
functions volitile, which is a non-starter in my application.
--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
Robert, >I think the guidelines are fairly clear on what types of functions should be >declared with which types. But the key is that these are guidelines, not hard >and fast rules, since there may be times when you need to ignore them. > > In 7.4 they where indeed guidelines. In 8.x the semantics of the function attribute are meant to be enforced. I quote Tom from this thread: "You don't have a lot of room to complain because 8.0 started to enforce that.". >"users" shouldn't care. the function developer should determine the details >and "users" shouldn't have to think about it. > > The "user" in this context, is the user of the function attribute, i.e. the function developer. Sorry if I was unclear. >Well, personally I prefered the way thing worked in 7.4, but I'm willing to >live with the 8.x method. If you forcibly prevent the work around though, >you better provide a work around, and if that mean a fourth function type I >could live with that; it's certainly better than marking these types of >functions volitile, which is a non-starter in my application. > > Right, I also preferred 7.4 since I consider it less ambiguous then the current solution. But a much better solution is just around the corner and it would be unfortunate if some fairly rare scenarios are used as an argument to prevent it. Especially since those scenarios can be easily catered for by introducing a fourth type. Regards, Thomas Hallgren
Tom Lane wrote:> Thomas Hallgren <thhal@mailblocks.com> writes:>>>The Rationale for my opinion is that since there is a needto accomplish>>what Gaetano needs, there should be declarative power to express it and>>thus, prevent "unsafe" designs.We need a way to declare a function>>"stable with no _intrusive_ side effects".>>> What you think is non-intrusivemay not be so at the database's level.> Right, but the actual solution is far from be the good one. If you claim that an immutable function "must not" do update because otherwise the database could be in a inconsisten status, then we are in trouble permitting a non-immutable function to be called by an "immutable" one. I like see postgres stable as always was till now and I prefer seen my code completelly broken than see someone call a non-immutable function inside a "immutable" one and claim on this list that he lost data. I think a clean solution is enforce the check between functions call ( I prefer even only this one), and at the same time provide a "mutable" attribute for tables ( a mutable table can be updated even inside an immutable contest ). Regards Gaetano Mendola
Tom Lane wrote: > Gaetano Mendola <mendola@bigfoot.com> writes: > >>The fact that a non-volatile function can not perform >>update is a good improvement but on the other side will >>limit too much if I know what I'm doing. > > > I've got zero sympathy for this argument. It's been documented right > along that functions with side-effects must be marked volatile. You > don't have a lot of room to complain because 8.0 started to enforce that. > In practice you can circumvent the restriction by splitting the > function in two (ie, there is no check that a nonvolatile function > doesn't call any volatile functions). So if you insist on sticking > with an unsafe application design, you can do it with relatively > localized changes. I do not consider my design as "unsafe", this is for example how a cache works: expose a "read" without side effect but updating internal statistics. After all the read will not alter the data that it expose but other data that the user even don't know the existence. However I think that "that missing check" is "unsafe" and jeopardize the effort to avoid a wrong user design. Having say that I'm happy to know that what I did will continue to work splitting the function in two parts. Regards Gaetano Mendola