Thread: Rule updates and PQcmdstatus() issue
Hello all, PostgreSQL *still* has a bug where PQcmdStatus() won't return the number of rows updated. But that is essential for applications, since without it of course we don't know if the updates/delete/insert commands succeded. Even worst, on interfaces like Delphi/dbExpress the program will return an error message and rollback transaction thinking nothing have been updated. In other words, unusable. This render views useless (I either use view with rules and don't get my program working) and won't allow me to proper use security settings on PostgreSQL... This is a *major* issue in my opinion that appeared on a May thread but I can't see it done on version 7.2.2. Even worst, I can't see nothing on the TODO file. Will this fix finally appear on 7.3 ? Any ways to work around this ? How can I know at least if *something* succeeded, or how many rows (the proper behavior)? Thank you very much. ------------- Best regards,Steve Howe mailto:howe@carcass.dhs.org
Steve Howe wrote: > Hello all, > > PostgreSQL *still* has a bug where PQcmdStatus() won't return the > number of rows updated. But that is essential for applications, since > without it of course we don't know if the updates/delete/insert > commands succeded. Even worst, on interfaces like Delphi/dbExpress the > program will return an error message and rollback transaction thinking > nothing have been updated. In other words, unusable. > > This render views useless (I either use view with rules and don't get > my program working) and won't allow me to proper use security settings > on PostgreSQL... > > This is a *major* issue in my opinion that appeared on a May thread > but I can't see it done on version 7.2.2. Even worst, I can't see > nothing on the TODO file. > > Will this fix finally appear on 7.3 ? Any ways to work around this ? > How can I know at least if *something* succeeded, or how many rows > (the proper behavior)? I see on TODO: * Return proper effected tuple count from complex commands [return] and that "return" link has a discussion of possible fixes. Unfortunately, no fix was agreed upon so there is no fix in 7.3. And, on top of that, I can't even think of a workaround. At best, perhaps someone can write you a patch to fix this. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Hello Bruce, Friday, September 6, 2002, 3:22:13 PM, you wrote: BM> Steve Howe wrote: >> Hello all, >> >> PostgreSQL *still* has a bug where PQcmdStatus() won't return the >> number of rows updated. But that is essential for applications, since >> without it of course we don't know if the updates/delete/insert >> commands succeded. Even worst, on interfaces like Delphi/dbExpress the >> program will return an error message and rollback transaction thinking >> nothing have been updated. In other words, unusable. >> >> This render views useless (I either use view with rules and don't get >> my program working) and won't allow me to proper use security settings >> on PostgreSQL... >> >> This is a *major* issue in my opinion that appeared on a May thread >> but I can't see it done on version 7.2.2. Even worst, I can't see >> nothing on the TODO file. >> >> Will this fix finally appear on 7.3 ? Any ways to work around this ? >> How can I know at least if *something* succeeded, or how many rows >> (the proper behavior)? BM> I see on TODO: BM> * Return proper effected tuple count from complex commands [return] Sorry, I missed it because I check the v7.2.2 TODO. BM> and that "return" link has a discussion of possible fixes. BM> Unfortunately, no fix was agreed upon so there is no fix in 7.3. So all the databases that uses rules will still be broken ? I don't believe you guys are so unconcerned about this... BM> And, on top of that, I can't even think of a workaround. At best, BM> perhaps someone can write you a patch to fix this. Let's hope so... and I disagree about the 'write for me' point; it's for *everyone using rules*. They are useless, currently... and it's broken for months and nothing agreed until know... I just can't believe in it. What do you do when you have to update a view ? ------------- Best regards,Steve Howe mailto:howe@carcass.dhs.org
I am not any happier about it than you are. Your report is good because it is the first case where returning the wrong value actually breaks software. You may be able to justify adding a fix during beta by saying it is a bug fix. Of course, someone is going to have to generate a patch and champion the cause. This stuff doesn't happen by magic. --------------------------------------------------------------------------- Steve Howe wrote: > Hello Bruce, > > Friday, September 6, 2002, 3:22:13 PM, you wrote: > > BM> Steve Howe wrote: > >> Hello all, > >> > >> PostgreSQL *still* has a bug where PQcmdStatus() won't return the > >> number of rows updated. But that is essential for applications, since > >> without it of course we don't know if the updates/delete/insert > >> commands succeded. Even worst, on interfaces like Delphi/dbExpress the > >> program will return an error message and rollback transaction thinking > >> nothing have been updated. In other words, unusable. > >> > >> This render views useless (I either use view with rules and don't get > >> my program working) and won't allow me to proper use security settings > >> on PostgreSQL... > >> > >> This is a *major* issue in my opinion that appeared on a May thread > >> but I can't see it done on version 7.2.2. Even worst, I can't see > >> nothing on the TODO file. > >> > >> Will this fix finally appear on 7.3 ? Any ways to work around this ? > >> How can I know at least if *something* succeeded, or how many rows > >> (the proper behavior)? > > BM> I see on TODO: > > BM> * Return proper effected tuple count from complex commands [return] > Sorry, I missed it because I check the v7.2.2 TODO. > > BM> and that "return" link has a discussion of possible fixes. > BM> Unfortunately, no fix was agreed upon so there is no fix in 7.3. > So all the databases that uses rules will still be broken ? I don't > believe you guys are so unconcerned about this... > > BM> And, on top of that, I can't even think of a workaround. At best, > BM> perhaps someone can write you a patch to fix this. > Let's hope so... and I disagree about the 'write for me' point; it's > for *everyone using rules*. They are useless, currently... and it's > broken for months and nothing agreed until know... I just can't > believe in it. > What do you do when you have to update a view ? > > ------------- > Best regards, > Steve Howe mailto:howe@carcass.dhs.org > > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Hello Bruce, Friday, September 6, 2002, 9:52:18 PM, you wrote: BM> I am not any happier about it than you are. Your report is good because BM> it is the first case where returning the wrong value actually breaks BM> software. You may be able to justify adding a fix during beta by saying BM> it is a bug fix. Actually I think it must have happened with someone else, but they must have quit using rules or something... Actually I can't ensure security in the system without rules. BM> Of course, someone is going to have to generate a patch and champion the BM> cause. This stuff doesn't happen by magic. I understand your point. I just was hoping to see more concern about the issue by the developers... but that's been broken for months. Unhappily I can't do it myself because it would take weeks to get familiar with the inners of PostgreSQL... Let's hope someone realize how serious is this and make a fix. Thanks again... ------------- Best regards,Steve Howe mailto:howe@carcass.dhs.org
Steve Howe wrote: > Hello Bruce, > > Friday, September 6, 2002, 9:52:18 PM, you wrote: > > > BM> I am not any happier about it than you are. Your report is good because > BM> it is the first case where returning the wrong value actually breaks > BM> software. You may be able to justify adding a fix during beta by saying > BM> it is a bug fix. > Actually I think it must have happened with someone else, but they > must have quit using rules or something... > Actually I can't ensure security in the system without rules. > > BM> Of course, someone is going to have to generate a patch and champion the > BM> cause. This stuff doesn't happen by magic. > I understand your point. I just was hoping to see more concern about > the issue by the developers... but that's been broken for months. > > Unhappily I can't do it myself because it would take weeks to get > familiar with the inners of PostgreSQL... Well, there was a big discussion, and I did bring up the issue in early August to see if I could get a resolution to it and was told no conclusion could be made. I suggest you read the TODO detail on the item and make a proposal on how it _should_ work and if you can get agreement from everyone, you may be able to nag someone into doing a patch. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Hello Bruce, Friday, September 6, 2002, 10:58:13 PM, you wrote: BM> Well, there was a big discussion, and I did bring up the issue in early BM> August to see if I could get a resolution to it and was told no BM> conclusion could be made. BM> I suggest you read the TODO detail on the item and make a proposal on BM> how it _should_ work and if you can get agreement from everyone, you may BM> be able to nag someone into doing a patch. I think it should return the number of rows modified in the context of the view, and not exactly that of each of the tables affected. And this would not work well with PQcmdStatus() because it returns a single integer entry only. This was working on some previous build, wasn't it ? What was the previous behavior ? Shouldn't the patch follow that way ? ------------- Best regards,Steve Howe mailto:howe@carcass.dhs.org
Steve Howe <howe@carcass.dhs.org> writes: > BM> I suggest you read the TODO detail on the item and make a proposal on > BM> how it _should_ work and if you can get agreement from everyone, you may > BM> be able to nag someone into doing a patch. > I think it should return the number of rows modified in the context of > the view, and not exactly that of each of the tables affected. That's so vague as to be useless. What is "in the context of the view"? How does that notion help us resolve the uncertainties discussed in the TODO thread? > This was working on some previous build, wasn't it ? What was the > previous behavior ? Shouldn't the patch follow that way ? The old behavior was quite broken too, just not in a way that affected you. We will not be reverting the change that fatally broke it (namely altering the order of RULE applications for INSERTs) and so "go back to the old code" isn't a workable answer at all. I don't think fixing the code is the hard part; agreeing on what the behavior should be in complex cases is the hard part. regards, tom lane
Tom Lane wrote: > Steve Howe <howe@carcass.dhs.org> writes: > > BM> I suggest you read the TODO detail on the item and make a proposal on > > BM> how it _should_ work and if you can get agreement from everyone, you may > > BM> be able to nag someone into doing a patch. > > > I think it should return the number of rows modified in the context of > > the view, and not exactly that of each of the tables affected. > > That's so vague as to be useless. What is "in the context of the view"? > How does that notion help us resolve the uncertainties discussed in the > TODO thread? > > > This was working on some previous build, wasn't it ? What was the > > previous behavior ? Shouldn't the patch follow that way ? > > The old behavior was quite broken too, just not in a way that affected > you. We will not be reverting the change that fatally broke it (namely > altering the order of RULE applications for INSERTs) and so "go back > to the old code" isn't a workable answer at all. > > I don't think fixing the code is the hard part; agreeing on what the > behavior should be in complex cases is the hard part. Yes, Steve, if you want a fix, you better read the TODO detail and come up with a proposal and try to sell it to the group. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Hello Tom, Saturday, September 7, 2002, 5:42:33 PM, you wrote: TL> Steve Howe <howe@carcass.dhs.org> writes: >> BM> I suggest you read the TODO detail on the item and make a proposal on >> BM> how it _should_ work and if you can get agreement from everyone, you may >> BM> be able to nag someone into doing a patch. >> I think it should return the number of rows modified in the context of >> the view, and not exactly that of each of the tables affected. TL> That's so vague as to be useless. What is "in the context of the view"? TL> How does that notion help us resolve the uncertainties discussed in the TL> TODO thread? I just mean that PQcmdStatus() should not return a value for each changed table but how many rows "viewable by the view" it could change. Again, I'm not that aware of the inners of PostgreSQL to feel comfortable to do a better suggestion. >> This was working on some previous build, wasn't it ? What was the >> previous behavior ? Shouldn't the patch follow that way ? TL> The old behavior was quite broken too, just not in a way that affected TL> you. We will not be reverting the change that fatally broke it (namely TL> altering the order of RULE applications for INSERTs) and so "go back TL> to the old code" isn't a workable answer at all. I didn't mean to revert the code but to make it work like the older version did. I was unaware that it was broken too, but the removal now broke the whole views/rules so I wonder what could be worst... Also, it should have affected thousands of users, not just me. Unless nobody uses views... TL> I don't think fixing the code is the hard part; agreeing on what the TL> behavior should be in complex cases is the hard part. I understand your point and I'll try to give a proper solution but since I'm not familiar with the PostgreSQL inners, I wonder how good could it be... Thanks :) ------------- Best regards,Steve Howe mailto:howe@carcass.dhs.org
Steve Howe wrote: > > Hello all, > > PostgreSQL *still* has a bug where PQcmdStatus() won't return the > number of rows updated. But that is essential for applications, since > without it of course we don't know if the updates/delete/insert > commands succeded. Even worst, on interfaces like Delphi/dbExpress the > program will return an error message and rollback transaction thinking > nothing have been updated. In other words, unusable. > > This render views useless (I either use view with rules and don't get > my program working) and won't allow me to proper use security settings > on PostgreSQL... > > This is a *major* issue in my opinion that appeared on a May thread > but I can't see it done on version 7.2.2. Even worst, I can't see > nothing on the TODO file. > > Will this fix finally appear on 7.3 ? Any ways to work around this ? > How can I know at least if *something* succeeded, or how many rows > (the proper behavior)? And of course, in the case you insert into a real table you expect if a trigger procedure suppressed your original INSERT, but fired a cascade of other triggers by doing a mass UPDATE somewhere else instead, that all these caused UPDATEs and whatnot's are summed up and returned instead, right? Or what is proper behavior here? So please, "proper behavior" is not allways what your favorite tool expects. And just because you cannot "fix" your tool doesn't make that behavior any more "proper". Jan > > Thank you very much. > > ------------- > Best regards, > Steve Howe mailto:howe@carcass.dhs.org > > ---------------------------(end of broadcast)--------------------------- > TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
Steve Howe wrote: > > Hello Bruce, > > Friday, September 6, 2002, 9:52:18 PM, you wrote: > > BM> I am not any happier about it than you are. Your report is good because > BM> it is the first case where returning the wrong value actually breaks > BM> software. You may be able to justify adding a fix during beta by saying > BM> it is a bug fix. > Actually I think it must have happened with someone else, but they > must have quit using rules or something... > Actually I can't ensure security in the system without rules. > > BM> Of course, someone is going to have to generate a patch and champion the > BM> cause. This stuff doesn't happen by magic. > I understand your point. I just was hoping to see more concern about > the issue by the developers... but that's been broken for months. > > Unhappily I can't do it myself because it would take weeks to get > familiar with the inners of PostgreSQL... > > Let's hope someone realize how serious is this and make a fix. Seems you at least realized how serious it is. Even if you cannot code the "proper" solution, could you please make a complete table of all possible situations and the expected returns? With complete I mean including all combinations of rules, triggers, deferred constraints and the like. Or do you at least see now where in the discussion we got stuck? It doesn't help to cry for a quick hack that fixes your particular problem. That only leads to the situation that someday we have a final fix that changes the behavior for your case again and then you cry again and ask us not to break backwards compatibility. Thanks for your patience and understanding, Jan > > Thanks again... > ------------- > Best regards, > Steve Howe mailto:howe@carcass.dhs.org > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
Steve Howe wrote: > > Hello Bruce, > > Friday, September 6, 2002, 10:58:13 PM, you wrote: > > BM> Well, there was a big discussion, and I did bring up the issue in early > BM> August to see if I could get a resolution to it and was told no > BM> conclusion could be made. > > BM> I suggest you read the TODO detail on the item and make a proposal on > BM> how it _should_ work and if you can get agreement from everyone, you may > BM> be able to nag someone into doing a patch. > I think it should return the number of rows modified in the context of > the view, and not exactly that of each of the tables affected. And > this would not work well with PQcmdStatus() because it returns a > single integer entry only. > > This was working on some previous build, wasn't it ? What was the > previous behavior ? Shouldn't the patch follow that way ? In previous versions rules even fired in different orders. We cannot get back to that, because it was the reason for total failure of rules at all. So no, the patch should follow that way. You say that the return should be the rows modified in the context of the view. Er ... what is that? You mean only INSERTS, UPDATES and DELETES made by rule actions directly to any table referenced by the view itself count, not if a modification to another third table or view triggers back a modification to one of these base tables in return ... would that be through a rule or a trigger? What about a view over views, that has rules that in turn get rewritten by the rewrite rules of the views it consists of? What is that views context in detail? Jan > > ------------- > Best regards, > Steve Howe mailto:howe@carcass.dhs.org > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
> could you please make a complete table of all > possible situations and the expected returns? With complete I mean > including all combinations of rules, triggers, deferred constraints and > the like. Or do you at least see now where in the discussion we got > stuck? Imho only view rules (== instead rules) should affect the returned info. Not "non instead" rules, triggers or constraints. Those are imho supposed to be transparent as long as they don't abort the statement. Especially for triggers and constraints there is no room for flexibility, since other db's also don't modify the "affected rows" count for these. Think sqlca.sqlerrd[2] /* number of rows processed */! Andreas
Hello Jan, Monday, September 9, 2002, 11:26:20 AM, you wrote: JW> Steve Howe wrote: >> >> Hello Bruce, >> >> Friday, September 6, 2002, 9:52:18 PM, you wrote: >> >> BM> I am not any happier about it than you are. Your report is good because >> BM> it is the first case where returning the wrong value actually breaks >> BM> software. You may be able to justify adding a fix during beta by saying >> BM> it is a bug fix. >> Actually I think it must have happened with someone else, but they >> must have quit using rules or something... >> Actually I can't ensure security in the system without rules. >> >> BM> Of course, someone is going to have to generate a patch and champion the >> BM> cause. This stuff doesn't happen by magic. >> I understand your point. I just was hoping to see more concern about >> the issue by the developers... but that's been broken for months. >> >> Unhappily I can't do it myself because it would take weeks to get >> familiar with the inners of PostgreSQL... >> >> Let's hope someone realize how serious is this and make a fix. JW> Seems you at least realized how serious it is. Even if you cannot code "At least" ?... What do you mean by that ? JW> the "proper" solution, could you please make a complete table of all JW> possible situations and the expected returns? With complete I mean JW> including all combinations of rules, triggers, deferred constraints and JW> the like. Or do you at least see now where in the discussion we got JW> stuck? I had seen and the proposal was posted two days ago. JW> It doesn't help to cry for a quick hack that fixes your particular JW> problem. That only leads to the situation that someday we have a final JW> fix that changes the behavior for your case again and then you cry again JW> and ask us not to break backwards compatibility. See, I'm not crying. I'm just another user who needs something working. The whole problem was that the PostgreSQL knew the problem existed, had a brief discussion on the subject, and couldn't reach an agreement. That's ok for me, I understand... but releasing versions known to be broken is something I can't understand. ------------- Best regards,Steve Howe mailto:howe@carcass.dhs.org
Hello Jan, Monday, September 9, 2002, 11:26:20 AM, you wrote: JW> Seems you at least realized how serious it is. Even if you cannot code JW> the "proper" solution, could you please make a complete table of all JW> possible situations and the expected returns? With complete I mean JW> including all combinations of rules, triggers, deferred constraints and JW> the like. Or do you at least see now where in the discussion we got JW> stuck? By the way, I don't think triggers and constraints are in focus here, just as rules other then "DO INSTEAD". These should be transparent to the user. I suggest you to read the proposal posted to get aware of the discussion. Thanks. ------------- Best regards,Steve Howe mailto:howe@carcass.dhs.org
Hello Jan, Monday, September 9, 2002, 11:15:47 AM, you wrote: JW> Steve Howe wrote: >> >> Hello all, >> >> PostgreSQL *still* has a bug where PQcmdStatus() won't return the >> number of rows updated. But that is essential for applications, since >> without it of course we don't know if the updates/delete/insert >> commands succeded. Even worst, on interfaces like Delphi/dbExpress the >> program will return an error message and rollback transaction thinking >> nothing have been updated. In other words, unusable. >> >> This render views useless (I either use view with rules and don't get >> my program working) and won't allow me to proper use security settings >> on PostgreSQL... >> >> This is a *major* issue in my opinion that appeared on a May thread >> but I can't see it done on version 7.2.2. Even worst, I can't see >> nothing on the TODO file. >> >> Will this fix finally appear on 7.3 ? Any ways to work around this ? >> How can I know at least if *something* succeeded, or how many rows >> (the proper behavior)? JW> And of course, in the case you insert into a real table you expect if a JW> trigger procedure suppressed your original INSERT, but fired a cascade JW> of other triggers by doing a mass UPDATE somewhere else instead, that JW> all these caused UPDATEs and whatnot's are summed up and returned JW> instead, right? Or what is proper behavior here? What is documented, and what is expected: PQcmdStatus(), PQcmdTuples()and PQoidValue() returning the information they should be. JW> So please, "proper behavior" is not allways what your favorite tool JW> expects. And just because you cannot "fix" your tool doesn't make that JW> behavior any more "proper". Do you have any word more appropriate ? And just so that you know, I can't "fix" my tool because I have other job to do (and a lot of that and that job uses PostgreSQL), and unhappily I couldn't join the development team and thus I'm not aware of how it works internally. The reason isn't that I just don't have intellectual capacity. And it looks like *you* overhauled the query rewrite rule system, so what we are talking is something that must have passed through you. So instead of offending me, your "proper" behavior would be try to help and suggest a solution for the problem, as other developers are doing. Thanks again. ------------- Best regards,Steve Howe mailto:howe@carcass.dhs.org
> existed, had a brief discussion on the subject, and couldn't reach an > agreement. That's ok for me, I understand... but releasing versions > known to be broken is something I can't understand.-9' the postmaster If we didn't do that, then Postgresql never would have been released in the first place, nor any date between then and now. There has been, and currently is a ton of known broken, wonky, or incomplete stuff -- but it's felt that the current version has a lot more to offer than the previous version, so it's being released. This works for *all* software. If you never release, nothing gets better. I suspect it'll be several more major releases before we begin to consider it approaching completely functional. -- Rod Taylor
On Mon, 9 Sep 2002, Steve Howe wrote: > JW> Steve Howe wrote: > >> > >> Hello all, > >> > >> PostgreSQL *still* has a bug where PQcmdStatus() won't return the > >> number of rows updated. But that is essential for applications, since > >> without it of course we don't know if the updates/delete/insert > >> commands succeded. Even worst, on interfaces like Delphi/dbExpress the > >> program will return an error message and rollback transaction thinking > >> nothing have been updated. In other words, unusable. As a note, I assume you realize that it returning any number doesn't guarantee that the command succeeded if you assume succeeding means doing what the statement sent would appear to do. ;) Although I think we need to change the current behavior, we are turning a false "failure" into a potentially false "success" (I did an update, it said two rows were changed but there's no visible data change in the entire system?) Fortunately, the likely bad effects from the false "success" are probably only going to happen in somewhat degenerate cases. I quote "failure" and "success" because there's already a notion of success and failure which is raising an exception condition or not (AFAICT 0 rows is a completion condition - the statement succeeded but nothing was modified). As such, using the count to determine success of the statement is wrong for an interface, but it may be meaningful for applications attempting to apply some sort of business logic.
> -----Original Message----- > From: Rod Taylor [mailto:rbt@rbt.ca] > Sent: Monday, September 09, 2002 10:55 AM > To: Steve Howe > Cc: PostgreSQL-development > Subject: Re: [HACKERS] Rule updates and PQcmdstatus() issue > > > > existed, had a brief discussion on the subject, and > couldn't reach an > > agreement. That's ok for me, I understand... but releasing versions > > known to be broken is something I can't understand. > -9' the postmaster > > If we didn't do that, then Postgresql never would have been > released in the first place, nor any date between then and now. > > There has been, and currently is a ton of known broken, > wonky, or incomplete stuff -- but it's felt that the current > version has a lot more to offer than the previous version, so > it's being released. > > This works for *all* software. If you never release, nothing > gets better. > > > I suspect it'll be several more major releases before we > begin to consider it approaching completely functional. I believe that the surprise is at the focus, when it comes to a release. With commercial products (anyway) if you have any sort of show-stopper bug (crashing, incorrect results, etc.) you do not release the tool until the bug, and all others like it, are fixed. Bugs that have to do with appearance or convenience can be overlooked for a release as long as they are documented in the release notes. Now, it is not unlikely that there are unintentional show-stopper bugs that get through Q/A. But intentionally passing them through would be incompetent for a commercial enterprise. With open source projects, the empasis tends to be on features, with far less regard for correcting known problems. Even bugs that can cause a crash seem to be viewed as acceptable if they happen rarely. Now, at first blush, the open source strategy seems ludicrous. After all, who will want to use a product which could potentially (albeit unlikely) destroy your data or give wrong results? Then, after a bit of thought, you can see that the same sort of strategy as the open source projects *is* followed by one very large and very successful software giant. So maybe "burgeoning featuritis without extreme concern for robust stability" isn't such a stupid strategy after all. ;-) All kidding aside, I would like to see increased emphasis on stability and correctness. But I will admit that it is a lot less fun than adding new features.
On Mon, Sep 09, 2002 at 11:30:52AM -0700, Dann Corbit wrote: > > All kidding aside, I would like to see increased emphasis on stability > and correctness. But I will admit that it is a lot less fun than adding > new features. But in fairness, I think you'd be hard pressed to find a set of developers anywhere who take more seriously that the PostgreSQL core the responsibility to provide stable, correct software. I've reported show-stopping bugs to commercial database providers and on the PostgreSQL lists, and I'd be hard pressed to come up with an occasion where I received from a commercial software company service that was even 1/10th the quality and speed that I get here. A -- ---- Andrew Sullivan 204-4141 Yonge Street Liberty RMS Toronto, Ontario Canada <andrew@libertyrms.info> M2P 2A8 +1 416 646 3304 x110
On Mon, Sep 09, 2002 at 11:30:52AM -0700, Dann Corbit wrote: > > > > I suspect it'll be several more major releases before we > > begin to consider it approaching completely functional. > > I believe that the surprise is at the focus, when it comes to a release. > With commercial products (anyway) if you have any sort of show-stopper > bug (crashing, incorrect results, etc.) you do not release the tool > until the bug, and all others like it, are fixed. Bugs that have to do > with appearance or convenience can be overlooked for a release as long > as they are documented in the release notes. Now, it is not unlikely > that there are unintentional show-stopper bugs that get through Q/A. > But intentionally passing them through would be incompetent for a > commercial enterprise. Hmm, you don't have any drinking buddies who work QA, do you? _Lots_ of known, "eat your harddrive" bugs get classified as "to be fixed in future release" in commercial software, when the release date pressure grows. > With open source projects, the empasis tends to be on features, with far > less regard for correcting known problems. Even bugs that can cause a > crash seem to be viewed as acceptable if they happen rarely. Huh? I tend to see exactly the opposite. Actual crash and "wrong answer" bugs tend to get very prompt attention on all the open source projects I know and use. What _does_ get delayed or even ignored are "bug compability" problems, like this one. That is, software that relies on the "affected rows" count is in fact broken, since it's making assumptions about that number that were never promised in any standard or interface docs. <snip silly comparison to commercial software house> > All kidding aside, I would like to see increased emphasis on stability > and correctness. But I will admit that it is a lot less fun than adding > new features. And this has got to be trolling: PostgreSQL is one of the _most_ stability and correctness focused software projects I've ever known. In this particular case, the complaints about this issue where "Your bugfix broke my tool! make it better!" The answer was "We can't just put it back, that's an actual bug in there (rules firing in an unpredicatable order). What's the _correct_ behavior?" The people with the complaints then did not come up with a compelling, complete description of what the correct behavior should be. There's always been vague parts to the "desired behavior" like the phrase Tom pointed out: "in the context of the view" which was clarified to mean "viewable by the view", which is nearly impossible to code, if not an example of the halting problem. PostgreSQL as a project errs on the side of not coding the quick fix, in favor of waiting for the right answer. Sometimes too long, but this case isn't one of those, IMHO. Ross -- Ross Reedstrom, Ph.D. reedstrm@rice.edu Executive Director phone: 713-348-6166 Gulf Coast Consortium for Bioinformatics fax: 713-348-6182 Rice University MS-39 Houston, TX 77005
> > If we didn't do that, then Postgresql never would have been > > released in the first place, nor any date between then and now. > I believe that the surprise is at the focus, when it comes to a release. > With commercial products (anyway) if you have any sort of show-stopper > bug (crashing, incorrect results, etc.) you do not release the tool Most companies / groups (opensource or otherwise) will not hold back many bugfixes and features for the sake of getting an additional out of the way fix in as it tends to piss off the majority of the users. I'm afraid right now I see this as a very minor item which is heavily broken, meaning it's really really important to very few users. Not having foreign keys break when renaming a column or table will probably affect more people and is awaiting the next release. Ditto for security enhancements. I see these as more important -- since they affect me :) If the changes are agreed upon and fixed, great. It's a better product because of it. But forcing others to use an older version with equivelently broken items because the next one doesn't do everything perfectly doesn't make for progress. However, rest assured, with anything if you push and put in the work require it'll eventually go where you want it to. -- Rod Taylor
> -----Original Message----- > From: Ross J. Reedstrom [mailto:reedstrm@rice.edu] > Sent: Monday, September 09, 2002 12:26 PM > To: Dann Corbit > Cc: Rod Taylor; Steve Howe; PostgreSQL-development > Subject: Re: [HACKERS] Rule updates and PQcmdstatus() issue > > > On Mon, Sep 09, 2002 at 11:30:52AM -0700, Dann Corbit wrote: > > > > > > I suspect it'll be several more major releases before we > > > begin to consider it approaching completely functional. > > > > I believe that the surprise is at the focus, when it comes to a > > release. With commercial products (anyway) if you have any sort of > > show-stopper bug (crashing, incorrect results, etc.) you do not > > release the tool until the bug, and all others like it, are fixed. > > Bugs that have to do with appearance or convenience can be > overlooked > > for a release as long as they are documented in the release notes. > > Now, it is not unlikely that there are unintentional > show-stopper bugs > > that get through Q/A. But intentionally passing them > through would be > > incompetent for a commercial enterprise. > > Hmm, you don't have any drinking buddies who work QA, do you? I do have friends who work in Q/A. > _Lots_ of known, "eat your harddrive" bugs get classified as > "to be fixed in future release" in commercial software, when > the release date pressure grows. I have been programming since 1976 on literally many dozens of projects. There is no project on which I have been a part where such a thing would be allowed. On the other hand, the projects I tend to work on are the "these tools are used to run your business" MIS sorts of things. Perhaps other areas of development are different. > > With open source projects, the emphasis tends to be on > features, with > > far less regard for correcting known problems. Even bugs that can > > cause a crash seem to be viewed as acceptable if they happen rarely. > > Huh? I tend to see exactly the opposite. Actual crash and > "wrong answer" bugs tend to get very prompt attention on all > the open source projects I know and use. What _does_ get > delayed or even ignored are "bug compability" problems, like > this one. That is, software that relies on the "affected > rows" count is in fact broken, since it's making assumptions > about that number that were never promised in any standard or > interface docs. If this particular case is a case of someone relying on undocumented behavior, then there is no bug. If this is a case of relying upon documented behavior and the behavior changes, then there is a bug. > <snip silly comparison to commercial software house> > > > All kidding aside, I would like to see increased emphasis > on stability > > and correctness. But I will admit that it is a lot less fun than > > adding new features. > > And this has got to be trolling: PostgreSQL is one of the > _most_ stability and correctness focused software projects > I've ever known. There are very serious problems that have been in the release notes for a very long time and yet have never been addressed. Most of them are rather esoteric and won't affect most users. I have been on many projects that were far more concerned with correctness. As I have said, "no serious bugs are allowed in a release" is not uncommon on the commercial projects where I have experience. That includes 9 years as a subcontractor at Microsoft. If they have a serious bug that cannot be fixed, they will simply cut scope. But my experience was on MS (ITG) projects. Perhaps other branches of MS did not require the same rigor. On the other hand, PostgreSQL is more responsive in this area than any other open source project that I know of. > In this particular case, the complaints > about this issue where "Your bugfix broke my tool! make it > better!" The answer was "We can't just put it back, that's an > actual bug in there (rules firing in an unpredicatable > order). What's the _correct_ behavior?" The people with the > complaints then did not come up with a compelling, complete > description of what the correct behavior should be. There's > always been vague parts to the "desired behavior" like the > phrase Tom pointed out: "in the context of the view" which > was clarified to mean "viewable by the view", which is nearly > impossible to code, if not an example of the halting problem. This may be an example where the original poster is asking for something they should not be asking for. If the original poster was relying upon undocumented behavior, then there is nothing that needs to be done, and the resulting problem is the original poster's fault. > PostgreSQL as a project errs on the side of not coding the > quick fix, in favor of waiting for the right answer. > Sometimes too long, but this case isn't one of those, IMHO. You are probably right about this case. In fact, I am not defending the original poster's demand. I have no idea if their request has merit or not. I was merely expressing an opinion that a good standard to follow is to fix all outstanding showstopper bugs before making a release. Nothing more, nothing less. I am not attacking the PostgreSQL project or team. In fact, I think it is the finest piece of open source, freely available software on the planet. My request was more of an aside -- simply wishing out loud for an intense focus on fixing problems.
Steve Howe wrote: > > Hello Jan, > > Monday, September 9, 2002, 11:15:47 AM, you wrote: > > JW> So please, "proper behavior" is not allways what your favorite tool > JW> expects. And just because you cannot "fix" your tool doesn't make that > JW> behavior any more "proper". > Do you have any word more appropriate ? > [...] > And it looks like *you* overhauled the query rewrite rule system, so > what we are talking is something that must have passed through you. So > instead of offending me, your "proper" behavior would be try to help > and suggest a solution for the problem, as other developers are doing. See, and exactly here lies the problem. Indeed, I spent about 3 months of my spare time back in 95 or so to fix it, after I spent many more months over years to get familiar with the internals. Now, instead of even trying to spend some serious amount of time yourself, you give some vague hints about the functionality that might make your problems disappear, name that a proposal and expect someone else to do what you need for free. This is not exactly how open source works. We should surely keep this on a much more technical level and avoid any personal offendings. To do so, please explain to me why you think that triggers and constraints are out of focus here? What is the difference between a trigger, a rule and an instead rule from a business process oriented point of view? I think there is none at all. They are just different techniques to do one and the same, implement business logic in the database system. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
Hello Jan, Monday, September 9, 2002, 4:56:04 PM, you wrote: JW> Steve Howe wrote: >> >> Hello Jan, >> >> Monday, September 9, 2002, 11:15:47 AM, you wrote: >> >> JW> So please, "proper behavior" is not allways what your favorite tool >> JW> expects. And just because you cannot "fix" your tool doesn't make that >> JW> behavior any more "proper". >> Do you have any word more appropriate ? >> [...] >> And it looks like *you* overhauled the query rewrite rule system, so >> what we are talking is something that must have passed through you. So >> instead of offending me, your "proper" behavior would be try to help >> and suggest a solution for the problem, as other developers are doing. JW> See, and exactly here lies the problem. Indeed, I spent about 3 months JW> of my spare time back in 95 or so to fix it, after I spent many more JW> months over years to get familiar with the internals. JW> Now, instead of even trying to spend some serious amount of time JW> yourself, you give some vague hints about the functionality that might JW> make your problems disappear, name that a proposal and expect someone JW> else to do what you need for free. This is not exactly how open source JW> works. As I told you, this would demand weeks and I just don't have time to do it. Other developers offered to make a fix and asked me to do that proposal. And so I did. It's sad that just you don't seem to be trying to help in any way. Other developers had considered the proposal and are actually voting and giving constructive ideas on the subject. JW> We should surely keep this on a much more technical level and avoid any JW> personal offendings. To do so, please explain to me why you think that JW> triggers and constraints are out of focus here? What is the difference JW> between a trigger, a rule and an instead rule from a business process JW> oriented point of view? I think there is none at all. They are just JW> different techniques to do one and the same, implement business logic in JW> the database system. Because the affected commands are supposed to give you back information on what your INSERT/UPDATE/DELETE commands, not what is making behind the scenes. And it seems that other people in the thread agree with me, please read thread. Since you are probably very familiar with the rules system, why don't you vote on a proposal too, or just suggest yours. Your opinion is very important. I'm not saying I'm the truth owner; I'm just another developer who needs a feature working again. Thank you. ------------- Best regards,Steve Howe mailto:howe@carcass.dhs.org
Actually, this problem is part of a whole scope of problems that were in the Berkeley code, because rules, and inheritance, just have a certain contorting effect on SQL queries where it is difficult to get them working properly. If these features didn't come from Berkeley, I doubt we would have ever implemented them, so in some case there are inherited bugs from features that weren't 100% thought out when they were added many years ago. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Jan Wieck wrote: > We should surely keep this on a much more technical level and avoid any > personal offendings. To do so, please explain to me why you think that > triggers and constraints are out of focus here? What is the difference > between a trigger, a rule and an instead rule from a business process > oriented point of view? I think there is none at all. They are just > different techniques to do one and the same, implement business logic in > the database system. All the problems here are coming from INSTEAD rules. We don't have INSTEAD triggers or contraints. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Steve Howe wrote: > Because the affected commands are supposed to give you back > information on what your INSERT/UPDATE/DELETE commands, not what is > making behind the scenes. > > And it seems that other people in the thread agree with me, please > read thread. > > Since you are probably very familiar with the rules system, why don't > you vote on a proposal too, or just suggest yours. Your opinion is > very important. I'm not saying I'm the truth owner; I'm just another > developer who needs a feature working again. Jan actually did vote in the first round which appears in TODO.detail. He voted that if the INSTEAD rule had only _one_ statement, return that, if not, return nothing. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
On Mon, 9 Sep 2002, Bruce Momjian wrote: > Jan Wieck wrote: > > We should surely keep this on a much more technical level and avoid any > > personal offendings. To do so, please explain to me why you think that > > triggers and constraints are out of focus here? What is the difference > > between a trigger, a rule and an instead rule from a business process > > oriented point of view? I think there is none at all. They are just > > different techniques to do one and the same, implement business logic in > > the database system. > > All the problems here are coming from INSTEAD rules. We don't have > INSTEAD triggers or contraints. Sure we do, well sort of. :) Make a before trigger that does a different statement and returns NULL to abort the original action on that row.
On Mon, 2002-09-09 at 22:11, Bruce Momjian wrote: > Jan Wieck wrote: > > We should surely keep this on a much more technical level and avoid any > > personal offendings. To do so, please explain to me why you think that > > triggers and constraints are out of focus here? What is the difference > > between a trigger, a rule and an instead rule from a business process > > oriented point of view? I think there is none at all. They are just > > different techniques to do one and the same, implement business logic in > > the database system. > > All the problems here are coming from INSTEAD rules. We don't have > INSTEAD triggers or contraints. Well.. Triggers could be exclusively INSTEAD. A trigger could easily write a few things to a number of other tables, and return NULL in a BEFORE trigger which would prevent execution of the requested command. -- Rod Taylor
Hello Bruce, Monday, September 9, 2002, 11:13:20 PM, you wrote: BM> Steve Howe wrote: >> Because the affected commands are supposed to give you back >> information on what your INSERT/UPDATE/DELETE commands, not what is >> making behind the scenes. >> >> And it seems that other people in the thread agree with me, please >> read thread. >> >> Since you are probably very familiar with the rules system, why don't >> you vote on a proposal too, or just suggest yours. Your opinion is >> very important. I'm not saying I'm the truth owner; I'm just another >> developer who needs a feature working again. BM> Jan actually did vote in the first round which appears in TODO.detail. BM> He voted that if the INSTEAD rule had only _one_ statement, return that, BM> if not, return nothing. We still need Tom's word and Hiroshi, since they were the most related to the subject, and the other developer's opinion... :) ------------- Best regards,Steve Howe mailto:howe@carcass.dhs.org
> What is the difference > between a trigger, a rule and an instead rule from a business process > oriented point of view? I think there is none at all. They are just > different techniques to do one and the same, implement > business logic in the database system. The difference is how other db's work. They all ignore triggers and constraints in the sqlca.sqlerrd[2] "number of processed rows" count, that I see identical to our affected rows count. They all have views, but not many have rules :-) Pg's "instead rules" are the toolkit for views, and as such need special handling, imho. Andreas
On Mon, 2002-09-09 at 21:25, Ross J. Reedstrom wrote: > And this has got to be trolling: PostgreSQL is one of the _most_ > stability and correctness focused software projects I've ever known. In > this particular case, the complaints about this issue where "Your bugfix > broke my tool! make it better!" The answer was "We can't just put it > back, that's an actual bug in there (rules firing in an unpredicatable > order). Why is "rules firing in an unpredicatable order" a bug but "returned affected tuple count is wrong " just a compatibility issue ? Afaik, rule firing order has never been promised, while pqCmdTuples() has. > What's the _correct_ behavior?" The people with the complaints > then did not come up with a compelling, complete description of what > the correct behavior should be. There's always been vague parts to the > "desired behavior" like the phrase Tom pointed out: "in the context of > the view" which was clarified to mean "viewable by the view", which is > nearly impossible to code, if not an example of the halting problem. One approach could be to expose the tuple count at SQL level and then let the user decide what to return. > PostgreSQL as a project errs on the side of not coding the quick fix, > in favor of waiting for the right answer. Sometimes too long, but this > case isn't one of those, IMHO. You usually learn afterwards when it has been "too long" ;) -------------- Hannu
Stephan Szabo <sszabo@megazone23.bigpanda.com> writes: > On Mon, 9 Sep 2002, Bruce Momjian wrote: >> All the problems here are coming from INSTEAD rules. We don't have >> INSTEAD triggers or contraints. > Sure we do, well sort of. :) > Make a before trigger that does a different statement and returns NULL > to abort the original action on that row. I think we can reasonably leave the side-effects of triggers out of the discussion. PQcmdStatus numbers have never included side-effects of triggers in the past, and I see no reason for them to start now. I think it's reasonable to exclude both triggers and non-INSTEAD rules from the status count, on the grounds that these normally represent "add-on" actions and not the "real" action. The cases that get interesting are those that involve multiple INSTEAD actions (either from multiple INSTEAD rules, or a single rule with multiple commands in its body) and those cases where the INSTEAD action is a different type from the original command (eg, ON UPDATE DO INSTEAD INSERT...). regards, tom lane
Hannu Krosing <hannu@tm.ee> writes: > Why is "rules firing in an unpredicatable order" a bug but "returned > affected tuple count is wrong " just a compatibility issue ? > Afaik, rule firing order has never been promised, while pqCmdTuples() > has. There has never been any spec saying exactly what PQcmdTuples would give in complicated cases. The effective behavior pre-7.2 was that you'd get the result tag of the last action executed, but this was undocumented, and unsafe to rely on in multi-rule cases even then, considering that the firing order of rules was not predictable. What actually happened was this: in 7.2 we changed ON INSERT rule firing to execute non-INSTEAD rules after the original INSERT, rather than before it. In the old behavior, non-INSTEAD rules just plain did not work with an INSERT: they wouldn't see any "NEW" row, because it wasn't there yet when they ran. This is surely a bug fix in my book (and it is unrelated to the 7.3 change that provides predictable firing order of multiple rules). Now the side effect of that change was to change PQcmdTuples' behavior, because the "last action" was no longer the same thing as before. This broke various clients that were depending on the "last action" to be the original INSERT. The fix we applied was to redefine PQcmdTuples to return the result count of the original query regardless of firing order. This behavior is evidently not good for Steve, and I'm perfectly prepared to discuss modifying it some more --- but I don't want to have a PQcmdTuples behavior-of-the-month with new changes in every release. I want a discussed, agreed-to, well-defined behavior that we aren't going to revisit again in future releases. When we have that agreement we can implement it and forget it ... but if we apply a bandaid now and then change the behavior again later, we're just going to make life even harder for clients. I'd rather leave the behavior broken (by Steve's view anyway) but *the same as 7.2* than have a new but still- unsatisfactory definition in there for 7.3. I think the other developers have the same negative opinion about API churn as I do, and so when we couldn't get agreement about what to do back in May, we shelved the topic in hopes a fresh idea would come along. Now could we drop the name-calling and the bogus opinionating about how serious or not-serious this problem is, and concentrate on finding a satisfactory answer? regards, tom lane
On Tue, 10 Sep 2002, Tom Lane wrote: > Stephan Szabo <sszabo@megazone23.bigpanda.com> writes: > > On Mon, 9 Sep 2002, Bruce Momjian wrote: > >> All the problems here are coming from INSTEAD rules. We don't have > >> INSTEAD triggers or contraints. > > > Sure we do, well sort of. :) > > Make a before trigger that does a different statement and returns NULL > > to abort the original action on that row. > > I think we can reasonably leave the side-effects of triggers out of the > discussion. PQcmdStatus numbers have never included side-effects of > triggers in the past, and I see no reason for them to start now. > I agree, I was just commenting on the instead trigger comment.
Bruce Momjian wrote: > > Jan Wieck wrote: > > We should surely keep this on a much more technical level and avoid any > > personal offendings. To do so, please explain to me why you think that > > triggers and constraints are out of focus here? What is the difference > > between a trigger, a rule and an instead rule from a business process > > oriented point of view? I think there is none at all. They are just > > different techniques to do one and the same, implement business logic in > > the database system. > > All the problems here are coming from INSTEAD rules. We don't have > INSTEAD triggers or contraints. So a BEFORE INSERT trigger on table1 that does an UPDATE to table2 and then returns NULL is not effectively the same as an ON INSERT ... DO INSTEAD UPDATE ... rule? Hmmm, the end result is exactly the same so what do you call it? I think we will have no chance to really return the number of VIEW-tuples affected. So any implementation is only a guess and we could simply return fixed 42 if "some" tuples where affected at all. This return is as wrong (according to Steve) as everything else but at least we have a clear definition what it means. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
Jan Wieck wrote: > Bruce Momjian wrote: > > > > Jan Wieck wrote: > > > We should surely keep this on a much more technical level and avoid any > > > personal offendings. To do so, please explain to me why you think that > > > triggers and constraints are out of focus here? What is the difference > > > between a trigger, a rule and an instead rule from a business process > > > oriented point of view? I think there is none at all. They are just > > > different techniques to do one and the same, implement business logic in > > > the database system. > > > > All the problems here are coming from INSTEAD rules. We don't have > > INSTEAD triggers or contraints. > > So a BEFORE INSERT trigger on table1 that does an UPDATE to table2 and > then returns NULL is not effectively the same as an ON INSERT ... DO > INSTEAD UPDATE ... rule? Hmmm, the end result is exactly the same so > what do you call it? Well, yes, functionally it is the same and we would have trouble dealing with that too. I didn't know you could NULL return from a trigger and it would exit the statement. > I think we will have no chance to really return the number of > VIEW-tuples affected. So any implementation is only a guess and we could > simply return fixed 42 if "some" tuples where affected at all. This > return is as wrong (according to Steve) as everything else but at least > we have a clear definition what it means. Yes, my guess is that accumulating everything with the same tags is the closest we are going to get and does return the proper values in simple multi-statement INSTEAD rules. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Jan Wieck writes: > I think we will have no chance to really return the number of > VIEW-tuples affected. So any implementation is only a guess and we could > simply return fixed 42 if "some" tuples where affected at all. This > return is as wrong (according to Steve) as everything else but at least > we have a clear definition what it means. Maybe we should return something to the effect of "unknown, but something happened". I can see that returning 0 in case of doubt might confuse applications. -- Peter Eisentraut peter_e@gmx.net