Thread: Re: [GENERAL] +/- Inf for float8's
Gah, typo'ed the name of pgsql-hackers. This should be better. Sorry to those who got this twice, once on GENERAL, once on HACKERS. Ross On Mon, Aug 14, 2000 at 02:33:55PM +1000, Tim Allen wrote: > I'm just trying out PG7.0.2, with a view to upgrading from 6.5.3, and I've > found one quirk a little troublesome. Not sure whether I'll get any > sympathy, but I shall ask anyway :). > > We find it convenient to be able to store +/- infinity for float8 values > in some database tables. With Postgres 6.5.3, we were able to get away > with this by using the values -1.79769313486232e+308 for -Inf and > 1.79769313486232e+308 for Inf. This is probably not very portable, but > anyway, it worked fine for us, on both x86 Linux and SGI IRIX. One thing, > though, to get these numbers past the interface we had to put them in > quotes. It seemed as though there was one level of parsing that didn't > like these particular numbers, and one level of parsing that coped OK, and > using quotes got it past the first level. > > Now, however (unfortunately for us), this inconsistency in the interface > has been "fixed", and now we can't get this past the interface, either > quoted or not. Fixing inconsistencies is, of course, in general, a good > thing, which is why I'm not confident of getting much sympathy :). > Breaking working apps is never a good thing, but that's part of why it went from 6.X to 7.X. > So, any suggestions as to how we can store +/- infinity as a valid float8 > value in a database table? > Right: the SQL standard doesn't say anything about what to do for these cases for floats (except by defining the syntax of an approximate numeric constant as basically a float), but the IEEE754 does: as you discovered below, they're NaN, -Infinity, and +Infinity. > I notice, btw, that 'NaN' is accepted as a valid float8. Is there any > particular reason why something similar for, eg '-Inf' and 'Inf' doesn't > also exist? Just discovered, there is a special number 'Infinity', which > seems to be recognised, except you can't insert it into a table because it > reports an overflow error. Getting warm, it seems, but not there yet. And > there doesn't seem to be a negative equivalent. And this is a bug. From looking at the source, I see that Thomas added code to accept 'NaN' and 'Infinity' (but not '-Infinity'), and Tom Lane tweaked it, but it's never been able to get an Infinity all the way to the table, as far as I can see: the value gets set to HUGE_VAL, but the call to CheckFloat8Val compares against FLOAT8_MAX (and FLOAT8_MIN), and complains, since HUGE_VAL is _defined_ to be larger than DBL_MAX. And, there's no test case in the regression tests for inserting NaN or Infinity. (Shame on Thomas ;-) I think the right thing to do is move the call to CheckFloat8Val into a branch of the test for NaN and Infinity, thereby not calling it if we've been passed those constants. I'm compiling up a test of this right now, and I'll submit a patch to Bruce if it passes regression. Looks like that function hasn't been touch in a while, so the patch should apply to 7.0.X as well as current CVS. <some time later> Looks like it works, and passes the regression tests as they are. I'm patching the tests to include the cases 'NaN', 'Infinity', and '-Infinity' as valid float8s, and 'not a float' as an invalid representation, and rerunning to get output to submit with the patch. This might be a bit hairy, since there are 5 different expected/float8* files. Should I try to hand patch them to deal with the new rows, or let them be regenerated by people with the appropriate platforms? <later again> Bigger problem with changing the float8 regression tests: a lot of our math functions seem to be guarded with CheckFloat8Val(result), so, if we allow these values in a float8 column, most of the math functions with elog(). It strikes me that there must have been a reason for this at one time. There's even a #define UNSAFE_FLOATS, to disable these checks. By reading the comments in old copies of float.c, it looks like this was added for an old, buggy linux/Alpha libc that would throw floating point exceptions, otherwise. Is there an intrinsic problem with allowing values outside the range FLOAT8_MAX <= x =>FLOAT8_MIN ? 'ORDER BY' seems to still work, with 'Infinity' and '-Infinity' sorting properly. Having a 'NaN' in there breaks sorting however. That's a current, live bug. Could be fixed by treating 'NaN' as a different flavor of NULL. Probably a fairly deep change, however. Hmm, NULL in a float8 sorts to the end, regardless of ASC or DESC, is that right? Anyway, here's the patch for just float.c , if anyone wants to look at it. As I said, it passes the existing float8 regression tests, but raises a lot of interesting questions. Ross -- Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> NSBRI Research Scientist/Programmer Computer and Information Technology Institute Rice University, 6100 S. Main St., Houston, TX 77005
Attachment
> > So, any suggestions as to how we can store +/- infinity as a valid float8 > > value in a database table? > Right: the SQL standard doesn't say anything about what to do for these > cases for floats (except by defining the syntax of an approximate numeric > constant as basically a float), but the IEEE754 does: as you discovered > below, they're NaN, -Infinity, and +Infinity. Not all computers fully support IEEE754, though many new ones do. > > I notice, btw, that 'NaN' is accepted as a valid float8. Is there any > > particular reason why something similar for, eg '-Inf' and 'Inf' doesn't > > also exist? Just discovered, there is a special number 'Infinity', which > > seems to be recognised, except you can't insert it into a table because it > > reports an overflow error. Getting warm, it seems, but not there yet. And > > there doesn't seem to be a negative equivalent. > And this is a bug. From looking at the source, I see that Thomas added > code to accept 'NaN' and 'Infinity' (but not '-Infinity'), and Tom Lane > tweaked it, but it's never been able to get an Infinity all the way to > the table, as far as I can see: the value gets set to HUGE_VAL, but the > call to CheckFloat8Val compares against FLOAT8_MAX (and FLOAT8_MIN), > and complains, since HUGE_VAL is _defined_ to be larger than DBL_MAX. > And, there's no test case in the regression tests for inserting NaN or > Infinity. (Shame on Thomas ;-) Ah, I'm just trying to leave some rewarding work for other folks ;) > I think the right thing to do is move the call to CheckFloat8Val into a > branch of the test for NaN and Infinity, thereby not calling it if we've > been passed those constants. I'm compiling up a test of this right now, > and I'll submit a patch to Bruce if it passes regression. Looks like > that function hasn't been touch in a while, so the patch should apply > to 7.0.X as well as current CVS. istm that the existing protection (or something like it) is required for some platforms, while other platforms may be able to handle NaN and +/-Inf just fine. Seems like a job for autoconf to determine the FP capabilities of a system, unless Posix defines some way to tell. Of course, even then we'd need an autoconf test to deal with non-Posix platforms. > Looks like it works, and passes the regression tests as they are. I'm > patching the tests to include the cases 'NaN', 'Infinity', and '-Infinity' > as valid float8s, and 'not a float' as an invalid representation, and > rerunning to get output to submit with the patch. This might be a bit > hairy, since there are 5 different expected/float8* files. Should I try > to hand patch them to deal with the new rows, or let them be regenerated > by people with the appropriate platforms? How about setting up a separate test (say, ieee754.sql) so that non-compliant platforms can still pass the original FP test suite. Then other platforms can be added in as they are tested. Some platforms may need their compiler switches tweaked; I haven't checked the Alpha/DUnix configuration but I recall needing to fix some flags to get compiled code to move these edge cases around even just through subroutine calls. One example was in trying to call finite(), which threw an error during the call to it if the number was NaN or Infinity. Which sort of defeated the purpose of the call :) > Bigger problem with changing the float8 regression tests: a lot of our > math functions seem to be guarded with CheckFloat8Val(result), so, if we > allow these values in a float8 column, most of the math functions with > elog(). It strikes me that there must have been a reason for this at one > time. There's even a #define UNSAFE_FLOATS, to disable these checks. By > reading the comments in old copies of float.c, it looks like this was > added for an old, buggy linux/Alpha libc that would throw floating point > exceptions, otherwise. There are still reasons on some platforms, as noted above... > Is there an intrinsic problem with allowing values outside the range > FLOAT8_MAX <= x =>FLOAT8_MIN ? 'ORDER BY' seems to still work, with > 'Infinity' and '-Infinity' sorting properly. Having a 'NaN' in there > breaks sorting however. That's a current, live bug. Could be fixed > by treating 'NaN' as a different flavor of NULL. Probably a fairly deep > change, however. Hmm, NULL in a float8 sorts to the end, regardless of > ASC or DESC, is that right? NULL and NaN are not quite the same thing imho. If we are allowing NaN in columns, then it is *known* to be NaN. > Anyway, here's the patch for just float.c , if anyone wants to look > at it. As I said, it passes the existing float8 regression tests, but > raises a lot of interesting questions. Are you interested in pursuing this further? It seems like we might be able to move in the direction you suggest on *some* platforms, but we will need to scrub the math functions to be able to handle these edge cases. - Thomas
On Tue, Aug 15, 2000 at 03:27:55AM +0000, Thomas Lockhart wrote: > > Not all computers fully support IEEE754, though many new ones do. True - the question becomes: how new is new? Are we supporting ones that aren't? If so, that's fine. If not, it's a lot easier to fix. ;-) > > And, there's no test case in the regression tests for inserting NaN or > > Infinity. (Shame on Thomas ;-) > > Ah, I'm just trying to leave some rewarding work for other folks ;) And we appreciate the crumbs. Actually, it _was_ good practice grovelling out versions from CVS and matching log messages. > > istm that the existing protection (or something like it) is required for > some platforms, while other platforms may be able to handle NaN and > +/-Inf just fine. Seems like a job for autoconf to determine the FP > capabilities of a system, unless Posix defines some way to tell. Of > course, even then we'd need an autoconf test to deal with non-Posix > platforms. Yeah, need to get Peter Eisentraut involved, perhaps. Should actually be pretty simple: the #define is already there: UNSAFE_FLOATS. Define that, and the CheckFloat[48]Val functions just return true. > > How about setting up a separate test (say, ieee754.sql) so that > non-compliant platforms can still pass the original FP test suite. Then > other platforms can be added in as they are tested. Hmm, I wish we had clue what other systems might be non-compliant, and how. The question becomes one of if it's _possible_ to support NaN, +/-Inf on some platforms. Then, we end up with a difference in functionality. > > > Is there an intrinsic problem with allowing values outside the range > > FLOAT8_MAX <= x =>FLOAT8_MIN ? 'ORDER BY' seems to still work, with > > 'Infinity' and '-Infinity' sorting properly. Having a 'NaN' in there > > breaks sorting however. That's a current, live bug. Could be fixed > > by treating 'NaN' as a different flavor of NULL. Probably a fairly deep > > change, however. Hmm, NULL in a float8 sorts to the end, regardless of > > ASC or DESC, is that right? > > NULL and NaN are not quite the same thing imho. If we are allowing NaN > in columns, then it is *known* to be NaN. For the purposes of ordering, however, they are very similar. Neither one can be placed corectly with respect to the other values: NULL, because we don't know were it really is, NaN because we know it's not even on this axis. I'm suggesting the we fix sorting on floats to treat NaN as NULL, and sort it to the end. As it stands, sorting is broken, since it returns the NaN rows wherever they happen to be. This causes them to act as barriers, partitioning the returned set into seperately sorted sub sequences. > > > Anyway, here's the patch for just float.c , if anyone wants to look > > at it. As I said, it passes the existing float8 regression tests, but > > raises a lot of interesting questions. > > Are you interested in pursuing this further? It seems like we might be > able to move in the direction you suggest on *some* platforms, but we > will need to scrub the math functions to be able to handle these edge > cases. Sure. I'm no great floating point wiz, but I'm sure Tom and Don will jump on anything I get wrong. Duping the float tests and feeding them NaN/+/-Inf as a seperate test set is probably a good idea. The existing patch moves the call to CheckFloat8Val() inside float8in so it is only called if strtod() consumes all it's input, and does not set errno. Seems safe to me: if strtod() doesn't consume it's input, we check to make sure it's not NaN/+/-Inf (gah, need a shorthand word for those three values), else elog(). If it does, but sets errno, we catch that. Then, in belt-and-suspenders style, we call CheckFloat8Val(). For that check to fail, strtod() would have to consume its entire input, return +/-HUGE_VAL, and _not_ set errno to ERANGE. BTW, this also brings up something that got discussed before the last release, but never implemented. The original problem from Tim Allen had to do with using a work around for not having +/- Inf: storing the values -1.79769313486232e+308 and 1.79769313486232e+308. He was having trouble, since a pg_dump/restore cycle broke, do to rounding the values to out of range for floats. This wasn't caught by the current regression tests, but would have been caught by the suggested dump/restore/dump/compare dumps cycle someone suggested for exercizing the *out and *in functions. Ross -- Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> NSBRI Research Scientist/Programmer Computer and Information Technology Institute Rice University, 6100 S. Main St., Houston, TX 77005
Thomas - A general design question. There seems to be a good reason to allow +/-Inf in float8 columns: Tim Allen has an need for them, for example. That's pretty straight forward, they seem to act properly if the underlying float libs handle them. I'm not convinced NaN gives us anything useful, especially given how badly it breaks sorting. I've been digging into that code a little, and it's not going to be pretty. It strikes me as wrong to embed type specific info into the generic sorting routines. So, anyone have any ideas what NaN would be useful for? Especially given we have NULL available, which most (non DB) numeric applications don't. Ross -- Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> NSBRI Research Scientist/Programmer Computer and Information Technology Institute Rice University, 6100 S. Main St., Houston, TX 77005
> So, anyone have any ideas what NaN would be useful for? Especially given > we have NULL available, which most (non DB) numeric applications don't. Hmm. With Tom Lane's new fmgr interface, you *can* return NULL if you spot a NaN result. Maybe that is the best way to go about it; we'll stipulate that NaN and NULL are equivalent. And we'll further stipulate that if you are messing with NaN then you deserve what you get ;) - Thomas
I can't say whether its worth the trouble to add NaN, but I will say that NaN is not the same as NULL. NULL is missing data; NaN is 0./0. The difference is significant for numerical applications. Tim "Ross J. Reedstrom" wrote: > Thomas - > A general design question. There seems to be a good reason to allow +/-Inf > in float8 columns: Tim Allen has an need for them, for example. That's > pretty straight forward, they seem to act properly if the underlying float > libs handle them. > > I'm not convinced NaN gives us anything useful, especially given how > badly it breaks sorting. I've been digging into that code a little, > and it's not going to be pretty. It strikes me as wrong to embed type > specific info into the generic sorting routines. > > So, anyone have any ideas what NaN would be useful for? Especially given > we have NULL available, which most (non DB) numeric applications don't. > > Ross > -- > Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> > NSBRI Research Scientist/Programmer > Computer and Information Technology Institute > Rice University, 6100 S. Main St., Houston, TX 77005 -- Timothy H. Keitt National Center for Ecological Analysis and Synthesis 735 State Street, Suite 300, Santa Barbara, CA 93101 Phone: 805-892-2519, FAX: 805-892-2510 http://www.nceas.ucsb.edu/~keitt/
>I'm not convinced NaN gives us anything useful, especially given how >badly it breaks sorting. I've been digging into that code a little, >and it's not going to be pretty. It strikes me as wrong to embed type >specific info into the generic sorting routines. > >So, anyone have any ideas what NaN would be useful for? Especially given >we have NULL available, which most (non DB) numeric applications don't. > >Ross Just a wild guess, NaN could be used to indicated invalid numeric data. However, this seems odd because it should have been checked prior to being put in the DB. NULL is no value, +/- infinity could be just that or out of range, unless you want NaN to be out of range. Depending on your scheme for representation you could take an out of range value and store it as +/i infinity. These are just suggestions. Thomas
Hi, just to add my opinion on NaN in the IEEE standard. As far as I remember, IEEE numbers work as follows: 1 bit sign some bits base some bits exponent This allows you to do several things: interpret the exp bits as a normal integer and get - exp=below half: negative exponents - exp=half: exponent=0 - exp=above half: positive exponents - exp=all set: NaN, quite a few at that For all of these the sign can be either positive or negative, leading to pos/neg zero (quite a strange concept). With the NaNs, you get quite a few possibilities, but notably: - base=0 (NaN -- this is not a number, but an animal) - base=max (pos/neg infinity, depending on sign) Someone mentioned a representation for 0/0 and I might add that there are four possibilities:(( 1.)*0.) / (( 1.)*0.)(( 1.)*0.) / ((-1.)*0.)((-1.)*0.) / (( 1.)*0.)((-1.)*0.) / ((-1.)*0.) These (given commutativity, except that we're dealing with a finite representation, but predictable in that it is actually possible to factor out the sign) can be reduced to:( 1) * (0./0.)(-1) * (0./0.) which amounts to pos/neg infinity of some sort. Now my take on NULL vs NaN is that there should be a whole bunch of NULL, just like there is a whole bunch of NaN. Just off the top of my head, I could imagine "unknown", "unknowable", "out of range in direction X". But, alas, the SQL standard doesn't provide for such things (though the storage implementation would: but what would you do with comparisons, conversions and displays?). so long, Oliver
(Side note: Folks, we need a real bug/issue-tracking system. We just discussed this a month ago ("How PostgreSQL's floating point hurts everyone everywhere"). If anyone's interested in porting Bugzilla or some other such system to PostgreSQL and putting it into use, let me know.) Ross J. Reedstrom writes: > Yeah, need to get Peter Eisentraut involved, perhaps. Should actually be > pretty simple: the #define is already there: UNSAFE_FLOATS. Define that, > and the CheckFloat[48]Val functions just return true. Show me a system where it doesn't work and we'll get it to work. UNSAFE_FLOATS as it stands it probably not the most appropriate behaviour; it intends to speed things up, not make things portable. > > NULL and NaN are not quite the same thing imho. If we are allowing NaN > > in columns, then it is *known* to be NaN. > > For the purposes of ordering, however, they are very similar. Then we can also treat them similar, i.e. sort them all last or all first. If you have NaN's in your data you wouldn't be interested in ordering anyway. > we check to make sure it's not NaN/+/-Inf (gah, need a shorthand word > for those three values), else elog(). "non-finite values" Side note 2: The paper "How Java's floating point hurts everyone everywhere" provides for good context reading. Side note 3: Once you read that paper you will agree that using floating point with Postgres is completely insane as long as the FE/BE protocol is text-based. -- Peter Eisentraut Sernanders väg 10:115 peter_e@gmx.net 75262 Uppsala http://yi.org/peter-e/ Sweden
Thomas Lockhart writes: > > So, anyone have any ideas what NaN would be useful for? Especially given > > we have NULL available, which most (non DB) numeric applications don't. > > Hmm. With Tom Lane's new fmgr interface, you *can* return NULL if you > spot a NaN result. Maybe that is the best way to go about it; we'll > stipulate that NaN and NULL are equivalent. And we'll further stipulate > that if you are messing with NaN then you deserve what you get ;) I beg to differ, this behaviour would not be correct. Instead, this should happen: NULL < NULL => NULL NULL < 1.0 => NULL NULL < Nan => NULL 1.0 < NULL => NULL 1.0 < NaN => false NaN < NULL => NULL NaN < 1.0 => false Then all the NaN's sort either all first or all last before or after the NULLs. -- Peter Eisentraut Sernanders väg 10:115 peter_e@gmx.net 75262 Uppsala http://yi.org/peter-e/ Sweden
At 12:33 AM 8/20/00 +0200, Peter Eisentraut wrote: >(Side note: Folks, we need a real bug/issue-tracking system. We just >discussed this a month ago ("How PostgreSQL's floating point hurts >everyone everywhere"). If anyone's interested in porting Bugzilla or some >other such system to PostgreSQL and putting it into use, let me know.) OpenACS and arsDigita are using Ben Adida's software development manager, which includes a ticket-tracking module. It's still under development, but you can take a look at www.openacs.org/sdm to see how we're using it. It was developed for Postgres (which is what you see at the above URL) then ported to Oracle (which is what you arsDigita does). aD has also added some functionality which is supposed to be ported back to the Postgres version. Among other things it integrates with a todo list manager that maintains individual todo lists for developers ... you're assigned a bug, it ends up on your todo list. - Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert Serviceand other goodies at http://donb.photo.net.
At 09:01 PM 8/19/00 -0300, The Hermit Hacker wrote: > >Ben has an account on Hub, and aolserver has been installed, so if you >guys want to install and get this working, just tell me what else you need >as far as software and/or configurations are concerned and "it shall be >done" ... I've e-mailed Ben a "heads-up", though he monitors this list and will probably see your note. I'll be gone about five of the next 6-7 weeks mostly doing my annual stint as a field biologist where I'm only accessible once a day via radio by BLM Dispatch in Elko, Nevada so I'm afraid this (like many other things at the moment) will fall on Ben's shoulders... - Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert Serviceand other goodies at http://donb.photo.net.
Ben has an account on Hub, and aolserver has been installed, so if you guys want to install and get this working, just tell me what else you need as far as software and/or configurations are concerned and "it shall be done" ... On Sat, 19 Aug 2000, Don Baccus wrote: > At 12:33 AM 8/20/00 +0200, Peter Eisentraut wrote: > >(Side note: Folks, we need a real bug/issue-tracking system. We just > >discussed this a month ago ("How PostgreSQL's floating point hurts > >everyone everywhere"). If anyone's interested in porting Bugzilla or some > >other such system to PostgreSQL and putting it into use, let me know.) > > OpenACS and arsDigita are using Ben Adida's software development manager, > which includes a ticket-tracking module. It's still under development, > but you can take a look at www.openacs.org/sdm to see how we're using > it. > > It was developed for Postgres (which is what you see at the above URL) > then ported to Oracle (which is what you arsDigita does). aD has also > added some functionality which is supposed to be ported back to the > Postgres version. > > Among other things it integrates with a todo list manager that maintains > individual todo lists for developers ... you're assigned a bug, it > ends up on your todo list. > > > > > - Don Baccus, Portland OR <dhogaza@pacifier.com> > Nature photos, on-line guides, Pacific Northwest > Rare Bird Alert Service and other goodies at > http://donb.photo.net. > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy Systems Administrator @ hub.org primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
http://www.postgresql.org/bugs/ I was about to implement it once before and the directory disappeared. But anyway it's there. Vince. On Sat, 19 Aug 2000, The Hermit Hacker wrote: > > Ben has an account on Hub, and aolserver has been installed, so if you > guys want to install and get this working, just tell me what else you need > as far as software and/or configurations are concerned and "it shall be > done" ... > > On Sat, 19 Aug 2000, Don Baccus wrote: > > > At 12:33 AM 8/20/00 +0200, Peter Eisentraut wrote: > > >(Side note: Folks, we need a real bug/issue-tracking system. We just > > >discussed this a month ago ("How PostgreSQL's floating point hurts > > >everyone everywhere"). If anyone's interested in porting Bugzilla or some > > >other such system to PostgreSQL and putting it into use, let me know.) > > > > OpenACS and arsDigita are using Ben Adida's software development manager, > > which includes a ticket-tracking module. It's still under development, > > but you can take a look at www.openacs.org/sdm to see how we're using > > it. > > > > It was developed for Postgres (which is what you see at the above URL) > > then ported to Oracle (which is what you arsDigita does). aD has also > > added some functionality which is supposed to be ported back to the > > Postgres version. > > > > Among other things it integrates with a todo list manager that maintains > > individual todo lists for developers ... you're assigned a bug, it > > ends up on your todo list. > > > > > > > > > > - Don Baccus, Portland OR <dhogaza@pacifier.com> > > Nature photos, on-line guides, Pacific Northwest > > Rare Bird Alert Service and other goodies at > > http://donb.photo.net. > > > > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy > Systems Administrator @ hub.org > primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org > > -- ========================================================================== Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from $16.00/moat Pop4 Networking Online Campground Directory http://www.camping-usa.com Online Giftshop Superstore http://www.cloudninegifts.com ==========================================================================
At 08:34 PM 8/19/00 -0400, Vince Vielhaber wrote: > >http://www.postgresql.org/bugs/ > >I was about to implement it once before and the directory disappeared. >But anyway it's there. Cool, I tried it and broke it on my second click ... any particular reason to roll your own rather than use something that's already being used by several other development projects and is under active development for that reason? (i.e. the SDM) - Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert Serviceand other goodies at http://donb.photo.net.
Don Baccus wrote: > > >(Side note: Folks, we need a real bug/issue-tracking system. We just > >discussed this a month ago ("How PostgreSQL's floating point hurts > >everyone everywhere"). If anyone's interested in porting Bugzilla or some > >other such system to PostgreSQL and putting it into use, let me know.) istm that it is *not* that easy. We tried (very briefly) a bug tracking system. Whatever technical problems it had (which other tools may not), the fundamental problem is that the mailing lists do a *great* job of screening problem reports while also supporting and enhancing the "Postgres culture", whereas a "bug report tool" eliminates that traffic and requires one or a few people to pay attention to the bug list to manage new and existing bug reports. This has (or could have) a *huge* impact on the culture and tradition of Postgres development, which imho is one of the most satisfying, pleasant, and effective environments in open source development. So if we try to do something with a bug tracking system, we will need to figure out: o how to retain a free and helpful discussion on the mailing lists, and to not degrade into a "shut up and check the bug reports" response. o how to filter or qualify bug reports so that developers don't spend time having to do that. All imho of course ;) - Thomas
At 01:22 AM 8/20/00 +0000, Thomas Lockhart wrote: >istm that it is *not* that easy. We tried (very briefly) a bug tracking >system. Whatever technical problems it had (which other tools may not), >the fundamental problem is that the mailing lists do a *great* job of >screening problem reports while also supporting and enhancing the >"Postgres culture", whereas a "bug report tool" eliminates that traffic >and requires one or a few people to pay attention to the bug list to >manage new and existing bug reports. In the SDM you can, of course, ask to be notified of various events by e-mail. And there's a commenting facility so in essence a bug report or feature request starts a conversation thread. I don't recall saying that the SDM is simply a "bug report tool". There's quite a bit more to it, and the goal is to INCREASE interactivity, not decrease it. >This has (or could have) a *huge* impact on the culture and tradition of >Postgres development, which imho is one of the most satisfying, >pleasant, and effective environments in open source development. So if >we try to do something with a bug tracking system, we will need to >figure out: > >o how to retain a free and helpful discussion on the mailing lists, and >to not degrade into a "shut up and check the bug reports" response. This is a social, not software, engineering issue. >o how to filter or qualify bug reports so that developers don't spend >time having to do that. Developers don't have to filter or qualify bug reports e-mailed to the bugs list today? Who's doing it, then, and why can't they continue doing so if another tool is used to manage bug reports? The SDM allows a little more granularity than the single e-mail list aproach allows for. You can designate modules within a package. For instance, psql might be a module with Peter assigned as an administrator, and he might elect to get e-mail alerts whenever a bug is submitted to for psql. But he might not, for instance, be particularly interested in getting e-mail alerts on (say) the JDBC driver. There's a certain amount of delegation inherent in an approach like this, and developers focused on narrow portions of the product (and Peter came to mind because of psql, I'm not suggesting he only has a narrow interest in the product) can arrange to only get nagged, if you will, for stuff they've taken responsibility for. My guess is that such a system probably isn't as cozy and useful for developers, as you're implying. I think it might well be more friendly for users, though. Certainly the OpenACS and arsDigita communities - both fairly large though not as long in the tooth as PG, I might add - seem to appreciate having access to such a system. - Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert Serviceand other goodies at http://donb.photo.net.
On Sat, 19 Aug 2000, Don Baccus wrote: > At 08:34 PM 8/19/00 -0400, Vince Vielhaber wrote: > > > >http://www.postgresql.org/bugs/ > > > >I was about to implement it once before and the directory disappeared. > >But anyway it's there. > > Cool, I tried it and broke it on my second click ... any particular reason > to roll your own rather than use something that's already being used by > several other development projects and is under active development for > that reason? (i.e. the SDM) Like I said, the dir disappeared before I could commit it, probably some config stuff too. We tried a couple of already in use items and frankly I got tired of learning a new package that noone used anyway. I figured at least this one could be more of what we needed. It logs the problem in the database and emails the bugs list (I may have the wrong list addr in there too). The status can be changed, entries can be made as to the status. What did you do to break it and what broke? Vince. -- ========================================================================== Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from $16.00/moat Pop4 Networking Online Campground Directory http://www.camping-usa.com Online Giftshop Superstore http://www.cloudninegifts.com ==========================================================================
If Ben and gang are willing to work on the bug tracking system to get it to fit what we want/need, it would give a good example of OpenACS and gang for them to use ... we could just shot "wants" at them and they could improve as we go along? On Sat, 19 Aug 2000, Vince Vielhaber wrote: > On Sat, 19 Aug 2000, Don Baccus wrote: > > > At 08:34 PM 8/19/00 -0400, Vince Vielhaber wrote: > > > > > >http://www.postgresql.org/bugs/ > > > > > >I was about to implement it once before and the directory disappeared. > > >But anyway it's there. > > > > Cool, I tried it and broke it on my second click ... any particular reason > > to roll your own rather than use something that's already being used by > > several other development projects and is under active development for > > that reason? (i.e. the SDM) > > Like I said, the dir disappeared before I could commit it, probably some > config stuff too. We tried a couple of already in use items and frankly > I got tired of learning a new package that noone used anyway. I figured > at least this one could be more of what we needed. It logs the problem > in the database and emails the bugs list (I may have the wrong list > addr in there too). The status can be changed, entries can be made as > to the status. > > What did you do to break it and what broke? > > Vince. > -- > ========================================================================== > Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net > 128K ISDN from $22.00/mo - 56K Dialup from $16.00/mo at Pop4 Networking > Online Campground Directory http://www.camping-usa.com > Online Giftshop Superstore http://www.cloudninegifts.com > ========================================================================== > > > > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy Systems Administrator @ hub.org primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
On Sat, 19 Aug 2000, Don Baccus wrote: > >o how to filter or qualify bug reports so that developers don't spend > >time having to do that. > > Developers don't have to filter or qualify bug reports e-mailed to the > bugs list today? Who's doing it, then, and why can't they continue > doing so if another tool is used to manage bug reports? the problem as I see it with any bug tracking tool is someone has to close off those bugs when fixed ... right now, someone commits a bug fix, and then fires off an email to the list stating its fixed ... with a bug tracking system, then they have to go one more step, open up a web browser, login to the system, find the bug report and close it ...
On Sun, 20 Aug 2000, The Hermit Hacker wrote: > > If Ben and gang are willing to work on the bug tracking system to get it > to fit what we want/need, it would give a good example of OpenACS and gang > for them to use ... we could just shot "wants" at them and they could > improve as we go along? Fine by me, BUT I have no desire to learn how it works. If that's gonna be the end result then rm -Rf is my preference. This'll be the third or forth one so far, the others were pushed off the edge of the earth. Sorry to be so harsh but no matter what the bug tool is I can't see it lasting very long. This group has shown repeatedly that it's not as desired as it appears to be. Vince. > > On Sat, 19 Aug 2000, Vince Vielhaber wrote: > > > On Sat, 19 Aug 2000, Don Baccus wrote: > > > > > At 08:34 PM 8/19/00 -0400, Vince Vielhaber wrote: > > > > > > > >http://www.postgresql.org/bugs/ > > > > > > > >I was about to implement it once before and the directory disappeared. > > > >But anyway it's there. > > > > > > Cool, I tried it and broke it on my second click ... any particular reason > > > to roll your own rather than use something that's already being used by > > > several other development projects and is under active development for > > > that reason? (i.e. the SDM) > > > > Like I said, the dir disappeared before I could commit it, probably some > > config stuff too. We tried a couple of already in use items and frankly > > I got tired of learning a new package that noone used anyway. I figured > > at least this one could be more of what we needed. It logs the problem > > in the database and emails the bugs list (I may have the wrong list > > addr in there too). The status can be changed, entries can be made as > > to the status. > > > > What did you do to break it and what broke? > > > > Vince. > > -- > > ========================================================================== > > Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net > > 128K ISDN from $22.00/mo - 56K Dialup from $16.00/mo at Pop4 Networking > > Online Campground Directory http://www.camping-usa.com > > Online Giftshop Superstore http://www.cloudninegifts.com > > ========================================================================== > > > > > > > > > > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy > Systems Administrator @ hub.org > primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org > > -- ========================================================================== Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from $16.00/moat Pop4 Networking Online Campground Directory http://www.camping-usa.com Online Giftshop Superstore http://www.cloudninegifts.com ==========================================================================
FWIW, I agree with Thomas' comments: our last try at a bug-tracking system was a spectacular failure, and rather than just trying again with a technically better bug-tracker, we need to understand why we failed before. I think we do need to look for a better answer than what we have, but I do not have any faith in "install system FOO and all your problems will be solved". My take is that (a) a bug *tracking* system is not the same as a bug *reporting* system. A tracking system will be useless if it gets cluttered with non-bug reports, duplicate entries, etc. There must be a human filter controlling what gets entered into the system. (b) our previous try (with Keystone) was a failure in part because it was not even effective as a bug reporting system: it did not encourage people to fill in our standard "bug report form", with the result that bug reports were seriously incomplete w.r.t. version numbers, platforms, etc. This is a relatively simple technical deficiency, not a social-engineering problem, but it does point up the fact that one-size-fits-all solutions fit nobody. (c) fill-in-the-web-form reporting systems suck. They make it difficult to copy-and-paste query output, dump files, etc. Also, the window for entering text is always either too small or too large. Email with attachments is fundamentally superior. (d) divorcing the bug reporting system from the existing mailing list community is foolish, as Thomas pointed out. When a bug report is a non-bug (user error, etc) or fixed in a later version or just a duplicate, we tend to rely on the rest of the community to give the reporter a helpful response. Funneling reports into a separate system that is only read by a few key developers will lose. I'm not sure what I want, but I know what I don't want... regards, tom lane
Perhaps this is foolhardy, but it might be worth making a list of requirements, at least so we can tick moxes when considering any system... >(a) a bug *tracking* system is not the same as a bug *reporting* >system. A tracking system will be useless if it gets cluttered >with non-bug reports, duplicate entries, etc. There must be a human >filter controlling what gets entered into the system. 1. Human filtering of 'incoming' reports. 2. Separation of 'bug reports' from 'bugs'. >(b) our previous try (with Keystone) was a failure in part because >it was not even effective as a bug reporting system: it did not >encourage people to fill in our standard "bug report form", with the >result that bug reports were seriously incomplete w.r.t. version >numbers, platforms, etc. This is a relatively simple technical >deficiency, not a social-engineering problem, but it does point up >the fact that one-size-fits-all solutions fit nobody. 3. Web and email submissions should do data verification and reject incomplete reports (giving reasons). >(c) fill-in-the-web-form reporting systems suck. They make it >difficult to copy-and-paste query output, dump files, etc. >Also, the window for entering text is always either too small or too >large. Email with attachments is fundamentally superior. [I disagree with the above (web *can* work), but...] 4. Must support email AND web submissions, or at least email submissions and web reporting. >(d) divorcing the bug reporting system from the existing mailing >list community is foolish, as Thomas pointed out. 5. Must integrate with mailing lists. And to add some of my own (suggested) requirements: 6. Require: name, email address, OS & version, PG version, short description. 7. Optional: compiler & version, long description, file attachments. 8. Creation of 'bug reports' is a public function. Creation of 'bug entries' is a priv'd function. 9. Simple reporting - unprocessed bug reports, open bugs, bugs by module etc. I have tried to keep this relatively simple in an effort to define what we need to make it work in the current context. But I'm sure I've missed things.... <YABRS> As it happens, I have a Perl/PGSql based bug-tracking system that I give my clients access to, which does about 80-90% of this, and I would be willing to GPL it. Before anybody asks, the reason I rolled my own was because there weren't many that supported email and web submission, and the ones that did, did not support PG easily. This system also implements my prefferred model for bug reporting which is to separate (to use my terminology) 'Incidents' from 'Issues': an Incident is an event (or set of events) that causes a user to make a report. An Issue is an individual item that needs attention (usually a single bug). Typically users send email (or web forms) and I create one or more Issues. When two Incidents (bug reports) are made about the same Issue, then the system allows the two to be linked, so that when the Issue is fixed, all Incident owners are notified etc. The email integration does very little validation, and it is *not* integrated with a mailing list (but it is on my ToDo list). I had planned to do the following: - When a message is received and the subject does not start with 'Re:' (or 'Aw:'!), submit it as a bug report. If the bug report code returns an error, then reject it from the list. If the bug report works, then send it on to Majordomo & Mhonarc. - If the message starts with 'Re:' then just submit it to the list. Let me know if anyone would be interested, and I can set up a sample 'product' for people to play with and then, if it is still worth pursuing, make the code a little more presentable (and decrease it's reliance on my own perl libraries). Bear in mind that this is designed for an Apache/mod-perl environment, so may not be acceptable to some people. </YABRS> ---------------------------------------------------------------- Philip Warner | __---_____ Albatross Consulting Pty. Ltd. |----/ - \ (A.B.N. 75 008 659 498) | /(@) ______---_ Tel: (+61) 0500 83 82 81 | _________ \ Fax: (+61) 0500 83 82 82 | ___________ | Http://www.rhyme.com.au | / \| | --________-- PGP key available upon request, | / and from pgp5.ai.mit.edu:11371 |/
Tom Lane writes: > (a) a bug *tracking* system is not the same as a bug *reporting* > system. A tracking system will be useless if it gets cluttered > with non-bug reports, duplicate entries, etc. There must be a human > filter controlling what gets entered into the system. Letting any user submit bug reports directly into any such system is certainly not going to work, we'd have "query does not use index" 5 times a day. I consider the current *reporting* procedure pretty good; web forms are overrated in my mind. What I had in mind was more a databased incarnation of the TODO list. I mean, who are we kidding, we are writing a database and maintain the list of problems in flat text. The TODO list has already moved to the TODO.detail extension, but we could take it a bit further. I think currently too many issues get lost, or discussed over and over again. Many developers maintain their own little lists. The TODO list often cannot be deciphered by end users and hence does not get read. -- Peter Eisentraut Sernanders väg 10:115 peter_e@gmx.net 75262 Uppsala http://yi.org/peter-e/ Sweden
On Sun, 20 Aug 2000, Peter Eisentraut wrote: > Tom Lane writes: > > > (a) a bug *tracking* system is not the same as a bug *reporting* > > system. A tracking system will be useless if it gets cluttered > > with non-bug reports, duplicate entries, etc. There must be a human > > filter controlling what gets entered into the system. > > Letting any user submit bug reports directly into any such system is > certainly not going to work, we'd have "query does not use index" 5 times > a day. I consider the current *reporting* procedure pretty good; web forms > are overrated in my mind. > > What I had in mind was more a databased incarnation of the TODO list. I > mean, who are we kidding, we are writing a database and maintain the list > of problems in flat text. The TODO list has already moved to the > TODO.detail extension, but we could take it a bit further. > > I think currently too many issues get lost, or discussed over and over > again. Many developers maintain their own little lists. The TODO list > often cannot be deciphered by end users and hence does not get read. A TODO list that one can add comments to ...
Peter Eisentraut wrote: > > Tom Lane writes: > > > (a) a bug *tracking* system is not the same as a bug *reporting* > > system. A tracking system will be useless if it gets cluttered > > with non-bug reports, duplicate entries, etc. There must be a human > > filter controlling what gets entered into the system. > > Letting any user submit bug reports directly into any such system is > certainly not going to work, we'd have "query does not use index" 5 times > a day. I consider the current *reporting* procedure pretty good; web forms > are overrated in my mind. > > What I had in mind was more a databased incarnation of the TODO list. I > mean, who are we kidding, we are writing a database and maintain the list > of problems in flat text. The TODO list has already moved to the > TODO.detail extension, but we could take it a bit further. > I maintain my todo items for my projects in a postgres database. But there are a lot of issues to consider there too: - a table of projects (or topics) - a table of todo items with synopsis, full description, ... - a table of versions (item is planned to be solved in version, x.x.x, actually solved in y.y.y) - a table of developers - assign table (projects -> developers, items -> developers) - change type: bug,doc,rfe (request for enhancement),idc (internal design change), ... - change state (accepted, evaluated, fixed, rejected, incomplete, committed, ... - severity or priority of each item, project - search functionality Regards Wim
The Hermit Hacker wrote: > the problem as I see it with any bug tracking tool is someone has to close > off those bugs when fixed ... right now, someone commits a bug fix, and > then fires off an email to the list stating its fixed ... with a bug > tracking system, then they have to go one more step, open up a web > browser, login to the system, find the bug report and close it ... With something like the SDM, the developer can also just mark that bug fixed online and whoever requested notifications (like maybe the pgsql-hackers mailing list) can get a one-liner automated email. This is great for individual users who are particularly interested in seeing one bug fixed: they request notification on that bug, and only get notifications that pertain to it... -Ben
I'm putting on my suits-type suit for just a moment. In order to Conquer The Universe(tm) why don't we just call it "PG"? -dlj.
Tom Lane wrote: > FWIW, I agree with Thomas' comments: our last try at a bug-tracking > system was a spectacular failure, and rather than just trying again > with a technically better bug-tracker, we need to understand why we > failed before. Okay, well then this is an interesting discussion: the OpenACS project and my company (OpenForce) would very much be interested in discussing what you guys think would be a useful bug tracking system. What kind of features would it need? Nothing is too fancy here, the idea is to have the best possible system, one that focused developers like the PG team would use. Maybe we should take this offline? I'm happy to keep this discussion going and hear everything you've got to say: understanding what it would take to build a system you guys would use is *very* important data. -Ben
On Sun, Aug 20, 2000 at 12:33:00AM +0200, Peter Eisentraut wrote: <snip side comment about bug tracking. My input: for an email controllable system, take a look at the debian bug tracking system> > Show me a system where it doesn't work and we'll get it to work. > UNSAFE_FLOATS as it stands it probably not the most appropriate behaviour; > it intends to speed things up, not make things portable. > I agree. In the previous thread on this, Thomas suggested creating a flag that would allow control turning the CheckFloat8Val function calls into a macro NOOP. Sound slike a plan to me. > > > > NULL and NaN are not quite the same thing imho. If we are allowing NaN > > > in columns, then it is *known* to be NaN. > > > > For the purposes of ordering, however, they are very similar. > > Then we can also treat them similar, i.e. sort them all last or all first. > If you have NaN's in your data you wouldn't be interested in ordering > anyway. Right, but the problem is that NULLs are an SQL language feature, and there for rightly special cased directly in the sorting apparatus. NaN is type specific, and I'd be loath to special case it in the same place. As it happens, I've spent some time this weekend groveling through the sort (and index, as it happens) code, and have an idea for a type specific fix. Here's the deal, and an actual, honest to goodness bug in the current code. As it stands, we allow one non-finite to be stored in a float8 field: NaN, with partial parsing of 'Infinity'. As I reported last week, NaNs break sorts: they act as barriers, creating sorted subsections in the output. As those familiar with the code have already guessed, there is a more serious bug: NaNs break indicies on float8 fields, essentially chopping the index off at the first NaN. Fixing this turns out to be a one liner to btfloat8cmp. Fixing sorts is a bit tricker, but can be done: Currently, I've hacked the float8lt and float8gt code to sort NaN to after +/-Infinity. (since NULLs are special cased, they end up sorting after NaN). I don't see any problems with this solution, and it give the desired behavior. I've attached a patch which fixes all the sort and index problems, as well as adding input support for -Infinity. This is not a complete solution, since I haven't done anything with the CheckFloat8Val test. On my system (linux/glibc2.1) compiling with UNSAFE_FLOATS seems to work fine for testing. > > Side note 2: The paper "How Java's floating point hurts everyone > everywhere" provides for good context reading. http://http/cs.berkeley.edu/~wkahan/JAVAhurt.pdf ? I'll take a look at it when I get in to work Monday. > > Side note 3: Once you read that paper you will agree that using floating > point with Postgres is completely insane as long as the FE/BE protocol is > text-based. Probably. But it's not our job to enforce sanity, right? Another way to think about it is fixing the implementation so the deficiencies of the FE/BE stand out in a clearer light. ;-) Ross -- Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> NSBRI Research Scientist/Programmer Computer and Information Technology Institute Rice University, 6100 S. Main St., Houston, TX 77005
Attachment
what's wrong wth "Post-Gres-QL"? I find it soooo simple to pronounce *shrug* On Sun, 20 Aug 2000, David Lloyd-Jones wrote: > I'm putting on my suits-type suit for just a moment. > > In order to Conquer The Universe(tm) why don't we just call it "PG"? > > -dlj. > > > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy Systems Administrator @ hub.org primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
At 01:15 20/08/00 -0400, Ben Adida wrote: >Okay, well then this is an interesting discussion: the OpenACS project >and my company (OpenForce) would very much be interested in discussing >what you guys think would be a useful bug tracking system. So am I. > >Maybe we should take this offline? If you do decide to go offline, I'd appreciate some CCs... ---------------------------------------------------------------- Philip Warner | __---_____ Albatross Consulting Pty. Ltd. |----/ - \ (A.B.N. 75 008 659 498) | /(@) ______---_ Tel: (+61) 0500 83 82 81 | _________ \ Fax: (+61) 0500 83 82 82 | ___________ | Http://www.rhyme.com.au | / \| | --________-- PGP key available upon request, | / and from pgp5.ai.mit.edu:11371 |/
If you all don't mind, I think it'd be great to keep this discussion on the list. As some of you know, Great Bridge is working on its own project hosting site (including a PG-backed bug tracking module). We're not quite ready to pull back the curtain yet, but are getting close, and will be actively soliciting input (and hacks) from the community. The process of getting hacker requirements for such a system is a very useful one, IMHO... Regards, Ned Philip Warner wrote: > At 01:15 20/08/00 -0400, Ben Adida wrote: > >Okay, well then this is an interesting discussion: the OpenACS project > >and my company (OpenForce) would very much be interested in discussing > >what you guys think would be a useful bug tracking system. > > So am I. > > > > >Maybe we should take this offline? > > If you do decide to go offline, I'd appreciate some CCs... > > ---------------------------------------------------------------- > Philip Warner | __---_____ > Albatross Consulting Pty. Ltd. |----/ - \ > (A.B.N. 75 008 659 498) | /(@) ______---_ > Tel: (+61) 0500 83 82 81 | _________ \ > Fax: (+61) 0500 83 82 82 | ___________ | > Http://www.rhyme.com.au | / \| > | --________-- > PGP key available upon request, | / > and from pgp5.ai.mit.edu:11371 |/
"Ross J. Reedstrom" <reedstrm@rice.edu> writes: > ! switch (isinf(num)) > ! { > ! case -1: > ! PG_RETURN_CSTRING(strcpy(ascii, "-Infinity")); > ! break; > ! case 1: > ! PG_RETURN_CSTRING(strcpy(ascii, "Infinity")); > ! break; > ! default: > ! break; > ! } My man page for isinf() sez: isinf() returns a positive integer if x is +INFINITY, or a negative integer if x is -INFINITY. Otherwise it returnszero. so the above switch statement is making an unportable assumption about exactly which positive or negative value will be returned. > + if (isnan(arg2)) PG_RETURN_BOOL(1); PG_RETURN_BOOL(true), please... > ! if (isnan(a)) > ! PG_RETURN_INT32(1); Do not like this at all --- doesn't it make the result of btint4cmp(NaN, NaN) dependent on which argument chances to be first? Seems to me that you must consider two NaNs to be equal, unless you want to subdivide the category of NaNs. regards, tom lane
At 10:22 PM 8/20/00 -0400, Ned Lilly wrote: >If you all don't mind, I think it'd be great to keep this discussion on the >list. As some of you know, Great Bridge is working on its own project >hosting site (including a PG-backed bug tracking module). We're not quite >ready to pull back the curtain yet, but are getting close, and will be >actively soliciting input (and hacks) from the community. So - again - why roll your own instead of build upon a base which at least is already seeing some use, by some fairly large organizations (arsDigita is larger and more deeply funded than Great Bridge, making profits to boot, and I won't even start talking about AOL)? arsDigita is putting some developer effort into the SDM, so it's no longer just Ben and whoever he can rope into helping out. Couldn't you guys more profitably spend time, say, working on outer joins rather than doing something like this? The folks working on the SDM have a LOT more web/db development experience than whoever's rolling your bug tracking system. I keep sniffing the odor of NIH ... - Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert Serviceand other goodies at http://donb.photo.net.
>At 10:22 PM 8/20/00 -0400, Ned Lilly wrote: >>If you all don't mind, I think it'd be great to keep this discussion on the >>list. As some of you know, Great Bridge is working on its own project >>hosting site (including a PG-backed bug tracking module). We're not quite >>ready to pull back the curtain yet, but are getting close, and will be >>actively soliciting input (and hacks) from the community. Another implication which missed me first time 'round is that Great Bridge might be planning to have its own bug reporting system, separate from that used by the development community at large? I hope not. There should be one central place for bug reporting. If Great Bridge wants to run it, fine, also if Great Bridge wants to be able to incorporate some sort of prioritization system for those with paid support (or some other discriminatory system) it is still probably better to figure out a way to accomodate it rather than have two separate bug reporting systems. - Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert Serviceand other goodies at http://donb.photo.net.
On Mon, 21 Aug 2000, Don Baccus wrote: > At 10:22 PM 8/20/00 -0400, Ned Lilly wrote: > >If you all don't mind, I think it'd be great to keep this discussion on the > >list. As some of you know, Great Bridge is working on its own project > >hosting site (including a PG-backed bug tracking module). We're not quite > >ready to pull back the curtain yet, but are getting close, and will be > >actively soliciting input (and hacks) from the community. > > So - again - why roll your own instead of build upon a base which at least > is already seeing some use, by some fairly large organizations (arsDigita > is larger and more deeply funded than Great Bridge, making profits to boot, > and I won't even start talking about AOL)? arsDigita is putting some > developer > effort into the SDM, so it's no longer just Ben and whoever he can rope into > helping out. > > Couldn't you guys more profitably spend time, say, working on outer joins > rather than doing something like this? > > The folks working on the SDM have a LOT more web/db development experience > than whoever's rolling your bug tracking system. > > I keep sniffing the odor of NIH ... Could it be possible that folks are shying away because of having to install and learn an entire webserver and tools and then the bug tracker on top of that? Vince. -- ========================================================================== Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from $16.00/moat Pop4 Networking Online Campground Directory http://www.camping-usa.com Online Giftshop Superstore http://www.cloudninegifts.com ==========================================================================
Vince Vielhaber wrote: > Could it be possible that folks are shying away because of having > to install and learn an entire webserver and tools and then the > bug tracker on top of that? And so the solution is to build a totally new system? Coming from the Postgres team, this is relatively surprising. Postgres is a great tool. I dropped Oracle and learned Postgres because I thought it would eventually become a better tool, and because it was already better in many ways. It took time and effort to do so, but eventually it was the right thing to do because I can now make full use of a very powerful open-source database. It seems to me that the whole point of Open-Source vs. Not Invented Here is that you are *supposed* to go out and make the effort necessary to learn new tools that can then become extremely useful. If you accept the attitude of "it's not Apache/mod-perl so I'm not using it," then it's time to stop all criticism of those who use MySQL, Oracle, Windows, etc... Those people are *used* to their technology, and the only reason they refuse to switch is that they don't want to spend time learning something new. Just my 2 cents.... the useful tools are not always the ones everyone is using. -Ben
On Mon, 21 Aug 2000, Ben Adida wrote: > Vince Vielhaber wrote: > > > Could it be possible that folks are shying away because of having > > to install and learn an entire webserver and tools and then the > > bug tracker on top of that? > > And so the solution is to build a totally new system? In some cases yes. > Coming from the Postgres team, this is relatively surprising. Postgres is a great > tool. I dropped Oracle and learned Postgres because I thought it would eventually > become a better tool, and because it was already better in many ways. It took > time and effort to do so, but eventually it was the right thing to do because I > can now make full use of a very powerful open-source database. I am *NOT* "the Postgres team". But have you listened to what you & Don are suggesting that we, or for that matter anyone else in need of a bug tracking system, do? You want us to install the full blown arsDigita with all the bells and whistles just for a bug tracker. That's like saying I need a pickup truck to move a chair so I'm going to go out and get a new FreightLiner with a 55' trailer to do the job. > It seems to me that the whole point of Open-Source vs. Not Invented Here is that > you are *supposed* to go out and make the effort necessary to learn new tools > that can then become extremely useful. If you accept the attitude of "it's not > Apache/mod-perl so I'm not using it," then it's time to stop all criticism of > those who use MySQL, Oracle, Windows, etc... Those people are *used* to their > technology, and the only reason they refuse to switch is that they don't want to > spend time learning something new. You missed the point. It's called overkill. You needed a full blown database for your project. We need (although _want_ may be another story) a bug tracker - not a new webserver. Vince. -- ========================================================================== Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from $16.00/moat Pop4 Networking Online Campground Directory http://www.camping-usa.com Online Giftshop Superstore http://www.cloudninegifts.com ==========================================================================
On Mon, Aug 21, 2000 at 07:35:10AM -0700, Don Baccus wrote: > > >At 10:22 PM 8/20/00 -0400, Ned Lilly wrote: > >>If you all don't mind, I think it'd be great to keep this discussion on the > >>list. As some of you know, Great Bridge is working on its own project > >>hosting site (including a PG-backed bug tracking module). We're not quite > >>ready to pull back the curtain yet, but are getting close, and will be > >>actively soliciting input (and hacks) from the community. > > Another implication which missed me first time 'round is that Great Bridge > might be planning to have its own bug reporting system, separate from > that used by the development community at large? Cool your conspiracy theories. I'm not yet involved with either side of this discussion, but before it runs out of control... > I hope not. There should be one central place for bug reporting. If > Great Bridge wants to run it, fine, also if Great Bridge wants to be able > to incorporate some sort of prioritization system for those with paid > support (or some other discriminatory system) it is still probably better > to figure out a way to accomodate it rather than have two separate > bug reporting systems. The fact is that postgres already has a very good system for keeping track of issues from report to fix to verification. So far the main defect is the obvious one of "People don't know the history unless they troll the message archives or lurk". Everyone here is leery of "fixing" a working system. Especially when it entails modifying the working system to deal with a new issue database. Bug Database/Issue Trackers can be done in two ways. Someone can grab an off-the-shelf system like Bugzilla or this ArsTechnica thing and then try to make the project conform to it. So far, everyone I've talked to who has touched Bugzilla has said that it sucks. I don't know anything about this other proposed system but it will probably require a lot of time to even get people to use it regularly, much less use it well. The other method is to create the system to match the process in place. Since the postgres project is already very well organized, I personally would like to see the custom system, rather then make Bruce throw away his TODO list and use someone else's idea of an issue tracking system. It takes a lot more work--someone has to pay attention to what is going on with the project and make sure the database stays in sync, but in the long run, it is less disruptive and smoother to integrate into an already working project like this one. -- Adam Haberlach | "A farm tractor is not a motorcycle." adam@newsnipple.com | --California DMV 1999 http://www.newsnipple.com/ | Motorcycle Driver Handbook
Tom - Thanks for the review. Here's a new version of the patch, fixing the two you objected to. Unfotunately, I seem to have found another corner case in the existing code that needs fixing. Here's the one line version: Use of an index in an ORDER BY DESC result changes placement of NULLs (and NaNs, now) from last returned to first returned tuples Long version: While examining the output from ORDER BY queries, both using and not using an index, I came across a discrepancy: the explicit handling of NULLs in the tuplesort case always sorts NULLs to the end, regardless of direction of sort. Intellectually, I kind of like that: "We don't know what these are, let's just tack them on the end". I implemented NaN sorting to emulate that behavior. This also has the pleasant property that NULL (or NaN) are never returned as > or < any other possible value, should be expected. However, if an index is involved, the index gets built, and the NULL values are stored at one end of the index. So, when a ORDER BY DESC is requested, the index is just read backwards, sending the NULLs (and NaNs) first. (They're still not returned from a query with a clause such as WHERE f1 > 0.) An example of the output is attached, from the regress float8 table (with a NULL and non-finites added. Don't need the non-finites to to display the problem, though, since it's NULLs as well) Note the blank row, which is the NULL, moves from the bottom to the top in the last case, using the index. So, what way should we go here? Make ASC/DESC actual mirrors of each other in the direct sort case, as well? Hack the index scan to know about nodes that always go to the end? Document it as a quirk? (Not likely: selection of plan should never affect output.) To make the direct sort the same as the index read would work for NULL, but for NaN would either require allowing NaN to be returned as > Infinity, which doesn't happen now, or add another ordering operator that is only used for the sort case (use of '>' and '<' seems to be hardcoded all the way to the parser) Thoughts? Ross -- Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> NSBRI Research Scientist/Programmer Computer and Information Technology Institute Rice University, 6100 S. Main St., Houston, TX 77005
Attachment
"Ross J. Reedstrom" <reedstrm@rice.edu> writes: > While examining the output from ORDER BY queries, both using and not using > an index, I came across a discrepancy: the explicit handling of NULLs in > the tuplesort case always sorts NULLs to the end, regardless of direction > of sort. Yeah. I think that's widely considered a bug --- we have a TODO item to fix it. You might care to dig in the archives for prior discussions. > To make the direct sort the same as the index read would work for NULL, > but for NaN would either require allowing NaN to be returned as > > Infinity, which doesn't happen now, Seems to me the sort order should be -Infinitynormal values+Infinityother types of NaNNULL and the reverse in a descending sort. > or add another ordering operator that is only used for the sort case > (use of '>' and '<' seems to be hardcoded all the way to the parser) don't even think about that... regards, tom lane
At 10:28 AM 8/21/00 -0700, Adam Haberlach wrote: >On Mon, Aug 21, 2000 at 07:35:10AM -0700, Don Baccus wrote: >> Another implication which missed me first time 'round is that Great Bridge >> might be planning to have its own bug reporting system, separate from >> that used by the development community at large? > > Cool your conspiracy theories. I'm making an observation, that's all. Cool your own wild theories, please. > >> I hope not. There should be one central place for bug reporting. If >> Great Bridge wants to run it, fine, also if Great Bridge wants to be able >> to incorporate some sort of prioritization system for those with paid >> support (or some other discriminatory system) it is still probably better >> to figure out a way to accomodate it rather than have two separate >> bug reporting systems. > > The fact is that postgres already has a very good system for keeping >track of issues from report to fix to verification. <shrug> I didn't raise the subject, it was a core developer who started this thread with a semi-rant about it being about time that the project had decent bug tracking software. So apparently not everyone in the community agrees with your analysis. If there were consensus that the current system's great then Great Bridge wouldn't be looking at implementing something different, and Peter wouldn't be ranting that something better is needed. > So far the main defect >is the obvious one of "People don't know the history unless they troll the >message archives or lurk". Everyone here is leery of "fixing" a working >system. There seems to be some disagreement about how well it works. Again, I didn't raise the issue, I simply responded with a possible solution when one of the core developers raised it. And I know that Great Bridge wants to do something web-based - this isn't some fantasy I dreamed up when in a psychotic state. I'm only saying that if a different approach is to be taken, why not build on something that exists, is under active development, and is being driven by folks who are VERY open to working with the project to make the tool fit the project rather than vice-versa? > >Someone can grab an off-the-shelf system like Bugzilla or this ArsTechnica arsDigita >thing and then try to make the project conform to it. Read above. Ben's already posted that he's eager for design input. aD has already enhanced the thing based on their own needs, and there's no reason why Great Bridge and the Postgres crew can't do the same. >I don't >know anything about this other proposed system but it will probably require >a lot of time to even get people to use it regularly, much less use it well. Strange, OpenACS folk use it regularly and all we've done is put a "report a bug" link on the home page. I haven't heard so many arguments against change since the VT100 started replacing the KSR 35! - Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert Serviceand other goodies at http://donb.photo.net.
At 10:48 AM 8/21/00 -0400, Vince Vielhaber wrote: >Could it be possible that folks are shying away because of having >to install and learn an entire webserver and tools and then the >bug tracker on top of that? Learning to use AOLserver is going to be harder than writing a bugtracker and associated tools from scratch? I find that hard to believe. If it's true, of course they could run Apache, since arsDigita provides a module which implements the AOLserver API in Apache for exactly this reason, thus making it possible to run the toolkit (including the SDM) under Apache. - Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert Serviceand other goodies at http://donb.photo.net.
At 11:12 AM 8/21/00 -0400, Vince Vielhaber wrote: >I am *NOT* "the Postgres team". But have you listened to what you & Don >are suggesting that we, or for that matter anyone else in need of a bug >tracking system, do? You want us to install the full blown arsDigita with >all the bells and whistles just for a bug tracker. That's like saying I >need a pickup truck to move a chair so I'm going to go out and get a new >FreightLiner with a 55' trailer to do the job. A rather dubious analogy. It takes me less than half a day to install AOLserver, Postgres and the toolkit on a virgin system, including setting up user accounts, etc. And ... you never know. Other parts of the toolkit might turn out to be useful. If not, just leave them turned off. Take a look at openacs.org - do you find any traces of the e-commerce module there? The intranet company management module? What do you see? You see the use of perhaps 10% of the toolkit. This is slightly different than hauling an 18-wheeler around. Software and trucks bear little resemblence to each other, though Freightliner does have its home here in Portland, OR. And, of course, the SDM has a bit more functionality than a simple bugtracker, which is just one piece. It will be gaining more functionality over time, including increased integration with CVS (there is already some integration, i.e. the ability to browse the tree). >You missed the point. It's called overkill. You needed a full blown >database for your project. We need (although _want_ may be another story) >a bug tracker - not a new webserver. Then run it under Apache. - Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert Serviceand other goodies at http://donb.photo.net.
* Don Baccus <dhogaza@pacifier.com> [000821 11:48] wrote: > At 11:12 AM 8/21/00 -0400, Vince Vielhaber wrote: > > >You missed the point. It's called overkill. You needed a full blown > >database for your project. We need (although _want_ may be another story) > >a bug tracker - not a new webserver. > > Then run it under Apache. Sorry to jump in without reading the entire thread, but has GNATS (what the FreeBSD team uses) or Bugzilla come up as options? GNATS is a bit crusty but works pretty ok for us. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk."
On Mon, 21 Aug 2000, Don Baccus wrote: > At 10:48 AM 8/21/00 -0400, Vince Vielhaber wrote: > > >Could it be possible that folks are shying away because of having > >to install and learn an entire webserver and tools and then the > >bug tracker on top of that? > > Learning to use AOLserver is going to be harder than writing a > bugtracker and associated tools from scratch? I find that hard to > believe. Learning how to use it is only a tiny part of it. You still have to migrate your website to it. It's not a drop in replacement. So writing a bugtracker that will fit the environment vs learning a new webserver & migrating your website & rebuilding or rewriting custom apps ... For the average, busy admin, don't count too heavily on the latter. They're more likely to stick with what they know and trust regardless of how good something else is reported to be. > If it's true, of course they could run Apache, since arsDigita > provides a module which implements the AOLserver API in Apache > for exactly this reason, thus making it possible to run the > toolkit (including the SDM) under Apache. First I heard of this, but I'd also have concerns of it's reliability. It has to be real new. And if it fails it's not ars that looks bad, it's the site that's running it. Remember the flack over udmsearch? PostgreSQL was slammed over and over because udmsearch wasn't working right. Vince. -- ========================================================================== Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from $16.00/moat Pop4 Networking Online Campground Directory http://www.camping-usa.com Online Giftshop Superstore http://www.cloudninegifts.com ==========================================================================
On Mon, 21 Aug 2000, Don Baccus wrote: > At 11:12 AM 8/21/00 -0400, Vince Vielhaber wrote: > > >I am *NOT* "the Postgres team". But have you listened to what you & Don > >are suggesting that we, or for that matter anyone else in need of a bug > >tracking system, do? You want us to install the full blown arsDigita with > >all the bells and whistles just for a bug tracker. That's like saying I > >need a pickup truck to move a chair so I'm going to go out and get a new > >FreightLiner with a 55' trailer to do the job. > > A rather dubious analogy. > > It takes me less than half a day to install AOLserver, Postgres and the > toolkit on a virgin system, including setting up user accounts, etc. How familiar are you with it as opposed to most others on the net? > And ... you never know. Other parts of the toolkit might turn out to be > useful. > > If not, just leave them turned off. Take a look at openacs.org - do you > find any traces of the e-commerce module there? The intranet company > management module? What do you see? You see the use of perhaps 10% of > the toolkit. > > This is slightly different than hauling an 18-wheeler around. Software > and trucks bear little resemblence to each other, though Freightliner does > have its home here in Portland, OR. You really don't get it do you? I'm not comparing software and trucks, I'm comparing tools to do a job. Once you grasp that concept you'll be able to catch on to what I'm talking about - until then I'm just wasting my time. Vince. -- ========================================================================== Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from $16.00/moat Pop4 Networking Online Campground Directory http://www.camping-usa.com Online Giftshop Superstore http://www.cloudninegifts.com ==========================================================================
At 02:54 PM 8/21/00 -0400, Vince Vielhaber wrote: >On Mon, 21 Aug 2000, Don Baccus wrote: >> Learning to use AOLserver is going to be harder than writing a >> bugtracker and associated tools from scratch? I find that hard to >> believe. >Learning how to use it is only a tiny part of it. You still have to >migrate your website to it. It's not a drop in replacement. So >writing a bugtracker that will fit the environment vs learning a new >webserver & migrating your website & rebuilding or rewriting custom >apps ... For the average, busy admin, don't count too heavily on >the latter. They're more likely to stick with what they know and >trust regardless of how good something else is reported to be. So run the development portion of the site on a different server. Who said anything about migrating the entire postgres site to AOLserver??? >> If it's true, of course they could run Apache, since arsDigita >> provides a module which implements the AOLserver API in Apache >> for exactly this reason, thus making it possible to run the >> toolkit (including the SDM) under Apache. > >First I heard of this, but I'd also have concerns of it's reliability. >It has to be real new. Yep. Written by Robert Thau, one of the original eight core Apache developers, under contract to aD. >And if it fails it's not ars that looks bad, >it's the site that's running it. arsDigita gets on average greater than $500,000 to develop and deploy a website. If aD deploys one on Apache+mod_aolserver (they paid for the development of this module) and it falls over, do you really believe aD won't look bad? Seeing as they'd very likely be sued, I think you're wrong. - Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert Serviceand other goodies at http://donb.photo.net.
At 02:58 PM 8/21/00 -0400, Vince Vielhaber wrote: >On Mon, 21 Aug 2000, Don Baccus wrote: >> It takes me less than half a day to install AOLserver, Postgres and the >> toolkit on a virgin system, including setting up user accounts, etc. > >How familiar are you with it as opposed to most others on the net? Ben has already offered to help out and has an account on hub. If I weren't leaving town for five of the next six or so weeks, I would too. Still, we have strangers to all three pieces installing everything in a weekend, usually with a bit of help. If you were to install it, you'd be familiar with Postgres which is about 1/3 of the confusion for newcomers. >You really don't get it do you? Yes, I do. > I'm not comparing software and trucks, >I'm comparing tools to do a job. Once you grasp that concept you'll be >able to catch on to what I'm talking about - until then I'm just wasting >my time. Do I detect a flame? Ohhh... It's a toolkit, Vincent. Once you grasp the notion that using a wrench out of your toolbox doesn't mean you have to use every tool in the toolbox you'll be able to grasp what I'm talking about. I answered your previous question plainly. If you're not capable of understanding the answer, don't answer with an invitation for a flamefest you have no chance of winning, OK? - Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert Serviceand other goodies at http://donb.photo.net.
On Mon, Aug 21, 2000 at 01:39:32PM -0400, Tom Lane wrote: > "Ross J. Reedstrom" <reedstrm@rice.edu> writes: > > While examining the output from ORDER BY queries, both using and not using > > an index, I came across a discrepancy: the explicit handling of NULLs in > > the tuplesort case always sorts NULLs to the end, regardless of direction > > of sort. > > Yeah. I think that's widely considered a bug --- we have a TODO item to > fix it. You might care to dig in the archives for prior discussions. I'll take a look. > > > To make the direct sort the same as the index read would work for NULL, > > but for NaN would either require allowing NaN to be returned as > > > Infinity, which doesn't happen now, > > Seems to me the sort order should be > > -Infinity > normal values > +Infinity > other types of NaN > NULL > > and the reverse in a descending sort. > > > or add another ordering operator that is only used for the sort case > > (use of '>' and '<' seems to be hardcoded all the way to the parser) > > don't even think about that... Sure, but any ideas _how_ to get 'NaN > +Infinity' to happen only during a sort operation, and not when '>' is called explicity as a WHERE condition, by touching only type specific code? 'Cause I'd call it a bug to be able to say: SELECT * from foo where f8 > 'Infinity'; and get anything at all back. NULL is taken care of by special casing in the sort code, as I already mentioned, and can be fixed immediately. Ross -- Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> NSBRI Research Scientist/Programmer Computer and Information Technology Institute Rice University, 6100 S. Main St., Houston, TX 77005
At 04:34 PM 8/21/00 -0300, The Hermit Hacker wrote: >Ummm, just stupid question here, but I already posted that AOLserver was >installed and ready for you guys to implement this ... Ben already has an >account to get in and do it ... instead of arguing about it, why not just >do it? If nobody likes it/uses it, so be it ... its no skin off my >back. But right now the arguments that are going back and forth seem to >be sooooooo useless since they *seem* to involve technical issues that >aren't issues ... Well, this is a breath of fresh air ... Hopefully Ben will have time soon to do so. Unfortunately (well, not from my personal point of view!) I'm about to leave for five of the next six weeks, four of those spent where the internet doesn't reach (where nothing but BLM radio dispatch doesn't reach, to be more precise) so I'm not going to be able to help. Otherwise I'd just jump in. Of course, putting it up will just make its shortcomings apparent, and those who resist even the concept of change will ignore the fact that Ben and I have both stated that modifying the SDM to fit the project rather than modifying the project to fit the SDM and say, "see, this doesn't fit our needs!" blah blah blah. Just as the red herring you've mentioned has been raised. - Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert Serviceand other goodies at http://donb.photo.net.
Well, just to throw another piece of information into the mix, there is a new bug-tracking system under development by the folks at collab.net. They call it "scarab", and last I talked to them that they thought it would be ready for production use Real Soon Now: http://scarab.tigris.org/ I guess they regard this as a replacement for bugzilla. Some personal opinions: (1) I would actually like to see bugzilla fixed rather than replaced. The collab.net guys are into java servlets because they're CS geeks who are down on perl. Me, I'm a perl loyalist who thinks that Larry Wall is onto something -- mathematical elegance may not be the right standard to judge a computer language. (2) And maybe it'd be nice if the "religous wars" could be dropped in favor of pure objective technical decision making, but i don't think they can be: the social and the technical don't neatly split into two little piles. (A case in point: the argument that using a mailing list for bug control is somehow "warmer" or "more human" than a bug database.
On Mon, 21 Aug 2000, Don Baccus wrote: > At 02:58 PM 8/21/00 -0400, Vince Vielhaber wrote: > >On Mon, 21 Aug 2000, Don Baccus wrote: > > >> It takes me less than half a day to install AOLserver, Postgres and the > >> toolkit on a virgin system, including setting up user accounts, etc. > > > >How familiar are you with it as opposed to most others on the net? > > Ben has already offered to help out and has an account on hub. If I weren't > leaving town for five of the next six or so weeks, I would too. > > Still, we have strangers to all three pieces installing everything in a > weekend, usually with a bit of help. If you were to install it, you'd > be familiar with Postgres which is about 1/3 of the confusion for newcomers. > > >You really don't get it do you? > > Yes, I do. > > > I'm not comparing software and trucks, > >I'm comparing tools to do a job. Once you grasp that concept you'll be > >able to catch on to what I'm talking about - until then I'm just wasting > >my time. > > Do I detect a flame? Ohhh... > > It's a toolkit, Vincent. Once you grasp the notion that using a wrench > out of your toolbox doesn't mean you have to use every tool in the > toolbox you'll be able to grasp what I'm talking about. Yet you insist on shoving the entire toolbox down our throats every time there's a task to be done, Donnie. > I answered your previous question plainly. If you're not capable of > understanding the answer, don't answer with an invitation for a flamefest > you have no chance of winning, OK? When the day comes that you actually answer a question without telling the world that openacs, arsdigita, aolserver or whatever you want to call it is the answer and saviour to everything from world peace to who cares what else, I'll believe you answered a "question plainly". But that's not likely to ever happen. And since you think that anything I've said so far has been a flame I doubt you'd know one if it slapped you with a trout. One more thing, Donnie... ***PLONK*** Vince. -- ========================================================================== Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from $16.00/moat Pop4 Networking Online Campground Directory http://www.camping-usa.com Online Giftshop Superstore http://www.cloudninegifts.com ==========================================================================
[first off, I got rid of that awful cc: list..... ARggghhh....] Don Baccus wrote: > At 04:34 PM 8/21/00 -0300, The Hermit Hacker wrote: > >Ummm, just stupid question here, but I already posted that AOLserver was > >installed and ready for you guys to implement this ... Ben already has an > >account to get in and do it ... instead of arguing about it, why not just > Well, this is a breath of fresh air ... > Hopefully Ben will have time soon to do so. Unfortunately (well, not from I'm available to help some, and also have a hub account (as well as an operational AOLserver+PostgreSQL+OpenACS site to work on here....). -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
On Mon, 21 Aug 2000, Don Baccus wrote: > At 04:34 PM 8/21/00 -0300, The Hermit Hacker wrote: > > >Ummm, just stupid question here, but I already posted that AOLserver was > >installed and ready for you guys to implement this ... Ben already has an > >account to get in and do it ... instead of arguing about it, why not just > >do it? If nobody likes it/uses it, so be it ... its no skin off my > >back. But right now the arguments that are going back and forth seem to > >be sooooooo useless since they *seem* to involve technical issues that > >aren't issues ... > > Well, this is a breath of fresh air ... > > Hopefully Ben will have time soon to do so. Unfortunately (well, not from > my personal point of view!) I'm about to leave for five of the next six > weeks, four of those spent where the internet doesn't reach (where nothing > but BLM radio dispatch doesn't reach, to be more precise) so I'm not going > to be able to help. > > Otherwise I'd just jump in. > > Of course, putting it up will just make its shortcomings apparent, and > those who resist even the concept of change will ignore the fact that > Ben and I have both stated that modifying the SDM to fit the project > rather than modifying the project to fit the SDM and say, "see, this > doesn't fit our needs!" blah blah blah. > > Just as the red herring you've mentioned has been raised. Right now, you've thrown out "it can do this, it can do that" ... I've put forth the resources so that you can prove that it will work, its in your court now :)
On Mon, 21 Aug 2000, Vince Vielhaber wrote: > On Mon, 21 Aug 2000, Don Baccus wrote: > > > At 10:48 AM 8/21/00 -0400, Vince Vielhaber wrote: > > > > >Could it be possible that folks are shying away because of having > > >to install and learn an entire webserver and tools and then the > > >bug tracker on top of that? > > > > Learning to use AOLserver is going to be harder than writing a > > bugtracker and associated tools from scratch? I find that hard to > > believe. > > Learning how to use it is only a tiny part of it. You still have to > migrate your website to it. It's not a drop in replacement. So > writing a bugtracker that will fit the environment vs learning a new > webserver & migrating your website & rebuilding or rewriting custom > apps ... For the average, busy admin, don't count too heavily on the > latter. They're more likely to stick with what they know and trust > regardless of how good something else is reported to be. why not just run aolserver on a different port like we do?
At 7:48 PM -0300 8/20/00, The Hermit Hacker wrote: >what's wrong wth "Post-Gres-QL"? > >I find it soooo simple to pronounce *shrug* > >On Sun, 20 Aug 2000, David Lloyd-Jones wrote: > > > I'm putting on my suits-type suit for just a moment. > > > > In order to Conquer The Universe(tm) why don't we just call it "PG"? > > IMNSHO the "QL" are silent and you pronounce it Postgres. I consider the SQL query language to be merely a (major) feature added during the evolution of the system. Signature held pending an ISO 9000 compliant signature design and approval process. h.b.hotz@jpl.nasa.gov, or hbhotz@oxy.edu
On Mon, 21 Aug 2000, The Hermit Hacker wrote: > On Mon, 21 Aug 2000, Vince Vielhaber wrote: > > > On Mon, 21 Aug 2000, Don Baccus wrote: > > > > > At 10:48 AM 8/21/00 -0400, Vince Vielhaber wrote: > > > > > > >Could it be possible that folks are shying away because of having > > > >to install and learn an entire webserver and tools and then the > > > >bug tracker on top of that? > > > > > > Learning to use AOLserver is going to be harder than writing a > > > bugtracker and associated tools from scratch? I find that hard to > > > believe. > > > > Learning how to use it is only a tiny part of it. You still have to > > migrate your website to it. It's not a drop in replacement. So > > writing a bugtracker that will fit the environment vs learning a new > > webserver & migrating your website & rebuilding or rewriting custom > > apps ... For the average, busy admin, don't count too heavily on the > > latter. They're more likely to stick with what they know and trust > > regardless of how good something else is reported to be. > > why not just run aolserver on a different port like we do? If it has the functionality you desire and are looking for, sure. My contention is that the average site isn't going to do it for a couple of features and would roll their own instead. Vince. -- ========================================================================== Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from $16.00/moat Pop4 Networking Online Campground Directory http://www.camping-usa.com Online Giftshop Superstore http://www.cloudninegifts.com ==========================================================================
On Mon, 21 Aug 2000, Henry B. Hotz wrote: > At 7:48 PM -0300 8/20/00, The Hermit Hacker wrote: > >what's wrong wth "Post-Gres-QL"? > > > >I find it soooo simple to pronounce *shrug* > > > >On Sun, 20 Aug 2000, David Lloyd-Jones wrote: > > > > > I'm putting on my suits-type suit for just a moment. > > > > > > In order to Conquer The Universe(tm) why don't we just call it "PG"? > > > > > IMNSHO the "QL" are silent and you pronounce it Postgres. I consider > the SQL query language to be merely a (major) feature added during > the evolution of the system. as stated in another email, it is not pronounced Postgres ... postgres was a whole different beast based on a completely different query language ... if you refer to Postgres, you are, IMHO, refering to the only Postgres 4.2 which was the grand-daddy to what its evolved into ...
[trimmed cc: list] Vince Vielhaber wrote: > On Mon, 21 Aug 2000, Don Baccus wrote: > > Vince wrote: > > >You really don't get it do you? > > Yes, I do. Vince, Don really does 'get it' -- he's just pretty vehement about his 'getting it'. > > It's a toolkit, Vincent. Once you grasp the notion that using a wrench > > out of your toolbox doesn't mean you have to use every tool in the > > toolbox you'll be able to grasp what I'm talking about. > Yet you insist on shoving the entire toolbox down our throats every time > there's a task to be done, Donnie. One of the many useful features of OpenACS is that you get the whole toolbox -- a toolbox, as opposed to a 'box of tools' -- Jensen makes a nice profit selling toolboxes with matched tools -- neat, clean, trim, and don't look anything like my four-drawer toolbox made up of a melange of tools and a Wal-mart toolbox. OpenACS is like the Jensen toolset -- you get a matched case, and high-quality tools matches to the case. With OpenACS you get a framework that tools can be plugged into -- tools that were designed to be plugged in that way (well, it's not perfect -- but nothing is). Everything can be covered by the system-wide authentication module, user group module, etc. Everything is designed to work smoothly together. so you only use authentication+SDM -- so what. You can expand as you need to -- and it doesn't take up _that_ much space. PostgreSQL is much like OpenACS (barring the funny capitalization, but I digress): PostgreSQL is a toolbox of database routines and modules, tied together by a SQL parser and a large set of clients with matching arbiter/backends. Download postgresql-version.tar.gz, and you _have_ to get the C++ code -- even if you don't want it. You have to get pgaccess -- even if you won't use it. If you want to do meaningful development, you have to keep nearly the whole source tree around......etc. How is this different from the OpenACS model? Don't want a part of OpenACS? Nobody is preventing you from nuking that part from your installation tree -- just like no one is preventing someone from installing PostgreSQL in a client-only sort of way (much easier with an RPMset, but I again digress.....). PostgreSQL requires libpq -- OpenACS requires (but doesn't include) AOLserver -- that analogy is not perfect, but close. And, OpenACS can run just fine under Apache with mod_aolserver. Although, since Marc has an AOLserver available and running.... :-) and a killer db server (bigger :-))... Vince, Don: sparring like this is not productive. Both of you are excellent hackers -- I've seen both of your code. Let's just make it work, and see what the hackers think of it. > > I answered your previous question plainly. If you're not capable of > > understanding the answer, don't answer with an invitation for a flamefest > > you have no chance of winning, OK? Flamefests are unwinnable. All parties to flamwars get burned -- either directly, or indirectly. I have seen too many flamewars -- and it's not worth the risk to reputation to go too far with one. This one is mild so far -- on a scale from one to ten, this makes it to one-and-a-half thus far (I've been on news.groups and news.admin (and cross-posted to alt.flame) more than once several years back....). > When the day comes that you actually answer a question without telling > the world that openacs, arsdigita, aolserver or whatever you want to call > it is the answer and saviour to everything from world peace to who cares Vince, try it. You might like it. But, it does require some different thinking -- which if you don't have time to do, your loss. Or, to put it differently -- I just recently learned how to write CGI scripts. That may seem laughable -- but I had already written many dynamic web pages using AOLserver TCL -- who needs CGI? The current rush on PHP programmers shows this -- many PHP programmers wouldn't have a clue how to go about writing a CGI script -- and, guess what -- with PHP you don't need CGI. The AOLserver TCL API is like PHP on steroids in many case -- and, in many other cases, PHP out-API's AOLserver. And I do remember that you _have_ tried AOLserver, about a year ago.... -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
[cc: list trimmed] Vince Vielhaber wrote: > On Mon, 21 Aug 2000, The Hermit Hacker wrote: > > why not just run aolserver on a different port like we do? > If it has the functionality you desire and are looking for, sure. My > contention is that the average site isn't going to do it for a couple > of features and would roll their own instead. Isn't this the biggest reason people give for using MySQL and not PostgreSQL???? (re: transaction support, triggers, etc). -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
On Mon, 21 Aug 2000, Vince Vielhaber wrote: > > why not just run aolserver on a different port like we do? > > If it has the functionality you desire and are looking for, sure. My > contention is that the average site isn't going to do it for a couple > of features and would roll their own instead. Sounds like a big waste of time investment when few of us have much of it to start with ... I've talked to Ben and he's going to work on getting the OpenACS version up and running ... once he has that going, he's going to talk to you (Vince) about getting the look to match what our web site looks like ... once we have that in place, the next step will be to customize it so that it does what *we* want it to do ...
"Ross J. Reedstrom" <reedstrm@rice.edu> writes: >>>> or add another ordering operator that is only used for the sort case >>>> (use of '>' and '<' seems to be hardcoded all the way to the parser) >> >> don't even think about that... > Sure, but any ideas _how_ to get 'NaN > +Infinity' to happen only during a > sort operation, and not when '>' is called explicity as a WHERE condition, > by touching only type specific code? That's exactly what you shouldn't even think about. The entire index and sorting system is predicated on the assumption that '<' and related operators agree with the order induced by a btree index. You do not get to make the operators behave differently in the free-standing case than when they are used with an index. > 'Cause I'd call it a bug to be able to say: > SELECT * from foo where f8 > 'Infinity'; > and get anything at all back. I agree it's pretty arbitrary to define NaN as > Infinity, but the sort ordering is necessarily arbitrary. We can special-case NULL because that's a type-independent concept, but special-casing NaN is out of the question IMHO. Pick where you want it in the type-specific order, and live with it. regards, tom lane
On Mon, Aug 21, 2000 at 02:32:08PM -0500, Ross J. Reedstrom wrote: > On Mon, Aug 21, 2000 at 01:39:32PM -0400, Tom Lane wrote: > > > > Seems to me the sort order should be > > > > -Infinity > > normal values > > +Infinity > > other types of NaN > > NULL > > > > and the reverse in a descending sort. > > > > NULL is taken care of by special casing in the sort code, as I already mentioned, > and can be fixed immediately. > Grr, I take this back. By the time comparetup_* see the tuples, we've no idea which order we're sorting in, just a pointer to the appropriate sortop. <whine mode> Why does every thing I touch in pgsql end up pulling in down into the guts of the whole system? Even something that looks nicely factored at first, like the type system? I guess this stuff is _hard_. </whine mode> Sigh, back to fixing up referential integrity violations in the DB I'm finally upgrading from 6.5 to 7.0.X. (DBA life lesson number XX: implementing RI in the backend _from the very start of a project_ is a very good thing. Cleansing the data of cruft left from _not_ having RI in the backend, is a bad thing. And those clever, recursive self-referencing table structures for representing trees are a pain in the DBA to reload.) Ross -- Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> NSBRI Research Scientist/Programmer Computer and Information Technology Institute Rice University, 6100 S. Main St., Houston, TX 77005
At 5:16 PM -0300 8/21/00, The Hermit Hacker wrote: >On Mon, 21 Aug 2000, Henry B. Hotz wrote: > > > At 7:48 PM -0300 8/20/00, The Hermit Hacker wrote: > > >what's wrong wth "Post-Gres-QL"? > > > > > >I find it soooo simple to pronounce *shrug* > > > > > >On Sun, 20 Aug 2000, David Lloyd-Jones wrote: > > > > > > > I'm putting on my suits-type suit for just a moment. > > > > > > > > In order to Conquer The Universe(tm) why don't we just call it "PG"? > > > > > > > > IMNSHO the "QL" are silent and you pronounce it Postgres. I consider > > the SQL query language to be merely a (major) feature added during > > the evolution of the system. > >as stated in another email, it is not pronounced Postgres ... postgres was >a whole different beast based on a completely different query language >... if you refer to Postgres, you are, IMHO, refering to the only >Postgres 4.2 which was the grand-daddy to what its evolved into ... ;-) Signature held pending an ISO 9000 compliant signature design and approval process. h.b.hotz@jpl.nasa.gov, or hbhotz@oxy.edu
Hi Joe, We're looking at elements of the Tigris architecture for the site that I referenced in an earlier email (so Don, we're not reinventing the wheel). Scarab has apparently been in development for awhile now - the Tigris project is currently using Bugzilla until Scarab itself is ready for prime time, then they intend to switch it over. We've also spent some time looking at an app called BugRat (even ported it to Postgres from MySQL)... it seems to offer a pretty good mix of features as well. The greatbridge.org site will likely get started with either BugRat or Bugzilla, then transitition to Scarab when/if it's ready. BTW, the goal of the site (which I didn't really explain too well in my earlier message) will be to provide a hosting infrastructure for some related projects (like interfaces, or the apps and tools we're listing at http://www.greatbridge.com/tools/toollist.php)... also, all the software we develop internally will go up on that site. So please don't interpret our interest as getting in the middle of what the main Postgres project uses for bug tracking - although the requirements outlined in the earlier messages was helpful just for our own development of the greatbridge.org site. If Ben and the OpenACS gang are willing to put the time into a bug tracker for the project, I think that couldn't help but be a good thing. Regards, Ned Joe Brenner wrote: > Well, just to throw another piece of information into the > mix, there is a new bug-tracking system under development by > the folks at collab.net. They call it "scarab", and last I > talked to them that they thought it would be ready for > production use Real Soon Now: > > http://scarab.tigris.org/ > > I guess they regard this as a replacement for bugzilla. > > Some personal opinions: > > (1) I would actually like to see bugzilla fixed rather than > replaced. The collab.net guys are into java servlets > because they're CS geeks who are down on perl. Me, I'm a > perl loyalist who thinks that Larry Wall is onto something > -- mathematical elegance may not be the right standard to > judge a computer language. > > (2) And maybe it'd be nice if the "religous wars" could be > dropped in favor of pure objective technical decision > making, but i don't think they can be: the social and the > technical don't neatly split into two little piles. (A case > in point: the argument that using a mailing list for bug > control is somehow "warmer" or "more human" than a bug > database. >
At 1:25 AM -0400 8/21/00, Tom Lane wrote: >"Ross J. Reedstrom" <reedstrm@rice.edu> writes: > > ! if (isnan(a)) > > ! PG_RETURN_INT32(1); > >Do not like this at all --- doesn't it make the result of btint4cmp(NaN, >NaN) dependent on which argument chances to be first? Seems to me that >you must consider two NaNs to be equal, unless you want to subdivide >the category of NaNs. I'm pretty sure IEEE 754 says that NaN does not compare equal to anything, including themselves. I also believe that Infinity isn't equal to itself either, it's just bigger than anything else except NaN (which isn't littler either). Without having seen the start of this thread I think the biggest problem is that some of the results of compares depend on the mode that the FP hardware is put in. IEEE specifies some modes, but not how you set the mode you want on the system you are actually running on. For example I think comparing zero and -Infinity may return three possible results: 0 > -Infinity, 0 < -Infinity (because it was told to ignore the sign of Infinity), or an illegal value exception. Likewise signalling/non-signalling NaN's act different depending on mode settings. We need to first figure out what floating point behavior we want to support. Then we figure what mode settings provide that behavior with a minimum of overhead. Then we need to figure out how to set those modes on all the platforms we support. We will probably discover that not all required modes actually exist on all hardware platforms. I know that 68000 and SPARC are pretty good, but PowerPC punted some stuff to exception handlers which may not be correct on all OS's. I've heard that Java has some portability issues because Intel fudged some stuff in the newer hardware. Does anyone feel like tracing down how to set the modes for all the different systems that we try to run on? If there is interest then I might poke at a couple/three NetBSD ports and Solaris/SPARC. But only if there is interest. Sun has put some code out under GPL which will let you test for these special values and handle them, but that seems like a big hit for what should be a simple compare. I assume that we can't put GPL code into the main sources any more than the *BSD's do. Perhaps we could get away with it if it is only included if configure can't figure out how to set the modes properly? Signature held pending an ISO 9000 compliant signature design and approval process. h.b.hotz@jpl.nasa.gov, or hbhotz@oxy.edu
On Mon, Aug 21, 2000 at 04:37:21PM -0400, Tom Lane wrote: > "Ross J. Reedstrom" <reedstrm@rice.edu> writes: > >>>> or add another ordering operator that is only used for the sort case > >>>> (use of '>' and '<' seems to be hardcoded all the way to the parser) > >> > >> don't even think about that... > > > Sure, but any ideas _how_ to get 'NaN > +Infinity' to happen only during a > > sort operation, and not when '>' is called explicity as a WHERE condition, > > by touching only type specific code? > > That's exactly what you shouldn't even think about. The entire index > and sorting system is predicated on the assumption that '<' and related > operators agree with the order induced by a btree index. You do not get > to make the operators behave differently in the free-standing case than > when they are used with an index. Oh really? Then why do btree's have their own comparator functions, seperate from heap sorts, and datum sorts, and explicit use of '<' ? The current code infrastructure allows for the possibility that these may need to diverge, requiring the coders to keep them in sync. Annoying, that, but useful for edge cases. Since btree already uses it's own comparator, The only reason I can see that the parser drops in '<' and '>' as the name of the sorting operator to use for ORDER BY is convenience: the functions are there, and have the (mostly) correct behavior. Changing this would only require writing another set of operators for the parser to drop in, that are used only for sorting, so that the sort behavior could diverge slightly, by knowing how to sort NULLs and NaNs. Yes, it'd be a third set of operators to keep in sync with the btree and default ones, but it could give completely correct behavior. Hmm, I another thought: all the comparator code assumes (a<b || a>b || a==c) and therefor only test 2 of the three conditions, falling through to the third. In the three places I just looked, two fall thorough to the equal case, and one to the 'less than' case. If all three fell through to the 'greater than' case, it might work with no tweaking at all. I'll have to try that, first. Ross -- Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> NSBRI Research Scientist/Programmer Computer and Information Technology Institute Rice University, 6100 S. Main St., Houston, TX 77005
On Mon, Aug 21, 2000 at 05:30:21PM -0500, Ross J. Reedstrom wrote: > > Hmm, I another thought: all the comparator code assumes (a<b || a>b || a==c) > and therefor only test 2 of the three conditions, falling through to the > third. In the three places I just looked, two fall thorough to the equal case, > and one to the 'less than' case. If all three fell through to the 'greater than' > case, it might work with no tweaking at all. I'll have to try that, first. Looking again, I realize that the sort comparetup_* code doesn't have access to a operator to test for equality, so can't do this. Sigh. Time to go home, I think. Ross -- Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> NSBRI Research Scientist/Programmer Computer and Information Technology Institute Rice University, 6100 S. Main St., Houston, TX 77005
"Ross J. Reedstrom" <reedstrm@rice.edu> writes: > On Mon, Aug 21, 2000 at 04:37:21PM -0400, Tom Lane wrote: >> That's exactly what you shouldn't even think about. The entire index >> and sorting system is predicated on the assumption that '<' and related >> operators agree with the order induced by a btree index. You do not get >> to make the operators behave differently in the free-standing case than >> when they are used with an index. > Oh really? Then why do btree's have their own comparator functions, > seperate from heap sorts, and datum sorts, and explicit use of '<' ? Strictly and only to save a few function-call cycles. Some paths in the btree code need a three-way comparison (is A<B, or A=B, or A>B?) and about half the time you'd need two calls to type-specific comparator functions to make that determination if you only had the user-level operators available. This does *not* mean that you have license to make the 3-way comparator's behavior differ from the operators, because the operators are used too. Note also that it is a three-way comparison function, not four-way: there is no provision for answering "none of the above" (except when a NULL is involved, and that only works because it's special-cased without calling type-specific code at all). The reason the sort code doesn't use the comparator routine is strictly historical, AFAICT. It really should, for speed reasons; but there may not be a 3-way comparator associated with a given '<' operator, and we've got a longstanding convention that a user-selected sort order is specified by naming a particular '<'-like operator. It may also be worth pointing out that the sort code still assumes trichotomy: it tests A<B, and if that is false it tries B<A, and if that's also false then it assumes A=B. There's still no room for an "unordered" response. > The current code infrastructure allows for the possibility that these > may need to diverge, requiring the coders to keep them in > sync. Annoying, that, but useful for edge cases. It is annoying. Many of the datatypes where comparison is nontrivial actually use an underlying 3-way comparison routine that the boolean comparators call, so as to avoid code-divergence problems. > Changing this would only require writing another set of operators for > the parser to drop in, that are used only for sorting, No, because *the user-level operators must match the index*. How many times do I have to repeat that? The transformation that allows, say,SELECT * FROM tab WHERE foo > 33 AND foo < 42 to be implemented by an indexscan (of an index on foo) is fundamentally dependent on the assumption that the operators '>' and '<' induce the same ordering of data values as is stored in the index. Otherwise you can't scan a subrange of the index and know that you've hit all the matching rows. The planner actually takes considerable care to verify that the operators appearing in WHERE *do* match the index ordering --- that's what pg_opclass and pg_amop are all about. If you invent an internal set of operators that provide a different index ordering, you will find that the planner ignores your index. regards, tom lane
On Mon, Aug 21, 2000 at 11:34:45AM -0700, Don Baccus wrote: > Strange, OpenACS folk use it regularly and all we've done is put a "report > a bug" link on the home page. ...I'm not sure you noticed, but this project isn't OpenACS. It is an established project with decent management that only needs a few features. Switching everything over to a canned solution, no matter how good of a toolbox you feel it is, is not necessarily going to solve the few problems we have without causing a whole host of new ones... > I haven't heard so many arguments against change since the VT100 started > replacing the KSR 35! Oh, and in case you didn't hear it earlier... PLONK. -- Adam Haberlach | "A farm tractor is not a motorcycle." adam@newsnipple.com | --California DMV 1999 http://www.newsnipple.com/ | Motorcycle Driver Handbook
The Hermit Hacker wrote: > > what's wrong wth "Post-Gres-QL"? > > I find it soooo simple to pronounce *shrug* Mee too. And I'm not sure if anybody pronounces PG the same way. Is it PeeGee or PiG (in which case we'd have the wrong animal in our logo). Jan > > On Sun, 20 Aug 2000, David Lloyd-Jones wrote: > > > I'm putting on my suits-type suit for just a moment. > > > > In order to Conquer The Universe(tm) why don't we just call it "PG"? > > > > -dlj. > > > > > > > > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy > Systems Administrator @ hub.org > primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org > -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
On Mon, 21 Aug 2000, Adam Haberlach wrote: > On Mon, Aug 21, 2000 at 11:34:45AM -0700, Don Baccus wrote: > > > Strange, OpenACS folk use it regularly and all we've done is put a "report > > a bug" link on the home page. > > ...I'm not sure you noticed, but this project isn't OpenACS. It is > an established project with decent management that only needs a few features. > Switching everything over to a canned solution, no matter how good of a > toolbox you feel it is, is not necessarily going to solve the few problems > we have without causing a whole host of new ones... Ummmm, who was talking about switching anything over to a canned solution? *raised eyebrow* we are talking about allowing the OpenACS folks setup a bug tracking system for us and seeing if it servces us better then other attempts have in the past ...
At 09:40 PM 8/21/00 -0300, The Hermit Hacker wrote: >On Mon, 21 Aug 2000, Adam Haberlach wrote: > >> On Mon, Aug 21, 2000 at 11:34:45AM -0700, Don Baccus wrote: >> >> > Strange, OpenACS folk use it regularly and all we've done is put a "report >> > a bug" link on the home page. >> >> ...I'm not sure you noticed, but this project isn't OpenACS. It is >> an established project with decent management that only needs a few features. >> Switching everything over to a canned solution, no matter how good of a >> toolbox you feel it is, is not necessarily going to solve the few problems >> we have without causing a whole host of new ones... > >Ummmm, who was talking about switching anything over to a canned >solution? *raised eyebrow* we are talking about allowing the OpenACS >folks setup a bug tracking system for us and seeing if it servces us >better then other attempts have in the past ... Which it probably won't - customization is the key, we're not quite as stupid as Adam makes us out to be. Here's an idea: How about ignoring whether or not a new solution is borne, or the SDM can be customized to fit your needs with less work than, and concentrating on what features you want to see? Has anyone thought about this, or was the last attempt such a failure that people have given it no further thought? - Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert Serviceand other goodies at http://donb.photo.net.
Don Baccus wrote: > > How about ignoring whether or not a new solution is borne, or the SDM > can be customized to fit your needs with less work than, and concentrating > on what features you want to see? > > Has anyone thought about this, or was the last attempt such a failure that > people have given it no further thought? > I don't know what the major code contributors need beyond the TODO list. But I remember what went wrong with the older system -- people would post non-bug issues, and in large numbers, as bugs. And the system would "pend" those non-issues, assigning them to core developers, who, at the time, were very busy implementing MVCC and crushing real bugs by the hundreds. It seems all Peter wants is a system whereby authorized users (presumably those with CVS privileges) would have the ability to post and close bugs. Perhaps such a system might have prevented the duplicate work done recently on the "binary compatibility WRT functional indexes" issue. Just from lurking, I think the core developers' consensus was that anything which allows Joe User to open tickets, without a "front-line" of advanced users/minor code contributors willing to act as filters, would consume too much time. People with great frequency ignore the note on the web-site which reads "Note: You must post elsewhere first" with respect to the pgsql-hackers list. So I don't think it was an issue with the technology, but the process. Although, from what I've read, I suspect ArsDigita is light-years ahead of the Keystone software that was the "PostgreSQL Bug Tracking System". P.S.: I've been looking forward to seeing ArsDigita running on postgresql.org for some time. I suspect there would be some short-term pain, but substantial long-term gain. :-) Mike Mascari
On Tue, Aug 22, 2000 at 02:16:44PM +0200, Peter Eisentraut wrote: > Ross J. Reedstrom writes: > > > Fixing sorts is a bit tricker, but can be done: Currently, I've hacked > > the float8lt and float8gt code to sort NaN to after +/-Infinity. (since > > NULLs are special cased, they end up sorting after NaN). I don't see > > any problems with this solution, and it give the desired behavior. > > SQL 99, part 5, section 17.2 specifies that the sort order for ASC and > DESC is defined in terms of the particular type's < and > operators. > Therefore the NaN's must always be at the end. (Before or after NULL is > implementation-defined, btw.) I'm not sure what your suggesting, Peter. Which is 'the end'? And how does 'Therefore' follow from considering the type behavior of NaN and the < and > operators ? I think your suggesting that NaN always sort to one end, either greater than Infinity or less than -Infinity, regardless of sort direction. Therefore, depending on the direction of ORDER BY, NaNs will be returned either be first or last, not always last, as I've currently implemented. I agree with this, but my reason comes from the required treatment of NULLs. My reasoning is as follows: The standard says (17.2): The relative position of rows X and Y in the result is determined by comparing XV(i) and YV(i) according to the rulesof Subclause 8.2, "<comparison predicate>", in ISO/IEC 9075-2, where the <comp op> is the applicable <comp op>for K(i), [...] and Subclause 8.2 says: 2) Numbers are compared with respect to their algebraic value. However, NaN is _not_ algebraically > or < any other number: in fact, General Rule 1. of subclause 8.2 does deal with this: 5) X <comp op> Y is_unknown if X <comp op> Y is neither true_ nor false_ . So, we're left with not knowing where to put NaN. However, the only other case where the comparision is unknown is: a) If either XV or YV is the null value, then X <comp op> Y is unknown_ . And, going back to section 17.2: [...] where the <comp op> is the applicable <comp op> for K(i), with the following special treatment of null values.Whether a sort key value that is null is considered greater or less than a non-null value is implementation-defined,but all sort key values that are null shall either be considered greater than all non-null valuesor be considered less than all non-null values. So, NULLs go at one end (less or greater), always, so NaN should as well. And NULL will go outside them, since NULLs are required to be considered greater than (in our case) all non-null values (including NaN). Ross -- Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> NSBRI Research Scientist/Programmer Computer and Information Technology Institute Rice University, 6100 S. Main St., Houston, TX 77005 > Ross J. Reedstrom writes: > > > Fixing sorts is a bit tricker, but can be done: Currently, I've hacked > > the float8lt and float8gt code to sort NaN to after +/-Infinity. (since > > NULLs are special cased, they end up sorting after NaN). I don't see > > any problems with this solution, and it give the desired behavior. > > SQL 99, part 5, section 17.2 specifies that the sort order for ASC and > DESC is defined in terms of the particular type's < and > operators. > Therefore the NaN's must always be at the end. (Before or after NULL is > implementation-defined, btw.) > > > -- > Peter Eisentraut Sernanders väg 10:115 > peter_e@gmx.net 75262 Uppsala > http://yi.org/peter-e/ Sweden >
On Tue, Aug 22, 2000 at 08:22:21PM +0200, Peter Eisentraut wrote: > > Hmm, I'm getting the feeling that perhaps at this point we should > explicitly *not* support NaN at all. After all, the underlying reason for > offering them is to provide IEEE 754 compliant floating point arithmetic, > but if we start making compromises such as NaN == NaN or NaN > +Infinity > then we might as well not do it. In these cases I opine that if you can't > do something correctly then you should perhaps be honest and don't do > it. After all, users that want a "not-a-number" can use NULL in most > cases, and hard-core floating point users are going to fail miserably > with the FE/BE protocol anyway. > Pretty much were I have come to on this, as well. The point is to get the existing NaN to not break indicies or sorting. The simplest way is to disable it. Ross -- Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> NSBRI Research Scientist/Programmer Computer and Information Technology Institute Rice University, 6100 S. Main St., Houston, TX 77005
> On Tue, Aug 22, 2000 at 08:22:21PM +0200, Peter Eisentraut wrote: >> Hmm, I'm getting the feeling that perhaps at this point we should >> explicitly *not* support NaN at all. Well ... this is a database, not a number-crunching system. It seems to me that we should be able to store and retrieve NaNs (at least on IEEE-compliant platforms). But I'm less excited about whether the sorting/comparison operators we offer are 100% IEEE-compliant. It has been quite a few years since I looked closely at the IEEE FP specs, but I do still recall that they made a distinction between "IEEE aware" and "non IEEE aware" comparison operators --- specifically, the first kind understood about unordered comparisons and the second didn't. Perhaps we could satisfy both SQL and IEEE requirements if we stipulate that we implement only IEEE's "non-aware" comparisons? Worth looking at anyway. >> ... hard-core floating point users are going to fail miserably >> with the FE/BE protocol anyway. It would be a mistake to design backend behavior on the assumption that we'll never have an FE/BE protocol better than the one we have today. (You could actually fix this problem without any protocol change, just a SET variable to determine output precision for FP values. Well-written platforms will reproduce floats exactly given "%.17g" or more precision demand in sprintf. If that fails it's libc's bug not ours.) regards, tom lane
My assumption is that we never came up with any solution to this, right? > On Sun, Aug 20, 2000 at 12:33:00AM +0200, Peter Eisentraut wrote: > <snip side comment about bug tracking. My input: for an email controllable > system, take a look at the debian bug tracking system> > > > Show me a system where it doesn't work and we'll get it to work. > > UNSAFE_FLOATS as it stands it probably not the most appropriate behaviour; > > it intends to speed things up, not make things portable. > > > > I agree. In the previous thread on this, Thomas suggested creating a flag > that would allow control turning the CheckFloat8Val function calls into > a macro NOOP. Sound slike a plan to me. > > > > > > > NULL and NaN are not quite the same thing imho. If we are allowing NaN > > > > in columns, then it is *known* to be NaN. > > > > > > For the purposes of ordering, however, they are very similar. > > > > Then we can also treat them similar, i.e. sort them all last or all first. > > If you have NaN's in your data you wouldn't be interested in ordering > > anyway. > > Right, but the problem is that NULLs are an SQL language feature, and > there for rightly special cased directly in the sorting apparatus. NaN is > type specific, and I'd be loath to special case it in the same place. As > it happens, I've spent some time this weekend groveling through the sort > (and index, as it happens) code, and have an idea for a type specific fix. > > Here's the deal, and an actual, honest to goodness bug in the current code. > > As it stands, we allow one non-finite to be stored in a float8 field: > NaN, with partial parsing of 'Infinity'. > > As I reported last week, NaNs break sorts: they act as barriers, creating > sorted subsections in the output. As those familiar with the code have > already guessed, there is a more serious bug: NaNs break indicies on > float8 fields, essentially chopping the index off at the first NaN. > > Fixing this turns out to be a one liner to btfloat8cmp. > > Fixing sorts is a bit tricker, but can be done: Currently, I've hacked > the float8lt and float8gt code to sort NaN to after +/-Infinity. (since > NULLs are special cased, they end up sorting after NaN). I don't see > any problems with this solution, and it give the desired behavior. > > I've attached a patch which fixes all the sort and index problems, as well > as adding input support for -Infinity. This is not a complete solution, > since I haven't done anything with the CheckFloat8Val test. On my > system (linux/glibc2.1) compiling with UNSAFE_FLOATS seems to work fine > for testing. > > > > > Side note 2: The paper "How Java's floating point hurts everyone > > everywhere" provides for good context reading. > > http://http/cs.berkeley.edu/~wkahan/JAVAhurt.pdf ? I'll take a look at it > when I get in to work Monday. > > > > > Side note 3: Once you read that paper you will agree that using floating > > point with Postgres is completely insane as long as the FE/BE protocol is > > text-based. > > Probably. But it's not our job to enforce sanity, right? Another way to think > about it is fixing the implementation so the deficiencies of the FE/BE stand > out in a clearer light. ;-) > > Ross > -- > Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> > NSBRI Research Scientist/Programmer > Computer and Information Technology Institute > Rice University, 6100 S. Main St., Houston, TX 77005 > [ Attachment, skipping... ] -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Bruce Momjian writes: > My assumption is that we never came up with any solution to this, right? It stopped when we noticed that proper support for non-finite values will break indexing, because the relational trichotomy doesn't hold. -- Peter Eisentraut peter_e@gmx.net http://yi.org/peter-e/
[ continuing a discussion from last August ] Peter Eisentraut <peter_e@gmx.net> writes: > Bruce Momjian writes: >> My assumption is that we never came up with any solution to this, right? > It stopped when we noticed that proper support for non-finite values will > break indexing, because the relational trichotomy doesn't hold. I believe that's not a problem anymore. The current form of the float comparison functions will perform sorting and comparisons according to the sequence -infinity < normal values < infinity < NaN < NULL with all NaNs treated as equal. This may not be exactly what an IEEE purist would like, but given that we have to define *some* consistent sort order, it seems as reasonable as we can get. Accordingly, I suggest that Ross go back to work on persuading the code to treat infinities and NaNs properly in other respects. IIRC, there are still open issues concerning whether we still need/want CheckFloat8Val/CheckFloat4Val, what the I/O conversion functions should do on non-IEEE machines, etc. They all seemed soluble, though. regards, tom lane
Tom Lane writes: > [ continuing a discussion from last August ] [I was *just* thinking about this. Funny.] > I believe that's not a problem anymore. The current form of the float > comparison functions will perform sorting and comparisons according to > the sequence > > -infinity < normal values < infinity < NaN < NULL I was thinking about making NaN equivalent to NULL. That would give consistency in ordering, and probably also in arithmetic. Additionally, if the platform supports it we ought to make the Invalid Operation FP exception (which yields NaN) configurable: either get NULL or get an error. -- Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
Peter Eisentraut <peter_e@gmx.net> writes: > I was thinking about making NaN equivalent to NULL. Mumble ... in the thread last August, someone made the point that SQL's idea of NULL ("unknown value") is not really the same as a NaN ("I know that this is not a well-defined number"). Even though there's a lot of similarity in the behaviors, I'd be inclined to preserve that semantic distinction. If we did want to do this, the implication would be that all float-returning functions would be required to make sure they were not returning NaNs:if (isnan(x)) PG_RETURN_NULL();else PG_RETURN_FLOAT8(x); Possibly this logic could be folded right into the PG_RETURN_FLOAT macros. > if the platform supports it we ought to make the Invalid Operation FP > exception (which yields NaN) configurable: either get NULL or get an > error. Seems like we could equally well offer the switch as "either get NaN or get an error". Something to be kept in mind here is the likelihood of divergence in our behavior between IEEE and non-IEEE platforms. I don't object to that --- it's sort of the point --- but we should be aware of how much difference we're creating, and try to avoid unnecessary differences. Hmm ... I suppose an attraction of a NULL-vs-error, as opposed to NaN- vs-error, option is that it could theoretically be supported on NaN-less hardware. But is that realizable in practice? SIGFPE is messy. regards, tom lane
> > -infinity < normal values < infinity < NaN < NULL > I was thinking about making NaN equivalent to NULL. That would give > consistency in ordering, and probably also in arithmetic. Additionally, > if the platform supports it we ought to make the Invalid Operation FP > exception (which yields NaN) configurable: either get NULL or get an > error. I'd like to see the distinction between NaN and NULL retained, since the two "values" arise from different circumstances and under different conditions. If a particular app needs them to be equivalent, then that is easy enough to do with SQL or triggers. - Thomas On a modestly related note, I'm come over to the notion that the date/time value 'current' could be ripped out eventually. Tom, isn't that the only case for those types which bolluxes up caching of date/time types?
Thomas Lockhart <lockhart@fourpalms.org> writes: > On a modestly related note, I'm come over to the notion that the > date/time value 'current' could be ripped out eventually. Tom, isn't > that the only case for those types which bolluxes up caching of > date/time types? Yes, I believe so. At least, that was the consideration that led me to mark those functions noncachable ... regards, tom lane