Thread: Re: [GENERAL] +/- Inf for float8's

Re: [GENERAL] +/- Inf for float8's

From
"Ross J. Reedstrom"
Date:
Gah, typo'ed the name of pgsql-hackers. This should be better. Sorry
to those who got this twice, once on GENERAL, once on HACKERS.

Ross

On Mon, Aug 14, 2000 at 02:33:55PM +1000, Tim Allen wrote:
> I'm just trying out PG7.0.2, with a view to upgrading from 6.5.3, and I've
> found one quirk a little troublesome. Not sure whether I'll get any
> sympathy, but I shall ask anyway :).
>
> We find it convenient to be able to store +/- infinity for float8 values
> in some database tables. With Postgres 6.5.3, we were able to get away
> with this by using the values -1.79769313486232e+308 for -Inf and
> 1.79769313486232e+308 for Inf. This is probably not very portable, but
> anyway, it worked fine for us, on both x86 Linux and SGI IRIX. One thing,
> though, to get these numbers past the interface we had to put them in
> quotes. It seemed as though there was one level of parsing that didn't
> like these particular numbers, and one level of parsing that coped OK, and
> using quotes got it past the first level.
>
> Now, however (unfortunately for us), this inconsistency in the interface
> has been "fixed", and now we can't get this past the interface, either
> quoted or not. Fixing inconsistencies is, of course, in general, a good
> thing, which is why I'm not confident of getting much sympathy :).
>

Breaking working apps is never a good thing, but that's part of why it went
from 6.X to 7.X.

> So, any suggestions as to how we can store +/- infinity as a valid float8
> value in a database table?
>

Right: the SQL standard doesn't say anything about what to do for these
cases for floats (except by defining the syntax of an approximate numeric
constant as basically a float), but the IEEE754 does: as you discovered
below, they're NaN, -Infinity, and +Infinity.

> I notice, btw, that 'NaN' is accepted as a valid float8. Is there any
> particular reason why something similar for, eg '-Inf' and 'Inf' doesn't
> also exist? Just discovered, there is a special number 'Infinity', which
> seems to be recognised, except you can't insert it into a table because it
> reports an overflow error. Getting warm, it seems, but not there yet. And
> there doesn't seem to be a negative equivalent.

And this is a bug. From looking at the source, I see that Thomas added
code to accept 'NaN' and 'Infinity' (but not '-Infinity'), and Tom Lane
tweaked it, but it's never been able to get an Infinity all the way to
the table, as far as I can see: the value gets set to HUGE_VAL, but the
call to CheckFloat8Val compares against FLOAT8_MAX (and FLOAT8_MIN),
and complains, since HUGE_VAL is _defined_ to be larger than DBL_MAX.

And, there's no test case in the regression tests for inserting NaN or
Infinity. (Shame on Thomas ;-)

I think the right thing to do is move the call to CheckFloat8Val into a
branch of the test for NaN and Infinity, thereby not calling it if we've
been passed those constants. I'm compiling up a test of this right now,
and I'll submit a patch to Bruce if it passes regression. Looks like
that function hasn't been touch in a while, so the patch should apply
to 7.0.X as well as current CVS.

<some time later>

Looks like it works, and passes the regression tests as they are.  I'm
patching the tests to include the cases 'NaN', 'Infinity', and '-Infinity'
as valid float8s, and 'not a float' as an invalid representation, and
rerunning to get output to submit with the patch. This might be a bit
hairy, since there are 5 different expected/float8* files. Should I try
to hand patch them to deal with the new rows, or let them be regenerated
by people with the appropriate platforms?

<later again>

Bigger problem with changing the float8 regression tests: a lot of our
math functions seem to be guarded with CheckFloat8Val(result), so, if we
allow these values in a float8 column, most of the math functions with
elog(). It strikes me that there must have been a reason for this at one
time. There's even a #define UNSAFE_FLOATS, to disable these checks. By
reading the comments in old copies of float.c, it looks like this was
added for an old, buggy linux/Alpha libc that would throw floating point
exceptions, otherwise.

Is there an intrinsic problem with allowing values outside the range
FLOAT8_MAX <= x =>FLOAT8_MIN ? 'ORDER BY' seems to still work, with
'Infinity' and '-Infinity' sorting properly. Having a 'NaN' in there
breaks sorting however.  That's a current, live bug.  Could be fixed
by treating 'NaN' as a different flavor of NULL. Probably a fairly deep
change, however. Hmm, NULL in a float8 sorts to the end, regardless of
ASC or DESC, is that right?

Anyway, here's the patch for just float.c , if anyone wants to look
at it. As I said, it passes the existing float8 regression tests, but
raises a lot of interesting questions.

Ross
--
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu>
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St.,  Houston, TX 77005

Attachment

Re: Re: [GENERAL] +/- Inf for float8's

From
Thomas Lockhart
Date:
> > So, any suggestions as to how we can store +/- infinity as a valid float8
> > value in a database table?
> Right: the SQL standard doesn't say anything about what to do for these
> cases for floats (except by defining the syntax of an approximate numeric
> constant as basically a float), but the IEEE754 does: as you discovered
> below, they're NaN, -Infinity, and +Infinity.

Not all computers fully support IEEE754, though many new ones do.

> > I notice, btw, that 'NaN' is accepted as a valid float8. Is there any
> > particular reason why something similar for, eg '-Inf' and 'Inf' doesn't
> > also exist? Just discovered, there is a special number 'Infinity', which
> > seems to be recognised, except you can't insert it into a table because it
> > reports an overflow error. Getting warm, it seems, but not there yet. And
> > there doesn't seem to be a negative equivalent.
> And this is a bug. From looking at the source, I see that Thomas added
> code to accept 'NaN' and 'Infinity' (but not '-Infinity'), and Tom Lane
> tweaked it, but it's never been able to get an Infinity all the way to
> the table, as far as I can see: the value gets set to HUGE_VAL, but the
> call to CheckFloat8Val compares against FLOAT8_MAX (and FLOAT8_MIN),
> and complains, since HUGE_VAL is _defined_ to be larger than DBL_MAX.
> And, there's no test case in the regression tests for inserting NaN or
> Infinity. (Shame on Thomas ;-)

Ah, I'm just trying to leave some rewarding work for other folks ;)

> I think the right thing to do is move the call to CheckFloat8Val into a
> branch of the test for NaN and Infinity, thereby not calling it if we've
> been passed those constants. I'm compiling up a test of this right now,
> and I'll submit a patch to Bruce if it passes regression. Looks like
> that function hasn't been touch in a while, so the patch should apply
> to 7.0.X as well as current CVS.

istm that the existing protection (or something like it) is required for
some platforms, while other platforms may be able to handle NaN and
+/-Inf just fine. Seems like a job for autoconf to determine the FP
capabilities of a system, unless Posix defines some way to tell. Of
course, even then we'd need an autoconf test to deal with non-Posix
platforms.

> Looks like it works, and passes the regression tests as they are.  I'm
> patching the tests to include the cases 'NaN', 'Infinity', and '-Infinity'
> as valid float8s, and 'not a float' as an invalid representation, and
> rerunning to get output to submit with the patch. This might be a bit
> hairy, since there are 5 different expected/float8* files. Should I try
> to hand patch them to deal with the new rows, or let them be regenerated
> by people with the appropriate platforms?

How about setting up a separate test (say, ieee754.sql) so that
non-compliant platforms can still pass the original FP test suite. Then
other platforms can be added in as they are tested.

Some platforms may need their compiler switches tweaked; I haven't
checked the Alpha/DUnix configuration but I recall needing to fix some
flags to get compiled code to move these edge cases around even just
through subroutine calls. One example was in trying to call finite(),
which threw an error during the call to it if the number was NaN or
Infinity. Which sort of defeated the purpose of the call :)

> Bigger problem with changing the float8 regression tests: a lot of our
> math functions seem to be guarded with CheckFloat8Val(result), so, if we
> allow these values in a float8 column, most of the math functions with
> elog(). It strikes me that there must have been a reason for this at one
> time. There's even a #define UNSAFE_FLOATS, to disable these checks. By
> reading the comments in old copies of float.c, it looks like this was
> added for an old, buggy linux/Alpha libc that would throw floating point
> exceptions, otherwise.

There are still reasons on some platforms, as noted above...

> Is there an intrinsic problem with allowing values outside the range
> FLOAT8_MAX <= x =>FLOAT8_MIN ? 'ORDER BY' seems to still work, with
> 'Infinity' and '-Infinity' sorting properly. Having a 'NaN' in there
> breaks sorting however.  That's a current, live bug.  Could be fixed
> by treating 'NaN' as a different flavor of NULL. Probably a fairly deep
> change, however. Hmm, NULL in a float8 sorts to the end, regardless of
> ASC or DESC, is that right?

NULL and NaN are not quite the same thing imho. If we are allowing NaN
in columns, then it is *known* to be NaN.

> Anyway, here's the patch for just float.c , if anyone wants to look
> at it. As I said, it passes the existing float8 regression tests, but
> raises a lot of interesting questions.

Are you interested in pursuing this further? It seems like we might be
able to move in the direction you suggest on *some* platforms, but we
will need to scrub the math functions to be able to handle these edge
cases.
                   - Thomas


Re: Re: [GENERAL] +/- Inf for float8's

From
"Ross J. Reedstrom"
Date:
On Tue, Aug 15, 2000 at 03:27:55AM +0000, Thomas Lockhart wrote:
> 
> Not all computers fully support IEEE754, though many new ones do.

True - the question becomes: how new is new?  Are we supporting ones
that aren't?  If so, that's fine. If not, it's a lot easier to fix. ;-)

> > And, there's no test case in the regression tests for inserting NaN or
> > Infinity. (Shame on Thomas ;-)
> 
> Ah, I'm just trying to leave some rewarding work for other folks ;)

And we appreciate the crumbs. Actually, it _was_ good practice grovelling
out versions from CVS and matching log messages.

> 
> istm that the existing protection (or something like it) is required for
> some platforms, while other platforms may be able to handle NaN and
> +/-Inf just fine. Seems like a job for autoconf to determine the FP
> capabilities of a system, unless Posix defines some way to tell. Of
> course, even then we'd need an autoconf test to deal with non-Posix
> platforms.

Yeah, need to get Peter Eisentraut involved, perhaps. Should actually be
pretty simple: the #define is already there: UNSAFE_FLOATS. Define that,
and the CheckFloat[48]Val functions just return true. 

> 
> How about setting up a separate test (say, ieee754.sql) so that
> non-compliant platforms can still pass the original FP test suite. Then
> other platforms can be added in as they are tested.

Hmm, I wish we had clue what other systems might be non-compliant, and how.
The question becomes one of if it's _possible_ to support NaN, +/-Inf on
some platforms. Then, we end up with a difference in functionality.

> 
> > Is there an intrinsic problem with allowing values outside the range
> > FLOAT8_MAX <= x =>FLOAT8_MIN ? 'ORDER BY' seems to still work, with
> > 'Infinity' and '-Infinity' sorting properly. Having a 'NaN' in there
> > breaks sorting however.  That's a current, live bug.  Could be fixed
> > by treating 'NaN' as a different flavor of NULL. Probably a fairly deep
> > change, however. Hmm, NULL in a float8 sorts to the end, regardless of
> > ASC or DESC, is that right?
> 
> NULL and NaN are not quite the same thing imho. If we are allowing NaN
> in columns, then it is *known* to be NaN.

For the purposes of ordering, however, they are very similar. Neither one
can be placed corectly with respect to the other values: NULL, because
we don't know were it really is, NaN because we know it's not even on
this axis. I'm suggesting the we fix sorting on floats to treat NaN as
NULL, and sort it to the end. As it stands, sorting is broken, since it
returns the NaN rows wherever they happen to be. This causes them to
act as barriers, partitioning the returned set into seperately sorted
sub sequences.

> 
> > Anyway, here's the patch for just float.c , if anyone wants to look
> > at it. As I said, it passes the existing float8 regression tests, but
> > raises a lot of interesting questions.
> 
> Are you interested in pursuing this further? It seems like we might be
> able to move in the direction you suggest on *some* platforms, but we
> will need to scrub the math functions to be able to handle these edge
> cases.

Sure. I'm no great floating point wiz, but I'm sure Tom and Don will
jump on anything I get wrong. Duping the float tests and feeding them
NaN/+/-Inf as a seperate test set is probably a good idea.

The existing patch moves the call to CheckFloat8Val() inside float8in
so it is only called if strtod() consumes all it's input, and does not
set errno. Seems safe to me: if strtod() doesn't consume it's input,
we check to make sure it's not NaN/+/-Inf (gah, need a shorthand word
for those three values), else elog(). If it does, but sets errno,
we catch that. Then, in belt-and-suspenders style, we call
CheckFloat8Val(). For that check to fail, strtod() would have to consume
its entire input, return +/-HUGE_VAL, and _not_ set errno to ERANGE.

BTW, this also brings up something that got discussed before the last
release, but never implemented. The original problem from Tim Allen had
to do with using a work around for not having +/- Inf: storing the values
-1.79769313486232e+308 and 1.79769313486232e+308. He was having trouble,
since a pg_dump/restore cycle broke, do to rounding the values to out
of range for floats. This wasn't caught by the current regression tests, 
but would have been caught by the suggested dump/restore/dump/compare
dumps cycle someone suggested for exercizing the *out and *in functions.


Ross
-- 
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> 
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St.,  Houston, TX 77005


Re: Re: [GENERAL] +/- Inf for float8's

From
"Ross J. Reedstrom"
Date:
Thomas - 
A general design question. There seems to be a good reason to allow +/-Inf
in float8 columns: Tim Allen has an need for them, for example. That's
pretty straight forward, they seem to act properly if the underlying float
libs handle them.

I'm not convinced NaN gives us anything useful, especially given how
badly it breaks sorting. I've been digging into that code a little,
and it's not going to be pretty. It strikes me as wrong to embed type
specific info into the generic sorting routines.

So, anyone have any ideas what NaN would be useful for? Especially given
we have NULL available, which most (non DB) numeric applications don't.

Ross
-- 
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> 
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St.,  Houston, TX 77005



Re: Re: [GENERAL] +/- Inf for float8's

From
Thomas Lockhart
Date:
> So, anyone have any ideas what NaN would be useful for? Especially given
> we have NULL available, which most (non DB) numeric applications don't.

Hmm. With Tom Lane's new fmgr interface, you *can* return NULL if you
spot a NaN result. Maybe that is the best way to go about it; we'll
stipulate that NaN and NULL are equivalent. And we'll further stipulate
that if you are messing with NaN then you deserve what you get ;)
                      - Thomas


Re: Re: [GENERAL] +/- Inf for float8's

From
"Timothy H. Keitt"
Date:
I can't say whether its worth the trouble to add NaN, but I will say that NaN
is not the same as NULL.  NULL is missing data; NaN is 0./0.  The difference
is significant for numerical applications.

Tim

"Ross J. Reedstrom" wrote:

> Thomas -
> A general design question. There seems to be a good reason to allow +/-Inf
> in float8 columns: Tim Allen has an need for them, for example. That's
> pretty straight forward, they seem to act properly if the underlying float
> libs handle them.
>
> I'm not convinced NaN gives us anything useful, especially given how
> badly it breaks sorting. I've been digging into that code a little,
> and it's not going to be pretty. It strikes me as wrong to embed type
> specific info into the generic sorting routines.
>
> So, anyone have any ideas what NaN would be useful for? Especially given
> we have NULL available, which most (non DB) numeric applications don't.
>
> Ross
> --
> Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu>
> NSBRI Research Scientist/Programmer
> Computer and Information Technology Institute
> Rice University, 6100 S. Main St.,  Houston, TX 77005

--
Timothy H. Keitt
National Center for Ecological Analysis and Synthesis
735 State Street, Suite 300, Santa Barbara, CA 93101
Phone: 805-892-2519, FAX: 805-892-2510
http://www.nceas.ucsb.edu/~keitt/





Re: Re: [GENERAL] +/- Inf for float8's

From
Thomas Swan
Date:
>I'm not convinced NaN gives us anything useful, especially given how
>badly it breaks sorting. I've been digging into that code a little,
>and it's not going to be pretty. It strikes me as wrong to embed type
>specific info into the generic sorting routines.
>
>So, anyone have any ideas what NaN would be useful for? Especially given
>we have NULL available, which most (non DB) numeric applications don't.
>
>Ross

Just a wild guess, NaN could be used to indicated invalid numeric 
data.  However, this seems odd because it should have been checked prior to 
being put in the DB.

NULL is no value, +/- infinity could be just that or out of range, unless 
you want NaN to be out of range.   Depending on your scheme for 
representation you could take an out of range value and store it as +/i 
infinity.

These are just suggestions.

Thomas



NaN

From
Oliver Seidel
Date:
Hi,

just to add my opinion on NaN in the IEEE standard.  As far as I
remember, IEEE numbers work as follows:

1 bit sign
some bits base
some bits exponent

This allows you to do several things:

interpret the exp bits as a normal integer and get
- exp=below half: negative exponents
- exp=half: exponent=0
- exp=above half: positive exponents
- exp=all set: NaN, quite a few at that

For all of these the sign can be either positive or negative, leading
to pos/neg zero (quite a strange concept).

With the NaNs, you get quite a few possibilities, but notably:
- base=0 (NaN -- this is not a number, but an animal)
- base=max (pos/neg infinity, depending on sign)

Someone mentioned a representation for 0/0 and I might add that there
are four possibilities:(( 1.)*0.) / (( 1.)*0.)(( 1.)*0.) / ((-1.)*0.)((-1.)*0.) / (( 1.)*0.)((-1.)*0.) / ((-1.)*0.)
These (given commutativity, except that we're dealing with a finite
representation, but predictable in that it is actually possible to
factor out the sign) can be reduced to:( 1) * (0./0.)(-1) * (0./0.)
which amounts to pos/neg infinity of some sort.

Now my take on NULL vs NaN is that there should be a whole bunch of
NULL, just like there is a whole bunch of NaN.  Just off the top of my
head, I could imagine "unknown", "unknowable", "out of range in
direction X".  But, alas, the SQL standard doesn't provide for such
things (though the storage implementation would: but what would you do
with comparisons, conversions and displays?).

so long,

Oliver


Re: Re: [GENERAL] +/- Inf for float8's

From
Peter Eisentraut
Date:
(Side note: Folks, we need a real bug/issue-tracking system. We just
discussed this a month ago ("How PostgreSQL's floating point hurts
everyone everywhere"). If anyone's interested in porting Bugzilla or some
other such system to PostgreSQL and putting it into use, let me know.)

Ross J. Reedstrom writes:

> Yeah, need to get Peter Eisentraut involved, perhaps. Should actually be
> pretty simple: the #define is already there: UNSAFE_FLOATS. Define that,
> and the CheckFloat[48]Val functions just return true. 

Show me a system where it doesn't work and we'll get it to work.
UNSAFE_FLOATS as it stands it probably not the most appropriate behaviour;
it intends to speed things up, not make things portable.


> > NULL and NaN are not quite the same thing imho. If we are allowing NaN
> > in columns, then it is *known* to be NaN.
> 
> For the purposes of ordering, however, they are very similar.

Then we can also treat them similar, i.e. sort them all last or all first.
If you have NaN's in your data you wouldn't be interested in ordering
anyway.


> we check to make sure it's not NaN/+/-Inf (gah, need a shorthand word
> for those three values), else elog().

"non-finite values"


Side note 2: The paper "How Java's floating point hurts everyone
everywhere" provides for good context reading.

Side note 3: Once you read that paper you will agree that using floating
point with Postgres is completely insane as long as the FE/BE protocol is
text-based.


-- 
Peter Eisentraut                  Sernanders väg 10:115
peter_e@gmx.net                   75262 Uppsala
http://yi.org/peter-e/            Sweden



Re: Re: [GENERAL] +/- Inf for float8's

From
Peter Eisentraut
Date:
Thomas Lockhart writes:

> > So, anyone have any ideas what NaN would be useful for? Especially given
> > we have NULL available, which most (non DB) numeric applications don't.
> 
> Hmm. With Tom Lane's new fmgr interface, you *can* return NULL if you
> spot a NaN result. Maybe that is the best way to go about it; we'll
> stipulate that NaN and NULL are equivalent. And we'll further stipulate
> that if you are messing with NaN then you deserve what you get ;)

I beg to differ, this behaviour would not be correct. Instead, this should
happen:

NULL < NULL    => NULL
NULL < 1.0    => NULL
NULL < Nan    => NULL
1.0 < NULL    => NULL
1.0 < NaN    => false
NaN < NULL    => NULL
NaN < 1.0    => false

Then all the NaN's sort either all first or all last before or after the
NULLs.


-- 
Peter Eisentraut                  Sernanders väg 10:115
peter_e@gmx.net                   75262 Uppsala
http://yi.org/peter-e/            Sweden



Re: Re: [GENERAL] +/- Inf for float8's

From
Don Baccus
Date:
At 12:33 AM 8/20/00 +0200, Peter Eisentraut wrote:
>(Side note: Folks, we need a real bug/issue-tracking system. We just
>discussed this a month ago ("How PostgreSQL's floating point hurts
>everyone everywhere"). If anyone's interested in porting Bugzilla or some
>other such system to PostgreSQL and putting it into use, let me know.)

OpenACS and arsDigita are using Ben Adida's software development manager,
which includes a ticket-tracking module.  It's still under development,
but you can take a look at www.openacs.org/sdm to see how we're using
it.

It was developed for Postgres (which is what you see at the above URL)
then ported to Oracle (which is what you arsDigita does).  aD has also
added some functionality which is supposed to be ported back to the
Postgres version.

Among other things it integrates with a todo list manager that maintains
individual todo lists for developers ...  you're assigned a bug, it
ends up on your todo list.




- Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert
Serviceand other goodies at http://donb.photo.net.
 


Re: Re: [GENERAL] +/- Inf for float8's

From
Don Baccus
Date:
At 09:01 PM 8/19/00 -0300, The Hermit Hacker wrote:
>
>Ben has an account on Hub, and aolserver has been installed, so if you
>guys want to install and get this working, just tell me what else you need
>as far as software and/or configurations are concerned and "it shall be
>done" ...

I've e-mailed Ben a "heads-up", though he monitors this list and will probably
see your note.

I'll be gone about five of the next 6-7 weeks mostly doing my annual stint as
a field biologist where I'm only accessible once a day via radio by BLM
Dispatch
in Elko, Nevada so I'm afraid this (like many other things at the moment) will
fall on Ben's shoulders...



- Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert
Serviceand other goodies at http://donb.photo.net.
 


Re: Re: [GENERAL] +/- Inf for float8's

From
The Hermit Hacker
Date:
Ben has an account on Hub, and aolserver has been installed, so if you
guys want to install and get this working, just tell me what else you need
as far as software and/or configurations are concerned and "it shall be
done" ...

On Sat, 19 Aug 2000, Don Baccus wrote:

> At 12:33 AM 8/20/00 +0200, Peter Eisentraut wrote:
> >(Side note: Folks, we need a real bug/issue-tracking system. We just
> >discussed this a month ago ("How PostgreSQL's floating point hurts
> >everyone everywhere"). If anyone's interested in porting Bugzilla or some
> >other such system to PostgreSQL and putting it into use, let me know.)
> 
> OpenACS and arsDigita are using Ben Adida's software development manager,
> which includes a ticket-tracking module.  It's still under development,
> but you can take a look at www.openacs.org/sdm to see how we're using
> it.
> 
> It was developed for Postgres (which is what you see at the above URL)
> then ported to Oracle (which is what you arsDigita does).  aD has also
> added some functionality which is supposed to be ported back to the
> Postgres version.
> 
> Among other things it integrates with a todo list manager that maintains
> individual todo lists for developers ...  you're assigned a bug, it
> ends up on your todo list.
> 
> 
> 
> 
> - Don Baccus, Portland OR <dhogaza@pacifier.com>
>   Nature photos, on-line guides, Pacific Northwest
>   Rare Bird Alert Service and other goodies at
>   http://donb.photo.net.
> 

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 



Re: Re: [GENERAL] +/- Inf for float8's

From
Vince Vielhaber
Date:
http://www.postgresql.org/bugs/

I was about to implement it once before and the directory disappeared.
But anyway it's there.

Vince.


On Sat, 19 Aug 2000, The Hermit Hacker wrote:

> 
> Ben has an account on Hub, and aolserver has been installed, so if you
> guys want to install and get this working, just tell me what else you need
> as far as software and/or configurations are concerned and "it shall be
> done" ...
> 
> On Sat, 19 Aug 2000, Don Baccus wrote:
> 
> > At 12:33 AM 8/20/00 +0200, Peter Eisentraut wrote:
> > >(Side note: Folks, we need a real bug/issue-tracking system. We just
> > >discussed this a month ago ("How PostgreSQL's floating point hurts
> > >everyone everywhere"). If anyone's interested in porting Bugzilla or some
> > >other such system to PostgreSQL and putting it into use, let me know.)
> > 
> > OpenACS and arsDigita are using Ben Adida's software development manager,
> > which includes a ticket-tracking module.  It's still under development,
> > but you can take a look at www.openacs.org/sdm to see how we're using
> > it.
> > 
> > It was developed for Postgres (which is what you see at the above URL)
> > then ported to Oracle (which is what you arsDigita does).  aD has also
> > added some functionality which is supposed to be ported back to the
> > Postgres version.
> > 
> > Among other things it integrates with a todo list manager that maintains
> > individual todo lists for developers ...  you're assigned a bug, it
> > ends up on your todo list.
> > 
> > 
> > 
> > 
> > - Don Baccus, Portland OR <dhogaza@pacifier.com>
> >   Nature photos, on-line guides, Pacific Northwest
> >   Rare Bird Alert Service and other goodies at
> >   http://donb.photo.net.
> > 
> 
> Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
> Systems Administrator @ hub.org 
> primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 
> 
> 

-- 
==========================================================================
Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from
$16.00/moat Pop4 Networking       Online Campground Directory    http://www.camping-usa.com      Online Giftshop
Superstore   http://www.cloudninegifts.com
 
==========================================================================





Re: Re: [GENERAL] +/- Inf for float8's

From
Don Baccus
Date:
At 08:34 PM 8/19/00 -0400, Vince Vielhaber wrote:
>
>http://www.postgresql.org/bugs/
>
>I was about to implement it once before and the directory disappeared.
>But anyway it's there.

Cool, I tried it and broke it on my second click ... any particular reason
to roll your own rather than use something that's already being used by
several other development projects and is under active development for
that reason?  (i.e. the SDM)




- Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert
Serviceand other goodies at http://donb.photo.net.
 


Re: Re: [GENERAL] +/- Inf for float8's

From
Thomas Lockhart
Date:
Don Baccus wrote:
> 
> >(Side note: Folks, we need a real bug/issue-tracking system. We just
> >discussed this a month ago ("How PostgreSQL's floating point hurts
> >everyone everywhere"). If anyone's interested in porting Bugzilla or some
> >other such system to PostgreSQL and putting it into use, let me know.)

istm that it is *not* that easy. We tried (very briefly) a bug tracking
system. Whatever technical problems it had (which other tools may not),
the fundamental problem is that the mailing lists do a *great* job of
screening problem reports while also supporting and enhancing the
"Postgres culture", whereas a "bug report tool" eliminates that traffic
and requires one or a few people to pay attention to the bug list to
manage new and existing bug reports.

This has (or could have) a *huge* impact on the culture and tradition of
Postgres development, which imho is one of the most satisfying,
pleasant, and effective environments in open source development. So if
we try to do something with a bug tracking system, we will need to
figure out:

o how to retain a free and helpful discussion on the mailing lists, and
to not degrade into a "shut up and check the bug reports" response.

o how to filter or qualify bug reports so that developers don't spend
time having to do that.

All imho of course ;)
                  - Thomas


Re: Re: [GENERAL] +/- Inf for float8's

From
Don Baccus
Date:
At 01:22 AM 8/20/00 +0000, Thomas Lockhart wrote:

>istm that it is *not* that easy. We tried (very briefly) a bug tracking
>system. Whatever technical problems it had (which other tools may not),
>the fundamental problem is that the mailing lists do a *great* job of
>screening problem reports while also supporting and enhancing the
>"Postgres culture", whereas a "bug report tool" eliminates that traffic
>and requires one or a few people to pay attention to the bug list to
>manage new and existing bug reports.

In the SDM you can, of course, ask to be notified of various events
by e-mail.  And there's a commenting facility so in essence a bug
report or feature request starts a conversation thread.  

I don't recall saying that the SDM is simply a "bug report tool".  There's
quite a bit more to it, and the goal is to INCREASE interactivity, not
decrease it.

>This has (or could have) a *huge* impact on the culture and tradition of
>Postgres development, which imho is one of the most satisfying,
>pleasant, and effective environments in open source development. So if
>we try to do something with a bug tracking system, we will need to
>figure out:
>
>o how to retain a free and helpful discussion on the mailing lists, and
>to not degrade into a "shut up and check the bug reports" response.

This is a social, not software, engineering issue.

>o how to filter or qualify bug reports so that developers don't spend
>time having to do that.

Developers don't have to filter or qualify bug reports e-mailed to the
bugs list today?  Who's doing it, then, and why can't they continue doing
so if another tool is used to manage bug reports? 

The SDM allows a little more granularity than the single e-mail list
aproach allows for.  You can designate modules within a package.  For
instance, psql might be a module with Peter assigned as an administrator,
and he might elect to get e-mail alerts whenever a bug is submitted to
for psql.

But he might not, for instance, be particularly interested in getting
e-mail alerts on (say) the JDBC driver.

There's a certain amount of delegation inherent in an approach like
this, and developers focused on narrow portions of the product (and
Peter came to mind because of psql, I'm not suggesting he only has
a narrow interest in the product) can arrange to only get nagged, if
you will, for stuff they've taken responsibility for.

My guess is that such a system probably isn't as cozy and useful for
developers, as you're implying.

I think it might well be more friendly for users, though.  Certainly
the OpenACS and arsDigita communities - both fairly large though
not as long in the tooth as PG, I might add - seem to appreciate having
access to such a system.



- Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert
Serviceand other goodies at http://donb.photo.net.
 


Re: Re: [GENERAL] +/- Inf for float8's

From
Vince Vielhaber
Date:
On Sat, 19 Aug 2000, Don Baccus wrote:

> At 08:34 PM 8/19/00 -0400, Vince Vielhaber wrote:
> >
> >http://www.postgresql.org/bugs/
> >
> >I was about to implement it once before and the directory disappeared.
> >But anyway it's there.
> 
> Cool, I tried it and broke it on my second click ... any particular reason
> to roll your own rather than use something that's already being used by
> several other development projects and is under active development for
> that reason?  (i.e. the SDM)

Like I said, the dir disappeared before I could commit it, probably some
config stuff too.  We tried a couple of already in use items and frankly
I got tired of learning a new package that noone used anyway.  I figured
at least this one could be more of what we needed.  It logs the problem
in the database and emails the bugs list (I may have the wrong list
addr in there too).  The status can be changed, entries can be made as
to the status.  

What did you do to break it and what broke?

Vince.
-- 
==========================================================================
Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from
$16.00/moat Pop4 Networking       Online Campground Directory    http://www.camping-usa.com      Online Giftshop
Superstore   http://www.cloudninegifts.com
 
==========================================================================





Re: Re: [GENERAL] +/- Inf for float8's

From
The Hermit Hacker
Date:
If Ben and gang are willing to work on the bug tracking system to get it
to fit what we want/need, it would give a good example of OpenACS and gang
for them to use ... we could just shot "wants" at them and they could
improve as we go along?

On Sat, 19 Aug 2000, Vince Vielhaber wrote:

> On Sat, 19 Aug 2000, Don Baccus wrote:
> 
> > At 08:34 PM 8/19/00 -0400, Vince Vielhaber wrote:
> > >
> > >http://www.postgresql.org/bugs/
> > >
> > >I was about to implement it once before and the directory disappeared.
> > >But anyway it's there.
> > 
> > Cool, I tried it and broke it on my second click ... any particular reason
> > to roll your own rather than use something that's already being used by
> > several other development projects and is under active development for
> > that reason?  (i.e. the SDM)
> 
> Like I said, the dir disappeared before I could commit it, probably some
> config stuff too.  We tried a couple of already in use items and frankly
> I got tired of learning a new package that noone used anyway.  I figured
> at least this one could be more of what we needed.  It logs the problem
> in the database and emails the bugs list (I may have the wrong list
> addr in there too).  The status can be changed, entries can be made as
> to the status.  
> 
> What did you do to break it and what broke?
> 
> Vince.
> -- 
> ==========================================================================
> Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net
>  128K ISDN from $22.00/mo - 56K Dialup from $16.00/mo at Pop4 Networking
>         Online Campground Directory    http://www.camping-usa.com
>        Online Giftshop Superstore    http://www.cloudninegifts.com
> ==========================================================================
> 
> 
> 
> 

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 



Re: Re: [GENERAL] +/- Inf for float8's

From
The Hermit Hacker
Date:
On Sat, 19 Aug 2000, Don Baccus wrote:

> >o how to filter or qualify bug reports so that developers don't spend
> >time having to do that.
> 
> Developers don't have to filter or qualify bug reports e-mailed to the
> bugs list today?  Who's doing it, then, and why can't they continue
> doing so if another tool is used to manage bug reports?

the problem as I see it with any bug tracking tool is someone has to close
off those bugs when fixed ... right now, someone commits a bug fix, and
then fires off an email to the list stating its fixed ... with a bug
tracking system, then they have to go one more step, open up a web
browser, login to the system, find the bug report and close it ...




Re: Re: [GENERAL] +/- Inf for float8's

From
Vince Vielhaber
Date:
On Sun, 20 Aug 2000, The Hermit Hacker wrote:

> 
> If Ben and gang are willing to work on the bug tracking system to get it
> to fit what we want/need, it would give a good example of OpenACS and gang
> for them to use ... we could just shot "wants" at them and they could
> improve as we go along?

Fine by me, BUT I have no desire to learn how it works.  If that's gonna
be the end result then rm -Rf is my preference.  This'll be the third or
forth one so far, the others were pushed off the edge of the earth.  Sorry
to be so harsh but no matter what the bug tool is I can't see it lasting
very long.  This group has shown repeatedly that it's not as desired as
it appears to be.

Vince.

> 
> On Sat, 19 Aug 2000, Vince Vielhaber wrote:
> 
> > On Sat, 19 Aug 2000, Don Baccus wrote:
> > 
> > > At 08:34 PM 8/19/00 -0400, Vince Vielhaber wrote:
> > > >
> > > >http://www.postgresql.org/bugs/
> > > >
> > > >I was about to implement it once before and the directory disappeared.
> > > >But anyway it's there.
> > > 
> > > Cool, I tried it and broke it on my second click ... any particular reason
> > > to roll your own rather than use something that's already being used by
> > > several other development projects and is under active development for
> > > that reason?  (i.e. the SDM)
> > 
> > Like I said, the dir disappeared before I could commit it, probably some
> > config stuff too.  We tried a couple of already in use items and frankly
> > I got tired of learning a new package that noone used anyway.  I figured
> > at least this one could be more of what we needed.  It logs the problem
> > in the database and emails the bugs list (I may have the wrong list
> > addr in there too).  The status can be changed, entries can be made as
> > to the status.  
> > 
> > What did you do to break it and what broke?
> > 
> > Vince.
> > -- 
> > ==========================================================================
> > Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net
> >  128K ISDN from $22.00/mo - 56K Dialup from $16.00/mo at Pop4 Networking
> >         Online Campground Directory    http://www.camping-usa.com
> >        Online Giftshop Superstore    http://www.cloudninegifts.com
> > ==========================================================================
> > 
> > 
> > 
> > 
> 
> Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
> Systems Administrator @ hub.org 
> primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 
> 
> 

-- 
==========================================================================
Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from
$16.00/moat Pop4 Networking       Online Campground Directory    http://www.camping-usa.com      Online Giftshop
Superstore   http://www.cloudninegifts.com
 
==========================================================================





Bug tracking (was Re: +/- Inf for float8's)

From
Tom Lane
Date:
FWIW, I agree with Thomas' comments: our last try at a bug-tracking
system was a spectacular failure, and rather than just trying again
with a technically better bug-tracker, we need to understand why we
failed before.

I think we do need to look for a better answer than what we have, but
I do not have any faith in "install system FOO and all your problems
will be solved".

My take is that

(a) a bug *tracking* system is not the same as a bug *reporting*
system.  A tracking system will be useless if it gets cluttered
with non-bug reports, duplicate entries, etc.  There must be a human
filter controlling what gets entered into the system.

(b) our previous try (with Keystone) was a failure in part because
it was not even effective as a bug reporting system: it did not
encourage people to fill in our standard "bug report form", with the
result that bug reports were seriously incomplete w.r.t. version
numbers, platforms, etc.  This is a relatively simple technical
deficiency, not a social-engineering problem, but it does point up
the fact that one-size-fits-all solutions fit nobody.

(c) fill-in-the-web-form reporting systems suck.  They make it
difficult to copy-and-paste query output, dump files, etc.
Also, the window for entering text is always either too small or too
large.  Email with attachments is fundamentally superior.

(d) divorcing the bug reporting system from the existing mailing
list community is foolish, as Thomas pointed out.  When a bug report
is a non-bug (user error, etc) or fixed in a later version or just
a duplicate, we tend to rely on the rest of the community to give
the reporter a helpful response.  Funneling reports into a separate
system that is only read by a few key developers will lose.

I'm not sure what I want, but I know what I don't want...
        regards, tom lane


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Philip Warner
Date:
Perhaps this is foolhardy, but it might be worth making a list of
requirements, at least so we can tick moxes when considering any system...


>(a) a bug *tracking* system is not the same as a bug *reporting*
>system.  A tracking system will be useless if it gets cluttered
>with non-bug reports, duplicate entries, etc.  There must be a human
>filter controlling what gets entered into the system.

1. Human filtering of 'incoming' reports.

2. Separation of 'bug reports' from 'bugs'. 


>(b) our previous try (with Keystone) was a failure in part because
>it was not even effective as a bug reporting system: it did not
>encourage people to fill in our standard "bug report form", with the
>result that bug reports were seriously incomplete w.r.t. version
>numbers, platforms, etc.  This is a relatively simple technical
>deficiency, not a social-engineering problem, but it does point up
>the fact that one-size-fits-all solutions fit nobody.

3. Web and email submissions should do data verification and reject
incomplete reports (giving reasons).


>(c) fill-in-the-web-form reporting systems suck.  They make it
>difficult to copy-and-paste query output, dump files, etc.
>Also, the window for entering text is always either too small or too
>large.  Email with attachments is fundamentally superior.

[I disagree with the above (web *can* work), but...]

4. Must support email AND web submissions, or at least email submissions
and web reporting.


>(d) divorcing the bug reporting system from the existing mailing
>list community is foolish, as Thomas pointed out. 

5. Must integrate with mailing lists.


And to add some of my own (suggested) requirements:

6. Require: name, email address, OS & version, PG version, short description.

7. Optional: compiler & version, long description, file attachments.

8. Creation of 'bug reports' is a public function. Creation of 'bug
entries' is a priv'd function.

9. Simple reporting - unprocessed bug reports, open bugs, bugs by module etc. 


I have tried to keep this relatively simple in an effort to define what we
need to make it work in the current context. But I'm sure I've missed
things.... 


<YABRS>
As it happens, I have a Perl/PGSql based bug-tracking system that I give my
clients access to, which does about 80-90% of this, and I would be willing
to GPL it.

Before anybody asks, the reason I rolled my own was because there weren't
many that supported email and web submission, and the ones that did, did
not support PG easily. This system also implements my prefferred model for
bug reporting which is to separate (to use my terminology) 'Incidents' from
'Issues': an Incident is an event (or set of events) that causes a user to
make a report. An Issue is an individual item that needs attention (usually
a single bug). Typically users send email (or web forms) and I create one
or more Issues. When two Incidents (bug reports) are made about the same
Issue, then the system allows the two to be linked, so that when the Issue
is fixed, all Incident owners are notified etc.

The email integration does very little validation, and it is *not*
integrated with a mailing list (but it is on my ToDo list). I had planned
to do the following:

- When a message is received and the subject does not start with 'Re:' (or
'Aw:'!), submit it as a bug report. If the bug report code returns an
error, then reject it from the list. If the bug report works, then send it
on to Majordomo & Mhonarc.

- If the message starts with 'Re:' then just submit it to the list.

Let me know if anyone would be interested, and I can set up a sample
'product' for people to play with and then, if it is still worth pursuing,
make the code a little more presentable (and decrease it's reliance on my
own perl libraries).

Bear in  mind that this is designed for an Apache/mod-perl environment, so
may not be acceptable to some people. 
</YABRS>



----------------------------------------------------------------
Philip Warner                    |     __---_____
Albatross Consulting Pty. Ltd.   |----/       -  \
(A.B.N. 75 008 659 498)          |          /(@)   ______---_
Tel: (+61) 0500 83 82 81         |                 _________  \
Fax: (+61) 0500 83 82 82         |                 ___________ |
Http://www.rhyme.com.au          |                /           \|                                |    --________--
PGP key available upon request,  |  /
and from pgp5.ai.mit.edu:11371   |/


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Peter Eisentraut
Date:
Tom Lane writes:

> (a) a bug *tracking* system is not the same as a bug *reporting*
> system.  A tracking system will be useless if it gets cluttered
> with non-bug reports, duplicate entries, etc.  There must be a human
> filter controlling what gets entered into the system.

Letting any user submit bug reports directly into any such system is
certainly not going to work, we'd have "query does not use index" 5 times
a day. I consider the current *reporting* procedure pretty good; web forms
are overrated in my mind.

What I had in mind was more a databased incarnation of the TODO list. I
mean, who are we kidding, we are writing a database and maintain the list
of problems in flat text. The TODO list has already moved to the
TODO.detail extension, but we could take it a bit further.

I think currently too many issues get lost, or discussed over and over
again. Many developers maintain their own little lists. The TODO list
often cannot be deciphered by end users and hence does not get read.


-- 
Peter Eisentraut                  Sernanders väg 10:115
peter_e@gmx.net                   75262 Uppsala
http://yi.org/peter-e/            Sweden



Re: Bug tracking (was Re: +/- Inf for float8's)

From
The Hermit Hacker
Date:
On Sun, 20 Aug 2000, Peter Eisentraut wrote:

> Tom Lane writes:
> 
> > (a) a bug *tracking* system is not the same as a bug *reporting*
> > system.  A tracking system will be useless if it gets cluttered
> > with non-bug reports, duplicate entries, etc.  There must be a human
> > filter controlling what gets entered into the system.
> 
> Letting any user submit bug reports directly into any such system is
> certainly not going to work, we'd have "query does not use index" 5 times
> a day. I consider the current *reporting* procedure pretty good; web forms
> are overrated in my mind.
> 
> What I had in mind was more a databased incarnation of the TODO list. I
> mean, who are we kidding, we are writing a database and maintain the list
> of problems in flat text. The TODO list has already moved to the
> TODO.detail extension, but we could take it a bit further.
> 
> I think currently too many issues get lost, or discussed over and over
> again. Many developers maintain their own little lists. The TODO list
> often cannot be deciphered by end users and hence does not get read.

A TODO list that one can add comments to ...



Re: Bug tracking (was Re: +/- Inf for float8's)

From
Wim Ceulemans
Date:
Peter Eisentraut wrote:
> 
> Tom Lane writes:
> 
> > (a) a bug *tracking* system is not the same as a bug *reporting*
> > system.  A tracking system will be useless if it gets cluttered
> > with non-bug reports, duplicate entries, etc.  There must be a human
> > filter controlling what gets entered into the system.
> 
> Letting any user submit bug reports directly into any such system is
> certainly not going to work, we'd have "query does not use index" 5 times
> a day. I consider the current *reporting* procedure pretty good; web forms
> are overrated in my mind.
> 
> What I had in mind was more a databased incarnation of the TODO list. I
> mean, who are we kidding, we are writing a database and maintain the list
> of problems in flat text. The TODO list has already moved to the
> TODO.detail extension, but we could take it a bit further.
> 

I maintain my todo items for my projects in a postgres database. But
there are a lot of issues to consider there too:

- a table of projects (or topics)
- a table of todo items with synopsis, full description, ...
- a table of versions (item is planned to be solved in version, x.x.x,
actually solved in y.y.y)
- a table of developers
- assign table (projects -> developers, items -> developers)
- change type: bug,doc,rfe (request for enhancement),idc (internal
design change), ...
- change state (accepted, evaluated, fixed, rejected, incomplete,
committed, ...
- severity or priority of each item, project
- search functionality

Regards
Wim


Re: Re: [GENERAL] +/- Inf for float8's

From
Ben Adida
Date:
The Hermit Hacker wrote:

> the problem as I see it with any bug tracking tool is someone has to close
> off those bugs when fixed ... right now, someone commits a bug fix, and
> then fires off an email to the list stating its fixed ... with a bug
> tracking system, then they have to go one more step, open up a web
> browser, login to the system, find the bug report and close it ...

With something like the SDM, the developer can also just mark that bug fixed
online and whoever requested notifications (like maybe the pgsql-hackers
mailing list) can get a one-liner automated email. This is great for
individual users who are particularly interested in seeing one bug fixed:
they request notification on that bug, and only get notifications that
pertain to it...

-Ben



How Do You Pronounce "PostgreSQL"?

From
"David Lloyd-Jones"
Date:
I'm putting on my suits-type suit for just a moment.
In order to Conquer The Universe(tm) why don't we just call it "PG"?
-dlj.
 




Re: Bug tracking (was Re: +/- Inf for float8's)

From
Ben Adida
Date:
Tom Lane wrote:

> FWIW, I agree with Thomas' comments: our last try at a bug-tracking
> system was a spectacular failure, and rather than just trying again
> with a technically better bug-tracker, we need to understand why we
> failed before.

Okay, well then this is an interesting discussion: the OpenACS project
and my company (OpenForce) would very much be interested in discussing
what you guys think would be a useful bug tracking system. What kind of
features would it need? Nothing is too fancy here, the idea is to have
the best possible system, one that focused developers like the PG team
would use.

Maybe we should take this offline? I'm happy to keep this discussion
going and hear everything you've got to say: understanding what it would
take to build a system you guys would use is *very* important data.

-Ben



Re: Re: [GENERAL] +/- Inf for float8's

From
"Ross J. Reedstrom"
Date:
On Sun, Aug 20, 2000 at 12:33:00AM +0200, Peter Eisentraut wrote:
<snip side comment about bug tracking. My input: for an email controllable
system, take a look at the debian bug tracking system>

> Show me a system where it doesn't work and we'll get it to work.
> UNSAFE_FLOATS as it stands it probably not the most appropriate behaviour;
> it intends to speed things up, not make things portable.
>

I agree. In the previous thread on this, Thomas suggested creating a flag
that would allow control turning the  CheckFloat8Val function calls into
a macro NOOP. Sound slike a plan to me.

>
> > > NULL and NaN are not quite the same thing imho. If we are allowing NaN
> > > in columns, then it is *known* to be NaN.
> >
> > For the purposes of ordering, however, they are very similar.
>
> Then we can also treat them similar, i.e. sort them all last or all first.
> If you have NaN's in your data you wouldn't be interested in ordering
> anyway.

Right, but the problem is that NULLs are an SQL language feature, and
there for rightly special cased directly in the sorting apparatus. NaN is
type specific, and I'd be loath to special case it in the same place. As
it happens, I've spent some time this weekend groveling through the sort
(and index, as it happens) code, and have an idea for a type specific fix.

Here's the deal, and an actual, honest to goodness bug in the current code.

As it stands, we allow one non-finite to be stored in a float8 field:
NaN, with partial parsing of 'Infinity'.

As I reported last week, NaNs break sorts: they act as barriers, creating
sorted subsections in the output.  As those familiar with the code have
already guessed, there is a more serious bug: NaNs break indicies on
float8 fields, essentially chopping the index off at the first NaN.

Fixing this turns out to be a one liner to btfloat8cmp.

Fixing sorts is a bit tricker, but can be done: Currently, I've hacked
the float8lt and float8gt code to sort NaN to after +/-Infinity. (since
NULLs are special cased, they end up sorting after NaN). I don't see
any problems with this solution, and it give the desired behavior.

I've attached a patch which fixes all the sort and index problems, as well
as adding input support for -Infinity. This is not a complete solution,
since I haven't done anything with the CheckFloat8Val test. On my
system (linux/glibc2.1) compiling with UNSAFE_FLOATS seems to work fine
for testing.

>
> Side note 2: The paper "How Java's floating point hurts everyone
> everywhere" provides for good context reading.

http://http/cs.berkeley.edu/~wkahan/JAVAhurt.pdf ? I'll take a look at it
when I get in to work Monday.

>
> Side note 3: Once you read that paper you will agree that using floating
> point with Postgres is completely insane as long as the FE/BE protocol is
> text-based.

Probably. But it's not our job to enforce sanity, right? Another way to think
about it is fixing the implementation so the deficiencies of the FE/BE stand
out in a clearer light. ;-)

Ross
--
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu>
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St.,  Houston, TX 77005


Attachment

Re: How Do You Pronounce "PostgreSQL"?

From
The Hermit Hacker
Date:
what's wrong wth "Post-Gres-QL"?

I find it soooo simple to pronounce *shrug*

On Sun, 20 Aug 2000, David Lloyd-Jones wrote:

> I'm putting on my suits-type suit for just a moment.
>  
> In order to Conquer The Universe(tm) why don't we just call it "PG"?
>  
>                                                       -dlj.
>  
> 
> 

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 



Re: Bug tracking (was Re: +/- Inf for float8's)

From
Philip Warner
Date:
At 01:15 20/08/00 -0400, Ben Adida wrote:
>Okay, well then this is an interesting discussion: the OpenACS project
>and my company (OpenForce) would very much be interested in discussing
>what you guys think would be a useful bug tracking system. 

So am I.

>
>Maybe we should take this offline? 

If you do decide to go offline, I'd appreciate some CCs...


----------------------------------------------------------------
Philip Warner                    |     __---_____
Albatross Consulting Pty. Ltd.   |----/       -  \
(A.B.N. 75 008 659 498)          |          /(@)   ______---_
Tel: (+61) 0500 83 82 81         |                 _________  \
Fax: (+61) 0500 83 82 82         |                 ___________ |
Http://www.rhyme.com.au          |                /           \|                                |    --________--
PGP key available upon request,  |  /
and from pgp5.ai.mit.edu:11371   |/


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Ned Lilly
Date:
If you all don't mind, I think it'd be great to keep this discussion on the
list.  As some of you know, Great Bridge is working on its own project
hosting site (including a PG-backed bug tracking module).  We're not quite
ready to pull back the curtain yet, but are getting close, and will be
actively soliciting input (and hacks) from the community.

The process of getting hacker requirements for such a system is a very
useful one, IMHO...

Regards,
Ned


Philip Warner wrote:

> At 01:15 20/08/00 -0400, Ben Adida wrote:
> >Okay, well then this is an interesting discussion: the OpenACS project
> >and my company (OpenForce) would very much be interested in discussing
> >what you guys think would be a useful bug tracking system.
>
> So am I.
>
> >
> >Maybe we should take this offline?
>
> If you do decide to go offline, I'd appreciate some CCs...
>
> ----------------------------------------------------------------
> Philip Warner                    |     __---_____
> Albatross Consulting Pty. Ltd.   |----/       -  \
> (A.B.N. 75 008 659 498)          |          /(@)   ______---_
> Tel: (+61) 0500 83 82 81         |                 _________  \
> Fax: (+61) 0500 83 82 82         |                 ___________ |
> Http://www.rhyme.com.au          |                /           \|
>                                  |    --________--
> PGP key available upon request,  |  /
> and from pgp5.ai.mit.edu:11371   |/



Re: Re: [GENERAL] +/- Inf for float8's

From
Tom Lane
Date:
"Ross J. Reedstrom" <reedstrm@rice.edu> writes:
> !     switch (isinf(num))
> !     {
> !         case -1:
> !             PG_RETURN_CSTRING(strcpy(ascii, "-Infinity"));
> !             break;
> !         case 1:
> !             PG_RETURN_CSTRING(strcpy(ascii, "Infinity"));
> !             break;
> !         default:
> !             break;
> !     }

My man page for isinf() sez:
     isinf() returns a positive integer if x is +INFINITY, or a negative     integer if x is -INFINITY.  Otherwise it
returnszero.
 

so the above switch statement is making an unportable assumption about
exactly which positive or negative value will be returned.

> +     if (isnan(arg2)) PG_RETURN_BOOL(1); 

PG_RETURN_BOOL(true), please...

> !     if (isnan(a))
> !         PG_RETURN_INT32(1);

Do not like this at all --- doesn't it make the result of btint4cmp(NaN,
NaN) dependent on which argument chances to be first?  Seems to me that
you must consider two NaNs to be equal, unless you want to subdivide
the category of NaNs.
        regards, tom lane


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Don Baccus
Date:
At 10:22 PM 8/20/00 -0400, Ned Lilly wrote:
>If you all don't mind, I think it'd be great to keep this discussion on the
>list.  As some of you know, Great Bridge is working on its own project
>hosting site (including a PG-backed bug tracking module).  We're not quite
>ready to pull back the curtain yet, but are getting close, and will be
>actively soliciting input (and hacks) from the community.

So - again - why roll your own instead of build upon a base which at least
is already seeing some use, by some fairly large organizations (arsDigita
is larger and more deeply funded than Great Bridge, making profits to boot,
and I won't even start talking about AOL)?  arsDigita is putting some
developer
effort into the SDM, so it's no longer just Ben and whoever he can rope into
helping out.

Couldn't you guys more profitably spend time, say, working on outer joins
rather than doing something like this?

The folks working on the SDM have a LOT more web/db development experience
than whoever's rolling your bug tracking system.  

I keep sniffing the odor of NIH ...



- Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert
Serviceand other goodies at http://donb.photo.net.
 


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Don Baccus
Date:
>At 10:22 PM 8/20/00 -0400, Ned Lilly wrote:
>>If you all don't mind, I think it'd be great to keep this discussion on the
>>list.  As some of you know, Great Bridge is working on its own project
>>hosting site (including a PG-backed bug tracking module).  We're not quite
>>ready to pull back the curtain yet, but are getting close, and will be
>>actively soliciting input (and hacks) from the community.

Another implication which missed me first time 'round is that Great Bridge
might be planning to have its own bug reporting system, separate from 
that used by the development community at large?

I hope not.  There should be one central place for bug reporting.  If 
Great Bridge wants to run it, fine, also if Great Bridge wants to be able 
to incorporate some sort of prioritization system for those with paid
support (or some other discriminatory system) it is still probably better
to figure out a way to accomodate it rather than have two separate 
bug reporting systems.



- Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert
Serviceand other goodies at http://donb.photo.net.
 


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Vince Vielhaber
Date:
On Mon, 21 Aug 2000, Don Baccus wrote:

> At 10:22 PM 8/20/00 -0400, Ned Lilly wrote:
> >If you all don't mind, I think it'd be great to keep this discussion on the
> >list.  As some of you know, Great Bridge is working on its own project
> >hosting site (including a PG-backed bug tracking module).  We're not quite
> >ready to pull back the curtain yet, but are getting close, and will be
> >actively soliciting input (and hacks) from the community.
> 
> So - again - why roll your own instead of build upon a base which at least
> is already seeing some use, by some fairly large organizations (arsDigita
> is larger and more deeply funded than Great Bridge, making profits to boot,
> and I won't even start talking about AOL)?  arsDigita is putting some
> developer
> effort into the SDM, so it's no longer just Ben and whoever he can rope into
> helping out.
> 
> Couldn't you guys more profitably spend time, say, working on outer joins
> rather than doing something like this?
> 
> The folks working on the SDM have a LOT more web/db development experience
> than whoever's rolling your bug tracking system.  
> 
> I keep sniffing the odor of NIH ...

Could it be possible that folks are shying away because of having
to install and learn an entire webserver and tools and then the 
bug tracker on top of that?

Vince.
-- 
==========================================================================
Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from
$16.00/moat Pop4 Networking       Online Campground Directory    http://www.camping-usa.com      Online Giftshop
Superstore   http://www.cloudninegifts.com
 
==========================================================================





Re: Bug tracking (was Re: +/- Inf for float8's)

From
Ben Adida
Date:
Vince Vielhaber wrote:

> Could it be possible that folks are shying away because of having
> to install and learn an entire webserver and tools and then the
> bug tracker on top of that?

And so the solution is to build a totally new system?

Coming from the Postgres team, this is relatively surprising. Postgres is a great
tool. I dropped Oracle and learned Postgres because I thought it would eventually
become a better tool, and because it was already better in many ways. It took
time and effort to do so, but eventually it was the right thing to do because I
can now make full use of a very powerful open-source database.

It seems to me that the whole point of Open-Source vs. Not Invented Here is that
you are *supposed* to go out and make the effort necessary to learn new tools
that can then become extremely useful. If you accept the attitude of "it's not
Apache/mod-perl so I'm not using it," then it's time to stop all criticism of
those who use MySQL, Oracle, Windows, etc... Those people are *used* to their
technology, and the only reason they refuse to switch is that they don't want to
spend time learning something new.

Just my 2 cents.... the useful tools are not always the ones everyone is using.

-Ben



Re: Bug tracking (was Re: +/- Inf for float8's)

From
Vince Vielhaber
Date:
On Mon, 21 Aug 2000, Ben Adida wrote:

> Vince Vielhaber wrote:
> 
> > Could it be possible that folks are shying away because of having
> > to install and learn an entire webserver and tools and then the
> > bug tracker on top of that?
> 
> And so the solution is to build a totally new system?

In some cases yes.  
> Coming from the Postgres team, this is relatively surprising. Postgres is a great
> tool. I dropped Oracle and learned Postgres because I thought it would eventually
> become a better tool, and because it was already better in many ways. It took
> time and effort to do so, but eventually it was the right thing to do because I
> can now make full use of a very powerful open-source database.

I am *NOT* "the Postgres team".  But have you listened to what you & Don
are suggesting that we, or for that matter anyone else in need of a bug
tracking system, do?  You want us to install the full blown arsDigita with
all the bells and whistles just for a bug tracker.  That's like saying I 
need a pickup truck to move a chair so I'm going to go out and get a new
FreightLiner with a 55' trailer to do the job.

> It seems to me that the whole point of Open-Source vs. Not Invented Here is that
> you are *supposed* to go out and make the effort necessary to learn new tools
> that can then become extremely useful. If you accept the attitude of "it's not
> Apache/mod-perl so I'm not using it," then it's time to stop all criticism of
> those who use MySQL, Oracle, Windows, etc... Those people are *used* to their
> technology, and the only reason they refuse to switch is that they don't want to
> spend time learning something new.

You missed the point.  It's called overkill.  You needed a full blown
database for your project.  We need (although _want_ may be another story)
a bug tracker - not a new webserver.

Vince.
-- 
==========================================================================
Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from
$16.00/moat Pop4 Networking       Online Campground Directory    http://www.camping-usa.com      Online Giftshop
Superstore   http://www.cloudninegifts.com
 
==========================================================================





Re: Bug tracking (was Re: +/- Inf for float8's)

From
Adam Haberlach
Date:
On Mon, Aug 21, 2000 at 07:35:10AM -0700, Don Baccus wrote:
> 
> >At 10:22 PM 8/20/00 -0400, Ned Lilly wrote:
> >>If you all don't mind, I think it'd be great to keep this discussion on the
> >>list.  As some of you know, Great Bridge is working on its own project
> >>hosting site (including a PG-backed bug tracking module).  We're not quite
> >>ready to pull back the curtain yet, but are getting close, and will be
> >>actively soliciting input (and hacks) from the community.
> 
> Another implication which missed me first time 'round is that Great Bridge
> might be planning to have its own bug reporting system, separate from 
> that used by the development community at large?
Cool your conspiracy theories.  I'm not yet involved with either side
of this discussion, but before it runs out of control...

> I hope not.  There should be one central place for bug reporting.  If 
> Great Bridge wants to run it, fine, also if Great Bridge wants to be able 
> to incorporate some sort of prioritization system for those with paid
> support (or some other discriminatory system) it is still probably better
> to figure out a way to accomodate it rather than have two separate 
> bug reporting systems.
The fact is that postgres already has a very good system for keeping
track of issues from report to fix to verification.  So far the main defect
is the obvious one of "People don't know the history unless they troll the
message archives or lurk".  Everyone here is leery of "fixing" a working
system.  Especially when it entails modifying the working system to deal
with a new issue database.
Bug Database/Issue Trackers can be done in two ways.

Someone can grab an off-the-shelf system like Bugzilla or this ArsTechnica 
thing and then try to make the project conform to it.  So far, everyone 
I've talked to who has touched Bugzilla has said that it sucks.  I don't 
know anything about this other proposed system but it will probably require
a lot of time to even get people to use it regularly, much less use it well.

The other method is to create the system to match the process in place.
Since the postgres project is already very well organized, I personally would
like to see the custom system, rather then make Bruce throw away his TODO
list and use someone else's idea of an issue tracking system.  It takes a lot
more work--someone has to pay attention to what is going on with the project
and make sure the database stays in sync, but in the long run, it is less
disruptive and smoother to integrate into an already working project like
this one.


-- 
Adam Haberlach             | "A farm tractor is not a motorcycle."
adam@newsnipple.com        |   --California DMV 1999
http://www.newsnipple.com/ |        Motorcycle Driver Handbook


Re: Re: [GENERAL] +/- Inf for float8's

From
"Ross J. Reedstrom"
Date:
Tom -
Thanks for the review. Here's a new version of the patch, fixing the two
you objected to. Unfotunately, I seem to have found another corner case
in the existing code that needs fixing. Here's the one line version:

Use of an index in an ORDER BY DESC result changes placement of NULLs
(and NaNs, now) from last returned to first returned tuples

Long version:

While examining the output from ORDER BY queries, both using and not using
an index, I came across a discrepancy: the explicit handling of NULLs in
the tuplesort case always sorts NULLs to the end, regardless of direction
of sort. Intellectually, I kind of like that: "We don't know what these are,
let's just tack them on the end". I implemented NaN sorting to emulate that
behavior. This also has the pleasant property that NULL (or NaN) are never
returned as > or < any other possible value, should be expected.

However, if an index is involved, the index gets built, and the NULL
values are stored at one end of the index. So, when a ORDER BY DESC is
requested, the index is just read backwards, sending the NULLs (and NaNs)
first. (They're still not returned from a query with a clause such as
WHERE f1 > 0.)

An example of the output is attached, from the regress float8 table (with
a NULL and non-finites added. Don't need the non-finites to to display
the problem, though, since it's NULLs as well) Note the blank row,
which is the NULL, moves from the bottom to the top in the last case,
using the index.

So, what way should we go here? Make ASC/DESC actual mirrors of each other
in the direct sort case, as well? Hack the index scan to know about nodes
that always go to the end? Document it as a quirk? (Not likely: selection of
plan should never affect output.)

To make the direct sort the same as the index read would work for NULL,
but for NaN would either require allowing NaN to be returned as >
Infinity, which doesn't happen now, or add another ordering operator
that is only used for the sort case (use of '>' and '<' seems to be
hardcoded all the way to the parser)

Thoughts?

Ross

--
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu>
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St.,  Houston, TX 77005

Attachment

Re: Re: [GENERAL] +/- Inf for float8's

From
Tom Lane
Date:
"Ross J. Reedstrom" <reedstrm@rice.edu> writes:
> While examining the output from ORDER BY queries, both using and not using
> an index, I came across a discrepancy: the explicit handling of NULLs in
> the tuplesort case always sorts NULLs to the end, regardless of direction
> of sort.

Yeah.  I think that's widely considered a bug --- we have a TODO item to
fix it.  You might care to dig in the archives for prior discussions.

> To make the direct sort the same as the index read would work for NULL,
> but for NaN would either require allowing NaN to be returned as >
> Infinity, which doesn't happen now,

Seems to me the sort order should be
-Infinitynormal values+Infinityother types of NaNNULL

and the reverse in a descending sort.

> or add another ordering operator that is only used for the sort case
> (use of '>' and '<' seems to be hardcoded all the way to the parser)

don't even think about that...
        regards, tom lane


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Don Baccus
Date:
At 10:28 AM 8/21/00 -0700, Adam Haberlach wrote:
>On Mon, Aug 21, 2000 at 07:35:10AM -0700, Don Baccus wrote:

>> Another implication which missed me first time 'round is that Great Bridge
>> might be planning to have its own bug reporting system, separate from 
>> that used by the development community at large?
>
>    Cool your conspiracy theories.

I'm making an observation, that's all.  Cool your own wild theories, please.

>
>> I hope not.  There should be one central place for bug reporting.  If 
>> Great Bridge wants to run it, fine, also if Great Bridge wants to be able 
>> to incorporate some sort of prioritization system for those with paid
>> support (or some other discriminatory system) it is still probably better
>> to figure out a way to accomodate it rather than have two separate 
>> bug reporting systems.
>
>    The fact is that postgres already has a very good system for keeping
>track of issues from report to fix to verification.

<shrug> I didn't raise the subject, it was a core developer who started
this thread with a semi-rant about it being about time that the project
had decent bug tracking software.

So apparently not everyone in the community agrees with your analysis.  If
there were consensus that the current system's great then Great Bridge
wouldn't
be looking at implementing something different, and Peter wouldn't be ranting
that something better is needed.

> So far the main defect
>is the obvious one of "People don't know the history unless they troll the
>message archives or lurk".  Everyone here is leery of "fixing" a working
>system.

There seems to be some disagreement about how well it works.  Again, I didn't
raise the issue, I simply responded with a possible solution when one of the
core developers raised it.  And I know that Great Bridge wants to do something
web-based - this isn't some fantasy I dreamed up when in a psychotic state.

I'm only saying that if a different approach is to be taken, why not build
on something that exists, is under active development, and is being driven
by folks who are VERY open to working with the project to make the tool
fit the project rather than vice-versa?

>
>Someone can grab an off-the-shelf system like Bugzilla or this ArsTechnica 

arsDigita

>thing and then try to make the project conform to it.

Read above.  Ben's already posted that he's eager for design input.  aD has
already enhanced the thing based on their own needs, and there's no reason
why Great Bridge and the Postgres crew can't do the same.

>I don't 
>know anything about this other proposed system but it will probably require
>a lot of time to even get people to use it regularly, much less use it well.

Strange, OpenACS folk use it regularly and all we've done is put a "report
a bug" link on the home page.

I haven't heard so many arguments against change since the VT100 started
replacing the KSR 35!



- Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert
Serviceand other goodies at http://donb.photo.net.
 


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Don Baccus
Date:
At 10:48 AM 8/21/00 -0400, Vince Vielhaber wrote:

>Could it be possible that folks are shying away because of having
>to install and learn an entire webserver and tools and then the 
>bug tracker on top of that?

Learning to use AOLserver is going to be harder than writing a
bugtracker and associated tools from scratch?  I find that hard to
believe.  

If it's true, of course they could run Apache, since arsDigita
provides a module which implements the AOLserver API in Apache
for exactly this reason, thus making it possible to run the
toolkit (including the SDM) under Apache.



- Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert
Serviceand other goodies at http://donb.photo.net.
 


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Don Baccus
Date:
At 11:12 AM 8/21/00 -0400, Vince Vielhaber wrote:

>I am *NOT* "the Postgres team".  But have you listened to what you & Don
>are suggesting that we, or for that matter anyone else in need of a bug
>tracking system, do?  You want us to install the full blown arsDigita with
>all the bells and whistles just for a bug tracker.  That's like saying I 
>need a pickup truck to move a chair so I'm going to go out and get a new
>FreightLiner with a 55' trailer to do the job.

A rather dubious analogy. 

It takes me less than half a day to install AOLserver, Postgres and the
toolkit on a virgin system, including setting up user accounts, etc.

And ... you never know.  Other parts of the toolkit might turn out to be
useful.

If not, just leave them turned off.  Take a look at openacs.org - do you
find any traces of the e-commerce module there?  The intranet company
management module?  What do you see?  You see the use of perhaps 10% of
the toolkit.

This is slightly different than hauling an 18-wheeler around.   Software
and trucks bear little resemblence to each other, though Freightliner does
have its home here in Portland, OR.

And, of course, the SDM has a bit more functionality than a simple bugtracker,
which is just one piece.  It will be gaining more functionality over time,
including increased integration with CVS (there is already some integration,
i.e. the ability to browse the tree).

>You missed the point.  It's called overkill.  You needed a full blown
>database for your project.  We need (although _want_ may be another story)
>a bug tracker - not a new webserver.

Then run it under Apache.



- Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert
Serviceand other goodies at http://donb.photo.net.
 


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Alfred Perlstein
Date:
* Don Baccus <dhogaza@pacifier.com> [000821 11:48] wrote:
> At 11:12 AM 8/21/00 -0400, Vince Vielhaber wrote:
> 
> >You missed the point.  It's called overkill.  You needed a full blown
> >database for your project.  We need (although _want_ may be another story)
> >a bug tracker - not a new webserver.
> 
> Then run it under Apache.

Sorry to jump in without reading the entire thread, but has GNATS
(what the FreeBSD team uses) or Bugzilla come up as options?

GNATS is a bit crusty but works pretty ok for us.

-- 
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
"I have the heart of a child; I keep it in a jar on my desk."


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Vince Vielhaber
Date:
On Mon, 21 Aug 2000, Don Baccus wrote:

> At 10:48 AM 8/21/00 -0400, Vince Vielhaber wrote:
> 
> >Could it be possible that folks are shying away because of having
> >to install and learn an entire webserver and tools and then the 
> >bug tracker on top of that?
> 
> Learning to use AOLserver is going to be harder than writing a
> bugtracker and associated tools from scratch?  I find that hard to
> believe.  

Learning how to use it is only a tiny part of it.  You still have to
migrate your website to it.   It's not a drop in replacement.  So
writing a bugtracker that will fit the environment vs learning a new
webserver & migrating your website & rebuilding or rewriting custom 
apps ...   For the average, busy admin, don't count too heavily on
the latter.   They're more likely to stick with what they know and
trust regardless of how good something else is reported to be.

> If it's true, of course they could run Apache, since arsDigita
> provides a module which implements the AOLserver API in Apache
> for exactly this reason, thus making it possible to run the
> toolkit (including the SDM) under Apache.

First I heard of this, but I'd also have concerns of it's reliability.
It has to be real new.   And if it fails it's not ars that looks bad,
it's the site that's running it.  Remember the flack over udmsearch?
PostgreSQL was slammed over and over because udmsearch wasn't working
right.  

Vince.
-- 
==========================================================================
Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from
$16.00/moat Pop4 Networking       Online Campground Directory    http://www.camping-usa.com      Online Giftshop
Superstore   http://www.cloudninegifts.com
 
==========================================================================





Re: Bug tracking (was Re: +/- Inf for float8's)

From
Vince Vielhaber
Date:
On Mon, 21 Aug 2000, Don Baccus wrote:

> At 11:12 AM 8/21/00 -0400, Vince Vielhaber wrote:
> 
> >I am *NOT* "the Postgres team".  But have you listened to what you & Don
> >are suggesting that we, or for that matter anyone else in need of a bug
> >tracking system, do?  You want us to install the full blown arsDigita with
> >all the bells and whistles just for a bug tracker.  That's like saying I 
> >need a pickup truck to move a chair so I'm going to go out and get a new
> >FreightLiner with a 55' trailer to do the job.
> 
> A rather dubious analogy. 
> 
> It takes me less than half a day to install AOLserver, Postgres and the
> toolkit on a virgin system, including setting up user accounts, etc.

How familiar are you with it as opposed to most others on the net?
> And ... you never know.  Other parts of the toolkit might turn out to be
> useful.
> 
> If not, just leave them turned off.  Take a look at openacs.org - do you
> find any traces of the e-commerce module there?  The intranet company
> management module?  What do you see?  You see the use of perhaps 10% of
> the toolkit.
> 
> This is slightly different than hauling an 18-wheeler around.   Software
> and trucks bear little resemblence to each other, though Freightliner does
> have its home here in Portland, OR.

You really don't get it do you?  I'm not comparing software and trucks,
I'm comparing tools to do a job.  Once you grasp that concept you'll be
able to catch on to what I'm talking about - until then I'm just wasting
my time.

Vince.
-- 
==========================================================================
Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from
$16.00/moat Pop4 Networking       Online Campground Directory    http://www.camping-usa.com      Online Giftshop
Superstore   http://www.cloudninegifts.com
 
==========================================================================





Re: Bug tracking (was Re: +/- Inf for float8's)

From
Don Baccus
Date:
At 02:54 PM 8/21/00 -0400, Vince Vielhaber wrote:
>On Mon, 21 Aug 2000, Don Baccus wrote:

>> Learning to use AOLserver is going to be harder than writing a
>> bugtracker and associated tools from scratch?  I find that hard to
>> believe.  

>Learning how to use it is only a tiny part of it.  You still have to
>migrate your website to it.   It's not a drop in replacement.  So
>writing a bugtracker that will fit the environment vs learning a new
>webserver & migrating your website & rebuilding or rewriting custom 
>apps ...   For the average, busy admin, don't count too heavily on
>the latter.   They're more likely to stick with what they know and
>trust regardless of how good something else is reported to be.

So run the development portion of the site on a different server.  Who
said anything about migrating the entire postgres site to AOLserver???

>> If it's true, of course they could run Apache, since arsDigita
>> provides a module which implements the AOLserver API in Apache
>> for exactly this reason, thus making it possible to run the
>> toolkit (including the SDM) under Apache.
>
>First I heard of this, but I'd also have concerns of it's reliability.
>It has to be real new.

Yep.  Written by Robert Thau, one of the original eight core Apache
developers, under contract to aD.

>And if it fails it's not ars that looks bad,
>it's the site that's running it.

arsDigita gets on average greater than $500,000 to develop and deploy
a website.

If aD deploys one on Apache+mod_aolserver (they paid for the development
of this module) and it falls over, do you really believe aD won't look
bad?

Seeing as they'd very likely be sued, I think you're wrong.



- Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert
Serviceand other goodies at http://donb.photo.net.
 


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Don Baccus
Date:
At 02:58 PM 8/21/00 -0400, Vince Vielhaber wrote:
>On Mon, 21 Aug 2000, Don Baccus wrote:

>> It takes me less than half a day to install AOLserver, Postgres and the
>> toolkit on a virgin system, including setting up user accounts, etc.
>
>How familiar are you with it as opposed to most others on the net?

Ben has already offered to help out and has an account on hub.  If I weren't
leaving town for five of the next six or so weeks, I would too.

Still, we have strangers to all three pieces installing everything in a
weekend, usually with a bit of help.  If you were to install it, you'd
be familiar with Postgres which is about 1/3 of the confusion for newcomers.

>You really don't get it do you?

Yes, I do.

>  I'm not comparing software and trucks,
>I'm comparing tools to do a job.  Once you grasp that concept you'll be
>able to catch on to what I'm talking about - until then I'm just wasting
>my time.

Do I detect a flame?  Ohhh...

It's a toolkit, Vincent.  Once you grasp the notion that using a wrench
out of your toolbox doesn't mean you have to use every tool in the
toolbox you'll be able to grasp what I'm talking about.  

I answered your previous question plainly.  If you're not capable of
understanding the answer, don't answer with an invitation for a flamefest
you have no chance of winning, OK?



- Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert
Serviceand other goodies at http://donb.photo.net.
 


Re: Re: [GENERAL] +/- Inf for float8's

From
"Ross J. Reedstrom"
Date:
On Mon, Aug 21, 2000 at 01:39:32PM -0400, Tom Lane wrote:
> "Ross J. Reedstrom" <reedstrm@rice.edu> writes:
> > While examining the output from ORDER BY queries, both using and not using
> > an index, I came across a discrepancy: the explicit handling of NULLs in
> > the tuplesort case always sorts NULLs to the end, regardless of direction
> > of sort.
> 
> Yeah.  I think that's widely considered a bug --- we have a TODO item to
> fix it.  You might care to dig in the archives for prior discussions.

I'll take a look.

> 
> > To make the direct sort the same as the index read would work for NULL,
> > but for NaN would either require allowing NaN to be returned as >
> > Infinity, which doesn't happen now,
> 
> Seems to me the sort order should be
> 
>     -Infinity
>     normal values
>     +Infinity
>     other types of NaN
>     NULL
> 
> and the reverse in a descending sort.
> 
> > or add another ordering operator that is only used for the sort case
> > (use of '>' and '<' seems to be hardcoded all the way to the parser)
> 
> don't even think about that...

Sure, but any ideas _how_ to get 'NaN > +Infinity' to happen only during a
sort operation, and not when '>' is called explicity as a WHERE condition,
by touching only type specific code? 'Cause I'd call it a bug to be able
to say:

SELECT * from foo where f8 > 'Infinity';

and get anything at all back.

NULL is taken care of by special casing in the sort code, as I already mentioned,
and can be fixed immediately.

Ross
-- 
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> 
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St.,  Houston, TX 77005


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Don Baccus
Date:
At 04:34 PM 8/21/00 -0300, The Hermit Hacker wrote:

>Ummm, just stupid question here, but I already posted that AOLserver was
>installed and ready for you guys to implement this ... Ben already has an
>account to get in and do it ... instead of arguing about it, why not just
>do it?  If nobody likes it/uses it, so be it ... its no skin off my
>back.  But right now the arguments that are going back and forth seem to
>be sooooooo useless since they *seem* to involve technical issues that
>aren't issues ...

Well, this is a breath of fresh air ...

Hopefully Ben will have time soon to do so.  Unfortunately (well, not from
my personal point of view!) I'm about to leave for five of the next six
weeks, four of those spent where the internet doesn't reach (where nothing
but BLM radio dispatch doesn't reach, to be more precise) so I'm not going
to be able to help.

Otherwise I'd just jump in.

Of course, putting it up will just make its shortcomings apparent, and 
those who resist even the concept of change will ignore the fact that
Ben and I have both stated that modifying the SDM to fit the project
rather than modifying the project to fit the SDM and say, "see, this
doesn't fit our needs!" blah blah blah.

Just as the red herring you've mentioned has been raised.



- Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert
Serviceand other goodies at http://donb.photo.net.
 


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Joe Brenner
Date:
Well, just to throw another piece of information into the
mix, there is a new bug-tracking system under development by
the folks at collab.net.  They call it "scarab", and last I
talked to them that they thought it would be ready for
production use Real Soon Now: 
  http://scarab.tigris.org/

I guess they regard this as a replacement for bugzilla.  


Some personal opinions: 

(1) I would actually like to see bugzilla fixed rather than
replaced.  The collab.net guys are into java servlets
because they're CS geeks who are down on perl.  Me, I'm a
perl loyalist who thinks that Larry Wall is onto something
-- mathematical elegance may not be the right standard to
judge a computer language.

(2) And maybe it'd be nice if the "religous wars" could be
dropped in favor of pure objective technical decision
making, but i don't think they can be: the social and the
technical don't neatly split into two little piles.  (A case
in point: the argument that using a mailing list for bug
control is somehow "warmer" or "more human" than a bug
database.


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Vince Vielhaber
Date:
On Mon, 21 Aug 2000, Don Baccus wrote:

> At 02:58 PM 8/21/00 -0400, Vince Vielhaber wrote:
> >On Mon, 21 Aug 2000, Don Baccus wrote:
> 
> >> It takes me less than half a day to install AOLserver, Postgres and the
> >> toolkit on a virgin system, including setting up user accounts, etc.
> >
> >How familiar are you with it as opposed to most others on the net?
> 
> Ben has already offered to help out and has an account on hub.  If I weren't
> leaving town for five of the next six or so weeks, I would too.
> 
> Still, we have strangers to all three pieces installing everything in a
> weekend, usually with a bit of help.  If you were to install it, you'd
> be familiar with Postgres which is about 1/3 of the confusion for newcomers.
> 
> >You really don't get it do you?
> 
> Yes, I do.
> 
> >  I'm not comparing software and trucks,
> >I'm comparing tools to do a job.  Once you grasp that concept you'll be
> >able to catch on to what I'm talking about - until then I'm just wasting
> >my time.
> 
> Do I detect a flame?  Ohhh...
> 
> It's a toolkit, Vincent.  Once you grasp the notion that using a wrench
> out of your toolbox doesn't mean you have to use every tool in the
> toolbox you'll be able to grasp what I'm talking about.  

Yet you insist on shoving the entire toolbox down our throats every time
there's a task to be done, Donnie.

> I answered your previous question plainly.  If you're not capable of
> understanding the answer, don't answer with an invitation for a flamefest
> you have no chance of winning, OK?

When the day comes that you actually answer a question without telling
the world that openacs, arsdigita, aolserver or whatever you want to call
it is the answer and saviour to everything from world peace to who cares
what else, I'll believe you answered a "question plainly".  But that's
not likely to ever happen.   And since you think that anything I've said
so far has been a flame I doubt you'd know one if it slapped you with a
trout.   

One more thing, Donnie...   ***PLONK***

Vince.
-- 
==========================================================================
Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from
$16.00/moat Pop4 Networking       Online Campground Directory    http://www.camping-usa.com      Online Giftshop
Superstore   http://www.cloudninegifts.com
 
==========================================================================





Re: Bug tracking (was Re: +/- Inf for float8's)

From
Lamar Owen
Date:
[first off, I got rid of that awful cc: list..... ARggghhh....]
Don Baccus wrote:
> At 04:34 PM 8/21/00 -0300, The Hermit Hacker wrote:
> >Ummm, just stupid question here, but I already posted that AOLserver was
> >installed and ready for you guys to implement this ... Ben already has an
> >account to get in and do it ... instead of arguing about it, why not just
> Well, this is a breath of fresh air ...
> Hopefully Ben will have time soon to do so.  Unfortunately (well, not from

I'm available to help some, and also have a hub account (as well as an
operational AOLserver+PostgreSQL+OpenACS site to work on here....).

--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11


Re: Bug tracking (was Re: +/- Inf for float8's)

From
The Hermit Hacker
Date:
On Mon, 21 Aug 2000, Don Baccus wrote:

> At 04:34 PM 8/21/00 -0300, The Hermit Hacker wrote:
> 
> >Ummm, just stupid question here, but I already posted that AOLserver was
> >installed and ready for you guys to implement this ... Ben already has an
> >account to get in and do it ... instead of arguing about it, why not just
> >do it?  If nobody likes it/uses it, so be it ... its no skin off my
> >back.  But right now the arguments that are going back and forth seem to
> >be sooooooo useless since they *seem* to involve technical issues that
> >aren't issues ...
> 
> Well, this is a breath of fresh air ...
> 
> Hopefully Ben will have time soon to do so.  Unfortunately (well, not from
> my personal point of view!) I'm about to leave for five of the next six
> weeks, four of those spent where the internet doesn't reach (where nothing
> but BLM radio dispatch doesn't reach, to be more precise) so I'm not going
> to be able to help.
> 
> Otherwise I'd just jump in.
> 
> Of course, putting it up will just make its shortcomings apparent, and 
> those who resist even the concept of change will ignore the fact that
> Ben and I have both stated that modifying the SDM to fit the project
> rather than modifying the project to fit the SDM and say, "see, this
> doesn't fit our needs!" blah blah blah.
> 
> Just as the red herring you've mentioned has been raised.

Right now,  you've thrown out "it can do this, it can do that" ... I've
put forth the resources so that you can prove that it will work, its in
your court now :)




Re: Bug tracking (was Re: +/- Inf for float8's)

From
The Hermit Hacker
Date:
On Mon, 21 Aug 2000, Vince Vielhaber wrote:

> On Mon, 21 Aug 2000, Don Baccus wrote:
> 
> > At 10:48 AM 8/21/00 -0400, Vince Vielhaber wrote:
> > 
> > >Could it be possible that folks are shying away because of having
> > >to install and learn an entire webserver and tools and then the 
> > >bug tracker on top of that?
> > 
> > Learning to use AOLserver is going to be harder than writing a
> > bugtracker and associated tools from scratch?  I find that hard to
> > believe.  
> 
> Learning how to use it is only a tiny part of it.  You still have to
> migrate your website to it.  It's not a drop in replacement.  So
> writing a bugtracker that will fit the environment vs learning a new
> webserver & migrating your website & rebuilding or rewriting custom
> apps ...  For the average, busy admin, don't count too heavily on the
> latter.  They're more likely to stick with what they know and trust
> regardless of how good something else is reported to be.

why not just run aolserver on a different port like we do?




Re: How Do You Pronounce "PostgreSQL"?

From
"Henry B. Hotz"
Date:
At 7:48 PM -0300 8/20/00, The Hermit Hacker wrote:
>what's wrong wth "Post-Gres-QL"?
>
>I find it soooo simple to pronounce *shrug*
>
>On Sun, 20 Aug 2000, David Lloyd-Jones wrote:
>
> > I'm putting on my suits-type suit for just a moment.
> >
> > In order to Conquer The Universe(tm) why don't we just call it "PG"?
> >

IMNSHO the "QL" are silent and you pronounce it Postgres.  I consider 
the SQL query language to be merely a (major) feature added during 
the evolution of the system.


Signature held pending an ISO 9000 compliant
signature design and approval process.
h.b.hotz@jpl.nasa.gov, or hbhotz@oxy.edu


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Vince Vielhaber
Date:
On Mon, 21 Aug 2000, The Hermit Hacker wrote:

> On Mon, 21 Aug 2000, Vince Vielhaber wrote:
> 
> > On Mon, 21 Aug 2000, Don Baccus wrote:
> > 
> > > At 10:48 AM 8/21/00 -0400, Vince Vielhaber wrote:
> > > 
> > > >Could it be possible that folks are shying away because of having
> > > >to install and learn an entire webserver and tools and then the 
> > > >bug tracker on top of that?
> > > 
> > > Learning to use AOLserver is going to be harder than writing a
> > > bugtracker and associated tools from scratch?  I find that hard to
> > > believe.  
> > 
> > Learning how to use it is only a tiny part of it.  You still have to
> > migrate your website to it.  It's not a drop in replacement.  So
> > writing a bugtracker that will fit the environment vs learning a new
> > webserver & migrating your website & rebuilding or rewriting custom
> > apps ...  For the average, busy admin, don't count too heavily on the
> > latter.  They're more likely to stick with what they know and trust
> > regardless of how good something else is reported to be.
> 
> why not just run aolserver on a different port like we do?

If it has the functionality you desire and are looking for, sure.  My
contention is that the average site isn't going to do it for a couple
of features and would roll their own instead.

Vince.
-- 
==========================================================================
Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net128K ISDN from $22.00/mo - 56K Dialup from
$16.00/moat Pop4 Networking       Online Campground Directory    http://www.camping-usa.com      Online Giftshop
Superstore   http://www.cloudninegifts.com
 
==========================================================================





Re: How Do You Pronounce "PostgreSQL"?

From
The Hermit Hacker
Date:
On Mon, 21 Aug 2000, Henry B. Hotz wrote:

> At 7:48 PM -0300 8/20/00, The Hermit Hacker wrote:
> >what's wrong wth "Post-Gres-QL"?
> >
> >I find it soooo simple to pronounce *shrug*
> >
> >On Sun, 20 Aug 2000, David Lloyd-Jones wrote:
> >
> > > I'm putting on my suits-type suit for just a moment.
> > >
> > > In order to Conquer The Universe(tm) why don't we just call it "PG"?
> > >
> 
> IMNSHO the "QL" are silent and you pronounce it Postgres.  I consider 
> the SQL query language to be merely a (major) feature added during 
> the evolution of the system.

as stated in another email, it is not pronounced Postgres ... postgres was
a whole different beast based on a completely different query language
... if you refer to Postgres, you are,  IMHO, refering to the only
Postgres 4.2 which was the grand-daddy to what its evolved into ...




Re: Bug tracking (was Re: +/- Inf for float8's)

From
Lamar Owen
Date:
[trimmed cc: list]

Vince Vielhaber wrote:
> On Mon, 21 Aug 2000, Don Baccus wrote:
> > Vince wrote:
> > >You really don't get it do you?

> > Yes, I do.

Vince, Don really does 'get it' -- he's just pretty vehement about his
'getting it'.

> > It's a toolkit, Vincent.  Once you grasp the notion that using a wrench
> > out of your toolbox doesn't mean you have to use every tool in the
> > toolbox you'll be able to grasp what I'm talking about.
> Yet you insist on shoving the entire toolbox down our throats every time
> there's a task to be done, Donnie.

One of the many useful features of OpenACS is that you get the whole
toolbox -- a toolbox, as opposed to a 'box of tools' -- Jensen makes a
nice profit selling toolboxes with matched tools -- neat, clean, trim,
and don't look anything like my four-drawer toolbox made up of a melange
of tools and a Wal-mart toolbox.

OpenACS is like the Jensen toolset -- you get a matched case, and
high-quality tools matches to the case.  With OpenACS you get a
framework that tools can be plugged into -- tools that were designed to
be plugged in that way (well, it's not perfect -- but nothing is). 
Everything can be covered by the system-wide authentication module, user
group module, etc.  Everything is designed to work smoothly together. 
so you only use authentication+SDM -- so what.  You can expand as you
need to -- and it doesn't take up _that_ much space.

PostgreSQL is much like OpenACS (barring the funny capitalization, but I
digress): PostgreSQL is a toolbox of database routines and modules, tied
together by a SQL parser and a large set of clients with matching
arbiter/backends.  Download postgresql-version.tar.gz, and you _have_ to
get the C++ code -- even if you don't want it.  You have to get pgaccess
-- even if you won't use it.  If you want to do meaningful development,
you have to keep nearly the whole source tree around......etc.  How is
this different from the OpenACS model?  Don't want a part of OpenACS? 
Nobody is preventing you from nuking that part from your installation
tree -- just like no one is preventing someone from installing
PostgreSQL in a client-only sort of way (much easier with an RPMset, but
I again digress.....).

PostgreSQL requires libpq -- OpenACS requires (but doesn't include)
AOLserver -- that analogy is not perfect, but close.  

And, OpenACS can run just fine under Apache with mod_aolserver. 
Although, since Marc has an AOLserver available and running.... :-)  and
a killer db server (bigger :-))...

Vince, Don: sparring like this is not productive.  Both of you are
excellent hackers -- I've seen both of your code.  Let's just make it
work, and see what the hackers think of it.
> > I answered your previous question plainly.  If you're not capable of
> > understanding the answer, don't answer with an invitation for a flamefest
> > you have no chance of winning, OK?

Flamefests are unwinnable.  All parties to flamwars get burned -- either
directly, or indirectly.  I have seen too many flamewars -- and it's not
worth the risk to reputation to go too far with one.  This one is mild
so far -- on a scale from one to ten, this makes it to one-and-a-half
thus far (I've been on news.groups and news.admin (and cross-posted to
alt.flame) more than once several years back....).
> When the day comes that you actually answer a question without telling
> the world that openacs, arsdigita, aolserver or whatever you want to call
> it is the answer and saviour to everything from world peace to who cares

Vince, try it.  You might like it.  But, it does require some different
thinking -- which if you don't have time to do, your loss.  Or, to put
it differently -- I just recently learned how to write CGI scripts. 
That may seem laughable -- but I had already written many dynamic web
pages using AOLserver TCL -- who needs CGI?  The current rush on PHP
programmers shows this -- many PHP programmers wouldn't have a clue how
to go about writing a CGI script -- and, guess what -- with PHP you
don't need CGI.  The AOLserver TCL API is like PHP on steroids in many
case -- and, in many other cases, PHP out-API's AOLserver.  

And I do remember that you _have_ tried AOLserver, about a year ago....

--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Lamar Owen
Date:
[cc: list trimmed]

Vince Vielhaber wrote:
> On Mon, 21 Aug 2000, The Hermit Hacker wrote:

> > why not just run aolserver on a different port like we do?
> If it has the functionality you desire and are looking for, sure.  My
> contention is that the average site isn't going to do it for a couple
> of features and would roll their own instead.

Isn't this the biggest reason people give for using MySQL and not
PostgreSQL???? (re: transaction support, triggers, etc).

--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11


Re: Bug tracking (was Re: +/- Inf for float8's)

From
The Hermit Hacker
Date:
On Mon, 21 Aug 2000, Vince Vielhaber wrote:

> > why not just run aolserver on a different port like we do?
> 
> If it has the functionality you desire and are looking for, sure.  My
> contention is that the average site isn't going to do it for a couple
> of features and would roll their own instead.

Sounds like a big waste of time investment when few of us have much of it
to start with ...

I've talked to Ben and he's going to work on getting the OpenACS version
up and running ... once he has that going, he's going to talk to you
(Vince) about getting the look to match what our web site looks like
... once we have that in place, the next step will be to customize it so
that it does what *we* want it to do ... 




Re: Re: [GENERAL] +/- Inf for float8's

From
Tom Lane
Date:
"Ross J. Reedstrom" <reedstrm@rice.edu> writes:
>>>> or add another ordering operator that is only used for the sort case
>>>> (use of '>' and '<' seems to be hardcoded all the way to the parser)
>> 
>> don't even think about that...

> Sure, but any ideas _how_ to get 'NaN > +Infinity' to happen only during a
> sort operation, and not when '>' is called explicity as a WHERE condition,
> by touching only type specific code?

That's exactly what you shouldn't even think about.  The entire index
and sorting system is predicated on the assumption that '<' and related
operators agree with the order induced by a btree index.  You do not get
to make the operators behave differently in the free-standing case than
when they are used with an index.

> 'Cause I'd call it a bug to be able to say:
> SELECT * from foo where f8 > 'Infinity';
> and get anything at all back.

I agree it's pretty arbitrary to define NaN as > Infinity, but the sort
ordering is necessarily arbitrary.  We can special-case NULL because
that's a type-independent concept, but special-casing NaN is out of the
question IMHO.  Pick where you want it in the type-specific order, and
live with it.
        regards, tom lane


Re: Re: [GENERAL] +/- Inf for float8's

From
"Ross J. Reedstrom"
Date:
On Mon, Aug 21, 2000 at 02:32:08PM -0500, Ross J. Reedstrom wrote:
> On Mon, Aug 21, 2000 at 01:39:32PM -0400, Tom Lane wrote:
> > 
> > Seems to me the sort order should be
> > 
> >     -Infinity
> >     normal values
> >     +Infinity
> >     other types of NaN
> >     NULL
> > 
> > and the reverse in a descending sort.
> > 
> 
> NULL is taken care of by special casing in the sort code, as I already mentioned,
> and can be fixed immediately.
> 

Grr, I take this back. By the time comparetup_* see the tuples, we've no idea
which order we're sorting in, just a pointer to the appropriate sortop.

<whine mode> 
Why does every thing I touch in pgsql end up pulling in down into the
guts of the whole system? Even something that looks nicely factored
at first, like the type system? I guess this stuff is _hard_.
</whine mode>

Sigh, back to fixing up referential integrity violations in the DB
I'm finally upgrading from 6.5 to 7.0.X. (DBA life lesson number XX:
implementing RI in the backend _from the very start of a project_ is a
very good thing. Cleansing the data of cruft left from _not_ having RI in
the backend, is a bad thing. And those clever, recursive self-referencing
table structures for representing trees are a pain in the DBA to reload.)

Ross
-- 
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> 
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St.,  Houston, TX 77005


Re: How Do You Pronounce "PostgreSQL"?

From
"Henry B. Hotz"
Date:
At 5:16 PM -0300 8/21/00, The Hermit Hacker wrote:
>On Mon, 21 Aug 2000, Henry B. Hotz wrote:
>
> > At 7:48 PM -0300 8/20/00, The Hermit Hacker wrote:
> > >what's wrong wth "Post-Gres-QL"?
> > >
> > >I find it soooo simple to pronounce *shrug*
> > >
> > >On Sun, 20 Aug 2000, David Lloyd-Jones wrote:
> > >
> > > > I'm putting on my suits-type suit for just a moment.
> > > >
> > > > In order to Conquer The Universe(tm) why don't we just call it "PG"?
> > > >
> >
> > IMNSHO the "QL" are silent and you pronounce it Postgres.  I consider
> > the SQL query language to be merely a (major) feature added during
> > the evolution of the system.
>
>as stated in another email, it is not pronounced Postgres ... postgres was
>a whole different beast based on a completely different query language
>... if you refer to Postgres, you are,  IMHO, refering to the only
>Postgres 4.2 which was the grand-daddy to what its evolved into ...

;-)


Signature held pending an ISO 9000 compliant
signature design and approval process.
h.b.hotz@jpl.nasa.gov, or hbhotz@oxy.edu


Re: Bug tracking

From
Ned Lilly
Date:
Hi Joe,

We're looking at elements of the Tigris architecture for the site
that I referenced in an earlier email (so Don, we're not reinventing
the wheel).  Scarab has apparently been in development for awhile
now - the Tigris project is currently using Bugzilla until Scarab
itself is ready for prime time, then they intend to switch it over.

We've also spent some time looking at an app called BugRat (even
ported it to Postgres from MySQL)... it seems to offer a pretty good
mix of features as well.

The greatbridge.org site will likely get started with either BugRat
or Bugzilla, then transitition to Scarab when/if it's ready.  BTW,
the goal of the site (which I didn't really explain too well in my
earlier message) will be to provide a hosting infrastructure for
some related projects (like interfaces, or the apps and tools we're
listing at http://www.greatbridge.com/tools/toollist.php)... also,
all the software we develop internally will go up on that site.

So please don't interpret our interest as getting in the middle of
what the main Postgres project uses for bug tracking - although the
requirements outlined in the earlier messages was helpful just for
our own development of the greatbridge.org site.  If Ben and the
OpenACS gang are willing to put the time into a bug tracker for the
project, I think that couldn't help but be a good thing.

Regards,
Ned



Joe Brenner wrote:

> Well, just to throw another piece of information into the
> mix, there is a new bug-tracking system under development by
> the folks at collab.net.  They call it "scarab", and last I
> talked to them that they thought it would be ready for
> production use Real Soon Now:
>
>    http://scarab.tigris.org/
>
> I guess they regard this as a replacement for bugzilla.
>
> Some personal opinions:
>
> (1) I would actually like to see bugzilla fixed rather than
> replaced.  The collab.net guys are into java servlets
> because they're CS geeks who are down on perl.  Me, I'm a
> perl loyalist who thinks that Larry Wall is onto something
> -- mathematical elegance may not be the right standard to
> judge a computer language.
>
> (2) And maybe it'd be nice if the "religous wars" could be
> dropped in favor of pure objective technical decision
> making, but i don't think they can be: the social and the
> technical don't neatly split into two little piles.  (A case
> in point: the argument that using a mailing list for bug
> control is somehow "warmer" or "more human" than a bug
> database.
>



Re: Re: [GENERAL] +/- Inf for float8's

From
"Henry B. Hotz"
Date:
At 1:25 AM -0400 8/21/00, Tom Lane wrote:
>"Ross J. Reedstrom" <reedstrm@rice.edu> writes:
> > !     if (isnan(a))
> > !         PG_RETURN_INT32(1);
>
>Do not like this at all --- doesn't it make the result of btint4cmp(NaN,
>NaN) dependent on which argument chances to be first?  Seems to me that
>you must consider two NaNs to be equal, unless you want to subdivide
>the category of NaNs.

I'm pretty sure IEEE 754 says that NaN does not compare equal to 
anything, including themselves.  I also believe that Infinity isn't 
equal to itself either, it's just bigger than anything else except 
NaN (which isn't littler either).

Without having seen the start of this thread I think the biggest 
problem is that some of the results of compares depend on the mode 
that the FP hardware is put in.  IEEE specifies some modes, but not 
how you set the mode you want on the system you are actually running 
on.  For example I think comparing zero and -Infinity may return 
three possible results:  0 > -Infinity, 0 < -Infinity (because it was 
told to ignore the sign of Infinity), or an illegal value exception. 
Likewise signalling/non-signalling NaN's act different depending on 
mode settings.

We need to first figure out what floating point behavior we want to 
support.  Then we figure what mode settings provide that behavior 
with a minimum of overhead.  Then we need to figure out how to set 
those modes on all the platforms we support.  We will probably 
discover that not all required modes actually exist on all hardware 
platforms.  I know that 68000 and SPARC are pretty good, but PowerPC 
punted some stuff to exception handlers which may not be correct on 
all OS's.  I've heard that Java has some portability issues because 
Intel fudged some stuff in the newer hardware.

Does anyone feel like tracing down how to set the modes for all the 
different systems that we try to run on?  If there is interest then I 
might poke at a couple/three NetBSD ports and Solaris/SPARC.  But 
only if there is interest.

Sun has put some code out under GPL which will let you test for these 
special values and handle them, but that seems like a big hit for 
what should be a simple compare.  I assume that we can't put GPL code 
into the main sources any more than the *BSD's do.  Perhaps we could 
get away with it if it is only included if configure can't figure out 
how to set the modes properly?


Signature held pending an ISO 9000 compliant
signature design and approval process.
h.b.hotz@jpl.nasa.gov, or hbhotz@oxy.edu


Re: Re: [GENERAL] +/- Inf for float8's

From
"Ross J. Reedstrom"
Date:
On Mon, Aug 21, 2000 at 04:37:21PM -0400, Tom Lane wrote:
> "Ross J. Reedstrom" <reedstrm@rice.edu> writes:
> >>>> or add another ordering operator that is only used for the sort case
> >>>> (use of '>' and '<' seems to be hardcoded all the way to the parser)
> >> 
> >> don't even think about that...
> 
> > Sure, but any ideas _how_ to get 'NaN > +Infinity' to happen only during a
> > sort operation, and not when '>' is called explicity as a WHERE condition,
> > by touching only type specific code?
> 
> That's exactly what you shouldn't even think about.  The entire index
> and sorting system is predicated on the assumption that '<' and related
> operators agree with the order induced by a btree index.  You do not get
> to make the operators behave differently in the free-standing case than
> when they are used with an index.

Oh really? Then why do btree's have their own comparator functions,
seperate from heap sorts, and datum sorts, and explicit use of '<' ? The
current code infrastructure allows for the possibility that these may need
to diverge, requiring the coders to keep them in sync. Annoying, that, but
useful for edge cases.

Since btree already uses it's own comparator, The only reason I can
see that the parser drops in  '<' and '>' as the name of the sorting
operator to use for ORDER BY is convenience: the functions are there,
and have the (mostly) correct behavior. 

Changing this would only require writing another set of operators for
the parser to drop in, that are used only for sorting, so that the
sort behavior could diverge slightly, by knowing how to sort NULLs and
NaNs. Yes, it'd be a third set of operators to keep in sync with the
btree and default ones, but it could give completely correct behavior.

Hmm, I another thought: all the comparator code assumes (a<b || a>b || a==c)
and therefor only test 2 of the three conditions, falling through to the 
third. In the three places I just looked, two fall thorough to the equal case,
and one to the 'less than' case. If all three fell through to the 'greater than'
case, it might work with no tweaking at all. I'll have to try that, first.

Ross
-- 
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> 
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St.,  Houston, TX 77005


Re: Re: [GENERAL] +/- Inf for float8's

From
"Ross J. Reedstrom"
Date:
On Mon, Aug 21, 2000 at 05:30:21PM -0500, Ross J. Reedstrom wrote:
> 
> Hmm, I another thought: all the comparator code assumes (a<b || a>b || a==c)
> and therefor only test 2 of the three conditions, falling through to the 
> third. In the three places I just looked, two fall thorough to the equal case,
> and one to the 'less than' case. If all three fell through to the 'greater than'
> case, it might work with no tweaking at all. I'll have to try that, first.

Looking again, I realize that the sort comparetup_* code doesn't have
access to a operator to test for equality, so can't do this. Sigh. Time
to go home, I think.

Ross
-- 
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> 
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St.,  Houston, TX 77005



Re: Re: [GENERAL] +/- Inf for float8's

From
Tom Lane
Date:
"Ross J. Reedstrom" <reedstrm@rice.edu> writes:
> On Mon, Aug 21, 2000 at 04:37:21PM -0400, Tom Lane wrote:
>> That's exactly what you shouldn't even think about.  The entire index
>> and sorting system is predicated on the assumption that '<' and related
>> operators agree with the order induced by a btree index.  You do not get
>> to make the operators behave differently in the free-standing case than
>> when they are used with an index.

> Oh really? Then why do btree's have their own comparator functions,
> seperate from heap sorts, and datum sorts, and explicit use of '<' ?

Strictly and only to save a few function-call cycles.  Some paths in the
btree code need a three-way comparison (is A<B, or A=B, or A>B?) and
about half the time you'd need two calls to type-specific comparator
functions to make that determination if you only had the user-level
operators available.  This does *not* mean that you have license to make
the 3-way comparator's behavior differ from the operators, because the
operators are used too.  Note also that it is a three-way comparison
function, not four-way: there is no provision for answering "none of the
above" (except when a NULL is involved, and that only works because it's
special-cased without calling type-specific code at all).

The reason the sort code doesn't use the comparator routine is strictly
historical, AFAICT.  It really should, for speed reasons; but there may
not be a 3-way comparator associated with a given '<' operator, and
we've got a longstanding convention that a user-selected sort order is
specified by naming a particular '<'-like operator.  It may also be
worth pointing out that the sort code still assumes trichotomy: it
tests A<B, and if that is false it tries B<A, and if that's also false
then it assumes A=B.  There's still no room for an "unordered" response.

> The current code infrastructure allows for the possibility that these
> may need to diverge, requiring the coders to keep them in
> sync. Annoying, that, but useful for edge cases.

It is annoying.  Many of the datatypes where comparison is nontrivial
actually use an underlying 3-way comparison routine that the boolean
comparators call, so as to avoid code-divergence problems.

> Changing this would only require writing another set of operators for
> the parser to drop in, that are used only for sorting,

No, because *the user-level operators must match the index*.  How many
times do I have to repeat that?  The transformation that allows, say,SELECT * FROM tab WHERE foo > 33 AND foo < 42
to be implemented by an indexscan (of an index on foo) is fundamentally
dependent on the assumption that the operators '>' and '<' induce the
same ordering of data values as is stored in the index.  Otherwise you
can't scan a subrange of the index and know that you've hit all the
matching rows.  The planner actually takes considerable care to verify
that the operators appearing in WHERE *do* match the index ordering ---
that's what pg_opclass and pg_amop are all about.  If you invent an
internal set of operators that provide a different index ordering,
you will find that the planner ignores your index.
        regards, tom lane


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Adam Haberlach
Date:
On Mon, Aug 21, 2000 at 11:34:45AM -0700, Don Baccus wrote:

> Strange, OpenACS folk use it regularly and all we've done is put a "report
> a bug" link on the home page.
...I'm not sure you noticed, but this project isn't OpenACS.  It is
an established project with decent management that only needs a few features.
Switching everything over to a canned solution, no matter how good of a
toolbox you feel it is, is not necessarily going to solve the few problems
we have without causing a whole host of new ones...

> I haven't heard so many arguments against change since the VT100 started
> replacing the KSR 35!
Oh, and in case you didn't hear it earlier... PLONK.

-- 
Adam Haberlach             | "A farm tractor is not a motorcycle."
adam@newsnipple.com        |   --California DMV 1999
http://www.newsnipple.com/ |        Motorcycle Driver Handbook


Re: How Do You Pronounce "PostgreSQL"?

From
Jan Wieck
Date:
The Hermit Hacker wrote:
>
> what's wrong wth "Post-Gres-QL"?
>
> I find it soooo simple to pronounce *shrug*
   Mee  too.  And I'm not sure if anybody pronounces PG the same   way.  Is it PeeGee or PiG (in which case we'd have
the wrong   animal in our logo).
 


Jan

>
> On Sun, 20 Aug 2000, David Lloyd-Jones wrote:
>
> > I'm putting on my suits-type suit for just a moment.
> >
> > In order to Conquer The Universe(tm) why don't we just call it "PG"?
> >
> >                                                       -dlj.
> >
> >
> >
>
> Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
> Systems Administrator @ hub.org
> primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org
>


--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #




Re: Bug tracking (was Re: +/- Inf for float8's)

From
The Hermit Hacker
Date:
On Mon, 21 Aug 2000, Adam Haberlach wrote:

> On Mon, Aug 21, 2000 at 11:34:45AM -0700, Don Baccus wrote:
> 
> > Strange, OpenACS folk use it regularly and all we've done is put a "report
> > a bug" link on the home page.
> 
>     ...I'm not sure you noticed, but this project isn't OpenACS.  It is
> an established project with decent management that only needs a few features.
> Switching everything over to a canned solution, no matter how good of a
> toolbox you feel it is, is not necessarily going to solve the few problems
> we have without causing a whole host of new ones...

Ummmm, who was talking about switching anything over to a canned
solution? *raised eyebrow*  we are talking about allowing the OpenACS
folks setup a bug tracking system for us and seeing if it servces us
better then other attempts have in the past ... 




Re: Bug tracking (was Re: +/- Inf for float8's)

From
Don Baccus
Date:
At 09:40 PM 8/21/00 -0300, The Hermit Hacker wrote:
>On Mon, 21 Aug 2000, Adam Haberlach wrote:
>
>> On Mon, Aug 21, 2000 at 11:34:45AM -0700, Don Baccus wrote:
>> 
>> > Strange, OpenACS folk use it regularly and all we've done is put a
"report
>> > a bug" link on the home page.
>> 
>>     ...I'm not sure you noticed, but this project isn't OpenACS.  It is
>> an established project with decent management that only needs a few
features.
>> Switching everything over to a canned solution, no matter how good of a
>> toolbox you feel it is, is not necessarily going to solve the few problems
>> we have without causing a whole host of new ones...
>
>Ummmm, who was talking about switching anything over to a canned
>solution? *raised eyebrow*  we are talking about allowing the OpenACS
>folks setup a bug tracking system for us and seeing if it servces us
>better then other attempts have in the past ... 

Which it probably won't - customization is the key, we're not quite as
stupid as Adam makes us out to be.

Here's an idea:

How about ignoring whether or not a new solution is borne, or the SDM
can be customized to fit your needs with less work than, and concentrating
on what features you want to see?

Has anyone thought about this, or was the last attempt such a failure that
people have given it no further thought?



- Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert
Serviceand other goodies at http://donb.photo.net.
 


Re: Bug tracking (was Re: +/- Inf for float8's)

From
Mike Mascari
Date:
Don Baccus wrote:
>
> How about ignoring whether or not a new solution is borne, or the SDM
> can be customized to fit your needs with less work than, and concentrating
> on what features you want to see?
> 
> Has anyone thought about this, or was the last attempt such a failure that
> people have given it no further thought?
> 

I don't know what the major code contributors need beyond the
TODO list. But I remember what went wrong with the older system
-- people would post non-bug issues, and in large numbers, as
bugs. And the system would "pend" those non-issues, assigning
them to core developers, who, at the time, were very busy
implementing MVCC and crushing real bugs by the hundreds. It
seems all Peter wants is a system whereby authorized users
(presumably those with CVS privileges) would have the ability to
post and close bugs. Perhaps such a system might have prevented
the duplicate work done recently on the "binary compatibility WRT
functional indexes" issue. Just from lurking, I think the core
developers' consensus was that anything which allows Joe User to
open tickets, without a "front-line" of advanced users/minor code
contributors willing to act as filters, would consume too much
time. People with great frequency ignore the note on the web-site
which reads "Note: You must post elsewhere first" with respect to
the pgsql-hackers list.

So I don't think it was an issue with the technology, but the
process. Although, from what I've read, I suspect ArsDigita is
light-years ahead of the Keystone software that was the
"PostgreSQL Bug Tracking System".

P.S.: I've been looking forward to seeing ArsDigita running on
postgresql.org for some time. I suspect there would be some
short-term pain, but substantial long-term gain. :-)

Mike Mascari


Re: Re: [GENERAL] +/- Inf for float8's

From
"Ross J. Reedstrom"
Date:
On Tue, Aug 22, 2000 at 02:16:44PM +0200, Peter Eisentraut wrote:
> Ross J. Reedstrom writes:
> 
> > Fixing sorts is a bit tricker, but can be done: Currently, I've hacked
> > the float8lt and float8gt code to sort NaN to after +/-Infinity. (since
> > NULLs are special cased, they end up sorting after NaN). I don't see
> > any problems with this solution, and it give the desired behavior.
> 
> SQL 99, part 5, section 17.2 specifies that the sort order for ASC and
> DESC is defined in terms of the particular type's < and > operators.
> Therefore the NaN's must always be at the end. (Before or after NULL is
> implementation-defined, btw.)


I'm not sure what your suggesting, Peter.  Which is 'the end'? And how does
'Therefore' follow from considering the type behavior of NaN and the < and
> operators ? 

I think your suggesting that NaN always sort to one end, either greater
than Infinity or less than -Infinity, regardless of sort direction. 
Therefore, depending on the direction of ORDER BY, NaNs will be returned
either be first or last, not always last, as I've currently implemented.

I agree with this, but my reason comes from the required treatment of NULLs.  

My reasoning is as follows:

The standard says (17.2):
    The relative position of rows X and Y in the result is determined by    comparing XV(i) and YV(i) according to the
rulesof Subclause 8.2,    "<comparison predicate>", in ISO/IEC 9075-2, where the <comp op>    is the applicable <comp
op>for K(i), [...]
 

and Subclause 8.2 says:
        2) Numbers are compared with respect to their algebraic value.

However, NaN is _not_ algebraically > or < any other number: in fact,
General Rule 1. of subclause 8.2 does deal with this:
            5) X <comp op> Y is_unknown if X <comp op> Y is neither               true_ nor false_ .

So, we're left with not knowing where to put NaN.

However, the only other case where the comparision is unknown is:
           a) If either XV or YV is the null value, then                 X <comp op> Y is unknown_ .

And, going back to section 17.2:
    [...] where the <comp op> is the applicable <comp op> for K(i),    with the following special treatment of null
values.Whether a    sort key value that is null is considered greater or less than a    non-null value is
implementation-defined,but all sort key values    that are null shall either be considered greater than all non-null
valuesor be considered less than all non-null values.
 

So, NULLs go at one end (less or greater), always, so NaN should as well.
And NULL will go outside them, since NULLs are required to be considered
greater than (in our case) all non-null values (including NaN).

Ross
-- 
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> 
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St.,  Houston, TX 77005

> Ross J. Reedstrom writes:
> 
> > Fixing sorts is a bit tricker, but can be done: Currently, I've hacked
> > the float8lt and float8gt code to sort NaN to after +/-Infinity. (since
> > NULLs are special cased, they end up sorting after NaN). I don't see
> > any problems with this solution, and it give the desired behavior.
> 
> SQL 99, part 5, section 17.2 specifies that the sort order for ASC and
> DESC is defined in terms of the particular type's < and > operators.
> Therefore the NaN's must always be at the end. (Before or after NULL is
> implementation-defined, btw.)
> 
> 
> -- 
> Peter Eisentraut                  Sernanders väg 10:115
> peter_e@gmx.net                   75262 Uppsala
> http://yi.org/peter-e/            Sweden
> 


Re: Re: [GENERAL] +/- Inf for float8's

From
"Ross J. Reedstrom"
Date:
On Tue, Aug 22, 2000 at 08:22:21PM +0200, Peter Eisentraut wrote:
> 
> Hmm, I'm getting the feeling that perhaps at this point we should
> explicitly *not* support NaN at all. After all, the underlying reason for
> offering them is to provide IEEE 754 compliant floating point arithmetic,
> but if we start making compromises such as NaN == NaN or NaN > +Infinity
> then we might as well not do it. In these cases I opine that if you can't
> do something correctly then you should perhaps be honest and don't do
> it. After all, users that want a "not-a-number" can use NULL in most
> cases, and hard-core floating point users are going to fail miserably
> with the FE/BE protocol anyway.
> 

Pretty much were I have come to on this, as well. The point is to get
the existing NaN to not break indicies or sorting. The simplest way is
to disable it.

Ross
-- 
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> 
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St.,  Houston, TX 77005


Re: Re: [GENERAL] +/- Inf for float8's

From
Tom Lane
Date:
> On Tue, Aug 22, 2000 at 08:22:21PM +0200, Peter Eisentraut wrote:
>> Hmm, I'm getting the feeling that perhaps at this point we should
>> explicitly *not* support NaN at all.

Well ... this is a database, not a number-crunching system.  It seems
to me that we should be able to store and retrieve NaNs (at least on
IEEE-compliant platforms).  But I'm less excited about whether the
sorting/comparison operators we offer are 100% IEEE-compliant.

It has been quite a few years since I looked closely at the IEEE FP
specs, but I do still recall that they made a distinction between "IEEE
aware" and "non IEEE aware" comparison operators --- specifically, the
first kind understood about unordered comparisons and the second didn't.
Perhaps we could satisfy both SQL and IEEE requirements if we stipulate
that we implement only IEEE's "non-aware" comparisons?  Worth looking at
anyway.

>> ... hard-core floating point users are going to fail miserably
>> with the FE/BE protocol anyway.

It would be a mistake to design backend behavior on the assumption that
we'll never have an FE/BE protocol better than the one we have today.

(You could actually fix this problem without any protocol change,
just a SET variable to determine output precision for FP values.
Well-written platforms will reproduce floats exactly given "%.17g"
or more precision demand in sprintf.  If that fails it's libc's
bug not ours.)
        regards, tom lane


Re: Re: [GENERAL] +/- Inf for float8's

From
Bruce Momjian
Date:
My assumption is that we never came up with any solution to this, right?


> On Sun, Aug 20, 2000 at 12:33:00AM +0200, Peter Eisentraut wrote:
> <snip side comment about bug tracking. My input: for an email controllable
> system, take a look at the debian bug tracking system>
> 
> > Show me a system where it doesn't work and we'll get it to work.
> > UNSAFE_FLOATS as it stands it probably not the most appropriate behaviour;
> > it intends to speed things up, not make things portable.
> > 
> 
> I agree. In the previous thread on this, Thomas suggested creating a flag
> that would allow control turning the  CheckFloat8Val function calls into
> a macro NOOP. Sound slike a plan to me.
> 
> > 
> > > > NULL and NaN are not quite the same thing imho. If we are allowing NaN
> > > > in columns, then it is *known* to be NaN.
> > > 
> > > For the purposes of ordering, however, they are very similar.
> > 
> > Then we can also treat them similar, i.e. sort them all last or all first.
> > If you have NaN's in your data you wouldn't be interested in ordering
> > anyway.
> 
> Right, but the problem is that NULLs are an SQL language feature, and
> there for rightly special cased directly in the sorting apparatus. NaN is
> type specific, and I'd be loath to special case it in the same place. As
> it happens, I've spent some time this weekend groveling through the sort
> (and index, as it happens) code, and have an idea for a type specific fix.
> 
> Here's the deal, and an actual, honest to goodness bug in the current code.
> 
> As it stands, we allow one non-finite to be stored in a float8 field:
> NaN, with partial parsing of 'Infinity'.
> 
> As I reported last week, NaNs break sorts: they act as barriers, creating
> sorted subsections in the output.  As those familiar with the code have
> already guessed, there is a more serious bug: NaNs break indicies on
> float8 fields, essentially chopping the index off at the first NaN.
> 
> Fixing this turns out to be a one liner to btfloat8cmp.
> 
> Fixing sorts is a bit tricker, but can be done: Currently, I've hacked
> the float8lt and float8gt code to sort NaN to after +/-Infinity. (since
> NULLs are special cased, they end up sorting after NaN). I don't see
> any problems with this solution, and it give the desired behavior.
> 
> I've attached a patch which fixes all the sort and index problems, as well
> as adding input support for -Infinity. This is not a complete solution,
> since I haven't done anything with the CheckFloat8Val test. On my
> system (linux/glibc2.1) compiling with UNSAFE_FLOATS seems to work fine 
> for testing.
> 
> > 
> > Side note 2: The paper "How Java's floating point hurts everyone
> > everywhere" provides for good context reading.
> 
> http://http/cs.berkeley.edu/~wkahan/JAVAhurt.pdf ? I'll take a look at it
> when I get in to work Monday.
> 
> > 
> > Side note 3: Once you read that paper you will agree that using floating
> > point with Postgres is completely insane as long as the FE/BE protocol is
> > text-based.
> 
> Probably. But it's not our job to enforce sanity, right? Another way to think
> about it is fixing the implementation so the deficiencies of the FE/BE stand
> out in a clearer light. ;-)
> 
> Ross
> -- 
> Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> 
> NSBRI Research Scientist/Programmer
> Computer and Information Technology Institute
> Rice University, 6100 S. Main St.,  Houston, TX 77005
> 

[ Attachment, skipping... ]


--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: Re: [GENERAL] +/- Inf for float8's

From
Peter Eisentraut
Date:
Bruce Momjian writes:

> My assumption is that we never came up with any solution to this, right?

It stopped when we noticed that proper support for non-finite values will
break indexing, because the relational trichotomy doesn't hold.

-- 
Peter Eisentraut      peter_e@gmx.net       http://yi.org/peter-e/



Re: Re: [GENERAL] +/- Inf for float8's

From
Tom Lane
Date:
[ continuing a discussion from last August ]

Peter Eisentraut <peter_e@gmx.net> writes:
> Bruce Momjian writes:
>> My assumption is that we never came up with any solution to this, right?

> It stopped when we noticed that proper support for non-finite values will
> break indexing, because the relational trichotomy doesn't hold.

I believe that's not a problem anymore.  The current form of the float
comparison functions will perform sorting and comparisons according to
the sequence
-infinity < normal values < infinity < NaN < NULL

with all NaNs treated as equal.  This may not be exactly what an IEEE
purist would like, but given that we have to define *some* consistent
sort order, it seems as reasonable as we can get.

Accordingly, I suggest that Ross go back to work on persuading the code
to treat infinities and NaNs properly in other respects.  IIRC, there
are still open issues concerning whether we still need/want
CheckFloat8Val/CheckFloat4Val, what the I/O conversion functions should
do on non-IEEE machines, etc.  They all seemed soluble, though.
        regards, tom lane


Re: Re: [GENERAL] +/- Inf for float8's

From
Peter Eisentraut
Date:
Tom Lane writes:

> [ continuing a discussion from last August ]
[I was *just* thinking about this.  Funny.]

> I believe that's not a problem anymore.  The current form of the float
> comparison functions will perform sorting and comparisons according to
> the sequence
>
>     -infinity < normal values < infinity < NaN < NULL

I was thinking about making NaN equivalent to NULL.  That would give
consistency in ordering, and probably also in arithmetic.  Additionally,
if the platform supports it we ought to make the Invalid Operation FP
exception (which yields NaN) configurable:  either get NULL or get an
error.

-- 
Peter Eisentraut   peter_e@gmx.net   http://funkturm.homeip.net/~peter



Re: Re: [GENERAL] +/- Inf for float8's

From
Tom Lane
Date:
Peter Eisentraut <peter_e@gmx.net> writes:
> I was thinking about making NaN equivalent to NULL.

Mumble ... in the thread last August, someone made the point that SQL's
idea of NULL ("unknown value") is not really the same as a NaN ("I know
that this is not a well-defined number").  Even though there's a lot of
similarity in the behaviors, I'd be inclined to preserve that semantic
distinction.

If we did want to do this, the implication would be that all
float-returning functions would be required to make sure they were not
returning NaNs:if (isnan(x))    PG_RETURN_NULL();else    PG_RETURN_FLOAT8(x);
Possibly this logic could be folded right into the PG_RETURN_FLOAT
macros.

> if the platform supports it we ought to make the Invalid Operation FP
> exception (which yields NaN) configurable:  either get NULL or get an
> error.

Seems like we could equally well offer the switch as "either get NaN
or get an error".

Something to be kept in mind here is the likelihood of divergence in
our behavior between IEEE and non-IEEE platforms.  I don't object to
that --- it's sort of the point --- but we should be aware of how much
difference we're creating, and try to avoid unnecessary differences.
Hmm ... I suppose an attraction of a NULL-vs-error, as opposed to NaN-
vs-error, option is that it could theoretically be supported on NaN-less
hardware.  But is that realizable in practice?  SIGFPE is messy.
        regards, tom lane


Re: Re: [GENERAL] +/- Inf for float8's

From
Thomas Lockhart
Date:
> >       -infinity < normal values < infinity < NaN < NULL
> I was thinking about making NaN equivalent to NULL.  That would give
> consistency in ordering, and probably also in arithmetic.  Additionally,
> if the platform supports it we ought to make the Invalid Operation FP
> exception (which yields NaN) configurable:  either get NULL or get an
> error.

I'd like to see the distinction between NaN and NULL retained, since the
two "values" arise from different circumstances and under different
conditions. If a particular app needs them to be equivalent, then that
is easy enough to do with SQL or triggers.
                      - Thomas

On a modestly related note, I'm come over to the notion that the
date/time value 'current' could be ripped out eventually. Tom, isn't
that the only case for those types which bolluxes up caching of
date/time types?


Re: Re: [GENERAL] +/- Inf for float8's

From
Tom Lane
Date:
Thomas Lockhart <lockhart@fourpalms.org> writes:
> On a modestly related note, I'm come over to the notion that the
> date/time value 'current' could be ripped out eventually. Tom, isn't
> that the only case for those types which bolluxes up caching of
> date/time types?

Yes, I believe so.  At least, that was the consideration that led me
to mark those functions noncachable ...
        regards, tom lane