Thread: Just comments about math

Just comments about math

From
Edipo Elder Fernandes de Melo
Date:
Hi all, 
   Why the results below are different? (On my casio scientific calculator, 
the two expressions below are equals to 381.4) 

=# select (3+0.2*707/80)*80; ?column? 
----------    381.4 

=# SELECT 80*(3+(707/80)*0.2); ?column? 
----------      368 
   Answer: "select 707/80;" returns 8, while "select 707/80::float" returns 
8.8375. 
   Abracos, 
   Edipo Elder    [edipoelder@ig.com.br] 

_________________________________________________________
Oi! Voc� quer um iG-mail gratuito?
Ent�o clique aqui: http://registro.ig.com.br/



Re: Just comments about math

From
"Andrew G. Hammond"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2001 December 7 12:34 pm, Edipo Elder Fernandes de Melo wrote:

> =# select (3+0.2*707/80)*80;
>  ?column?
> ----------
>     381.4
>
> =# SELECT 80*(3+(707/80)*0.2);
>  ?column?
> ----------
>       368
>
>     Answer: "select 707/80;" returns 8, while "select 707/80::float"
> returns 8.8375.

SELECT 707.0/80;  --> 8.8375
 In your first expression, you change type to floating point in your first 
operation, but in your second expression  you don't change type until the 
second to last operation.  Integer division always returns an integer result, 
even when it's not the most mathematically intuitive way to do things.

- -- 
Andrew G. Hammond     mailto:drew@xyzzy.dhs.org   http://xyzzy.dhs.org/~drew/
56 2A 54 EF 19 C0 3B 43 72 69 5B E3 69 5B A1 1F                  613-389-5481
5CD3 62B0 254B DEB1 86E0  8959 093E F70A B457 84B1
"To blow recursion you must first blow recur" -- me
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iEYEARECAAYFAjwU0OcACgkQCT73CrRXhLFZBwCgjBrPcSCzGKkf91KEstX4F68G
SbcAniakO7mr044HoJtegKn8vdX/gWfE
=PlV9
-----END PGP SIGNATURE-----