Re: Same double precision operations, different results - Mailing list pgsql-hackers

<div dir="ltr">Thx Tom!!<div style="style"><br /></div><div style="style">Yep, I focused on a<span
style="color:rgb(0,0,0);font-family:verdana,sans-serif;font-size:12px;line-height:18.234375px">lltrigonometric
functionstake arguments and return values of type </span><tt class=""
style="font-size:12px;color:rgb(0,0,0);line-height:18.234375px">doubleprecision</tt><span
style="color:rgb(0,0,0);font-family:verdana,sans-serif;font-size:12px;line-height:18.234375px">.Looking for the error I
lostfocus on numeric values directly on the select and diff inner calculations. </span></div><div style="style"><span
style="color:rgb(0,0,0);font-family:verdana,sans-serif;font-size:12px;line-height:18.234375px"><br/></span></div><div
style="style"><spanstyle="color:rgb(0,0,0);font-family:verdana,sans-serif;font-size:12px;line-height:18.234375px">I
thinkbest approach will be maintain double precision on trigonometric calculations for faster and cast to numeric
beforeacos operation.</span></div><div style="style"><span
style="color:rgb(0,0,0);font-family:verdana,sans-serif;font-size:12px;line-height:18.234375px"><br/></span></div><div
style="style"><spanstyle="color:rgb(0,0,0);font-family:verdana,sans-serif;font-size:12px;line-height:18.234375px">Thank
youman!</span></div><div style="style"><span
style="color:rgb(0,0,0);font-family:verdana,sans-serif;font-size:12px;line-height:18.234375px"><br/></span></div><div
style="style"><spanstyle="color:rgb(0,0,0);font-family:verdana,sans-serif;font-size:12px;line-height:18.234375px"><br
/></span></div></div><divclass="gmail_extra"><br /><br /><div class="gmail_quote">2014-02-13 18:26 GMT+01:00 Tom Lane
<spandir="ltr"><<a href="mailto:tgl@sss.pgh.pa.us" target="_blank">tgl@sss.pgh.pa.us</a>></span>:<br
/><blockquoteclass="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div
class="">DanielVázquez <<a href="mailto:daniel2d2art@gmail.com">daniel2d2art@gmail.com</a>> writes:<br /> >
Pleasecan someone tell me why this behavior? Thx!!<br /><br /></div>You're confusing numeric and double precision; in
particularthis<br /> calculation is *not* being done in float8, but numeric:<br /><div class=""><br /> > # select
(0.766238989559398* 0.766238989559398 * 1 + 0.642555686986733 *<br /> > 0.642555686986733) calc;<br /><br
/></div><divclass="">> (*) Why this calculation produces 1 and not 0.999999999999999633651488135693<br /><br
/></div>Actually,it's not producing 1, but a smidgen more:<br /><br /> regression=# set extra_float_digits TO 3;<br />
SET<br/> regression=# select ( cast (<br /><div class="">   cos(radians(39.9826557))<br />    *
cos(radians(39.9826557))<br/>    * cos(radians(-0.04773120000004383) - radians(-0.04773120000004383))<br />    +
sin(radians(39.9826557))<br/>    * sin(radians(39.9826557)) as double precision )<br />  );<br /></div>       float8<br
/>---------------------<br />  1.00000000000000022<br /> (1 row)<br /><br /> You've got roundoff error either way, but
thisway happens to be in the<br /> direction that makes acos() complain.<br /><br />                         regards,
tomlane<br /></blockquote></div><br /></div>