Re: Rounding problems - Mailing list pgsql-general
From | Justin |
---|---|
Subject | Re: Rounding problems |
Date | |
Msg-id | 49FDAD45.4070104@emproshunts.com Whole thread Raw |
In response to | Rounding problems ("Paolo Saudin" <paolo@ecometer.it>) |
Responses |
R: Rounding problems
|
List | pgsql-general |
<br /><br /> Paolo Saudin wrote: <blockquote cite="mid:002801c9cbeb$f6b59c20$e420d460$@it" type="cite"><style> <!--/* Font Definitions */@font-face{font-family:Wingdings;panose-1:5 0 0 0 0 0 0 0 0 0;} @font-face{font-family:"Cambria Math";panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face{font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;}/* Style Definitions */p.MsoNormal, li.MsoNormal, div.MsoNormal{margin:0cm;margin-bottom:.0001pt;font-size:11.0pt;font-family:"Calibri","sans-serif";} a:link, span.MsoHyperlink{mso-style-priority:99;color:blue;text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed{mso-style-priority:99;color:purple;text-decoration:underline;} p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph{mso-style-priority:34;margin-top:0cm;margin-right:0cm;margin-bottom:0cm;margin-left:36.0pt;margin-bottom:.0001pt;font-size:11.0pt;font-family:"Calibri","sans-serif";} span.StileMessaggioDiPostaElettronica17{mso-style-type:personal-compose;font-family:"Calibri","sans-serif";color:windowtext;} .MsoChpDefault{mso-style-type:export-only;} @page Section1{size:612.0pt 792.0pt;margin:70.85pt 2.0cm 2.0cm 2.0cm;} div.Section1{page:Section1;}/* List Definitions */@list l0{mso-list-id:361589963;mso-list-type:hybrid;mso-list-template-ids:-1476211862-1560001026 68157443 68157445 68157441 6815744368157445 68157441 68157443 68157445;} @list l0:level1{mso-level-start-at:0;mso-level-number-format:bullet;mso-level-text:\F06E;mso-level-tab-stop:none;mso-level-number-position:left;text-indent:-18.0pt;font-family:Wingdings;mso-fareast-font-family:Calibri;mso-bidi-font-family:"Times NewRoman";} @list l1{mso-list-id:892347903;mso-list-type:hybrid;mso-list-template-ids:686571502 512508354 68157443 68157445 6815744168157443 68157445 68157441 68157443 68157445;} @list l1:level1{mso-level-start-at:0;mso-level-number-format:bullet;mso-level-text:\F06E;mso-level-tab-stop:none;mso-level-number-position:left;text-indent:-18.0pt;font-family:Wingdings;mso-fareast-font-family:Calibri;mso-bidi-font-family:"Times NewRoman";} @list l2{mso-list-id:989483843;mso-list-type:hybrid;mso-list-template-ids:-722197268 311070654 68157443 68157445 6815744168157443 68157445 68157441 68157443 68157445;} @list l2:level1{mso-level-start-at:0;mso-level-number-format:bullet;mso-level-text:\F06E;mso-level-tab-stop:none;mso-level-number-position:left;text-indent:-18.0pt;font-family:Wingdings;mso-fareast-font-family:Calibri;mso-bidi-font-family:"Times NewRoman";} ol{margin-bottom:0cm;} ul{margin-bottom:0cm;} --> </style><div class="Section1"><p class="MsoNormal">Hi,<p class="MsoNormal"> <p class="MsoNormal">I have a problem witha query wich simple aggregate values. In the sample below I have two values, 1.3 and 1.4. Rounding their average withone decimals, should give 1.4.<p class="MsoNormal">The first query with - cast( tables_seb.tbl_arvier_chamencon.id_1 AS numeric) AS value - give the expected result, while the second one with - tables_seb.tbl_arvier_chamencon.id_1AS value - give 1.3. <p class="MsoNormal"> <p class="MsoNormal">Which could be the reason??<p class="MsoNormal"> <br /></div></blockquote> My first thought is whats with all the castings??? <br /><br />Castings are mostly likely the cause of your problems,<b> </b>What is <b>tbl_arvier_chamencon.id_1 </b>data type???<br/><br /> I'm guessing its something other than numeric. All other floating point data types will have problemscaused by Binary Floating-Point Arithmetic<br /><br /> Numeric data type uses different functions to do its mathfor the stated purpose of being exact yet being allot slower. <br /><br /> In one query casting is done prior to avg()yet in the other casting is done after avg(). This will allow Postgres to use different functions to calculate averagegiving an unexpected result.<br />
pgsql-general by date: