Thread: Precision problems with storage of float/double
Hi, I have found that floats and doubles that I save to the database with the ODBC driver may lose precision. The exact bit patternof my double/float is not what is actually saved in the database. This causes difficulties in situations where youneed to compare current record contents with the actual data that was inserted. It appears to be related to conversion from the double/float to text before it is sent to the database. I made a simple patch that appears to fix the issue (by adding more decimal digits in the string conversion), and a testcase that tests the issue.
Attachment
(2014/06/16 1:51), John Smith wrote: > Hi, > > I have found that floats and doubles that I save to the database with the ODBC driver may lose precision. The exact bitpattern of my double/float is not what is actually saved in the database. This causes difficulties in situations whereyou need to compare current record contents with the actual data that was inserted. > > It appears to be related to conversion from the double/float to text before it is sent to the database. > > I made a simple patch that appears to fix the issue (by adding more decimal digits in the string conversion), and a testcase that tests the issue. Thanks. As far as I examined, 9 decimal digits for 32-bit reals and 17 decimal digits for 64-bit doubles are sufficient to guarantee the complete recovery. Is it OK to change decimal digits in your patch to those ones? Hmm I noticed that server's output should be changed in addition to this change. The setting of extra_float_digits should be changed from 2 to 3. regards, Hiroshi Inoue -- I am using the free version of SPAMfighter. SPAMfighter has removed 10915 of my spam emails to date. Get the free SPAMfighter here: http://www.spamfighter.com/len Do you have a slow PC? Try a Free scan http://www.spamfighter.com/SLOW-PCfighter?cid=sigen
Hi Hiroshi I googled this a little, and I think you are right. 17 and 9 should be enough. I don't have any problems with you changingthe patch. ---------------------------------------- > Date: Tue, 17 Jun 2014 12:24:51 +0900 > From: inoue@tpf.co.jp > To: nukemd@hotmail.com > CC: pgsql-odbc@postgresql.org > Subject: Re: [ODBC] Precision problems with storage of float/double > > (2014/06/16 1:51), John Smith wrote: >> Hi, >> >> I have found that floats and doubles that I save to the database with the ODBC driver may lose precision. The exact bitpattern of my double/float is not what is actually saved in the database. This causes difficulties in situations whereyou need to compare current record contents with the actual data that was inserted. >> >> It appears to be related to conversion from the double/float to text before it is sent to the database. >> >> I made a simple patch that appears to fix the issue (by adding more decimal digits in the string conversion), and a testcase that tests the issue. > > Thanks. > > As far as I examined, 9 decimal digits for 32-bit reals and 17 decimal > digits for 64-bit doubles are sufficient to guarantee the complete > recovery. > Is it OK to change decimal digits in your patch to those ones? > > Hmm I noticed that server's output should be changed in addition > to this change. The setting of extra_float_digits should be changed > from 2 to 3. > > regards, > Hiroshi Inoue > > > > -- > I am using the free version of SPAMfighter. > SPAMfighter has removed 10915 of my spam emails to date. > Get the free SPAMfighter here: http://www.spamfighter.com/len > > Do you have a slow PC? Try a Free scan > http://www.spamfighter.com/SLOW-PCfighter?cid=sigen >