Re: width_bucket issue - Mailing list pgsql-general

From Christophe Pettus
Subject Re: width_bucket issue
Date
Msg-id D1114C6A-9A61-468C-817E-86D467198569@thebuild.com
Whole thread Raw
In response to width_bucket issue  (Raphaël Berbain <raphael.berbain@gmail.com>)
List pgsql-general
> On Jul 24, 2018, at 13:02, Raphaël Berbain <raphael.berbain@gmail.com> wrote:
> I'd expect b1 = b2 = 2. What am I missing?

The problem appears to be due to rounding during the intermediate calculations.  In compute_bucket() in numeric.c:

        div_var(&operand_var, &bound1_var, result_var,
                select_div_scale(&operand_var, &bound1_var), true);

... produces 0.99999999999999999999 for that particular value, instead of 1, and the subsequent +1 and FLOOR() result
in1 instead of 2. 
--
-- Christophe Pettus
   xof@thebuild.com



pgsql-general by date:

Previous
From: Rory Falloon
Date:
Subject: Re: Dealing with latency to replication slave; what to do?
Next
From: Jeff Janes
Date:
Subject: Re: Dealing with latency to replication slave; what to do?