Re: Bug in numeric multiplication - Mailing list pgsql-hackers

From Dean Rasheed
Subject Re: Bug in numeric multiplication
Date
Msg-id CAEZATCWys0N9PVQUA27iUjw0rDauAm35nwGHJ7PKFacxtXnFiQ@mail.gmail.com
Whole thread Raw
In response to Re: Bug in numeric multiplication  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Bug in numeric multiplication  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
On 21 September 2015 at 16:09, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> I wrote:
>> Dean Rasheed <dean.a.rasheed@gmail.com> writes:
>>> The problem then arises in the final carry propagation pass. During
>>> this phase of the computation, the carry from one digit (which can be
>>> a shade under INT_MAX / NBASE) is added to the next digit, and that's
>>> where the overflow happens.
>
>> Nice catch!  I think the comment could use a little more work, but I'll
>> adjust it and push.
>
> After trying to rework the comment to explain what maxdig really meant
> after your changes, I came to the conclusion that it'd be better to do
> it as per attached.  Does this look sane to you?
>

Yes that looks better. It's still the same amount of extra headroom
(21), but I think it's clearer your way.

Regards,
Dean



pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Bug in numeric multiplication
Next
From: Dmitry Dolgov
Date:
Subject: Re: jsonb_set array append hack?