Re: [PATCH] Fix old thinko in formula to compute sweight in numeric_sqrt(). - Mailing list pgsql-hackers

From Dean Rasheed
Subject Re: [PATCH] Fix old thinko in formula to compute sweight in numeric_sqrt().
Date
Msg-id CAEZATCUhSn_1aQ63qguMF=4WmvqYirjM2nC8rPFjz2VAdHmcmQ@mail.gmail.com
Whole thread Raw
In response to Re: [PATCH] Fix old thinko in formula to compute sweight in numeric_sqrt().  ("Joel Jacobson" <joel@compiler.org>)
Responses Re: [PATCH] Fix old thinko in formula to compute sweight in numeric_sqrt().
List pgsql-hackers
On Tue, 31 Jan 2023 at 15:05, Joel Jacobson <joel@compiler.org> wrote:
>
> I also think the performance impact no matter how small isn't worth it,
> but a comment based on your comments would be very valuable IMO.
>
> Below is an attempt at summarising your text, and to avoid the performance impact,
> maybe an #if so we get the general correct formula for DEC_DIGITS 1 or 2,
> and the reduced hand-optimised form for DEC_DIGITS 4?
> That could also improve readabilty, since readers perhaps more easily would see
> the relation between sweight and arg.weight, for the only DEC_DIGITS case we care about.
>

That seems a bit wordy, given the context of this comment. I think
it's sufficient to just give the formula, and note that it simplifies
when DEC_DIGITS is even (not just 4):

    /*
     * Assume the input was normalized, so arg.weight is accurate.  The result
     * then has at least sweight = floor(arg.weight * DEC_DIGITS / 2 + 1)
     * digits before the decimal point.  When DEC_DIGITS is even, we can save
     * a few cycles, since the division is exact and there is no need to
     *  round down.
     */
#if DEC_DIGITS == ((DEC_DIGITS / 2) * 2)
    sweight = arg.weight * DEC_DIGITS / 2 + 1;
#else
    if (arg.weight >= 0)
        sweight = arg.weight * DEC_DIGITS / 2 + 1;
    else
        sweight = 1 - (1 - arg.weight * DEC_DIGITS) / 2;
#endif

Regards,
Dean



pgsql-hackers by date:

Previous
From: Greg Stark
Date:
Subject: Re: Timeline ID hexadecimal format
Next
From: Tom Lane
Date:
Subject: Re: MacOS: xsltproc fails with "warning: failed to load external entity"