Re: Numeric 508 datatype - Mailing list pgsql-patches

From Bruce Momjian
Subject Re: Numeric 508 datatype
Date
Msg-id 200512021717.jB2HHEh15358@candle.pha.pa.us
Whole thread Raw
In response to Re: Numeric 508 datatype  (Bruce Momjian <pgman@candle.pha.pa.us>)
Responses Re: Numeric 508 datatype  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-patches
Bruce Momjian wrote:
> OK, seems all objections have been dealt with so it goes into the patch
> queue.  I will ask on 'general'.
>
> The only downside I see is that I can't impress people by doing:
>
>     SELECT factorial(4000);
>
> I don't suppose the _impression_ factor is worth two bytes per value.
> Shame.
>
> I suppose people wanting to do such manipulations will have to store the
> numbers as text and use a server-side library like perl to do
> calculations.

Oops, I was wrong about this.  The patch changes the maximum _specified_
precision:

      /*
    !  * Hardcoded precision limit - arbitrary, but must be small enough that
    !  * dscale values will fit in 14 bits.
       */
    ! #define NUMERIC_MAX_PRECISION     1000

      /*
       * Internal limits on the scales chosen for calculation results
    --- 15,23 ----
      #define _PG_NUMERIC_H_

      /*
    !  * Hardcoded precision limit - maximum that can fit in Numeric storage
       */
    ! #define NUMERIC_MAX_PRECISION     508

but in fact, our computational precision is 4096, and we silently
overflow for values greater than that:

    test=> create table test(x numeric);
    CREATE TABLE
    test=> insert into test values (factorial(4000));
    INSERT 0 1

The length is 4096 digits, and so is factorial(10000) --- clearly wrong.
I now see in the TODO:

    * Change NUMERIC to enforce the maximum precision, and increase it

So we are really decreasing the specified precision from 1000 to 508,
and the computational precision from 4096 to 508.  Is there any plan to
fix the silent overflow problem?  Is that in the patch?  I don't see it.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

pgsql-patches by date:

Previous
From: Chris Browne
Date:
Subject: Re: [HACKERS] Should libedit be preferred to libreadline?
Next
From: Bruce Momjian
Date:
Subject: Re: Case Conversion Fix for MB Chars