Thread: Commit ab596105b55 - BRIN minmax-multi indexes

Rushabh Lathia <rushabh.lathia@gmail.com> writes:
> Commit mentioned in the $subject changed the FirstBootstrapObjectId
> (transam.h) from 12000 to 13000.  I was trying to understand the reason
> behind this change, but was not able to gather that information. Also didn't
> find anything in the commit message either.

As of right now, genbki.pl's OID counter reaches 12036, so it's
pretty clear that 12000 no longer works.  (I have this figure in
my head because I noted it while working on [1].)  13000 might
well be an excessive jump though.  Do you have a concrete problem
with it?

            regards, tom lane

[1] https://www.postgresql.org/message-id/flat/3737988.1618451008@sss.pgh.pa.us

Hi,

On 4/15/21 4:03 PM, Rushabh Lathia wrote:
> Hi,
> 
> Commit mentioned in the $subject changed the FirstBootstrapObjectId
> (transam.h) from 12000 to 13000.  I was trying to understand the reason
> behind this change, but was not able to gather that information. Also didn't
> find anything in the commit message either.
> 
> Can you please explain those changes? Is it accidental or intentional?
> 

Yeah, it's an intentional change - I should have mentioned it explicitly
in the thread, probably.

We're assigning OIDs to catalog entries, at different phases, and each
phase has a range or OIDs to ensure the values are unique. The first
phase is genkbi.pl which transforms the .dat files, assigns OIDs in the
[FirstGenbkiObjectId, FirstBootstrapObjectId) range.

However, patches are adding new stuff to the .dat files, so we may hit
the upper limit. The minmax patch happened to add enough new entries to
hit it, i.e. the genbki.pl needed OIDs above FirstBootstrapObjectId and
the compilation would fail. Try lowering the value back to 12000 and run
run "make check" again - it'll fail.

The limits are mostly arbitrary, the primary purpose is to ensure the
OIDs are unique etc. So the patch simply added 1000 values to the genbki
range, to fix this.

Not sure what'll happen once we fill all those ranges, but we're quite
far from that, I think. It took us ~20 years to get 2000 OIDs in the
genbki range, and the bootstrap has ~1000 OIDs. So we've used only about
half the values between 10k and 16k, so far ...


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 4/15/21 4:29 PM, Rushabh Lathia wrote:
> 
> 
> On Thu, Apr 15, 2021 at 7:49 PM Tom Lane <tgl@sss.pgh.pa.us
> <mailto:tgl@sss.pgh.pa.us>> wrote:
> 
>     Rushabh Lathia <rushabh.lathia@gmail.com
>     <mailto:rushabh.lathia@gmail.com>> writes:
>     > Commit mentioned in the $subject changed the FirstBootstrapObjectId
>     > (transam.h) from 12000 to 13000.  I was trying to understand the
>     reason
>     > behind this change, but was not able to gather that information.
>     Also didn't
>     > find anything in the commit message either.
> 
>     As of right now, genbki.pl <http://genbki.pl>'s OID counter reaches
>     12036, so it's
>     pretty clear that 12000 no longer works.  (I have this figure in
>     my head because I noted it while working on [1].)  13000 might
>     well be an excessive jump though.  Do you have a concrete problem
>     with it?
> 

Yeah, the bump from 12000 to 13000 might be unnecessarily large. But
considering the bootstrap uses only about 1000 OIDs from the >=13000
range, I don't see this as a problem. Surely we can move the ranges in
the future, if needed?

> 
> In EDB Advance Server, it has their own set of system objects.  Due
> to mentioned commit (where it changes the FirstBootstrapObjectId to 13000),
> now system objects exceeding the FirstNormalObjectId.  
> 

I haven't checked what the EDBAS does exactly, but how could it hit
16384 because of custom catalogs? I haven't checked what exactly is
EDBAS doing, but surely it does not have thousands of catalogs, right?

It's OK to lower FirstBootstrapObjectId to e.g. 12500 during the merge,
if that solves the issue for EDBAS. As I said, those ranges are mostly
arbitrary anyway, and EDBAS already has catalogs differences.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Rushabh Lathia <rushabh.lathia@gmail.com> writes:
> On Thu, Apr 15, 2021 at 7:49 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> As of right now, genbki.pl's OID counter reaches 12036, so it's
>> pretty clear that 12000 no longer works.  (I have this figure in
>> my head because I noted it while working on [1].)  13000 might
>> well be an excessive jump though.  Do you have a concrete problem
>> with it?

> In EDB Advance Server, it has their own set of system objects.  Due
> to mentioned commit (where it changes the FirstBootstrapObjectId to 13000),
> now system objects exceeding the FirstNormalObjectId.

You might want to rethink where you're allocating those OIDs.  Even if
we didn't move FirstBootstrapObjectId today, it's inevitably going to
creep up over time.

As I recall the discussions about this, we'd expected that add-on products
that need OIDs in the bootstrap range would take them from the 8K-10K
range, not above FirstBootstrapObjectId.  Because of the possibility of
having lots of system locales creating lots of collations, the amount of
available OID space above FirstBootstrapObjectId is not as predictable as
you might wish.  (I suspect eventually we're going to have to back off
the idea of creating every possible locale at bootstrap, but we haven't
addressed that yet.)

We are overlapping development use of the 8K-10K OID range with it being
available for add-ons post-release, which might make it hard to do testing
against HEAD.  But you could renumber the not-yet-frozen objects' IDs out
of the way whenever you want to make a merge.

            regards, tom lane