Re: [RFC] Removing "magic" oids - Mailing list pgsql-hackers

Hi,

On 2018-09-29 20:48:10 -0700, Andres Freund wrote:
> In my opinion the current WITH OIDs system has numerous weaknesses:
> 
> 1) The fact that oids are so magic means that if we get pluggable
>    storage, the design of the potential pluggable systems is constrained
>    and similar magic has to be present everywhere.
> 
> 2) The fact that the oids in each table have the same counter to be
>    based on means that oid wraparounds have much worse consequences
>    performance wise than necessary. E.g. once the global counter has
>    wrapped, all toast tables start to be significantly slower.
> 
>    It would be much better if most database objects had their own
>    counters.
> 
> 3) For some oid using objects (toast, large objects at the very least)
>    it'd be quite worthwhile to switch to 8 byte ids.  Currently that's
>    hard to do, because it'd break on-disk compatibility.
> 
> 4) There's a lot of special case code around for dealing with oids.
> 
> 5a) The fact that system table oids don't show up in selects by default
>    makes it more work than necessary to look at catalogs.
> 
> 5b) Similarly, it's fairly annoying when debugging not to trivially see
>    oids for catalog structs.
> 
> 
> I think we should drop WITH OIDs support.  pg_dump should convert WITH
> OIDs tables into tables that have an explicit oid column (with an
> appropriate default function), pg_upgrade should refuse to upgrade them.
> We've defaulted WITH OIDs to off for quite a while now, so that's
> hopefully not going to be too painful.
> 
> For catalog tables, I think we should just add explicit oid columns.
> That obviously requires a fair amount of change, but it's not too bad.
> One issue here is that we we want to support "manual" inserts both for
> initdb, and for emergency work.
> 
> 
> The attached *PROTOTYPE* *WIP* patch removes WITH OIDs support, converts
> catalog table oids to explicit oid columns, and makes enough
> infrastructure changes to make plain pg_regress pass (after the
> necessary changes of course).  Only superficial psql and no pg_dump
> changes.
> 
> There's plenty of issues with the prototype, but overall I'm pretty
> pleased.
> 
> 
> There's three major areas I'm not so sure about:
> 
> 1) There's a few places dealing with system tables that don't deal with
>    a hardcoded system table. Since there's no notion of "table has oid"
>    and "which column is the oid column) anymore, we need some way to
>    deal with that.  So far I've just extended existing objectaddress.c
>    code to deal with that, but it's not that pretty.
> 
> 2) We need to be able to manually insert into catalog tables. Both
>    initdb and emergency surgery.  My current hack is doing so directly
>    in nodeModifyTable.c but that's beyond ugly.  I think we should add
>    an explicit DEFAULT clause to those columns with something like
>    nextoid('tablename', 'name_of_index') or such.
> 
> 3) The quickest way to deal with the bootstrap code was to just assign
>    all oids for oid carrying tables that don't yet have any assigned.
>    That doesn't generally seem terrible, although it's certainly badly
>    implemented right now.  That'd mean we'd have three ranges of oids
>    probably, unless we somehow have the bootstrap code advance the
>    in-database oid counter to a right state before we start with
>    post-bootstrap work.  I like the idea of all bootstrap time oids
>    being determined by genbki.pl (I think that'll allow to remove some
>    code too).
> 
> 
> I do wonder about ripping the oid counter out entirely, and replacing it
> with sequences. But that seems like a separate project.
> 
> 
> I'll polish this up further in the not too far away future.  But I'd
> really like to get some feedback before I sink more time into this.

Does anybody have engineering / architecture level comments about this
proposal?

Greetings,

Andres Freund