Thread: 32bit OID wrap around concerns
One scenario is to use an oid to identify a toast value. As the oid generation is mono increased within a database instance, it can gets wrap around after 2^32 generations. After that:
1. GetNewOidWithIndex() could gets unbounded performance as it needs to by pass already in use values of its own.
2. These by-passed oids are actually usable by other toast indices, but are wasted. This can lead to further aggravated OID wrap around.
Do we think above scenario is something we shall worry about? Especially for large databases.
Thanks,
Qingqing
Qingqing
Qingqing Zhou wrote: > One scenario is to use an oid to identify a toast value. As the oid > generation is mono increased within a database instance, it can gets wrap > around after 2^32 generations. After that: > 1. GetNewOidWithIndex() could gets unbounded performance as it needs to by > pass already in use values of its own. > 2. These by-passed oids are actually usable by other toast indices, but are > wasted. This can lead to further aggravated OID wrap around. > > Do we think above scenario is something we shall worry about? Especially > for large databases. IMO in theory it sucks that toast values use the shared OID generator, though in practice I have never seen a problem due to that. Often, toast value creation is interleaved across several tables, so the shared generator does not have to skip too many values at once each time. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> Qingqing Zhou wrote:
>> Do we think above scenario is something we shall worry about? Especially
>> for large databases.
> IMO in theory it sucks that toast values use the shared OID generator,
> though in practice I have never seen a problem due to that. Often,
> toast value creation is interleaved across several tables, so the shared
> generator does not have to skip too many values at once each time.
Yeah, I was just explaining that to some Salesforce colleagues. The fact
that we use a single shared OID generator (and not one per table) greatly
reduces the risk of long runs of consecutively-assigned OIDs in any one
table.
There is an issue if you do a dump and restore: the toast OIDs used in any
one table will be consecutive after that, because we load all the data for
each table sequentially. It might be worth tweaking the generation rules
to try to avoid that scenario. Still, there's not been any field
complaints indicating that this is a problem.
regards, tom lane
On Mon, Mar 2, 2015 at 12:38 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> There is an issue if you do a dump and restore: the toast OIDs used in any
> one table will be consecutive after that, because we load all the data for
> each table sequentially.
With these consecutive oids, the "aggravated oid wrap around" can be
illustrated by the following example:- table A oids in [minOid, 2^31)- table B oids in [2^31, 2^32]
After 1st time wrap around, and follows an insert(A), it immediately
bumps oid to 2^31. Now it is B's turn, it has to bump it to 2^32 and
wrap it around the 2nd time. This extreme example shows that we can
incur another wrap around with 2 toast insertions.
It is not clear how above example can be translated into real world
scenario probabilistically ("many more interleaved ranges (down) with
many more concurrency (up)"), but the "aggravated" issue seems there.
Regards,
Qingqing