Thread: Is monotonous xid8 is a right way to do?

Pavel Borisov <pashkin.elfe@gmail.com> writes:

> Hi hackers!
>
> Now we have two data types xid and xid8. The first one (xid) makes a
> numeric ring, and xid8 are monotonous.
>
> As per [1] "Unlike xid values, xid8 values increase strictly monotonically
> and cannot be reused in the lifetime of a database cluster."
>
> As a consequence of [1] xid8 can have min/max functions (committed in [2]),
> which xid can not have.
>
> When working on 64xid patch [3] we assume that even 64xid's technically can
> be wraparound-ed, although it's very much unlikely. I wonder what is
> expected to be with xid8 values at this (unlikely) 64xid wraparound?

Even if a cluster was consuming a million XIDs per second, it would take
over half a million years to wrap around the 64bit range. Is that really
something we should worry about?

ilmari@[local]:5432 ~=# select 2::numeric^64/10^9/3600/24/365;
┌──────────────────┐
│     ?column?     │
├──────────────────┤
│ 584942.417355072 │
└──────────────────┘

- ilmari

Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> writes:

> Even if a cluster was consuming a million XIDs per second, it would take
> over half a million years to wrap around the 64bit range. Is that really
> something we should worry about?
>
> ilmari@[local]:5432 ~=# select 2::numeric^64/10^9/3600/24/365;

Oops, that should be 10^6, not 10^9. I was dithering over whether to do
it as a million or a billion per second.  For a billion XIDs per second
it would last a mere half millennium.

> ┌──────────────────┐
> │     ?column?     │
> ├──────────────────┤
> │ 584942.417355072 │
> └──────────────────┘
>
> - ilmari

On Fri, 1 Apr 2022 at 14:13, Pavel Borisov <pashkin.elfe@gmail.com> wrote:
>
> Hi hackers!
>
> Now we have two data types xid and xid8. The first one (xid) makes a numeric ring, and xid8 are monotonous.
>
> As per [1] "Unlike xid values, xid8 values increase strictly monotonically and cannot be reused in the lifetime of a
databasecluster."
 
>
> As a consequence of [1] xid8 can have min/max functions (committed in [2]), which xid can not have.
>
> When working on 64xid patch [3] we assume that even 64xid's technically can be wraparound-ed, although it's very much
unlikely.I wonder what is expected to be with xid8 values at this (unlikely) 64xid wraparound?
 
>
> What do you think about this? Wouldn't it be better to change xid8 to form a numeric ring like xid? I think it is
necessaryfor any 64-wraparound-enabled implementation of 64xids.
 
>
> Please feel free to share your thoughts.

Assuming that each Xid is WAL-logged (or at least one in 8) we won't
see xid8 wraparound, as our WAL is byte-addressable with only 64 bits
used as the identifier. As such, we can only fit a maximum of 2^61
xid8s in our WAL; which is less than what would be needed to wrap
around.

Addressed another way: If we'd have a system that consumed one xid
every CPU clock; then the best available x86 hardware right now would
currently consume ~ 5.5B xids every second. This would still leave
around 100 years of this system running non-stop before we'd be
hitting xid8 wraparound (= 2^64 / 5.5e9 (xid8 /sec) / 3600 (min /hour)
/ 730 (hour / month)/ 12 (month /year)).

I don't think we'll have to consider that an issue for now. Maybe,
eventually, if we start doing distributed transactions where
transaction IDs are reasonably consumed at a rate higher than 5B /sec
(and not logged at that rate) we can start considering this to be a
problem.

A different and more important issue (IMO) is that the xlog record
header currently only supports 32-bit xids -- long-running
transactions can reasonably see a xid4 wraparound in their lifetime.

Enjoy,

Matthias van de Meent