From 916e6a6121f03b65f361e1ca064fb5d4b6181cee Mon Sep 17 00:00:00 2001 From: Peter Geoghegan Date: Sat, 22 Apr 2023 12:41:00 -0700 Subject: [PATCH v3 5/9] Move Interpreting XID stamps from tuple headers. This is intended to be fairly close to a mechanical change. It isn't entirely mechanical, though, since the original wording has been slightly modified for it to work in context. Structuring things this way should make life a little easier for doc translators. --- doc/src/sgml/maintenance.sgml | 80 ++++------------------ doc/src/sgml/xact.sgml | 125 ++++++++++++++++++++++++++++------ 2 files changed, 120 insertions(+), 85 deletions(-) diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml index e130dfdbd..cb6f28e1e 100644 --- a/doc/src/sgml/maintenance.sgml +++ b/doc/src/sgml/maintenance.sgml @@ -447,75 +447,25 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu wraparound - - wraparound - of transaction IDs - + + wraparound + of transaction IDs + - PostgreSQL's - MVCC transaction semantics - depend on being able to compare transaction ID (XID) - numbers: a row version with an insertion XID greater than the current - transaction's XID is in the future and should not be visible - to the current transaction. But since transaction IDs have limited size - (32 bits) a cluster that runs for a long time (more - than 4 billion transactions) would suffer transaction ID - wraparound: the XID counter wraps around to zero, and all of a sudden - transactions that were in the past appear to be in the future — which - means their output become invisible. In short, catastrophic data loss. - (Actually the data is still there, but that's cold comfort if you cannot - get at it.) To avoid this, it is necessary to vacuum every table - in every database at least once every two billion transactions. + PostgreSQL's MVCC transaction semantics depend on + being able to compare transaction + ID numbers (XID) to determine + whether or not the row is visible to each query's MVCC snapshot + (see ). But since + on-disk storage of transaction IDs in heap pages uses a truncated + 32-bit representation to save space (rather than the full 64-bit + representation), it is necessary to vacuum every table in every + database at least once every two billion + transactions (though far more frequent vacuuming is typical). - - The reason that periodic vacuuming solves the problem is that - VACUUM will mark rows as frozen, indicating that - they were inserted by a transaction that committed sufficiently far in - the past that the effects of the inserting transaction are certain to be - visible to all current and future transactions. - Normal XIDs are - compared using modulo-232 arithmetic. This means - that for every normal XID, there are two billion XIDs that are - older and two billion that are newer; another - way to say it is that the normal XID space is circular with no - endpoint. Therefore, once a row version has been created with a particular - normal XID, the row version will appear to be in the past for - the next two billion transactions, no matter which normal XID we are - talking about. If the row version still exists after more than two billion - transactions, it will suddenly appear to be in the future. To - prevent this, PostgreSQL reserves a special XID, - FrozenTransactionId, which does not follow the normal XID - comparison rules and is always considered older - than every normal XID. - Frozen row versions are treated as if the inserting XID were - FrozenTransactionId, so that they will appear to be - in the past to all normal transactions regardless of wraparound - issues, and so such row versions will be valid until deleted, no matter - how long that is. - - - - - In PostgreSQL versions before 9.4, freezing was - implemented by actually replacing a row's insertion XID - with FrozenTransactionId, which was visible in the - row's xmin system column. Newer versions just set a flag - bit, preserving the row's original xmin for possible - forensic use. However, rows with xmin equal - to FrozenTransactionId (2) may still be found - in databases pg_upgrade'd from pre-9.4 versions. - - - Also, system catalogs may contain rows with xmin equal - to BootstrapTransactionId (1), indicating that they were - inserted during the first phase of initdb. - Like FrozenTransactionId, this special XID is treated as - older than every normal XID. - - - controls how old an XID value has to be before rows bearing that XID will be diff --git a/doc/src/sgml/xact.sgml b/doc/src/sgml/xact.sgml index b467660ee..0762442e1 100644 --- a/doc/src/sgml/xact.sgml +++ b/doc/src/sgml/xact.sgml @@ -22,6 +22,8 @@ single-statement transactions. + + Virtual Transaction IDs Every transaction is identified by a unique VirtualTransactionId (also called @@ -46,29 +48,111 @@ started, particularly if the transaction started with statements that only performed database reads. + - - The internal transaction ID type xid is 32 bits wide - and wraps around every - 4 billion transactions. A 32-bit epoch is incremented during each - wraparound. There is also a 64-bit type xid8 which - includes this epoch and therefore does not wrap around during the - life of an installation; it can be converted to xid by casting. - The functions in - return xid8 values. Xids are used as the - basis for PostgreSQL's MVCC concurrency mechanism and streaming - replication. - + + Permanent Transaction IDs + + The internal transaction ID type xid is 32 bits wide + and wraps around every + 4 billion transactions. A 32-bit epoch is incremented during each + wraparound. There is also a 64-bit type xid8 which + includes this epoch and therefore does not wrap around during the + life of an installation; it can be converted to xid by casting. + The functions in + return xid8 values. Xids are used as the + basis for PostgreSQL's MVCC concurrency mechanism and streaming + replication. + - - When a top-level transaction with a (non-virtual) xid commits, - it is marked as committed in the pg_xact - directory. Additional information is recorded in the - pg_commit_ts directory if is enabled. - + + When a top-level transaction with a (non-virtual) xid commits, + it is marked as committed in the pg_xact + directory. Additional information is recorded in the + pg_commit_ts directory if is enabled. + + + <type>TransactionId</type> comparison rules + + The system often needs to compare t_xmin + and t_xmax fields for MVCC snapshot + visibility checks. + + + + We use a truncated 32-bit representation of transaction IDs, rather + than using the full 64-bit representation. The 2.1 billion XIDs + distance invariant must be preserved because + transaction IDs stored in heap row headers use a truncated 32-bit + representation (rather than the full 64-bit representation). Since + all unfrozen transaction IDs from heap tuple headers + must be from the same transaction ID epoch (or + from a space in the 64-bit representation that spans two adjoining + transaction ID epochs), there isn't any need to store a separate + epoch field in each tuple header (see for further details). This + scheme has the advantage of requiring much less space than a design + that stores an XID epoch alongside each XID stored in each heap + tuple header. It has the disadvantage of constraining the system's + ability to allocate new XIDs (in the worst case scenario where + xidStopLimit mode is used to preserve the + distance invariant). + + + + VACUUM will + mark tuple headers frozen, indicating + that all eligible rows on the page were inserted by a transaction + that committed sufficiently far in the past that the effects of the + inserting transaction are certain to be visible to all current and + future transactions. Normal XIDs are compared using + modulo-232 arithmetic. This means that + for every normal XID, there are two billion XIDs that are + older and two billion that are newer; + another way to say it is that the normal XID space is circular with + no endpoint. Therefore, once a row version has been created with a + particular normal XID, the row version will appear to be in + the past for the next two billion transactions, no matter + which normal XID we are talking about. If the row version still + exists after more than two billion transactions, it will suddenly + appear to be in the future. To prevent this, + PostgreSQL reserves a special XID, + FrozenTransactionId, which does not follow the + normal XID comparison rules and is always considered older than + every normal XID. Frozen row versions are treated as if the + inserting XID were FrozenTransactionId, so that + they will appear to be in the past to all normal + transactions regardless of wraparound issues, and so such row + versions will be valid until deleted, no matter how long that is. + + + + + In PostgreSQL versions before 9.4, freezing was + implemented by actually replacing a row's insertion XID + with FrozenTransactionId, which was visible in the + row's xmin system column. Newer versions just set a flag + bit, preserving the row's original xmin for possible + forensic use. However, rows with xmin equal + to FrozenTransactionId (2) may still be found + in databases pg_upgrade'd from pre-9.4 versions. + + + Also, system catalogs may contain rows with xmin equal + to BootstrapTransactionId (1), indicating that they were + inserted during the first phase of initdb. + Like FrozenTransactionId, this special XID is treated as + older than every normal XID. + + + + + + + Global Transaction Identifiers In addition to vxid and xid, prepared transactions are also assigned Global Transaction @@ -77,6 +161,7 @@ prepared transactions. The mapping of GID to xid is shown in pg_prepared_xacts. + -- 2.40.1