Thread: typo: XIDs are actually compared using modulo-2^32 arithmetic
Hi, It seems there is a typo here: http://www.postgresql.org/docs/devel/static/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND where we say that we compare XIDs using arithmetic modulo 2^31, which should instead be 2^32 (as it is with uint32, e.g. xid_age). Best wishes, Dr. Gianni Ciolli - 2ndQuadrant Italia PostgreSQL Training, Services and Support gianni.ciolli@2ndquadrant.it | www.2ndquadrant.it
Gianni Ciolli <gianni.ciolli@2ndquadrant.it> writes: > It seems there is a typo here: > http://www.postgresql.org/docs/devel/static/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND > where we say that we compare XIDs using arithmetic modulo 2^31, which > should instead be 2^32 (as it is with uint32, e.g. xid_age). [ thinks about that for awhile... ] Yeah, I think you're right. Patch pushed, thanks! regards, tom lane
<p dir="ltr">I don't have a source tree handy but iirc we treaty 2^31 values as being in the past and 2^31 values as beingin the future.<p dir="ltr">I've been trying to think how to protect better against the recent vacuum freeze bug. Ifsomeone ruins vacuum freeze now and has any wrapped values they'll destroy their possibly recoverable data.<p dir="ltr">Itseems to me we shouldn't really need 2^31 values in the future. If vacuum or hot pruning comes across an xidfar in the future, say a million xids further into the future than the most recent transaction, then it should signalan error rather than just treat it as being in the future.<p dir="ltr">-- <br /> greg<div class="gmail_quote">On 12Dec 2013 09:41, "Tom Lane" <<a href="mailto:tgl@sss.pgh.pa.us">tgl@sss.pgh.pa.us</a>> wrote:<br type="attribution"/><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Gianni Ciolli <<a href="mailto:gianni.ciolli@2ndquadrant.it">gianni.ciolli@2ndquadrant.it</a>>writes:<br /> > It seems there is a typohere:<br /> > <a href="http://www.postgresql.org/docs/devel/static/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND" target="_blank">http://www.postgresql.org/docs/devel/static/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND</a><br/> > wherewe say that we compare XIDs using arithmetic modulo 2^31, which<br /> > should instead be 2^32 (as it is with uint32,e.g. xid_age).<br /><br /> [ thinks about that for awhile... ] Yeah, I think you're right.<br /> Patch pushed, thanks!<br/><br /> regards, tom lane<br /><br /><br /> --<br /> Sent via pgsql-hackers mailing list(<a href="mailto:pgsql-hackers@postgresql.org">pgsql-hackers@postgresql.org</a>)<br /> To make changes to your subscription:<br/><a href="http://www.postgresql.org/mailpref/pgsql-hackers" target="_blank">http://www.postgresql.org/mailpref/pgsql-hackers</a><br/></blockquote></div>
On 2013-12-14 20:19:11 +0000, Greg Stark wrote: > I don't have a source tree handy but iirc we treaty 2^31 values as being in > the past and 2^31 values as being in the future. > > I've been trying to think how to protect better against the recent vacuum > freeze bug. If someone ruins vacuum freeze now and has any wrapped values > they'll destroy their possibly recoverable data. Fortunately that's exceedingly unlikely to happen. There's basically two consequences the bug can have: a) we don't freeze tuples on pages that are already marked all-visible because we're doing a partial scan and thus don'tscan them. b) (9.2+) we don't freeze tuples on a page not marked all visible, because a buffer is pinned and we skip those when !scan_all. a) can lead to the tuple vanishing again because they are reported as being in progress, after 2^31 xids passed. But by virtue of being on an all-visible page, they are fully hinted. Which means, that after the wraparound they will be reported as delete-in-progress or insert-in-progress. Luckily neither will get vacuumed away. They will "just" be invisible. What can happen with b) is that the clog gets truncated to somewhere between the real relfrozenxid and the computed relfrozenxid. In that case we'll get errors when later doing a HeapTupleSatisfiesVacuum/HTSMVCC. But it's quite likely that the tuple will get vacuumed at some point before 2^31 xids have passed since its not marked all visible and thus will be scanned with each future vacuum. So, for the data to be removed permanently you'd have to hit b) with partial vacuums (scan_all vacuums do wait!) several times in a row. That seems unlikely. > It seems to me we shouldn't really need 2^31 values in the future. If > vacuum or hot pruning comes across an xid far in the future, say a million > xids further into the future than the most recent transaction, then it > should signal an error rather than just treat it as being in the future. Yea, I have wondered about that as well. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services