Re: cheaper snapshots redux - Mailing list pgsql-hackers

<div class="Section1"><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">>If you know what transactions were running the last time a snapshot summary was written and what
>transactionshave ended since then, you can work out the new xmin on the fly.  I have working >code for this and
it'sactually quite simple.</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span
style="font-size:
8.0pt"> </span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">I believe one method to do same is as follows:</span></font><p class="MsoPlainText"><font face="Courier New"
size="1"><spanstyle="font-size:
 
8.0pt">Let us assume at some point of time the snapshot and completed XID list is somewhat as follows:</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt">Snapshot</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">{</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">Xmin – 5</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">Xip[] – 8 10 12</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">Xmax  - 15</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">}</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">Committed XIDS – 8, 10 , 12, 18, 20, 21</span></font><p class="MsoPlainText"><font face="Courier New"
size="1"><spanstyle="font-size:
 
8.0pt">So it means 16,17,19 are running transactions. So it will behave as follows:</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt">Xmin – 16</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">Xmax – 21</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">Xip[] – 17,19</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt"> </span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">But if we do above way to calculate Xmin, we need to check in existing Xip array and committed Xid array to find
Xmin.Won’t this cause reasonable time even though it is outside lock time if Xip and Xid are large.</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt"> </span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">> Because GetSnapshotData() computes a new value for RecentGlobalXmin by scanning the ProcArray.  > This
isn'tcosting a whole lot extra right now because the xmin and xid fields are normally in > the same cache line, so
onceyou've looked at one of them it doesn't cost that much extra to </span></font><p class="MsoPlainText"><font
face="CourierNew" size="1"><span style="font-size:
 
8.0pt">> look at the other.  If, on the other hand, you're not looking at (or even locking) the </span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt">> ProcArray, then doing so just to recomputed RecentGlobalXmin sucks.</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt"> </span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">Yes, this is more time as compare to earlier, but if our approach to calculate Xmin is like above point, then
oneextra read outside lock should not matter. However if for above point approach is different then it will be
costlier.</span></font><pclass="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
 
8.0pt"> </span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">***************************************************************************************</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt">This e-mail and attachments contain confidential information from HUAWEI, which is intended only for the person
orentity whose address is listed above. Any use of the information contained herein in any way (including, but not
limitedto, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient's)
isprohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete
it!</span></font><pclass="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
 
8.0pt"> </span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt"> </span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">-----Original Message-----<br /> From: Robert Haas [mailto:robertmhaas@gmail.com] <br /> Sent: Monday, September
12,2011 7:39 PM<br /> To: Amit Kapila<br /> Cc: pgsql-hackers@postgresql.org<br /> Subject: Re: [HACKERS] cheaper
snapshotsredux</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
 
8.0pt"> </span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">On Sun, Sep 11, 2011 at 11:08 PM, Amit Kapila <amit.kapila@huawei.com> wrote:</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt">>   In the approach mentioned in your idea, it mentioned that once after</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt">> taking snapshot, only committed XIDs will be updated and sometimes snapshot</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt">> itself.</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">> </span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">>   So when the xmin will be updated according to your idea as snapshot will</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt">> not be updated everytime so xmin cannot be latest always.</span></font><p class="MsoPlainText"><font
face="CourierNew" size="1"><span style="font-size:
 
8.0pt"> </span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">If you know what transactions were running the last time a snapshot</span></font><p class="MsoPlainText"><font
face="CourierNew" size="1"><span style="font-size:
 
8.0pt">summary was written and what transactions have ended since then, you</span></font><p class="MsoPlainText"><font
face="CourierNew" size="1"><span style="font-size:
 
8.0pt">can work out the new xmin on the fly.  I have working code for this</span></font><p class="MsoPlainText"><font
face="CourierNew" size="1"><span style="font-size:
 
8.0pt">and it's actually quite simple.</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span
style="font-size:
8.0pt"> </span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">>>RecentGlobalXmin doesn't need to be completely up to date, and in fact</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt">> recomputing it on every snapshot becomes prohibitively expensive with this</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt">> approach.  I'm still struggling with the best way to handle that.</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt">> </span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">>   RecentGlobalXmin and RecentXmin are mostly updated with snapshots xmin</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt">> and that too outside ProcArrayLock, so why it will be expensive if you have</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt">> updated xmin.</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span
style="font-size:
8.0pt"> </span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">Because GetSnapshotData() computes a new value for RecentGlobalXmin by</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt">scanning the ProcArray.  This isn't costing a whole lot extra right</span></font><p class="MsoPlainText"><font
face="CourierNew" size="1"><span style="font-size:
 
8.0pt">now because the xmin and xid fields are normally in the same cache</span></font><p class="MsoPlainText"><font
face="CourierNew" size="1"><span style="font-size:
 
8.0pt">line, so once you've looked at one of them it doesn't cost that much</span></font><p class="MsoPlainText"><font
face="CourierNew" size="1"><span style="font-size:
 
8.0pt">extra to look at the other.  If, on the other hand, you're not looking</span></font><p
class="MsoPlainText"><fontface="Courier New" size="1"><span style="font-size:
 
8.0pt">at (or even locking) the ProcArray, then doing so just to recompute</span></font><p class="MsoPlainText"><font
face="CourierNew" size="1"><span style="font-size:
 
8.0pt">RecentGlobalXmin sucks.</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span
style="font-size:
8.0pt"> </span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">-- </span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">Robert Haas</span></font><p class="MsoPlainText"><font face="Courier New" size="1"><span style="font-size:
8.0pt">EnterpriseDB: http://www.enterprisedb.com</span></font><p class="MsoPlainText"><font face="Courier New"
size="1"><spanstyle="font-size:
 
8.0pt">The Enterprise PostgreSQL Company</span></font></div>