<div dir="ltr">On Tue, May 10, 2016 at 9:02 AM, Tom Lane <<a
href="mailto:tgl@sss.pgh.pa.us">tgl@sss.pgh.pa.us</a>>wrote:<br />> Kevin Grittner <<a
href="mailto:kgrittn@gmail.com">kgrittn@gmail.com</a>>writes:<br />>> There were 75 samples each of "disabled"
and"reverted" in the<br />>> spreadsheet. Averaging them all, I see this:<br />><br />>> reverted:
290,660TPS<br />>> disabled: 292,014 TPS<br />><br />>> That's a 0.46% overall increase in performance
withthe patch,<br />>> disabled, compared to reverting it. I'm surprised that you<br />>> consider that to
bea "clearly measurable difference". I mean, it<br />>> was measured and it is a difference, but it seems to be
wellwithin<br />>> the noise. Even though it is based on 150 samples, I'm not sure we<br />>> should
considerit statistically significant.<br />><br />> You don't have to guess about that --- compare it to the
standard<br/>> deviation within each group.<br /><br />My statistics skills are rusty, but I thought that just gives
you<br/>an effect size, not any idea of whether the effect is statistically<br />significant.<br /><br />Does anyone
withsharper skills in this area than I want to opine<br />on whether there is a statistically significant difference
between<br/>the numbers on "master-default-disabled" lines and "master-revert"<br />lines in the old_snap.ods file
attachedto an earlier post on this<br />thread?<br /><br />--<br />Kevin Grittner<br />EDB: <a
href="http://www.enterprisedb.com">http://www.enterprisedb.com</a><br/>The Enterprise PostgreSQL Company</div>
