Re: Production block comparison facility - Mailing list pgsql-hackers

<p dir="ltr">If you're always going FPW then there's no point in the rest of the record. The point here was to find
problemsso that users could run normally with confidence. <p dir="ltr">The cases you might want to run in the mode you
describeare the build farm or integration testing. When treating your application on the next release of postgres it
wouldbe nice to have tests for the replication in your workload given the experience in 9.3. <p dir="ltr">Even without
theconstant full page writes a live production system could do a FPW comparison after a FPW if it was in a consistent
state.That would give standbys periodic verification at low costs.<p dir="ltr">-- <br /> greg<div
class="gmail_quote">On22 Jul 2014 12:28, "Simon Riggs" <<a
href="mailto:simon@2ndquadrant.com">simon@2ndquadrant.com</a>>wrote:<br type="attribution" /><blockquote
class="gmail_quote"style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> On 22 July 2014 08:49,
MichaelPaquier <<a href="mailto:michael.paquier@gmail.com">michael.paquier@gmail.com</a>> wrote:<br /> > On
Sun,Jul 20, 2014 at 5:31 PM, Simon Riggs <<a href="mailto:simon@2ndquadrant.com">simon@2ndquadrant.com</a>>
wrote:<br/> >> The block comparison facility presented earlier by Heikki would not be<br /> >> able to be
usedin production systems. ISTM that it would be desirable<br /> >> to have something that could be used in that
way.<br/> >><br /> >> ISTM easy to make these changes<br /> >><br /> >> * optionally generate a
FPWfor every WAL record, not just first<br /> >> change after checkpoint<br /> >> full_page_writes =
'always'<br/> >><br /> >> * when an FPW arrives, optionally run a check to see if it compares<br />
>>correctly against the page already there, when running streaming<br /> >> replication without a recovery
target.We could skip reporting any<br /> >> problems until the database is consistent<br /> >>
full_page_write_check= on<br /> >><br /> >> The above changes seem easy to implement.<br /> >><br />
>>With FPW compression, this would be a usable feature in production.<br /> >><br /> >> Comments?<br
/>><br /> > This is an interesting idea, and it would be easier to use than what<br /> > has been submitted
forCF1. However, full_page_writes set to "always"<br /> > would generate a large amount of WAL even for small
records,<br/> > increasing I/O for the partition holding pg_xlog, and the frequency of<br /> > checkpoints run on
system.Is this really something suitable for<br /> > production?<br /><br /> For critical systems, yes, I think it
is.<br/><br /> It would be possible to make that user selectable for particular<br /> transactions or tables.<br /><br
/>> Then, looking at the code, we would need to tweak XLogInsert for the<br /> > WAL record construction to
alwaysdo a FPW and to update<br /> > XLogCheckBufferNeedsBackup. Then for the redo part, we would need to<br /> >
dosome extra operations in the area of<br /> > RestoreBackupBlock/RestoreBackupBlockContents, including masking<br
/>> operations before comparing the content of the FPW and the current<br /> > page.<br /> ><br /> > Does
thatsound right?<br /><br /> Yes, it doesn't look very much code because it fits well with existing<br />
approaches.<br/><br /> --<br />  Simon Riggs                   <a href="http://www.2ndQuadrant.com/"
target="_blank">http://www.2ndQuadrant.com/</a><br/>  PostgreSQL Development, 24x7 Support, Training & Services<br
/><br/><br /> --<br /> Sent via pgsql-hackers mailing list (<a
href="mailto:pgsql-hackers@postgresql.org">pgsql-hackers@postgresql.org</a>)<br/> To make changes to your
subscription:<br/><a href="http://www.postgresql.org/mailpref/pgsql-hackers"
target="_blank">http://www.postgresql.org/mailpref/pgsql-hackers</a><br/></blockquote></div>