Thread: pgsql: Introduce new page checksum algorithm and module.
Introduce new page checksum algorithm and module. Isolate checksum calculation to its own module, so that bufpage knows little if anything about the details of the calculation. This implementation is a modified FNV-1a hash checksum, details of which are given in the new checksum.c header comments. Basic implementation only, so we fix the output value. Later related commits will add version numbers to pg_control, compiler optimization flags and memory barriers. Ants Aasma, reviewed by Jeff Davis and Simon Riggs Branch ------ master Details ------- http://git.postgresql.org/pg/commitdiff/43e7a668499b8a69a62cc539a0fbe6983384339c Modified Files -------------- src/backend/storage/page/Makefile | 2 +- src/backend/storage/page/bufpage.c | 36 ++++---- src/backend/storage/page/checksum.c | 160 +++++++++++++++++++++++++++++++++++ src/include/storage/checksum.h | 23 +++++ 4 files changed, 201 insertions(+), 20 deletions(-)
On Mon, April 29, 2013 10:08, Simon Riggs wrote: > Introduce new page checksum algorithm and module. I get these errors (but on an existing database), so apparently a catversion bump was needed? psql: FATAL: invalid page in block 0 of relation global/12639 psql: FATAL: invalid page in block 0 of relation global/12639 2013-04-29 10:23:12.553 CEST 12292 FATAL: invalid page in block 0 of relation global/12639 2013-04-29 10:23:12.561 CEST 12295 WARNING: page verification failed, calculated checksum 64163 but expected 59696 2013-04-29 10:23:12.561 CEST 12295 FATAL: invalid page in block 0 of relation global/12639 2013-04-29 10:23:12.567 CEST 12299 WARNING: page verification failed, calculated checksum 64163 but expected 59696 2013-04-29 10:23:12.567 CEST 12299 FATAL: invalid page in block 0 of relation global/12639 thanks, Erik Rijkers
On 29 April 2013 09:28, Erikjan Rijkers <er@xs4all.nl> wrote: > On Mon, April 29, 2013 10:08, Simon Riggs wrote: >> Introduce new page checksum algorithm and module. > > I get these errors (but on an existing database), so apparently a catversion bump was needed? You noticed I had committed, upgraded your database, which was running a non-default option using alpha code, ran tests on it and wrote an email back, all within 20 min? I take it this is just a test and the point is just a theoretical one based on an automated test suite? I'm not sure that we test for upgrade compatibility anywhere in the community tests. Would you care to share your test suite with the community, it sounds useful. Thanks for the report. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Mon, April 29, 2013 11:03, Simon Riggs wrote: > On 29 April 2013 09:28, Erikjan Rijkers <er@xs4all.nl> wrote: >> On Mon, April 29, 2013 10:08, Simon Riggs wrote: >>> Introduce new page checksum algorithm and module. >> >> I get these errors (but on an existing database), so apparently a catversion bump was needed? > > You noticed I had committed, upgraded your database, which was running > a non-default option using alpha code, ran tests on it and wrote an > email back, all within 20 min? > > I take it this is just a test and the point is just a theoretical one > based on an automated test suite? I'm not sure that we test for > upgrade compatibility anywhere in the community tests. > > Would you care to share your test suite with the community, it sounds useful. > > Thanks for the report. > Well, when I see a commit (via email), I do a pull and I look if it needs initdb on the basis of the two variables below (catalog_version and control_version); if an initdb is not needed, I immediately (but not automatically) compile it underneath my dev database - after all, compile-check-install only takes a couple of minutes. So really, the automaton is me ;-) Because of the errors, I assumed that an initdb is needed and therefore a catversion bump was missing. But that is guesswork on my part; maybe there is something else. I suppose a non-initdb recompile could be automated on the basis of these two: catalog_version=$( grep CATALOG_VERSION_NO $dir/src/include/catalog/catversion.h | cut -f 2 ); and control_version=$( grep -h define.PG_CONTROL_VERSION $dir/src/include/catalog/pg_control.h | cut -f 2 ); where $dir points to the src tree at hand. So I have no further test-suite, I'm afraid, and the monster bash-script that compiles instances from patches is too horrible to inflict on others... Thanks, Erik Rijkers