pgsql: Allow I/O reliability checks using 16-bit checksums - Mailing list pgsql-committers

From Simon Riggs
Subject pgsql: Allow I/O reliability checks using 16-bit checksums
Date
Msg-id E1UJ2T8-0008Cm-W3@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Allow I/O reliability checks using 16-bit checksums

Checksums are set immediately prior to flush out of shared buffers
and checked when pages are read in again. Hint bit setting will
require full page write when block is dirtied, which causes various
infrastructure changes. Extensive comments, docs and README.

WARNING message thrown if checksum fails on non-all zeroes page;
ERROR thrown but can be disabled with ignore_checksum_failure = on.

Feature enabled by an initdb option, since transition from option off
to option on is long and complex and has not yet been implemented.
Default is not to use checksums.

Checksum used is WAL CRC-32 truncated to 16-bits.

Simon Riggs, Jeff Davis, Greg Smith
Wide input and assistance from many community members. Thank you.

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/96ef3b8ff1cf1950e897fd2f766d4bd9ef0d5d56

Modified Files
--------------
contrib/pg_upgrade/controldata.c          |   32 ++++-
contrib/pg_upgrade/pg_upgrade.h           |    1 +
doc/src/sgml/config.sgml                  |   24 ++++
doc/src/sgml/ref/initdb.sgml              |   14 ++
src/backend/access/gist/gistget.c         |    8 +-
src/backend/access/hash/hash.c            |    6 +-
src/backend/access/heap/heapam.c          |   96 +++++++++-----
src/backend/access/heap/pruneheap.c       |    2 +-
src/backend/access/heap/rewriteheap.c     |    4 +
src/backend/access/heap/visibilitymap.c   |   48 +++++--
src/backend/access/nbtree/nbtinsert.c     |   10 +-
src/backend/access/nbtree/nbtree.c        |    3 +-
src/backend/access/nbtree/nbtsort.c       |    3 +
src/backend/access/nbtree/nbtutils.c      |    6 +-
src/backend/access/rmgrdesc/xlogdesc.c    |    4 +
src/backend/access/spgist/spginsert.c     |    3 +
src/backend/access/transam/README         |   22 +++
src/backend/access/transam/xlog.c         |  111 ++++++++++++++-
src/backend/bootstrap/bootstrap.c         |    7 +-
src/backend/commands/matview.c            |    2 +
src/backend/commands/sequence.c           |    2 +-
src/backend/commands/tablecmds.c          |    2 +
src/backend/commands/vacuumlazy.c         |   14 +-
src/backend/storage/buffer/bufmgr.c       |  214 +++++++++++++++++++++++-----
src/backend/storage/buffer/localbuf.c     |    7 +-
src/backend/storage/freespace/README      |    4 +-
src/backend/storage/freespace/freespace.c |   10 +-
src/backend/storage/freespace/fsmpage.c   |    2 +-
src/backend/storage/page/bufpage.c        |  183 ++++++++++++++++++++++--
src/backend/utils/misc/guc.c              |   16 ++
src/backend/utils/time/tqual.c            |    4 +-
src/bin/initdb/initdb.c                   |   19 ++-
src/bin/pg_controldata/pg_controldata.c   |    2 +
src/bin/pg_resetxlog/pg_resetxlog.c       |    2 +
src/include/access/heapam_xlog.h          |    2 +-
src/include/access/visibilitymap.h        |    4 +-
src/include/access/xlog.h                 |    3 +
src/include/catalog/pg_control.h          |    8 +-
src/include/storage/bufmgr.h              |    3 +-
src/include/storage/bufpage.h             |    5 +-
40 files changed, 766 insertions(+), 146 deletions(-)


pgsql-committers by date:

Previous
From: Andrew Dunstan
Date:
Subject: pgsql: Silence compiler warnings about unused values.
Next
From: Simon Riggs
Date:
Subject: pgsql: Add new README file for pages/checksums