Re: Online checksums verification in the backend - Mailing list pgsql-hackers

From Andres Freund
Subject Re: Online checksums verification in the backend
Date
Msg-id 20201029191045.6mvdab3ae3nnpiai@alap3.anarazel.de
Whole thread Raw
In response to Re: Online checksums verification in the backend  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
Hi,

On 2020-10-29 11:17:29 -0700, Andres Freund wrote:
> The code does IO while holding the buffer mapping lock. That seems
> *entirely* unacceptable to me. That basically locks 1/128 of shared
> buffers against concurrent mapping changes, while reading data that is
> likely not to be on disk?  Seriously?

Also, uh, I don't think the locking of the buffer table provides you
with the full guarantees CheckBuffer() seems to assume:

 * Check the state of a buffer without loading it into the shared buffers. To
 * avoid torn pages and possible false positives when reading data, a shared
 * LWLock is taken on the target buffer pool partition mapping, and we check
 * if the page is in shared buffers or not.  An I/O lock is taken on the block
 * to prevent any concurrent activity from happening.

this doesn't actually prevent all concurrent write IO, unless you hold
an appropriate lock on the relation. There's a few places that use
smgrwrite()/smgrextend() to write out data bypassing shared buffers.

Maybe that isn't a problem for the uses of CheckBuffer() is envisioned
for, but that'd need a pretty detailed explanation as to when it's safe
to use CheckBuffer() for which blocks.

Greetings,

Andres Freund



pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: Online checksums verification in the backend
Next
From: Michael Banck
Date:
Subject: Re: [PATCH] remove pg_archivecleanup and pg_standby