Re: Proposal for Updating CRC32C with AVX-512 Algorithm. - Mailing list pgsql-hackers

From Nathan Bossart
Subject Re: Proposal for Updating CRC32C with AVX-512 Algorithm.
Date
Msg-id Z1tgSblZeRgrL9U3@nathan
Whole thread Raw
In response to Re: Proposal for Updating CRC32C with AVX-512 Algorithm.  (Andres Freund <andres@anarazel.de>)
Responses Re: Proposal for Updating CRC32C with AVX-512 Algorithm.
List pgsql-hackers
On Thu, Dec 12, 2024 at 10:45:29AM -0500, Andres Freund wrote:
> Frankly, we should just move away from using CRCs. They're good for cases
> where short runs of bit flips are much more likely than other kinds of errors
> and where the amount of data covered by them has a low upper bound. That's not
> at all the case for WAL records. It'd not matter too much if CRCs were cheap
> to compute - but they aren't.  We should instead move to some more generic
> hashing algorithm, decent ones are much faster.

Upthread [0], I wondered aloud about trying to reuse the page checksum code
for this.  IIRC there was a lot of focus on performance when that was
added, and IME it catches problems decently well.

[0] https://postgr.es/m/ZrUcX2kq-0doNBea%40nathan

-- 
nathan



pgsql-hackers by date:

Previous
From: Michael Harris
Date:
Subject: Re: FileFallocate misbehaving on XFS
Next
From: Sami Imseih
Date:
Subject: Re: Controlling the usage of a user-defined cast