Re: Zstandard support for toast compression - Mailing list pgsql-hackers

From Robert Haas
Subject Re: Zstandard support for toast compression
Date
Msg-id CA+TgmoZm07cwAxu2mQmPn3Grvw7n7nS5J_H=gk0W=hXVyQZ5JA@mail.gmail.com
Whole thread Raw
In response to Re: Zstandard support for toast compression  (Michael Paquier <michael@paquier.xyz>)
List pgsql-hackers
On Mon, May 23, 2022 at 12:33 AM Michael Paquier <michael@paquier.xyz> wrote:
> On Thu, May 19, 2022 at 04:12:01PM -0400, Robert Haas wrote:
> > On Thu, May 19, 2022 at 4:20 AM Michael Paquier <michael@paquier.xyz> wrote:
> >> Btw, shouldn't we have something a bit more, err, extensible for the
> >> design of an extensible varlena header?  If we keep it down to some
> >> bitwise information, we'd be fine for a long time but it would be
> >> annoying to review again an extended design if we need to extend it
> >> with more data.
> >
> > What do you have in mind?
>
> A per-varlena checksum was one thing that came into my mind.

It's a bit hard for me to believe that such a thing would be
desirable. I think it makes more sense to checksum blocks than datums,
because:

(1) There might be a lot of really small datums, and storing checksums
for all of them could be costly, or
(2) The datums could on the other hand be really big, and then the
checksum is pretty non-specific about where the problem has happened.

YMMV, of course.

-- 
Robert Haas
EDB: http://www.enterprisedb.com



pgsql-hackers by date:

Previous
From: Jan Wieck
Date:
Subject: Re: Limiting memory allocation
Next
From: Robert Haas
Date:
Subject: Re: Zstandard support for toast compression