Re: [PATCHES] Post-special page storage TDE support - Mailing list pgsql-hackers

From David Christensen
Subject Re: [PATCHES] Post-special page storage TDE support
Date
Msg-id CAOxo6XKPWaPP+VaETgQiHbR51Tq5HAdWs6-hE6S9+WLi1U4qPA@mail.gmail.com
Whole thread Raw
In response to Re: [PATCHES] Post-special page storage TDE support  (Matthias van de Meent <boekewurm+postgres@gmail.com>)
Responses Re: [PATCHES] Post-special page storage TDE support
List pgsql-hackers
Per some offline discussion with Stephen and incorporating some of the
feedback I've gotten I'm including the following changes/revisions:

1. Change the signature of any macros that rely on a dynamic component
to look like a function so you can more easily determine in-code
whether something is truly a constant/compile time calculation or a
runtime one.

2. We use a new page flag for whether "extended page features" are
enabled on the given page.  If this is set then we look for the 1-byte
trailer with the bitflag of the number of features. We allow space for
7 page features and are reserving the final hi bit for future
use/change of interpretation to accommodate more.

3. Consolidate the extended checksums into a 56-bit checksum that
immediately precedes the 1-byte flag.  Choice of 64-bit checksum is
arbitrary just based on some MIT-licensed code I found, so just
considering this proof of concept, not necessarily promoting that
specific calculation.  (I think I included some additional checksum
variants from the earlier revision for ease of testing various
approaches.)

4. Ensure the whole area is MAXALIGN and fixed a few bugs that were
pointed out in this thread.

Patches are:

1. make select_views stable, prerequisite for anything that is messing
with tuples on page sizes

2. add reserved_page_size handling and rework existing code to account
for this additional space usage

3. main PageFeatures-related code; introduce that abstraction layer,
along with the trailing byte on the page with the enabled features for
this specific page.  We also add an additional param to PageInit()
with the page features active on this page; currently all call sites
are using the cluster-wide cluster_page_features as the parameter, so
all pages share what is stored in the control file based on initdb
options.  However, routines which query page features look on the
actual page itself, so in fact we are able to piecemeal at the
page/relation level if we so desire, or turn off for specific types of
pages, say. Also includes the additional pd_flags bit to enable that
interpretation.

4. Actual extended checksums PageFeature. Rather than two separate
implementations as in the previous patch series, we are using 56 bits
of a 64-bit checksum, stored as the high 7 bytes of the final 8 in the
page where this is enabled.

5. wasted_space PageFeature just to demo multiple features in play.

Thanks,

David

Attachment

pgsql-hackers by date:

Previous
From: Mark Dilger
Date:
Subject: Re: Tables not getting vacuumed in postgres
Next
From: Tom Lane
Date:
Subject: Non-emergency patch for bug #17679