Re: Extensible storage manager API - SMGR hook Redux - Mailing list pgsql-hackers

From Kirill Reshke
Subject Re: Extensible storage manager API - SMGR hook Redux
Date
Msg-id CALdSSPiDcYuxiFVQ1=968bCrb_NdQABg+QYPpzgSG-cZ3tn35w@mail.gmail.com
Whole thread Raw
In response to Re: Extensible storage manager API - SMGR hook Redux  (Matthias van de Meent <boekewurm+postgres@gmail.com>)
Responses Re: Extensible storage manager API - SMGR hook Redux
List pgsql-hackers


On Mon, 4 Dec 2023 at 22:21, Matthias van de Meent <boekewurm+postgres@gmail.com> wrote:
On Mon, 4 Dec 2023 at 17:51, Kirill Reshke <reshkekirill@gmail.com> wrote:
>
> So, 0002 patch uses the `get_tablespace` function, which searches Catalog to tablespace SMGR id. I wonder how `smgr_redo` would work with it?

That's a very good point I hadn't considered in detail yet. Quite
clearly, the current code is wrong in assuming that the catalog is
accessible, and it should probably be stored in a way similar to
pg_filenode.map in a file managed outside the buffer pool.

Hmm, pg_filenode.map  is a nice idea. So, simply maintain TableSpaceOId -> smgr id mapping in a separate file and update the whole file on any changes, right?
Looks reasonable to me, but it is clear that this solution can be really slow in some patterns, like if we create many-many tablespaces(the way you suggested it in the per-relation SMGR feature). Maybe we can store data in files somehow separately, and only update one chunk per operation.

Anyway, if we use a `pg_filenode.map` - like solution, we need to reuse its code infrasture, right? For example, it seems that code that calculates checksums can be reused.
So, we need to refactor code here, define something like FileMap API maybe. Or is it not really worth it? We can just write similar code twice.

pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: trying again to get incremental backup
Next
From: Nathan Bossart
Date:
Subject: Re: optimize atomic exchanges