Allow table AM's to cache stuff in relcache - Mailing list pgsql-hackers

From Heikki Linnakangas
Subject Allow table AM's to cache stuff in relcache
Date
Msg-id dc0a5fe7-9adc-84f7-40e8-a3cde9529a5f@iki.fi
Whole thread Raw
Responses Re: Allow table AM's to cache stuff in relcache
List pgsql-hackers
Index AM's can cache stuff in RelationData->rd_amcache. In the zedstore 
table AM we've been hacking on, I'd like to also use rd_amcache to cache 
some information, but that's not currently possible, because rd_amcache 
can only be used by index AMs, not table AMs.

Attached patch allows rd_amcache to also be used by table AMs.

While working on this, I noticed that the memory management of relcache 
entries is quite complicated. Most stuff that's part of a relcache entry 
is allocated in CacheMemoryContext. But some fields have a dedicated 
memory context to hold them, like rd_rulescxt for rules and rd_pdcxt for 
partition information. And indexes have rd_indexcxt to hold all kinds of 
index support info.

In the patch, I documented that rd_amcache must be allocated in 
CacheMemoryContext, or in rd_indexcxt if it's an index. It works, but 
it's a bit weird. It would nice to have one memory context in every 
relcache entry, to hold all the stuff related to it, including 
rd_amcache. In other words, it would be nice if we had "rd_indexcxt" for 
tables, too, not just indexes. That would allow tracking memory usage 
more accurately, if you're debugging an out of memory situation for example.

However, the special contexts like rd_rulescxt and rd_pdcxt would still 
be needed, because of the way RelationClearRelation preserves them, when 
rebuilding the relcache entry for an open relation. So I'm not sure how 
much it would really simplify things. Also, there's some overhead for 
having extra memory contexts, and some people already complain that the 
relcache uses too much memory.

Alternatively, we could document that rd_amcache should always be 
allocated in CacheMemoryContext, even for indexes. That would make the 
rule for pg_amcache straightforward. There's no particular reason why 
rd_amcache has to be allocated in rd_indexcxt, except for how it's 
accounted for in memory context dumps.

- Heikki

Attachment

pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: pg_dump multi VALUES INSERT
Next
From: Tom Lane
Date:
Subject: Re: Allow table AM's to cache stuff in relcache