Thread: index access method documentation light on details on ii_AmCache

The following documentation comment has been logged on the website:

Page: https://www.postgresql.org/docs/15/index-functions.html
Description:

So, if I cache something in ii_AmCache during a call to my aminsert
callback...

When, if ever, does it get freed?

Having looked at example code, I don't actually see anything doing this in
insert paths, so presumably there's some point at which this happens
automatically, possibly as part of the Memory Context thing, maybe related
to the ii_Context which seems to be getting used, but I can't find anything
anywhere documenting that. This may well be completely obvious, or intended
to be implied by "it can allocate space in indexInfo->ii_Context", but it's
not exceptionally obvious to me as a newcomer to the code. (By contrast, the
ambuild docs say to palloc a data structure, but don't mention a context for
it; no idea whether it should be in a particular context.)

Actually, in full generality, I have not been able to find a section of the
documentation which explains the memory-context stuff at all. I found a blog
post elsewhere suggesting that it's just "the memory context will be freed
and thus everything associated with it". This implies that there's no
straightforward way for an index to do end-of-insert maintenance after all
the inserts from a given query are complete, except to do it after every
tuple just in case it's the last tuple, I guess?

The following documentation comment has been logged on the website:

Page: https://www.postgresql.org/docs/15/index-functions.html

Description:

So, if I cache something in ii_AmCache during a call to my aminsert

callback...

When, if ever, does it get freed?

Having looked at example code, I don't actually see anything doing this in

insert paths, so presumably there's some point at which this happens

automatically, possibly as part of the Memory Context thing, maybe related

to the ii_Context which seems to be getting used, but I can't find anything

anywhere documenting that. This may well be completely obvious, or intended

to be implied by "it can allocate space in indexInfo->ii_Context", but it's

not exceptionally obvious to me as a newcomer to the code. (By contrast, the

ambuild docs say to palloc a data structure, but don't mention a context for

it; no idea whether it should be in a particular context.)

Actually, in full generality, I have not been able to find a section of the

documentation which explains the memory-context stuff at all. I found a blog

post elsewhere suggesting that it's just "the memory context will be freed

and thus everything associated with it". This implies that there's no

straightforward way for an index to do end-of-insert maintenance after all

the inserts from a given query are complete, except to do it after every

tuple just in case it's the last tuple, I guess?

On Wed, 13 Sep 2023 15:48:41 -0300
"Euler Taveira" <euler@eulerto.com> wrote:

> Unless you change it, ii_Context is CurrentMemoryContext (see
> makeIndexInfo). Hence, your AM state information is freed when the
> current memory context is freed.

A thing I am now wondering:

Is there anything in the postgresql documentation which explains this,
or says when a given memory context will be freed? Like, is the
ii_Context there for the lifetime of the index? For the lifetime of the
query? For the lifetime of a single insert operation?

I've found a reference to it in a third-party blog post, but I actually
can't find anything in the docs explaining memory contexts, and a bit
of experimenting makes me think they are doing things that aren't
obvious to me. For instance, I tried *not* deleting a memory context
when done with it, then looping doing a query which tried to create and
populate it, and... memory usage did not go up. Making me think that
maybe it gets reused? Or maybe it's implicitly deleted because it had
a parent context?

-s

On Wed, 13 Sep 2023 15:48:41 -0300

"Euler Taveira" <euler@eulerto.com> wrote:

> Unless you change it, ii_Context is CurrentMemoryContext (see

> makeIndexInfo). Hence, your AM state information is freed when the

> current memory context is freed.

A thing I am now wondering:

Is there anything in the postgresql documentation which explains this,

or says when a given memory context will be freed? Like, is the

ii_Context there for the lifetime of the index? For the lifetime of the

query? For the lifetime of a single insert operation?

On Thu, 14 Sep 2023 10:06:03 -0300
"Euler Taveira" <euler@eulerto.com> wrote:

> AFAICS there isn't a chapter dedicated to memory contexts in the
> documentation. Did you check the README that I pointed out in the
> previous email? Most of the developer information is available in
> README files in the source code. Server Programming and Internals
> contain useful information for Postgres hackers too.

... I did not check that out, I will go look at the readmes. The
documentation had enough documentation on things like "here's the list
of functions to implement" that I hadn't gone looking for other
documentation.

Thanks!

-s