GiST memory allocation - Mailing list pgsql-hackers

From Neil Conway
Subject GiST memory allocation
Date
Msg-id 4186490D.2020506@samurai.com
Whole thread Raw
Responses Re: GiST memory allocation  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
Memory allocation in access/gist/gist.c is pretty heinous, IMHO. There 
are retail pallocs and pfrees all over the place, and the requirements 
for which allocations need to be released and by whom is pretty messy. 
AFAICS, GiST doesn't take any advantage of the palloc() infrastructure 
beyond treating palloc() as a better malloc().

One relatively easy fix would be to create a per-backend "GiST context" 
as a static variable in gist.c. At the external entry points to gist.c 
(of which there are only three, gistbuild(), gistinsert(), and 
gistbulkdelete()), the memory context would be initialized if necessary 
and switched into. palloc() will still be used to allocate memory, but 
most/all of the retail pfrees can be eliminated; instead, we can reset 
the private memory context before returning to the caller of the GiST 
entry point (the observation here is that 99% of the allocations done by 
gist.c are for internal use only -- we rarely allocate anything that 
needs to live longer than the current GiST API call). Additional resets 
can be done where appropriate to reduce memory consumption -- for 
example, we will definitely need a reset in gistbuildCallback().

I haven't looked that carefully at the various clients of the GiST API 
(e.g. contrib/btree_gist and so on), but I think this should simplify 
them as well: rather than needing to ensure they pfree all their 
allocations, they can now assume that they are invoked in a relatively 
short-lived memory context.

One downside is that the GiST API won't be reentrant, but I don't think 
that's a major problem.

Comments? Is there a better way to clean up GiST's memory allocation?

(I looked primarily at gist.c; from a brief glance at gistscan.c, it 
doesn't seem nearly so bad.)

-Neil


pgsql-hackers by date:

Previous
From: Thomas Hallgren
Date:
Subject: Re: charset/collation in values
Next
From: Tom Lane
Date:
Subject: Re: psql and schemas