Traffic jams in fn_extra - Mailing list pgsql-hackers

From Paul Ramsey
Subject Traffic jams in fn_extra
Date
Msg-id E6DC7D1083A346EEA9A2448F5ED35223@cleverelephant.ca
Whole thread Raw
Responses Re: Traffic jams in fn_extra  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
As an extension with a lot of CPU load, we (postgis) tend to use flinfo->fn_extra a lot, for caching things that are
intensiveto calculate at the start of a query and reuse throughout subsequent functions calls. 
 

- coordinate projection objects
- indexes of the edges of large geometries
- other kinds of indexes of the edges of large geometries :)
- schemas of lidar pointcloud collections

As we've added different kinds of caching, in our own project, we've banged up against problems of multiple functions
tryingto stuff information into the same pointer, and ended up putting an extra container of our own into fn_extra, to
holdthe different kinds of stuff we might want to store, a GenericCacheCollection
 

https://github.com/postgis/postgis/blob/svn-trunk/libpgcommon/lwgeom_cache.c#L46-L48

As (by now) a connoisseur of fn_extra caching, I've noticed while reading bits of PgSQL code that there are far more
placesthat stuff state into fn_extra than I ever knew, and that they do so without any substantial concern that other
statemight already be there. (Well, that's not true, they usually check for NULL and they give up if fn_extra is
alreadyin use.) The range types, I was surprised to find doing some performance caching in fn_extra. The set-returning
functionmacros use it to hold state. And many others I'm sure.
 

Would it be good/wise to add another, better managed, slot to flinfo, one that isn't just void* but is a hash? (Or, has
somemanagement macros to handle it and make it a hash* if it's null, whatever API makes sense) so that multiple bits of
codecan cache state over function calls without banging into one another?
 

flinfo->fn_extra_hash perhaps?

If this sounds OK, I'd be honored to try and make it my first submission to PgSQL.

P. 

-- 
Paul Ramsey
http://cleverelephant.ca
http://postgis.net





pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: better atomics - v0.2
Next
From: Alexander Korotkov
Date:
Subject: Re: GIN improvements part2: fast scan