Re: About Custom Aggregates, C Extensions and Memory - Mailing list pgsql-hackers

From Tom Lane
Subject Re: About Custom Aggregates, C Extensions and Memory
Date
Msg-id 608868.1755272106@sss.pgh.pa.us
Whole thread Raw
In response to About Custom Aggregates, C Extensions and Memory  (Marthin Laubscher <postgres@lobeshare.co.za>)
Responses Re: About Custom Aggregates, C Extensions and Memory
List pgsql-hackers
Marthin Laubscher <postgres@lobeshare.co.za> writes:
> A custom aggregate seems the best vehicle for what I seek to implement. Given the processing involved, it’s probably
bestwritten in C. 
> That makes the aggregate and opaque value encoded and compressed to an internal format that allows direct equality
testingand comparison. For everything else it needs to be decoded into memory, worked on and then encoded into a value
asexpected by the database ecosystem.  
> The challenge being that decoding and encoding presents a massive overhead (easily 2 orders of magnitude or more)
comparedto the lightning fast operations to e.g. add or remove a value from the aggregate while in memory, killing
performanceand limiting potential. 

Yeah.  What you want is to declare the aggregate as having transtype
"internal" (which basically means that ExecAgg will store a pointer
for you) and make that pointer point to a data structure kept in the
"aggcontext", which will have a suitable lifespan.  json_agg() might
be a suitable example to look at.  Keep in mind that the finalfn
mustn't modify the stored state, as there are optimizations where
it'll be applied more than once.

            regards, tom lane



pgsql-hackers by date:

Previous
From: Sami Imseih
Date:
Subject: Re: shmem_startup_hook called twice on Windows
Next
From: Sami Imseih
Date:
Subject: Re: shmem_startup_hook called twice on Windows