Thread: Set Returning C-Function with cache over multiple calls (with different arguments)
Set Returning C-Function with cache over multiple calls (with different arguments)
From
Thilo Schneider
Date:
Dear list,
Currently I am working on a user C-Function which should create a cache object on the first call and afterwards return
aset of computed values for each argument combination it is called with.
My Problem is how to get the cache object saved over multiple calls. Without the SRF I could use
fcinfo->flinfo->fn_extrafor my pointer to the data. This is now used by the FuncCallContext structure. This structure
isdestroyed every time SRF_RETURN_DONE is called, thus user_fctx also is not the way to go.
As a minimal example look at the function provided below.
--------------------------------------------------- snip ---------------------------------------------------
PG_FUNCTION_INFO_V1(test);
Datum test(PG_FUNCTION_ARGS)
{
MemoryContext old_context;
FuncCallContext *funcctx;
if (SRF_IS_FIRSTCALL()) {
funcctx = SRF_FIRSTCALL_INIT();
// This is the structure potentially generated in previous calls
str = funcctx->user_fctx;
// If the structure does not exist or the geometry array has changed, it has to be created.
if ( ! str) {
elog(NOTICE, "create new");
old_context = MemoryContextSwitchTo(fcinfo->flinfo->fn_mcxt);
// Fill str with data around here ...
MemoryContextSwitchTo(old_context);
funcctx->user_fctx = str;
}
}
funcctx = SRF_PERCALL_SETUP();
SRF_RETURN_DONE(funcctx);
}
--------------------------------------------------- snip ---------------------------------------------------
To make the problem perfectly clear the SQL-Code this should work with:
--------------------------------------------------- snip ---------------------------------------------------
CREATE OR REPLACE FUNCTION test(int, int[] )
RETURNS SETOF int AS 'myfunc', 'test'
LANGUAGE 'c' IMMUTABLE STRICT
COST 1;
SELECT test(number, array(SELECT integers FROM another_table)) FROM numbers;
--------------------------------------------------- snip ---------------------------------------------------
As creating the cache object is by far the most expensive part in the desired function, it should be possible to create
thecache only once over the whole query - using only the arguments in the array, which do not change over multiple
calls.
Is there a way around this problem? Another pointer I could use and do not know of yet?
Thanks in advance,
Thilo Schneider
Re: Set Returning C-Function with cache over multiple calls (with different arguments)
From
Merlin Moncure
Date:
On Mon, Jan 11, 2010 at 2:45 AM, Thilo Schneider
<Thilo.Schneider@math.uni-giessen.de> wrote:
> Dear list,
>
> Currently I am working on a user C-Function which should create a cache object on the first call and afterwards
returna set of computed values for each argument combination it is called with.
>
> My Problem is how to get the cache object saved over multiple calls. Without the SRF I could use
fcinfo->flinfo->fn_extrafor my pointer to the data. This is now used by the FuncCallContext structure. This structure
isdestroyed every time SRF_RETURN_DONE is called, thus user_fctx also is not the way to go.
>
> As a minimal example look at the function provided below.
>
> --------------------------------------------------- snip ---------------------------------------------------
> PG_FUNCTION_INFO_V1(test);
> Datum test(PG_FUNCTION_ARGS)
> {
> MemoryContext old_context;
> FuncCallContext *funcctx;
>
>
> if (SRF_IS_FIRSTCALL()) {
> funcctx = SRF_FIRSTCALL_INIT();
>
> // This is the structure potentially generated in previous calls
> str = funcctx->user_fctx;
>
> // If the structure does not exist or the geometry array has changed, it has to be created.
> if ( ! str) {
> elog(NOTICE, "create new");
>
> old_context = MemoryContextSwitchTo(fcinfo->flinfo->fn_mcxt);
>
> // Fill str with data around here ...
>
> MemoryContextSwitchTo(old_context);
>
> funcctx->user_fctx = str;
> }
>
> }
> funcctx = SRF_PERCALL_SETUP();
> SRF_RETURN_DONE(funcctx);
> }
> --------------------------------------------------- snip ---------------------------------------------------
>
> To make the problem perfectly clear the SQL-Code this should work with:
>
> --------------------------------------------------- snip ---------------------------------------------------
> CREATE OR REPLACE FUNCTION test(int, int[] )
> RETURNS SETOF int AS 'myfunc', 'test'
> LANGUAGE 'c' IMMUTABLE STRICT
> COST 1;
>
> SELECT test(number, array(SELECT integers FROM another_table)) FROM numbers;
> --------------------------------------------------- snip ---------------------------------------------------
>
> As creating the cache object is by far the most expensive part in the desired function, it should be possible to
createthe cache only once over the whole query - using only the arguments in the array, which do not change over
multiplecalls.
>
> Is there a way around this problem? Another pointer I could use and do not know of yet?
have you ruled out simply keeping a static pointer around and using malloc()?
merlin
Re: Set Returning C-Function with cache over multiple calls (with different arguments)
From
Thilo Schneider
Date:
Dear list,
I solved my own problem - as so often, once you write it down and press the send button you get the idea.
The problem was:
> Currently I am working on a user C-Function which should create a cache object on the first call and afterwards
returna set of computed values for each argument combination it is called with.
>
> My Problem is how to get the cache object saved over multiple calls. Without the SRF I could use
fcinfo->flinfo->fn_extrafor my pointer to the data. This is now used by the FuncCallContext structure. This structure
isdestroyed every time SRF_RETURN_DONE is called, thus user_fctx also is not the way to go.
My solution:
--------------------------------------------------- snip ---------------------------------------------------
struct myData {
FuncCallContext *funcctx;
// own Data
int cachedObject;
} myData
PG_FUNCTION_INFO_V1(test);
Datum test(PG_FUNCTION_ARGS)
{
MemoryContext old_context;
FuncCallContext *funcctx;
myData *str;
// Get fn_extra
str = fcinfo->flinfo->fn_extra;
if ( ! str) {
elog(NOTICE, "create new");
old_context = MemoryContextSwitchTo(fcinfo->flinfo->fn_mcxt);
// Fill str with data around here ...
MemoryContextSwitchTo(old_context);
str->funcctx = NULL;
}
// This is the situation the SRF-macros expect:
fcinfo->flinfo->fn_extra = str->funcctx;
if (SRF_IS_FIRSTCALL()) {
funcctx = SRF_FIRSTCALL_INIT();
// Your commands
}
funcctx = SRF_PERCALL_SETUP();
// This is the macro SRF_RETURN_DONE(funcctx);
// Before we finally return we save our str in fn_extra and fn_extra in str->funcctx.
do {
ReturnSetInfo *rsi;
end_MultiFuncCall(fcinfo, funcctx);
rsi = (ReturnSetInfo *) fcinfo->resultinfo;
rsi->isDone = ExprEndResult;
// -- Modify macro here --
str->funcctx = fcinfo->flinfo->fn_extra;
fcinfo->flinfo->fn_extra = str;
// -- End modification --
PG_RETURN_NULL();
} while (0);
// Of course, SRF_RETURN_DATUM has to be adapted the same way!
}
--------------------------------------------------- snip ---------------------------------------------------
Regards,
Thilo Schneider