Thread: Problems with SPI memory management

Hi!
I posted the following message to the general list but no answer.Could
you please help?

I have some problems with the SPI memory management (at least I think
this is the problem).

What I'm trying to do is to get the Query related to a select statement,
alter it and produce a new SPI_plan that will execute. To do so, I
retrieve the query from the _SPI_plan->qtlist, alter it (seems OK in
nodeToString) and then use some SPI functions copied in my code to
produce the new _SPI_plan. In more details, I begin a new my_SPI
session, call my_SPI_prepare and pass the query. The code is almost the
same with that in spi.c apart from that I do not call pg_parse and
pg_analyze_and_rewrite.Then I begin a new SPI session and pass the new
plan.

Now, the problem:

Everything works fine only when I execute my function just after I have
initiated a new psql session. When I try to call it again, the Query
structure inside the new plan seems empty (all fields shown by
nodeToString seem to have no values). Even if I reload my shared library
(load command),the same problem appears. I allocate memory for the Query
object with palloc() outside a SPI or my_SPI session.

Any hints?

Thanks in advance!
Ntinos Katsaros

PS: I'm not very good at C!

"Katsaros Kwn/nos" <ntinos@aueb.gr> writes:
> What I'm trying to do is to get the Query related to a select statement,
> alter it and produce a new SPI_plan that will execute. To do so, I
> retrieve the query from the _SPI_plan->qtlist, alter it (seems OK in
> nodeToString) and then use some SPI functions copied in my code to
> produce the new _SPI_plan. In more details, I begin a new my_SPI
> session, call my_SPI_prepare and pass the query. The code is almost the
> same with that in spi.c apart from that I do not call pg_parse and
> pg_analyze_and_rewrite.Then I begin a new SPI session and pass the new
> plan.

What do you mean by "begin a new SPI session" --- do you SPI_finish()
and SPI_connect()?  (If so, why bother?)  I think that SPI_finish()
would release the memory in which the previous plan existed.  If you're
not copying the entire parse tree this would lead to trouble.

You'd be well advised to be doing this sort of hackery in a build with
--enable-cassert.  That turns on CLOBBER_FREED_MEMORY which makes
misuse of freed memory a whole lot more obvious.
        regards, tom lane

On Wed, 2004-09-15 at 19:51, Tom Lane wrote:
> You'd be well advised to be doing this sort of hackery in a build with
> --enable-cassert.  That turns on CLOBBER_FREED_MEMORY which makes
> misuse of freed memory a whole lot more obvious.

I did this but when I try do create a function the following message is
displayed:

" ERROR:  could not load library
"/usr/local/pgsql2/PLANSandTESTS/libtimes.so":
/usr/local/pgsql2/PLANSandTESTS/libtimes.so: undefined symbol:
assert_enabled "

Any ideas? Is there any file I should include during compilation? I
couldn't find any HOW-TO or something.

Regards,
Ntinoas Katsaros

"Katsaros Kwn/nos" <ntinos@aueb.gr> writes:
> On Wed, 2004-09-15 at 19:51, Tom Lane wrote:
>> You'd be well advised to be doing this sort of hackery in a build with
>> --enable-cassert.  That turns on CLOBBER_FREED_MEMORY which makes
>> misuse of freed memory a whole lot more obvious.

> I did this but when I try do create a function the following message is
> displayed:

> " ERROR:  could not load library
> "/usr/local/pgsql2/PLANSandTESTS/libtimes.so":
> /usr/local/pgsql2/PLANSandTESTS/libtimes.so: undefined symbol:
> assert_enabled "

You need to rebuild the whole backend, not just your function.

            regards, tom lane

On Wed, 2004-09-15 at 19:51, Tom Lane wrote:
> "Katsaros Kwn/nos" <ntinos@aueb.gr> writes:
> > What I'm trying to do is to get the Query related to a select statement,
> > alter it and produce a new SPI_plan that will execute. To do so, I
> > retrieve the query from the _SPI_plan->qtlist, alter it (seems OK in
> > nodeToString) and then use some SPI functions copied in my code to
> > produce the new _SPI_plan. In more details, I begin a new my_SPI
> > session, call my_SPI_prepare and pass the query. The code is almost the
> > same with that in spi.c apart from that I do not call pg_parse and
> > pg_analyze_and_rewrite.Then I begin a new SPI session and pass the new
> > plan.
> 
> What do you mean by "begin a new SPI session" --- do you SPI_finish()
> and SPI_connect()?  (If so, why bother?)  I think that SPI_finish()
> would release the memory in which the previous plan existed.  If you're
> not copying the entire parse tree this would lead to trouble.

After all these, I managed to transform a query like 
"select * from t1 where age>8 and age<50" into an even simpler:
"select * from t1 where age>8". This works fine (big deal....:-) ).

The new problem is when I try to alter a select statement like this:
"select t1.name,t2.age from t1,t2 where t1.name=t2.name and t2.age>2".
Even if I don't try to alter the query structure ,by just giving the old
Query to the planner, the system crashes (signal 11, something goes
wrong with memory references) instead of returning the same plan back.
The strange thing is that with "debug_print_plan = true" the planner
returns a plan completly different than expected (i.e. returned by
explain verbose) and then crashes.

Any ideas...?

Regards,
Ntinos Katsaros