Thread: Re: [GENERAL] Memory Errors...
"Ian Harding" <ianh@tpchd.org> writes:
> It is pltcl [not plpgsql]
Ah. I don't think we've done much of any work on plugging leaks in
pltcl :-(.
> It hurts when I do this:
> drop function memleak();
> create function memleak() returns int as '
> for {set counter 1} {$counter < 100000} {incr counter} {
> set sql "select ''foo''"
> spi_exec "$sql"
> }
> ' language 'pltcl';
> select memleak();
Yeah, I see very quick memory exhaustion also :-(. Looks like the
spi_exec call is the culprit, but I'm not sure exactly why ...
anyone have time to look at this?
regards, tom lane
I said:
> Yeah, I see very quick memory exhaustion also :-(. Looks like the
> spi_exec call is the culprit, but I'm not sure exactly why ...
> anyone have time to look at this?
On looking a little more closely, it's clear that pltcl_SPI_exec()
should be, and is not, calling SPI_freetuptable() once it's done with
the tuple table returned by SPI_exec(). This needs to be done in all
the non-elog code paths after SPI_exec has returned SPI_OK_SELECT.
pltcl_SPI_execp() has a similar problem, and there may be comparable
bugs in other pltcl routines (not to mention other sources of memory
leaks, but I think this is the problem for your example).
I have no time to work on this right now; any volunteers out there?
regards, tom lane
Tom Lane wrote: > I said: > >>Yeah, I see very quick memory exhaustion also :-(. Looks like the >>spi_exec call is the culprit, but I'm not sure exactly why ... >>anyone have time to look at this? > > > On looking a little more closely, it's clear that pltcl_SPI_exec() > should be, and is not, calling SPI_freetuptable() once it's done with > the tuple table returned by SPI_exec(). This needs to be done in all > the non-elog code paths after SPI_exec has returned SPI_OK_SELECT. > pltcl_SPI_execp() has a similar problem, and there may be comparable > bugs in other pltcl routines (not to mention other sources of memory > leaks, but I think this is the problem for your example). > > I have no time to work on this right now; any volunteers out there? > I can give it a shot, but probably not until the weekend. I haven't really followed this thread closely, and don't know tcl very well, so it would help if someone can send me a minimal tcl function which triggers the problem. Thanks, Joe
> "Ian Harding" <ianh@tpchd.org> writes: > > It is pltcl [not plpgsql] Quick, minor point, in the manner of a question: Why is the pltcl directory called tcl where all the other pls are pl<language>? That's in src/pl of course. Also in my anoncvs fetch which is a few weeks old now being from the day before beta freeze. -- Nigel J. Andrews Director --- Logictree Systems Limited Computer Consultants
Nigel J. Andrews wrote: > > > "Ian Harding" <ianh@tpchd.org> writes: > > > It is pltcl [not plpgsql] > > Quick, minor point, in the manner of a question: > > Why is the pltcl directory called tcl where all the other pls are pl<language>? I asked the same question a while ago. I asked about changing it but others didn't want the change. It is hard to rename stuff in CVS and keep proper history. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
"Nigel J. Andrews" <nandrews@investsystems.co.uk> writes:
> Why is the pltcl directory called tcl where all the other pls are pl<language>?
Consistency? We don't need no steenking consistency!
Personally I'd prefer to remove the pl prefix from the other
subdirectories of src/pl/ ... it seems redundantly wasted excessive
typing ;-) And I'd have preferred to flatten out the src/ subdirectory
of src/pl/[pl]pgsql, which is likewise redundant and inconsistent with
the other PLs.
However, it's fairly painful to make any such change without losing
the CVS version history for the moved files, which is Not a Good Thing.
Or breaking our ability to reconstitute old releases from the CVS tree,
which is Much Worse. So I'm afraid we're stuck with this historical
mischance.
regards, tom lane
On Thu, 19 Sep 2002, Joe Conway wrote: > Tom Lane wrote: > > I said: > > > >>Yeah, I see very quick memory exhaustion also :-(. Looks like the > >>spi_exec call is the culprit, but I'm not sure exactly why ... > >>anyone have time to look at this? > > > > > > On looking a little more closely, it's clear that pltcl_SPI_exec() > > should be, and is not, calling SPI_freetuptable() once it's done with > > the tuple table returned by SPI_exec(). This needs to be done in all > > the non-elog code paths after SPI_exec has returned SPI_OK_SELECT. > > pltcl_SPI_execp() has a similar problem, and there may be comparable > > bugs in other pltcl routines (not to mention other sources of memory > > leaks, but I think this is the problem for your example). > > > > I have no time to work on this right now; any volunteers out there? > > > > I can give it a shot, but probably not until the weekend. > > I haven't really followed this thread closely, and don't know tcl very well, > so it would help if someone can send me a minimal tcl function which triggers > the problem. I can probably take a look at this tomorrow, already started by looking at the pltcl_SPI_exec routine. I think a quick glance at ...init_unknown() also shows a lack of tuptable freeing. -- Nigel J. Andrews
Nigel J. Andrews wrote: > On Thu, 19 Sep 2002, Joe Conway wrote: >>I can give it a shot, but probably not until the weekend. >> >>I haven't really followed this thread closely, and don't know tcl very well, >>so it would help if someone can send me a minimal tcl function which triggers >>the problem. > > I can probably take a look at this tomorrow, already started by looking at the > pltcl_SPI_exec routine. I think a quick glance at ...init_unknown() also shows > a lack of tuptable freeing. > OK -- let me know if you can't find the time and I'll jump back in to it. Joe
Tom Lane writes: > On looking a little more closely, it's clear that pltcl_SPI_exec() > should be, and is not, calling SPI_freetuptable() once it's done with > the tuple table returned by SPI_exec(). This needs to be done in all > the non-elog code paths after SPI_exec has returned SPI_OK_SELECT. There's a note in the PL/Python documentation that it's leaking memory if SPI plans are used. Maybe that's related and someone could take a look at it. -- Peter Eisentraut peter_e@gmx.net
I'll try to have a look-see by the end of the weekend. Any code that can reproduce it or is it ANY code that uses SPI? Greg On Fri, 2002-09-20 at 11:39, Peter Eisentraut wrote: > Tom Lane writes: > > > On looking a little more closely, it's clear that pltcl_SPI_exec() > > should be, and is not, calling SPI_freetuptable() once it's done with > > the tuple table returned by SPI_exec(). This needs to be done in all > > the non-elog code paths after SPI_exec has returned SPI_OK_SELECT. > > There's a note in the PL/Python documentation that it's leaking memory if > SPI plans are used. Maybe that's related and someone could take a look at > it. > > -- > Peter Eisentraut peter_e@gmx.net > > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster
On 20 Sep 2002, Greg Copeland wrote: > I'll try to have a look-see by the end of the weekend. Any code that > can reproduce it or is it ANY code that uses SPI? > > Greg > > > On Fri, 2002-09-20 at 11:39, Peter Eisentraut wrote: > > Tom Lane writes: > > > > > On looking a little more closely, it's clear that pltcl_SPI_exec() > > > should be, and is not, calling SPI_freetuptable() once it's done with > > > the tuple table returned by SPI_exec(). This needs to be done in all > > > the non-elog code paths after SPI_exec has returned SPI_OK_SELECT. > > > > There's a note in the PL/Python documentation that it's leaking memory if > > SPI plans are used. Maybe that's related and someone could take a look at > > it. I've added the call to free the tuptable just as in the pltcl patch I submited earlier (which I can't remember if I've seen in the list so I may well resend). However, the comments in the code imply there might be another leak with prepared plans. I'm looking into that so I won't be sending this patch just yet. -- Nigel J. Andrews
Well, it looks like it was already taken to the mat. ;) Greg On Thu, 2002-09-19 at 16:58, Joe Conway wrote: > Nigel J. Andrews wrote: > > On Thu, 19 Sep 2002, Joe Conway wrote: > >>I can give it a shot, but probably not until the weekend. > >> > >>I haven't really followed this thread closely, and don't know tcl very well, > >>so it would help if someone can send me a minimal tcl function which triggers > >>the problem. > > > > I can probably take a look at this tomorrow, already started by looking at the > > pltcl_SPI_exec routine. I think a quick glance at ...init_unknown() also shows > > a lack of tuptable freeing. > > > > OK -- let me know if you can't find the time and I'll jump back in to it. > > Joe > > > > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/users-lounge/docs/faq.html