Thread: Python interface memory leak?

Does the python interface leak memory? We are seeing the process grow
with basically every select. Any suggestions on what's going on? There
are no cycles, and a trivial program (basically a loop around a select)
demonstrates the problem.

This is 7.1.2 on RH7.[01].

    Stephen

Are you disposing the results of each select call? Not being familiar with
Python, as well as not knowing which interface for Python you are using, or
the code used to invoke the selects, I can only surmise that the result data
from each select is being stored in RAM and never garbage collected. At
least that could be responsible for significant memory jumps. For small
jumps, it could simply be the accumulation of multiple objects. So 1) you
may have to explicitly destroy result data and 2) you may need to ensure
that you are re-using the same object.

Without more info, any guess is as good as another.

Keary Suska
Esoteritech, Inc.
"Leveraging Open Source for a better Internet"

> From: Stephen Robert Norris <srn@commsecure.com.au>
> Date: 16 Oct 2001 22:00:35 +1000
> To: pgsql-general@postgresql.org
> Subject: [GENERAL] Python interface memory leak?
>
> Does the python interface leak memory? We are seeing the process grow
> with basically every select. Any suggestions on what's going on? There
> are no cycles, and a trivial program (basically a loop around a select)
> demonstrates the problem.
>
> This is 7.1.2 on RH7.[01].
>
> Stephen
>

At 16/10/01 14.00, you wrote:
>Does the python interface leak memory? We are seeing the process grow
>with basically every select. Any suggestions on what's going on? There
>are no cycles, and a trivial program (basically a loop around a select)
>demonstrates the problem.
>
>This is 7.1.2 on RH7.[01].
>
>         Stephen

The python interface stores query datas in memory. If you don't clean such
datas, the used memory will grow up...

If this is not the case, try the PoPy postgres driver for Python
(http://popy.sourgeforge.net or
http://freshmeat.net/redir/popy/8258/url_tgz/PoPy-2.0.7.tar.gz ).

Regards,



Doct. Eng. Denis
Gasparin                                             denis@edistar.com
---------------------------------------------------------------------------------------
Programmer & System
Administrator                                http://www.edistar.com
---------------------------------------------------------------------------------------
                              Well alas we've seen it all before
                               Knights in armour, days of yore
                          The same old fears and the same old crimes
                            We haven't changed since ancient times

                               -- Iron Hand -- Dire Straits --
---------------------------------------------------------------------------------------

On Wed, 2001-10-17 at 16:56, Denis Gasparin wrote:
> At 16/10/01 14.00, you wrote:
> >Does the python interface leak memory? We are seeing the process grow
> >with basically every select. Any suggestions on what's going on? There
> >are no cycles, and a trivial program (basically a loop around a select)
> >demonstrates the problem.
> >
> >This is 7.1.2 on RH7.[01].
> >
> >         Stephen
>
> The python interface stores query datas in memory. If you don't clean such
> datas, the used memory will grow up...

Actually, it's a bug. I spent most of today with a co-worker looking for
it. There's a Py_DECREF missing in the fetch routine, which makes it
leak the results of every fetch. Who do I send the (one-line) fix to?

    Stephen

Stephen Robert Norris <srn@commsecure.com.au> writes:
> Actually, it's a bug. I spent most of today with a co-worker looking for
> it. There's a Py_DECREF missing in the fetch routine, which makes it
> leak the results of every fetch. Who do I send the (one-line) fix to?

pgsql-patches list.

            regards, tom lane