On Tue, 2006-03-14 at 22:06 -0800, maarten roosendaal wrote:
> We are currently having a problem that our Postgres DB
> is throwing an SQL error which states that it's 'out
> of memory'.
>
> What we have is a DB with 1 table that has 3.9 million
> records. We need to find certain records that are to
> be processed by a Java App so we do a "select id from
> table where type=a and condition in (1, 2) order by id
> limit 2000". When this query gets executed we see the
> memory on the DB Server increasing and after it has
> finishes it drops a bit but we see it growing a few MB
> per few minutes. This has caused an out of memory
> after the system has been processing for a day or 2.
> The query is heavy because of the order by but that
> does not explain why the memory is increasing.
Could be a memory leak in PG sort, so please explain further.
> We use a DAO which extends HibernateDaoSupport and the
> method (findIds) has been marked as
> propagation_required. So we assume Spring manages the
> transaction and thus closing of the resultset. Here's
> part of the DAO method: Query q =
> getSession(false).createQuery(query);
> q.setMaxResults(RESULT_SIZE);
> List list = q.list();
>
> No rocketscience but somehow this causes a problem on
> the DB Server.
>
> Does anyone have an idea where to look?
The java backtrace mentions bulk update. Where does that fit into this?
You've got a rather large stack of software there and you need to
isolate the problem. If you are running multiple system components on
one system then it is possible that a memory leak in one component can
cause a problem in another. A memory intensive task such as sort would
then be likely to highlight the problem, but that doesn't mean its the
cause of the leak.
If you can reproduce this problem with a simple repetitive test case
using a script executing psql, that will help. If you cannot, then that
points to a failure in another system component.
Right now, you've not told us much about the query, what release level
you are using etc. Are you staying connected and re-issuing the same
command repeatedly or? We'll need to know more about the memory growth:
which processes does it effect.
We'll also need to know about any custom datatypes involved, or any
parts of the query executing user developed code.
Best Regards, Simon Riggs
