Re: "out of memory" error - Mailing list pgsql-general

Rachel,

Most likely your data set coming back is too large to process all at
once.  You need to set the fetch size on the statement to control the
number of records coming back per request for them.

ps.setFetchSize( 100 );  where ps is a prepared statement.  The result
set will use a cursor to only bring back 100 records at a time and will
automatically fetch the next hundred until the entire result set has
been brought back.

Make sure that your JDBC statements don't have a semi colon at the end
of them otherwise the setFetchSize will be ignored and the entire result
set will be returned at once.

--sean


Rachel McConnell wrote:

> Hello,
>
> I have a Java web application using Hibernate to connect to a
> PostgreSQL backend.  I am seeing the below stack trace during
> processing of a set of data consisting of around 1000 objects; for a
> 200 object set I don't see the exception.
>
> I believe the salient point is the Out Of Memory bit - marked below by
> *****'s
> The fact that this exception occurs during a call to
> org.postgresql.util.PSQLException.parseServerError() seems disturbing
> too.
>
> * the JVM or servlet container might be out of memory, however the db
> is on a different machine than the server so this doesn't seem likely
>
> * Postgres might be out of memory.  How would I check for this?
> Subsequent requests go through just fine, as do any requests I make
> directly using psql.
>
> * something Else is happening
>
> Any advice on how to attack this issue would be much appreciated!
>
> Rachel McConnell
>
>
> WARN  [     SystemErrorInterceptor ] - java.util.NoSuchElementException
> java.util.NoSuchElementException
>         at java.util.AbstractList$Itr.next(AbstractList.java:426)
>         at java.util.Collections$6.nextElement(Collections.java:2479)
>         at
> com.enginegreen.denali.interceptor.SystemErrorInterceptor.intercept(SystemErrorInterceptor.java:89)
>
>         at
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
>
>         at
> com.opensymphony.xwork.interceptor.TimerInterceptor.intercept(TimerInterceptor.java:62)
>
>         at
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
>
>         at
> com.opensymphony.xwork.DefaultActionProxy.execute(DefaultActionProxy.java:116)
>
>         at
> com.opensymphony.webwork.dispatcher.ServletDispatcher.serviceAction(ServletDispatcher.java:182)
>
>         at
> com.opensymphony.webwork.dispatcher.ServletDispatcher.service(ServletDispatcher.java:162)
>
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:103)
>         at
> com.caucho.server.http.FilterChainServlet.doFilter(FilterChainServlet.java:96)
>
>         at
> com.enginegreen.denali.chrome.filter.ChromeFilter.doFilter(ChromeFilter.java:57)
>
>         at
> com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88)
>
>         at
> com.enginegreen.denali.interceptor.HibernateFilter.doFilter(HibernateFilter.java:42)
>
>         at
> com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88)
>
>         at
> com.enginegreen.denali.action.DenaliFilter.doFilter(DenaliFilter.java:28)
>         at
> com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88)
>
>         at com.caucho.server.http.Invocation.service(Invocation.java:315)
>         at
> com.caucho.server.http.CacheInvocation.service(CacheInvocation.java:135)
>         at
> com.caucho.server.http.RunnerRequest.handleRequest(RunnerRequest.java:346)
>
>         at
> com.caucho.server.http.RunnerRequest.handleConnection(RunnerRequest.java:274)
>
>         at com.caucho.server.TcpConnection.run(TcpConnection.java:139)
>         at java.lang.Thread.run(Thread.java:534)
> WARN  [     SystemErrorInterceptor ] -
> com.enginegreen.denali.persistence.PersistenceException:
> net.sf.hibernate.JDBCException: could not initialize collection:
> [com.enginegreen.ledger.model.InvoiceSummary.orderSummarySet#78828]
>         at
> com.enginegreen.ledger.invoice.impl.BillingClerkImpl.renderInvoicesAsPDF(BillingClerkImpl.java:429)
>
>         at
> com.enginegreen.denali.domain.billing.BillingManagerImpl.finalizeNextBillingPeriod(BillingManagerImpl.java:248)
>
>         at
> com.enginegreen.denali.domain.billing.SecureBillingManager.finalizeNextBillingPeriod(SecureBillingManager.java:96)
>
>         at
> com.enginegreen.denali.action.billing.GenerateInvoicesAction.doExecute(GenerateInvoicesAction.java:121)
>
>         at
> com.enginegreen.denali.action.WorkflowActionSupport.execute(WorkflowActionSupport.java:65)
>
>         at
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:172)
>
>         at
> com.enginegreen.denali.interceptor.ActionSourceInterceptor.intercept(ActionSourceInterceptor.java:50)
>
>         at
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
>
>         at
> com.enginegreen.denali.interceptor.NullInterceptor.intercept(NullInterceptor.java:25)
>
>         at
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
>
>         at
> com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:37)
>
>         at
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
>
>         at
> com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:37)
>
>         at
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
>
>         at
> com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:37)
>
>         at
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
>
>         at
> com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:37)
>
>         at
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
>
>         at
> com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:37)
>
>         at
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
>
>         at
> com.enginegreen.denali.interceptor.security.AdminSecurityInterceptor.intercept(AdminSecurityInterceptor.java:31)
>
>         at
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
>
>         at
> com.enginegreen.denali.interceptor.SystemErrorInterceptor.intercept(SystemErrorInterceptor.java:47)
>
>         at
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
>
>         at
> com.opensymphony.xwork.interceptor.TimerInterceptor.intercept(TimerInterceptor.java:62)
>
>         at
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
>
>         at
> com.opensymphony.xwork.DefaultActionProxy.execute(DefaultActionProxy.java:116)
>
>         at
> com.opensymphony.webwork.dispatcher.ServletDispatcher.serviceAction(ServletDispatcher.java:182)
>
>         at
> com.opensymphony.webwork.dispatcher.ServletDispatcher.service(ServletDispatcher.java:162)
>
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:103)
>         at
> com.caucho.server.http.FilterChainServlet.doFilter(FilterChainServlet.java:96)
>
>         at
> com.enginegreen.denali.chrome.filter.ChromeFilter.doFilter(ChromeFilter.java:57)
>
>         at
> com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88)
>
>         at
> com.enginegreen.denali.interceptor.HibernateFilter.doFilter(HibernateFilter.java:42)
>
>         at
> com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88)
>
>         at
> com.enginegreen.denali.action.DenaliFilter.doFilter(DenaliFilter.java:28)
>         at
> com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88)
>
>         at com.caucho.server.http.Invocation.service(Invocation.java:315)
>         at
> com.caucho.server.http.CacheInvocation.service(CacheInvocation.java:135)
>         at
> com.caucho.server.http.RunnerRequest.handleRequest(RunnerRequest.java:346)
>
>         at
> com.caucho.server.http.RunnerRequest.handleConnection(RunnerRequest.java:274)
>
>         at com.caucho.server.TcpConnection.run(TcpConnection.java:139)
>         at java.lang.Thread.run(Thread.java:534)
> Caused by: net.sf.hibernate.JDBCException: could not initialize
> collection:
> [com.enginegreen.ledger.model.InvoiceSummary.orderSummarySet#78828]
>         at
> net.sf.hibernate.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:287)
>
>         at
> net.sf.hibernate.impl.SessionImpl.initializeCollection(SessionImpl.java:3226)
>
>         at
> net.sf.hibernate.collection.PersistentCollection.forceInitialization(PersistentCollection.java:340)
>
>         at
> net.sf.hibernate.impl.SessionImpl.initializeNonLazyCollections(SessionImpl.java:3089)
>
>         at
> net.sf.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:138)
>
>         at net.sf.hibernate.loader.Loader.doList(Loader.java:950)
>         at net.sf.hibernate.loader.Loader.list(Loader.java:941)
>         at
> net.sf.hibernate.hql.QueryTranslator.list(QueryTranslator.java:834)
>         at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:1512)
>         at net.sf.hibernate.impl.QueryImpl.list(QueryImpl.java:39)
>         at
> com.enginegreen.ledger.invoice.impl.BillingClerkImpl.renderInvoicesAsPDF(BillingClerkImpl.java:414)
>
>         ... 42 more
>
> ******
>
> Caused by: org.postgresql.util.PSQLException: ERROR: out of memory
>
>         at
> org.postgresql.util.PSQLException.parseServerError(PSQLException.java:139)
>
>         at
> org.postgresql.core.QueryExecutor.executeV3(QueryExecutor.java:154)
>         at
> org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:101)
>         at
> org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:43)
>         at
> org.postgresql.jdbc1.AbstractJdbc1Statement.execute(AbstractJdbc1Statement.java:515)
>
>         at
> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:50)
>
>         at
> org.postgresql.jdbc1.AbstractJdbc1Statement.executeQuery(AbstractJdbc1Statement.java:231)
>
>         at
> net.sf.hibernate.impl.BatcherImpl.getResultSet(BatcherImpl.java:87)
>         at net.sf.hibernate.loader.Loader.getResultSet(Loader.java:795)
>         at net.sf.hibernate.loader.Loader.doQuery(Loader.java:189)
>         at
> net.sf.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:133)
>
>         at net.sf.hibernate.loader.Loader.loadCollection(Loader.java:910)
>         at net.sf.hibernate.loader.Loader.loadCollection(Loader.java:885)
>         at
> net.sf.hibernate.loader.OneToManyLoader.initialize(OneToManyLoader.java:80)
>
>         at
> net.sf.hibernate.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:284)
>
>         ... 52 more
>
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
>