High memory retention by QueryExecutorImpl.pgStream.encoding.decoderArray - Mailing list pgsql-jdbc

From Atul Deopujari
Subject High memory retention by QueryExecutorImpl.pgStream.encoding.decoderArray
Date
Msg-id CAGkoPheL6vFLtM=71Vn+8sdrRj9LaRyFuc+E_U=5R5z=FdNNEQ@mail.gmail.com
Whole thread Raw
Responses Re: High memory retention by QueryExecutorImpl.pgStream.encoding.decoderArray  (Dave Cramer <pg@fastcrypt.com>)
List pgsql-jdbc
Hello
I am trying to figure out reasons for high memory utilization by PgConnection object. My application occasionally runs out of memory and the heap dump shows PgConnection as one of the largest consumers of memory.

The heap dump shows PgConnection having a retained size of 640 MB.
We use a connection pool of 8 connections. Each connection object is retaining about 80 MB. If I drill down into the PgConnection object, I see that QueryExecutorImpl taking almost all memory within which it is the pgStream.encoding.decoderArray taking all the memory.

Not all connection objects are 80 MB all the time. Sometimes the heap dump also shows that some connection objects are as low as 3 MB in size.

I would like to understand under what circumstances can pgStream.encoding.decoderArray retain 80MB. I have checked places in the code and have ensured that we are doing resultSet.close, stmt.close and conn.close wherever we are using them. We do have big objects in the database. I have tried to reproduce this problem using a simple program that queries tuples that are large text, some are even more than 500MB. I haven't been able to see such high memory usage of QueryExecutorImpl in my local setup.

We use the following JDBC driver and the server
  • JDBC driver build number: 42.2.2
  • Server version: 10.4 (Ubuntu 10.4-2.pgdg14.04+1)

Will appreciate if someone can throw light on the reasons for excessive object retention in  QueryExecutorImpl.pgStream.encoding.decoderArray which can help me dig this problem further.

Thanks,
Atul

pgsql-jdbc by date:

Previous
From: Švorc Martin
Date:
Subject: RE: How to send queries to master and failover to slave ?
Next
From: Dave Cramer
Date:
Subject: Re: High memory retention by QueryExecutorImpl.pgStream.encoding.decoderArray