Re: High memory retention by QueryExecutorImpl.pgStream.encoding.decoderArray - Mailing list pgsql-jdbc

From Atul Deopujari
Subject Re: High memory retention by QueryExecutorImpl.pgStream.encoding.decoderArray
Date
Msg-id CAGkoPhed3=TcWUj0oX933L3sisRBEouv-_WB6NxPEH8HrwWZjA@mail.gmail.com
Whole thread Raw
In response to Re: High memory retention by QueryExecutorImpl.pgStream.encoding.decoderArray  (Dave Cramer <pg@fastcrypt.com>)
List pgsql-jdbc

--
Atul

On Tue, 5 Mar 2019 at 16:30, Dave Cramer <pg@fastcrypt.com> wrote:
Hello,

It would be useful if you could try the latest driver as that may have fixed the problem, alternatively it is also what we are developing against.

As you already have a heap dump can you shed any more light on what's going on?

Regards

On Mon, 4 Mar 2019 at 11:01, Atul Deopujari <atuldeopujari@gmail.com> wrote:
Hello
I am trying to figure out reasons for high memory utilization by PgConnection object. My application occasionally runs out of memory and the heap dump shows PgConnection as one of the largest consumers of memory.

The heap dump shows PgConnection having a retained size of 640 MB.
We use a connection pool of 8 connections. Each connection object is retaining about 80 MB. If I drill down into the PgConnection object, I see that QueryExecutorImpl taking almost all memory within which it is the pgStream.encoding.decoderArray taking all the memory.

Not all connection objects are 80 MB all the time. Sometimes the heap dump also shows that some connection objects are as low as 3 MB in size.

I would like to understand under what circumstances can pgStream.encoding.decoderArray retain 80MB. I have checked places in the code and have ensured that we are doing resultSet.close, stmt.close and conn.close wherever we are using them. We do have big objects in the database. I have tried to reproduce this problem using a simple program that queries tuples that are large text, some are even more than 500MB. I haven't been able to see such high memory usage of QueryExecutorImpl in my local setup.

We use the following JDBC driver and the server
  • JDBC driver build number: 42.2.2
  • Server version: 10.4 (Ubuntu 10.4-2.pgdg14.04+1)

Will appreciate if someone can throw light on the reasons for excessive object retention in  QueryExecutorImpl.pgStream.encoding.decoderArray which can help me dig this problem further.

Thanks,
Atul

pgsql-jdbc by date:

Previous
From: Dave Cramer
Date:
Subject: Re: High memory retention by QueryExecutorImpl.pgStream.encoding.decoderArray
Next
From: Philippe Ebersohl
Date:
Subject: Re: A method to asynchronously LISTEN ?