RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed - Mailing list pgsql-bugs

From Matt Carter
Subject RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
Date
Msg-id BL3PR08MB72838C12A57F4D9668D27AF0996DA@BL3PR08MB7283.namprd08.prod.outlook.com
Whole thread Raw
In response to Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
List pgsql-bugs
Tom Lane <tgl@sss.pgh.pa.us>  wrote:
> Please try varying the connection type and encryption.

Good idea.  Here's what I found:

Test 1:
  Configuration: Original (SQLAlchemy + NullPool)
  libpq: 16.0.7
  SQLAlchemy: Yes
  Leak Rate: 803 KB/min

Test 2:
  Configuration: Pure psycopg2 (default)
  libpq: 16.0.7
  SQLAlchemy: NO
  Leak Rate: 801 KB/min

Test 3:
  Configuration: Pure psycopg2 + gssencmode=disable
  libpq: 16.0.7
  SQLAlchemy: NO
  Leak Rate: 858 KB/min

Test 4:
  Configuration: Pure psycopg2 + SSL only
  libpq: 16.0.7
  SQLAlchemy: NO
  Leak Rate: 861 KB/min

Test 5:
  Configuration: Pure psycopg2
  libpq: 13.0.11
  SQLAlchemy: NO
  Leak Rate: 17 KB/min

So, it seems none of these changes avoids the leak:
- Removing SQLAlchemy
- Disabling GSSAPI encryption
- Using SSL-only
- Changing PostgreSQL DB Server version from 16 to 13

The only changes that I found that avoid the leak are:
- Changing libpq version from 16 to 13, or
- Changing psycopg version from 2 to 3.

Tom, since your C test showed no leak, the issue is likely in how psycopg2 calls libpq, not pure libpq itself.

I guess I should report this to the psycopg2 project.

pgsql-bugs by date:

Previous
From: Andres Freund
Date:
Subject: Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
Next
From: Andrew Dunstan
Date:
Subject: Re: BUG #19409: Function jsonb_strip_nulls() changed from immutable to stable.