Re: Lots of memory allocated when reassigning Large Objects - Mailing list pgsql-hackers

On Mon, Nov 29, 2021 at 01:49:24PM +0100, Guillaume Lelarge wrote:
> One of our customers had a very bad issue while trying to reassign objects
> from user A to user B. He had a lot of them, and the backend got very
> hungry for memory. It finally all went down when the linux kernel decided
> to kill the backend (-9 of course).

> Memory is released at the end of the reassignment. So it's definitely not
> leaked forever, but only during the operation, which looks like a missing
> pfree (or something related). I've tried to find something like that in the
> code somewhere, but to no avail. I'm pretty sure I missed something, which
> is the reason for this email :)

I reproduced the issue like this.

psql postgres -c 'CREATE ROLE two WITH login superuser'
psql postgres two -c "SELECT lo_import('/dev/null') FROM generate_series(1,22111)" >/dev/null
psql postgres -c 'SET client_min_messages=debug; SET log_statement_stats=on;' -c 'begin; REASSIGN OWNED BY two TO
pryzbyj;rollback;'
 

I didn't find the root problem, but was able to avoid the issue by creating a
new mem context.  I wonder if there are a bunch more issues like this.

unpatched:
!       33356 kB max resident size

patched:
!       21352 kB max resident size

diff --git a/src/backend/catalog/pg_shdepend.c b/src/backend/catalog/pg_shdepend.c
index 9ea42f805f..cbe7f04983 100644
--- a/src/backend/catalog/pg_shdepend.c
+++ b/src/backend/catalog/pg_shdepend.c
@@ -65,6 +65,7 @@
 #include "storage/lmgr.h"
 #include "utils/acl.h"
 #include "utils/fmgroids.h"
+#include "utils/memutils.h"
 #include "utils/syscache.h"
 
 typedef enum
@@ -1497,6 +1498,11 @@ shdepReassignOwned(List *roleids, Oid newrole)
         while ((tuple = systable_getnext(scan)) != NULL)
         {
             Form_pg_shdepend sdepForm = (Form_pg_shdepend) GETSTRUCT(tuple);
+            MemoryContext cxt, oldcxt;
+
+            cxt = AllocSetContextCreate(CurrentMemoryContext, "shdepReassignOwned cxt",
+                    ALLOCSET_DEFAULT_SIZES);
+            oldcxt = MemoryContextSwitchTo(cxt);
 
             /*
              * We only operate on shared objects and objects in the current
@@ -1598,8 +1604,12 @@ shdepReassignOwned(List *roleids, Oid newrole)
                     elog(ERROR, "unexpected classid %u", sdepForm->classid);
                     break;
             }
+
             /* Make sure the next iteration will see my changes */
             CommandCounterIncrement();
+
+            MemoryContextSwitchTo(oldcxt);
+            MemoryContextDelete(cxt);
         }
 
         systable_endscan(scan);