Re: [patch] Cache invalidation for I/O Workers - Mailing list pgsql-hackers

From shihao zhong
Subject Re: [patch] Cache invalidation for I/O Workers
Date
Msg-id CAGRkXqQuhvb60xmaRoGkAuJG0AscvjRWCM+n2f_=2F-VzhUYhw@mail.gmail.com
Whole thread
In response to Re: [patch] Cache invalidation for I/O Workers  (Alexandre Felipe <o.alexandre.felipe@gmail.com>)
Responses Re: [patch] Cache invalidation for I/O Workers
List pgsql-hackers
Hi Alexandre,

Thanks for the patch. I read v1, here are my suggestions.

+ if (msg->id == SHAREDINVALSMGR_ID)
+ {
+ RelFileLocatorBackend rlocator;
+ rlocator.locator = msg->sm.rlocator;
+ rlocator.backend = (msg->sm.backend_hi << 16) | (int) msg->sm.backend_lo;
+ smgrreleaserellocator(rlocator);
+ }

Since 21d9c3ee4ef, smgrreleaserellocator() only closes the fds. The SMgrRelation
entry stays in the hash table. Backends free those in AtEOXact_SMgr(), but an
IO worker never runs a transaction, so entries for dropped relations
pile up. The fd leak turns into a memory leak. We had runs into production
issue because of that.


bgwriter and checkpointer have the same problem. They call
smgrdestroyall() once per checkpoint cycle, see the comment in
bgwriter.c. Could the IO worker just do the same every so often? That
is option 1 in the bug report. It needs no sinval slot for up to 32
workers, and it is small enough to backpatch to 18, where
io_method=worker is the default.

The patch also needs a pgindent run.

Thanks,
Shihao

pgsql-hackers by date:

Previous
From: Noah Misch
Date:
Subject: Re: Serverside SNI support in libpq
Next
From: Henson Choi
Date:
Subject: Re: Row pattern recognition