Re: Let's make PostgreSQL multi-threaded - Mailing list pgsql-hackers

From Robert Haas
Subject Re: Let's make PostgreSQL multi-threaded
Date
Msg-id CA+Tgmoadhk-yHnjrPq34iaZ_UXPLm3QAiqwyyvfYXFoPU3CTHw@mail.gmail.com
Whole thread Raw
In response to Re: Let's make PostgreSQL multi-threaded  (Hannu Krosing <hannuk@google.com>)
List pgsql-hackers
On Wed, Jun 14, 2023 at 3:46 PM Hannu Krosing <hannuk@google.com> wrote:
> I remember a few times when memory leaks in some PostGIS packages
> cause slow memory exhaustion and the simple fix was limiting
> connection lifetime to something between 15 min and an hour.
>
> The main problem here is that PostGIS uses a few tens of other GPL GIS
> related packages which are all changing independently and thus it is
> quite hard to be sure that none of these have developed a leak. And
> you also likely can not just stop upgrading these as they also contain
> security fixes.
>
> I have no idea what the fix could be in case of threaded server.

Presumably, when a thread exits, we
MemoryContextDelete(TopMemoryContext). If the leak is into any memory
context managed by PostgreSQL, this still frees the memory. But it
might not be. Right now, if a library does a malloc() that it doesn't
free() every once in a while, it's no big deal. If it does it too
often, it's a problem now, too. But if it does it only every now and
then, process exit will prevent accumulation over time. In a threaded
model, that isn't true any longer: those allocations will accumulate
until we OOM.

And IMHO that's definitely a very significant downside of this
direction. I don't think it should be dispositive because such
problems are, hopefully, fixable, whereas some of the problems caused
by the process model are basically unfixable except by not using it
any more. However, if we lived in a world where both models were
supported and a particular user said, "hey, I'm sticking with the
process model because I don't trust my third-party libraries not to
leak," I would be like "yep, I totally get it."

--
Robert Haas
EDB: http://www.enterprisedb.com



pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: Should heapam_estimate_rel_size consider fillfactor?
Next
From: Robert Haas
Date:
Subject: Re: trying again to get incremental backup