Re: Trim the heap free memory - Mailing list pgsql-hackers

From shawn wang
Subject Re: Trim the heap free memory
Date
Msg-id CA+T=_GUddKqkiVHHj1NgZ3htqKo317rHPHSPdnY=XTGA6f8HVA@mail.gmail.com
Whole thread Raw
In response to Re: Trim the heap free memory  (Tomas Vondra <tomas@vondra.me>)
List pgsql-hackers
Thank you very much for your response.
 
To propose something less abstract / more tangible, I think we should do
something like this:

1) add a bit of code for glibc-based systems, that adjusts selected
malloc parameters using mallopt() during startup

2) add a GUC that enables this, with the default being the regular glibc
behavior (with dynamic adjustment of various thresholds)

I believe that the issue here arises from design incompatibilities between the complex engineering code and ptmalloc.
Modifying malloc parameters through mallopt is not user-friendly for database users and can be overly complex.
Moreover, setting certain parameters may lead to performance issues.
Monitoring memory usage should be a common practice for all database users.
With my signal-based approach, we can trigger a trim operation
when high memory usage is detected or by setting up a scheduled task.
This reduces the complexity for users and also helps in lowering memory consumption.
Of course, this solution is not perfect and does not address the problem elegantly from a fundamental perspective.
However, it has proven effective in the user environment.
I have set up a scheduled task to execute a function every 10 minutes for processes exceeding 50MB.
This has reduced memory usage from 87% to 30% on a 64GB system.

Best regards

Shawn

pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: [PATCH] initdb: Treat empty -U argument as unset username
Next
From: Tom Lane
Date:
Subject: Re: A assert failure when initdb with track_commit_timestamp=on