Hi,
On 2024-11-28 17:30:32 +0100, Dmitry Dolgov wrote:
> The assumption about picking up a lowest address is just how it works right now
> on Linux, this fact is already used in the patch. The idea that we could put
> upper boundary on the size of other mappings based on total available memory
> comes from the fact that anonymous mappings, that are much larger than memory,
> will fail without overcommit.
The overcommit issue shouldn't be a big hurdle - by mmap()ing with
MAP_NORESERVE the space isn't reserved. Then madvise with MADV_POPULATE_WRITE
can be used to actually populate the used range of the mapping and MADV_REMOVE
can be used to shrink the mapping again.
> With overcommit it becomes different, but if allocations are hitting that
> limit I can imagine there are bigger problems than shared buffer resize.
I'm fairly sure it'll not work to just disregard issues around overcommit. A
overly large memory allocation, without MAP_NORESERVE, will actually reduce
the amount of memory that can be used for other allocations. That's obviously
problematic, because you'll now have a smaller shared buffers, but can't use
the memory for work_mem type allocations...
Greetings,
Andres Freund
