Hi,
for quite some time, I've been under the impression, that there's still
one disadvantage left from using processes instead of threads: we can
only use statically sized chunks of shared memory. Every component that
wants to use shared memory needs to pre-allocate whatever it thinks is
sufficient. It cannot enlarge its share, nor can unused memory be
allocated to other components.
Having written a very primitive kind of a dynamic memory allocator for
imessages [1], I've always wanted a better alternative. So I've
investigated a bit, refactored step-by-step, and finally came up with
the attached, lock based dynamic shared memory allocator. Its interface
is as simple as malloc() and free(). A restart of the postmaster should
truncate the whole area.
Being a component which needs to pre-allocate its area in shared memory
in advance, you need to define a maximum size for the pool of
dynamically allocatable memory. That's currently defined in shmem.h
instead of a GUC.
This kind of feature has been requested at the Tokyo Clusting Meeting
(by myself) in 2009 and is listed on the Wiki [2].
I'm now using that allocator as the basis for a reworked imessages
patch, which I've attached as well. Both are tested as a basis for
Postgres-R.
While I think other components could use this dynamic memory allocator,
too, I didn't write any code for that. Imessages currently is the only
user available. (So please apply the dynshmem patch first, then
imessages).
Comments?
Greetings from Oxford, and thanks to Joachim Wieland for providing me
the required Internet connectivity ;-)
Markus Wanner
[1]: Postgres-R: internal messages
http://archives.postgresql.org/message-id/4886DB0B.1090508@bluegap.ch
[2]: Mentioned Cluster Feature
http://wiki.postgresql.org/wiki/ClusterFeatures#Dynamic_shared_memory_allocation
For git adicts: here's a git repository with both patches applied:
http://git.postgres-r.org/?p=imessages;a=summary
