On Wed, Sep 20, 2017 at 6:14 PM, Gaddam Sai Ram <gaddamsairam.n@zohocorp.com> wrote: > Thank you very much! That fixed my issue! :) > I was in an assumption that pinning the area will increase its lifetime but > yeah after taking memory context into consideration its working fine!
So far the success rate in confusing people who first try to make long-lived DSA areas and DSM segments is 100%. Basically, this is all designed to ensure automatic cleanup of resources in short-lived scopes.
90% ;)
I got it working with no significant issues for a long lived segment used to store a pool of shm_mq pairs used for a sort of "connection listener" bgworker. Though I only used DSM+ToC, not DSA. But TBH that may well be luck, as I tend to routinely use memory contexts scoped to the operational lifetime of a subsystem, making most problems like this just vanish without my realising they were there in the first place. Usually.
I pretty much shamelessly cribbed from test_shm_mq for the ToC stuff though. It's simple enough when you read it in use, but I'd be lucky to do it without an example.
I had lots more problems with shm_mq than DSM. shm_mq is very obviously designed for short-lived scopes, and falls down badly if you have a pool of queues you want to re-use after the peer detaches. You have to track "in use" flags separately to the shm_mq's own, because it doesn't clear its stored PGPROC entries for receiver/sender on detach. Once you know neither sender nor receiver is still attached, you can memset() the area and create a new queue in it.
You can't just reset the queue for a new peer, and have to do quite a dance to make sure it's safe detach from, overwrite, re-create and re-attach to.
Good luck for your graph project. I think you're going to have to expend a lot of energy trying to avoid memory leaks if your DSA lives as long as the database cluster, since error paths won't automatically free any memory you allocated in it.
Yeah, that's going to be hard. You might land up having lots and lots of little DSM segments.
There is a kind of garbage collection for palloc'd memory and also for other resources like file handles, but if you're using a big long lived DSA area you have nothing like that.
We need, IMO, a DSA-backed heirachical MemoryContext system.
We can't use the exact MemoryContext API as-is due to the need for far pointers though :(
