Hi Hackers,
I would like your opinion on something that I think would be very useful.
I am using the terminology of financial systems because I think many of you will be familiar with that and will facilitate the understanding.
MEMORY FUND: The memory fund can grant memory loans for any backend, however it doesn't have any memory of its own, it will just give a contract that allows the backend to allocate memory beyond its own work_mem.
LEASE: A lease is a contract between the fund and one of the backends that grants a certain amount of memory for that backend.
+-----------------------------------------------------+
| Memory Fund |
| +----------------------------------------------+ |
| | Memory Bank Ledger | |
| | - Total pool: 10GB | |
| | - Available: 5GB | |
| | - Leases: [(pid1, 3GB), (pid2, 2GB)] | |
| +----------------------------------------------+ |
+-----------------------------------------------------+
| | |
+----+----+ +----+----+ +----+----+
|Backend 1| |Backend 2| |Backend 3|
| 100MB | | 100MB | | 100MB |
| +3GB | | +2GB | | |
+---------+ +---------+ +---------+
(own heap) (own heap) (own heap)
Planning: Approve(2GB) -> available -= 2GB, approvals[me] = 2GB
Execution: Lease(500MB) -> approvals[me] = 1.5GB, leases[me] = 500MB
Lease(500MB) -> approvals[me] = 1GB, leases[me] = 1GB
Return(500MB) -> leases[me] = 500MB, available += 500MB
...
Cleanup: Return(500MB) -> leases[me] = 0, available += 500MB
Close(2GB) -> approvals[me] = 0, available += 1GB
The bank could implement both blocking and non-blocking requests. Non blocking requests could be made before deciding to go for a temp file. But the planner could be extended by considering leveraged plans.
Thoughts?
