Thread: Attach to shared memory after fork()
Fork is an expensive operation[1]. The major cost is the mm(VMA PTE...) copy.
ARM is especially weak on fork, which will invalid TLB entries one by one, and this is an expensive operation[2]. We could easily got 100% CPU on ARM machine. We also meet fork problem in x86, but not as serious as arm.
We can avoid this by enable hugepage(and 2MB doesn’t help us under arm, we got a huge shared buffer), but we still think it is a problem.
So I propose to remove shared buffers from postmaster and shmat them after fork. Not all of them, we still keep necessary shared memories in postmaster. Or maybe we just need to give up fork like under Windows?
Any good idea about it?
D5.10 TLB maintenance requirements and the TLB maintenance instructions:
Break-before-make sequence on changing from an old translation table entry to a new translation table entryrequires the following steps:
1. Replace the old translation table entry with an invalid entry, and execute a DSB instruction.
2. Invalidate the translation table entry with a broadcast TLB invalidation instruction, and execute a DSBinstruction to ensure the completion of that invalidation.
3. Write the new translation table entry, and execute a DSB instruction to ensure that the new entry is visible.
1. Replace the old translation table entry with an invalid entry, and execute a DSB instruction.
2. Invalidate the translation table entry with a broadcast TLB invalidation instruction, and execute a DSBinstruction to ensure the completion of that invalidation.
3. Write the new translation table entry, and execute a DSB instruction to ensure that the new entry is visible.
Regards.
Yuhang Qiu.
On 4/26/21 11:56 PM, 邱宇航(烛远) wrote: > Fork is an expensive operation[1]. The major cost is the mm(VMA > PTE...) copy. > > ARM is especially weak on fork, which will invalid TLB entries one by > one, and this is an expensive operation[2]. We could easily got 100% > CPU on ARM machine. We also meet fork problem in x86, but not as > serious as arm. > > We can avoid this by enable hugepage(and 2MB doesn’t help us under > arm, we got a huge shared buffer), but we still think it is a problem. > > So I propose to remove shared buffers from postmaster and shmat them > after fork. Not all of them, we still keep necessary shared memories > in postmaster. Or maybe we just need to give up fork like under Windows? > Windows has CreateProcess, which isn't available elsewhere. If you build with EXEC_BACKEND on *nix it will fork() followed by exec(), the classic Unix pattern. You can benchmark that but I doubt you will like the results. This is one of the reasons for using a connection pooler like pgbouncer, which can vastly reduce the number of new process creations Postgres has to do. Better shared memory management might be more promising. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
"=?UTF-8?B?6YKx5a6H6IiqKOeDm+i/nCk=?=" <yuhang.qyh@alibaba-inc.com> writes:
> Fork is an expensive operation[1].
Yeah, it's not hugely cheap.
> So I propose to remove shared buffers from postmaster and shmat them
> after fork.
This proposal seems moderately insane. In the first place, it
introduces failure modes we could do without, and in the second place,
how is it not strictly *more* expensive than what happens now? You
still have to end up with all those TLB entries mapped in the child.
(If your kernel is unable to pass down shared-memory TLBs effectively,
ISTM that's a kernel shortcoming not a Postgres architectural problem.)
regards, tom lane