Re: Let's make PostgreSQL multi-threaded - Mailing list pgsql-hackers

From Heikki Linnakangas
Subject Re: Let's make PostgreSQL multi-threaded
Date
Msg-id 29c2ae59-29b5-51cb-a2aa-bbd03e1a83e9@iki.fi
Whole thread Raw
In response to Re: Let's make PostgreSQL multi-threaded  (Ranier Vilela <ranier.vf@gmail.com>)
Responses Re: Let's make PostgreSQL multi-threaded
Re: Let's make PostgreSQL multi-threaded
List pgsql-hackers
On 05/06/2023 12:26, Ranier Vilela wrote:
> First, it is not clear what are the benefits of architecture change?
> 
> Performance?

I doubt it makes much performance difference, at least not initially. It 
might help a little with backend startup time, and maybe some other 
things. And might reduce the overhead of context switches and TLB cache 
misses.

In the long run, a single-process architecture makes it easier to have 
shared catalog caches, plan cache, etc. which can improve performance. 
And it can make it easier to launch helper threads for things where 
worker processes would be too heavy-weight. But those benefits will 
require more work, they won't happen just by replacing processes with 
threads.

The ease of developing things like that is my motivation.

> Development becomes much more complicated and error-prone.

I don't agree with that.

We currently bend over backwards to make all allocations fixed-sized in 
shared memory. You learn to live with that, but a lot of things would be 
simpler if you could allocate and free in shared memory more freely. 
It's no panacea, you still need to be careful with locking and 
concurrency. But a lot simpler.

We have built dynamic shared memory etc. over the years to work around 
the limitations of shared memory. But it's still a lot more complicated.

Code that doesn't need to communicate with other processes/threads is 
simple to write in either model.

> There are still many low-hanging fruit to be had that can improve 
> performance.
> And the code can gradually and safely remove multithreading barriers.
> 
> 1. gradual reduction of global variables
> 2. introduction of local context structures
> 3. shrink current structures (to fit in 32, 64 boundaries)
> 
> 4. scope reduction

Right, the reason I started this thread is to explicitly note that it is 
a worthy goal. If it's not, the above steps would be pointless. But if 
we agree that it is a worthy goal, we can start to incrementally work 
towards it.

-- 
Heikki Linnakangas
Neon (https://neon.tech)




pgsql-hackers by date:

Previous
From: Bruce Momjian
Date:
Subject: Re: Let's make PostgreSQL multi-threaded
Next
From: Heikki Linnakangas
Date:
Subject: Re: Let's make PostgreSQL multi-threaded