Thanks for clarifying the questions!
> On Sep 15, 2020, at 12:41 PM, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
>
> On Tue, Sep 15, 2020 at 9:27 AM Li Japin <japinli@hotmail.com> wrote:
>>
>> For now, postgres use single process to send, receive and replay the WAL when we use stream replication,
>> is there any point to parallelize this process? If it does, how do we start?
>>
>> Any thoughts?
>>
>
> I think we must ask few questions:
>
> 1. What's the major gain we get out of this? Is it that the time to
> stream gets reduced or something else?
I think when the database failover, we might shorten the recovery time from the parallel stream replication.
> If the answer to the above point is something solid, then
> 2. How do we distribute the work to multiple processes?
> 3. Do we need all of the workers to maintain the order in which they
> read WAL files(on the publisher) and apply the changes(on the
> subscriber?)
> 3. Do we want to map the sender/publisher workers to
> receiver/subscriber workers on a one-to-one basis? If not, how do we
> do it?
> 4. How do sender and receiver workers communicate?
> 5. What if we have multiple subscribers/receivers?
>
> I'm no expert in replication, I may be wrong as well. Others may have
> better thoughts.
>
Maybe we can distribute the work to multiple processes according by the WAL record type.
In the first step, I think we can parallel the replay process. We can classify the WAL by WAL type or RmgrId,
and then parallel those WAL replay if possible.
Then, we can think how to parallel WalReceiver and WalSender.
Best regards
Japin Li.
