Re: Parallelize stream replication process - Mailing list pgsql-hackers

From Bharath Rupireddy
Subject Re: Parallelize stream replication process
Date
Msg-id CALj2ACU104ZL9r6x_y-1cKy5jOZvb3c4B6LnKQ9OHGxMhSTOQg@mail.gmail.com
Whole thread Raw
In response to Parallelize stream replication process  (Li Japin <japinli@hotmail.com>)
Responses Re: Parallelize stream replication process
Re: Parallelize stream replication process
List pgsql-hackers
On Tue, Sep 15, 2020 at 9:27 AM Li Japin <japinli@hotmail.com> wrote:
>
> For now, postgres use single process to send, receive and replay the WAL when we use stream replication,
> is there any point to parallelize this process? If it does, how do we start?
>
> Any thoughts?
>

I think we must ask few questions:

1. What's the major gain we get out of this? Is it that the time to
stream gets reduced or something else?
If the answer to the above point is something solid, then
2. How do we distribute the work to multiple processes?
3. Do we need all of the workers to maintain the order in which they
read WAL files(on the publisher) and apply the changes(on the
subscriber?)
3. Do we want to map the sender/publisher workers to
receiver/subscriber workers on a one-to-one basis? If not, how do we
do it?
4. How do sender and receiver workers communicate?
5. What if we have multiple subscribers/receivers?

I'm no expert in replication, I may be wrong as well. Others may have
better thoughts.

With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com



pgsql-hackers by date:

Previous
From: "MauMau"
Date:
Subject: Re: I'd like to discuss scaleout at PGCon
Next
From: Tom Lane
Date:
Subject: Re: pg_restore causing deadlocks on partitioned tables