At Thu, 9 Mar 2023 11:00:46 +0530, Amit Kapila <amit.kapila16@gmail.com> wrote in
> On Wed, Mar 8, 2023 at 9:20 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> > On Wed, Mar 8, 2023 at 3:30 AM Nathan Bossart <nathandbossart@gmail.com> wrote:
> > >
> >
> > > IMO the current set of
> > > trade-offs (e.g., unavoidable bloat and WAL buildup) would make this
> > > feature virtually unusable for a lot of workloads, so it's probably worth
> > > exploring an alternative approach.
> >
> > It might require more engineering effort for alternative approaches
> > such as one I proposed but the feature could become better from the
> > user perspective. I also think it would be worth exploring it if we've
> > not yet.
> >
>
> Fair enough. I think as of now most people think that we should
> consider alternative approaches for this feature. The two ideas at a
If we can notify subscriber of the transaction start time, will that
solve the current problem? If not, or if it is not possible, +1 to
look for other solutions.
> high level are that the apply worker itself first flushes the decoded
> WAL (maybe only when time-delay is configured) or have a separate
> walreceiver process as we have for standby. I think we need to analyze
> the pros and cons of each of those approaches and see if they would be
> useful even for other things on the apply side.
My understanding of the requirements here is that the publisher should
not hold changes, the subscriber should not hold data reads, and all
transactions including two-phase ones should be applied at once upon
committing. Both sides need to respond to the requests from the other
side. We expect apply-delay of several hours or more. My thoughts
considering the requirements are as follows:
If we expect delays of several hours or more, I don't think it's
feasible to stack received changes in the process memory. So, if
apply-delay is in effect, I think it would be better to process
transactions through files regardless of process configuration.
I'm not sure whether we should have a separate process for protocol
processing. On one hand, it would simplify the protocol processing
part, but on the other hand, changes would always have to be applied
through files. If we plan to integrate the paths with and without
apply-delay by the file-passing method, this might work. If we want to
maintain the current approach when not applying apply-delay, I think
we would have to implement it in a single process, but I feel the
protocol processing part could become complicated.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
