Re: Perform streaming logical transactions by background workers and parallel apply - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: Perform streaming logical transactions by background workers and parallel apply
Date
Msg-id CAA4eK1Kt67RdW0WTR-LTxasj3pyukPCYhfA0arDUNnsz2wh03A@mail.gmail.com
Whole thread Raw
In response to Re: Perform streaming logical transactions by background workers and parallel apply  (Masahiko Sawada <sawada.mshk@gmail.com>)
List pgsql-hackers
On Fri, Nov 18, 2022 at 10:31 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Fri, Nov 18, 2022 at 1:47 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Fri, Nov 18, 2022 at 8:01 AM Peter Smith <smithpb2250@gmail.com> wrote:
> > >
> > > On Fri, Nov 18, 2022 at 11:36 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > > >
> > > ...
> > > > ---
> > > > The streaming parameter has the new value "parallel" for "streaming"
> > > > option to enable the parallel apply. It fits so far but I think the
> > > > parallel apply feature doesn't necessarily need to be tied up with
> > > > streaming replication. For example, we might want to support parallel
> > > > apply also for non-streaming transactions in the future. It might be
> > > > better to have another option, say "parallel", to control parallel
> > > > apply behavior. The "parallel" option can be a boolean option and
> > > > setting parallel = on requires streaming = on.
> > > >
> >
> > If we do that then how will the user be able to use streaming
> > serialize mode (write to file for streaming transactions) as we have
> > now? Because after we introduce parallelism for non-streaming
> > transactions, the user would want parallel = on irrespective of the
> > streaming mode. Also, users may wish to only parallelize large
> > transactions because of additional overhead for non-streaming
> > transactions for transaction dependency tracking, etc. So, the user
> > may wish to have a separate knob for large transactions as the patch
> > has now.
>
> One idea for that would be to make it enum. For example, setting
> parallel = "streaming" works for that.
>

Yeah, but then we will have two different parameters (parallel and
streaming) to control streaming behavior. This will be confusing say
when the user says parallel = 'streaming' and streaming = off, we need
to probably disallow such settings but not sure if it would be any
better than allowing parallelism for large xacts by streaming
parameter.

-- 
With Regards,
Amit Kapila.



pgsql-hackers by date:

Previous
From: "Drouvot, Bertrand"
Date:
Subject: Avoid double lookup in pgstat_fetch_stat_tabentry()
Next
From: Tom Lane
Date:
Subject: Re: Optimize join selectivity estimation by not reading MCV stats for unique join attributes