Re: Perform streaming logical transactions by background workers and parallel apply - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: Perform streaming logical transactions by background workers and parallel apply
Date
Msg-id CAA4eK1JoQrbeyhSmCJyAwN2k36oeJhkDscyp9QJjY5UPDUYhuA@mail.gmail.com
Whole thread Raw
In response to Re: Perform streaming logical transactions by background workers and parallel apply  (Peter Smith <smithpb2250@gmail.com>)
Responses Re: Perform streaming logical transactions by background workers and parallel apply
List pgsql-hackers
On Fri, Nov 18, 2022 at 8:01 AM Peter Smith <smithpb2250@gmail.com> wrote:
>
> On Fri, Nov 18, 2022 at 11:36 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> ...
> > ---
> > The streaming parameter has the new value "parallel" for "streaming"
> > option to enable the parallel apply. It fits so far but I think the
> > parallel apply feature doesn't necessarily need to be tied up with
> > streaming replication. For example, we might want to support parallel
> > apply also for non-streaming transactions in the future. It might be
> > better to have another option, say "parallel", to control parallel
> > apply behavior. The "parallel" option can be a boolean option and
> > setting parallel = on requires streaming = on.
> >

If we do that then how will the user be able to use streaming
serialize mode (write to file for streaming transactions) as we have
now? Because after we introduce parallelism for non-streaming
transactions, the user would want parallel = on irrespective of the
streaming mode. Also, users may wish to only parallelize large
transactions because of additional overhead for non-streaming
transactions for transaction dependency tracking, etc. So, the user
may wish to have a separate knob for large transactions as the patch
has now.

>
> FWIW, I tend to agree with this idea but for a different reason. In
> this patch, the 'streaming' parameter had become a kind of hybrid
> boolean/enum. AFAIK there are no other parameters anywhere that use a
> hybrid pattern like this so I was thinking it may be better not to be
> different.
>

I think we have a similar pattern for GUC parameters like
constraint_exclusion (see constraint_exclusion_options),
backslash_quote (see backslash_quote_options), etc.

-- 
With Regards,
Amit Kapila.



pgsql-hackers by date:

Previous
From: Richard Guo
Date:
Subject: Re: Optimize join selectivity estimation by not reading MCV stats for unique join attributes
Next
From: Masahiko Sawada
Date:
Subject: Re: Perform streaming logical transactions by background workers and parallel apply