Re: Perform streaming logical transactions by background workers and parallel apply - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: Perform streaming logical transactions by background workers and parallel apply
Date
Msg-id CAA4eK1KkrkAzO-=_k_6s5V1+uNUFkvxMK-33MaFw=JCw5Hqs3w@mail.gmail.com
Whole thread Raw
In response to RE: Perform streaming logical transactions by background workers and parallel apply  ("houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>)
Responses Re: Perform streaming logical transactions by background workers and parallel apply  (Amit Kapila <amit.kapila16@gmail.com>)
List pgsql-hackers
On Fri, Dec 16, 2022 at 2:47 PM houzj.fnst@fujitsu.com
<houzj.fnst@fujitsu.com> wrote:
>
> > ---
> > +       active_workers = list_copy(ParallelApplyWorkerPool);
> > +
> > +       foreach(lc, active_workers)
> > +       {
> > +               int                     slot_no;
> > +               uint16          generation;
> > +               ParallelApplyWorkerInfo *winfo =
> > (ParallelApplyWorkerInfo *) lfirst(lc);
> > +
> > +               LWLockAcquire(LogicalRepWorkerLock, LW_SHARED);
> > +               napplyworkers =
> > logicalrep_pa_worker_count(MyLogicalRepWorker->subid);
> > +               LWLockRelease(LogicalRepWorkerLock);
> > +
> > +               if (napplyworkers <=
> > max_parallel_apply_workers_per_subscription / 2)
> > +                       return;
> > +
> >
> > Calling logicalrep_pa_worker_count() with lwlock for each worker seems
> > not efficient to me. I think we can get the number of workers once at
> > the top of this function and return if it's already lower than the
> > maximum pool size. Otherwise, we attempt to stop extra workers.
>
> How about we directly check the length of worker pool list here which
> seems simpler and don't need to lock ?
>

I don't see any problem with that. Also, if such a check is safe then
can't we use the same in pa_free_worker() as well? BTW, shouldn't
pa_stop_idle_workers() try to free/stop workers unless the active
number reaches below max_parallel_apply_workers_per_subscription?

-- 
With Regards,
Amit Kapila.



pgsql-hackers by date:

Previous
From: Niyas Sait
Date:
Subject: Re: [PATCH] Add native windows on arm64 support
Next
From: vinay kumar
Date:
Subject: Feature request to add rows extracted using pg_dump utility