Re: Force streaming every change in logical decoding - Mailing list pgsql-hackers

On Wed, Dec 14, 2022 at 5:29 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Wed, Dec 14, 2022 at 2:15 PM shiy.fnst@fujitsu.com
> <shiy.fnst@fujitsu.com> wrote:
> >
> > Please see the attached patch. I also fix Peter's comments[1]. The GUC name and
> > design are still under discussion, so I didn't modify them.
> >
>
> Let me summarize the discussion on name and design till now. As per my
> understanding, we have three requirements: (a) In publisher, stream
> each change of transaction instead of waiting till
> logical_decoding_work_mem or commit; this will help us to reduce the
> test timings of current and future tests for replication of
> in-progress transactions; (b) In publisher, serialize each change
> instead of waiting till logical_decoding_work_mem; this can help
> reduce the test time of tests related to serialization of changes in
> logical decoding; (c) In subscriber, during parallel apply for
> in-progress transactions (a new feature being discussed at [1]) allow
> the system to switch to serialize mode (no more space in shared memory
> queue between leader and parallel apply worker either due to a
> parallel worker being busy or waiting on some lock) while sending
> changes.
>
> Having a GUC that controls these actions/features will allow us to
> write tests with shorter duration and better predictability as
> otherwise, they require a lot of changes. Apart from tests, these also
> help to easily debug the required code. So they fit the Developer
> Options category of GUC [2].
>
> We have discussed three different ways to provide GUC for these
> features. (1) Have separate GUCs like force_server_stream_mode,
> force_server_serialize_mode, force_client_serialize_mode (we can use
> different names for these) for each of these; (2) Have two sets of
> GUCs for server and client. We can have logical_decoding_mode with
> values as 'stream' and 'serialize' for the server and then
> logical_apply_serialize = true/false for the client. (3) Have one GUC
> like logical_replication_mode with values as 'server_stream',
> 'server_serialize', 'client_serialize'.
>
> The names used here are tentative mainly to explain each of the
> options, we can use different names once we decide among the above.
>
> Thoughts?

I think option 2 makes sense because stream/serialize are two related
options and both are dependent on the same parameter
(logical_decoding_work_mem) so having a single know is logical.  And
another GUC for serializing logical apply.

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com