Re: Force streaming every change in logical decoding - Mailing list pgsql-hackers

From shveta malik
Subject Re: Force streaming every change in logical decoding
Date
Msg-id CAJpy0uBO2W9YhKPLbiTWANZsrhjyarPg1N1qswY4CYhh49eQVA@mail.gmail.com
Whole thread Raw
In response to RE: Force streaming every change in logical decoding  ("shiy.fnst@fujitsu.com" <shiy.fnst@fujitsu.com>)
List pgsql-hackers


On Fri, Dec 23, 2022 at 2:03 PM shiy.fnst@fujitsu.com <shiy.fnst@fujitsu.com> wrote:
>
> On Fri, Dec 23, 2022 1:50 PM Amit Kapila <amit.kapila16@gmail.com>
> >
> > On Thu, Dec 22, 2022 at 6:18 PM shiy.fnst@fujitsu.com
> > <shiy.fnst@fujitsu.com> wrote:
> > >
> > >
> > > Besides, I tried to reduce data size in streaming subscription tap tests by this
> > > new GUC (see 0002 patch). But I didn't covert all streaming tap tests
> > because I
> > > think we also need to cover the case that there are lots of changes. So, 015*
> > is
> > > not modified. And 017* is not modified because streaming transactions and
> > > non-streaming transactions are tested alternately in this test.
> > >
> >
> > I think we can remove the newly added test from the patch and instead
> > combine the 0001 and 0002 patches. I think we should leave the
> > 022_twophase_cascade as it is because it can impact code coverage,
> > especially the below part of the test:
> > # 2PC PREPARE with a nested ROLLBACK TO SAVEPOINT
> > $node_A->safe_psql(
> >     'postgres', "
> >     BEGIN;
> >     INSERT INTO test_tab VALUES (9999, 'foobar');
> >     SAVEPOINT sp_inner;
> >     INSERT INTO test_tab SELECT i, md5(i::text) FROM
> > generate_series(3, 5000) s(i);
> >
> > Here, we will stream first time after the subtransaction, so can
> > impact the below part of the code in ReorderBufferStreamTXN:
> > if (txn->snapshot_now == NULL)
> > {
> > ...
> > dlist_foreach(subxact_i, &txn->subtxns)
> > {
> > ReorderBufferTXN *subtxn;
> >
> > subtxn = dlist_container(ReorderBufferTXN, node, subxact_i.cur);
> > ReorderBufferTransferSnapToParent(txn, subtxn);
> > }
> > ...
> >
>
> OK, I removed the modification in 022_twophase_cascade.pl and combine the two patches.
>
> Please see the attached patch.
> I also fixed Kuroda-san's comments[1].
>
> [1] https://www.postgresql.org/message-id/TYAPR01MB5866CD99CF86EAC84119BC91F5E99%40TYAPR01MB5866.jpnprd01.prod.outlook.com
>
> Regards,
> Shi yu

Hello,
I ran tests (4 runs) on both versions (v5 and v6) in release mode. And the data looks promising, time is reduced now:

  HEAD                                 V5                                      Delta (sec)
2:20.535307                      2:15.865241                             4.670066
2:19.220917                      2:14.445312                             4.775605
2:22.492128                      2:17.35755                               5.134578
2:20.737309                      2:15.564306                             5.173003

HEAD                                V6                                         Delta (sec)
2:20.535307                   2:15.363567                                 5.17174
2:19.220917                   2:15.079082                                 4.14.1835
2:22.492128                   2:16.244139                                 6.247989
2:20.737309                   2:16.108033                                 4.629276

thanks
Shveta

pgsql-hackers by date:

Previous
From: "Anton A. Melnikov"
Date:
Subject: Re: [BUG] pg_upgrade test fails from older versions.
Next
From: Bharath Rupireddy
Date:
Subject: Re: Improve WALRead() to suck data directly from WAL buffers when possible