RE: Perform streaming logical transactions by background workers and parallel apply - Mailing list pgsql-hackers

From wangw.fnst@fujitsu.com
Subject RE: Perform streaming logical transactions by background workers and parallel apply
Date
Msg-id OS3PR01MB62756BD9482EB6BB1CA4CD4D9EAA9@OS3PR01MB6275.jpnprd01.prod.outlook.com
Whole thread Raw
In response to RE: Perform streaming logical transactions by background workers and parallel apply  ("wangw.fnst@fujitsu.com" <wangw.fnst@fujitsu.com>)
Responses RE: Perform streaming logical transactions by background workers and parallel apply
Re: Perform streaming logical transactions by background workers and parallel apply
List pgsql-hackers
On Wed, Jun 8, 2022 3:13 PM I wrote:
> Attach the new patches.(only changed 0001 and 0003)

I tried to improve the patches by following points:

1. Initialize variable include_abort_lsn to false. It reports a warning in
cfbot. (see patch v10-0001)
BTW, I merged the patch that added the new GUC (see v9-0004) into patch 0001.

2. Because of the improvement #2 in [1], the foreign key could not be detected
when checking trigger function. So added additional checks for the foreign key.
(see patch 0004)

3. Adding a check for the partition table when trying to apply changes in the
apply background worker. (see patch 0004)
In additional, the partition cache map on subscriber have several bugs (see
thread [2]). Because patch 0004 is developed based on the patches in [2], so I
merged the patches(v4-0001~v4-0003) in [2] into a temporary patch 0003 here.
After the patches in [2] is committed, I will delete patch 0003 and rebase
patch 0004.

4. Improve constraint checking in a separate patch as suggested by Amit-san in
[3] #6.(see patch 0005)
I added a new field "bool subretry" in catalog pg_subscription. I use this
field to indicate whether the transaction that we are going to process has
failed before.
If apply worker/bgworker was exit with an error, this field will be set to
true; If we successfully apply a transaction, this field will be set to false.
If we retry to apply a streaming transaction, whether the user sets the
streaming option to "on" or "apply", we will apply the transaction in the apply
worker.

Attach the new patches.
Only changed patches 0001, 0004 and added new separate patch 0005.

[1] -
https://www.postgresql.org/message-id/OS3PR01MB6275208A2F8ED832710F65E09EA49%40OS3PR01MB6275.jpnprd01.prod.outlook.com
[2] -
https://www.postgresql.org/message-id/flat/OSZPR01MB6310F46CD425A967E4AEF736FDA49%40OSZPR01MB6310.jpnprd01.prod.outlook.com
[3] - https://www.postgresql.org/message-id/CAA4eK1Jt08SYbRt_-rbSWNg%3DX9-m8%2BRdP5PosfnQgyF-z8bkxQ%40mail.gmail.com

Regards,
Wang wei

Attachment

pgsql-hackers by date:

Previous
From: "huangning290@yahoo.com"
Date:
Subject: GIN index partial match
Next
From: "houzj.fnst@fujitsu.com"
Date:
Subject: RE: Skipping schema changes in publication