Re: Parallel Apply - Mailing list pgsql-hackers

From Shlok Kyal
Subject Re: Parallel Apply
Date
Msg-id CANhcyEXuDB3SSsQGADwCfLxn8J=D_Gf7T2wfgX4vBa-=de5F8A@mail.gmail.com
Whole thread
In response to RE: Parallel Apply  ("Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>)
List pgsql-hackers
> The version number now becomes v27 by the internal reviews. I adjusted each
> patches to work well and pass tests, and fixed test codes to extract the
> dependent transaction correctly.

Hi,

I found one more issue. It is related to changes in 0007 patch:

Parallel apply can resume while a transaction for which dependency
recording was stopped is still in progress.
This can allow a later transaction to be applied first, causing an
update to be skipped and the subscriber to diverge from the publisher.

When dependency memory reaches the limit, 'maybe_suspend_parallel_apply()' sets:
+ parallel_apply_suspended = true;

Subsequent changes are applied without recording dependencies:
+ /*
+ * If the estimated memory usage of dependency tracking exceeds the limit,
+ * suspend parallel apply until the usage drops below half of the limit.
+ */
+ if (maybe_suspend_parallel_apply(winfo))
+ return;

However, maybe_resume_parallel_apply() considers only memory usage:
+ if (dependency_mem_usage > PARALLEL_DEPENDENCY_MEMORY_LIMIT / 2)
+ return;
+
+ parallel_apply_suspended = false;
It does not check whether a transaction with unrecorded changes is
still in progress.

Attached a test to reproduce the issue.

Thanks,
Shlok Kyal

Attachment

pgsql-hackers by date:

Previous
From: David Geier
Date:
Subject: Re: [PATCH v1] Batch B-tree TIDs when building a bitmap
Next
From: David Geier
Date:
Subject: Use mbsnrtowcs() in char2wchar() to avoid temporary string copies