Thread: Re: locking [user] catalog tables vs 2pc vs logical rep
Hi. The attached PG docs patch about catalog deadlocks was previously implemented in another thread [1], but it seems more relevant to this one. PSA. ------ [1] https://www.postgresql.org/message-id/CAA4eK1K%2BSeT31pxwL5iTvXq%3DJhZpG_cUJLFhiz-eD%2BJr-WAPeg%40mail.gmail.com Kind Regards, Peter Smith. Fujitsu Australia.
Attachment
On Tuesday, June 1, 2021 4:33 PM Peter Smith <smithpb2250@gmail.com>
> To: Andres Freund <andres@anarazel.de>
> Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>; Amit Kapila
> <amit.kapila16@gmail.com>; Markus Wanner
> <markus.wanner@enterprisedb.com>
> Subject: Re: locking [user] catalog tables vs 2pc vs logical rep
>
> Hi.
>
> The attached PG docs patch about catalog deadlocks was previously
> implemented in another thread [1], but it seems more relevant to this one.
>
> PSA.
Thank you for providing the patch.
I have updated your patch to include some other viewpoints.
For example, CLUSTER command scenario
that also causes hang of PREPARE in synchronous mode.
We get this deadlock, using the 2PC patch-set.
FYI, the scenario is
(1) create a table with a trigger
(2) create pub and sub in synchronous mode
(3) then, execute CLUSTER pg_trigger USING pg_trigger_oid_index,
and do some operations (e.g. INSERT) on the trigger-attached table and PREPARE
The mechanism of this is
walsender tries to take a lock on pg_trigger if the table has a trigger,
but, pg_trigger is already locked by the CLUSTER command, which leads to the deadlock.
Then, this scenario requires some operations on the table which has trigger
because it invokes the walsender to take the lock described above.
I also included the description about TRUNCATE on user_catalog_table
in the patch. Please have a look at this patch.
Best Regards,
Takamichi Osumi
Attachment
On Thu, Jun 3, 2021 at 9:18 AM osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com> wrote: > > On Tuesday, June 1, 2021 4:33 PM Peter Smith <smithpb2250@gmail.com> > > To: Andres Freund <andres@anarazel.de> > > Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>; Amit Kapila > > <amit.kapila16@gmail.com>; Markus Wanner > > <markus.wanner@enterprisedb.com> > > Subject: Re: locking [user] catalog tables vs 2pc vs logical rep > > > > Hi. > > > > The attached PG docs patch about catalog deadlocks was previously > > implemented in another thread [1], but it seems more relevant to this one. > > > > PSA. > Thank you for providing the patch. > I have updated your patch to include some other viewpoints. > > For example, CLUSTER command scenario > that also causes hang of PREPARE in synchronous mode. > We get this deadlock, using the 2PC patch-set. > > FYI, the scenario is > (1) create a table with a trigger > (2) create pub and sub in synchronous mode > (3) then, execute CLUSTER pg_trigger USING pg_trigger_oid_index, > and do some operations (e.g. INSERT) on the trigger-attached table and PREPARE > > The mechanism of this is > walsender tries to take a lock on pg_trigger if the table has a trigger, > but, pg_trigger is already locked by the CLUSTER command, which leads to the deadlock. > Then, this scenario requires some operations on the table which has trigger > because it invokes the walsender to take the lock described above. > > I also included the description about TRUNCATE on user_catalog_table > in the patch. Please have a look at this patch. 1) I was not able to generate html docs with the attached patch: logicaldecoding.sgml:1128: element sect1: validity error : Element sect1 content does not follow the DTD, expecting (sect1info? , (title , subtitle? , titleabbrev?) , (toc | lot | index | glossary | bibliography)* , (((calloutlist | glosslist | bibliolist | itemizedlist | orderedlist | segmentedlist | simplelist | variablelist | caution | important | note | tip | warning | literallayout | programlisting | programlistingco | screen | screenco | screenshot | synopsis | cmdsynopsis | funcsynopsis | classsynopsis | fieldsynopsis | constructorsynopsis | destructorsynopsis | methodsynopsis | formalpara | para | simpara | address | blockquote | graphic | graphicco | mediaobject | mediaobjectco | informalequation | informalexample | informalfigure | informaltable | equation | example | figure | table | msgset | procedure | sidebar | qandaset | task | anchor | bridgehead | remark | highlights | abstract | authorblurb | epigraph | indexterm | beginpage)+ , (refentry* | sect2* | simplesect*)) | refentry+ | sect2+ | simplesect+) , (toc | lot | index | glossary | bibliography)*), got (title sect2 sect2 note ) </sect1> 2) You could change hang to deadlock: + logical decoding of published table within the same transaction leads to a hang. Regards, Vignesh
On Thu, Jun 3, 2021 at 9:18 AM osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com> wrote: > > On Tuesday, June 1, 2021 4:33 PM Peter Smith <smithpb2250@gmail.com> > > To: Andres Freund <andres@anarazel.de> > > Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>; Amit Kapila > > <amit.kapila16@gmail.com>; Markus Wanner > > <markus.wanner@enterprisedb.com> > > Subject: Re: locking [user] catalog tables vs 2pc vs logical rep > > > > Hi. > > > > The attached PG docs patch about catalog deadlocks was previously > > implemented in another thread [1], but it seems more relevant to this one. > > > > PSA. > Thank you for providing the patch. > I have updated your patch to include some other viewpoints. > I suggest creating a synchronous replication part of the patch for back-branches as well. -- With Regards, Amit Kapila.
On Thursday, June 3, 2021 7:07 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Thu, Jun 3, 2021 at 9:18 AM osumi.takamichi@fujitsu.com
> <osumi.takamichi@fujitsu.com> wrote:
> > Thank you for providing the patch.
> > I have updated your patch to include some other viewpoints.
> >
>
> I suggest creating a synchronous replication part of the patch for
> back-branches as well.
You are right. Please have a look at the attached patch-set.
Needless to say, the patch for HEAD has descriptions that depend on
the 2pc patch-set.
Best Regards,
Takamichi Osumi
Attachment
On Thursday, June 3, 2021 1:09 PM vignesh C <vignesh21@gmail.com> wrote:
> On Thu, Jun 3, 2021 at 9:18 AM osumi.takamichi@fujitsu.com
> <osumi.takamichi@fujitsu.com> wrote:
> > Thank you for providing the patch.
> > I have updated your patch to include some other viewpoints.
> >
> > I also included the description about TRUNCATE on user_catalog_table
> > in the patch. Please have a look at this patch.
>
> 1) I was not able to generate html docs with the attached patch:
> logicaldecoding.sgml:1128: element sect1: validity error : Element...
Thank you for your review.
I fixed the patch to make it pass to generate html output.
Kindly have a look at the v03.
> 2) You could change hang to deadlock:
> + logical decoding of published table within the same
> transaction leads to a hang.
Yes. I included your point. Thanks.
Best Regards,
Takamichi Osumi
On Mon, Jun 7, 2021 at 4:18 AM osumi.takamichi@fujitsu.com
<osumi.takamichi@fujitsu.com> wrote:
>
> On Thursday, June 3, 2021 7:07 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > On Thu, Jun 3, 2021 at 9:18 AM osumi.takamichi@fujitsu.com
> > <osumi.takamichi@fujitsu.com> wrote:
> > > Thank you for providing the patch.
> > > I have updated your patch to include some other viewpoints.
> > >
> >
> > I suggest creating a synchronous replication part of the patch for
> > back-branches as well.
> You are right. Please have a look at the attached patch-set.
> Needless to say, the patch for HEAD has descriptions that depend on
> the 2pc patch-set.
>
1)
+ <para>
+ The use of any command to take an ACCESS EXCLUSIVE lock on
[user] catalog tables
+ can cause the deadlock of logical decoding in synchronous
mode. This means that
+ at the transaction commit or prepared transaction, the command
hangs or the server
+ becomes to block any new connections. To avoid this, users
must refrain from such
+ operations.
+ </para>
Can we change it something like:
Logical decoding of transactions in synchronous replication mode
requires access to system tables and/or user catalog tables, hence
user should refrain from taking exclusive lock on system tables and/or
user catalog tables or refrain from executing commands like cluster
command which will take exclusive lock on system tables internally. If
not the transaction will get blocked at commit/prepare time because of
a deadlock.
2) I was not sure if we should include the examples below or the above
para is enough, we can hear from others and retain it if required:
+ <para>
+ When <command>COMMIT</command> is conducted for a transaction that has
+ issued explicit <command>LOCK</command> on
<structname>pg_class</structname>
+ with logical decoding, the deadlock occurs. Also, committing
one that runs
+ <command>CLUSTER</command> <structname>pg_class</structname> is another
+ deadlock scenario.
+ </para>
+
+ <para>
+ Similarly, executing <command>PREPARE TRANSACTION</command>
+ after <command>LOCK</command> command on
<structname>pg_class</structname> and
+ logical decoding of published table within the same
transaction leads to the deadlock.
+ Clustering <structname>pg_trigger</structname> by
<command>CLUSTER</command> command
+ brings about the deadlock as well, when published table has a
trigger and any operations
+ that will be decoded are conducted on the same table.
+ </para>
+
+ <para>
+ The deadlock can happen when users execute <command>TRUNCATE</command>
+ on user_catalog_table under the condition that output plugin
have reference to it.
</para>
Regards,
Vignesh
On Mon, Jun 7, 2021 at 9:26 AM vignesh C <vignesh21@gmail.com> wrote:
>
> On Mon, Jun 7, 2021 at 4:18 AM osumi.takamichi@fujitsu.com
> <osumi.takamichi@fujitsu.com> wrote:
> >
> > On Thursday, June 3, 2021 7:07 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > On Thu, Jun 3, 2021 at 9:18 AM osumi.takamichi@fujitsu.com
> > > <osumi.takamichi@fujitsu.com> wrote:
> > > > Thank you for providing the patch.
> > > > I have updated your patch to include some other viewpoints.
> > > >
> > >
> > > I suggest creating a synchronous replication part of the patch for
> > > back-branches as well.
> > You are right. Please have a look at the attached patch-set.
> > Needless to say, the patch for HEAD has descriptions that depend on
> > the 2pc patch-set.
> >
>
> 1)
> + <para>
> + The use of any command to take an ACCESS EXCLUSIVE lock on
> [user] catalog tables
> + can cause the deadlock of logical decoding in synchronous
> mode. This means that
> + at the transaction commit or prepared transaction, the command
> hangs or the server
> + becomes to block any new connections. To avoid this, users
> must refrain from such
> + operations.
> + </para>
>
> Can we change it something like:
> Logical decoding of transactions in synchronous replication mode
> requires access to system tables and/or user catalog tables, hence
> user should refrain from taking exclusive lock on system tables and/or
> user catalog tables or refrain from executing commands like cluster
> command which will take exclusive lock on system tables internally. If
> not the transaction will get blocked at commit/prepare time because of
> a deadlock.
>
I think this is better than what the patch has proposed. I suggest
minor modifications to your proposed changes. Let's write the above
para as: "In synchronous replication setup, a deadlock can happen, if
the transaction has locked [user] catalog tables exclusively. This is
because logical decoding of transactions can lock catalog tables to
access them. To avoid this users must refrain from taking an exclusive
lock on [user] catalog tables. This can happen in the following ways:"
+ <para>
+ When <command>COMMIT</command> is conducted for a transaction that has
+ issued explicit <command>LOCK</command> on
<structname>pg_class</structname>
+ with logical decoding, the deadlock occurs. Also, committing
one that runs
+ <command>CLUSTER</command> <structname>pg_class</structname> is another
+ deadlock scenario.
</para>
The above points need to be mentioned in the <itemizedlist> fashion.
See <sect2 id="continuous-archiving-caveats"> for an example. I think
the above point can be split as follows.
<listitem>
<para>
User has issued an explicit <command>LOCK</command> on
<structname>pg_class</structname> (or any other catalog table) in a
transaction. Now when we try to decode such a transaction, a deadlock
can happen.
</para>
</listitem>
Similarly, write separate points for Cluster and Truncate.
One more comment is that for HEAD, first just create a patch with
synchronous replication-related doc changes and then write a separate
patch for prepared transactions.
> 2) I was not sure if we should include the examples below or the above
> para is enough,
>
It is better to give examples but let's use the format as I suggested above.
--
With Regards,
Amit Kapila.
On Mon, Jun 7, 2021 at 10:44 AM Amit Kapila <amit.kapila16@gmail.com> wrote: > > One more comment is that for HEAD, first just create a patch with > synchronous replication-related doc changes and then write a separate > patch for prepared transactions. > I noticed that docs for "Synchronous replication support for Logical Decoding" has been introduced by commit 49c0864d7ef5227faa24f903902db90e5c9d5d69 which goes till 9.6. So, I think you need to create a patch for 9.6 as well unless one of the existing patches already applies in 9.6. -- With Regards, Amit Kapila.
On Monday, June 7, 2021 6:22 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Mon, Jun 7, 2021 at 10:44 AM Amit Kapila <amit.kapila16@gmail.com>
> wrote:
> >
> > One more comment is that for HEAD, first just create a patch with
> > synchronous replication-related doc changes and then write a separate
> > patch for prepared transactions.
> >
>
> I noticed that docs for "Synchronous replication support for Logical Decoding"
> has been introduced by commit
> 49c0864d7ef5227faa24f903902db90e5c9d5d69 which goes till 9.6. So, I think
> you need to create a patch for 9.6 as well unless one of the existing patches
> already applies in 9.6.
OK. I could apply PG10's patch to 9.6.
Also, I've made a separate patch for 2PC description.
On the other hand, I need to mention that
there are some gaps to cause failures to apply patches
between supported versions.
(e.g. applying a patch for HEAD to stable PG13 fails)
To address the gaps between the versions,
I needed to conduct some manual fixes.
Therefore, please note that the content of patch
between PG12 and PG13 are almost same
like PG9.6 and PG10, but, I prepared
independent patches for HEAD and PG11,
in order to make those applied in a comfortable manner.
Kindly have a look at the updated patch-set.
They all passed the test of make html.
Best Regards,
Takamichi Osumi
Attachment
- HEAD_deadlock_documentation_of_logical_decoding_v04.patch
- HEAD_with_2PC_deadlock_documentation_of_logical_decoding_v04.patch
- PG10_deadlock_documentation_of_logical_decoding_v04.patch
- PG11_deadlock_documentation_of_logical_decoding_v04.patch
- PG12_deadlock_documentation_of_logical_decoding_v04.patch
- PG13_deadlock_documentation_of_logical_decoding_v04.patch
- PG96_deadlock_documentation_of_logical_decoding_v04.patch
On Tue, Jun 8, 2021 at 1:34 PM osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com> wrote: > > On Monday, June 7, 2021 6:22 PM Amit Kapila <amit.kapila16@gmail.com> wrote: > > On Mon, Jun 7, 2021 at 10:44 AM Amit Kapila <amit.kapila16@gmail.com> > > wrote: > > > > > > One more comment is that for HEAD, first just create a patch with > > > synchronous replication-related doc changes and then write a separate > > > patch for prepared transactions. > > > > > > > I noticed that docs for "Synchronous replication support for Logical Decoding" > > has been introduced by commit > > 49c0864d7ef5227faa24f903902db90e5c9d5d69 which goes till 9.6. So, I think > > you need to create a patch for 9.6 as well unless one of the existing patches > > already applies in 9.6. > OK. I could apply PG10's patch to 9.6. > Also, I've made a separate patch for 2PC description. > > On the other hand, I need to mention that > there are some gaps to cause failures to apply patches > between supported versions. > (e.g. applying a patch for HEAD to stable PG13 fails) > > To address the gaps between the versions, > I needed to conduct some manual fixes. > Therefore, please note that the content of patch > between PG12 and PG13 are almost same > like PG9.6 and PG10, but, I prepared > independent patches for HEAD and PG11, > in order to make those applied in a comfortable manner. > > > Kindly have a look at the updated patch-set. > They all passed the test of make html. Thanks for the updated patch. I have few comments: 1) Should we list the actual system tables like pg_class,pg_trigger, etc instead of any other catalog table? User has issued an explicit LOCK on pg_class (or any other catalog table) 2) Here This means deadlock, after this we mention deadlock again for each of the examples, we can remove it if redundant. This can happen in the following ways: 3) Should [user] catalog tables be catalog tables or user catalog tables [user] catalog tables Regards, Vignesh
On Tue, Jun 8, 2021 at 6:24 PM vignesh C <vignesh21@gmail.com> wrote: > > Thanks for the updated patch. > > I have few comments: > 1) Should we list the actual system tables like pg_class,pg_trigger, > etc instead of any other catalog table? > User has issued an explicit LOCK on pg_class (or any other catalog table) > I think the way it is mentioned is okay. We don't need to specify other catalog tables. > 2) Here This means deadlock, after this we mention deadlock again for > each of the examples, we can remove it if redundant. > This can happen in the following ways: > 3) Should [user] catalog tables be catalog tables or user catalog tables > [user] catalog tables > The third point is not clear. Can you please elaborate by quoting the exact change from the patch? -- With Regards, Amit Kapila.
On Wednesday, June 9, 2021 12:06 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Tue, Jun 8, 2021 at 6:24 PM vignesh C <vignesh21@gmail.com> wrote:
> >
> > Thanks for the updated patch.
> >
> > I have few comments:
> > 1) Should we list the actual system tables like pg_class,pg_trigger,
> > etc instead of any other catalog table?
> > User has issued an explicit LOCK on pg_class (or any other catalog
> > table)
> >
>
> I think the way it is mentioned is okay. We don't need to specify other catalog
> tables.
Okay.
> > 2) Here This means deadlock, after this we mention deadlock again for
> > each of the examples, we can remove it if redundant.
> > This can happen in the following ways:
I think this sentence works to notify that commands described below
are major scenarios naturally, to the readers. Then, I don't want to remove it.
If you somehow feel that the descriptions are redundant,
how about unifying all listitems as nouns. like below ?
* An explicit <command>LOCK</command> on <structname>pg_class</structname> (or any other catalog table) in a
transaction
* Reordering <structname>pg_class</structname> by <command>CLUSTER</command> command in a transaction
* Executing <command>TRUNCATE</command> on user_catalog_table
> > 3) Should [user] catalog tables be catalog tables or user catalog
> > tables [user] catalog tables
> >
>
> The third point is not clear. Can you please elaborate by quoting the exact
> change from the patch?
IIUC, he means to replace all descriptions "[user] catalog tables"
with "catalog tables or user catalog tables" in the patch,
because seemingly we don't use square brackets to describe optional clause in
normal descriptions(like outside of synopsis and I don't find any example for this).
But, even if so, I would like to keep the current square brackets description,
which makes sentence short and simple.
Best Regards,
Takamichi Osumi
On Wed, Jun 9, 2021 at 12:03 PM osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com> wrote: > > On Wednesday, June 9, 2021 12:06 PM Amit Kapila <amit.kapila16@gmail.com> wrote: > > On Tue, Jun 8, 2021 at 6:24 PM vignesh C <vignesh21@gmail.com> wrote: > > > > 3) Should [user] catalog tables be catalog tables or user catalog > > > tables [user] catalog tables > > > > > > > The third point is not clear. Can you please elaborate by quoting the exact > > change from the patch? > IIUC, he means to replace all descriptions "[user] catalog tables" > with "catalog tables or user catalog tables" in the patch, > because seemingly we don't use square brackets to describe optional clause in > normal descriptions(like outside of synopsis and I don't find any example for this). > But, even if so, I would like to keep the current square brackets description, > which makes sentence short and simple. > +1. -- With Regards, Amit Kapila.
On Tuesday, June 8, 2021 5:04 PM I wrote:
> On Monday, June 7, 2021 6:22 PM Amit Kapila <amit.kapila16@gmail.com>
> wrote:
> > On Mon, Jun 7, 2021 at 10:44 AM Amit Kapila <amit.kapila16@gmail.com>
> > wrote:
> > >
> > > One more comment is that for HEAD, first just create a patch with
> > > synchronous replication-related doc changes and then write a
> > > separate patch for prepared transactions.
> >
> > I noticed that docs for "Synchronous replication support for Logical Decoding"
> > has been introduced by commit
> > 49c0864d7ef5227faa24f903902db90e5c9d5d69 which goes till 9.6. So, I
> > think you need to create a patch for 9.6 as well unless one of the
> > existing patches already applies in 9.6.
> OK. I could apply PG10's patch to 9.6.
> Also, I've made a separate patch for 2PC description.
>
> On the other hand, I need to mention that there are some gaps to cause failures
> to apply patches between supported versions.
> (e.g. applying a patch for HEAD to stable PG13 fails)
I scrutinized this POV and checked the gaps between supported versions.
In terms of the section where the patch want to fix,
there are only 2 major gaps between PG10 and PG11 - [1]
and between PG13 and HEAD - [2]. In other words,
the patch-set should be 4 types.
* patch for HEAD
* additional patch for HEAD based on 2PC patch-set
* patch for from PG11 to PG13
* patch for PG9.6 and PG10
> To address the gaps between the versions, I needed to conduct some manual
> fixes.
> Therefore, please note that the content of patch between PG12 and PG13 are
> almost same like PG9.6 and PG10, but, I prepared independent patches for
> HEAD and PG11, in order to make those applied in a comfortable manner.
Therefore, I was wrong.
I didn't need the specific independent patch for PG11.
I'll fix the patch-set accordingly in the next version.
[1] how we finish xref tag is different between PG10 and PG11
--- logicaldecoding.sgml_PG11 2021-06-09 04:38:18.214163527 +0000
+++ logicaldecoding.sgml_PG10 2021-06-09 04:37:50.533163527 +0000
@@ -730,9 +698,9 @@
replication</link> solutions with the same user interface as synchronous
replication for <link linkend="streaming-replication">streaming
replication</link>. To do this, the streaming replication interface
- (see <xref linkend="logicaldecoding-walsender"/>) must be used to stream out
+ (see <xref linkend="logicaldecoding-walsender">) must be used to stream out
data. Clients have to send <literal>Standby status update (F)</literal>
- (see <xref linkend="protocol-replication"/>) messages, just like streaming
+ (see <xref linkend="protocol-replication">) messages, just like streaming
replication clients do.
</para>
[2] in HEAD, we have a new sect1 after "Synchronous Replication Support for Logical Decoding"
--- logicaldecoding.sgml_PG13 2021-06-09 05:10:34.927163527 +0000
+++ logicaldecoding.sgml_HEAD 2021-06-09 05:08:12.810163527 +0000
@@ -747,4 +1089,177 @@
</para>
</note>
</sect1>
+
+ <sect1 id="logicaldecoding-streaming">
+ <title>Streaming of Large Transactions for Logical Decoding</title>
+
+ <para>
+ The basic output plugin callbacks (e.g., <function>begin_cb</function>,
...
Best Regards,
Takamichi Osumi
On Wed, Jun 9, 2021 at 12:03 PM osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com> wrote: > > On Wednesday, June 9, 2021 12:06 PM Amit Kapila <amit.kapila16@gmail.com> wrote: > > On Tue, Jun 8, 2021 at 6:24 PM vignesh C <vignesh21@gmail.com> wrote: > > > > > > Thanks for the updated patch. > > > > > > I have few comments: > > > 1) Should we list the actual system tables like pg_class,pg_trigger, > > > etc instead of any other catalog table? > > > User has issued an explicit LOCK on pg_class (or any other catalog > > > table) > > > > > > > I think the way it is mentioned is okay. We don't need to specify other catalog > > tables. > Okay. > > > > > 2) Here This means deadlock, after this we mention deadlock again for > > > each of the examples, we can remove it if redundant. > > > This can happen in the following ways: > I think this sentence works to notify that commands described below > are major scenarios naturally, to the readers. Then, I don't want to remove it. > > If you somehow feel that the descriptions are redundant, > how about unifying all listitems as nouns. like below ? > > * An explicit <command>LOCK</command> on <structname>pg_class</structname> (or any other catalog table) in a transaction > * Reordering <structname>pg_class</structname> by <command>CLUSTER</command> command in a transaction > * Executing <command>TRUNCATE</command> on user_catalog_table > This looks good to me. Keep the 2PC documentation patch also on the same lines. Regards, Vignesh
On Thursday, June 10, 2021 1:14 PM vignesh C <vignesh21@gmail.com>
> On Wed, Jun 9, 2021 at 12:03 PM osumi.takamichi@fujitsu.com
> <osumi.takamichi@fujitsu.com> wrote:
> >
> > On Wednesday, June 9, 2021 12:06 PM Amit Kapila
> <amit.kapila16@gmail.com> wrote:
> > > On Tue, Jun 8, 2021 at 6:24 PM vignesh C <vignesh21@gmail.com> wrote:
> > > >
> > > > Thanks for the updated patch.
> > > >
> > > > I have few comments:
> > > > 1) Should we list the actual system tables like
> > > > pg_class,pg_trigger, etc instead of any other catalog table?
> > > > User has issued an explicit LOCK on pg_class (or any other catalog
> > > > table)
> > > >
> > >
> > > I think the way it is mentioned is okay. We don't need to specify
> > > other catalog tables.
> > Okay.
> >
> >
> > > > 2) Here This means deadlock, after this we mention deadlock again
> > > > for each of the examples, we can remove it if redundant.
> > > > This can happen in the following ways:
> > I think this sentence works to notify that commands described below
> > are major scenarios naturally, to the readers. Then, I don't want to remove
> it.
> >
> > If you somehow feel that the descriptions are redundant, how about
> > unifying all listitems as nouns. like below ?
> >
> > * An explicit <command>LOCK</command> on
> > <structname>pg_class</structname> (or any other catalog table) in a
> > transaction
> > * Reordering <structname>pg_class</structname> by
> > <command>CLUSTER</command> command in a transaction
> > * Executing <command>TRUNCATE</command> on user_catalog_table
> >
>
> This looks good to me. Keep the 2PC documentation patch also on the same
> lines.
Yeah, of course. Thanks for your confirmation.
Best Regards,
Takamichi Osumi
On Thursday, June 10, 2021 1:30 PM I wrote:
> On Thursday, June 10, 2021 1:14 PM vignesh C <vignesh21@gmail.com>
> > On Wed, Jun 9, 2021 at 12:03 PM osumi.takamichi@fujitsu.com
> > <osumi.takamichi@fujitsu.com> wrote:
> > >
> > > On Wednesday, June 9, 2021 12:06 PM Amit Kapila
> > <amit.kapila16@gmail.com> wrote:
> > > > On Tue, Jun 8, 2021 at 6:24 PM vignesh C <vignesh21@gmail.com>
> wrote:
> > > > >
> > > > > Thanks for the updated patch.
> > > > >
> > > > > I have few comments:
> > > > > 1) Should we list the actual system tables like
> > > > > pg_class,pg_trigger, etc instead of any other catalog table?
> > > > > User has issued an explicit LOCK on pg_class (or any other
> > > > > catalog
> > > > > table)
> > > > >
> > > >
> > > > I think the way it is mentioned is okay. We don't need to specify
> > > > other catalog tables.
> > > Okay.
> > >
> > >
> > > > > 2) Here This means deadlock, after this we mention deadlock
> > > > > again for each of the examples, we can remove it if redundant.
> > > > > This can happen in the following ways:
> > > I think this sentence works to notify that commands described below
> > > are major scenarios naturally, to the readers. Then, I don't want to
> > > remove
> > it.
> > >
> > > If you somehow feel that the descriptions are redundant, how about
> > > unifying all listitems as nouns. like below ?
> > >
> > > * An explicit <command>LOCK</command> on
> > > <structname>pg_class</structname> (or any other catalog table) in a
> > > transaction
> > > * Reordering <structname>pg_class</structname> by
> > > <command>CLUSTER</command> command in a transaction
> > > * Executing <command>TRUNCATE</command> on
> user_catalog_table
> > >
> >
> > This looks good to me. Keep the 2PC documentation patch also on the
> > same lines.
> Yeah, of course. Thanks for your confirmation.
Hi, attached the updated patch-set.
I've conducted some updates.
(1) Added commit messages for all patches
(2) Sorted out the descriptions of listitem to make them look uniform
(3) Removed PG11-specific patch and unified the patch from PG11 to PG13,
which will keep the documents cleanliness for future back-patching, if any.
(4) Removed unnecessary space after 'id'
In v04, there was an unneeded space like below. Fixed.
In the same logicaldecoding.sgml doc, there is no space after 'id' for sec2.
+ <sect2 id ="logicaldecoding-synchronous-caveats">
+ <title>Caveats</title>
(5) Fixed the reference accurately by replacing link tag with xref tag.
In v04, I let the reference be inaccurate, because the linkend points to the caveats
but the link word was "Synchronous Replication Support for Logical Decoding".
+ [user] catalog tables exclusively. To avoid this users must refrain from
+ having locks on catalog tables (e.g. explicit <command>LOCK</command> command)
+ in such transactions.
+ (See <link linkend="logicaldecoding-synchronous-caveats">Synchronous
+ Replication Support for Logical Decoding</link> for the details.)
So, in v05, I've fixed this to point out the caveats directly.
+ [user] catalog tables exclusively. To avoid this users must refrain from
+ having locks on catalog tables (e.g. explicit <command>LOCK</command> command)
+ in such transactions.
+ (See <xref linkend="logicaldecoding-synchronous-caveats"/> for the details.)
Kindly have a look at the patch-set.
Best Regards,
Takamichi Osumi
Attachment
- HEAD_deadlock_documentation_of_logical_decoding_v05.patch
- HEAD_with_2PC_deadlock_documentation_of_logical_decoding_v05.patch
- PG10_deadlock_documentation_of_logical_decoding_v05.patch
- PG11_deadlock_documentation_of_logical_decoding_v05.patch
- PG12_deadlock_documentation_of_logical_decoding_v05.patch
- PG13_deadlock_documentation_of_logical_decoding_v05.patch
- PG96_deadlock_documentation_of_logical_decoding_v05.patch
On Fri, Jun 11, 2021 at 6:57 AM osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com> wrote: > > On Thursday, June 10, 2021 1:30 PM I wrote: > > On Thursday, June 10, 2021 1:14 PM vignesh C <vignesh21@gmail.com> > > > On Wed, Jun 9, 2021 at 12:03 PM osumi.takamichi@fujitsu.com > > > <osumi.takamichi@fujitsu.com> wrote: > > > > > > > > On Wednesday, June 9, 2021 12:06 PM Amit Kapila > > > <amit.kapila16@gmail.com> wrote: > > > > > On Tue, Jun 8, 2021 at 6:24 PM vignesh C <vignesh21@gmail.com> > > wrote: > > > > > > > > > > > > Thanks for the updated patch. > > > > > > > > > > > > I have few comments: > > > > > > 1) Should we list the actual system tables like > > > > > > pg_class,pg_trigger, etc instead of any other catalog table? > > > > > > User has issued an explicit LOCK on pg_class (or any other > > > > > > catalog > > > > > > table) > > > > > > > > > > > > > > > > I think the way it is mentioned is okay. We don't need to specify > > > > > other catalog tables. > > > > Okay. > > > > > > > > > > > > > > 2) Here This means deadlock, after this we mention deadlock > > > > > > again for each of the examples, we can remove it if redundant. > > > > > > This can happen in the following ways: > > > > I think this sentence works to notify that commands described below > > > > are major scenarios naturally, to the readers. Then, I don't want to > > > > remove > > > it. > > > > > > > > If you somehow feel that the descriptions are redundant, how about > > > > unifying all listitems as nouns. like below ? > > > > > > > > * An explicit <command>LOCK</command> on > > > > <structname>pg_class</structname> (or any other catalog table) in a > > > > transaction > > > > * Reordering <structname>pg_class</structname> by > > > > <command>CLUSTER</command> command in a transaction > > > > * Executing <command>TRUNCATE</command> on > > user_catalog_table > > > > > > > > > > This looks good to me. Keep the 2PC documentation patch also on the > > > same lines. > > Yeah, of course. Thanks for your confirmation. > Hi, attached the updated patch-set. > > I've conducted some updates. > > (1) Added commit messages for all patches > (2) Sorted out the descriptions of listitem to make them look uniform > (3) Removed PG11-specific patch and unified the patch from PG11 to PG13, > which will keep the documents cleanliness for future back-patching, if any. > > (4) Removed unnecessary space after 'id' > > In v04, there was an unneeded space like below. Fixed. > In the same logicaldecoding.sgml doc, there is no space after 'id' for sec2. > > + <sect2 id ="logicaldecoding-synchronous-caveats"> > + <title>Caveats</title> > > (5) Fixed the reference accurately by replacing link tag with xref tag. > > In v04, I let the reference be inaccurate, because the linkend points to the caveats > but the link word was "Synchronous Replication Support for Logical Decoding". > > + [user] catalog tables exclusively. To avoid this users must refrain from > + having locks on catalog tables (e.g. explicit <command>LOCK</command> command) > + in such transactions. > + (See <link linkend="logicaldecoding-synchronous-caveats">Synchronous > + Replication Support for Logical Decoding</link> for the details.) > > So, in v05, I've fixed this to point out the caveats directly. > > + [user] catalog tables exclusively. To avoid this users must refrain from > + having locks on catalog tables (e.g. explicit <command>LOCK</command> command) > + in such transactions. > + (See <xref linkend="logicaldecoding-synchronous-caveats"/> for the details.) > > Kindly have a look at the patch-set. > Thanks for the updated patch: Few comments: 1) We have used Reordering and Clustering for the same command, we could rephrase similarly in both places. + <para> + Reordering <structname>pg_class</structname> by <command>CLUSTER</command> + command in a transaction. + </para> + <para> + Clustering <structname>pg_trigger</structname> and decoding <command>PREPARE + TRANSACTION</command>, if any published table have a trigger and any + operations that will be decoded are conducted. + </para> + </listitem> 2) Here user_catalog_table should be user catalog table + <para> + Executing <command>TRUNCATE</command> on user_catalog_table in a transaction. + </para> Regards, Vignesh
On Friday, June 11, 2021 2:13 PM vignesh C <vignesh21@gmail.com> wrote:
> Thanks for the updated patch:
> Few comments:
> 1) We have used Reordering and Clustering for the same command, we could
> rephrase similarly in both places.
> + <para>
> + Reordering <structname>pg_class</structname> by
> <command>CLUSTER</command>
> + command in a transaction.
> + </para>
>
> + <para>
> + Clustering <structname>pg_trigger</structname> and decoding
> <command>PREPARE
> + TRANSACTION</command>, if any published table have a trigger
> and any
> + operations that will be decoded are conducted.
> + </para>
> + </listitem>
>
> 2) Here user_catalog_table should be user catalog table
> + <para>
> + Executing <command>TRUNCATE</command> on
> user_catalog_table
> in a transaction.
> + </para>
Thanks for your review.
Attached the patch-set that addressed those two comments.
I also fixed the commit message a bit in the 2PC specific patch to HEAD.
No other changes.
Please check.
Best Regards,
Takamichi Osumi
Attachment
- HEAD_deadlock_documentation_of_logical_decoding_v06.patch
- HEAD_with_2PC_deadlock_documentation_of_logical_decoding_v06.patch
- PG10_deadlock_documentation_of_logical_decoding_v06.patch
- PG11_deadlock_documentation_of_logical_decoding_v06.patch
- PG12_deadlock_documentation_of_logical_decoding_v06.patch
- PG13_deadlock_documentation_of_logical_decoding_v06.patch
- PG96_deadlock_documentation_of_logical_decoding_v06.patch
On Mon, Jun 14, 2021 at 5:33 PM osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com> wrote: > > On Friday, June 11, 2021 2:13 PM vignesh C <vignesh21@gmail.com> wrote: > > Thanks for the updated patch: > > Few comments: > > 1) We have used Reordering and Clustering for the same command, we could > > rephrase similarly in both places. > > + <para> > > + Reordering <structname>pg_class</structname> by > > <command>CLUSTER</command> > > + command in a transaction. > > + </para> > > > > + <para> > > + Clustering <structname>pg_trigger</structname> and decoding > > <command>PREPARE > > + TRANSACTION</command>, if any published table have a trigger > > and any > > + operations that will be decoded are conducted. > > + </para> > > + </listitem> > > > > 2) Here user_catalog_table should be user catalog table > > + <para> > > + Executing <command>TRUNCATE</command> on > > user_catalog_table > > in a transaction. > > + </para> > Thanks for your review. > > Attached the patch-set that addressed those two comments. > I also fixed the commit message a bit in the 2PC specific patch to HEAD. > No other changes. > > Please check. Thanks for the updated patches, the patch applies cleanly in all branches. Please add a commitfest entry for this, so that we don't miss it. Regards, Vignesh
On Tuesday, June 15, 2021 1:51 PM vignesh C <vignesh21@gmail.com> wrote: > > Attached the patch-set that addressed those two comments. > > I also fixed the commit message a bit in the 2PC specific patch to HEAD. > > No other changes. > > > > Please check. > > Thanks for the updated patches, the patch applies cleanly in all branches. > Please add a commitfest entry for this, so that we don't miss it. Thank you. I've registered the patch-set in [1]. I'll wait for other reviews from other developers, if any. [1] - https://commitfest.postgresql.org/33/3170/ Best Regards, Takamichi Osumi
On Mon, Jun 14, 2021 at 5:33 PM osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com> wrote: > > On Friday, June 11, 2021 2:13 PM vignesh C <vignesh21@gmail.com> wrote: > > Attached the patch-set that addressed those two comments. > Few minor comments: 1. + <listitem> + <para> + Clustering <structname>pg_class</structname> in a transaction. Can we change above to: Perform <command>CLUSTER</command> on <structname>pg_class</structname> in a transaction. 2. + <listitem> + <para> + Executing <command>TRUNCATE</command> on user catalog table in a transaction. + </para> Square brackets are missing for user. 3. + <indexterm> + <primary>Overview</primary> + </indexterm> .. .. + <indexterm> + <primary>Caveats</primary> + </indexterm> Why are these required when we already have titles? I have seen other places in the docs where we use titles for Overview and Caveats but they didn't have similar usage. 4. <para> + Performing <command>PREPARE TRANSACTION</command> after <command>LOCK</command> + command on <structname>pg_class</structname> and logical decoding of published + table. Can we change above to: <command>PREPARE TRANSACTION</command> after <command>LOCK</command> command on <structname>pg_class</structname> and allow logical decoding of two-phase transactions. 5. + <para> + Clustering <structname>pg_trigger</structname> and decoding <command>PREPARE + TRANSACTION</command>, if any published table have a trigger and any + operations that will be decoded are conducted. + </para> Can we change above to: <command>PREPARE TRANSACTION</command> after <command>CLUSTER</command> command on <structname>pg_trigger</structname> and allow logical decoding of two-phase transactions. This will lead to deadlock only when published table have a trigger. -- With Regards, Amit Kapila.
On Wed, Jun 16, 2021 at 3:51 PM Amit Kapila <amit.kapila16@gmail.com> wrote: > > On Mon, Jun 14, 2021 at 5:33 PM osumi.takamichi@fujitsu.com > <osumi.takamichi@fujitsu.com> wrote: > > > > On Friday, June 11, 2021 2:13 PM vignesh C <vignesh21@gmail.com> wrote: > > > > Attached the patch-set that addressed those two comments. > > > > Few minor comments: > 1. > + <listitem> > + <para> > + Clustering <structname>pg_class</structname> in a transaction. > > Can we change above to: Perform <command>CLUSTER</command> on > <structname>pg_class</structname> in a transaction. > > 2. > + <listitem> > + <para> > + Executing <command>TRUNCATE</command> on user catalog table > in a transaction. > + </para> > > Square brackets are missing for user. > > 3. > + <indexterm> > + <primary>Overview</primary> > + </indexterm> > .. > .. > + <indexterm> > + <primary>Caveats</primary> > + </indexterm> > > Why are these required when we already have titles? I have seen other > places in the docs where we use titles for Overview and Caveats but > they didn't have similar usage. > Even I felt this was not required. I had checked other places and also prepared doc by removing it, it works fine. Regards, Vignesh
On Wednesday, June 16, 2021 7:21 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Mon, Jun 14, 2021 at 5:33 PM osumi.takamichi@fujitsu.com
> <osumi.takamichi@fujitsu.com> wrote:
> >
> > On Friday, June 11, 2021 2:13 PM vignesh C <vignesh21@gmail.com>
> wrote:
> >
> > Attached the patch-set that addressed those two comments.
> >
>
> Few minor comments:
> 1.
> + <listitem>
> + <para>
> + Clustering <structname>pg_class</structname> in a transaction.
>
> Can we change above to: Perform <command>CLUSTER</command> on
> <structname>pg_class</structname> in a transaction.
Looks better.
>
> 2.
> + <listitem>
> + <para>
> + Executing <command>TRUNCATE</command> on user catalog
> table
> in a transaction.
> + </para>
>
> Square brackets are missing for user.
Thanks for catching it. You are right.
> 3.
> + <indexterm>
> + <primary>Overview</primary>
> + </indexterm>
> ..
> ..
> + <indexterm>
> + <primary>Caveats</primary>
> + </indexterm>
>
> Why are these required when we already have titles? I have seen other places
> in the docs where we use titles for Overview and Caveats but they didn't have
> similar usage.
Sorry, this was a mistake. We didn't need those sections.
> 4.
> <para>
> + Performing <command>PREPARE TRANSACTION</command>
> after
> <command>LOCK</command>
> + command on <structname>pg_class</structname> and logical
> decoding of published
> + table.
>
> Can we change above to: <command>PREPARE
> TRANSACTION</command> after <command>LOCK</command>
> command on <structname>pg_class</structname> and allow logical
> decoding of two-phase transactions.
>
> 5.
> + <para>
> + Clustering <structname>pg_trigger</structname> and decoding
> <command>PREPARE
> + TRANSACTION</command>, if any published table have a trigger
> and any
> + operations that will be decoded are conducted.
> + </para>
>
> Can we change above to: <command>PREPARE
> TRANSACTION</command> after <command>CLUSTER</command>
> command on <structname>pg_trigger</structname> and allow logical
> decoding of two-phase transactions. This will lead to deadlock only when
> published table have a trigger.
Yeah, I needed the nuance to turn on logical decoding of two-phase transactions...
Your above suggestions are much tidier and more accurate than mine.
I agree with your all suggestions.
Best Regards,
Takamichi Osumi
On Thu, Jun 17, 2021 at 8:41 AM osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com> wrote: > > On Wednesday, June 16, 2021 7:21 PM Amit Kapila <amit.kapila16@gmail.com> wrote: > > On Mon, Jun 14, 2021 at 5:33 PM osumi.takamichi@fujitsu.com > > <osumi.takamichi@fujitsu.com> wrote: > > > > > > On Friday, June 11, 2021 2:13 PM vignesh C <vignesh21@gmail.com> > > wrote: > > > > > > Attached the patch-set that addressed those two comments. > > > > > > > Few minor comments: > > 1. > > + <listitem> > > + <para> > > + Clustering <structname>pg_class</structname> in a transaction. > > > > Can we change above to: Perform <command>CLUSTER</command> on > > <structname>pg_class</structname> in a transaction. > Looks better. > > > > > 2. > > + <listitem> > > + <para> > > + Executing <command>TRUNCATE</command> on user catalog > > table > > in a transaction. > > + </para> > > > > Square brackets are missing for user. > Thanks for catching it. You are right. > > > > 3. > > + <indexterm> > > + <primary>Overview</primary> > > + </indexterm> > > .. > > .. > > + <indexterm> > > + <primary>Caveats</primary> > > + </indexterm> > > > > Why are these required when we already have titles? I have seen other places > > in the docs where we use titles for Overview and Caveats but they didn't have > > similar usage. > Sorry, this was a mistake. We didn't need those sections. > > > > 4. > > <para> > > + Performing <command>PREPARE TRANSACTION</command> > > after > > <command>LOCK</command> > > + command on <structname>pg_class</structname> and logical > > decoding of published > > + table. > > > > Can we change above to: <command>PREPARE > > TRANSACTION</command> after <command>LOCK</command> > > command on <structname>pg_class</structname> and allow logical > > decoding of two-phase transactions. > > > > 5. > > + <para> > > + Clustering <structname>pg_trigger</structname> and decoding > > <command>PREPARE > > + TRANSACTION</command>, if any published table have a trigger > > and any > > + operations that will be decoded are conducted. > > + </para> > > > > Can we change above to: <command>PREPARE > > TRANSACTION</command> after <command>CLUSTER</command> > > command on <structname>pg_trigger</structname> and allow logical > > decoding of two-phase transactions. This will lead to deadlock only when > > published table have a trigger. > Yeah, I needed the nuance to turn on logical decoding of two-phase transactions... > Your above suggestions are much tidier and more accurate than mine. > I agree with your all suggestions. > Pushed! -- With Regards, Amit Kapila.
On Thu, Jun 17, 2021 at 4:27 PM Amit Kapila <amit.kapila16@gmail.com> wrote: > > On Thu, Jun 17, 2021 at 8:41 AM osumi.takamichi@fujitsu.com > <osumi.takamichi@fujitsu.com> wrote: > > Pushed! > [Responding to Simon's comments] > If LOCK and TRUNCATE is advised against on all user catalog tables, why would CLUSTER only apply to pg_class? Surely itslocking > level is the same as LOCK? > Cluster will also apply to all user catalog tables. I think we can extend it slightly as we have mentioned for Lock. > The use of "[user]" isn't fully explained, so it might not be clear that this applies to both Postgres catalog tables andany user tables > that have been nominated as catalogs. Probably worth linking to the "Capabilities" section to explain. > Sounds reasonable. > It would be worth coalescing the following sections into a single page, since they are just a few lines each: > Streaming Replication Protocol Interface > Logical Decoding SQL Interface > System Catalogs Related to Logical Decoding > I think this is worth considering but we might want to discuss this as a separate change/patch. -- With Regards, Amit Kapila.
On Thu, Jun 17, 2021 at 12:57 PM Amit Kapila <amit.kapila16@gmail.com> wrote: > > On Thu, Jun 17, 2021 at 4:27 PM Amit Kapila <amit.kapila16@gmail.com> wrote: > > > > On Thu, Jun 17, 2021 at 8:41 AM osumi.takamichi@fujitsu.com > > <osumi.takamichi@fujitsu.com> wrote: > > > > Pushed! > > > [Responding to Simon's comments] > > > If LOCK and TRUNCATE is advised against on all user catalog tables, why would CLUSTER only apply to pg_class? Surelyits locking > > level is the same as LOCK? > > > > Cluster will also apply to all user catalog tables. I think we can > extend it slightly as we have mentioned for Lock. OK, good. > > The use of "[user]" isn't fully explained, so it might not be clear that this applies to both Postgres catalog tablesand any user tables > > that have been nominated as catalogs. Probably worth linking to the "Capabilities" section to explain. > > > > Sounds reasonable. > > > It would be worth coalescing the following sections into a single page, since they are just a few lines each: > > Streaming Replication Protocol Interface > > Logical Decoding SQL Interface > > System Catalogs Related to Logical Decoding > > > > I think this is worth considering but we might want to discuss this as > a separate change/patch. Makes sense. Thanks -- Simon Riggs http://www.EnterpriseDB.com/
On Thursday, June 17, 2021 10:34 PM Simon Riggs <simon.riggs@enterprisedb.com> wrote:
> On Thu, Jun 17, 2021 at 12:57 PM Amit Kapila <amit.kapila16@gmail.com>
> wrote:
> > On Thu, Jun 17, 2021 at 4:27 PM Amit Kapila <amit.kapila16@gmail.com>
> wrote:
> > >
> > > On Thu, Jun 17, 2021 at 8:41 AM osumi.takamichi@fujitsu.com
> > > <osumi.takamichi@fujitsu.com> wrote:
> > >
> > > Pushed!
> > >
> > [Responding to Simon's comments]
> >
> > > If LOCK and TRUNCATE is advised against on all user catalog tables,
> > > why would CLUSTER only apply to pg_class? Surely its locking level is the
> same as LOCK?
> > >
> >
> > Cluster will also apply to all user catalog tables. I think we can
> > extend it slightly as we have mentioned for Lock.
>
> OK, good.
>
> > > The use of "[user]" isn't fully explained, so it might not be clear
> > > that this applies to both Postgres catalog tables and any user tables that
> have been nominated as catalogs. Probably worth linking to the "Capabilities"
> section to explain.
> > >
> >
> > Sounds reasonable.
Simon, I appreciate your suggestions and yes,
if the user catalog table is referenced by the output plugin,
it can be another cause of the deadlock.
I'm going to post the patch for the those two changes, accordingly.
Best Regards,
Takamichi Osumi
On Friday, June 18, 2021 11:41 AM osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com> wrote:
> On Thursday, June 17, 2021 10:34 PM Simon Riggs
> <simon.riggs@enterprisedb.com> wrote:
> > On Thu, Jun 17, 2021 at 12:57 PM Amit Kapila <amit.kapila16@gmail.com>
> > wrote:
> > > On Thu, Jun 17, 2021 at 4:27 PM Amit Kapila
> > > <amit.kapila16@gmail.com>
> > wrote:
> > > >
> > > > On Thu, Jun 17, 2021 at 8:41 AM osumi.takamichi@fujitsu.com
> > > > <osumi.takamichi@fujitsu.com> wrote:
> > > >
> > > > Pushed!
> > > >
> > > [Responding to Simon's comments]
> > >
> > > > If LOCK and TRUNCATE is advised against on all user catalog
> > > > tables, why would CLUSTER only apply to pg_class? Surely its
> > > > locking level is the
> > same as LOCK?
> > > >
> > >
> > > Cluster will also apply to all user catalog tables. I think we can
> > > extend it slightly as we have mentioned for Lock.
> >
> > OK, good.
> >
> > > > The use of "[user]" isn't fully explained, so it might not be
> > > > clear that this applies to both Postgres catalog tables and any
> > > > user tables that
> > have been nominated as catalogs. Probably worth linking to the
> "Capabilities"
> > section to explain.
> > > >
> > >
> > > Sounds reasonable.
> Simon, I appreciate your suggestions and yes, if the user catalog table is
> referenced by the output plugin, it can be another cause of the deadlock.
>
> I'm going to post the patch for the those two changes, accordingly.
Hi, I've made the patch-set to cover the discussion above for all-supported versions.
Please have a look at those.
Best Regards,
Takamichi Osumi
Attachment
- HEAD_extend_logical_decoding_caveats_in_sync_mode_v01.patch
- PG10_extend_logical_decoding_caveats_in_sync_mode_v01.patch
- PG11_extend_logical_decoding_caveats_in_sync_mode_v01.patch
- PG12_extend_logical_decoding_caveats_in_sync_mode_v01.patch
- PG13_extend_logical_decoding_caveats_in_sync_mode_v01.patch
- PG96_extend_logical_decoding_caveats_in_sync_mode_v01.patch
On Fri, Jun 18, 2021 at 2:25 PM osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com> wrote: > > On Friday, June 18, 2021 11:41 AM osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com> wrote: > > Simon, I appreciate your suggestions and yes, if the user catalog table is > > referenced by the output plugin, it can be another cause of the deadlock. > > > > I'm going to post the patch for the those two changes, accordingly. > Hi, I've made the patch-set to cover the discussion above for all-supported versions. > Please have a look at those. > I have slightly modified your patch, see if the attached looks okay to you? This is just a HEAD patch, I'll modify the back-branch patches accordingly. -- With Regards, Amit Kapila.
Attachment
On Saturday, June 19, 2021 6:51 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Fri, Jun 18, 2021 at 2:25 PM osumi.takamichi@fujitsu.com
> <osumi.takamichi@fujitsu.com> wrote:
> >
> > On Friday, June 18, 2021 11:41 AM osumi.takamichi@fujitsu.com
> <osumi.takamichi@fujitsu.com> wrote:
>
> > > Simon, I appreciate your suggestions and yes, if the user catalog
> > > table is referenced by the output plugin, it can be another cause of the
> deadlock.
> > >
> > > I'm going to post the patch for the those two changes, accordingly.
> > Hi, I've made the patch-set to cover the discussion above for all-supported
> versions.
> > Please have a look at those.
>
> I have slightly modified your patch, see if the attached looks okay to you? This
> is just a HEAD patch, I'll modify the back-branch patches accordingly.
Thank you for updating the patch.
The patch becomes much better. Yet, I have one suggestion.
* doc/src/sgml/logicaldecoding.sgml
<itemizedlist>
<listitem>
<para>
Issuing an explicit <command>LOCK</command> on <structname>pg_class</structname>
- (or any other catalog table) in a transaction.
+ (or any other [user] catalog table) in a transaction.
</para>
</listitem>
<listitem>
<para>
- Perform <command>CLUSTER</command> on <structname>pg_class</structname> in
- a transaction.
+ Perform <command>CLUSTER</command> on <structname>pg_class</structname> (or any
+ other [user] catalog table) in a transaction.
</para>
</listitem>
<listitem>
<para>
<command>PREPARE TRANSACTION</command> after <command>LOCK</command> command
- on <structname>pg_class</structname> and allow logical decoding of two-phase
- transactions.
+ on <structname>pg_class</structname> (or any other [user] catalog table) and
+ allow logical decoding of two-phase transactions.
</para>
</listitem>
<listitem>
<para>
<command>PREPARE TRANSACTION</command> after <command>CLUSTER</command>
- command on <structname>pg_trigger</structname> and allow logical decoding of
- two-phase transactions. This will lead to deadlock only when published table
- have a trigger.
+ command on <structname>pg_trigger</structname> (or any other [user] catalog
+ table) and allow logical decoding of two-phase transactions. This will lead
+ to deadlock only when published table have a trigger.
Now we have the four paren supplementary descriptions,
not to make users miss any other [user] catalog tables.
Because of this, the built output html gives me some redundant
impression, for that parts. Accordingly, couldn't we move them
to outside of the itemizedlist section in a simple manner ?
For example, to insert a sentence below the list,
after removing the paren descriptions in the listitem, which says
"Note that those commands that can cause deadlock apply to not only
explicitly indicated system catalog tables above but also any other [user] catalog table."
Best Regards,
Takamichi Osumi
On Sun, Jun 20, 2021 at 9:28 AM osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com> wrote: > > On Saturday, June 19, 2021 6:51 PM Amit Kapila <amit.kapila16@gmail.com> wrote: > > On Fri, Jun 18, 2021 at 2:25 PM osumi.takamichi@fujitsu.com > > <osumi.takamichi@fujitsu.com> wrote: > > > > > > On Friday, June 18, 2021 11:41 AM osumi.takamichi@fujitsu.com > > <osumi.takamichi@fujitsu.com> wrote: > > > > > > Simon, I appreciate your suggestions and yes, if the user catalog > > > > table is referenced by the output plugin, it can be another cause of the > > deadlock. > > > > > > > > I'm going to post the patch for the those two changes, accordingly. > > > Hi, I've made the patch-set to cover the discussion above for all-supported > > versions. > > > Please have a look at those. > > > > I have slightly modified your patch, see if the attached looks okay to you? This > > is just a HEAD patch, I'll modify the back-branch patches accordingly. > Thank you for updating the patch. > The patch becomes much better. Yet, I have one suggestion. > > * doc/src/sgml/logicaldecoding.sgml > <itemizedlist> > <listitem> > <para> > Issuing an explicit <command>LOCK</command> on <structname>pg_class</structname> > - (or any other catalog table) in a transaction. > + (or any other [user] catalog table) in a transaction. > </para> > </listitem> > > <listitem> > <para> > - Perform <command>CLUSTER</command> on <structname>pg_class</structname> in > - a transaction. > + Perform <command>CLUSTER</command> on <structname>pg_class</structname> (or any > + other [user] catalog table) in a transaction. > </para> > </listitem> > > <listitem> > <para> > <command>PREPARE TRANSACTION</command> after <command>LOCK</command> command > - on <structname>pg_class</structname> and allow logical decoding of two-phase > - transactions. > + on <structname>pg_class</structname> (or any other [user] catalog table) and > + allow logical decoding of two-phase transactions. > </para> > </listitem> > > <listitem> > <para> > <command>PREPARE TRANSACTION</command> after <command>CLUSTER</command> > - command on <structname>pg_trigger</structname> and allow logical decoding of > - two-phase transactions. This will lead to deadlock only when published table > - have a trigger. > + command on <structname>pg_trigger</structname> (or any other [user] catalog > + table) and allow logical decoding of two-phase transactions. This will lead > + to deadlock only when published table have a trigger. > > > Now we have the four paren supplementary descriptions, > not to make users miss any other [user] catalog tables. > Because of this, the built output html gives me some redundant > impression, for that parts. Accordingly, couldn't we move them > to outside of the itemizedlist section in a simple manner ? > > For example, to insert a sentence below the list, > after removing the paren descriptions in the listitem, which says > "Note that those commands that can cause deadlock apply to not only > explicitly indicated system catalog tables above but also any other [user] catalog table." > Sounds reasonable to me. /but also any other/but also to any other/, to seems to be missing in the above line. Kindly send an update patch. -- With Regards, Amit Kapila.
On Sunday, June 20, 2021 3:23 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Sun, Jun 20, 2021 at 9:28 AM osumi.takamichi@fujitsu.com
> <osumi.takamichi@fujitsu.com> wrote:
> > * doc/src/sgml/logicaldecoding.sgml
...
> >
> > Now we have the four paren supplementary descriptions, not to make
> > users miss any other [user] catalog tables.
> > Because of this, the built output html gives me some redundant
> > impression, for that parts. Accordingly, couldn't we move them to
> > outside of the itemizedlist section in a simple manner ?
> >
> > For example, to insert a sentence below the list, after removing the
> > paren descriptions in the listitem, which says "Note that those
> > commands that can cause deadlock apply to not only explicitly
> > indicated system catalog tables above but also any other [user] catalog table."
>
> Sounds reasonable to me. /but also any other/but also to any other/, to
> seems to be missing in the above line. Kindly send an update patch.
Excuse me, I don't understand the second sentence.
I wrote "but also" clause in my example.
Also, attached the patch for the change to the HEAD.
Best Regards,
Takamichi Osumi
Attachment
On Sunday, June 20, 2021 9:50 PM I wrote:
> On Sunday, June 20, 2021 3:23 PM Amit Kapila <amit.kapila16@gmail.com>
> wrote:
> > On Sun, Jun 20, 2021 at 9:28 AM osumi.takamichi@fujitsu.com
> > <osumi.takamichi@fujitsu.com> wrote:
> > > * doc/src/sgml/logicaldecoding.sgml
> ...
> > >
> > > Now we have the four paren supplementary descriptions, not to make
> > > users miss any other [user] catalog tables.
> > > Because of this, the built output html gives me some redundant
> > > impression, for that parts. Accordingly, couldn't we move them to
> > > outside of the itemizedlist section in a simple manner ?
> > >
> > > For example, to insert a sentence below the list, after removing the
> > > paren descriptions in the listitem, which says "Note that those
> > > commands that can cause deadlock apply to not only explicitly
> > > indicated system catalog tables above but also any other [user] catalog
> table."
> >
> > Sounds reasonable to me. /but also any other/but also to any other/,
> > to seems to be missing in the above line. Kindly send an update patch.
> Excuse me, I don't understand the second sentence.
> I wrote "but also" clause in my example.
>
> Also, attached the patch for the change to the HEAD.
I've updated the patch to follow the correction Amit-san mentioned.
Please check.
Best Regards,
Takamichi Osumi
Attachment
On Mon, Jun 21, 2021 at 8:48 AM osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com> wrote: > > On Sunday, June 20, 2021 9:50 PM I wrote: > > On Sunday, June 20, 2021 3:23 PM Amit Kapila <amit.kapila16@gmail.com> > > wrote: > > > On Sun, Jun 20, 2021 at 9:28 AM osumi.takamichi@fujitsu.com > > > <osumi.takamichi@fujitsu.com> wrote: > > > > * doc/src/sgml/logicaldecoding.sgml > > ... > > > > > > > > Now we have the four paren supplementary descriptions, not to make > > > > users miss any other [user] catalog tables. > > > > Because of this, the built output html gives me some redundant > > > > impression, for that parts. Accordingly, couldn't we move them to > > > > outside of the itemizedlist section in a simple manner ? > > > > > > > > For example, to insert a sentence below the list, after removing the > > > > paren descriptions in the listitem, which says "Note that those > > > > commands that can cause deadlock apply to not only explicitly > > > > indicated system catalog tables above but also any other [user] catalog > > table." > > > > > > Sounds reasonable to me. /but also any other/but also to any other/, > > > to seems to be missing in the above line. Kindly send an update patch. > > Excuse me, I don't understand the second sentence. > > I wrote "but also" clause in my example. > > > > Also, attached the patch for the change to the HEAD. > I've updated the patch to follow the correction Amit-san mentioned. > Please check. > Pushed. -- With Regards, Amit Kapila.