Thread: Inherited UPDATE/DELETE vs async execution

Inherited UPDATE/DELETE vs async execution

From
Etsuro Fujita
Date:
I noticed this while working on the
EXPLAIN-ANALYZE-for-async-capable-nodes issue:

EXPLAIN (VERBOSE, COSTS OFF)
DELETE FROM async_pt;
                           QUERY PLAN
----------------------------------------------------------------
 Delete on public.async_pt
   Foreign Delete on public.async_p1 async_pt_1
   Foreign Delete on public.async_p2 async_pt_2
   Delete on public.async_p3 async_pt_3
   ->  Append
         ->  Async Foreign Delete on public.async_p1 async_pt_1
               Remote SQL: DELETE FROM public.base_tbl1
         ->  Async Foreign Delete on public.async_p2 async_pt_2
               Remote SQL: DELETE FROM public.base_tbl2
         ->  Seq Scan on public.async_p3 async_pt_3
               Output: async_pt_3.tableoid, async_pt_3.ctid
(11 rows)

DELETE FROM async_pt;
server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
connection to server was lost

The cause for this would be that direct-update plans are mistakenly
treated as async-capable ones, as shown in the EXPLAIN output.  To
fix, I think we should modify postgresPlanDirectModify() so that it
clears the async-capable flag if it is set.  Attached is a patch for
that.  Maybe I am missing something, though.

Best regards,
Etsuro Fujita

Attachment

Re: Inherited UPDATE/DELETE vs async execution

From
Justin Pryzby
Date:
On Sat, May 08, 2021 at 01:20:51AM +0900, Etsuro Fujita wrote:
> I noticed this while working on the
> EXPLAIN-ANALYZE-for-async-capable-nodes issue:
> 
> DELETE FROM async_pt;
> server closed the connection unexpectedly

Confirmed, +Tomas, and added at
https://wiki.postgresql.org/wiki/PostgreSQL_14_Open_Items

-- 
Justin



Re: Inherited UPDATE/DELETE vs async execution

From
Etsuro Fujita
Date:
On Mon, May 10, 2021 at 9:20 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
> On Sat, May 08, 2021 at 01:20:51AM +0900, Etsuro Fujita wrote:
> > I noticed this while working on the
> > EXPLAIN-ANALYZE-for-async-capable-nodes issue:
> >
> > DELETE FROM async_pt;
> > server closed the connection unexpectedly
>
> Confirmed, +Tomas, and added at
> https://wiki.postgresql.org/wiki/PostgreSQL_14_Open_Items

Thanks for that!

Maybe my explanation was not good, but actually, this is a follow-up
for commits 27e1f1456 and 86dc90056, which were independently
discussed and committed, and IIUC, the batch-insert work by Tomas
would not be related to this.  So I’ll work on it unless Tom (or
anyone else) wants to.

Best regards,
Etsuro Fujita



Re: Inherited UPDATE/DELETE vs async execution

From
Amit Langote
Date:
Fujita-san,

On Sat, May 8, 2021 at 1:21 AM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
> I noticed this while working on the
> EXPLAIN-ANALYZE-for-async-capable-nodes issue:
>
> EXPLAIN (VERBOSE, COSTS OFF)
> DELETE FROM async_pt;
>                            QUERY PLAN
> ----------------------------------------------------------------
>  Delete on public.async_pt
>    Foreign Delete on public.async_p1 async_pt_1
>    Foreign Delete on public.async_p2 async_pt_2
>    Delete on public.async_p3 async_pt_3
>    ->  Append
>          ->  Async Foreign Delete on public.async_p1 async_pt_1
>                Remote SQL: DELETE FROM public.base_tbl1
>          ->  Async Foreign Delete on public.async_p2 async_pt_2
>                Remote SQL: DELETE FROM public.base_tbl2
>          ->  Seq Scan on public.async_p3 async_pt_3
>                Output: async_pt_3.tableoid, async_pt_3.ctid
> (11 rows)
>
> DELETE FROM async_pt;
> server closed the connection unexpectedly
>     This probably means the server terminated abnormally
>     before or while processing the request.
> connection to server was lost
>
> The cause for this would be that direct-update plans are mistakenly
> treated as async-capable ones, as shown in the EXPLAIN output.

I guess that happens because the ForeignScan nodes responsible for
scanning or direct-updating/deleting from child foreign tables appear
under an Append as of 86dc90056, whereas before they would appear as
child plans of a ModifyTable node.  IIUC, it's the Append that causes
the async_capable flag to be set in those ForeignScan nodes.

>  To
> fix, I think we should modify postgresPlanDirectModify() so that it
> clears the async-capable flag if it is set.  Attached is a patch for
> that.  Maybe I am missing something, though.

I see that your patch is to disable asynchronous execution in
ForeignScan nodes responsible for direct update/delete, but why not do
the same for other ForeignScan nodes too?  Or the other way around --
is it because fixing the crash that occurs in the former's case would
be a significant undertaking for little gain?

--
Amit Langote
EDB: http://www.enterprisedb.com



Re: Inherited UPDATE/DELETE vs async execution

From
Etsuro Fujita
Date:
Amit-san,

On Mon, May 10, 2021 at 9:21 PM Amit Langote <amitlangote09@gmail.com> wrote:
> On Sat, May 8, 2021 at 1:21 AM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
> > I noticed this while working on the
> > EXPLAIN-ANALYZE-for-async-capable-nodes issue:
> >
> > EXPLAIN (VERBOSE, COSTS OFF)
> > DELETE FROM async_pt;
> >                            QUERY PLAN
> > ----------------------------------------------------------------
> >  Delete on public.async_pt
> >    Foreign Delete on public.async_p1 async_pt_1
> >    Foreign Delete on public.async_p2 async_pt_2
> >    Delete on public.async_p3 async_pt_3
> >    ->  Append
> >          ->  Async Foreign Delete on public.async_p1 async_pt_1
> >                Remote SQL: DELETE FROM public.base_tbl1
> >          ->  Async Foreign Delete on public.async_p2 async_pt_2
> >                Remote SQL: DELETE FROM public.base_tbl2
> >          ->  Seq Scan on public.async_p3 async_pt_3
> >                Output: async_pt_3.tableoid, async_pt_3.ctid
> > (11 rows)
> >
> > DELETE FROM async_pt;
> > server closed the connection unexpectedly
> >     This probably means the server terminated abnormally
> >     before or while processing the request.
> > connection to server was lost
> >
> > The cause for this would be that direct-update plans are mistakenly
> > treated as async-capable ones, as shown in the EXPLAIN output.
>
> I guess that happens because the ForeignScan nodes responsible for
> scanning or direct-updating/deleting from child foreign tables appear
> under an Append as of 86dc90056, whereas before they would appear as
> child plans of a ModifyTable node.  IIUC, it's the Append that causes
> the async_capable flag to be set in those ForeignScan nodes.

That's right.

The inherited update/delete work is great!  Thanks for that!

> >  To
> > fix, I think we should modify postgresPlanDirectModify() so that it
> > clears the async-capable flag if it is set.  Attached is a patch for
> > that.  Maybe I am missing something, though.
>
> I see that your patch is to disable asynchronous execution in
> ForeignScan nodes responsible for direct update/delete, but why not do
> the same for other ForeignScan nodes too?

I just thought it would be better to execute other ForeignScan nodes
asynchronously for performance, if they are async-capable.

> Or the other way around --
> is it because fixing the crash that occurs in the former's case would
> be a significant undertaking for little gain?

Yeah, I think it would be a good idea to support "Async Foreign
Delete" in the former's case.  And actually, I tried to do so, but I
didn't, because it seemed to take time.  I might be missing something,
though.

Thanks!

Best regards,
Etsuro Fujita



Re: Inherited UPDATE/DELETE vs async execution

From
Amit Langote
Date:
Fujita-san,

On Tue, May 11, 2021 at 5:56 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
> On Mon, May 10, 2021 at 9:21 PM Amit Langote <amitlangote09@gmail.com> wrote:
> > On Sat, May 8, 2021 at 1:21 AM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
> > > I noticed this while working on the
> > > EXPLAIN-ANALYZE-for-async-capable-nodes issue:
> > >
> > > EXPLAIN (VERBOSE, COSTS OFF)
> > > DELETE FROM async_pt;
> > >                            QUERY PLAN
> > > ----------------------------------------------------------------
> > >  Delete on public.async_pt
> > >    Foreign Delete on public.async_p1 async_pt_1
> > >    Foreign Delete on public.async_p2 async_pt_2
> > >    Delete on public.async_p3 async_pt_3
> > >    ->  Append
> > >          ->  Async Foreign Delete on public.async_p1 async_pt_1
> > >                Remote SQL: DELETE FROM public.base_tbl1
> > >          ->  Async Foreign Delete on public.async_p2 async_pt_2
> > >                Remote SQL: DELETE FROM public.base_tbl2
> > >          ->  Seq Scan on public.async_p3 async_pt_3
> > >                Output: async_pt_3.tableoid, async_pt_3.ctid
> > > (11 rows)
> > >
> > > DELETE FROM async_pt;
> > > server closed the connection unexpectedly
> > >     This probably means the server terminated abnormally
> > >     before or while processing the request.
> > > connection to server was lost
> > >
> > > The cause for this would be that direct-update plans are mistakenly
> > > treated as async-capable ones, as shown in the EXPLAIN output.
> >
> > I guess that happens because the ForeignScan nodes responsible for
> > scanning or direct-updating/deleting from child foreign tables appear
> > under an Append as of 86dc90056, whereas before they would appear as
> > child plans of a ModifyTable node.  IIUC, it's the Append that causes
> > the async_capable flag to be set in those ForeignScan nodes.
>
> That's right.
>
> The inherited update/delete work is great!  Thanks for that!

Thanks.

> > >  To
> > > fix, I think we should modify postgresPlanDirectModify() so that it
> > > clears the async-capable flag if it is set.  Attached is a patch for
> > > that.  Maybe I am missing something, though.
> >
> > I see that your patch is to disable asynchronous execution in
> > ForeignScan nodes responsible for direct update/delete, but why not do
> > the same for other ForeignScan nodes too?
>
> I just thought it would be better to execute other ForeignScan nodes
> asynchronously for performance, if they are async-capable.

Okay, so I take it that making these ForeignScan nodes (that only
fetch the data) asynchronous doesn't interfere with update/delete
subsequently being performed over presumably the same connection to
the remote server.

> > Or the other way around --
> > is it because fixing the crash that occurs in the former's case would
> > be a significant undertaking for little gain?
>
> Yeah, I think it would be a good idea to support "Async Foreign
> Delete" in the former's case.  And actually, I tried to do so, but I
> didn't, because it seemed to take time.

Ah I see.  I guess it makes sense to prevent such cases in v14 as your
patch does, and revisit this in the future.
--
Amit Langote
EDB: http://www.enterprisedb.com



Re: Inherited UPDATE/DELETE vs async execution

From
Etsuro Fujita
Date:
Amit-san,

On Tue, May 11, 2021 at 9:53 PM Amit Langote <amitlangote09@gmail.com> wrote:
> On Tue, May 11, 2021 at 5:56 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
> > On Mon, May 10, 2021 at 9:21 PM Amit Langote <amitlangote09@gmail.com> wrote:
> > > On Sat, May 8, 2021 at 1:21 AM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:

> > > >  To
> > > > fix, I think we should modify postgresPlanDirectModify() so that it
> > > > clears the async-capable flag if it is set.  Attached is a patch for
> > > > that.  Maybe I am missing something, though.
> > >
> > > I see that your patch is to disable asynchronous execution in
> > > ForeignScan nodes responsible for direct update/delete, but why not do
> > > the same for other ForeignScan nodes too?
> >
> > I just thought it would be better to execute other ForeignScan nodes
> > asynchronously for performance, if they are async-capable.
>
> Okay, so I take it that making these ForeignScan nodes (that only
> fetch the data) asynchronous doesn't interfere with update/delete
> subsequently being performed over presumably the same connection to
> the remote server.

Good point!  I don't think it would interfere with the update/delete,
because in that case postgres_fdw would actually perform the
update/delete and the asynchronous foreign scans serially rather than
concurrently.  (They wouldn't be perfomed in parallel unless they use
different connections, in other words.)

> > > Or the other way around --
> > > is it because fixing the crash that occurs in the former's case would
> > > be a significant undertaking for little gain?
> >
> > Yeah, I think it would be a good idea to support "Async Foreign
> > Delete" in the former's case.  And actually, I tried to do so, but I
> > didn't, because it seemed to take time.
>
> Ah I see.  I guess it makes sense to prevent such cases in v14 as your
> patch does, and revisit this in the future.

+1

Here is a rebased version of the patch.  I'm planning to apply this tommorow.

Thanks for the comment!

Best regards,
Etsuro Fujita

Attachment

Re: Inherited UPDATE/DELETE vs async execution

From
Amit Langote
Date:
On Wed, May 12, 2021 at 6:45 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
> On Tue, May 11, 2021 at 9:53 PM Amit Langote <amitlangote09@gmail.com> wrote:
> > Okay, so I take it that making these ForeignScan nodes (that only
> > fetch the data) asynchronous doesn't interfere with update/delete
> > subsequently being performed over presumably the same connection to
> > the remote server.
>
> Good point!  I don't think it would interfere with the update/delete,
> because in that case postgres_fdw would actually perform the
> update/delete and the asynchronous foreign scans serially rather than
> concurrently.  (They wouldn't be perfomed in parallel unless they use
> different connections, in other words.)

I see, that makes sense.

> > > > Or the other way around --
> > > > is it because fixing the crash that occurs in the former's case would
> > > > be a significant undertaking for little gain?
> > >
> > > Yeah, I think it would be a good idea to support "Async Foreign
> > > Delete" in the former's case.  And actually, I tried to do so, but I
> > > didn't, because it seemed to take time.
> >
> > Ah I see.  I guess it makes sense to prevent such cases in v14 as your
> > patch does, and revisit this in the future.
>
> +1
>
> Here is a rebased version of the patch.  I'm planning to apply this tommorow.

+   /*
+    * Finally, unset the async-capable flag if it is set.
+    */

Would it make sense to expand here even just a bit on why we must do this?

-- 
Amit Langote
EDB: http://www.enterprisedb.com



Re: Inherited UPDATE/DELETE vs async execution

From
Etsuro Fujita
Date:
On Thu, May 13, 2021 at 3:32 PM Amit Langote <amitlangote09@gmail.com> wrote:
> On Wed, May 12, 2021 at 6:45 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
> > Here is a rebased version of the patch.  I'm planning to apply this tommorow.
>
> +   /*
> +    * Finally, unset the async-capable flag if it is set.
> +    */
>
> Would it make sense to expand here even just a bit on why we must do this?

+1  How about something like this?

"Finally, unset the async-capable flag if it is set, as we currently
don't support asynchronous execution of direct modifications."

Best regards,
Etsuro Fujita



Re: Inherited UPDATE/DELETE vs async execution

From
Etsuro Fujita
Date:
On Thu, May 13, 2021 at 5:00 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
>
> On Thu, May 13, 2021 at 3:32 PM Amit Langote <amitlangote09@gmail.com> wrote:
> > On Wed, May 12, 2021 at 6:45 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
> > > Here is a rebased version of the patch.  I'm planning to apply this tommorow.
> >
> > +   /*
> > +    * Finally, unset the async-capable flag if it is set.
> > +    */
> >
> > Would it make sense to expand here even just a bit on why we must do this?
>
> +1  How about something like this?
>
> "Finally, unset the async-capable flag if it is set, as we currently
> don't support asynchronous execution of direct modifications."

Pushed after modifying the comment as such.  I think we could improve
it later.  :-)

Thanks for the comment!

Will close this in the open items list.

Best regards,
Etsuro Fujita



Re: Inherited UPDATE/DELETE vs async execution

From
Amit Langote
Date:
On Thu, May 13, 2021 at 8:10 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
> On Thu, May 13, 2021 at 5:00 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
> > On Thu, May 13, 2021 at 3:32 PM Amit Langote <amitlangote09@gmail.com> wrote:
> > > On Wed, May 12, 2021 at 6:45 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
> > > > Here is a rebased version of the patch.  I'm planning to apply this tommorow.
> > >
> > > +   /*
> > > +    * Finally, unset the async-capable flag if it is set.
> > > +    */
> > >
> > > Would it make sense to expand here even just a bit on why we must do this?
> >
> > +1  How about something like this?
> >
> > "Finally, unset the async-capable flag if it is set, as we currently
> > don't support asynchronous execution of direct modifications."
>
> Pushed after modifying the comment as such.  I think we could improve
> it later.  :-)

Looks good as pushed, thank you.

-- 
Amit Langote
EDB: http://www.enterprisedb.com