Re: POC: postgres_fdw insert batching - Mailing list pgsql-hackers

From Amit Langote
Subject Re: POC: postgres_fdw insert batching
Date
Msg-id CA+HiwqESrO5BKHtWZZTSbK26OWk-Uhx6DYtdVEkpdaoZ=uW5BA@mail.gmail.com
Whole thread Raw
In response to RE: POC: postgres_fdw insert batching  ("tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>)
Responses RE: POC: postgres_fdw insert batching  ("tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>)
List pgsql-hackers
On Sat, Jan 16, 2021 at 12:00 AM tsunakawa.takay@fujitsu.com
<tsunakawa.takay@fujitsu.com> wrote:
> From: Amit Langote <amitlangote09@gmail.com>
> > Okay, so maybe not moving the whole logic into the FDW's
> > BeginForeignModify(), but at least if we move this...
> >
> > @@ -441,6 +449,72 @@ ExecInsert(ModifyTableState *mtstate,
> > +       /*
> > +        * Determine if the FDW supports batch insert and determine the
> > batch
> > +        * size (a FDW may support batching, but it may be disabled for the
> > +        * server/table). Do this only once, at the beginning - we don't want
> > +        * the batch size to change during execution.
> > +        */
> > +       if (resultRelInfo->ri_FdwRoutine->GetForeignModifyBatchSize &&
> > +           resultRelInfo->ri_FdwRoutine->ExecForeignBatchInsert &&
> > +           resultRelInfo->ri_BatchSize == 0)
> > +           resultRelInfo->ri_BatchSize =
> > +
> > resultRelInfo->ri_FdwRoutine->GetForeignModifyBatchSize(resultRelInfo);
> >
> > ...into ExecInitModifyTable(), ExecInsert() only needs the following block:
>
> Does ExecInitModifyTable() know all leaf partitions where the tuples produced by VALUES or SELECT go?  ExecInsert()
doesn'tfind the target leaf partition for the first time through the call to ExecPrepareTupleRouting()?  Leaf
partitionscan have different batch_size settings. 

Good thing you reminded me that this is about inserts, and in that
case no, ExecInitModifyTable() doesn't know all leaf partitions, it
only sees the root table whose batch_size doesn't really matter.  So
it's really ExecInitRoutingInfo() that I would recommend to set
ri_BatchSize; right after this block:

/*
 * If the partition is a foreign table, let the FDW init itself for
 * routing tuples to the partition.
 */
if (partRelInfo->ri_FdwRoutine != NULL &&
    partRelInfo->ri_FdwRoutine->BeginForeignInsert != NULL)
    partRelInfo->ri_FdwRoutine->BeginForeignInsert(mtstate, partRelInfo);

Note that ExecInitRoutingInfo() is called only once for a partition
when it is initialized after being inserted into for the first time.

For a non-partitioned targets, I'd still say set ri_BatchSize in
ExecInitModifyTable().

--
Amit Langote
EDB: http://www.enterprisedb.com



pgsql-hackers by date:

Previous
From: "tsunakawa.takay@fujitsu.com"
Date:
Subject: RE: POC: postgres_fdw insert batching
Next
From: Tomas Vondra
Date:
Subject: Re: list of extended statistics on psql