Re: POC: postgres_fdw insert batching - Mailing list pgsql-hackers

From Amit Langote
Subject Re: POC: postgres_fdw insert batching
Date
Msg-id CA+HiwqFdmt6YF=fRymQu=7HXFLQomExbSv3d9ztrsh-Yo2-nPg@mail.gmail.com
Whole thread Raw
In response to Re: POC: postgres_fdw insert batching  (Tomas Vondra <tomas.vondra@enterprisedb.com>)
List pgsql-hackers
On Thu, Feb 18, 2021 at 8:38 AM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
> On 2/17/21 8:36 PM, Tomas Vondra wrote:
> > Thanks. The patch seems reasonable, but it's a bit too large for a fix,
> > so I'll go ahead and push one of the previous fixes restricting batching
> > to plain INSERT commands. But this seems useful, so I suggest adding it
> > to the next commitfest.
>
> I've pushed the v4 fix, adding the CMD_INSERT to execPartition.
>
> I think we may need to revise the relationship between FDW and places
> that (may) call GetForeignModifyBatchSize. Currently, these places need
> to be quite well synchronized - in a way, the issue was (partially) due
> to postgres_fdw and core not agreeing on the details.

Agreed.

> In particular, create_foreign_modify sets batch_size for CMD_INSERT and
> leaves it 0 otherwise. So GetForeignModifyBatchSize() returned 0 later,
> triggering an assert.
>
> It's probably better to require GetForeignModifyBatchSize() to always
> return a valid batch size (>= 1). If batching is not supported, just
> return 1.

That makes sense.

How about the attached?

-- 
Amit Langote
EDB: http://www.enterprisedb.com

Attachment

pgsql-hackers by date:

Previous
From: "Hou, Zhijie"
Date:
Subject: RE: Determine parallel-safety of partition relations for Inserts
Next
From: Amit Langote
Date:
Subject: Re: Parallel INSERT (INTO ... SELECT ...)