Re: [POC] Fast COPY FROM command for the table with foreign partitions - Mailing list pgsql-hackers

From Andrey V. Lepikhov
Subject Re: [POC] Fast COPY FROM command for the table with foreign partitions
Date
Msg-id da7ed3f5-b596-2549-3710-4cc2a602ec17@postgrespro.ru
Whole thread Raw
In response to Re: [POC] Fast COPY FROM command for the table with foreign partitions  (Amit Langote <amitlangote09@gmail.com>)
Responses Re: [POC] Fast COPY FROM command for the table with foreign partitions  (Amit Langote <amitlangote09@gmail.com>)
List pgsql-hackers
On 9/9/20 5:51 PM, Amit Langote wrote:
> On Wed, Sep 9, 2020 at 6:42 PM Alexey Kondratov
> <a.kondratov@postgrespro.ru> wrote:
>> On 2020-09-09 11:45, Andrey V. Lepikhov wrote:
>>> This does not seem very convenient and will lead to errors in the
>>> future. So, I agree with Amit.
>>
>> And InitResultRelInfo() may set ri_usesMultiInsert to false by default,
>> since it's used only by COPY now. Then you won't need this in several
>> places:
>>
>> +       resultRelInfo->ri_usesMultiInsert = false;
>>
>> While the logic of turning multi-insert on with all the validations
>> required could be factored out of InitResultRelInfo() to a separate
>> routine.
> 
> Interesting idea.  Maybe better to have a separate routine like Alexey says.
Ok. I rewrited the patch 0001 with the Alexey suggestion.
Patch 0002... required minor changes (new version see in attachment).

Also I added some optimization (see 0003 and 0004 patches). Here we 
execute 'COPY .. FROM  STDIN' at foreign server only once, in the 
BeginForeignCopy routine. It is a proof-of-concept patches.

Also I see that error messages processing needs to be rewritten. Unlike 
the INSERT operation applied to each row, here we find out copy errors 
only after sending the END of copy. Currently implementations 0002 and 
0004 provide uninformative error messages for some cases.

-- 
regards,
Andrey Lepikhov
Postgres Professional

Attachment

pgsql-hackers by date:

Previous
From: Surafel Temesgen
Date:
Subject: Re: Evaluate expression at planning time for two more cases
Next
From: Kyotaro Horiguchi
Date:
Subject: Re: v13: CLUSTER segv with wal_level=minimal and parallel index creation