Re: Parallel copy - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: Parallel copy
Date
Msg-id CAA4eK1L=dcpGWnXOEvY15XXrGBazUi0uUsuJ69k9G0JBfjhc6Q@mail.gmail.com
Whole thread Raw
In response to Re: Parallel copy  (Amit Kapila <amit.kapila16@gmail.com>)
Responses Re: Parallel copy
List pgsql-hackers
On Thu, Oct 29, 2020 at 11:45 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Tue, Oct 27, 2020 at 7:06 PM vignesh C <vignesh21@gmail.com> wrote:
> >
> [latest version]
>
> I think the parallel-safety checks in this patch
> (v9-0002-Allow-copy-from-command-to-process-data-from-file) are
> incomplete and wrong.
>

One more point, I have noticed that some time back [1], I have given
one suggestion related to the way workers process the set of lines
(aka chunk). I think you can try by increasing the chunk size to say
100, 500, 1000 and use some shared counter to remember the number of
chunks processed.

[1] - https://www.postgresql.org/message-id/CAA4eK1L-Xgw1zZEbGePmhBBWmEmLFL6rCaiOMDPnq2GNMVz-sg%40mail.gmail.com

-- 
With Regards,
Amit Kapila.



pgsql-hackers by date:

Previous
From: Daniil Zakhlystov
Date:
Subject: Re: libpq compression
Next
From: Amit Kapila
Date:
Subject: Re: Track statistics for streaming of in-progress transactions