Re: An idea for parallelizing COPY within one backend - Mailing list pgsql-hackers

From Andrew Dunstan
Subject Re: An idea for parallelizing COPY within one backend
Date
Msg-id 47C593D0.6080600@dunslane.net
Whole thread Raw
In response to Re: An idea for parallelizing COPY within one backend  ("Florian G. Pflug" <fgp@phlo.org>)
Responses Re: An idea for parallelizing COPY within one backend  (Brian Hurt <bhurt@janestcapital.com>)
Re: An idea for parallelizing COPY within one backend  ("Florian G. Pflug" <fgp@phlo.org>)
List pgsql-hackers

Florian G. Pflug wrote:
>
>> Would it be possible to determine when the copy is starting that this 
>> case holds, and not use the parallel parsing idea in those cases?
>
> In theory, yes. In pratice, I don't want to be the one who has to 
> answer to an angry user who just suffered a major drop in COPY 
> performance after adding an ENUM column to his table.
>
>

I am yet to be convinced that this is even theoretically a good path to 
follow. Any sufficiently large table could probably be partitioned and 
then we could use the parallelism that is being discussed for pg_restore 
without any modification to the backend at all. Similar tricks could be 
played by an external bulk loader for third party data sources.

cheers

andrew


pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: An idea for parallelizing COPY within one backend
Next
From: Brian Hurt
Date:
Subject: Re: An idea for parallelizing COPY within one backend