Re: Having some problems with concurrent COPY commands - Mailing list pgsql-performance

From Shaun Thomas
Subject Re: Having some problems with concurrent COPY commands
Date
Msg-id CAB78C+DTuu11NXEho=sxmyztgx77gHVog0NK5Cb7TFt_Sbav2w@mail.gmail.com
Whole thread Raw
In response to Re: Having some problems with concurrent COPY commands  (Andres Freund <andres@anarazel.de>)
Responses Re: Having some problems with concurrent COPY commands
List pgsql-performance
On Tue, Oct 13, 2015 at 7:23 AM, Andres Freund <andres@anarazel.de> wrote:
> and send the results.

Whelp, I'm an idiot. I can't account for how I did it, but I can only
assume I didn't export my ports in the tests properly. I ran
everything again and there's a marked difference between 9.3 and 9.4.
The parallel copy times still inflate, but only from 1.4s to 2.5s at 4
procs. Though it gets a bit dicey after that.

I tried to see what the growth curve looks like, but the numbers are
wildly inconsistent after 4 procs. Even at 6, it went anywhere from
4.3 to 7s for each COPY, even while no checkpoint is running. COPY
time definitely increases with each additional process though, which
is likely expected. I was hoping the lock improvements in 9.5 would
improve this area too, but performance is the same on 9.5 (yes I'm
sure this time).

I can still send the perfs, but I suspect they're not exceptionally
useful anymore. :)

As a side note, using INSERT instead scales almost exactly linearly.
This would be useful, except that INSERT is already at least a
magnitude slower than COPY. Hah.

--
Shaun Thomas
bonesmoses@gmail.com
http://bonesmoses.org/


pgsql-performance by date:

Previous
From: Andres Freund
Date:
Subject: Re: Having some problems with concurrent COPY commands
Next
From: David Rowley
Date:
Subject: Re: Having some problems with concurrent COPY commands