Re: concurrent COPY performance - Mailing list pgsql-hackers

From Merlin Moncure
Subject Re: concurrent COPY performance
Date
Msg-id b42b73150906161433n5ce5d4f3r5bc9253bf8c545e2@mail.gmail.com
Whole thread Raw
In response to concurrent COPY performance  (Stefan Kaltenbrunner <stefan@kaltenbrunner.cc>)
Responses Re: concurrent COPY performance
Re: concurrent COPY performance
List pgsql-hackers
On Tue, Jun 16, 2009 at 12:47 PM, Stefan
Kaltenbrunner<stefan@kaltenbrunner.cc> wrote:
> Hi!
>
> I have been doing some bulk loading testing recently - mostly with a focus
> on answering why we are "only" getting a (max of) cores/2(up to around 8
> cores even less with more) speedup using parallel restore.
> What I found is that on some fast IO-subsystem we are CPU bottlenecked on
> concurrent copy which is able to utilize WAL bypass (and scale up to around
> cores/2) and performance without wal bypass is very bad.
> In the WAL logged case we are only able to get a 50% speedup using the
> second process already and we are never able to scale better than 3x (up to
> 8 cores) and performance degrades even after that point.

how are you bypassing wal?  do I read this properly that on your 8
core system you are getting 4x speedup with wal bypass and 3x speedup
without?

merlin


pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: machine-readable explain output
Next
From: Andrew Dunstan
Date:
Subject: Re: concurrent COPY performance