Re: parallel pg_restore - Mailing list pgsql-hackers

From Joshua Drake
Subject Re: parallel pg_restore
Date
Msg-id 20080923165137.32902742@jd-laptop
Whole thread Raw
In response to Re: parallel pg_restore  (Andrew Dunstan <andrew@dunslane.net>)
List pgsql-hackers
On Tue, 23 Sep 2008 16:50:43 -0400
Andrew Dunstan <andrew@dunslane.net> wrote:

> 
> 
> Simon Riggs wrote:
> > On Tue, 2008-09-23 at 12:43 -0700, Joshua Drake wrote:
> >   
> >> On Tue, 23 Sep 2008 08:44:19 +0100
> >> Simon Riggs <simon@2ndQuadrant.com> wrote:
> >>
> >>     
> >>> On Mon, 2008-09-22 at 15:05 -0400, Andrew Dunstan wrote:
> >>>
> >>>       
> >>>> j and m happen to be two of those that are available.
> >>>>
> >>>> I honestly don't have a terribly strong opinion about what it
> >>>> should be called. I can live with jobs or multi-threads.
> >>>>         
> >>> Perhaps we can use -j for jobs and -m for memory, so we can set
> >>> memory available across all threads with a single total value.
> >>>
> >>> I can live with jobs or multi-threads also, whichever we decide.
> >>> Neither one is confusing to explain.
> >>>
> >>>       
> >> Memory? Where did that come from. Andrew is that in your spec?
> >>     
> >
> > No, but it's in mine. As I said upthread, no point in making it more
> > parallel than memory allows. Different operations need more/less
> > memory than others, so we must think about that also. We can
> > quickly work out how big a table is, so we can work out how much
> > memory it will need to perform sorts for index builds and thus how
> > many parallel builds can sensibly take place.
> >
> >   
> 
> If that ever happens it will certainly not be in this go round.
> 
> In fact, we have some anecdotal evidence that the point of dimishing 
> returns is not reached until a fairly high degree of parallelism is
> used (Joshua's and my client has been using 24 threads, I believe).

Against 8 cores but yes.

Joshua D. Drake

-- 
The PostgreSQL Company since 1997: http://www.commandprompt.com/ 
PostgreSQL Community Conference: http://www.postgresqlconference.org/
United States PostgreSQL Association: http://www.postgresql.us/




pgsql-hackers by date:

Previous
From: Bruce Momjian
Date:
Subject: Re: Proposal of SE-PostgreSQL patches (for CommitFest:Sep)
Next
From: KaiGai Kohei
Date:
Subject: Re: Proposal of SE-PostgreSQL patches (for CommitFest:Sep)