Re: problem with large maintenance_work_mem settings and - Mailing list pgsql-hackers

From Simon Riggs
Subject Re: problem with large maintenance_work_mem settings and
Date
Msg-id 1142002079.16417.9.camel@localhost.localdomain
Whole thread Raw
In response to Re: problem with large maintenance_work_mem settings and  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
On Fri, 2006-03-10 at 09:31 -0500, Tom Lane wrote:
> Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes:
> > LOG:  begin index sort: unique = f, workMem = 8024000, randomAccess = f
> > LOG:  switching to external sort with 28658 tapes: CPU 4.18s/13.96u sec 
> > elapsed 32.43 sec
> > LOG:  finished writing run 1 to tape 0: CPU 173.56s/3425.85u sec elapsed 
> > 3814.82 sec
> > LOG:  performsort starting: CPU 344.17s/7013.20u sec elapsed 7715.45 sec
> > LOG:  finished writing final run 2 to tape 1: CPU 347.19s/7121.78u sec 
> > elapsed 7827.25 sec
> > LOG:  performsort done (except 2-way final merge): CPU 348.25s/7132.99u 
> > sec elapsed 7846.47 sec
> 
> > after that the postmaster is now consuming 99% CPU for about 22 hours(!) 
> 
> I'll look into it, but I was already wondering if we shouldn't bound the
> number of tapes somehow.  It's a bit hard to believe that 28000 tapes is
> a sane setting.

Well, since they are not actually tapes, why not?

I thought you had changed the memory settings so that the 28658 was a
maximum, not the actual number it used. In the above example we are only
using 2 runs (tapes)... so it should just read in run 1 and then run 2
into memory and complete the final merge.

Seems reasonable to limit tapes to say 10,000. 28000 tapes allows you to
sort 448 TB without any merging...

Best Regards, Simon Riggs




pgsql-hackers by date:

Previous
From: Stefan Kaltenbrunner
Date:
Subject: Re: problem with large maintenance_work_mem settings and
Next
From: Tom Lane
Date:
Subject: Re: problem with large maintenance_work_mem settings and