Re: optimizing CleanupTempFiles - Mailing list pgsql-hackers

From Simon Riggs
Subject Re: optimizing CleanupTempFiles
Date
Msg-id 1221684417.3913.2164.camel@ebony.2ndQuadrant
Whole thread Raw
In response to optimizing CleanupTempFiles  (Alvaro Herrera <alvherre@commandprompt.com>)
Responses Re: optimizing CleanupTempFiles  (Alvaro Herrera <alvherre@commandprompt.com>)
List pgsql-hackers
On Wed, 2008-09-17 at 16:25 -0400, Alvaro Herrera wrote:

> We've been profiling a large system (8 CPUs, 64 GB of memory, some
> dozens of disks) which seems rather more swamped than it should.  Part
> of the problem seems to come from CleanupTempFiles, the second entry in
> oprofile output.

I'm glad you've observed this also. I saw it about two years ago but
wasn't able to convince anyone else it existed at the time.

> I can see two simple ways to solve this problem.  One is to create a
> second array that keeps pointers to the temp files in VfdCache.  Then,
> on CleanupTempFiles we scan that array instead of VfdCache directly.
> 
> The second one is to use a separate VfdCache for temp files, but this
> seems much more involved, requiring touch almost all of fd.c.
> 
> Of course, perhaps there's another solution which involves rethinking
> fd.c in a more thorough fashion, but I'm not sure how right offhand.

Simple solution is to have a state variable so you can see whether a
backend has created an temp files in this transaction. Most don't, so I
think the above two solutions are overkill. If we created any, scan for
them, if not, don't. Just a simple boolean state, just as we have for
AtEOXact_RelationCache().

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support



pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: optimizing CleanupTempFiles
Next
From: Alvaro Herrera
Date:
Subject: Re: optimizing CleanupTempFiles