Profiling vs autovacuum (was Re: building 8.3beta2 w/ 'make check' consumes A LOT of disk space) - Mailing list pgsql-hackers

From Tom Lane
Subject Profiling vs autovacuum (was Re: building 8.3beta2 w/ 'make check' consumes A LOT of disk space)
Date
Msg-id 4285.1194145581@sss.pgh.pa.us
Whole thread Raw
In response to Re: building 8.3beta2 w/ 'make check' consumes A LOT of disk space  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Profiling vs autovacuum
List pgsql-hackers
I wrote:
> However, accumulation of zillions of gmon.out files is definitely a
> downside of the approach; one that I've noticed myself.  I've also
> noticed that it takes a heck of a long time to rm -rf $PGDATA once
> you've built up a few tens of thousands of gprof subdirectories.  What's
> worse, this accumulation will occur pretty quick even if you're not
> doing anything with the DB, because of autovacuum process launches.

> I wonder if we need to rein in gmon.out accumulation somehow, and if
> so how?  This isn't an issue for ordinary users but I can see it
> becoming a PITA for developers.

On reflection, it seems like the worst part of this is the steady
accumulation of gprof files from autovacuum workers, which are unlikely
to be of interest at all (if you want to profile vacuuming, you'd
probably issue manual vacuum commands anyway).  So I propose something
like the attached, untested patch to force all AV workers to dump to the
same gmon.out file.  Comments?

            regards, tom lane

*** src/backend/storage/ipc/ipc.c.orig    Wed Jul 25 15:58:56 2007
--- src/backend/storage/ipc/ipc.c    Sat Nov  3 23:00:41 2007
***************
*** 126,131 ****
--- 126,136 ----
           *    $PGDATA/gprof/8845/gmon.out
           *        ...
           *
+          * To avoid undesirable disk space bloat, autovacuum workers are
+          * discriminated against: all their gmon.out files go into the same
+          * subdirectory.  Without this, an installation that is "just sitting
+          * there" nonetheless eats megabytes of disk space every few seconds.
+          *
           * Note that we do this here instead of in an on_proc_exit()
           * callback because we want to ensure that this code executes
           * last - we don't want to interfere with any other on_proc_exit()
***************
*** 133,139 ****
           */
          char gprofDirName[32];

!         snprintf(gprofDirName, 32, "gprof/%d", (int) getpid());

          mkdir("gprof", 0777);
          mkdir(gprofDirName, 0777);
--- 138,147 ----
           */
          char gprofDirName[32];

!         if (IsAutoVacuumWorkerProcess())
!             snprintf(gprofDirName, 32, "gprof/avworker");
!         else
!             snprintf(gprofDirName, 32, "gprof/%d", (int) getpid());

          mkdir("gprof", 0777);
          mkdir(gprofDirName, 0777);

pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: should I worry?
Next
From: Gregory Stark
Date:
Subject: Re: Profiling vs autovacuum