Re: Too-many-files errors on OS X - Mailing list pgsql-hackers

From Kevin Brown
Subject Re: Too-many-files errors on OS X
Date
Msg-id 20040222115736.GJ2608@filer
Whole thread Raw
In response to Too-many-files errors on OS X  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Too-many-files errors on OS X  (Tom Lane <tgl@sss.pgh.pa.us>)
Re: Too-many-files errors on OS X  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
Tom Lane wrote:
> However, it seems that the real problem here is that we are so far off
> base about how many files we can open.  I wonder whether we should stop
> relying on sysconf() and instead try to make some direct probe of the
> number of files we can open.  I'm imagining repeatedly open() until
> failure at some point during postmaster startup, and then save that
> result as the number-of-openable-files limit.

I strongly favor this method.  In particular, the probe should probably
be done after all shared libraries have been loaded and initialized.

I originally thought that each shared library that was loaded would eat
a file descriptor (since I thought it would be implemented via mmap())
but that doesn't seem to be the case, at least under Linux (for those
who are curious, you can close the underlying file after you perform
the mmap() and the mapped region still works).  If it's true under any
OS then it would certainly be prudent to measure the available file
descriptors after the shared libs have been loaded (another reason is
that the init function of a library might itself open a file and keep
it open, but this isn't likely to happen very often).

> I also notice that OS X 10.3 seems to have working SysV semaphore
> support.  I am tempted to change template/darwin to use SysV where
> available, instead of Posix semaphores.  I wonder whether inheriting
> 100-or-so open file descriptors every time we launch a backend isn't
> in itself a nasty performance hit, quite aside from its effect on how
> many normal files we can open.

I imagine this could easily be tested.  I rather doubt that the
performance hit would be terribly large, but we certainly shouldn't rule
it out without testing it first.


-- 
Kevin Brown                          kevin@sysexperts.com


pgsql-hackers by date:

Previous
From: Scott Goodwin
Date:
Subject: Re: Mac OS X, PostgreSQL, PL/Tcl
Next
From: Dave Cramer
Date:
Subject: Re: Pl/Java - next step?