Thread: PostgreSQL running out of file handles

Hi,

A few days back the load increased on our database server to the point 
where it could not get enough file handles.  This causes the backends to 
crash, get restarted only to crash again, on and on.

We fixed it by bumping kern.maxfiles, but was just wondering if this is 
a scenario that PostgreSQL should handle more gracefully?

Chris

On Fri, May 13, 2005 at 09:45:15AM +0800, Christopher Kings-Lynne wrote:

> A few days back the load increased on our database server to the point 
> where it could not get enough file handles.  This causes the backends to 
> crash, get restarted only to crash again, on and on.
> 
> We fixed it by bumping kern.maxfiles, but was just wondering if this is 
> a scenario that PostgreSQL should handle more gracefully?

We certainly go great lengths to prevent this problem ... what version
is this?

-- 
Alvaro Herrera (<alvherre[a]surnet.cl>)
"The important things in the world are problems with society that we don't
understand at all. The machines will become more complicated but they won't
be more complicated than the societies that run them."    (Freeman Dyson)

>>A few days back the load increased on our database server to the point 
>>where it could not get enough file handles.  This causes the backends to 
>>crash, get restarted only to crash again, on and on.
>>
>>We fixed it by bumping kern.maxfiles, but was just wondering if this is 
>>a scenario that PostgreSQL should handle more gracefully?
> 
> We certainly go great lengths to prevent this problem ... what version
> is this?

7.4.6

Chris

Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:
> A few days back the load increased on our database server to the point 
> where it could not get enough file handles.  This causes the backends to 
> crash, get restarted only to crash again, on and on.

> We fixed it by bumping kern.maxfiles, but was just wondering if this is 
> a scenario that PostgreSQL should handle more gracefully?

I suppose you are running on some BSD variant?  BSD is notorious for
promising more than it can deliver with respect to number of open files
per process.  This is a kernel bug, not a Postgres bug.

You can adjust Postgres' max_files_per_process setting to compensate for
the kernel's lying about its capabilities.

(Postgres is in fact one of the most robust applications I know of
in terms of not going belly-up in response to EMFILE or ENFILE.
However, if there are not any spare descriptors to close, there's
not a lot we can do except fail.)
        regards, tom lane

> I suppose you are running on some BSD variant?  BSD is notorious for
> promising more than it can deliver with respect to number of open files
> per process.  This is a kernel bug, not a Postgres bug.

Good guess.  Freebsd 4.8 or so.

Chris

> I suppose you are running on some BSD variant?  BSD is notorious for
> promising more than it can deliver with respect to number of open files
> per process.  This is a kernel bug, not a Postgres bug.
> 
> You can adjust Postgres' max_files_per_process setting to compensate for
> the kernel's lying about its capabilities.
> 
> (Postgres is in fact one of the most robust applications I know of
> in terms of not going belly-up in response to EMFILE or ENFILE.
> However, if there are not any spare descriptors to close, there's
> not a lot we can do except fail.)
> 
>             regards, tom lane