> Had this happen at a previous employer, and it definitely is bad. I
> believe we had to do a reboot to clear it up. And we saw the problem a
> couple of times since the sys admin wasn't able to deduce what had
> happened the first time we got it. IIRC the problem hit somewhere around
> 150 connections, so we ran with 128 max. I think this is a safe number
> on most servers these days (running linux as least) though out of the
> box I might be more inclined to limit it to 64. If you do hit a file
> descriptor problem, *you are hosed*.
Just yesterday I managed to hose my new Postgres installation during a
particular benchmarking run. Postgres did restart itself nicely though. I
have no idea why that particular run caused problems when all other runs
with identical settings didn't. I checked the log and saw file descriptor
probs. I was doing 128 connections with 128 max connetions. This was the
log:
> 2003-02-12 04:16:15 LOG: PGSTAT: cannot open temp stats file
> /usr/local/pgsql/data/global/pgstat.tmp.41388: Too many open files in
> system
> 2003-02-12 04:16:15 LOG: PGSTAT: cannot open temp stats file
> /usr/local/pgsql/data/global/pgstat.tmp.41388: Too many open files in
> system
> 2003-02-12 04:16:39 PANIC: could not open transaction-commit log
> directory
> (/usr/local/pgsql/data/pg_clog): Too many open files in system
> 2003-02-12 04:16:39 LOG: statement: SET autocommit TO 'on';VACUUM
> ANALYZE
> 2003-02-12 04:16:39 LOG: PGSTAT: cannot open temp stats file
> /usr/local/pgsql/data/global/pgstat.tmp.41388: Too many open files in
> system
This was the MIB:
> kern.maxfiles: 1064
> kern.maxfilesperproc: 957
This was the solution:
> sysctl -w kern.maxfiles=65536
> sysctl -w kern.maxfilesperproc=8192
>
> .. and then stick
>
> kern.maxfiles=65536
> kern.maxfilesperproc=8192
>
> in /etc/sysctl.conf so its set during a reboot.
Which just goes to highlight the importance of rigorously testing a
production installation...
Chris
