Thread: Errors: Too many open files
During busy hours, Postgres and Apache are reporting errors that there are too many open files, and sometimes Postgres will crash. I've searched all over for a way to increase this resource limit. I have tried: ulimit -n 8192 (seems only to work for the current shell session) echo "8192" > /proc/sys/fs/file-max (did not seem to make a difference) adding this entry to /etc/security/limits.conf: postgres hard nofile 4096 (did not seem to make a difference) I have read the Postgres Administration docs "3.5.2 Resource Limits" but I am still unable to find the solution. Can anybody tell me how to increase the resource allocation, and exactly where to make this change? Any help would be greatly appreciated, _Ken Creason webmaster@fcs.net
Sorry, I should have mentioned: RedHat Linux 7.1 _Ken At 05:28 PM 12/6/01, you wrote: >During busy hours, Postgres and Apache are reporting >errors that there are too many open files, and sometimes >Postgres will crash. I've searched all over for a way >to increase this resource limit. >I have tried: > >ulimit -n 8192 >(seems only to work for the current shell session) > >echo "8192" > /proc/sys/fs/file-max >(did not seem to make a difference) > >adding this entry to /etc/security/limits.conf: >postgres hard nofile 4096 >(did not seem to make a difference) > >I have read the Postgres Administration docs >"3.5.2 Resource Limits" but I am still unable to find >the solution. > >Can anybody tell me how to increase the resource allocation, >and exactly where to make this change? > >Any help would be greatly appreciated, >_Ken Creason >webmaster@fcs.net > > > >---------------------------(end of broadcast)--------------------------- >TIP 6: Have you searched our list archives? > >http://archives.postgresql.org
I tried adding the ulimit -n 8192 to the startup script (/etc/rc.d/init.d/postgres) but got an error on the restart ... something about unary operator expected, I think. _Ken At 07:01 PM 12/6/01, you wrote: >On Thu, Dec 06, 2001 at 05:28:30PM -0800, Ken wrote: >> ulimit -n 8192 >> (seems only to work for the current shell session) > >How did you start your system? Why not add that to the startup >script? > >> Can anybody tell me how to increase the resource allocation, >> and exactly where to make this change? > >Not without knowing your OS and such. But there's a section about it >in the docs. > >-- >---- >Andrew Sullivan 87 Mowat Avenue >Liberty RMS Toronto, Ontario Canada ><andrew@libertyrms.info> M6K 3E3 > +1 416 646 3304 x110
Ken <mailinglists@creason.com> writes:
> During busy hours, Postgres and Apache are reporting
> errors that there are too many open files, and sometimes
> Postgres will crash.
You need to raise the kernel's file table size (most Unixen call
this kernel parameter NFILE; also NINODE might be interesting).
It's a system-wide limit, not a per-process limit.
> echo "8192" > /proc/sys/fs/file-max
> (did not seem to make a difference)
This *looks* plausible, but I'm not familiar enough with Linux to know
if that's the right incantation or not. Also, did you check to see
what the original value is? I'm not sure that 8k is more than the
default ...
regards, tom lane
I believe the original value of /proc/sys/fs/file-max was 1024, but I may very well be mistaken. I have increased that value now to 65536 in hopes that it makes the difference. Thank you Andrew, Frank and Tom for your kind advice. _Ken At 07:29 AM 12/7/01, you wrote: >Ken <mailinglists@creason.com> writes: >> During busy hours, Postgres and Apache are reporting >> errors that there are too many open files, and sometimes >> Postgres will crash. > >You need to raise the kernel's file table size (most Unixen call >this kernel parameter NFILE; also NINODE might be interesting). >It's a system-wide limit, not a per-process limit. > >> echo "8192" > /proc/sys/fs/file-max >> (did not seem to make a difference) > >This *looks* plausible, but I'm not familiar enough with Linux to know >if that's the right incantation or not. Also, did you check to see >what the original value is? I'm not sure that 8k is more than the >default ... > > regards, tom lane > >---------------------------(end of broadcast)--------------------------- >TIP 4: Don't 'kill -9' the postmaster
Tom Lane wrote:
> Ken <mailinglists@creason.com> writes:
> > During busy hours, Postgres and Apache are reporting
> > errors that there are too many open files, and sometimes
> > Postgres will crash.
>
> You need to raise the kernel's file table size (most Unixen call
> this kernel parameter NFILE; also NINODE might be interesting).
> It's a system-wide limit, not a per-process limit.
>
> > echo "8192" > /proc/sys/fs/file-max
> > (did not seem to make a difference)
>
> This *looks* plausible, but I'm not familiar enough with Linux to know
> if that's the right incantation or not. Also, did you check to see
> what the original value is? I'm not sure that 8k is more than the
> default ...
It is the right place. IIRC the default is either 4 or 8K. I
usually set it to 32K or even more (depends on the number of
tables and number of backends I expect).
Jan
--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
> > > echo "8192" > /proc/sys/fs/file-max > > (did not seem to make a difference) > You'll want to add this to /etc/sysctl.conf as well so that it adds this at bootup. This works on newer Linux (RedHat) distros. see /sbin/sysctl as well... _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com
> > > echo "8192" > /proc/sys/fs/file-max > > > (did not seem to make a difference) You'll probably want to do something like this: echo '16384' > /proc/sys/fs/file-max echo '65536' > /proc/sys/fs/inode-max The documentation in the kernel sources mentions that inode-max should be something like two or three times the file-max. Also, depending on your local configuration, you may need to use ulimit to raise the file limits for that individual user. It's so much easier than years ago, when you had to recompile the kernel to raise those limits. ; ) steve
Hi Ken,
we´ve had the same problem with a samba fileserver, which regularly
choked with a "too many open filehandles" message. We found out that
the value 8192 is nowadays the standard for /proc/sys/fs/file-max and
you can set it to much higher values. For example, the SuSE people say,
that for their Email-Server it should be set to 65536. There might also
be a value /proc/sys/fs/inode-max (may be only for certain file
systems), which according to SuSE might also be risen to 131072.
Maybe it helps.
mfg Frank Finner
On 07-Dec-01 Ken sat down, thought for a long time and then wrote:
> During busy hours, Postgres and Apache are reporting
> errors that there are too many open files, and sometimes
> Postgres will crash. I've searched all over for a way
> to increase this resource limit.
> I have tried:
>
> ulimit -n 8192
> (seems only to work for the current shell session)
>
> echo "8192" > /proc/sys/fs/file-max
> (did not seem to make a difference)
>
> adding this entry to /etc/security/limits.conf:
> postgres hard nofile 4096
> (did not seem to make a difference)
--
Frank Finner
And now there is no turning back at all.
(M. Moorcock, "Elric Of Melnibone")"