Thread: postgresql 7.1 file descriptor
Hi, I am running postgresql - 7.1 on Redhat 2.2.16-22 kernel. Sometimes this the system becomes totally unusable with "VFS: file-max limit 4096 reached" error? The system is setup to have 4096 file descriptor as default. I was wondering how many file descriptor does postgres open per backend process? Thank you, --thanh
jade <jade@vanzoest.com> writes:
> I am running postgresql - 7.1 on Redhat 2.2.16-22 kernel.
> Sometimes this the system becomes totally unusable with
> "VFS: file-max limit 4096 reached" error? The system is setup to have
> 4096 file descriptor as default. I was wondering how many file descriptor
> does postgres open per backend process?
Potentially lots. Increase your system's NFILE limit, or reduce the max
number of backends you will allow PG to start, or update to 7.2 which
will let you set a per-backend file limit (see MAX_FILES_PER_PROCESS;
AFAIR that parameter did not exist in 7.1).
regards, tom lane
jade wrote: > > Hi, > > I am running postgresql - 7.1 on Redhat 2.2.16-22 kernel. > Sometimes this the system becomes totally unusable with > "VFS: file-max limit 4096 reached" error? The system is setup to have > 4096 file descriptor as default. I was wondering how many file descriptor > does postgres open per backend process? One for each file in your database, one for the communication's socket, etc. Your best bet is to add more files.
On Tue, 7 May 2002, jade wrote: > > Hi, > > > I am running postgresql - 7.1 on Redhat 2.2.16-22 kernel. > Sometimes this the system becomes totally unusable with > "VFS: file-max limit 4096 reached" error? The system is setup to have > 4096 file descriptor as default. I was wondering how many file descriptor > does postgres open per backend process? I think it depends on what those backends are doing. I just ran a pgbench -c 100 -t 10 on my machine, and went from "2253 712 8192" in /proc/sys/fs/file-nr to "5316 3825 8192" in the same file. According to /usr/src/linux-2.4/Documentation/sysctl/fs.txt on my machine, the first number is the number of file handles allocated, the second is the number in use, and the third is the max. So, running 100 simos on my personal workstation used about 3000 file handles allocated, with about the same number showing use. To easiest way to change these settings is through the use of sysctl. If your machine doesn't have sysctl installed, find it and install it, it's a VERY easy way to change system / kernel settings. After install, look at man sysctl for more info. East explanation, sysctl -a shows all settings, sysctl -p process and sets all settings found in /etc/sysctl.conf On production servers, the number for file-max is often set to 32768 or higher (on our big boxes we have it set to 65536 and often see usage of well over 20000 under load.)
On Wed, 8 May 2002, Tom Lane wrote: > jade <jade@vanzoest.com> writes: > > I am running postgresql - 7.1 on Redhat 2.2.16-22 kernel. > > Sometimes this the system becomes totally unusable with > > "VFS: file-max limit 4096 reached" error? The system is setup to have > > 4096 file descriptor as default. I was wondering how many file descriptor > > does postgres open per backend process? > > Potentially lots. Increase your system's NFILE limit, or reduce the max > number of backends you will allow PG to start, or update to 7.2 which > will let you set a per-backend file limit (see MAX_FILES_PER_PROCESS; > AFAIR that parameter did not exist in 7.1). Hi All, Thanks for all the response -- it was all very helpful! I don't want to limit the my backend processes because each response is consider "important" so running any backend limitation is consider unaccceptable. So, I've increased the system NFIle limit.. so far so good :-) Are they any way to pool connections through a single backend process? thanks, Thanh