Thread: postgresql 7.1 file descriptor

postgresql 7.1 file descriptor

From
jade
Date:
Hi,


I am running postgresql - 7.1 on Redhat 2.2.16-22 kernel.
Sometimes this the system becomes totally unusable with
"VFS: file-max limit 4096 reached" error?  The system is setup to have
4096 file descriptor as default.  I was wondering how many file descriptor
does postgres open per backend process?

Thank you,
--thanh




Re: [HACKERS] postgresql 7.1 file descriptor

From
Tom Lane
Date:
jade <jade@vanzoest.com> writes:
> I am running postgresql - 7.1 on Redhat 2.2.16-22 kernel.
> Sometimes this the system becomes totally unusable with
> "VFS: file-max limit 4096 reached" error?  The system is setup to have
> 4096 file descriptor as default.  I was wondering how many file descriptor
> does postgres open per backend process?

Potentially lots.  Increase your system's NFILE limit, or reduce the max
number of backends you will allow PG to start, or update to 7.2 which
will let you set a per-backend file limit (see MAX_FILES_PER_PROCESS;
AFAIR that parameter did not exist in 7.1).
        regards, tom lane


Re: postgresql 7.1 file descriptor

From
mlw
Date:
jade wrote:
> 
> Hi,
> 
> I am running postgresql - 7.1 on Redhat 2.2.16-22 kernel.
> Sometimes this the system becomes totally unusable with
> "VFS: file-max limit 4096 reached" error?  The system is setup to have
> 4096 file descriptor as default.  I was wondering how many file descriptor
> does postgres open per backend process?

One for each file in your database, one for the communication's socket, etc.
Your best bet is to add more files.


Re: [HACKERS] postgresql 7.1 file descriptor

From
Scott Marlowe
Date:
On Tue, 7 May 2002, jade wrote:

> 
> Hi,
> 
> 
> I am running postgresql - 7.1 on Redhat 2.2.16-22 kernel.
> Sometimes this the system becomes totally unusable with
> "VFS: file-max limit 4096 reached" error?  The system is setup to have
> 4096 file descriptor as default.  I was wondering how many file descriptor
> does postgres open per backend process?

I think it depends on what those backends are doing.  I just ran a pgbench 
-c 100 -t 10 on my machine, and went from "2253    712     8192" in 
/proc/sys/fs/file-nr to "5316    3825    8192" in the same file.

According to /usr/src/linux-2.4/Documentation/sysctl/fs.txt on my machine, 
the first number is the number of file handles allocated, the second is 
the number in use, and the third is the max.

So, running 100 simos on my personal workstation used about 3000 file 
handles allocated, with about the same number showing use.

To easiest way to change these settings is through the use of sysctl.  If 
your machine doesn't have sysctl installed, find it and install it, it's a 
VERY easy way to change system / kernel settings.

After install, look at man sysctl for more info.

East explanation, sysctl -a shows all settings, sysctl -p process and sets 
all settings found in /etc/sysctl.conf

On production servers, the number for file-max is often set to 32768 or 
higher (on our big boxes we have it set to 65536 and often see usage of 
well over 20000 under load.)



Re: [HACKERS] postgresql 7.1 file descriptor

From
jade
Date:
On Wed, 8 May 2002, Tom Lane wrote:

> jade <jade@vanzoest.com> writes:
> > I am running postgresql - 7.1 on Redhat 2.2.16-22 kernel.
> > Sometimes this the system becomes totally unusable with
> > "VFS: file-max limit 4096 reached" error?  The system is setup to have
> > 4096 file descriptor as default.  I was wondering how many file descriptor
> > does postgres open per backend process?
>
> Potentially lots.  Increase your system's NFILE limit, or reduce the max
> number of backends you will allow PG to start, or update to 7.2 which
> will let you set a per-backend file limit (see MAX_FILES_PER_PROCESS;
> AFAIR that parameter did not exist in 7.1).

Hi All,

Thanks for all the response -- it was all very helpful! I don't want to limit
the my backend processes because each response is consider "important" so
running any backend limitation is consider unaccceptable.  So,  I've increased
the system NFIle limit.. so far so good :-)

Are they any way to pool connections through a single backend process?

thanks,
Thanh