Re: Bump soft open file limit (RLIMIT_NOFILE) to hard limit on startup - Mailing list pgsql-hackers

From Jelte Fennema-Nio
Subject Re: Bump soft open file limit (RLIMIT_NOFILE) to hard limit on startup
Date
Msg-id DES3UPNP0UXY.35SEHGJ8F8OEQ@jeltef.nl
Whole thread Raw
In response to Re: Bump soft open file limit (RLIMIT_NOFILE) to hard limit on startup  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
On Tue Feb 11, 2025 at 8:42 PM CET, Andres Freund wrote:
> Not sure that's quite the right thing to do for postmaster. What I'd start
> with is to increase the soft limit to
>   "already used files" + max_files_per_process.

I needed to rebase this patch, and that made me finally take the time to
do the restoration of the file limit for subprocesses properly: In
previous versions of this patch it restored the limit before the call to
system() and it didn't restore it at all for popen. This latest version
the patch adds custom pg_system() and pg_popen() functions that restore
the limits in the child process right after the fork, but before the
exec.

There are two reasons to do this:
1. Any executables that still use select(2) will get clear "out of file
   descriptors" errors instead of failing in mysterious ways.
2. Future looking when we'll have multi-threading (which this change is
   needed for) it would be problematic to restore the original limit
   temporarily in the postgres process tree. Some other thread might
   want to open a file while the limit is too low. By only calling
   setrlimit with the lower value in the child process there's not a
   single moment where the original Postgres process has a too low limit.

Attachment

pgsql-hackers by date:

Previous
From: jian he
Date:
Subject: Re: UPDATE run check constraints for affected columns only
Next
From: Mihail Nikalayeu
Date:
Subject: Re: Adding REPACK [concurrently]