Thread: Question: Why Are File Descriptors Not Closed and Accounted for PostgreSQL Backends?
Question: Why Are File Descriptors Not Closed and Accounted for PostgreSQL Backends?
From
Srinath Reddy Sadipiralla
Date:
Hi PostgreSQL Community,
when a backend process starts, pq_init is called where it opens a FD during CreateWaitEventSet()
if (!AcquireExternalFD())
{
/* treat this as though epoll_create1 itself returned EMFILE */
elog(ERROR, "epoll_create1 failed: %m");
}
set->epoll_fd = epoll_create1(EPOLL_CLOEXEC);
but we didn't closed or called ReleaseExternalFD() for accounting,lets say if we have multiple clients connected and
areactively running queries, won't the max number of open FDs (ulimit -n) limit of the system gets reached and cause
"Toomany open files issue"?
Regards
Srinath Reddy
Re: Question: Why Are File Descriptors Not Closed and Accounted for PostgreSQL Backends?
From
Heikki Linnakangas
Date:
On 24/05/2024 15:17, Srinath Reddy Sadipiralla wrote:
> Hi PostgreSQL Community,
> when a backend process starts, pq_init is called where it opens a FD during CreateWaitEventSet()
>
>
> if (!AcquireExternalFD())
> {
> /* treat this as though epoll_create1 itself returned EMFILE */
> elog(ERROR, "epoll_create1 failed: %m");
> }
> set->epoll_fd = epoll_create1(EPOLL_CLOEXEC);
>
>
> but we didn't closed or called ReleaseExternalFD() for accounting
Yes we do, see FreeWaitEventSet().
The WaitEventSet created fro pq_init() is never explicitly free'd
though, because it's created in the per-connection backend process. When
the connection is terminated, the backend process exits, cleaning up any
resources including the WaitEventSet.
--
Heikki Linnakangas
Neon (https://neon.tech)
Re: Question: Why Are File Descriptors Not Closed and Accounted for PostgreSQL Backends?
From
Srinath Reddy Sadipiralla
Date:
Thanks for the reply,yeah i know about FreeWaitEventSet() but that is being used in few places but not for handling backends.
i got it that FDs like FeBeWaitSet->epoll_fd will be free'd when connection is terminated but as i mentioned wouldn't it be an issue if the connection is long living lets take idle which can be running queries for long time,what if we have multiple connections like this running queries using multiple system FDs and reach the limit,cause they are using FDs ,so they may not be free'd.
i got it that FDs like FeBeWaitSet->epoll_fd will be free'd when connection is terminated but as i mentioned wouldn't it be an issue if the connection is long living lets take idle which can be running queries for long time,what if we have multiple connections like this running queries using multiple system FDs and reach the limit,cause they are using FDs ,so they may not be free'd.
---- On Fri, 24 May 2024 19:15:54 +0530 Heikki Linnakangas <hlinnaka@iki.fi> wrote ---
On 24/05/2024 15:17, Srinath Reddy Sadipiralla wrote:
> Hi PostgreSQL Community,
> when a backend process starts, pq_init is called where it opens a FD during CreateWaitEventSet()
>
>
> if (!AcquireExternalFD())
> {
> /* treat this as though epoll_create1 itself returned EMFILE */
> elog(ERROR, "epoll_create1 failed: %m");
> }
> set->epoll_fd = epoll_create1(EPOLL_CLOEXEC);
>
>
> but we didn't closed or called ReleaseExternalFD() for accounting
Yes we do, see FreeWaitEventSet().
The WaitEventSet created fro pq_init() is never explicitly free'd
though, because it's created in the per-connection backend process. When
the connection is terminated, the backend process exits, cleaning up any
resources including the WaitEventSet.
--
Heikki Linnakangas
Neon (https://neon.tech)