pgsql: Fix running out of file descriptors for spill files. - Mailing list pgsql-committers

From Amit Kapila
Subject pgsql: Fix running out of file descriptors for spill files.
Date
Msg-id E1imurV-0000KX-CO@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Fix running out of file descriptors for spill files.

Currently while decoding changes, if the number of changes exceeds a
certain threshold, we spill those to disk.  And this happens for each
(sub)transaction.  Now, while reading all these files, we don't close them
until we read all the files.  While reading these files, if the number of
such files exceeds the maximum number of file descriptors, the operation
errors out.

Use PathNameOpenFile interface to open these files as that internally has
the mechanism to release kernel FDs as needed to get us under the
max_safe_fds limit.

Reported-by: Amit Khandekar
Author: Amit Khandekar
Reviewed-by: Amit Kapila
Backpatch-through: 9.4
Discussion: https://postgr.es/m/CAJ3gD9c-sECEn79zXw4yBnBdOttacoE-6gAyP0oy60nfs_sabQ@mail.gmail.com

Branch
------
REL_11_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/3e3a7973523570089fa69533dcb3aa0122636d8c

Modified Files
--------------
src/backend/replication/logical/reorderbuffer.c | 47 +++++++++++++++----------
src/test/recovery/t/006_logical_decoding.pl     | 39 +++++++++++++++++++-
2 files changed, 66 insertions(+), 20 deletions(-)


pgsql-committers by date:

Previous
From: Peter Geoghegan
Date:
Subject: pgsql: Revise BTP_HAS_GARBAGE nbtree VACUUM comments.
Next
From: Peter Eisentraut
Date:
Subject: pgsql: Fix comment in test