Re: 2pc leaks fds - Mailing list pgsql-hackers

From Antonin Houska
Subject Re: 2pc leaks fds
Date
Msg-id 26247.1586157152@antos
Whole thread Raw
In response to Re: 2pc leaks fds  (Andres Freund <andres@anarazel.de>)
Responses Re: 2pc leaks fds  (Andres Freund <andres@anarazel.de>)
Re: 2pc leaks fds  (Andres Freund <andres@anarazel.de>)
Re: 2pc leaks fds  (Antonin Houska <ah@cybertec.at>)
List pgsql-hackers
Andres Freund <andres@anarazel.de> wrote:

> > From what I can see, the error is that the code only bothers closing
> > WALOpenSegment->seg when switching to a new segment, but we need also
> > to close it when finishing the business in XLogReaderFree().
> 
> Yea, I came to the same conclusion and locally fixed it the same way
> (except having the close a bit earlier in XLogReaderFree()).

It's still not quite clear to me why the problem starts to appear after
0dc8ead46.  This patch does not remove any close() call from XLogReaderFree().

> But I'm not sure it's quite the right idea. I'm not sure I fully
> understand the design of 0dc8ead46, but it looks to me like it's
> intended to allow users of the interface to have different ways of
> opening files. If we just close() the fd that'd be a bit more limited.

It should have allowed users to have different ways to *locate the segment*
file. The WALSegmentOpen callback could actually return file path instead of
the file descriptor and let WALRead() perform the opening/closing, but then
the WALRead function would need to be aware whether it is executing in backend
or in frontend (so it can use the correct function to open/close the file).

I was aware of the problem that the correct function should be used to open
the file and that's why this comment was added (although "mandatory" would be
more suitable than "preferred"):

 * BasicOpenFile() is the preferred way to open the segment file in backend
 * code, whereas open(2) should be used in frontend.
 */
typedef int (*WALSegmentOpen) (XLogSegNo nextSegNo, WALSegmentContext *segcxt,
                               TimeLineID *tli_p);

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com



pgsql-hackers by date:

Previous
From: Noah Misch
Date:
Subject: Re: [HACKERS] WAL logging problem in 9.4.3?
Next
From: Amit Kapila
Date:
Subject: Re: pg_stat_statements issue with parallel maintenance (Was Re: WALusage calculation patch)