Re: [HACKERS] on_dsm_detach() callback and parallel tuplesort BufFile resource management - Mailing list pgsql-hackers

From Robert Haas
Subject Re: [HACKERS] on_dsm_detach() callback and parallel tuplesort BufFile resource management
Date
Msg-id CA+TgmobTnN4fibvHx6tqgQ0VorPsyHaMhfacb6qRhi=_JHcx6Q@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] on_dsm_detach() callback and parallel tuplesort BufFile resource management  (Thomas Munro <thomas.munro@enterprisedb.com>)
Responses Re: [HACKERS] on_dsm_detach() callback and parallel tuplesort BufFile resource management  (Peter Geoghegan <pg@bowt.ie>)
List pgsql-hackers
On Thu, Mar 9, 2017 at 7:29 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
> Suppressing ENOENT because it's not clear which backend actually owns
> a file is a bit different from using it to detect that there are no
> more segments...

+1, emphatically.

> Obviously I screwed some things up in the code I
> posted, but I think the general idea that the DSM segment owns the
> files on disk is a good one.

+1 to that, too.  The DSM has exactly the lifetime that we want here;
no backend or resowner involved in the transaction does.

> I figure that it (via the detach hook)
> should be able to delete the files using only data in shmem, without
> reference to any backend-local memory, so that file cleanup is
> entirely disconnected from backend-local resource cleanup (fds and
> memory).

That's one approach.  Another approach is to somehow tie the two
together; for example, I thought about teaching FileClose that where
it currently calls unlink() to get rid of the temporary file, it would
first go check some shared reference count and decrement it, skipping
the unlink if the result was >0.  But that only works if you have a
separate chunk of shared memory per File, which seems like it won't
work for what you need.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: [HACKERS] [COMMITTERS] pgsql: Fix hard-coded relkind constants in pg_dump.c.
Next
From: Amit Kapila
Date:
Subject: Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers