Re: [HACKERS] Orphaned files in base/[oid] - Mailing list pgsql-hackers

From Tom Lane
Subject Re: [HACKERS] Orphaned files in base/[oid]
Date
Msg-id 29046.1502736046@sss.pgh.pa.us
Whole thread Raw
In response to Re: [HACKERS] Orphaned files in base/[oid]  (Chris Travers <chris.travers@adjust.com>)
Responses Re: [HACKERS] Orphaned files in base/[oid]  (Andres Freund <andres@anarazel.de>)
Re: [HACKERS] Orphaned files in base/[oid]  (Chris Travers <chris.travers@adjust.com>)
List pgsql-hackers
Chris Travers <chris.travers@adjust.com> writes:
> On Mon, Aug 14, 2017 at 6:33 PM, Andres Freund <andres@anarazel.de> wrote:
>> I think the fix here is to call RemovePgTempFiles() during
>> crash-restarts, instead of just full starts. The previously stated need
>> to be able to inspect temp files after a crash can be less impactfully
>> fulfilled with restart_after_crash = false.

> But that only clears temp files right?
> I am less concerned about the temp files because a restart clears them.

It will clear temp files, and also temp tables.

> The bigger issue I see are with the orphaned base files.

It would be possible to have orphaned non-temp tables if you'd suffered
a crash during the transaction that created those tables.  Ordinarily
a newly-created table file wouldn't be that large, but if your workflow
created tables and shoved boatloads of data into them in the same
transaction, it's not so hard to see this becoming an issue.

The core problem with zapping non-temp table files is that you can't
do that unless you're sure you have consistent, up-to-date pg_class
data that nobody else is busy adding to.  It's hard to see an external
application being able to do that safely.  You certainly can't do it
at the point in the postmaster startup cycle where we currently do
the other things --- for those, we rely only on filesystem naming
conventions to identify what to zap.
        regards, tom lane



pgsql-hackers by date:

Previous
From: Chris Travers
Date:
Subject: Re: [HACKERS] Orphaned files in base/[oid]
Next
From: Andres Freund
Date:
Subject: Re: [HACKERS] Orphaned files in base/[oid]