Thread: possible bug: orphaned files left after immediate shutdown during DDL

Case:

  BEGIN;
  CREATE TABLE foo AS SELECT generate_series(1,1000);
  CHECKPOINT;
  SELECT relfilenode FROM pg_class WHERE relname='foo';

Let's say that returns 23456. Send the postmaster a SIGQUIT (immediate
shutdown), and then restart. The file 23456 is still in the filesystem,
but there's no record in pg_class for it. I don't see any obvious path
where it will be removed, so it looks like it will just stay there
forever.

My question is: is this a conscious decision to be paranoid during
recovery, or is this a bug? Or is there some reason that properly
determining which files should be removed at recovery time is
challenging?

Regards,
    Jeff Davis

Jeff Davis <pgsql@j-davis.com> writes:
> Case:

>   BEGIN;
>   CREATE TABLE foo AS SELECT generate_series(1,1000);
>   CHECKPOINT;
>   SELECT relfilenode FROM pg_class WHERE relname='foo';

> Let's say that returns 23456. Send the postmaster a SIGQUIT (immediate
> shutdown), and then restart. The file 23456 is still in the filesystem,
> but there's no record in pg_class for it. I don't see any obvious path
> where it will be removed, so it looks like it will just stay there
> forever.

> My question is: is this a conscious decision to be paranoid during
> recovery, or is this a bug?

It's intentional ... not that other people haven't complained about it
before.  Remember that what you have done is forced a crash, and
recovery from it is crash recovery.  If we proactively removed such
files we would very possibly be destroying evidence of forensic value.

IMO, immediate shutdown is not a tool to be used at random, and this
isn't something we need to fix.

            regards, tom lane

On Wed, 2011-02-09 at 22:58 -0500, Tom Lane wrote:
> It's intentional ... not that other people haven't complained about it
> before.  Remember that what you have done is forced a crash, and
> recovery from it is crash recovery.  If we proactively removed such
> files we would very possibly be destroying evidence of forensic value.

I thought that might be the case, but I wasn't able to find any previous
discussions.

It might be a good idea to issue a warning during recovery, however,
like "possible orphaned file ...". I'm not sure if it's worth the
bookkeeping effort though.

Regards,
    Jeff Davis

Jeff Davis wrote:
> On Wed, 2011-02-09 at 22:58 -0500, Tom Lane wrote:
> > It's intentional ... not that other people haven't complained about it
> > before.  Remember that what you have done is forced a crash, and
> > recovery from it is crash recovery.  If we proactively removed such
> > files we would very possibly be destroying evidence of forensic value.
>
> I thought that might be the case, but I wasn't able to find any previous
> discussions.
>
> It might be a good idea to issue a warning during recovery, however,
> like "possible orphaned file ...". I'm not sure if it's worth the
> bookkeeping effort though.

I thought we had a TODO item about removing orphaned files, but I don't
see it now, perhaps because I thought we had fixed that.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + It's impossible for everything to be true. +