Thread: Could not remove file messages

Hi:

We've been getting a message in our logs on a semi-regular basis in response to a cron job that drops
and recreates a number of databases using the dropdb command. The message indicates that a directory
cannot be removed, and sure enough, when I look in the file system, the 11210840 directory exists,
although it's empty. When I check the oids of my databases, 11210840 doesn't exist which is not
surprising given the successful drop database message at the end.

I was wondering why this might be happening and if there is an easy, safe way to clean up those
directories.

We're running 8.1.4. Relevant log messages below:

01:45:59 PDT WARNING:  could not remove file or directory "base/11210840": File exists
01:45:59 PDT WARNING:  could not remove database directory "base/11210840"
01:45:59 PDT LOG:  transaction ID wrap limit is 2147484146, limited by database "postgres"
01:46:00 PDT LOG:  duration: 3754.761 ms  statement: DROP DATABASE cdstest;

Thanks!
Mija

Mija Lee <mija@scharp.org> writes:
> We're running 8.1.4. Relevant log messages below:

> 01:45:59 PDT WARNING:  could not remove file or directory "base/11210840": File exists
> 01:45:59 PDT WARNING:  could not remove database directory "base/11210840"

Hm, on what platform?  There are known reasons for this to happen on
Windows (basically, that removing files can be delayed if the bgwriter
process still has them open).  I'm not sure to what extent that's been
fixed in later PG releases.  But you really ought to be running 8.1.9
anyway ...

            regards, tom lane

Solaris 10. We are using the solaris package of postgres, which is why we are
using 8.1.4 (I believe).

Mija Lee <mija@scharp.org> writes:
> Solaris 10. We are using the solaris package of postgres, which is why we are
> using 8.1.4 (I believe).

OK, that lets out the Windows theory.  Do you maybe have the database
directory mounted via NFS?  NFS has its own set of strange behaviors...
eg this one:
http://www.sunmanagers.org/archives/2000/0924.html
You said the directory appeared empty, but did you check for dot-files?

            regards, tom lane

Yes, we are using nfs mounted data directories. There are no .nfs files in the
subdirectory, but as I understand it, that doesn't necessarily mean that was not
the source of the problem. I am going to try moving the data directory locally
on a test server and see if I still get the messages.

Thanks so much for the help!
Mija

Mija Lee <mija@scharp.org> writes:
> Yes, we are using nfs mounted data directories. There are no .nfs files in the
> subdirectory, but as I understand it, that doesn't necessarily mean that was not
> the source of the problem. I am going to try moving the data directory locally
> on a test server and see if I still get the messages.

OK --- if that is it, it's fairly likely that moving to 8.1.7 or later,
or 8.2.2 or later, would solve the problem.  If Sun hasn't got something
at least as up-to-date as that, they are falling down on the job.

            regards, tom lane

Hi,

On Thu, 2007-08-30 at 13:32 -0700, Mija Lee wrote:

> Solaris 10. We are using the solaris package of postgres, which is why
> we are  using 8.1.4 (I believe).

Blastwave has 8.2.3 in their stable tree, and 8.2.4 in their testing
tree:

http://www.blastwave.org/testing/index_cron.html

Regards,
--
Devrim GÜNDÜZ
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Managed Services, Shared and Dedicated Hosting
Co-Authors: plPHP, ODBCng - http://www.commandprompt.com/

Hi -

I just wanted to follow up on this. We've recently upgraded to 8.2.4. I still
get this message and the empty subdirectories with the later version of
postgres. I have seen the .nfs files temporarily in the subdirectories that
remain so I am sure that you are right about the cause, Tom.

Unless you know that this is being worked on for the next version of postgres, I
think I'll just write a script that removes empty subdirectories from the pgdata
dir that are more than a week old or so, unless you think this is a bad idea.

Mija

Mija Lee <mija@scharp.org> writes:
> I just wanted to follow up on this. We've recently upgraded to 8.2.4. I still
> get this message and the empty subdirectories with the later version of
> postgres. I have seen the .nfs files temporarily in the subdirectories that
> remain so I am sure that you are right about the cause, Tom.

Hmm ... I had hoped that the fixes we put in for Windows would solve
this for NFS too, but apparently not.

> Unless you know that this is being worked on for the next version of
> postgres, I think I'll just write a script that removes empty
> subdirectories from the pgdata dir that are more than a week old or
> so, unless you think this is a bad idea.

Well, any fix will probably not appear in production for some months, so
you might as well put in the script if the empty directories bother you.

            regards, tom lane