Re: Postgresql13_beta1 (could not rename temporary statistics file)Windows 64bits - Mailing list pgsql-hackers
| From | Ranier Vilela |
|---|---|
| Subject | Re: Postgresql13_beta1 (could not rename temporary statistics file)Windows 64bits |
| Date | |
| Msg-id | CAEudQApVGty9RiL7ymVFfpBMGUGKtdQmuEntjCYSb8ifLRkcWA@mail.gmail.com Whole thread Raw |
| In response to | Re: Postgresql13_beta1 (could not rename temporary statistics file)Windows 64bits (Justin Pryzby <pryzby@telsasoft.com>) |
| List | pgsql-hackers |
Em ter., 16 de jun. de 2020 às 01:10, Justin Pryzby <pryzby@telsasoft.com> escreveu:
On Mon, Jun 15, 2020 at 11:49:33PM -0300, Ranier Vilela wrote:
> I can confirm that the problem is in pgrename (dirmod.c),
> something is not OK, with MoveFileEx, even with the
> (MOVEFILE_REPLACE_EXISTING) flag.
>
> Replacing MoveFileEx, with
> unlink (to);
> rename (from, to);
>
> #if defined (WIN32) &&! defined (__ CYGWIN__)
> unlink(to);
> while (rename (from, to)! = 0)
> #else
> while (rename (from, to) <0)
> #endif
>
> The log messages have disappeared.
>
> I suspect that if the target (to) file exists, MoveFileEx, it is failing to
> rename, even with the flag enabled.
>
> Windows have the native rename function (
> https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/rename-wrename?view=vs-2019
> )
> However, it fails if the target (to) file exists.
>
> Question, is it acceptable delete the target file, if it exists, to
> guarantee the rename?
I don't think so - the idea behind writing $f.tmp and renaming it to $f is that
it's *atomic*.
"atomic" here means, rename operation only, see.
1. create $f.tmp
2. write
3. close
4. rename to $f
I found an earlier thread addressing the same problem - we probably both
should've found this earlier.
https://commitfest.postgresql.org/27/2230/
https://www.postgresql.org/message-id/flat/CAPpHfds9trA6ipezK3BsuuOSQwEmESiqj8pkOxACFJpoLpcoNw%40mail.gmail.com#9b04576b717175e9dbf03cc991977d3f
Very interesting.
That link, gave me a light.
Users report this same situation in production environments with a high load.
Here I have only one notebook, with 8gb ram, windows 10 64 bits (2004), msvc 2019 (64 bits).
Without any load, just starting and stopping the server, the messages appear in the log.
Here I have only one notebook, with 8gb ram, windows 10 64 bits (2004), msvc 2019 (64 bits).
Without any load, just starting and stopping the server, the messages appear in the log.
That thread goes back to 2017, so I don't think this is a new problem in v13.
I'm not sure why you wouldn't also see the same behavior in v12.
Windows Server 2003 (32 bits) with Postgres 9.6 (32 bits), none log (could rename).`
Windows Server 2016 (64 bits) with Postgres 12 (64 bits), have log (cound rename).
Yes V12 can confirm, too.
File postgresql-Mon.log:
2019-10-14 11:56:26 GMT [4844]: [1-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
File postgresql-Sat.log:
2019-09-28 10:15:52 GMT [4804]: [2-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
2019-10-05 12:01:23 GMT [4792]: [1-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
2019-10-05 23:55:31 GMT [4792]: [2-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
File postgresql-Sun.log:
2019-10-06 07:43:36 GMT [4792]: [3-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
File postgresql-Tue.log:
2019-10-01 07:31:46 GMT [4804]: [3-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
2019-10-22 14:44:25 GMT [3868]: [1-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
File postgresql-Wed.log:
2019-10-02 22:20:52 GMT [4212]: [1-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
2019-10-09 11:05:02 GMT [3712]: [1-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
2019-10-14 11:56:26 GMT [4844]: [1-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
File postgresql-Sat.log:
2019-09-28 10:15:52 GMT [4804]: [2-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
2019-10-05 12:01:23 GMT [4792]: [1-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
2019-10-05 23:55:31 GMT [4792]: [2-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
File postgresql-Sun.log:
2019-10-06 07:43:36 GMT [4792]: [3-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
File postgresql-Tue.log:
2019-10-01 07:31:46 GMT [4804]: [3-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
2019-10-22 14:44:25 GMT [3868]: [1-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
File postgresql-Wed.log:
2019-10-02 22:20:52 GMT [4212]: [1-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
2019-10-09 11:05:02 GMT [3712]: [1-1] user=,db=,app=,client= LOG: could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied
There's a couple patches in that thread, and the latest patch was rejected.
Maybe you'd want to test them out and provide feedback.
I will try.
BTW, the first patch does:
! if (filePresent)
! {
! if (ReplaceFile(to, from, NULL, REPLACEFILE_IGNORE_MERGE_ERRORS, 0, 0))
! break;
! }
! else
! {
! if (MoveFileEx(from, to, MOVEFILE_REPLACE_EXISTING))
! break;
! }
Since it's racy to first check if the file exists, I would've thought we should
instead do:
ret = ReplaceFile()
if ret == OK:
break
else if ret == FileDoesntExist:
if MoveFileEx():
break
Or, should we just try to create a file first, to allow ReplaceFile() to always
work ?
This seemingly, it works too.
+ while (rename(from, to) != 0 && !MoveFileEx(from, to, MOVEFILE_REPLACE_EXISTING | MOVEFILE_COPY_ALLOWED))
Attached a patch.
regards,
Ranier Vilela
Attachment
pgsql-hackers by date: