One small detail I'd like to draw attention to is this bit, which differs slightly from the [non-working] previous attempts by mapping to two different errors:
+ * If there's no O_CREAT flag, then we'll pretend the file is + * invisible. With O_CREAT, we have no choice but to report that + * there's a file in the way (which wouldn't happen on Unix).
When GetTempFileName() finds a duplicated file name and the file is pending for deletion, it fails with an "ERROR_ACCESS_DENIED" error code. That may describe the situation better than "ERROR_FILE_EXISTS".