Re: stat() on Windows might cause error if target file is largerthan 4GB - Mailing list pgsql-hackers

From Michael Paquier
Subject Re: stat() on Windows might cause error if target file is largerthan 4GB
Date
Msg-id 20180910064454.GG19122@paquier.xyz
Whole thread Raw
In response to stat() on Windows might cause error if target file is larger than4GB  ("Higuchi, Daisuke" <higuchi.daisuke@jp.fujitsu.com>)
Responses Re: stat() on Windows might cause error if target file is largerthan 4GB
List pgsql-hackers
On Mon, Sep 10, 2018 at 02:01:53AM +0000, Higuchi, Daisuke wrote:
> This mail is about following bug report post:
> https://www.postgresql.org/message-id/flat/153442391458.1505.9181095584291689853%40wrigleys.postgresql.org

Thanks for following up on this thread.  I marked that as one of my
TODOs but could not move around to look at it seriously.

> This is because _stat64i32() is used for stat() on Windows, I think.
> Seeing following URL, _stat64i32() could use 32 bit, it means 4GB is max
> size.
> https://msdn.microsoft.com/en-us/library/14h5k7ff.aspx
>
> When I create the simple application to use stat() on VS2013,
> stat() could not deal with 4GB file and failed with errno=132.

I don't quite follow the logic here, we map stat() to pgwin32_safestat()
when compiling with MSVC.

> I think the pg_dump occurs EOVERFLOW, but Windows' errno.h does not have
> this errno, so 'Unknown error' is output.

The error mapping happens in win32error.c, so we'd want to update this
table to make sure that EOVERFLOW is the errno reported is correct.

> So, pgwin32_safestat() should be changed to solve this problem.
> Do you have any idea or comments?

Yes, the fix needs to happen there.  It seems to me that what we are
looking for here is to complete the calculation of st_size with
nFileSizeHigh, so as we are able to compile a correct 64-bit integer
size, allowing support for larger files on Win32, which is something
that fsync() support for pg_dump has actually changed by opening stat()
to be called for files with a size larger than 4GB.  (I need badly to
setup a Windows machine...)
--
Michael

Attachment

pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: How to find local logical replication origin?
Next
From: Amit Kapila
Date:
Subject: Re: SerializeParamList vs machines with strict alignment