Thread: BUG #15975: Copy from large text files on Windows returns "iso-8859-1 error"

The following bug has been logged on the website:

Bug reference:      15975
Logged by:          Johann Oskarsson
Email address:      johann@myrkraverk.com
PostgreSQL version: 11.5
Operating system:   Windows 10
Description:

When I try to load a large text file into PostgreSQL 11 on Windows, I get an
"unknown error."  This has been discussed on StackOverflow here:
https://stackoverflow.com/q/53523051/264751

It turns out that trying to COPY FROM a large textfile, say several
gigabytes, will always fail.  According to Stack Overflow, this only happens
on PostgreSQL 11.  The earlier releases don't seem to have this problem.

Example, given

  create table pwned( passwd sha1 not null, count int4 not null );

Then

  copy pwned( passwd, count ) from
'c:\src\postgres\pwned\pwned-passwords-sha1-ordered-by-hash-v5.txt' with (
format text, delimiter ':' );

gives

  ERROR:  could not stat file
"c:\src\postgres\pwned\pwned-passwords-sha1-ordered-by-hash-v5.txt": Unknown
error

Workaround is to pipe the file through a program, as discussed here:
https://wp.me/p4w9PF-di

PG Bug reporting form <noreply@postgresql.org> writes:
> It turns out that trying to COPY FROM a large textfile, say several
> gigabytes, will always fail.

This is being worked on.  If you're in a position to review or test
the patch being discussed here:

https://commitfest.postgresql.org/24/2189/

please do.

> According to Stack Overflow, this only happens
> on PostgreSQL 11.

Seems unlikely.  There's never been support in PG for working around
Windows' weirdness in this area (ie lack of stat() support for the case).

            regards, tom lane