Thread: BUG #16127: PostgreSQL 12.1 on Windows 2008 R2 copy table from ‘large 2GB csv’report “iso-8859-1 error”

The following bug has been logged on the website:

Bug reference:      16127
Logged by:          yanliang lei
Email address:      leiyanliang@highgo.com
PostgreSQL version: 12.1
Operating system:   Windows 2008 R2
Description:

D:\>dir
 驱动器 D 中的卷是 新加卷

D:\Program Files\PostgreSQL\12>cd bin

D:\Program Files\PostgreSQL\12\bin>psql -d postgres -U posgres
用户 posgres 的口令:
psql: 错误: 无法连接到服务器:FATAL:  password authentication failed for user "p
osgres"

D:\Program Files\PostgreSQL\12\bin>psql -d postgres -U postgres
用户 postgres 的口令:
psql (12.1)
输入 "help" 来获取帮助信息.

postgres=# select version();
                          version
------------------------------------------------------------
 PostgreSQL 12.1, compiled by Visual C++ build 1914, 64-bit
(1 行记录)


postgres=# CREATE TABLE github_events
postgres-# (
postgres(#     event_id bigint,
postgres(#     event_type text,
postgres(#     event_public boolean,
postgres(#     repo_id bigint,
postgres(#     payload jsonb,
postgres(#     repo jsonb,
postgres(#     user_id bigint,
postgres(#     org jsonb,
postgres(#     created_at timestamp
postgres(# );
CREATE TABLE
postgres=# \timing
启用计时功能.
postgres=# set client_encoding='UTF8';
SET
时间:1.185 ms
postgres=# set lc_messages=en_us;
SET
时间:23.467 ms
postgres=# copy github_events from  'd:\large_events.csv' csv;
ERROR:  could not stat file "d:\large_events.csv": Unknown error
----->>>Please note this error!!!
时间:4.558 ms
postgres=# select version();
                          version
------------------------------------------------------------
 PostgreSQL 12.1, compiled by Visual C++ build 1914, 64-bit
----->>>>PostgreSQL version is 12.1
(1 行记录)


时间:0.280 ms
postgres=#

Please note:
1.  d:\large_events.csv this file is large 2GB。
2.  the software “postgresql-12.1-1-windows-x64.exe” is downloaded from
enterprisedb website.


On Wed, Nov 20, 2019 at 12:42:38PM +0000, PG Bug reporting form wrote:
>The following bug has been logged on the website:
>
>Bug reference:      16127
>Logged by:          yanliang lei
>Email address:      leiyanliang@highgo.com
>PostgreSQL version: 12.1
>Operating system:   Windows 2008 R2
>Description:
>
>D:\>dir
> 驱动器 D 中的卷是 新加卷
>
>D:\Program Files\PostgreSQL\12>cd bin
>
>D:\Program Files\PostgreSQL\12\bin>psql -d postgres -U posgres
>用户 posgres 的口令:
>psql: 错误: 无法连接到服务器:FATAL:  password authentication failed for user "p
>osgres"
>
>D:\Program Files\PostgreSQL\12\bin>psql -d postgres -U postgres
>用户 postgres 的口令:
>psql (12.1)
>输入 "help" 来获取帮助信息.
>
>postgres=# select version();
>                          version
>------------------------------------------------------------
> PostgreSQL 12.1, compiled by Visual C++ build 1914, 64-bit
>(1 行记录)
>
>
>postgres=# CREATE TABLE github_events
>postgres-# (
>postgres(#     event_id bigint,
>postgres(#     event_type text,
>postgres(#     event_public boolean,
>postgres(#     repo_id bigint,
>postgres(#     payload jsonb,
>postgres(#     repo jsonb,
>postgres(#     user_id bigint,
>postgres(#     org jsonb,
>postgres(#     created_at timestamp
>postgres(# );
>CREATE TABLE
>postgres=# \timing
>启用计时功能.
>postgres=# set client_encoding='UTF8';
>SET
>时间:1.185 ms
>postgres=# set lc_messages=en_us;
>SET
>时间:23.467 ms
>postgres=# copy github_events from  'd:\large_events.csv' csv;
>ERROR:  could not stat file "d:\large_events.csv": Unknown error
>----->>>Please note this error!!!

This probably means the PostgreSQL user/process does not have access to
the file, either because it does not exist, lack of privileges, an AV
system blocking the access, or something like that.

I suggest you seach in the Windows Event Log, and various other logs you
might have there.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 



PG Bug reporting form <noreply@postgresql.org> writes:
> postgres=# copy github_events from  'd:\large_events.csv' csv;
> ERROR:  could not stat file "d:\large_events.csv": Unknown error
> ----->>>Please note this error!!!

> 1.  d:\large_events.csv this file is large 2GB

Given that, I wonder if this isn't the same issue being worked
on over here:

https://www.postgresql.org/message-id/flat/15858-9572469fd3b73263@postgresql.org
https://commitfest.postgresql.org/25/2189/

If you could help review/test that patch, it would make it more
likely to get fixed soon.

            regards, tom lane



On Wed, Nov 20, 2019 at 10:47:41AM -0500, Tom Lane wrote:
> Given that, I wonder if this isn't the same issue being worked
> on over here:
>
> https://www.postgresql.org/message-id/flat/15858-9572469fd3b73263@postgresql.org
> https://commitfest.postgresql.org/25/2189/
>
> If you could help review/test that patch, it would make it more
> likely to get fixed soon.

I think that's the same issue with stat() not working for files larger
than 2GB on Windows.
--
Michael

Attachment


On Thu, Nov 28, 2019 at 5:16 AM sirlipeng <sirlipeng@gmail.com> wrote:
On Wed, Nov 20, 2019 at 09:42:49PM +0800, leiyanliang@highgo.com wrote:

>In that case, I think Tom is right it's likely the same issue as [1],
>although that starts with a report about issues with 4GB. Maybe that's
>due to using different windows version 2008 R2 vs. 2012 R2, though.
>
>It'd be good if you could test the latest patch [2] in that thread.
>You'll need to build PostgreSQL on Windows, though - I don't have much
>experience with that, but there are wiki pages about doing that either
>using Visual Studio [3] or mingw [4].

I have had the same problem on both win2008r2(64bit) and win10(64bit) with postgreSQL12.1.

The problem has been solved when the above path been applied on win2008r2(64bit) and win10(64bit).

It works:
postgres=# copy github_events from '\\vmware-host\Shared Folders\share\large_eve
nts.csv' csv;
COPY 1146625


Great! Thanks for testing.

Regards,

Juan José Santamaría Flecha