Re: [PERFORMANCE] Error loading 37G CSV file "invalid string enlargement request size 65536" - Mailing list pgsql-performance

From Tom Lane
Subject Re: [PERFORMANCE] Error loading 37G CSV file "invalid string enlargement request size 65536"
Date
Msg-id 27806.1208901323@sss.pgh.pa.us
Whole thread Raw
In response to [PERFORMANCE] Error loading 37G CSV file "invalid string enlargement request size 65536"  ("Adonias Malosso" <malosso@gmail.com>)
List pgsql-performance
"Adonias Malosso" <malosso@gmail.com> writes:
> I�m running a copy for a 37G CSV and receiving the following error:
> "invalid string enlargement request size 65536"

AFAICS this would only happen if you've got an individual line of COPY
data exceeding 1GB.  (PG versions later than 8.2 give a slightly more
helpful "out of memory" error in such a case.)

Most likely, that was not your intention, and the real problem is
incorrect quoting/escaping in the CSV file, causing COPY to think
that a large number of physical lines should be read as one logical line.

            regards, tom lane

pgsql-performance by date:

Previous
From: "Adonias Malosso"
Date:
Subject: [PERFORMANCE] Error loading 37G CSV file "invalid string enlargement request size 65536"
Next
From: Ivan Voras
Date:
Subject: Re: Suspicious top output