Thread: BUG #16424: COPY Command fails for CSV formath
The following bug has been logged on the website: Bug reference: 16424 Logged by: Vivek Kamath Email address: vivek4422@gmail.com PostgreSQL version: 10.6 Operating system: Linux Unix Description: Hi, I am copying a csv file where the fields are quoted with " and separated by ,. I am using below syntax to copy data using below syntax psql -c "COPY table_name from filename WITH (FORMAT csv)". On Analysis I observed that any data that even number of double quotes in it are copied successfully but fails when it has odd number of double quotes in it. For ex 1234,"Test "hi" ","Test bye" will be loaded as 1234,Test hi, Test bye in table successfully but my expectation is to load data as 1234,Test "hi", Test bye where as 1234,"Test "hi ", "Test bye" will fail with error ERROR: unterminated CSV quoted field.. In this case my expectation is to load data as 1234,Test "hi, Test bye. Also since all this is is happening as part of the batch. I cannot pre-process the data as the same data set needs to be loaded in Redshift DB. Copy command works perfectly fine with this data set in redshift. Idea is load whatever comes in between the double quotes ("......") as is in varchar field. Please suggest some ideas as how this can be achieved.
On Friday, May 8, 2020, PG Bug reporting form <noreply@postgresql.org> wrote:
The following bug has been logged on the website:
Bug reference: 16424
Logged by: Vivek Kamath
Email address: vivek4422@gmail.com
PostgreSQL version: 10.6
Operating system: Linux Unix
Description:
Hi,
I am copying a csv file where the fields are quoted with " and separated by
,.
I am using below syntax to copy data using below syntax
psql -c "COPY table_name from filename WITH (FORMAT csv)".
Replace that command with something that will handle the poorly structured csv data you are receiving. COPY is very linear and unforgiving when it comes to accepting input so you need something that will feed copy that well-formed data it expects).
David J.