Re: [GENERAL] import CSV file to a table - Mailing list pgsql-general

From Günce Kaya
Subject Re: [GENERAL] import CSV file to a table
Date
Msg-id CAAV2-mWE5Kf=55gEG1ZEYgFwf-QGoYc9PRHVxJ8wfZgEFwzW4A@mail.gmail.com
Whole thread Raw
In response to Re: [GENERAL] import CSV file to a table  (Rob Sargent <robjsargent@gmail.com>)
List pgsql-general
Hi,

When I open a new thread, I didn't know exactly what is true words to research. I read all of your posts and I think CSV parsing is the point to me. I've created my script and I share it to record. 

#cat cargo2.sh
#!/bin/bash
while IFS=, read uor_desc crime_type zip_code ucr_hierarchy date_reported date_occured
do

echo "select * from insertorders('$uor_desc', '$crime_type', '$zip_code', '$fucr_hierarchy', '$date_reported', '$date_occured');"

done < test.txt;

Thank you for your advices and helps.

Regards,

Gunce

2017-03-08 20:00 GMT+03:00 Rob Sargent <robjsargent@gmail.com>:


On 03/08/2017 09:52 AM, Karl Czajkowski wrote:
On Mar 08, Rob Sargent modulated:

Yes Karl, I agree.  I admitted as much.  But if it's clean, as in
free of quoted commas, life is much more simple.  I've lost site of
whether or not the OP knows his situation w.r.t. to this.  The awk
line will tell him and for a one-off load this can make a world of
difference in complexity - two bash lines and a COPY.

Maybe I didn't understand your awk? I thought it was counting commas
in lines.  This isn't the same as counting commas in records.

     this,is,record,one
     "this,,","is
     ,,record","two
     ,,,"

this has three commas on each line and definitely is not suitable
for naive CSV handling.


Karl
In essence it does count commas but plus one :).  $NF is number of fields defined by commas so one more field than number of commas. If you think/hope the file is simple and well formatted, this is a pretty quick check. But if you're looking for a general solution, you need a real csv parser.  I recall being quite surprised and amused to learn there is an actual standard for csv format. (Naturally if you have one to hand, you don't need the awk line.)



--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general



--
Gunce Kaya

pgsql-general by date:

Previous
From: Patrick B
Date:
Subject: [GENERAL] count case when - PG 9.2
Next
From: Tom Lane
Date:
Subject: Re: [GENERAL] Unable to start postgresql