Hi Aaron:
On Tue, Jan 6, 2015 at 7:06 PM, Aaron Botsis <aaron@bt-r.com> wrote:
> Hi Francisco, I=E2=80=99m aware, but still consider this to be a bug, or =
at least
> a great opportunity for an enhancement. :)
>
Maybe, but you are going to have a problem.
> This had bitten me for the third time while trying to import some json
> data. It=E2=80=99d be great to bypass the copy escaping (and possibly oth=
er meta
> characters) when the column type is json or jsonb. I=E2=80=99d be happy t=
o try and
> write it and submit a patch if folks believe this is an acceptable way to
> go=E2=80=A6 That said, I should probably read what the process is for thi=
s kind of
> thing :)
>
Reading this, you are talking about 'the column being json'. COPY needs to
do the escaping at the same time it's constructing the columns. The present
way is easy to do, read char by char, if it's a escape, process next char
acumulating into current field, otherwise see whether it is a field or
record separator and act accordingly. It's also layered, when you construct
the records for copy you get all the field data, turn them into escaped
strings, join them by the field separator and spit them out followed by a
record separator ( practical implementations may do this virtually ).
Intermixing this with the 'I'm in a json column' would need to pass
information from the upper layer, and make it more difficult and, specially
error prone. What do you do if ( using the standard delimiters ) your json
value has embeded newlines and tabs ( which, IIRC, are legal in several
places inside the json ). And all this to make some incorrectly formatted
files read ( which can be correctly formatted with a perl one liner or
something similar ). I'm not the one to decide, but I will vote against
including that ( but do not trust me too much, I would also vote against
including 'csv' which I consider the root of many evils ).
Francisco Olarte.
