Re: Ragged CSV import - Mailing list pgsql-hackers

From Robert Haas
Subject Re: Ragged CSV import
Date
Msg-id 603c8f070909092019n26f7d82di3a666d332d66207b@mail.gmail.com
Whole thread Raw
In response to Re: Ragged CSV import  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Ragged CSV import
List pgsql-hackers
On Wed, Sep 9, 2009 at 11:01 PM, Tom Lane<tgl@sss.pgh.pa.us> wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>> Robert Haas wrote:
>>> I wonder whether it would be appropriate to do something like
>>> implement a method by which copy could return text[] and then one
>>> could write wrappers around that functionality to do this as well as
>>> other things.
>
>> Anything along these lines in C is likely to be far larger than what I
>> had in mind, which was a fairly minor enhancement.
>
> I think Robert's got a point though.  What you're talking about is a
> fairly specialized single-purpose feature, which nonetheless is going to
> require a lot of infrastructure (for example, teaching psql's \copy
> about it).  Perhaps, for approximately the same amount of overhead,
> we could provide something more general.
>
> I don't agree with the idea of injecting something "behind" copy though.
> The thought that comes to mind for me is something "in front of" copy,
> that is, give it the text of each line and let it do a text-to-text
> transformation before COPY chews on it.

That seems to me to be a whole lot less useful.  As I see it, the
whole point of any enhancement in this area is to reuse the parsing
code.   If I have a CSV file (or some other format COPY understands),
I don't want to have to write my own parser for that format in order
to do some simple data transformation (like dropping columns >n).

Previous agreements nonwithstanding, I think letting COPY do general
transformations is a great idea.  But I'm a lot more skeptical about
one specific transformation without some framework of which that case
is the first instance.

...Robert


pgsql-hackers by date:

Previous
From: Andrew Dunstan
Date:
Subject: Re: Ragged CSV import
Next
From: Andrew Gierth
Date:
Subject: Re: LATERAL