Re: psql copy command - 1 char limitation on delimiter - Mailing list pgsql-general

From Steve Crawford
Subject Re: psql copy command - 1 char limitation on delimiter
Date
Msg-id 4CBCA9A3.6020608@pinpointresearch.com
Whole thread Raw
In response to Re: psql copy command - 1 char limitation on delimiter  (Bruce Momjian <bruce@momjian.us>)
List pgsql-general
On 10/12/2010 08:28 PM, Bruce Momjian wrote:
> Steve Crawford wrote:
>
>> On 09/25/2010 07:03 AM, Tom Lane wrote:
>>
>>> rey<reywang@optonline.net>   writes:
>>>
>>>
>>>> Why limit this to a single character?
>>>>
>>>>
>>> Performance.  Believe it or not, breaking fields at the delimiter is
>>> a significant factor in COPY speed.
>>>
>>>             regards, tom lane
>>>
>>>
>>>
>> I agree that that multi-character (or even regex) delimiters would be
>> useful. Would it be reasonable for the copy process to differentiate
>> between single character delimiters which could be processed in
>> "high-speed" mode and multi-character or regex delimiters which would be
>> available as needed albeit at the expense of a performance hit?
>>
> I am not sure you are aware but Postgres never confuses delimiters from
> data because it uses a backslash before literal data that matches
> delimiters.
>
>
Yes, I am. But the discussion was about using multi-character strings as
delimiters.

But while I have encountered files using multiple-character delimiters,
I'm finding myself leaning toward the camp that says that such cases are
better processed externally by Perl/Python/sed/awk/Ruby/ETL/etc.
Especially given the "fun" of defining how to properly escape a
regex-matching string in a regex delimited file.

Cheers,
Steve


pgsql-general by date:

Previous
From: "Joshua D. Drake"
Date:
Subject: PgWest 2010: Two weeks away
Next
From: "Turner, John J"
Date:
Subject: Re: installing from source in Windows