Thread: Bulk COPY end of copy delimiter
Today while trying to do a bulk COPY of data into a table, the process aborted with the following error message: ERROR: end-of-copy marker corrupt CONTEXT: COPY tbl_logged_event, line 178519: "606447014,1492,2005-02-24 03:16:14,2005-02-23 20:27:48,win_applog,,error,adsmclientservice,nt author..." Googling the error, we found reference to the '\.' (backslash-period) being an "end-of-copy marker". Unfortunately, our data contains the backslash-period character sequence. Is there any know fix or workaround for this condition? We're using Postgresql 7.3.9 and also running tests on an 8.0.1 system. Thanks in advance, --- Steve ___________________________________________________________________________________ Steven Rosenstein IT Architect/Developer | IBM Virtual Server Administration Voice/FAX: 845-689-2064 | Cell: 646-345-6978 | Tieline: 930-6001 Text Messaging: 6463456978 @ mobile.mycingular.com Email: srosenst @ us.ibm.com "Learn from the mistakes of others because you can't live long enough to make them all yourself." -- Eleanor Roosevelt
Hi, On Mon, 4 Apr 2005, Steven Rosenstein wrote: > > > > > Today while trying to do a bulk COPY of data into a table, the process > aborted with the following error message: > > ERROR: end-of-copy marker corrupt > CONTEXT: COPY tbl_logged_event, line 178519: "606447014,1492,2005-02-24 > 03:16:14,2005-02-23 20:27:48,win_applog,,error,adsmclientservice,nt > author..." > > Googling the error, we found reference to the '\.' (backslash-period) being > an "end-of-copy marker". Unfortunately, our data contains the > backslash-period character sequence. Is there any know fix or workaround > for this condition? Any sequence \. in COPY input data should be escaped as \\. If this data was generated by pg_dump then its a problem, but I haven't seen any other reports of this. Can I assume that you've generated the data for bulk load yourself? If so, there is discussion of escaping characters here: http://www.postgresql.org/docs/8.0/static/sql-copy.html. Gavin
Your assumption is correct. The data was generated out of a DB2 database,
and uses commas as field delimiters.
Thank you for the workaround,
--- Steve
___________________________________________________________________________________
Steven Rosenstein
IT Architect/Developer | IBM Virtual Server Administration
Voice/FAX: 845-689-2064 | Cell: 646-345-6978 | Tieline: 930-6001
Text Messaging: 6463456978 @ mobile.mycingular.com
Email: srosenst @ us.ibm.com
"Learn from the mistakes of others because you can't live long enough to
make them all yourself." -- Eleanor Roosevelt
Gavin Sherry
<swm@alcove.com.a
u> To
Sent by: Steven Rosenstein/New
pgsql-performance York/IBM@IBMUS
-owner@postgresql cc
.org pgsql-performance@postgresql.org
Subject
Re: [PERFORM] Bulk COPY end of copy
04/04/2005 08:00 delimiter
PM
Hi,
On Mon, 4 Apr 2005, Steven Rosenstein wrote:
>
>
>
>
> Today while trying to do a bulk COPY of data into a table, the process
> aborted with the following error message:
>
> ERROR: end-of-copy marker corrupt
> CONTEXT: COPY tbl_logged_event, line 178519: "606447014,1492,2005-02-24
> 03:16:14,2005-02-23 20:27:48,win_applog,,error,adsmclientservice,nt
> author..."
>
> Googling the error, we found reference to the '\.' (backslash-period)
being
> an "end-of-copy marker". Unfortunately, our data contains the
> backslash-period character sequence. Is there any know fix or workaround
> for this condition?
Any sequence \. in COPY input data should be escaped as \\. If this data
was generated by pg_dump then its a problem, but I haven't seen any other
reports of this. Can I assume that you've generated the data for bulk load
yourself? If so, there is discussion of escaping characters here:
http://www.postgresql.org/docs/8.0/static/sql-copy.html.
Gavin
---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly