Re: Bulkloading using COPY - ignore duplicates? - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Bulkloading using COPY - ignore duplicates?
Date
Msg-id 7632.1010009907@sss.pgh.pa.us
Whole thread Raw
In response to Re: Bulkloading using COPY - ignore duplicates?  (Bruce Momjian <pgman@candle.pha.pa.us>)
Responses Re: Bulkloading using COPY - ignore duplicates?  (Bruce Momjian <pgman@candle.pha.pa.us>)
List pgsql-hackers
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I think we can allow something like:
> COPY FROM '/tmp/x' WITH ERRORS 2

> Yes, I realize we need subtransactions or something, but we should add
> it to the TODO list if it is a valid request, right?

Well, I don't like that particular API in any case.  Why would I think
that 2 errors are okay and 3 are not, if I'm loading a
many-thousand-line COPY file?  Wouldn't it matter *what* the errors
are, at least as much as how many there are?  "Discard duplicate rows"
is one thing, but "ignore bogus data" (eg, unrecognizable timestamps)
is not the same animal at all.

As someone already remarked, the correct, useful form of such a feature
is to echo the rejected lines to some sort of output file that I can
look at afterwards.  How many errors there are is not the issue.
        regards, tom lane


pgsql-hackers by date:

Previous
From: Bruce Momjian
Date:
Subject: Re: Bulkloading using COPY - ignore duplicates?
Next
From: Laurette Cisneros
Date:
Subject: Re: bug in join?