Hi,
sorry for answering a bit later than I hoped. Here is my review so far:
Contents
======
This patch starts to address in my opinion one of COPY's shortcoming, which is error handling. PK and exclusion errors are taken care of, but not (yet?) other types of errors.
Documentation is updated, "\h copy" also and some regression tests are added.
Initial Run
=======
Patch applies (i've tested v6) cleanly.
make: OK
make install: OK
make check: OK
make installcheck: OK
Performance
========
I've tested the patch on a 1.1G file with 10 000 000 lines. Each test was done 15 times on a small local VM. Table is without constraints.
head: 38,93s
head + patch: 38,76s
Another test was one a 0.1GB file with 1 000 000 lines. Each test done 10 times on a small local VM and the table has a pk.
COPY 4,550s
COPY CONFLICT 4,595s
COPY CONFLICT with only one pk error 10,529s
COPY CONFLICT pk error every 100 lines 10,859s
COPY CONFLICT pk error every 1000 lines 10,879s
I did not test exclusions so far.
Thoughts
======
I find the feature useful in itself. One big question for me is can it be improved later on to handle other types of errors (like check constraints for example) ? A "-1" for the error limit would be very useful in my opinion.
I am also afraid that the name "error_limit" might mislead users into thinking that all error types are handled. But I do not have a better suggestion without making this clause much longer...
I've had a short look at the code, but this will need review by someone else.
Anyway, thanks a lot for taking the time to work on it.
Anthony
