Use subtraction is very inefficient, a project called pg_bulkload support the
skip errors ,and it does not useing subtraction. It performance is very good.
So I want to imitate pg_bulkload to implementation skip errors of copy.
if i do the following thing to copy :
1. disable all of trigger of table;
2. only skip the following errors:
* format error of tuple;
* check constraint violation;
* unique or primary key constraint violation;
other errors will be abort current transcation except above three errors.
3. bad tuple will be deleted and the resource owner per tuples is reseased
when a unique key violation, xmax of the tuple will be marked to current
transaction id as it not be seen as good, and all of index entries of the bad
tuple will be real delete at next VACUUM.
Is it right to skip errors for copy from?
Date: 2014-06-17 21:40
Subject: Re: [HACKERS] How to implement the skip errors for copy from ?
xbzhang <xbzhang@kingbase.com.cn> writes:
> LWlocks can record in resource owner per tuples, so they can be released��at rigth way, but the memory allocated on memory context��is one problem.Are there any others problems?
See AbortSubTransaction(), CleanupSubTransaction(), and the rather large
number of subroutines they call. Almost everything that code does is
connected to cleaning up something that might have been left unfinished
after an elog(ERROR) took control away in the middle of some code
sequence.
In addition, you can't just wave your hands and presto the bad tuple is
not there anymore. For example, the failure might have been a unique key
violation in some index or other. Not only is the bad tuple already on
disk, but possibly so are index entries for it in other indexes. In
general the only way to get rid of those index entries is a VACUUM.
So you really have to have a subtransaction whose XID is what you mark
the new tuple with, and then rolling back the subtransaction is what
causes the new tuple to not be seen as good. (Actually getting rid of
it will be left for the next VACUUM.)
regards, tom lane
-----
???????????
????:AVG - www.avg.com
??:2013.0.3480 / ?????:3955/7691 - ????:06/17/14
