Tom,
> Sure ... but you'll find that it's not large enough to be useful.
> Once you remove all the interesting consistency checks such as
> unique indexes and foreign keys, the COPY will tend to go through
> just fine, and then you're still stuck trying to weed out bad data
> without very good tools for it. The only errors we could really
> separate out without subtransaction fencing are extremely trivial
> ones like too many or too few fields on a line ... which can be
> caught with a sed script.
Speaking as someone who did a LOT of DW load design only a couple years ago,
I'll say that the "special case" of no triggers, no constraint checks except
length, and type-safety check actually constitutes about 50% of DW bulk
loading. The only exception to that is unique indexes, which would normally
be included and would be the difficult thing.
Also, "special case bulk loading" would in fact give users of other types of
applications a lot more flexibility -- they could always load into a holding
table just to clean up the type safety issues and then merge into the real
table.
So I don't agree that the "load into new partition without dependancies" is
too much of a special case to be worth pursuing. It might be a bad idea for
other reasons, but not because it's too obscure.
--Josh
