Re: Practical error logging for very large COPY - Mailing list pgsql-hackers

From Greg Stark
Subject Re: Practical error logging for very large COPY
Date
Msg-id 8764qkodnf.fsf@stark.xeocode.com
Whole thread Raw
In response to Re: Practical error logging for very large COPY  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Practical error logging for very large COPY  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
Tom Lane <tgl@sss.pgh.pa.us> writes:

> Martijn van Oosterhout <kleptog@svana.org> writes:
> > On Tue, Nov 22, 2005 at 09:58:44AM -0500, Tom Lane wrote:
> >> The general problem that needs to be solved is "trap any error that
> >> occurs during attempted insertion of a COPY row, and instead of aborting
> >> the copy, record the data and the error message someplace else".
> 
> > Actually, there are really only a few errors people want to trap I
> > imagine:
> 
> You've forgotten bad data, eg "foo" in an integer field, or an
> untranslatable multibyte character.  The bad-data problem is what lets
> out trigger-based solutions, or indeed anything that presumes that the
> bad data can be forced into a particular representation.

I think that's precisely the point here though. There are basically two
categories of errors:

1) Data that can be parsed and loaded but generates some sort of constraint  violation such as a UNIQUE violation,
foreignkey violation, or other  constraint violation.
 

2) Data that can't be parsed as the correct data type at all.

It would be nice to be able to have the former loaded into an actual table
where it can be queried and perhaps fixed and reloaded.

The latter clearly cannot. I would say it should just generate a log entry.


-- 
greg



pgsql-hackers by date:

Previous
From: Andreas Pflug
Date:
Subject: Re: TODO item "%Allow pg_hba.conf be controlled via SQL"
Next
From: Grzegorz Jaskiewicz
Date:
Subject: Re: order by, for custom types