Re: [HACKERS] GSOC'17 project introduction: Parallel COPY executionwith errors handling - Mailing list pgsql-hackers

From Pavel Stehule
Subject Re: [HACKERS] GSOC'17 project introduction: Parallel COPY executionwith errors handling
Date
Msg-id CAFj8pRA4pznAjyT44xxv-ryiVzao5DJvUHh9kXBSCWYGQ9w_MQ@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] GSOC'17 project introduction: Parallel COPY executionwith errors handling  (Peter Eisentraut <peter.eisentraut@2ndquadrant.com>)
List pgsql-hackers


2018-03-03 15:02 GMT+01:00 Peter Eisentraut <peter.eisentraut@2ndquadrant.com>:
On 3/3/18 00:48, Tom Lane wrote:
> I don't think that can possibly work.  It would only be safe if, between
> the thrower and the catcher, there were no other levels of control
> operating according to the normal error-handling rules.  But input
> functions certainly cannot assume that they are only called by COPY,
> so how could they safely throw a "soft error"?

That assumes that throwing a soft error in a context that does not
handle it specially is not safe.  I'd imagine in such situations the
soft error just behaves like a normal exception.

This soft error solves what? If somebody use fault tolerant COPY, then he will not be satisfied, when only format errors will be ignored. Any constraints, maybe trigger errors should be ignored too.

But is true, so some other databases raises soft error on format issues, and doesn't raise a exception. Isn't better to use some alternative algorithm like

 under subtransaction try to insert block of 1000 lines. When some fails do rollback and try to import per block of 100 lines, and try to find wrong line?

or another way - for this specific case reduce the cost of subtransaction? This is special case, subtransaction under one command.

Regards

Pavel

 

--
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

pgsql-hackers by date:

Previous
From: Peter Eisentraut
Date:
Subject: Re: Rewriting the test of pg_upgrade as a TAP test - take two
Next
From: Peter Eisentraut
Date:
Subject: Re: JIT compiling with LLVM v11