Timely reporting of COPY errors - Mailing list pgsql-hackers

From Martijn van Oosterhout
Subject Timely reporting of COPY errors
Date
Msg-id 20080416202907.GA26340@svana.org
Whole thread Raw
Responses Re: Timely reporting of COPY errors  (Stephen Frost <sfrost@snowman.net>)
Re: Timely reporting of COPY errors  (Tom Lane <tgl@sss.pgh.pa.us>)
Re: Timely reporting of COPY errors  (Bruce Momjian <bruce@momjian.us>)
List pgsql-hackers
Hi,

I notice that while doing bulk-loads that any errors detected by the
backend arn't noticed by libpq until right at the end. Is this
intentional? Looking at the code we have this comment in putCopyData:
 /*  * Process any NOTICE or NOTIFY messages that might be pending in the  * input buffer.  Since the server might
generatemany notices during the  * COPY, we want to clean those out reasonably promptly to prevent  * indefinite
expansionof the input buffer.  (Note: the actual read of  * input data into the input buffer happens down inside
pqSendSome,but  * it's not authorized to get rid of the data again.)  */ 

Except that pqSendSome won't try reading anything until it has a
problem writing. Since the backend will consume copy data indefinitly,
the error message sits in the kernel buffers until the end.

Is there anything that can be done? I've tried putting in
PQconsumeInput in places but it doesn't appear to help.

Any ideas?
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> Please line up in a tree and maintain the heap invariant while
> boarding. Thank you for flying nlogn airlines.

pgsql-hackers by date:

Previous
From: Andrew Dunstan
Date:
Subject: Re: How to submit a patch
Next
From: Stephen Frost
Date:
Subject: Re: Timely reporting of COPY errors