Re: Synchronisation problems in COPY IN - Mailing list pgsql-jdbc
From | Maciek Sakrejda |
---|---|
Subject | Re: Synchronisation problems in COPY IN |
Date | |
Msg-id | AANLkTinJrszRRTAvbaKKD2MZY=H+hYM9FVdJA4BdYg-H@mail.gmail.com Whole thread Raw |
In response to | Re: Synchronisation problems in COPY IN (Kris Jurka <books@ejurka.com>) |
List | pgsql-jdbc |
There is some moderately bogus behavior with respect to closing a connection while doing a COPY (see list archives). I wonder if that could be involved? If memory serves, the issue was that a Terminate message (sent on Connection.close() to indicate that the client wants to end the protocol connection) is *not* synchronized relative to COPY protocol operations. This means that *theoretically*, your Terminate message could be shimmed right into the middle of a CopyData message. In practice, I've only ever seen an error from the server complaining about an unexpected message type (Terminate) during COPY, since the client is *supposed* to send CopyDone or CopyFail before a Terminate while a COPY is in effect. Still, I think it might be possible to get message contents munged. I think your code *might* be susceptible to that, since you kick off a bunch of COPY operations in separate threads and then call Connection.close() in the main thread, without waiting for the COPY threads to finish if I'm reading that right--should that if (t.isAlive) {} be something while (t.isAlive()) {}? Can you capture (e.g., tcpdump) the data stream that gets sent to the server? It'd be interesting to see if Terminate actually is getting munged into the COPY data or if we're looking at another issue entirely. For what it's worth, I investigated a fix, but (1) it's a pain in the ass to get the logic right without Java 5's synchronization primitives (in fact, if we do this again, it might be worth just implementing a ReadWriteLock unless JDBC is planning to drop 1.4 support) and (2) based on preliminary testing, there was an unacceptable performance regression in doing the locking right. I can try to revisit the patch (or dig it out and post it as is), but we may need to rethink this entirely. First of all, though, it'd be nice to figure out if this is actually the problem affecting you. If the problem I described is still just theoretical, I wouldn't worry about it. Since it involves the client closing the Connection while having active COPY operations in other threads, this is wonky *application* behavior any (well, almost any) way you look at it. --- Maciek Sakrejda | System Architect | Truviso 1065 E. Hillsdale Blvd., Suite 215 Foster City, CA 94404 (650) 242-3500 Main www.truviso.com On Mon, Sep 13, 2010 at 7:17 AM, Kris Jurka <books@ejurka.com> wrote: > > > On Mon, 13 Sep 2010, Kim Bisgaard wrote: > >> We are occasionally experiencing problems with our JDBC implementation of >> COPY IN. >> >> We have cut our program down to the attached example. > > Do you get the failure you've shown from this example? I get another > failure from the connection being close before the last copy thread can > complete its write operation. This seems like an expected failure, but I > can't reproduce yours. > > org.postgresql.util.PSQLException: Database connection failed when canceling > copy operation > at > org.postgresql.core.v3.QueryExecutorImpl.cancelCopy(QueryExecutorImpl.java:796) > at > org.postgresql.core.v3.CopyOperationImpl.cancelCopy(CopyOperationImpl.java:32) > at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:150) > at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:126) > at EvejrDBCopy.run(EvejrDBCopy.java:34) > >> We use PostgreSQL 8.4, and JDBC 8.4-702. >> > > Your stacktrace shows that you are using 8.4-701, not 702. 702 had some > fixes to the copy code when using a Reader, but your code shows you are > using an InputStream, so those changes shouldn't be relevent. > > Kris Jurka > > -- > Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-jdbc >
pgsql-jdbc by date: