Re: Loading binary data into the database - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Loading binary data into the database
Date
Msg-id 25023.964107817@sss.pgh.pa.us
Whole thread Raw
In response to Re: Loading binary data into the database  (Adriaan Joubert <a.joubert@albourne.com>)
List pgsql-hackers
Adriaan Joubert <a.joubert@albourne.com> writes:
>> Such as?  IIRC, the reason we disabled it was precisely that there was
>> no support on the client side.  (What's worse, there's no support in
>> the FE/BE protocol either.  I don't see how you could have made this
>> work...)

> I issued a 'copy binary <table> from stdin;' and then sent the data with
> PQputnbytes

How did you get out of COPY state?  In binary mode CopyFrom will only
recognize EOF as end of data, and there's no provision in the FE/BE
protocol for making it see an EOF.  You'd have had to break the
connection to get out of that --- and I'd have expected the loss of
connection to cause a transaction abort, preventing your data from
getting committed.  (If it didn't abort, that's a bug that needs to be
fixed... if the line drops halfway through a copy, you don't want it
to commit do you?)

The real bottom line here is that the FE/BE protocol would need to be
changed to support binary copy properly, and no one's excited about
putting more work into the existing protocol, nor about the ensuing
compatibility problems.


> Perhaps libpq is not the answer. I've even been thinking about writing a
> SPI function that acts as a CORBA server -- but decided that that is just
> too ugly to contemplate. So what is the solution?

A CORBA-based replacement protocol has been discussed seriously, though
I haven't noticed any work getting done on it lately.  Feel free to
pitch in if you think it's a good idea.
        regards, tom lane


pgsql-hackers by date:

Previous
From: Larry Rosenman
Date:
Subject: Stupid Me...
Next
From: Tom Lane
Date:
Subject: Re: About TOAST and indices