Re: raw output from copy - Mailing list pgsql-hackers

From Pavel Stehule
Subject Re: raw output from copy
Date
Msg-id CAFj8pRC9BLiJ+5c0Zo_AaJTsvpioREEfv2vEeYq0GC=3zuHCqA@mail.gmail.com
Whole thread Raw
In response to Re: raw output from copy  ("Daniel Verite" <daniel@manitou-mail.org>)
Responses Re: raw output from copy  (Pavel Stehule <pavel.stehule@gmail.com>)
List pgsql-hackers


2016-03-04 15:54 GMT+01:00 Daniel Verite <daniel@manitou-mail.org>:
        Corey Huinker wrote:

> So, for me, RAW is the right solution, or at least *a* right solution.

Questions on how to extract from a bytea column come up on a regular
basis, as in [1] [2] [3], or [4] a few days ago, and so far the answers
are to encode the contents in text and decode them in an additional
step, or use COPY BINARY and filter out the headers.

But none of this is as straightforward and efficient as the proposed
COPY RAW.
Also the conversion to text can't be used at all on very large
contents (>512MB), as mentioned in another recent thread [5]
(this is the same reason why pg_dump can't dump such rows),
but COPY RAW doesn't have this limitation.

Technically COPY BINARY should be sufficient, but it seems that
people dislike having to deal with its headers.
Also it's not supported by any of the drivers of popular
script languages that otherwise provide COPY in text format
(DBD::Pg, php, psycopg2...)
Maybe the RAW format would have a better chance to get support
there, because of its simplicity.

exactly - I would to decrease dependency on PostgreSQL internals. Working with clean content is simple and possible with any environment without unclean operations.

Regards

Pavel
 

[1]
http://www.postgresql.org/message-id/038517CEB6DE43BD8422D7947B6BE8D8@fanliji
ng

[2] http://www.postgresql.org/message-id/4C8272C4.1000008@arcor.de

[3] http://stackoverflow.com/questions/6730729

[4] http://www.postgresql.org/message-id/56C66565.50107@consistentstate.com

[5] http://www.postgresql.org/message-id/14620.1456851036@sss.pgh.pa.us


Best regards,
--
Daniel Vérité
PostgreSQL-powered mailer: http://www.manitou-mail.org
Twitter: @DanielVerite

pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: psql completion for ids in multibyte string
Next
From: Robert Haas
Date:
Subject: Re: psql completion for ids in multibyte string