Re: New Copy Formats - avro/orc/parquet - Mailing list pgsql-general

From Adrian Klaver
Subject Re: New Copy Formats - avro/orc/parquet
Date
Msg-id 78a8f0bd-4b6f-b791-b6ce-80b19e2b6d1b@aklaver.com
Whole thread Raw
In response to New Copy Formats - avro/orc/parquet  (Nicolas Paris <niparisco@gmail.com>)
Responses Re: New Copy Formats - avro/orc/parquet  (Nicolas Paris <niparisco@gmail.com>)
List pgsql-general
On 02/10/2018 07:13 AM, Nicolas Paris wrote:
> Hello
> 
> I d'found useful to be able to import/export from postgres to those modern data
> formats:
> - avro (c writer=https://avro.apache.org/docs/1.8.2/api/c/index.html)
> - parquet (c++ writer=https://github.com/apache/parquet-cpp)
> - orc (all writers=https://github.com/apache/orc)
> 
> Something like :
> COPY table TO STDOUT ORC;
> 
> Would be lovely.
> 
> This would greatly enhance how postgres integrates in big-data ecosystem.
> 
> Any thought ?

https://www.postgresql.org/docs/10/static/sql-copy.html

"PROGRAM

     A command to execute. In COPY FROM, the input is read from standard 
output of the command, and in COPY TO, the output is written to the 
standard input of the command.

     Note that the command is invoked by the shell, so if you need to 
pass any arguments to shell command that come from an untrusted source, 
you must be careful to strip or escape any special characters that might 
have a special meaning for the shell. For security reasons, it is best 
to use a fixed command string, or at least avoid passing any user input 
in it.
"

> 
> Thanks
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com


pgsql-general by date:

Previous
From: Nicolas Paris
Date:
Subject: New Copy Formats - avro/orc/parquet
Next
From: Nicolas Paris
Date:
Subject: Re: New Copy Formats - avro/orc/parquet