Re: New Copy Formats - avro/orc/parquet - Mailing list pgsql-general

From Tomas Vondra
Subject Re: New Copy Formats - avro/orc/parquet
Date
Msg-id fd8c103f-8448-775f-39d5-d69c8aaf0560@2ndquadrant.com
Whole thread Raw
In response to Re: New Copy Formats - avro/orc/parquet  ("David G. Johnston" <david.g.johnston@gmail.com>)
List pgsql-general
On 02/10/2018 04:38 PM, David G. Johnston wrote:
> On Saturday, February 10, 2018, Nicolas Paris <niparisco@gmail.com
> <mailto:niparisco@gmail.com>> wrote:
> 
>     Hello
> 
>     I d'found useful to be able to import/export from postgres to those
>     modern data
>     formats:
>     - avro (c writer=https://avro.apache.org/docs/1.8.2/api/c/index.html
>     <https://avro.apache.org/docs/1.8.2/api/c/index.html>)
>     - parquet (c++ writer=https://github.com/apache/parquet-cpp
>     <https://github.com/apache/parquet-cpp>)
>     - orc (all writers=https://github.com/apache/orc
>     <https://github.com/apache/orc>)
> 
>     Something like :
>     COPY table TO STDOUT ORC;
> 
>     Would be lovely.
> 
>     This would greatly enhance how postgres integrates in big-data
>     ecosystem.
> 
>  
> It would be written "... with (format 'orc')" and your best bet would be
> to create an extension.  I don't think that having such code in core (or
> contrib) is desirable.
> 

I don't think make this extensible by an extension (i.e. the formats
supported by COPY are hard-coded in core). But I agree that if we are to
add multiple new formats, it'd be nice to allow doing that in extension.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


pgsql-general by date:

Previous
From: "David G. Johnston"
Date:
Subject: Re: New Copy Formats - avro/orc/parquet
Next
From: Tomas Vondra
Date:
Subject: Re: New Copy Formats - avro/orc/parquet