Re: Make COPY extendable in order to support Parquet and other formats - Mailing list pgsql-hackers

From Aleksander Alekseev
Subject Re: Make COPY extendable in order to support Parquet and other formats
Date
Msg-id CAJ7c6TOvJE_kwBAoo7-6QT8PRvAgc2SrPfUQJOKAFTq_QeEiFw@mail.gmail.com
Whole thread Raw
In response to Re: Make COPY extendable in order to support Parquet and other formats  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Make COPY extendable in order to support Parquet and other formats
List pgsql-hackers
Andres, Tom,

> > I suspect that we'd first need a patch to refactor the existing copy code a
> > good bit to clean things up. After that it hopefully will be possible to plug
> > in a new format without being too intrusive.
>
> I think that step 1 ought to be to convert the existing formats into
> plug-ins, and demonstrate that there's no significant loss of performance.

Yep, this looks like a promising strategy to me too.

> I know little about parquet - can it support FROM STDIN efficiently?

Parquet is a compressed binary format with data grouped by columns
[1]. I wouldn't assume that this is a primary use case for this
particular format.

[1]: https://parquet.apache.org/docs/file-format/

-- 
Best regards,
Aleksander Alekseev



pgsql-hackers by date:

Previous
From: "Drouvot, Bertrand"
Date:
Subject: Re: Missing reference to pgstat_replslot.c in pgstat.c
Next
From: Amit Kapila
Date:
Subject: Re: Perform streaming logical transactions by background workers and parallel apply