Re: export to parquet - Mailing list pgsql-general

From Chris Travers
Subject Re: export to parquet
Date
Msg-id CAKt_ZfvxgX4ZO5r0hWrRzP9b+X9cnVVshk_CZ=4kX-y5AG5Wew@mail.gmail.com
Whole thread Raw
In response to export to parquet  (Scott Ribe <scott_ribe@elevated-dev.com>)
Responses Re: export to parquet
List pgsql-general


On Wed, Aug 26, 2020 at 9:00 PM Scott Ribe <scott_ribe@elevated-dev.com> wrote:
I have no Hadoop, no HDFS. Just looking for the easiest way to export some PG tables into Parquet format for testing--need to determine what kind of space reduction we can get before deciding whether to look into it more.

Any suggestions on particular tools? (PG 12, Linux)

For simple exporting, the simplest thing is a single-node instance of Spark.

You can read parquet files in Postgres using https://github.com/adjust/parquet_fdw if you so desire but it does not support writing as parquet files are basically immutable.
 

--
Scott Ribe
scott_ribe@elevated-dev.com
https://www.linkedin.com/in/scottribe/







--
Best Wishes,
Chris Travers

Efficito:  Hosted Accounting and ERP.  Robust and Flexible.  No vendor lock-in.

pgsql-general by date:

Previous
From: Scott Ribe
Date:
Subject: export to parquet
Next
From: Scott Ribe
Date:
Subject: Re: export to parquet