Re: Load a csv or a avro? - Mailing list pgsql-general

From Kashif Zeeshan
Subject Re: Load a csv or a avro?
Date
Msg-id CAAPsdhd=JfvVaNMnJ1eUAuBts3664Q0r8JE-OBijvEuTQO4P3w@mail.gmail.com
Whole thread Raw
In response to Load a csv or a avro?  (sud <suds1434@gmail.com>)
Responses Re: Load a csv or a avro?
List pgsql-general
Hi

There are different data formats available, following are few points for there performance implications

1. CSV : It's easy to use and widely supported but it can be slower due to parsing overload.
2. Binary : Its faster to load but not human understandable.

Hope this helps.

Regards
Kashif Zeeshan

On Fri, Jul 5, 2024 at 2:08 PM sud <suds1434@gmail.com> wrote:

Hello all, 

Its postgres database. We have option of getting files in csv and/or in avro format messages from another system to load it into our postgres database. The volume will be 300million messages per day across many files in batches.

My question was, which format should we chose in regards to faster data loading performance ? and if any other aspects to it also should be considered apart from just loading performance?

pgsql-general by date:

Previous
From: sud
Date:
Subject: Load a csv or a avro?
Next
From: Josef Šimánek
Date:
Subject: Re: Load a csv or a avro?