Avro files are smaller due to compression so needing less I/O time. whereas CSV files are simpler but larger in size so read/write will need more time.
COPY command works very well with CSV files whereas ETL process is required for handling Avro.
pá 5. 7. 2024 v 11:08 odesílatel sud <suds1434@gmail.com> napsal: > > Hello all, > > Its postgres database. We have option of getting files in csv and/or in avro format messages from another system to load it into our postgres database. The volume will be 300million messages per day across many files in batches. > > My question was, which format should we chose in regards to faster data loading performance ? and if any other aspects to it also should be considered apart from just loading performance?
