Re: [GENERAL] COPY: row is too big - Mailing list pgsql-general

From Steve Crawford
Subject Re: [GENERAL] COPY: row is too big
Date
Msg-id CAEfWYyy5WXU__oRf0iMWthpRWZdJRNW2kt9v3da1BswtFKFP1Q@mail.gmail.com
Whole thread Raw
In response to Re: [GENERAL] COPY: row is too big  (Adrian Klaver <adrian.klaver@aklaver.com>)
Responses Re: [GENERAL] COPY: row is too big  (Adrian Klaver <adrian.klaver@aklaver.com>)
List pgsql-general
... 
Numeric is expensive type - try to use float instead, maybe double.

If I am following the OP correctly the table itself has all the columns declared as varchar. The data in the CSV file is a mix of text, date and numeric, presumably cast to text on entry into the table.

But a CSV *is* purely text - no casting to text is needed. Conversion is only needed when the strings in the CSV are text representations of *non*-text data.

I'm guessing that the OP is using all text fields to deal with possibly flawed input data and then validating and migrating the data in subsequent steps. In that case, an ETL solution may be a better approach. Many options, both open- closed- and hybrid-source exist.

Cheers,
Steve

pgsql-general by date:

Previous
From: Adrian Klaver
Date:
Subject: Re: [GENERAL] COPY: row is too big
Next
From: Adrian Klaver
Date:
Subject: Re: [GENERAL] COPY: row is too big