Re: COPY and file_fdw with fixed column widths - Mailing list pgsql-hackers

From Tom Lane
Subject Re: COPY and file_fdw with fixed column widths
Date
Msg-id 34330.1430250382@sss.pgh.pa.us
Whole thread Raw
In response to COPY and file_fdw with fixed column widths  (Bruce Momjian <bruce@momjian.us>)
Responses Re: COPY and file_fdw with fixed column widths
List pgsql-hackers
Bruce Momjian <bruce@momjian.us> writes:
> I know COPY doesn't support importing files with fixed column widths,
> i.e. we can't say field1 is the first 30 characters, and field2 is the
> rest of the line.  We need a unique delimiter at column 31.  (Commercial
> Ingres does support this ability.)

> I know we tell most people to use sed, Perl, or an ETL tool to convert
> files into a format COPY understands, and I think that is a reasonable
> answer.  However, the file_fdw also reads our COPY format, and in that
> case, the data file might be updated regularly and running an ETL
> process on it every time it is read is inconvenient.

COPY is, and has always been intended to be, as fast as possible; loading
format transformation abilities onto it seems like a fundamental mistake.
Therefore, if you wish file_fdw were more flexible, I think the answer is
to create a variant of file_fdw that doesn't use COPY but some other
mechanism.
        regards, tom lane



pgsql-hackers by date:

Previous
From: Merlin Moncure
Date:
Subject: Re: cache invalidation for PL/pgsql functions
Next
From: Bruce Momjian
Date:
Subject: Re: COPY and file_fdw with fixed column widths