On 01/15/2011 07:41 PM, Andrew Dunstan wrote:
>
>
> On 01/15/2011 12:29 PM, Andrew Dunstan wrote:
>>
>> I've been waiting for the latest FDW patches as patiently as I can,
>> and I've been reviewing them this morning, in particular the file_fdw
>> patch and how it interacts with the newly exposed COPY API. Overall
>> it seems to be a whole lot cleaner, and the wholesale duplication of
>> the copy code is gone, so it's much nicer and cleaner. So now I'd
>> like to add a new option to it: "textarray". This option would
>> require that the foreign table have exactly one field, of type
>> text[], and would compose all the field strings read from the file
>> for each record into the array (however many there are). This would
>> require a few changes to contrib/file_fdw/file_fdw.c and a few
>> changes to src/backend/commands/copy.c, which I can probably have
>> done in fairly short order, Deo Volente. This will allow something like:
>>
>> CREATE FOREIGN TABLE arr_text (
>> t text[]
>> ) SERVER file_server
>> OPTIONS (format 'csv', filename '/path/to/ragged.csv', textarray
>> 'true');
>> SELECT t[3]::int as a, t[1]::timestamptz as b, t[99] as not_there
>> FROM arr_text;
>>
>>
>
> A WIP patch is attached. It's against Shigeru Hanada's latest FDW
> patches. It's surprisingly tiny. Right now it probably leaks memory
> like a sieve, and that's the next thing I'm going to chase down.
>
>
Updated patch attached, that should use memory better.
cheers
andrew
