Re: WIP patch: add (PRE|POST)PROCESSOR options to COPY - Mailing list pgsql-hackers

From Tom Lane
Subject Re: WIP patch: add (PRE|POST)PROCESSOR options to COPY
Date
Msg-id 14768.1347976777@sss.pgh.pa.us
Whole thread Raw
In response to Re: WIP patch: add (PRE|POST)PROCESSOR options to COPY  ("Etsuro Fujita" <fujita.etsuro@lab.ntt.co.jp>)
Responses Re: WIP patch: add (PRE|POST)PROCESSOR options to COPY  ("Etsuro Fujita" <fujita.etsuro@lab.ntt.co.jp>)
List pgsql-hackers
"Etsuro Fujita" <fujita.etsuro@lab.ntt.co.jp> writes:
> Maybe my explanation was insufficient.  Let me add one thing to my earlier
> explanation.  The submitted patch allows the psql \copy instruction to be
> executed like:

> $ echo '/bin/gunzip -c $1' > decompress.sh
> $ chmod +x decompress.sh

> postgres=# \copy foo FROM '/home/pgsql/foo.csv.gz' WITH (format 'csv',
> preprocessor '/home/pgsql/decompress.sh')

> In this example, command "/home/pgsql/decompress.sh /home/pgsql/foo.csv.gz" is
> executed on client side, by using popen(), and command "COPY foo FROM STDIN WITH
> (format 'csv')" is sent to backend.  I apologize for not providing you with
> enough explanation.

Well, in that case, you've got not only an explanation problem but a
syntax problem, because that syntax is utterly misleading.  Anybody
looking at it would think that the "format" option is one of the options
being sent to the backend.  The code required to pull it out of there
has got to be grossly overcomplicated (and likely bugprone), too.

I think it would be better to present this as something like

\copy foo from '/home/pgsql/decompress.sh /home/pgsql/foo.csv.gz |' with format 'csv'

which would cue any reasonably Unix-savvy person that what's happening
is a popen on the client side.  It'd probably be a whole lot less
complicated to implement, too.
        regards, tom lane



pgsql-hackers by date:

Previous
From: Amit kapila
Date:
Subject: Re: Proof of concept: auto updatable views [Review of Patch]
Next
From: Marco Nenciarini
Date:
Subject: Re: [PATCH] Support for Array ELEMENT Foreign Keys