Re: WIP patch: add (PRE|POST)PROCESSOR options to COPY - Mailing list pgsql-hackers

From Etsuro Fujita
Subject Re: WIP patch: add (PRE|POST)PROCESSOR options to COPY
Date
Msg-id 001001cd9610$343cb7d0$9cb62770$@lab.ntt.co.jp
Whole thread Raw
In response to Re: WIP patch: add (PRE|POST)PROCESSOR options to COPY  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: WIP patch: add (PRE|POST)PROCESSOR options to COPY  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
> From: Tom Lane [mailto:tgl@sss.pgh.pa.us]

> "Etsuro Fujita" <fujita.etsuro@lab.ntt.co.jp> writes:
> > Maybe my explanation was insufficient.  Let me add one thing to my earlier
> > explanation.  The submitted patch allows the psql \copy instruction to be
> > executed like:
> 
> > $ echo '/bin/gunzip -c $1' > decompress.sh
> > $ chmod +x decompress.sh
> 
> > postgres=# \copy foo FROM '/home/pgsql/foo.csv.gz' WITH (format 'csv',
> > preprocessor '/home/pgsql/decompress.sh')

> Well, in that case, you've got not only an explanation problem but a
> syntax problem, because that syntax is utterly misleading.  Anybody
> looking at it would think that the "format" option is one of the options
> being sent to the backend.  The code required to pull it out of there
> has got to be grossly overcomplicated (and likely bugprone), too.
> 
> I think it would be better to present this as something like
> 
> \copy foo from '/home/pgsql/decompress.sh /home/pgsql/foo.csv.gz |' with
> format 'csv'
> 
> which would cue any reasonably Unix-savvy person that what's happening
> is a popen on the client side.  It'd probably be a whole lot less
> complicated to implement, too.

Great!

I have a question.  I think it would be also better to extend the syntax for the
SQL COPY command in the same way, ie,

COPY foo from '/home/pgsql/decompress.sh /home/pgsql/foo.csv.gz |' with format
'csv'

Is this okay?

Thanks,

Best regards,
Etsuro Fujita




pgsql-hackers by date:

Previous
From: Bruce Momjian
Date:
Subject: Re: [ADMIN] pg_upgrade from 9.1.3 to 9.2 failed
Next
From: Tom Lane
Date:
Subject: Re: WIP patch: add (PRE|POST)PROCESSOR options to COPY