Re: Re: new set of psql patches for loading (saving) datafrom (to) text, binary files - Mailing list pgsql-hackers

From Andres Freund
Subject Re: Re: new set of psql patches for loading (saving) datafrom (to) text, binary files
Date
Msg-id 20170406010242.gmbvfotggbcx45pr@alap3.anarazel.de
Whole thread Raw
In response to [HACKERS] Re: new set of psql patches for loading (saving) data from (to) text,binary files  (Pavel Stehule <pavel.stehule@gmail.com>)
Responses Re: Re: new set of psql patches for loading (saving) datafrom (to) text, binary files  (Stephen Frost <sfrost@snowman.net>)
List pgsql-hackers
On 2017-03-18 17:51:48 +0100, Pavel Stehule wrote:
> What is done:
> 
> create table foo foo(a bytea);
> 
> -- import
> insert into foo values($1)
> \gloadfrom ~/xxx.jpg bytea
> 
> -- export
> \pset format binary
> select a from foo
> \g ~/xxx2.jpg
> 
> tested on import 55MB binary file
> 
> Comments, notes?

I don't like the API here much.  Loading requires knowledge of some
magic $1 value and allows only a single column, printing doesn't mean
much when there's multiple columns / rows.

I think the loading side of things should be redesigned into a more
general facility for providing query parameters.  E.g. something like
\setparam $1 'whateva'
\setparamfromfile $2 'somefile'
\setparamfromprogram $3 cat /frakbar

which then would get used in the next query sent to the server.  That'd
allow importing multiple columns, and it'd be useful for other purposes
than just loading binary data.


I don't yet have a good idea how to deal with moving individual cells
into files, so they can be loaded.  One approach would be to to have
something like

\storequeryresult filename_template.%row.%column

which'd then print the current query buffer into the relevant file after
doing replacement on %row and %column.

I don't think we can find an API we agree upon in the next 48h...

- Andres



pgsql-hackers by date:

Previous
From: Tatsuo Ishii
Date:
Subject: Re: pgbench - allow to store select results intovariables
Next
From: David Rowley
Date:
Subject: Re: [COMMITTERS] pgsql: Collect and use multi-columndependency stats