Re: Re: new set of psql patches for loading (saving) datafrom (to) text, binary files - Mailing list pgsql-hackers

From Andres Freund
Subject Re: Re: new set of psql patches for loading (saving) datafrom (to) text, binary files
Date
Msg-id 20170406011406.wofxewbn2fhksx4r@alap3.anarazel.de
Whole thread Raw
In response to [HACKERS] new set of psql patches for loading (saving) data from (to) text,binary files  (Pavel Stehule <pavel.stehule@gmail.com>)
List pgsql-hackers
Hi,

On 2017-04-05 21:07:59 -0400, Stephen Frost wrote:
> * Andres Freund (andres@anarazel.de) wrote:
> > I don't like the API here much.  Loading requires knowledge of some
> > magic $1 value and allows only a single column, printing doesn't mean
> > much when there's multiple columns / rows.
> 
> > I think the loading side of things should be redesigned into a more
> > general facility for providing query parameters.  E.g. something like
> > \setparam $1 'whateva'
> > \setparamfromfile $2 'somefile'
> > \setparamfromprogram $3 cat /frakbar
> >
> > which then would get used in the next query sent to the server.  That'd
> > allow importing multiple columns, and it'd be useful for other purposes
> > than just loading binary data.
> 
> I tend to agree that the loading side should probably be thought through
> more.
> 
> > I don't yet have a good idea how to deal with moving individual cells
> > into files, so they can be loaded.  One approach would be to to have
> > something like
> > 
> > \storequeryresult filename_template.%row.%column
> > 
> > which'd then print the current query buffer into the relevant file after
> > doing replacement on %row and %column.
> 
> I don't actually agree that there's a problem having the output from a
> query stored direclty in binary form into a single file.  The above
> approach seems to imply that every binary result must go into an
> independent file, and perhaps that would be useful in some cases, but I
> don't see it as required.

Well, it'd not be enforced - it'd depend on your template.  But for a
lot of types of files, it'd not make sense to store multiple
columns/rows in file. Particularly for ones where printing them out to
files is actually meaningful (i.e. binary ones).

- Andres



pgsql-hackers by date:

Previous
From: David Rowley
Date:
Subject: Re: [COMMITTERS] pgsql: Collect and use multi-columndependency stats
Next
From: Robert Haas
Date:
Subject: Re: strange parallel query behavior after OOM crashes