Re: Proposal to add --single-row to psql - Mailing list pgsql-hackers

From Pavel Stehule
Subject Re: Proposal to add --single-row to psql
Date
Msg-id CAFj8pRD395WOcqx7sieAKVbbNP_G1tkY84SJ7rdCWC1A2KkwYA@mail.gmail.com
Whole thread Raw
In response to Proposal to add --single-row to psql  (Christopher Manning <c@christophermanning.org>)
List pgsql-hackers
Hello

It is redundant to current FETCH_COUNT implementation, so I don't see sense to use it together. Maybe we can drop FETCH_COUNT and replace it by --single-row mode and probably it can simplify code.

Regards

Pavel



2013/4/23 Christopher Manning <c@christophermanning.org>
psql currently collects the query result rows in memory before writing them to a file and can cause out of memory problems for large results in low memory environments like ec2. I can't use COPY TO STDOUT or FETCH_COUNT since I'm using Redshift and it doesn't support [writing to STDOUT](http://docs.aws.amazon.com/redshift/latest/dg/r_COPY.html) or [CURSOR](https://forums.aws.amazon.com/thread.jspa?threadID=122664).

[Single Row Mode is available in Postgres 9.2](http://www.postgresql.org/docs/9.2/static/libpq-single-row-mode.html) but [it doesn't look like](http://www.postgresql.org/docs/9.2/static/app-psql.html) you can tell psql to use single row mode when writing to a file (using --output).

I'm proposing to add a --single-row option to psql that would allow the result rows of a query to be streamed to a file without collecting them in memory first. 

I'm new to the postgres source, but I was considering doing this by adding an elseif at [this line in bin/psql/common.c](https://github.com/postgres/postgres/blob/master/src/bin/psql/common.c#L955) that would call [PQsetSingleRowMode](https://github.com/postgres/postgres/blob/master/src/interfaces/libpq/fe-exec.c#L1581) and ideally use something very similar to [ExecQueryUsingCursor](https://github.com/postgres/postgres/blob/master/src/bin/psql/common.c#L1081)

Please let me know if that would be an acceptable addition and if there's anything in particular I should be aware of when adding the feature.

Thank you,
Christopher


pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: putting a bgworker to rest
Next
From: Tom Lane
Date:
Subject: Re: Proposal to add --single-row to psql