Re: BUG #1756: PQexec eats huge amounts of memory - Mailing list pgsql-bugs

From Tom Lane
Subject Re: BUG #1756: PQexec eats huge amounts of memory
Date
Msg-id 16737.1121718340@sss.pgh.pa.us
Whole thread Raw
In response to BUG #1756: PQexec eats huge amounts of memory  ("Denis Vlasenko" <vda@ilport.com.ua>)
List pgsql-bugs
"Kevin Grittner" <Kevin.Grittner@wicourts.gov> writes:
> Are you saying that if I execute a statement like:
> select * from "TableWithHundredsOfMillionsOfRows"

> that the entire table will be copied to a result set before returning
> the first row?

libpq does things that way, yes.

> Is this result set built in RAM on the server side?

No.

> I am interested primarily in the behavior under JDBC, although psql is
> also significant.

JDBC does things its own way; I believe it is possible to fetch a
resultset in a streaming fashion in JDBC, but you'd have to ask them.

The point of this discussion is simply that the current libpq API does
not permit streaming resultsets.  This is not a limitation of the
server, nor of the protocol, nor even really of libpq itself if you
were willing to invent a suitable new set of API definitions for a
streaming-resultset interface.  But we aren't going to change the
existing API to do streaming behind the client's back, because it won't
be transparent.

            regards, tom lane

pgsql-bugs by date:

Previous
From: "Kevin Grittner"
Date:
Subject: Re: BUG #1756: PQexec eats huge amounts of memory
Next
From: "Dirk Jagdmann"
Date:
Subject: BUG #1774: ecpg preprocessor produces a wrong varchar struct