Re: Streaming LIBPQ? Sliding Window in LIBPQ? - Mailing list pgsql-general

From Gregory Stark
Subject Re: Streaming LIBPQ? Sliding Window in LIBPQ?
Date
Msg-id 87oddefkzn.fsf@oxford.xeocode.com
Whole thread Raw
In response to Re: Streaming LIBPQ? Sliding Window in LIBPQ?  (Martijn van Oosterhout <kleptog@svana.org>)
Responses Re: Streaming LIBPQ? Sliding Window in LIBPQ?  (Martijn van Oosterhout <kleptog@svana.org>)
List pgsql-general
"Martijn van Oosterhout" <kleptog@svana.org> writes:

> On Wed, Nov 28, 2007 at 09:11:07AM -0600, Abraham, Danny wrote:
>> Is there a way to break the PGresult array to chuncks  Like Oracle?
>>
>> I mean, without changing the text of given queries,
>>
>> can we  somehow limit the memory consumption of a client that is using
>> LIBPQ?
>
> Use non-blocking mode to get the data in chunks. You can't easy control
> the number of rows you get each time though...

This doesn't really help. It's nonblocking but you still can't get libpq to
actually give you a result data structure until the entire results have
accumulated.

The only interface like this libpq supports is to use cursors in your SQL and
then FETCH n for each chunk. There's no libpq support for receiving results
incrementally.

If you're writing a driver implementing the protocol from scratch you could
expose chunks of results to the application but there's no protocol-level
support for it so you can't directly control the rate at which results arrive
or the chunk size or anything like that.

--
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com
  Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL training!

pgsql-general by date:

Previous
From: Tom Lane
Date:
Subject: Re: [Re] Re: [Re] Re: Unknown winsock error 10061while dumping a big database
Next
From: Chad Hendren
Date:
Subject: PostgresSQL vs. Informix