Re: C++, Postgres , libpqxx huge query - Mailing list pgsql-general

From Tom Lane
Subject Re: C++, Postgres , libpqxx huge query
Date
Msg-id 23853.1399214050@sss.pgh.pa.us
Whole thread Raw
In response to Re: C++, Postgres , libpqxx huge query  (Andres Freund <andres@2ndquadrant.com>)
Responses Re: C++, Postgres , libpqxx huge query  (alexandros_e <alexandros.ef@gmail.com>)
Re: C++, Postgres , libpqxx huge query  (Alexandros Efentakis <efentakis@imis.athena-innovation.gr>)
List pgsql-general
Andres Freund <andres@2ndquadrant.com> writes:
> On 2014-05-04 01:57:43 -0700, alexandros_e wrote:
>> I have to execute an SQL query to Postgres by the following code. The query
>> returns a huge number of rows (40M or more) and has 4 integer fields: When I
>> use a workstation with 32Gb everything works but on a 16Gb workstation the
>> query is very slow (due to swapping I guess). Is there any way to tell the
>> C++ to load rows at batches, without waiting the entire dataset? With Java I
>> never had these issues before, due to the probably better JDBC driver.

> Try looking into either using a serverside cursor or COPY.

Another possibility is libpq's recently-introduced row-at-a-time mode:

http://www.postgresql.org/docs/9.3/static/libpq-single-row-mode.html

though I'm not sure how effectively that's supported by libpqxx.

            regards, tom lane


pgsql-general by date:

Previous
From: Adrian Klaver
Date:
Subject: Re: Server continuously enters to recovery mode.
Next
From: Glen Eustace
Date:
Subject: Monitoring Pg servers with Microsoft SCOM