Re: Java Out-of-memory errors on attempts to read tables with millions of rows

From: Evil Azrael
Subject: Re: Java Out-of-memory errors on attempts to read tables with millions of rows
Date: ,
Msg-id: 71226588.20030714190744@evilazrael.de
(view: Whole thread, Raw)
In response to: Java Out-of-memory errors on attempts to read tables with millions of rows  (Rich Cullingford)
List: pgsql-performance

I think you want to use a Cursor for browsing the data.

Christoph Nelles


Am Montag, 14. Juli 2003 um 18:53 schrieben Sie:

RC> Greetings,
RC> We have several tables (in a PG 7.3.3 database on RH Linux 7.3) with 2M+
RC> rows (each row 300-400 bytes in length) that we SELECT into a JDBC
RC> ResultSet for display to the user. We expected that the driver would not
RC> actually transmit data from the database until the application began
RC> issuing getXXX() calls. (IIRC, this is the way the Oracle driver works,
RC> and we had created a buffering mechanism to use it.) Instead, the driver
RC> appears to be attempting to create the whole rowset in Java memory
RC> before returning, and the application runs out of memory. (Java has been
RC> configured to use up to 1.5G on the machine this occurs on.)

RC> Now the SELECT is preceded by a COUNT of the rows that the same query
RC> would return, so perhaps that's what's causing the problem. But the
RC> question is, is this the way a ResultSet is supposed to work? Are there
RC> any configuration options available that modify this behavior? Are there
RC> commercial implementations of PG JDBC that don't have this problem?
RC> (Shame on me, but I have to ask. :)

RC> Any help will be greatly appreciated!

RC>                                  Rich Cullingford
RC>                                  


RC> ---------------------------(end of broadcast)---------------------------
RC> TIP 9: the planner will ignore your desire to choose an index scan if your
RC>       joining column's datatypes do not match



--
Mit freundlichen Grüssen
Evil Azrael                            mailto:



pgsql-performance by date:

From: Tom Lane
Date:
Subject: Re: optimizer picks smaller table to drive nested loops?
From: "Shridhar Daithankar"
Date:
Subject: Re: Sanity check requested