Thread: JVM crash when select count(*) on large table through JDBC
Hello,
I have a curious problem, which I would like to know whether somebody
can reproduce this: From a JVM using JDBC, I issue
select count(*) from big_table
where big table has more than 20 million entries (timestamps and
numbers). Upon PreparedStatement.executeQuery(), the VM quits, without
any message.
Java VM: 1.4.2-16 (latest one)
Pg-Jdbc: JDBC3 Postgresql Driver, Version 8.3-603
Server: PostgreSQL 8.1.11 (on Debian etch)
The most surprising thing is the lack of any error message (I had
expected a TimeOutException, or OutOfMemoryError, or the like), as if
someone had called System.exit().
The same commando, when executed through pgAdmin III, works.
As suggested on the website (http://jdbc.postgresql.org/lists.html), I
have searched the mailing lists, to no avail so far.
I would highly appreciate any reproduction of these results, or
suggestions for remedy.
Kind regards,
Kai Ruhl
On 22-Feb-08, at 11:25 AM, Kai Ruhl wrote: > Hello, > > I have a curious problem, which I would like to know whether somebody > can reproduce this: From a JVM using JDBC, I issue > > select count(*) from big_table > > where big table has more than 20 million entries (timestamps and > numbers). Upon PreparedStatement.executeQuery(), the VM quits, without > any message. > It's my understanding that regardless of what a java program is doing (aside from calling exit, or jni code) , it should never crash. I would say there is a bug in your jvm.
Just did a quick test with synthetic data (20M rows 3 columns: serial, timestamp, and text). And it worked ok both on Java (with driver postgresql-8.1-407.jdbc3.jar) and PgAdmin. Perhaps you could try to build a test case to expose the problem? On Fri, Feb 22, 2008 at 1:30 PM, Dave Cramer <pg@fastcrypt.com> wrote: > > On 22-Feb-08, at 11:25 AM, Kai Ruhl wrote: > > > Hello, > > > > I have a curious problem, which I would like to know whether somebody > > can reproduce this: From a JVM using JDBC, I issue > > > > select count(*) from big_table > > > > where big table has more than 20 million entries (timestamps and > > numbers). Upon PreparedStatement.executeQuery(), the VM quits, without > > any message. > > > It's my understanding that regardless of what a java program is doing > (aside from calling exit, or jni code) , it should never crash. > > I would say there is a bug in your jvm. > > > > ---------------------------(end of broadcast)--------------------------- > TIP 6: explain analyze is your friend >
On Fri, 22 Feb 2008, Andres Olarte wrote: > Just did a quick test with synthetic data (20M rows 3 columns: serial, > timestamp, and text). And it worked ok both on Java (with driver > postgresql-8.1-407.jdbc3.jar) and PgAdmin. > > Perhaps you could try to build a test case to expose the problem? From a client perspective, select count(*) should be the same no matter how many rows there are and what types of columns are in the table, so a test case won't be helpful. If the JVM is just up and dying, that's the JVM's fault not a pg issue. Kris Jurka
I know it's the same to count 1 o 20M rows, but it takes much longer to count 20M. I really don't know much about the internal of the driver, but maybe some kind of time out? On Fri, Feb 22, 2008 at 2:33 PM, Kris Jurka <books@ejurka.com> wrote: > > On Fri, 22 Feb 2008, Andres Olarte wrote: > > > Just did a quick test with synthetic data (20M rows 3 columns: serial, > > timestamp, and text). And it worked ok both on Java (with driver > > postgresql-8.1-407.jdbc3.jar) and PgAdmin. > > > > Perhaps you could try to build a test case to expose the problem? > > From a client perspective, select count(*) should be the same no matter > how many rows there are and what types of columns are in the table, so a > test case won't be helpful. If the JVM is just up and dying, that's the > JVM's fault not a pg issue. > > Kris Jurka >
As Kris said and I said. If your jvm crashes it's a jvm problem. I know people that have queries that take 20 minutes or more... Dave On 22-Feb-08, at 3:39 PM, Andres Olarte wrote: > I know it's the same to count 1 o 20M rows, but it takes much longer > to count 20M. I really don't know much about the internal of the > driver, but maybe some kind of time out? > > On Fri, Feb 22, 2008 at 2:33 PM, Kris Jurka <books@ejurka.com> wrote: >> >> On Fri, 22 Feb 2008, Andres Olarte wrote: >> >>> Just did a quick test with synthetic data (20M rows 3 columns: >>> serial, >>> timestamp, and text). And it worked ok both on Java (with driver >>> postgresql-8.1-407.jdbc3.jar) and PgAdmin. >>> >>> Perhaps you could try to build a test case to expose the problem? >> >> From a client perspective, select count(*) should be the same no >> matter >> how many rows there are and what types of columns are in the table, >> so a >> test case won't be helpful. If the JVM is just up and dying, >> that's the >> JVM's fault not a pg issue. >> >> Kris Jurka >> > > ---------------------------(end of > broadcast)--------------------------- > TIP 7: You can help support the PostgreSQL project by donating at > > http://www.postgresql.org/about/donate