Thread: Performance of jdbc insert statements and select nextval
i found the following problem:
Summary:
10000 inserts with pgadmin tool into one table (primary key bigserial) can be done wihtin 3 seconds. If i perform the same job with jdbc this takes 13 seconds. The insert statement is prepared only once, the statement for fetching the nextval also. If i omit the select nextval execution time improves to 8 seconds. Can anybody tell me why jdbc is 3 to four times slower than pgadmin? I am using Postgresql 8.2 and 8.3 Database server, java 1.5 and 1.6 and i tried the drivers 8.2 and 8.3 in type 3 and type 4. The execution times are always identical.
Precise description:
The problem results from the following application problem. We are developing a web application with geronimo application server, JPA persistence technology and PostgreSQL database server. The application is tested on Ubuntu Linux and Windows XP, with postgresql 8.2 and 8.3 servers on java 1.5 and 1.6 platform. I made a test with "logical" insert of objects which results in 50000 inserts can be done within 2 minutes, what means 416 inserts per second, or 1 insert in 2/1000 sec (2ms). The cost estimate with explain for an insert is on the other hand is 0,4 ms. So, where is the gap? I left JPA and tried jdbc. Here i found out: 10000 inserts in sql script can be done by PGAdmin within 3 seconds, a simple jdbc loader program needs 13 seconds. Leaving the id generation to the database, this can be done within 8 sec (but in my jpa world i of course need the id, and there was a severe problem of leaving this task to the jpa model. The automatic id generation (which of course is supported by jpa) did not work in postgresql if the tables are in a schema).
This is my jdbc test program code:
Yet now i have tried to other database servers. MySql GA and Oracle XE (10g Express). Both servers lead to the same performance (10000 inserts within 13 seconds). So, PostgreSQL is not less or more performant than others. Following a hint from Graig Ringer. May it be that a firewall or virus scanner forces the performance slow down? This perhaps might explain why the system does not scale with faster hardware.
Thanks, Ralf
BGS Beratungsgesellschaft Software Systemplanung AG | ||||
Niederlassung Nord Ebertstra�e 21 26382 Wilhelmshaven Fon: +49 (0) 4421 / 9683-700 Fax: +49 (0) 4421 / 9683-790 www.bgs-ag.de | Gesch�ftssitz Mainz Registergericht Amtsgericht Mainz HRB 62 50 | Aufsichtsratsvorsitzender Klaus Hellwig Vorstand Hanspeter Gau Hermann Kiefer Nils Manegold |
Attachment
On Fri, 20 Feb 2009, ralf.baumhof@bgs-ag.de wrote: > [insert with pgadmin is faster than JDBC] As I explained previously I believe pgadmin is sending all of your inserts in one network roundtrip rather than a single insert at a time. So for example: StringBuffer sql = new StringBuffer(); for (int i=0; i<100; i++) { sql.append("INSERT INTO mytable (a) VALUES ("); sql.append(i).append(")"); } Statement.execute(sql.toString()) will be faster than for (int i=0; i<100; i++) { String sql = "INSERT INTO mytable (a) VALUES (" + i + ")"; Statement.execute(sql); } because the second sample has to make 100 network trips instead of just one. > I made a test with "logical" insert of objects which results in 50000 > inserts can be done within 2 minutes, what means 416 inserts per second, > or 1 insert in 2/1000 sec (2ms). The cost estimate with explain for an > insert is on the other hand is 0,4 ms. Explain cost estimates do not have units of milliseconds, so the comparison is invalid. Kris Jurka
thanks for your hint.
i have tested this, and you'r right. Packaging my requests into chunks of 1000 each, i can execute 10000 inserts within 2 seconds. This is exactly the performance of PGAdmin Tool and 5 times faster then performing an execute per each object. By the way, when i was talking about explain i was thinking of explain analyse and this statement produces a total runtime in milliseconds. So the execution time of 0,2 ms per insert is close to the cost estimate of explain (analyse).
Thanks for your help,
Ralf
Kris Jurka <books@ejurka.com> Gesendet von: pgsql-jdbc-owner@postgresql.org 20.02.2009 18:59 |
|
On Fri, 20 Feb 2009, ralf.baumhof@bgs-ag.de wrote:
> [insert with pgadmin is faster than JDBC]
As I explained previously I believe pgadmin is sending all of your inserts
in one network roundtrip rather than a single insert at a time. So for
example:
StringBuffer sql = new StringBuffer();
for (int i=0; i<100; i++) {
sql.append("INSERT INTO mytable (a) VALUES (");
sql.append(i).append(")");
}
Statement.execute(sql.toString())
will be faster than
for (int i=0; i<100; i++) {
String sql = "INSERT INTO mytable (a) VALUES (" + i + ")";
Statement.execute(sql);
}
because the second sample has to make 100 network trips instead of just
one.
> I made a test with "logical" insert of objects which results in 50000
> inserts can be done within 2 minutes, what means 416 inserts per second,
> or 1 insert in 2/1000 sec (2ms). The cost estimate with explain for an
> insert is on the other hand is 0,4 ms.
Explain cost estimates do not have units of milliseconds, so the
comparison is invalid.
Kris Jurka
--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc
BGS Beratungsgesellschaft Software Systemplanung AG | ||||
Niederlassung Nord Ebertstra�e 21 26382 Wilhelmshaven Fon: +49 (0) 4421 / 9683-700 Fax: +49 (0) 4421 / 9683-790 www.bgs-ag.de | Gesch�ftssitz Mainz Registergericht Amtsgericht Mainz HRB 62 50 | Aufsichtsratsvorsitzender Klaus Hellwig Vorstand Hanspeter Gau Hermann Kiefer Nils Manegold |