Re: performance for high-volume log insertion - Mailing list pgsql-performance

From Kris Jurka
Subject Re: performance for high-volume log insertion
Date
Msg-id Pine.BSO.4.64.0904261301200.19948@leary.csoft.net
Whole thread Raw
In response to Re: performance for high-volume log insertion  (Thomas Kellerer <spam_eater@gmx.net>)
Responses Re: performance for high-volume log insertion
Re: performance for high-volume log insertion
List pgsql-performance

On Thu, 23 Apr 2009, Thomas Kellerer wrote:

> Out of curiosity I did some tests through JDBC.
>
> Using a single-column (integer) table, re-using a prepared statement
> took about 7 seconds to insert 100000 rows with JDBC's batch interface
> and a batch size of 1000
>

As a note for non-JDBC users, the JDBC driver's batch interface allows
executing multiple statements in a single network roundtrip.  This is
something you can't get in libpq, so beware of this for comparison's sake.

> I also played around with batch size. Going beyond 200 didn't make a big
> difference.
>

Despite the size of the batch passed to the JDBC driver, the driver breaks
it up into internal sub-batch sizes of 256 to send to the server.  It does
this to avoid network deadlocks from sending too much data to the server
without reading any in return.  If the driver was written differently it
could handle this better and send the full batch size, but at the moment
that's not possible and we're hoping the gains beyond this size aren't too
large.

Kris Jurka

pgsql-performance by date:

Previous
From: Adam Ruth
Date:
Subject:
Next
From: Thomas
Date:
Subject: Re: performance for high-volume log insertion