Performance tuning for copy in - Mailing list pgsql-admin

From Joe Maldonado
Subject Performance tuning for copy in
Date
Msg-id 1081954461.3941.17.camel@joem
Whole thread Raw
List pgsql-admin
Hello,
Currently I can get no more than 40k records/second to be copied
in to postgres.  This is also the case when running multiple
cocurrent copy in commands from different connections.  It
seems linear in that 1 process will copy in at 40k/s and 2
will copy in at about 20k/s, etc.

I need to know whether I can get better performance from
the db with the hardware specified below.  Changing from
mirrored to stripe RAID is not an option :( at this time
as is turning off fsync (which did not work much better).


-Joe Maldonado

Configuration:
PostgreSQL 7.4
configure --libdir=/usr/lib \
          --includedir=/usr/include \
          --bindir=/usr/bin \
          --with-krb5=/usr/kerberos \
          --with-openssl=/usr \
          --enable-thread-safety \
          --enable-debug

OS: RedHat 9
Drive Config:
sdb1: /pg_clog
sdb2: /pg_xlog
sdc1: /pg_data (RAID 1)

hdparm output shows average of 45MB/s
iostat shows fairly constant 10MB/s written to pg_data
and 5-8MB/s to WAL.

CPU: 2x 2.0G Xeon



pgsql-admin by date:

Previous
From: Bruno Wolff III
Date:
Subject: Re: Real time replication of PG DBs accross two servers -
Next
From: Jodi Kanter
Date:
Subject: XTG Data Modeller