2) Tune ext3. The default configuration wrecks high-write situations. Look into data=writeback for mounting, turning off atime (I hope you've done this already) updates, and also modifying the scheduler to the elevator model. This is poorly documented in Linux (like just about everything), but it's crtical.
I'm using noatime and data=writeback already. I changed my scheduler from the default anticipatory to deadline and saw an improvement, but I haven't yet delved into playing with specific elevator tunable values per-device.
3) Use 8.1 and strongly look at Bizgres. The data partitioning is critical.
I've just started down the path of getting 8.1 running with a larger block size, and tommorow I'm going to look at Bizgres's partitioning as opposed to some manual schemes. Will the current Bizgres have a lot of the performance enhancements of 8.1 already (or will 8.1 or 8.2 eventually get Bizgres's partitioning?)?
