Re: Test lab - Mailing list pgsql-hackers

Here are a couple of additions to the performance test lab discussion.   I hope you will find these useful.  

1.) Test tools.  The Bristlecone testing package I presented at the PG Fall 2007 Conference is now available at http://bristlecone.continuent.org.  There are two main tools:  Evaluator and Benchmark.  Evaluator generates a CPU-intensive mixed load.  Benchmark generates very specific loads with systematically varying parameters.  I have been using bristlecone to do a lot of testing of MySQL and PostgreSQL, since we have middleware that runs on both.  I plan to follow Josh's request and run some of the current benchmarks to compare 8.2.5 vs. 8.3 performance.   So far most of my tests have compared MySQL and PostgreSQL vs. our middleware but I recently started to compare the databases directly.   One initial result:  MySQL appears to be much faster at streaming very large result sets.  

2.) Test hardware.  We have a number of hosts in Grenoble, France that are available to help set up a European lab.    We gave away 4 to the postgresql.fr folks but if there's anyone else within driving (or trucking distance) we still have at least a dozen 1U rack mountable Compaq units.  They are in a garage and winter will soon be upon the Alps, so we need to try to unload them.   Unluckily we overbought hardware in this location but with luck this can be someone else's good fortune.   It probably won't help in the US of A due to shipping costs. 

Please look at Bristlecone.  It's very early on but I have found these tools to be exceedingly useful.   Among other things it should be possible to add features that allow us to do regression testing on performance,  something that is a pain for "normal" test frameworks.   

Cheers, Robert