"Guillaume Smet" <guillaume.smet@gmail.com> writes:
> And the most important point IMHO is that we must be aware of the
> trade-offs we make. We might have some cases where the CPU trade-off
> is not worth the I/O improvement (and probably the other case too).
> We really need a test framework to be able to perform daily benchmarks
> in various situations through the whole release cycle.
Well the problem is that what benchmark you choose dictates what areas you
need to concentrate on. The industy-standard benchmark so far has been TPC-C
which is only really concerned with I/O. Published benchmarks typically have
2-4 processors and hundreds of drives...
In the future TPC-E will load up a few more CPU resource hogs like relational
integrity checks, but even there it's fundamentally going to be a disk-bound
benchmark.
> I currently have compiled a version per month from january to now to
> perform my own tests (mostly CPU bound). If anyone wants me to perform
> specific pgbench load (I know it's not perfect but it's the most
> convenient tool we have currently), ping me. The box is only a Core2
> duo box with 2 GB of RAM and a SATA disk. So it's quite easy to be I/O
> bound :).
It would be nice to have infrastructure similar to the buldfarm running a
standard set of benchmarks every day. It would be fascinating to see the
graphs day-by-day of performance. Hopefully we wouldn't see too many dips and
just a steady increase over time.
-- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's On-Demand Production
Tuning
