Re: Is there a meaningful benchmark? - Mailing list pgsql-general

It isn't actually possible at this stage for me to benchmark "the
application" because it doesn't yet exist.  There are a number of
potential projects floating around, with as yet unwritten
specifications, to run on different platforms ranging from embedded to
larger servers.  People just want to hear what I think is a good RDBMS
to use.  My opinion won't necessarily be followed.

Nobody at this point is expecting the RDBMS to become a bottleneck, if
they are planning to actually use one at all.  However someone is sure
to ask the question, for an average application with an average
database, how is performance?

Even if such a question is answered, it isn't going to be the only
factor.  For example I have collected reasonable numbers already on
footprints of different RDBMSs, because embedded guys might find that
important if they're restricted to 64MB of flash.  On the other hand if
they went with some of the newer solid state drives with gigs of space,
then a few packages using 10s of MB wouldn't be such a problem any more.

In short, all bets are off and I'm just looking for baseline
information.  This is just a general feasibility and technology
exploration phase.

I'm aware of the limitations of hard numbers, but the more simple
information I have in different dimensions, the easier it is to convince
people not to lock in too early.

Thanks for the info, I'll check some of those references.

-Will


-----Original Message-----
From: Dann Corbit [mailto:DCorbit@connx.com]
Sent: 19 March 2009 17:16
To: Will Rutherdale (rutherw); pgsql-general@postgresql.org
Subject: RE: [GENERAL] Is there a meaningful benchmark?

The only way to get an answer to a question like this is to actually
benchmark the application you have in mind.
And the answer won't be very good unless you have an expert on each
given system install and tune the application.

There is a regular benchmark that is run against the PostgreSQL
database.  I don't remember where to find the graphs.  Probably, someone
on the list can tell us the location.


Here are some benchmark figures:
http://tweakers.net/reviews/657/6
http://www.spec.org/jAppServer2004/results/res2007q3/jAppServer2004-2007
0606-00065.html
http://www.informationweek.com/news/software/linux/showArticle.jhtml?art
icleID=201001901
http://www.kaltenbrunner.cc/blog/index.php?/archives/21-guid.html
http://benchw.sourceforge.net/benchw_results_open3.html


P.S.
PostgreSQL seems to scale pretty well:
http://www.computerworld.com/action/article.do?command=viewArticleBasic&
taxonomyId=18&articleId=9087918&intsrc=hm_topic


My opinion:
Most benchmarks are run by someone with an axe to grind.  I never
believe them.  The TPC benchmarks are probably the most trustworthy,
because they have to be certified.  But a fast TPC/whatever benchmark is
no guarantee that *your* application will run fast.  So if you want to
evaluation several different technologies do your own benchmark.  Do
your own calculations to find out the total cost of ownership over the
lifetime of the project.  Examine all the features that are available,
and what kind of technical support is possible.  Consider the impact of
product licensing.  What happens if you need to scale up to titanic
volume?  After you have thought all factors over very carefully, make
your choice.

If you rely on someone else to do the work for you, it's really begging
for trouble.  MySQL guys will show you why MySQL is faster.  PostgreSQL
guys will show you why PostgreSQL is faster.  Oracle guys will show you
why Oracle is faster.  SQL*Server guys will show you why SQL*Server is
faster.  DB/2 guys will show you why DB/2 is faster.  Now, none of them
are lying (at least hopefully) but they are experts in their own domain
and not in the domain of the other product and they are also going to
choose those tight little corners where their product has the biggest
advantage.

IMO-YMMV.
P.S.
I'm a PostgreSQL fan and so I am likely to (perhaps unconsciously) favor
PostgreSQL in my remarks.