Re: Postgres vr.s Oracle - Mailing list pgsql-advocacy

On Wed, Dec 10, 2008 at 2:08 PM, Brian Hurt <bhurt@janestcapital.com> wrote:
> Thought I'd point this blog post out to the list:
> http://www.miketec.org/serendipity/index.php?/archives/7-Oracle-and-Postgres-Redux.html

Hmm... I wonder how scientific his benchmarking is and how well his
Oracle system was tuned.  Because I've done quite a few performance
comparisons between Postgres 8.3 and 8.4-dev against a well-tuned
Oracle8i instance on Linux, and 8i (from 1999) beats the latest
versions of Postgres quite handily on the same hardware.  And, while I
have received permission from Oracle to publish the result, I haven't
had the time to write up the blog entry yet.

Outside of simple curiosity, my reason for running the benchmark was
simply to show that in terms of performance, Oracle had it right over
10 years ago and that our continual discussions about leaving things
to the OS and file system developers (because they know how to manage
memory/data better than we do) is pointless.  It illustrates that if
Postgres ever wants to step into this century and take advantage of
newer hardware configurations, we need to accept the fact that PG's
inherent design has serious performance-related flaws which need to be
addressed sooner rather than later.  Similarly, I ran the same tests
against Oracle 10g and 11g, and a properly tuned Oracle system is
10-100x faster than Postgres on lots of operations in both OLTP and
DSS workloads, but because I didn't expect Postgres to be close to
Oracle these days, I went back to comparing against 8i (Standard
Edition) just to make my point.

Regardless of what some people on this list tell you, one of the main
reasons Oracle and other vendors don't like people performing external
benchmarks is because the majority of people screw them up.  Proper
benchmarking is something that takes time to do and you have to have a
good amount of experience ensuring the SUT environment is exactly the
same for both databases.  Similarly, the majority of people don't know
how to tune an Oracle system properly, which is why they get bad
results.  Whether that's the case in this test or not, is unknown.
There certainly isn't enough information in that blog entry to
determine if the system was tuned properly or whether the tests were
even performed correctly, yet it was forwarded on as gospel.
Similarly, the benchmark was performed by someone a bit jaded against
Oracle, which doesn't bode well for presuming he was unbiased.

I can't believe the number of people on these lists who continually
forward piss-poor benchmarks just because they think it somehow
solidifies their belief that PG is somehow equal to or better than
commercial databases in terms of overall performance.  Sure, there are
cases where PG is faster than Oracle... but they few and far between.
If Postgres works well for you, use it.  If not, use something else...
just please don't push half-baked benchmarks performed by jaded people
who are so uninformed that they think Don Burleson is the
be-all-end-all of Oracle tuning consultants.

-Jonah