Re: Benchmark comparing PostgreSQL, MySQL and Oracle - Mailing list pgsql-performance

From Jonah H. Harris
Subject Re: Benchmark comparing PostgreSQL, MySQL and Oracle
Date
Msg-id 36e682920902200939y9e80176q507f6a835782eada@mail.gmail.com
Whole thread Raw
In response to Benchmark comparing PostgreSQL, MySQL and Oracle  (Sergio Lopez <sergio.lopez@nologin.es>)
Responses Re: Benchmark comparing PostgreSQL, MySQL and Oracle  (Sergio Lopez <sergio.lopez@nologin.es>)
Re: Benchmark comparing PostgreSQL, MySQL and Oracle  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-performance
On Fri, Feb 20, 2009 at 6:28 AM, Sergio Lopez <sergio.lopez@nologin.es> wrote:
Hi,

I've made a benchmark comparing PostgreSQL, MySQL and Oracle under three environments: GNU/Linux-x86, Solaris-x86 (same machine as GNU/Linux) and Solaris-SPARC. I think you might find it interesting:

http://blogs.nologin.es/slopez/archives/17-Benchmarking-Databases-I.-Volatile-Storage..html

Sorry Segio,

In addition to violating your Oracle license, you need to learn a couple things about benchmarking.

First of all, you need to do some research on the benchmark kit itself, rather than blindly downloading and using one.  BenchmarkSQL has significant bugs in it which affect the result.  I can say that authoritatively as I worked on/with it for quite awhile.  Don't trust any result that comes from BenchmarkSQL.  If you fix the bugs, Oracle (out of the box in OLTP config) will come out 60%.

Oracle comes out twice as fast as PG on Linux.  And, unless you're using a significant number of warehouses, MySQL+InnoDB will come out better than PG as well.

Second, I didn't see anything in your Oracle settings for parallelism and I/O tuning.  Did you set them?  And, based on what you presented, you didn't set configure the SGA appropriately given the hardware mentioned.  What was your log buffer set to?

Third, did you manually analyze the Oracle/MySQL databases, because BenchmarkSQL will automatically analyze Postgres' tables to help the optimizer... did you do the same for the other databases?

Fourth, it didn't look like you tuned PG properly either.  What was shared_buffers, wal_buffers, and wal_sync_method set to?

Fifth, did you do an out-of-the-box install of Oracle, or a custom one?  If out of the box, did you choose OLTP or General?

There's lots of other things I could go on about in regard to flushing all the caches prior to starting the benchmarks, filesystem options, etc.

Not trying to be rude, but *THIS* is why Oracle, IBM, Microsoft, et al. don't want people running benchmarks without their permission.  When performing benchmarks, there are a lot of things to take into consideration.  If you're just performing out-of-the-box tests, then that's fine, but you have to make sure the benchmark kit doesn't optimize itself for any one of those databases (which it does for PG).

--
Jonah H. Harris, Senior DBA
myYearbook.com

pgsql-performance by date:

Previous
From: Sergio Lopez
Date:
Subject: Re: Benchmark comparing PostgreSQL, MySQL and Oracle
Next
From: Sergio Lopez
Date:
Subject: Re: Benchmark comparing PostgreSQL, MySQL and Oracle