Re: Benchmark comparing PostgreSQL, MySQL and Oracle - Mailing list pgsql-performance

Hi all,

As the author of BenchmarkSQL and the founder of EnterpriseDB....  I
can assure you that BenchmarkSQL was NOT written specifically for
PostgreSQL.    It is intended to be a completely database agnostic
tpc-c like java based benchmark.

However; as Jonah correctly points out in painstaking detail:
PostgreSQL is good, but...  Oracle, MySQL/Innodb and and and don't
necessarily suck.  :-)

--Luss

PS:   Submit a patch to BenchmarkSQL and I'll update it.


On 2/20/09, Sergio Lopez <sergio.lopez@nologin.es> wrote:
> El Fri, 20 Feb 2009 16:54:58 -0500
> Robert Haas <robertmhaas@gmail.com> escribió:
>
>> On Fri, Feb 20, 2009 at 4:34 PM, Jonah H. Harris
>> <jonah.harris@gmail.com> wrote:
>> > On Fri, Feb 20, 2009 at 3:40 PM, Merlin Moncure
>> > <mmoncure@gmail.com> wrote:
>> >>
>> >> ISTM you are the one throwing out unsubstantiated assertions
>> >> without data to back it up.  OP ran benchmark. showed
>> >> hardware/configs, and demonstrated result.  He was careful to
>> >> hedge expectations and gave rationale for his analysis methods.
>> >
>> > As I pointed out in my last email, he makes claims about PG being
>> > faster than Oracle and MySQL based on his results.  I've already
>> > pointed out significant tuning considerations, for both Postgres
>> > and Oracle, which his benchmark did not take into account.
>> >
>> > This group really surprises me sometimes.  For such a smart group
>> > of people, I'm not sure why everyone seems to have a problem
>> > pointing out design flaws, etc. in -hackers, yet when we want to
>> > look good, we'll overlook blatant flaws where benchmarks are
>> > concerned.
>>
>> The biggest flaw in the benchmark by far has got to be that it was
>> done with a ramdisk, so it's really only measuring CPU consumption.
>> Measuring CPU consumption is interesting, but it doesn't have a lot to
>> do with throughput in real-life situations.  The benchmark was
>> obviously constructed to make PG look good, since the OP even mentions
>> on the page that the reason he went to ramdisk was that all of the
>> databases, *but particularly PG*, had trouble handling all those
>> little writes.  (I wonder how much it would help to fiddle with the
>> synchronous_commit settings.  How do MySQL and Oracle alleviate this
>> problem and we can usefully imitate any of it?)
>>
>
> The benchmark is NOT constructed to make PostgreSQL look good, that
> never was my intention. All databases suffered the I/O bottleneck for
> their redo/xlog/binary_log files, specially PostgreSQL but closely
> followed by Oracle. For some reason MySQL seems to deal better with I/O
> contention, but still gives numbers that are less than the half it gives
> with tmpfs.
>
> While using the old array (StorageTek T3), I've played with
> synchronous_commit, wal_sync_method, commit_delay... and only setting
> wal_sync_method = open_datasync (which, in Solaris, completly disables
> I/O syncing) gave better results, for obvious reasons.
>
> Anyway, I think that in the next few months I'll be able to repeat the
> tests with a nice SAN, and then we'll have new numbers that will be
> more near to "real-world situations" (but synthetic benchmarks always
> are synthetic benchmarks) and also we'll be able to compare them with
> this ones to see how I/O contetion impacts on each database.
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>