Re: PostgreSQL x Oracle - Mailing list pgsql-general

Oops! andrew@libertyrms.info (Andrew Sullivan) was seen spray-painting on a wall:
> On Mon, Feb 10, 2003 at 08:59:14AM -0500, Christopher Browne wrote:
>> No, the answer is "We don't know which is faster," and it is quite
>> certain that we /can't/ know with any degree of certainty.
>>
>> The licensing arrangements for Oracle (and many similar products) deny
>> the ability to do performance comparisons.
>
> No they don't.  The deny the ability to _publish_ the benchmarks.  If
> you have sufficient funds and time, you could do all the benchmarks
> yourself.

Fair enough.

But the point still stands that the licenses deny the ability to make
public claims about relative performance.

If you happened to do some benchmarks (on dev6, if it ever gets
working :-)), then I'd be quite well placed to look at the results,
but it wouldn't help anybody making public claims about their relative
peformance.

> You could also rely on the TPC for data.  Since they publish the
> test specs, the comparison is at least apples to apples.  They
> happen to be really strange, bio-engineered apples, with each system
> hand-crafted for the purposes of the test at hand.  And of course,
> the deeper pockets of Oracle provide ample oppotunity for them to
> try more often.  But you still get actually useful comparisons in
> that case.  Whether they are sufficiently analogous to the
> application you are trying to build is another question entirely.
> (Admittedly, in the absense of a rich patron, PostgreSQL is not
> going to have any TPC numbers.)

I'm still fairly skeptical of the TPC data; the results I have seen
commonly involve fairly contrived sorts of systems.  They wind up
combining Oracle + some hardware configuration that may never actually
be sold to anyone + Tuxedo with a hand-tuned overall set of
configuration.

Will that configuration be usefully analagous with what anyone is
actually planning to use?  It's difficult to say.

There's actually, by the same token, some "real-world" merit to some
of the MySQL benchmarketing.  Consider: they may only present the DB
activity that allows MySQL to look good, that involves direct keyed
access to individual DB entries, which is where it "shines."  For
someone that is using the "LAMP" development model, it is more than
plausible that their data access methods correspond fairly well with
the benchmarks.

In other words, someone that is using the database wisely will use it
for the things it actually is good at, which, for the "M word" will
involve having a very few processes doing a few updates and a bunch of
processes doing a lot of reads.

> I see frequently suggestions that benchmarks are useless because
> they measure the wrong things, or that they are skewed for this or
> that case.  That doesn't mean that good tests are impossible. I'm
> not a real big fan of the TPC's policies, but they do have some
> well-crafted test specifications.

Unfortuantely, a lot of the older TPC specs have proven susceptible to
"hacks" where the data proves to be almost totally non-interdependent,
so that by throwing extra CPUs and extra disks at the benchmarks, you
can get very nearly linear scalability.  Based on history, I'm
skeptical that the "hacks" for more recent benchmarks just haven't
been found yet...
--
output = reverse("ac.notelrac.teneerf@" "454aa")
http://www3.sympatico.ca/cbbrowne/finances.html
"Robot: Your plastic pal who's fun to be with."
-- Marketing Division, Sirius Cybernetics Corp.