Re: [GENERAL] Re: [PHP3] Re: PostgreSQL vs Mysql comparison - Mailing list pgsql-general

hi all...


> Note that it would be much better to write a benchmark that can be run
> against any SQL server.  As the MySQL benchmarks are designed to do
> exactly this, I would suggest that you use the MySQL benchmark suit as
> a base for this!

i disagree heartily. standardized benchmarks such as these are rediculous wastes
of time, especially when done with database engines. why?

because, as has been shown time and time again on both sides of any fence you
look at in our wonderful world of software, benchmarks usually favour one side
or the other due to the fact that they are standard. well, software isn't
standard. MySQL and PostgreSQL use different methods for handling queries,
indecies, data, etc.. should we expect the exact same queries to perform equally
well on both? no! it will most probably favour one or the other, depending how
it is written. i'm not saying this is always intentional, its merely a fact
about software and standardized benchmarking as i see it.

what is better? well, what is software used for? to run arbitrary
queries/functions? NO! to solve a problem, complete a task. SO. give me a task.
allow me to use the software to complete that task. because in the real world,
i don't write the same queries in the same way for two different database
engines. i don't use X the same way i use Windows. i don't use GIMP the same
way i use photoshop.

benchmarks are a waste of time. problem solutions designec for the products at
hand show the REAL potential of the systems and can't be screwed with. after
all, all parties are going to use everything they can to make it fast. same
results. different paths. different software. apples and apples.

its not the method. its the answer to the question.

> I don't think the important things is just to optimize some specific
> queries; It's much more important to test a lot of different types of
> queries.

on this i agree. so, lets optimize all the queries (or not =), but leave it to
the hackers at their discretion.

> I think it's important to using queries that are common to many
> applications.
>
> For normal queries there isn't that much to optimize.  There is of
> course also the option to add some test to 'solve some problem'.  In
> this case one can use different methods to solve the query for
> different databases;  Normally these kind of tests are more
> interesting than useful as this isn't normally portable between
> database servers.

portability is a non issue, in my mind. when i use a database engine, i use
every nook and cranny of it when querying, etc to get performance. example:
would you forgo (sp) using PL/SQL in Oracle because it isn't supported the
exact same way everywhere (anywhere?) else? of course not. portability is a non
issue. problem solution power when given to a skilled user is. or an unskilled
user.

i think such tests are more telling than benchmarks. benchmarks are
interesting, but completely uninterpretable in a meaningful way (see above).
problem solution capability shows the true power of a system, esp. if that
system has unique ways of doing things that REALLY make a difference.

> Aaron> this is open source, remember? its about choice. its about the right tool for
> Aaron> the right job. its about a FUDless environment where we get the tools to do
> Aaron> what we NEED to do.
>
> This is exactly the aim of the MySQL benchmarks.  (They are GPL)

this (GPLing) is indeed applaudable!

> Aaron> as coordinator, i'd be willing to collect the final parameters, communicate
> Aaron> with representatives from each side (probably the developers?) and post the
> Aaron> results (after running them, of course :-)
>
> In this case;  Can you take a look at the MySQL benchmark suite and
> comment on this?

i believe my comments above stand. i do not support benchmarking. it only works
when you have two identical systems. but the point of such exercises is to
differences between DIFFERENT systems. =)

that said (again), i'll go grab the crash_me test suite and look it over, toss
in my 2 cents, if i have that much at the end of the exercise =)

--
Aaron J. Seigo
Sys Admin