Re: Two weeks to feature freeze - Mailing list pgsql-hackers

"Dann Corbit" <DCorbit@connx.com> writes:
> Look at this:
> http://www.mysql.com/information/crash-me.php?mysql_4_1=on&postgres=on

This looks a little cleaner than the last time I looked at it (more than
three years ago), but it's still fundamentally a marketing effort.  It
is not an exercise in spec compliance measurement, because there are
hundreds of bullet points that all look exactly alike, whether they are
measuring spec-required elements, random vendor extensions to the spec,
or spec violations.  To take just one example of the latter,
"Calculate 1--1" is still shown with a green star for MySQL and a
failure for Postgres, when a more correct reading would be "Fails to
recognize SQL-standard -- comment syntax" for MySQL.  And yes, they were
called out on this three years ago, and no they haven't fixed the entry
since then.  I should believe that there is any good faith on their
part?

For another example, take a close look at the "Quoting" section, which
purports to measure compliance to the spec's ideas about how to quote
an identifier.  Postgres accepts double-quoted identifiers per spec,
including doubled double quotes per spec, and rejects bracketed or
backquoted identifiers per spec.  MySQL is apparently spec compliant on
just one of those four points.  Curious that they manage to end up with
a better looking display than us in this section; in particular note
that Postgres is specifically claimed *not* to handle double-quoted
identifiers.  (Memory is fuzzy after three years, but IIRC when you look
at the actual test code being used, it tests more than whether double
quoted identifiers are allowed, and really is failing us on some quite
unrelated detail.)

Another point worth mentioning is that most of the numerical limits
shown in the table have nothing to do with actual server limits, but
with random limitations of their test process.  For instance, I'm not
sure what "max index part length 235328" really means, but I am pretty
sure it's got nothing to do with the Postgres server.  Or look at
"constant string size in SELECT 16777207" ... nope, there's no such
limit.  (If they'd put a "+" in there then it'd be okay, but no.)
I still remember watching crash-me trying to measure the max query
length of Postgres 7.0: the crashme client process dumped core before
Postgres did, after which the controlling script announced that we
weren't crash-safe.

> So far, I have seen three problems pointed out (out of 600+ tests).

These are the high spots from three-year-old memories.  Do you really
want a detailed analysis?  A quick look at their table recalls plenty
of bogosity to my mind.

A last point is that this table is comparing MySQL 4.1 (bleeding edge
alpha release) against PG 7.2 (one full major release behind the times).
While I cannot really blame the MySQL guys for not being up-to-the-
minute on everyone else's releases, this does emphasize the key point,
namely that this isn't a fair comparison run by disinterested parties
but a marketing effort of, by, and for MySQL.
        regards, tom lane