RE: TPC (was Great Bridge benchmark results for Postgres, 4others) - Mailing list pgsql-general

This benchmark had a lot of value for the job that was going to use ODBC.
Pretty obvious that Postgres blows away everyone else in the ODBC dept.  I'm
not sure if this shows that PGsql is best-performer, or if it just shows
that the other db's have sucky ODBC implimentations.

Too bad it doesn't show us what the performance would have been with native
drivers.  Hopefully someone will develop an interface that each db supports
at full speed on a minimum-functionality level (like ODBC, but faster).
Maybe that would shed some more light.  Perl::DBI comes close to this, but
your still relying on the quality of the module's implimentation.  Oh well.

But, I must detail that I will be using PGsql for my current .com project
(online ordering, etc).  This choice is over Sybase, Interbase, and MySQL.
I found the price / performance / features just couldn't be beat with PGsql.



> -----Original Message-----
> From: pgsql-general-owner@hub.org
> [mailto:pgsql-general-owner@hub.org]On
> Behalf Of Ned Lilly
> Sent: Monday, August 14, 2000 9:35 PM
> To: Alex Pilosov
> Cc: The Hermit Hacker; PostgreSQL General
> Subject: Re: TPC (was [GENERAL] Great Bridge benchmark results for
> Postgres, 4others)
>
>
> Hi Alex,
>
> Absolutely, as I said, we did this benchmarking for our own
> internal due diligence in
> understanding PostgreSQL's capabilities.  It's not intended
> to be a formal big-iron TPC
> test, like you see at tpc.org.  The software we used was one
> commercial vendor's
> implementation of the published AS3AP and TPC-C specs - it's
> the same one used by a lot
> of trade magazines.
>
> Benchmarking will be a significant part of Great Bridge's
> ongoing contribution to the
> PostgreSQL community - starting with these relatively simple
> tests, and scaling up to
> larger systems over time.
>
> Regards,
> Ned
>
> Alex Pilosov wrote:
>
> > A more interesting benchmark would be to compare TPC/C
> results on same
> > kind of hardware other vendors use for THEIR TPC
> benchmarks, which are
> > posted on tpc.org, as well as comparing price/performance of each.
> >
> > TPC as run by company 'commissioned by GB' cannot be validated and
> > accepted into TPC database, they must be run under close
> supervision by
> > TPC-approved monitors. I hope GB actually springs for the
> price of running
> > the REAL TPC benchmark (last I heard it was around 25k$).
> >
> > To see how postgres performs on low-end (for TPC low-end is
> <8 processors)
> > would be interesting to say the least.
> >
> > A problem with a real TPC is the strong suggestion to run a
> transaction
> > manager, to improve speed. No transaction manager supports
> postgres yet.
> >
> > Another note on TPC is that they require to include as a final price
> > support contract, on which GreatBridge should be able to compete.
> >
> > On Mon, 14 Aug 2000, Ned Lilly wrote:
> >
> > > Marc's right... we opted for ODBC to ensure as much of an
> "apples to apples"
> > > comparison as possible.  Of the 5 databases we tested, a
> native driver existed for
> > > only the two (ahem) unnamed proprietary products -
> Postgres, Interbase, and MySQL
> > > had to rely on ODBC.  So we used the vendor's own ODBC
> for each of the other two
> > > cases.
> > >
> > > <disclaimer>
> > > As with all benchmarks, your mileage will vary according
> to hardware, OS, and of
> > > course the specific application.  What we attempted to do
> here was use two
> > > industry-standard benchmarks and treat all five products the same.
> > > </disclaimer>
> > >
> > > Presumably, if the vendor had taken the time to write a
> native driver for
> > > Postgres, the results would have seen an even bigger
> kick.  We don't have any
> > > reason to think that the results for all five tests in
> native driver mode would be
> > > out of proportion to the results we got through ODBC.
> > >
> > > Regards,
> > > Ned
> > >
> > >
> > > The Hermit Hacker wrote:
> > >
> > > > On Mon, 14 Aug 2000, Steve Wolfe wrote:
> > > >
> > > > > > 1) Using only ODBC drivers.  I don't know how much
> of an impact a driver
> > > > > can
> > > > > > make but it would seem that using native drivers
> would shutdown one source
> > > > > > of objections.
> > > > >
> > > > >   Using ODBC is guaranteed to slow down the
> benchmark.  I've seen native
> > > > > database drivers beat ODBC by anywhere from a factor
> of two to an order of
> > > > > magnitude.
> > > >
> > > > I haven't had a chance to take a look at the benchmarks
> yet, having just
> > > > seen this, but *if* Great Bridge performed their
> benchmarks such that all
> > > > the databases were access via ODBC, then they are using an
> > > > 'apples-to-apples' approach, as each will have similar
> slowdowns as a
> > > > result ...
> > >
> > >
>
>