Thread: strange postgresql x mysql comparison in forrester analyse

I got link on free downloadable document from Microsoft Server.

http://www.microsoft.com/presspass/itanalyst/docs/06-30-09EnterpriseDatabaseManagementSystems.aspx

Still I though so Forrester analysis are well. I was surprised bad
informations about PostgreSQL there. An author probably doesn't read
some about PostgreSQL last 5 years. I thing so we should to warn about
very bad work of Forrester agency.

Pavel Stehule

p.s. Who use Ingres? I though so this database is dead - and in this
document I reading, so The best representatives of OSS databases are
Ingres, MySQL and PostgreSQL.

2009/10/14 Pavel Stehule <pavel.stehule@gmail.com>
>
> I got link on free downloadable document from Microsoft Server.
>
> http://www.microsoft.com/presspass/itanalyst/docs/06-30-09EnterpriseDatabaseManagementSystems.aspx
>
> Still I though so Forrester analysis are well. I was surprised bad
> informations about PostgreSQL there. An author probably doesn't read
> some about PostgreSQL last 5 years. I thing so we should to warn about
> very bad work of Forrester agency.
>
> Pavel Stehule
>
> p.s. Who use Ingres? I though so this database is dead - and in this
> document I reading, so The best representatives of OSS databases are
> Ingres, MySQL and PostgreSQL.
>

Sounds like a pile of FUD: "PostgreSQL lags behind. PostgreSQL has
some good capabilities across the board but lags in
performance, scalability, administration, application development,
support for disparate data
types, and VLDBs."

They mark Postgres down as having the weakest strategy and weakest
offering.  Strangely enough they put Database availability for MSSQL
as the 2nd highest and Postgres as 2nd lowest, despite us having a
history of downtime of our MSSQL servers here (one quite frequently)
and at my previous place of work, and no downtime ever with Postgres.

This made me laugh too: "PostgreSQL: An offering that lags in
enterprise database features and functionality. Although
PostgreSQL offers a good set of basic database features and
functionality, it lags in enterpriseclass
capabilities for availability, security, programmability, and
performance. In the past, Sun
and Fujitsu have supported PostgreSQL — and more recently EnterpriseDB
— but its enterprise
adoption has been slow, although it has the second-largest developer
community after MySQL."

"Issues: Although PostgreSQL has good features and functionality, it
is not a proven enterpriseclass
DBMS to support mission-critical deployments."

Even though there are so many examples to the contrary.

All quite ridiculous, and they're either commenting without doing any
research, or being commissioned to put their commissioner in the right
light.  In either case it's horrendously inaccurate and misleading.
The "report" contributes nothing of value.

Thom

Forrester interviewed 21 vendor and user companies, including ibM, ingres, Microsoft, oracle, PostgreSQl, Sun Microsystems, and Sybase.

Pavel Stehule wrote:
> I got link on free downloadable document from Microsoft Server.
>

LOL at them evaluating the PostgreSQL "Company financials".

And "calls with seven of each vendor's current customers"
seems a bit interesting.   What does the postgres project
actually sell?  T-shirts?

On Thu, Oct 15, 2009 at 5:19 PM, Ron Mayer
<rm_pg@cheapcomplexdevices.com> wrote:
> Pavel Stehule wrote:
>> I got link on free downloadable document from Microsoft Server.
>>
>
> LOL at them evaluating the PostgreSQL "Company financials".
>
> And "calls with seven of each vendor's current customers"
> seems a bit interesting.   What does the postgres project
> actually sell?  T-shirts?

And fluffy stuffed elephants here in Europe :-)

http://andreas.scherbaum.la/blog/archives/524-FOSDEM-2009-is-over.html

See the bottom two pictures.


--
Dave Page
EnterpriseDB UK:   http://www.enterprisedb.com

On 15/10/2009 17:25, Dave Page wrote:
> And fluffy stuffed elephants here in Europe :-)
>
> http://andreas.scherbaum.la/blog/archives/524-FOSDEM-2009-is-over.html
>
> See the bottom two pictures.

Cool! Will those be available in Paris in November? My daughter has a
weakness for stuffed (toy) animals...

Ray.

------------------------------------------------------------------
Raymond O'Donnell, Director of Music, Galway Cathedral, Ireland
rod@iol.ie
Galway Cathedral Recitals: http://www.galwaycathedral.org/recitals
------------------------------------------------------------------

Raymond O'Donnell wrote:
> On 15/10/2009 17:25, Dave Page wrote:
>> And fluffy stuffed elephants here in Europe :-)
>>
>> http://andreas.scherbaum.la/blog/archives/524-FOSDEM-2009-is-over.html
>>
>> See the bottom two pictures.
>
> Cool! Will those be available in Paris in November? My daughter has a
> weakness for stuffed (toy) animals...

I fully expect Forrester to interview her on the toy's scalability
features next time they're looking for a "customer" of the postgres
project to interview.

On Thu, Oct 15, 2009 at 8:50 PM, Raymond O'Donnell <rod@iol.ie> wrote:
> On 15/10/2009 17:25, Dave Page wrote:
>> And fluffy stuffed elephants here in Europe :-)
>>
>> http://andreas.scherbaum.la/blog/archives/524-FOSDEM-2009-is-over.html
>>
>> See the bottom two pictures.
>
> Cool! Will those be available in Paris in November? My daughter has a
> weakness for stuffed (toy) animals...

I'm not sure about the big ones (they're really expensive anyway).

Are you planning on bringing any of the small ones Ads?


--
Dave Page
EnterpriseDB UK:   http://www.enterprisedb.com

On Thu, Oct 15, 2009 at 2:43 PM, Ron Mayer
<rm_pg@cheapcomplexdevices.com> wrote:

> I fully expect Forrester to interview her on the toy's scalability
> features next time they're looking for a "customer" of the postgres
> project to interview.

For elephant swag that scales well, I would recommend elephant balloons.

--
Regards,
Richard Broersma Jr.

Visit the Los Angeles PostgreSQL Users Group (LAPUG)
http://pugs.postgresql.org/lapug

On Wed, 14 Oct 2009, Thom Brown wrote:

> This made me laugh too: "PostgreSQL: An offering that lags in enterprise
> database features and functionality...

I hate to break it to you, but by the criteria listed for what's an
"enterprise" database:

"support for application development, high availability, disaster
recovery, security, high performance, a wide range of data types, and
backup and recovery."

And later:

"support for high availability, security, performance, manageability, and
integration with applications."

PostgreSQL *is* weak.  It's got no integrated replication or
high-availability to that business people can see, strictly low-level
command-line tools for backup and management, lack of any obvious fancy
development tools known to work with the product, outright rejection of
feel-good security measures unless they are actually effective...all
completely valid things to criticize.  If these things are the criteria
you use to measure "enterprise", it's completely fair to say PostgreSQL
doesn't match the competition he's comparing against.  I don't think the
reports is all that biased, besides the fact that what the analyst (and
lots of other business people too!) feel is important doesn't match the
priorities of the PG community.

Let's consider the specific criticisms:

"PostgreSQL has some good capabilities across the board but lags in
performance, scalability, administration, application development, support
for disparate data types, and VLDBs."

And consider each of them:

Performance:  PostgreSQL flat out fails on some of the common TPC
benchmark queries because it doesn't have support for features needed to
execute on them within the timeframe required (Jignesh at Sun did a good
report on which it does and doesn't handle a while back).  If stuff like
that is your benchmark, performance really is bad.  Do not be confused
because PG works great on *most* database tasks, there are plenty it's
miserable at compared with the commercial offerings they're comparing
against.

Scalability:  No integrated support for any sort of replication,
clustering, or connection pooling?  You've just failed as far as this part
of the market is concerned.

Administration:  I like powerful command line tools even if they're
cryptic.  The market this report is written to does not.

Application development:  the tools people PostgreSQL apps with are great
if you're got a UNIX-ish background.  They look pretty primitive to those
who don't get that though.  Would you bet your business that the
PostgreSQL .Net driver is high quality?  That's the sort of stuff that's
being evaluated here.  (Not to pick on the authors of that driver, I know
that code has been moving along nicely, just the most obvious example of
priority disconnect between this community and the market at large I could
think of).

Support for disparate data types:  no idea what that's supposed to mean,
here I think the analyst may have missed the power of the Postgres type
system.

VLDBs:  At the point this was written, there wasn't even any clear
in-place upgrade path for PostgreSQL database.  Instant thumbs-down from
most large database prospects.  There's plenty of other missing features
here too; Simon made a nice list at
http://wiki.postgresql.org/wiki/Simon_Riggs%27_Development_Projects#Very_Large_Database_.28VLDB.29

Again, these items are probably not your priorities or you wouldn't be
using PostgreSQL, but I think the analyst is right that they're often
those of the customers they're aiming the report at.  I'm quite pleased at
the ever expanding reach of applications PostgreSQL is appropriate for,
but to be both fair and accurate here you really need to temper that with
recognizing how many it's just not right for.  Yet!

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

On Oct 14, 2009, at 7:20 AM, Thom Brown wrote:
> Sounds like a pile of FUD: "PostgreSQL lags behind. PostgreSQL has
> some good capabilities across the board but lags in
> performance, scalability, administration, application development,
> support for disparate data
> types, and VLDBs."


Sounds like we're starting to hit too close to home for at least some
company that has pockets to influence Forrester. I expect we'll be
seeing more of this, not less.
--
Decibel!, aka Jim C. Nasby, Database Architect  decibel@decibel.org
Give your computer some brain candy! www.distributed.net Team #1828

On Friday 16. October 2009, Greg Smith wrote:
>On Wed, 14 Oct 2009, Thom Brown wrote:
>> This made me laugh too: "PostgreSQL: An offering that lags in
>> enterprise database features and functionality...
>
>I hate to break it to you, but by the criteria listed for what's an
>"enterprise" database:
>
>"support for application development, high availability, disaster
>recovery, security, high performance, a wide range of data types, and
>backup and recovery."
>
>And later:
>
>"support for high availability, security, performance, manageability,
> and integration with applications."
>
>PostgreSQL *is* weak.

Greg, thank you for what I think is a very accurate analysis on what's
the trouble with PostgreSQL from a PHB point of view. I myself am a
all-out geek, and I'm extremely satisfied with Pg just the way it is.
But I totally see your points.

So, I think that the main question now, in order to take
PostgreSQL «mainstream», is to add the features the PHBs want. That is,
_if_ it is regarded, by the community, as a valid goal for PostgreSQL
to gain PHB cred.

What can be done to get Pg this kind of cred? Can developers be
attracted who have a hunch about how to make PostgreSQL a little more
streamlined from that angle? Or should the community as a whole,
perhaps, aim for  more general «end user friendliness»?

Or, are we satisfied with keeping Pg as a fringe product, made by and
for geeks?
--
Leif Biberg Kristensen | Registered Linux User #338009
Me And My Database: http://solumslekt.org/blog/

Greg,

> Again, these items are probably not your priorities or you wouldn't be
> using PostgreSQL, but I think the analyst is right that they're often
> those of the customers they're aiming the report at.  I'm quite pleased
> at the ever expanding reach of applications PostgreSQL is appropriate
> for, but to be both fair and accurate here you really need to temper
> that with recognizing how many it's just not right for.  Yet!

The problem isn't the commercial databases. I'd agree that we lag well
behind Oracle on most of those things.

It's the other OSDBs he's comparing against; I'm pretty familiar with
the capabilities of both INGRES and MySQL, and in most of those
categories, both of them lag behind PostgreSQL.  Yet they were rated
higher because the analyst gets his data from the corporate marketing
departments, and does not actually do any independant analysis.

In other words, I made the mistake of being honest with the analyst
rather than hyping, which worked with Forrester in the past.  Just not
anymore.

--Josh Berkus

On Fri, Oct 16, 2009 at 4:36 PM, Josh Berkus <josh@agliodbs.com> wrote:

> In other words, I made the mistake of being honest with the analyst
> rather than hyping, which worked with Forrester in the past.  Just not
> anymore.

I don't think its a mistake for a person wearing the hat of community
leader and core member to be honest about the capabilities of
PostgreSQL.  But to make the playing field even, should future
inquires from Forrester be directed to the marketing wings of
companies that use PostgreSQL.


--
Regards,
Richard Broersma Jr.

Visit the Los Angeles PostgreSQL Users Group (LAPUG)
http://pugs.postgresql.org/lapug

On Oct 16, 2009, at 6:18 PM, Leif B. Kristensen wrote:

> What can be done to get Pg this kind of cred?

This has probably been beat to death, but still... I think native
master/slave replication support (e.g., WAL shipping to slaves that
can be queried) would be a big one.

Yours,

Tom

On 16 Oct 2009, at 23:18, Leif B. Kristensen wrote:

>> And later:
>>
>> "support for high availability, security, performance, manageability,
>> and integration with applications."
>>
>> PostgreSQL *is* weak.
>
> [snip]
>
> What can be done to get Pg this kind of cred? Can developers be
> attracted who have a hunch about how to make PostgreSQL a little more
> streamlined from that angle? Or should the community as a whole,
> perhaps, aim for  more general «end user friendliness»?
>
> Or, are we satisfied with keeping Pg as a fringe product, made by and
> for geeks?

Just to add in another point of view - these Oracle or dare I say it
MS SQL type features are great for comparing databases in a top trumps
fashion but they're not the only way of expanding the scope of what
Postgres is used for. I create postgres apps using an agile, i.e.
quick development platform I developed that's now open source. I'm
talking about it at PGday.EU. For certain entrepreneurial
organisations or those in changing environments, the ability to use
rapid prototyping / development of databases is more important than
having the highest TPC benchmark. In fact, Oracle also are into this,
for example I think they've won the www.radrace.org competition a few
times.

Having said that, my argument has always been that using postgres does
give built in scalability should things take off with your app, even
though smaller companies don't have to invest at the start in
licenses. If you suddenly get an exponential increase in simultaneous
users, there is an inbuilt ability to handle it. I think there's
another talk at the conference on this topic too - looking forward to
it. I'm not going to complain if Pg continues development on this road.

Oliver Kohll
oliver@gtwm.co.uk / 0845 456 1810 / 07814 828608
www.gtportalbase.com

Greg Smith wrote:
> I hate to break it to you, but by the criteria listed for what's an
> "enterprise" database:
> "support for application development, high availability, disaster
> recovery, security, high performance, a wide range of data types, and
> backup and recovery. ...
> PostgreSQL *is* weak.

The analyst is making an argument like "Linux is weaker than Windows
because it has no graphical user interface in the kernel" - and
we're falling for it.

> And consider each of them:
>
> Performance:  ...Do not be confused
> because PG works great on *most* database tasks, there are plenty it's
> miserable at ...

Would have been nice if they had pointed to the benchmark they
had in mind.   The only well known published benchmark I
see (on spec.org) that compares postgres to many of these other
databases made us look OK to me.

> Scalability:  No integrated support for any sort of replication,
> clustering, or connection pooling?  You've just failed as far as this
> part of the market is concerned.

Postgres has this - in exactly the same way that Linux has a GUI.

> Administration:  I like powerful command line tools even if they're
> cryptic.  The market this report is written to does not.

And in the same way, Linux administration tools are nonexistant
in the kernel either.

Which makes it a good thing that postgres has pgadmin and webmin.

> Application development:  the tools people PostgreSQL apps with are
> great if you're got a UNIX-ish background.  They look pretty primitive
> to those who don't get that though.  Would you bet your business that
> the PostgreSQL .Net driver is high quality?

I'd bet it's on par with freetds / unixodbc and other tools to get
linux apps to work with sqlserver.


> Support for disparate data types:  no idea what that's supposed to mean,
> here I think the analyst may have missed the power of the Postgres type
> system.

I totally agree we fall short here - for example most commercial
database vendors offer GIS data types.

Fortunately if we consider postgres as a platform, we have one
available from a third party.

> VLDBs:  At the point this was written, there wasn't even any clear
> in-place upgrade path for PostgreSQL database.  Instant thumbs-down from
> most large database prospects.  There's plenty of other missing features
> here too; Simon made a nice list at

For VLDB work, I agree F/OSS postgres doesn't really stand up, and
you need proprietary add-ons/forks.  However I note that many of the
larges databases in the world (Yahoo's Ebay's, etc) are indeed postgres
with extensions.

If the analysts report excludes any proprietary, I agree that
postgres as well as any other vendors who need proprietary
code should score low here.

OTOH, if the analyst's report permits proprietary code, it seems
postgres should score on top here.

> Again, these items are probably not your priorities or you wouldn't be
> using PostgreSQL, but I think the analyst is right that they're often
> those of the customers they're aiming the report at.

I think the problem is that we're comparing an apples (just the core
postgres kernel) with oranges (the database kernels plus all the
supporting tools and apps from other vendors).

If you look at postgres as a broader platform, we do very well,
especially if you include non-BSD (both gpl and proprietary)
licensed extensions.

On Sun, 18 Oct 2009, Ron Mayer wrote:

> Would have been nice if they had pointed to the benchmark they had in
> mind.  The only well known published benchmark I see (on spec.org) that
> compares postgres to many of these other databases made us look OK to
> me.

Found the talk I was alluding to:
http://blogs.sun.com/jkshah/entry/postgresql_east_2008_talk_postgresql
Note the TCP-H summary on P26.  Out of the 21 queries in that standard
benchmark load, PostgreSQL basically doesn't handle 9 of them.  Makes it
hard for businesses to trust you can deploy it as a generic database
application for data-warehouse purposes knowing there are some sizable
holes there.  And it's difficult to push back and dispute claims of
benchmark issues with the database vs. the commercial products knowing
it's not hard to discover said holes.

> I think the problem is that we're comparing an apples (just the core
> postgres kernel) with oranges (the database kernels plus all the
> supporting tools and apps from other vendors).

Sure, but the larger point I was suggesting is that the way the analyst is
evaluating like that matches that of more business people than the
community may like to acknowledge.  There's a reason people bundle all
that stuff with their core database products.

Ultimately this is a hard issue to field.  If the comparison piece here
was instead EDB's bundling of Postgres with the whole fairly well
integrated development tool stack they make available now (note how big
the list at http://www.enterprisedb.com/products/download.do is getting
nowadays), the larger comparison might go off a bit better.  You know that
would raise a completely different set of criticism of the study from
within this community though.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

Greg Smith wrote:
> On Sun, 18 Oct 2009, Ron Mayer wrote:
>
>> Would have been nice if they had pointed to the benchmark they had in
>> mind.  The only well known published benchmark I see (on spec.org)
>> that compares postgres to many of these other databases made us look
>> OK to me.
>
> Found the talk I was alluding to:
> http://blogs.sun.com/jkshah/entry/postgresql_east_2008_talk_postgresql
> Note the TCP-H summary on P26.  Out of the 21 queries in that standard
> benchmark load, PostgreSQL basically doesn't handle 9 of them.  Makes it
> hard for businesses to trust you can deploy it as a generic database
> application for data-warehouse purposes knowing there are some sizable
> holes there.  And it's difficult to push back and dispute claims of
> benchmark issues with the database vs. the commercial products knowing
> it's not hard to discover said holes.

well for a long time our main problem with TPC-H was that we actually
delivered the wrong(!) answer (due to the half done SQL spec interval
implementation) to a number of queries there.
This issue was also mentioned in a number of other
benchmarks/comparisions like
http://monetdb.cwi.nl/projects/monetdb/SQL/Benchmark/TPCH/index.html.

While 8.4 should now run those queries correctly we are still far away
from being a serious competitor on a dataware house workload like this
so I agree that we still have ways to got...




Stefan