Thread: Thoughs after discussions at OSCON
Hi all,
I've been doing some reflecting on some things I saw and heard at the
OSCON this year, and I thought I'd note them here, more to prime
discussion than to propose any strong conclusions.
1. The first item I think bears mention is the number of occasions I
had people ask me whether the project is losing steam, or "losing
out" to MySQL, or being no threat to Oracle.
Coming from (as I do) the position that MySQL is increasingly
irrelevant to the community's efforts, I found this pretty
surprising. What it tells me, though, is that this message is
getting through to people, even if we don't think it's a real or
meaningful one. (My alarm at this is no doubt increased by my
reading a biography of J. Robert Oppenheimer right now; but that
example only serves to remind me more than ever that pursuing
rational argument in the face of hysterical opinion is not always an
approach that will yield happy results.)
In a different way, since I'm not particularly interested in
"threatening" Oracle, I'm not sure why people were anxious to set up
that sort of battle.
In any case, this leads me to believe that we need to find some way
to begin to undermine all the FUD out there, before we get into such
deep water that we can't get back to shore. I don't know how to do
this; but I came away with the impression that this is much more
important than it used to be. I was surprised, for instance, to see
Oracle in a prominent place near by us on the floor. I think MySQL
has taken from them just enough money now that they're seeing free
software as a real threat, and given the things I've heard from
Oracle reps I've talked to, I know they think Postgres is a much
bigger deal to them than is MySQL.
2. The second item that struck me is that, in spite of what I
heard in (1), even more large companies are looking at betting the
farm on Postgres. I think this is excellent news, but we've
certainly got to find a way both to make this a safe choice for
themselves and others (by getting people "out of the closet"), and to
make it sustainable (by building up a community of administrators who
can run these systems). I think the latter consideration is
particularly important: if people start adopting this system, and
find it's impossible to find administrators, PostgreSQL will be
written off as impossible to use. It would be a really bad time for
the DBA mixture to get "lean".
3. There is a sort of change coming about once again in the free
software world. It appeared to me that there were a lot more
"enterprise" and pointy-hair types this year than in the two previous
years I've been there. This might, of course, be a function of the
burgeoning of my own points, but I suspect that it rather marks a
resurgence of commercial interest in free software that sort of had
its wind knocked out by the bursted bubble in 2000 or 2001. This set
of commercially interested people are much more sceptical than the
last round, which probably means that they'll have better legs; but
also, that they're likely to tell us some things we don't really want
to hear. We're going to have to be particularly sensitive to those
sorts of marketing concerns in order to deal with (1) and (2). I
think we can do it for sure, but I was reminded on more than one
occasion of the tendency of some in the free software community (and
not here in particular) to point and scream the moment a profit
motive is floated. This isn't, by the way, to criticise anyone here.
It was merely an observation generally of the conference, and some of
the hallway conversations I heard (by no means most, let alone all).
Note that this is _not_ a claim that we need to pander to commercial
interests and make salespeople happy. It's rather to say that I
won't be surprised if we see more folks coming from corporations,
and wanting free things from the community. We're going to have to
be sure that we understand their issues before accepting or
rejecting their proposals, because very often they'll bring a
perspective we'd never have thought about.
In any case, those are some not-too-random thoughts I had on my
return.
A
--
Andrew Sullivan | ajs@crankycanuck.ca
Information security isn't a technological problem. It's an economics
problem.
--Bruce Schneier
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > I've been doing some reflecting on some things I saw and heard at the > OSCON this year, and I thought I'd note them here, more to prime > discussion than to propose any strong conclusions. Random responses from a non-OSCONer: > The first item I think bears mention is the number of occasions I > had people ask me whether the project is losing steam, or "losing > out" to MySQL, or being no threat to Oracle. We certainly are not gaining "geek mindshare" as fast as we should. It doesn't help that O'Reilly seems to be in bed with MySQL AB (exhibit one: the joint MySQL conference). We need to look at all the things that MySQL is doing right, because our technical superiority alone is not going to save us. I've also started to think lately that our BSD license may be an even greater asset than our feature set. More about that later in a PP rant. :) > I was surprised, for instance, to see Oracle in a prominent place > near by us on the floor. Oracle was there? Anyone talk to them? What part of their product is "open source" anyway? > find it's impossible to find administrators, PostgreSQL will be > written off as impossible to use. It would be a really bad time for > the DBA mixture to get "lean". It would be nice if we could emphasize that "if you can admin Oracle, you can admin Postgres." While the systems are vastly different, PG is so much easier to admin than Oracle I haven't met an Oracle DBA once who was not amazed at how easy it was to use after dealing with the thousands of knobs and levers that Oracle gives you. The important things such as memory usage, disk partitioning, tablespaces, and explain plans are all still there, they are just easier to use. :) > This set of commercially interested people are much more sceptical > than the last round, which probably means that they'll have better > legs; but also, that they're likely to tell us some things we don't > really want to hear. I think the community overall does a pretty good job of this, although we could perhaps do responses on mailing lists more of a "interesting, can you tell us how you see this feature working" and less of the "feel free to fund someone to write it for you." - -- Greg Sabino Mullane greg@turnstep.com PGP Key: 0x14964AC8 200508082158 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -----BEGIN PGP SIGNATURE----- iD8DBQFC+A8bvJuQZxSWSsgRAsxDAJ41FFg8F+q4dWYd5SNwL8+BPQ6xDgCgynyy BfNPg+YM+T6dTISCRrWNpa8= =C2yx -----END PGP SIGNATURE-----
>Oracle was there? Anyone talk to them? What part of their product is >"open source" anyway? > > > Oracle has done ALOT for open source. That is why they were there. Sincerely, Joshua D. Drake
> Random responses from a non-OSCONer: > >> The first item I think bears mention is the number of occasions I >> had people ask me whether the project is losing steam, or "losing >> out" to MySQL, or being no threat to Oracle. > > We certainly are not gaining "geek mindshare" as fast as we should. > It doesn't help that O'Reilly seems to be in bed with MySQL AB > (exhibit one: the joint MySQL conference). It seems to me that APress is a plausible publisher to "bias towards;" the last couple of books that I have found *very* interesting were published by them. They have published some things O'Reilly wouldn't (on zsh, Common Lisp), in areas that actually have gotten them sales (as in "having to do second printings"). Lisp people got in something of a snit because O'Reilly had a published policy that they wouldn't take such books. The *wise* move was and is to take would-be book offerings elsewhere. > We need to look at all the things that MySQL is doing right, because > our technical superiority alone is not going to save us. I've also > started to think lately that our BSD license may be an even greater > asset than our feature set. More about that later in a PP rant. :) > >> I was surprised, for instance, to see Oracle in a prominent place >> near by us on the floor. > > Oracle was there? Anyone talk to them? What part of their product is > "open source" anyway? I didn't go and visit; I barely heard a peep about Oracle at the conference. >> find it's impossible to find administrators, PostgreSQL will be >> written off as impossible to use. It would be a really bad time >> for the DBA mixture to get "lean". > > It would be nice if we could emphasize that "if you can admin > Oracle, you can admin Postgres." While the systems are vastly > different, PG is so much easier to admin than Oracle I haven't met > an Oracle DBA once who was not amazed at how easy it was to use > after dealing with the thousands of knobs and levers that Oracle > gives you. The important things such as memory usage, disk > partitioning, tablespaces, and explain plans are all still there, > they are just easier to use. :) That's only true if the Oracle DBA takes seriously the jump to PostgreSQL. If they carry the baggage of the attitude that Oracle's marketing folk dearly want to press on them, then they may prefer to play the "it hasn't got the knobs and levers and is therefore inferior" game. -- (reverse (concatenate 'string "moc.liamg" "@" "enworbbc")) http://cbbrowne.com/info/ If we were meant to fly, we wouldn't keep losing our luggage.
>It seems to me that APress is a plausible publisher to "bias towards;" >the last couple of books that I have found *very* interesting were >published by them. > > As an Oreilly author and I might get in trouble for this but I would suggest someone besides Oreilly as well. They made me change our license on Pratical PostgreSQL for the second edition so that it won't be viewable for free on the web anymore. From a business perspective it makes sense because they want the Safari stuff they are doing to be more profitable but still... >They have published some things O'Reilly wouldn't (on zsh, Common >Lisp), in areas that actually have gotten them sales (as in "having to >do second printings"). > > Well don't let that fool you though. That could mean they bothered to sell 5k+ books. Pratical PostgreSQL's first run was only 5k we had to go to second printing the first month it was out. 5k copies sold will NOT make up the money lost in the development of the book. You need at least 10-15k copies sold for that. >Lisp people got in something of a snit because O'Reilly had a >published policy that they wouldn't take such books. The *wise* move >was and is to take would-be book offerings elsewhere. > > > Sincerely, Joshua D. Drake
Andrew Sullivan wrote: >Hi all, > >I've been doing some reflecting on some things I saw and heard at the >OSCON this year, and I thought I'd note them here, more to prime >discussion than to propose any strong conclusions. > >1. The first item I think bears mention is the number of occasions I >had people ask me whether the project is losing steam, or "losing >out" to MySQL, or being no threat to Oracle. > > The biggest thing I noticed, from spending some time volunteering at the booth, was that most people wanted to talk about only one thing: performance. To me, that shows that the biggest misconception out there is that PostgreSQL is slow (or even that it is fast, but that's all it's got going for it). Thus secondly, there is the depressing observation that the majority of developers haven't a clue what the relational model is really good for. They want to wring every possible bit of speed out of a database while piling all sorts of constraints into application space. That's pretty much the norm for most open source applications I have seen. In fact, even some specifically PostgreSQL-based applications are fairly light in their use of constraints and server-side functions. It's a catch-22: one of the big problems with getting OS developers to use PostgreSQL's advanced capabilities is that they usually want their applications to remain compatible with MySQL, rather than limit potential deployment opportunities. Ironically, it seems to me that if MySQL gets more sophisticated, allowing real constraints, triggers, etc... that it will then actually be *easier* to get developers to switch to PostgreSQL. Anyway, I am really not too worried about the popularity of PostgreSQL. Even based on my non-scientific habit of just bringing it up in conversation, I find there is a lot more recognition of it these past couple years. The local Linux User group here in South Florida has even asked me to make a presentation on PostgreSQL. One of the difficulties with the FUD out there is that if M[a certain DBMS] claims it has a certain feature, then people just accept it without question (Any X is as good as anyone else's X). Maybe it's a good idea to put out some material explaining how much difference there can be in two different implementations of such a thing as (views/triggers/procedures/constraints), and the pitfalls that can happen because of this. I should say that the second thing people wanted most to discuss was replication. Again, a feature where often any_x == x, without differentiation between good or bad implementations. Regards, Rick Morris (NetCompass.us) (P.S. Had a great time hanging out with everyone; thanks for the warm welcome.)
On Tuesday 09 August 2005 07:36, Christopher Browne wrote:
> > Random responses from a non-OSCONer:
> >> The first item I think bears mention is the number of occasions I
> >> had people ask me whether the project is losing steam, or "losing
> >> out" to MySQL, or being no threat to Oracle.
> >
> > We certainly are not gaining "geek mindshare" as fast as we should.
> > It doesn't help that O'Reilly seems to be in bed with MySQL AB
> > (exhibit one: the joint MySQL conference).
>
So if you have been to an oscon in recent years, you'll be familiar with the
"open source trends" that Tim O'Rielly gives every year. Basically the idea
behind the talk is that you can spot trends within open source based on book
sales for a given topic. The problem of course is that every year we end up
looking bad because there are about 47,000 my$ql/oracle books and only 1
postgresql book, which is now several years out of date. I had to explain
this again this year to folks who weren't aware of the discrepency, but took
notice that postgresql comprises a very small dot on the orielly radar
screen. In fact, if you look in the catalog they provided to oscon attendees,
which includes about 1/2 dozen oreilly friendly publishers, you'll notice
there isn't a single postgresql book included.
> It seems to me that APress is a plausible publisher to "bias towards;"
> the last couple of books that I have found *very* interesting were
> published by them.
>
Having now met several of the Apress folks and knowing a bunch more, I can
tell you Apress is very excited about PostgreSQL's potential in the book
market. As most folks are aware, they recently released an updated version
of the begining database w/ postgresql book, which they acquired the rights
to from wrox. Another good sign is that they are currently working with
Mitch Pirtle on a Mambo book, which should include bits on using Mambo w/
PostgreSQL. Now I'll grant that these folks are a business, so thier
enthusiasm is directly tied to thier sales, but from what I have seen they
are by far the most pro-postgresql publisher around right now.
As a side note, I am currently reading the book "Joel on Software" from
Apress, which discusses software development strategies in an informal and
entertaining manner. I'd recommend it to anyone who does development or
project managment, it's at least as entertaining as any of the LISP books ;-)
--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
On Mon, 2005-08-08 at 21:50 -0700, Joshua D. Drake wrote: > Well don't let that fool you though. That could mean they bothered to > sell 5k+ books. Pratical > PostgreSQL's first run was only 5k we had to go to second printing the > first month it was out. > > 5k copies sold will NOT make up the money lost in the development of the > book. You need > at least 10-15k copies sold for that. Just a thought on this. Self-publishing is another route to take; if you print 5000 softcover books for about $10K and then sell them for $29.99 each, well, do the math :-) Of course, there's a huge downside - you have to do a ton of work. You have to provide a camera-ready PDF file, set up a web site to take orders, do all the marketing and promo, have a place to keep 10 cubic feet of books, handle order fulfillment (i.e., lugging 10-20 copies a day down to the post office), and so on. But it's a thought... Yours, Tom
On August 9, 2005 06:38 am, Tom Copeland wrote: > Just a thought on this. Self-publishing is another route to take; if > you print 5000 softcover books for about $10K and then sell them for > $29.99 each, well, do the math :-) > > Of course, there's a huge downside - you have to do a ton of work. You > have to provide a camera-ready PDF file, set up a web site to take > orders, do all the marketing and promo, have a place to keep 10 cubic > feet of books, handle order fulfillment (i.e., lugging 10-20 copies a > day down to the post office), and so on. Two other downside factors: distributing network and editing. An individual may know their material but that has nothingto do with knowing how to write and having the means to get it out to the people who want to read it.
On Tue, Aug 09, 2005 at 12:52:29AM -0400, Rick Morris wrote:
> got going for it). Thus secondly, there is the depressing observation
> that the majority of developers haven't a clue what the relational model
> is really good for. They want to wring every possible bit of speed out
> of a database while piling all sorts of constraints into application
> space. That's pretty much the norm for most open source applications I
> have seen.
At the risk of sending your depression into total free-fall, I'll
note that many proprietary applications, including those developed
for Oracle, suffer this problem as well. Programmers who understand
a database-backed system are much less common than they should be.
And you're _really_ hosed if the person doing the hiring doesn't
understand relational systems: you end up with a whole raft of
programmers, none of whom has had a Date with the clue stick. (Sorry
about that, folks. It was irresistable.) To the extent that's true,
however, those programmers also have practically no incentive to move
from MySQL, save for licensing. And, as one of the PHP folks said to
me for the second year in a row, "Why would I move? MySQL does what
I need, and when I need to go bigger, I use Oracle." Apparently,
"But Postgres is the one that's free," isn't an answer. Go know.
> without question (Any X is as good as anyone else's X). Maybe it's a
> good idea to put out some material explaining how much difference there
> can be in two different implementations of such a thing as
> (views/triggers/procedures/constraints), and the pitfalls that can
> happen because of this.
Given the troubles IBM has, with all their advertising and white
paper money, making such arguments against Oracle, I don't think that
will be a rich seam. I agree that this is one of the things I'm
troubled about in MySQL's case: they now can justly claim that they
have transactions (well, most of the time), that they have a strict
implementation of SQL (well, if you turn it on), that they have
stored procedures (pretty much), that they support subqueries (in
some positions) &c. For a long time, I considered MySQL an
annoyance, because one was always having to discuss this toy in the
same breath as Postgres. But while Pg has been busy polishing real
industrial-grade features, MySQL has been _marketing_ themselves as
industrial-grade. And since the people who read _Network World_, who
are unfortunately also often the people in charge of IT procurement
budgets, don't know the difference (and probably never will) between
"subselects in some cases" and "subselects" (for instance), I think
our problem is about to get harder.
That isn't to say that (for instance) the 8.1 features aren't
welcome, nor even that I don't appreciate what the difference is.
But a year ago, I was bearish on the survival of MySQL through the
MySQL AB funding period. I'm not any more, and I suppose that's why
I'm made nervous.
A
--
Andrew Sullivan | ajs@crankycanuck.ca
I remember when computers were frustrating because they *did* exactly what
you told them to. That actually seems sort of quaint now.
--J.D. Baldwin
On Tue, 2005-08-09 at 07:02 -0400, Robert Bernier wrote: > On August 9, 2005 06:38 am, Tom Copeland wrote: > > Just a thought on this. Self-publishing is another route to take; if > > you print 5000 softcover books for about $10K and then sell them for > > $29.99 each, well, do the math :-) > > > > Of course, there's a huge downside - you have to do a ton of work. You > > have to provide a camera-ready PDF file, set up a web site to take > > orders, do all the marketing and promo, have a place to keep 10 cubic > > feet of books, handle order fulfillment (i.e., lugging 10-20 copies a > > day down to the post office), and so on. > > Two other downside factors: distributing network and editing. An individual may know their material but that has nothingto do with knowing how to write and having the means to get it out to the people who want to read it. Yup, quite true. There are a lot of pitfalls... but the upside is pretty good too :-) Yours, tom
Andrew Sullivan wrote: >On Tue, Aug 09, 2005 at 12:52:29AM -0400, Rick Morris wrote: > > > >>got going for it). Thus secondly, there is the depressing observation >>that the majority of developers haven't a clue what the relational model >>is really good for. They want to wring every possible bit of speed out >>of a database while piling all sorts of constraints into application >>space. That's pretty much the norm for most open source applications I >>have seen. >> >> > >At the risk of sending your depression into total free-fall, I'll >note that many proprietary applications, including those developed >for Oracle, suffer this problem as well. Programmers who understand >a database-backed system are much less common than they should be. >And you're _really_ hosed if the person doing the hiring doesn't >understand relational systems: you end up with a whole raft of >programmers, none of whom has had a Date with the clue stick. (Sorry >about that, folks. It was irresistable.) > heh... > To the extent that's true, >however, those programmers also have practically no incentive to move >from MySQL, save for licensing. And, as one of the PHP folks said to >me for the second year in a row, "Why would I move? MySQL does what >I need, and when I need to go bigger, I use Oracle." Apparently, >"But Postgres is the one that's free," isn't an answer. Go know. > > Well, I have spent years debating with the PHP folks at forums.devshed.com, and one of the best replies I have found to that argument is "Have you ever tried going from MySQL to Oracle? Moving from MySQL to Oracle is a LOT more painful than moving from PostgreSQL to Oracle". > > >>without question (Any X is as good as anyone else's X). Maybe it's a >>good idea to put out some material explaining how much difference there >>can be in two different implementations of such a thing as >>(views/triggers/procedures/constraints), and the pitfalls that can >>happen because of this. >> >> > >Given the troubles IBM has, with all their advertising and white >paper money, making such arguments against Oracle, I don't think that >will be a rich seam. I agree that this is one of the things I'm >troubled about in MySQL's case: they now can justly claim that they >have transactions (well, most of the time), that they have a strict >implementation of SQL (well, if you turn it on), that they have >stored procedures (pretty much), that they support subqueries (in >some positions) &c. For a long time, I considered MySQL an >annoyance, because one was always having to discuss this toy in the >same breath as Postgres. But while Pg has been busy polishing real >industrial-grade features, MySQL has been _marketing_ themselves as >industrial-grade. And since the people who read _Network World_, who >are unfortunately also often the people in charge of IT procurement >budgets, don't know the difference (and probably never will) between >"subselects in some cases" and "subselects" (for instance), I think >our problem is about to get harder. > >That isn't to say that (for instance) the 8.1 features aren't >welcome, nor even that I don't appreciate what the difference is. >But a year ago, I was bearish on the survival of MySQL through the >MySQL AB funding period. I'm not any more, and I suppose that's why >I'm made nervous. > > Yes, I am convinced that we will never have a *majority* of developers who know enough and care enough about PostgreSQL's serious features. All I care about is enough to keep PostgreSQL a going concern. With that in mind,it seems the real question is one of strategy, rather than focusing on feature firepower: - MySQL understands the playbook well, having taken a lesson or two from Mr. Gates himself, I believe. They clearly understand that "saying it makes it so" in this market. Also, they have (sorry to say) a much more catchy name (although cloyingly cutesy). So PostgreSQL will have a hard fight trying to win the popularity award. - But, like you said, there are quite a few companies taking a more serious interest in PostgreSQL. This is different from winning the single developer. Think about the strategic difference. We all know that many large companies use PostgreSQL for internal projects. Verio comes to mind, here in Florida. I liken it to the number of companies that use FreeBSD instead of Linux (also something you will find at Verio). FreeBSD will never be as popular as Linux, but there are many companies who use it extensively because they have found it suits their needs perfectly; they just don't waste a lot of breath *advertising* that fact. Companies that use Linux tend to shout it from the rooftops because it makes good press. (Please... not trying to start a FreeBSD-vs-Linux war, just noting a strategic similarity). Note that there is much *less* feature distinction between it and Linux than between My/Pg, but still, FreeBSD will always be around, because it has achieved a sort of balance in the market. Individual users tend to use Linux, but corporate deployments might have 1000 FreeBSD boxes that no one ever hears about. I think that again is very similar to the PostgreSQL use cases. - Thus, taking the above under advisement, it would seem best not to fight MySQL on their ground, nor to spend much time on feature distinction, but to forge more serious relationships with organizations who have serious needs. The guys that don't have serious data management needs will never perceive a real reason to move to PostgreSQL, but those who have been bitten a few times (as I was once upon a time before moving to PostgreSQL ;-) ) are a lot more open to the possibility. I think the ratio between FreeBSD/Linux has reached a sort of self-correcting balance that will go on for a long time. How can we reach that sort of balance with PostgreSQL/MySQL? - In debunking the FUD, there are plenty of independent people whose rants are more successful than any whitepaper put out by postgresql.org could be. The "MySQL gotchas" page is a perfect example. So I agree; best not to waste breath putting out "official" arguments against MySQL or exposing this or that flaw in implementation. The independent guys seem to carry more weight, because of perceived impartiality. The best PR, it seems to me, is the somewhat jovial relationship between FreeBSD and Linux (Theo de Raadt notwithstanding). Kind of an implied "we're working on the same things" approach, even though we know there is a big difference). But again, I am still not as worried as you seem to be. I have been involved in several small-to-medium project rollouts at various companies since 2000, and every single time I had no difficulty convincing the boss to go with PostgreSQL. Ditto with several friends of mine. I think there is a lot more of that out there than you realize. On these sorts of projects, you win over the head developer, not the IT procurement guy. That's the ticket ;-). Regards, Rick Morris
On Tue, Aug 09, 2005 at 09:49:28AM -0400, Rick Morris wrote:
>
> - Thus, taking the above under advisement, it would seem best not to
> fight MySQL on their ground, nor to spend much time on feature
> distinction, but to forge more serious relationships with organizations
> who have serious needs.
I think this is a capital idea. It seems this is a place to focus.
A
--
Andrew Sullivan | ajs@crankycanuck.ca
Information security isn't a technological problem. It's an economics
problem.
--Bruce Schneier
Folks: > > Just a thought on this. Self-publishing is another route to take; if > > you print 5000 softcover books for about $10K and then sell them for > > $29.99 each, well, do the math :-):-) If anyone on this list wants to write a PostgreSQL book, and has the time, I can get you a publisher in no time flat. -- Josh Berkus Aglio Database Solutions San Francisco
Andrew, > 1. The first item I think bears mention is the number of occasions I > had people ask me whether the project is losing steam, or "losing > out" to MySQL, or being no threat to Oracle. Re: MySQL, you may have talked to different people than me. However, my experience was that the people who were agitated about "competition" with MySQL were PostgreSQL community members, not third parties. That is, there was *vastly* more talk of MySQL at the PostgreSQL BOF than I heard in the hallways. Probably Bruce and I just should have cut it off sooner ... -- Josh Berkus Aglio Database Solutions San Francisco
On Tue, Aug 09, 2005 at 07:34:20AM -0400, Andrew Sullivan wrote: > On Tue, Aug 09, 2005 at 12:52:29AM -0400, Rick Morris wrote: > > > got going for it). Thus secondly, there is the depressing observation > > that the majority of developers haven't a clue what the relational model > > is really good for. They want to wring every possible bit of speed out > > of a database while piling all sorts of constraints into application > > space. That's pretty much the norm for most open source applications I > > have seen. > > At the risk of sending your depression into total free-fall, I'll > note that many proprietary applications, including those developed > for Oracle, suffer this problem as well. Programmers who understand > a database-backed system are much less common than they should be. > And you're _really_ hosed if the person doing the hiring doesn't > understand relational systems: you end up with a whole raft of > programmers, none of whom has had a Date with the clue stick. (Sorry > about that, folks. It was irresistable.) To the extent that's true, > however, those programmers also have practically no incentive to move > from MySQL, save for licensing. And, as one of the PHP folks said to > me for the second year in a row, "Why would I move? MySQL does what > I need, and when I need to go bigger, I use Oracle." Apparently, > "But Postgres is the one that's free," isn't an answer. Go know. > Lack of understanding of relational modelling is a big problem. People design there databases w/application centric enforcements which play well on mysql but violates Date's central rule about relational databases: the integrity of the data is defined in the database and cannot be circumvented by applications. Learn, Educate. Learn More. Educate More. --elein > > without question (Any X is as good as anyone else's X). Maybe it's a > > good idea to put out some material explaining how much difference there > > can be in two different implementations of such a thing as > > (views/triggers/procedures/constraints), and the pitfalls that can > > happen because of this. > > Given the troubles IBM has, with all their advertising and white > paper money, making such arguments against Oracle, I don't think that > will be a rich seam. I agree that this is one of the things I'm > troubled about in MySQL's case: they now can justly claim that they > have transactions (well, most of the time), that they have a strict > implementation of SQL (well, if you turn it on), that they have > stored procedures (pretty much), that they support subqueries (in > some positions) &c. For a long time, I considered MySQL an > annoyance, because one was always having to discuss this toy in the > same breath as Postgres. But while Pg has been busy polishing real > industrial-grade features, MySQL has been _marketing_ themselves as > industrial-grade. And since the people who read _Network World_, who > are unfortunately also often the people in charge of IT procurement > budgets, don't know the difference (and probably never will) between > "subselects in some cases" and "subselects" (for instance), I think > our problem is about to get harder. > > That isn't to say that (for instance) the 8.1 features aren't > welcome, nor even that I don't appreciate what the difference is. > But a year ago, I was bearish on the survival of MySQL through the > MySQL AB funding period. I'm not any more, and I suppose that's why > I'm made nervous. > > A > > -- > Andrew Sullivan | ajs@crankycanuck.ca > I remember when computers were frustrating because they *did* exactly what > you told them to. That actually seems sort of quaint now. > --J.D. Baldwin > > ---------------------------(end of broadcast)--------------------------- > TIP 1: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly >
On Tue, Aug 09, 2005 at 08:47:42AM -0700, Josh Berkus wrote:
> Re: MySQL, you may have talked to different people than me. However, my
> experience was that the people who were agitated about "competition" with
> MySQL were PostgreSQL community members, not third parties. That is, there
That may be the case. This was an impression formed mostly at the
booth, where I was fielding questions from people who were explicitly
asking me whether Postgres was going to be around to compete with
MySQL.
Explaining how licensing and publicity worked was helpful in this
context. But also, given that this is a discussion I sometimes find
myself having in my day job, it's an issue I'm particularly sensitive
to. (It's also why I was so happy to hear Aaron Thul's (sp?)
presentation.)
A
--
Andrew Sullivan | ajs@crankycanuck.ca
In the future this spectacle of the middle classes shocking the avant-
garde will probably become the textbook definition of Postmodernism.
--Brad Holland
elein wrote: > > >Lack of understanding of relational modelling is a big problem. >People design there databases w/application centric enforcements >which play well on mysql but violates Date's central rule about >relational databases: the integrity of the data is defined in the >database and cannot be circumvented by applications. > > > I hate to sound extremely pessimistic here but I don't really think that this is the problem, having had this discussion with many people who should know better or at least have CS degrees (which I don't have). This also serves to explain the success of MySQL. The problem is pervasive in both proprietary and open source apps because the developer sees the RDBMS merely as a simple data store for his/her application. The idea that the database might serve several apps never occurs to them. Hence it makes real sense to put as much logic into the application level as possible because if you put it in the database, then that makes more work if you have to switch RDBMS's. So MySQL works fine when people see it as basically an SQL interface for a set of application resources possibly residing on another system. It is "good enough" for this. But the problem is only masked in this situation. The RDBMS is an incredibly powerful tool for application integration. However, this is often not in the best interests of the developers who would rather lock you into their program. Therefore an app-specific database design makes economic sense. The only people that get hurt by this are the customers. So again, making connections to businesses so that we get a chance to explain ourselves is the only way we will be able to compete. Best Wishes, Chris Travers Metatron Technology Consulting
elein wrote: > On Tue, Aug 09, 2005 at 07:34:20AM -0400, Andrew Sullivan wrote: > >>On Tue, Aug 09, 2005 at 12:52:29AM -0400, Rick Morris wrote: >> >> >>>got going for it). Thus secondly, there is the depressing observation >>>that the majority of developers haven't a clue what the relational model >>>is really good for. They want to wring every possible bit of speed out >>>of a database while piling all sorts of constraints into application >>>space. That's pretty much the norm for most open source applications I >>>have seen. >> >>At the risk of sending your depression into total free-fall, I'll >>note that many proprietary applications, including those developed >>for Oracle, suffer this problem as well. Programmers who understand >>a database-backed system are much less common than they should be. >>And you're _really_ hosed if the person doing the hiring doesn't >>understand relational systems: you end up with a whole raft of >>programmers, none of whom has had a Date with the clue stick. (Sorry >>about that, folks. It was irresistable.) To the extent that's true, >>however, those programmers also have practically no incentive to move >>from MySQL, save for licensing. And, as one of the PHP folks said to >>me for the second year in a row, "Why would I move? MySQL does what >>I need, and when I need to go bigger, I use Oracle." Apparently, >>"But Postgres is the one that's free," isn't an answer. Go know. >> > > Lack of understanding of relational modelling is a big problem. > People design there databases w/application centric enforcements > which play well on mysql but violates Date's central rule about > relational databases: the integrity of the data is defined in the > database and cannot be circumvented by applications. > > Learn, Educate. Learn More. Educate More. Agreed. That is actually the fun part. I have personally been responsible for educating at least a few Devshed-ers on why the relational model matters, and why the word 'relational' is not directly referring (heh...) to foreign key relationships. Speaking of education, I think the coolest thing we can do is put together a library of examples that show how to save time and effort with the relational model, while increasing the value of your data. "Why the Relational Model Saves you Time", or something like that. I say this because that's exactly the sort of resource I would have *loved* when starting out to learn PostgreSQL and really grok the relational model. There are so many areas where a DBMS like PostgreSQL can make a difference, but developers don't even have enough of a handle on the concepts to know this. While http://techdocs.postgresql.org is a great resource, it is (of course) focused more on many different technical aspects of PostgreSQL, rather than teaching the overall concepts, with simple, clear examples. Back when I was learning this stuff, I had to spend forever just searching for an example of how to make a multi-table view updatable. It took me just as long before this to see why this was an incredibly useful thing. Most documentation and articles I have seen start from the technical term or concept, rather than starting from the use case and explaining what technical term or concept applies. Some titles that might catch attention (hope these aren't too cute): - Automation: Hand off your Headaches to the Database - Tying Together Multiple Applications through the Database - Making Sure X Happens Every Time Y Applies - Making the Database Speak your Language with Custom Datatypes and Operators. - You Should Never, Ever Have to Repeat Yourself! - What is a Relation and Why does that Matter? - Dealing with Time-Sensitive Data etc... //cough... Of course, In saying this, I suppose it would be hypocritical of me not to put up my hand and volunteer some time on this. I can definitely offer some good material for PHP developers and people designing business-oriented web apps. Regards, Rick Morris - > > --elein > > >>>without question (Any X is as good as anyone else's X). Maybe it's a >>>good idea to put out some material explaining how much difference there >>>can be in two different implementations of such a thing as >>>(views/triggers/procedures/constraints), and the pitfalls that can >>>happen because of this. >> >>Given the troubles IBM has, with all their advertising and white >>paper money, making such arguments against Oracle, I don't think that >>will be a rich seam. I agree that this is one of the things I'm >>troubled about in MySQL's case: they now can justly claim that they >>have transactions (well, most of the time), that they have a strict >>implementation of SQL (well, if you turn it on), that they have >>stored procedures (pretty much), that they support subqueries (in >>some positions) &c. For a long time, I considered MySQL an >>annoyance, because one was always having to discuss this toy in the >>same breath as Postgres. But while Pg has been busy polishing real >>industrial-grade features, MySQL has been _marketing_ themselves as >>industrial-grade. And since the people who read _Network World_, who >>are unfortunately also often the people in charge of IT procurement >>budgets, don't know the difference (and probably never will) between >>"subselects in some cases" and "subselects" (for instance), I think >>our problem is about to get harder. >> >>That isn't to say that (for instance) the 8.1 features aren't >>welcome, nor even that I don't appreciate what the difference is. >>But a year ago, I was bearish on the survival of MySQL through the >>MySQL AB funding period. I'm not any more, and I suppose that's why >>I'm made nervous. >> >>A >> >>-- >>Andrew Sullivan | ajs@crankycanuck.ca >>I remember when computers were frustrating because they *did* exactly what >>you told them to. That actually seems sort of quaint now. >> --J.D. Baldwin >> >>---------------------------(end of broadcast)--------------------------- >>TIP 1: if posting/reading through Usenet, please send an appropriate >> subscribe-nomail command to majordomo@postgresql.org so that your >> message can get through to the mailing list cleanly >> > > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Have you searched our list archives? > > http://archives.postgresql.org > >
> -----Original Message----- > From: pgsql-advocacy-owner@postgresql.org > [mailto:pgsql-advocacy-owner@postgresql.org]On Behalf Of Rick Morris > Sent: Tuesday, August 09, 2005 12:04 PM > To: PostgreSQL Advocacy > Subject: Re: [pgsql-advocacy] Thoughs after discussions at OSCON > > > Agreed. That is actually the fun part. I have personally been > responsible for educating at least a few Devshed-ers on why the > relational model matters, and why the word 'relational' is > not directly > referring (heh...) to foreign key relationships. > Aye, I've benefitted a few times from Rick's input over at devshed, and while there's more activity in the MySQL forums I have seen an increase (small, but noticeable) in people looking to migrate to PostgreSQL from MySQL especially since the windows version was released. -b
ajs@crankycanuck.ca (Andrew Sullivan) writes:
> On Tue, Aug 09, 2005 at 09:49:28AM -0400, Rick Morris wrote:
>>
>> - Thus, taking the above under advisement, it would seem best not to
>> fight MySQL on their ground, nor to spend much time on feature
>> distinction, but to forge more serious relationships with organizations
>> who have serious needs.
>
> I think this is a capital idea. It seems this is a place to focus.
Absolutely.
The same would be true of "fighting Oracle on their ground."
One thought suggested by someone outside of "the community" is that
one of the things that the various "more popular" systems have gotten
'right' is the notion of having a "Foo Press" that is responsible for
deploying books about their systems.
This is true for various values of "Foo" including the members of the
set {"Microsoft", "Oracle", "MySQL"}.
Contrast with some systems that people hardly ever hear of...
- There is exactly ONE book available on MaxDB, by No Starch Press
- There is exactly ONE book available on SQLite, by New Riders
- APress has ONE book on Firebird...
- It is not comforting that O'Reilly only has one PostgreSQL book :-(
One "capital idea" might well be to establish a "PostgreSQL Press,"
whether in cooperation with an existing publisher or otherwise. The
timing of this would be pretty critical; we're likely at a fairly
critical point in time for it now, for it to happen or not.
There's immediate room for a multiplicity of titles, including:
- Publishing a "dead tree" edition of the 'official' docs
- Perhaps a work on Performance Tuning (I have had some notes
collecting for a couple years, albeit with NOTHING added in
at least a year :-().
- Something on Slony-I
There are enough interested folk, in my view, for at least those three
as _obvious_ first works.
FYI, nothing is new...
Back before MySQL had anything to do with SAPDB, I commented on the
SAPDB mailing list the VITAL need for literature on it; the paucity of
interest in MaxDB is quite consistent with how recent it was when a
book was first released on the subject...
<http://lists.mysql.org/maxdb/2178>
-------------------------------------------------------------------------------
> Thanks. I've downloaded and printed the online docs. I was just
> wondering/hoping (if) there were other publications out there. It
> would be great if SAP could develop 'Oracle Press' like
> publications, that were written by developers, that could take us
> into the 'subtle' nuances of the system.
That only is likely to happen if/when there appears to be a market for
selling such publications.
Oracle has clearly _built_ a market for certification and
documentation; that has involved spending a _lot_ of money on what
might be called "business development." It's not evident that it is
in SAP's interests to spend that kind of money on SAP-DB; they aren't
likely to see billions of dollars in licensing revenues walking in
their door as a side-effect [or as a prime effect!].
It may be plausible to convince some publisher [M+T, New Riders,
O'Reilly come to mind] to publish something, were SAP to pay something
in; it's still not obvious that this is a good way for SAP to spend
their money.
What _would_ be nice would be if there was some documentation in
"libre" form that could readily be published in "dead tree" form where
it would cost ~$20 USD to get the docs printed for a single copy. I'm
just not sure what "dead tree publishers" are readily available to do
that sort of thing.
-------------------------------------------------------------------------------
--
(format nil "~S@~S" "cbbrowne" "acm.org")
http://www.ntlug.org/~cbbrowne/sap.html
Rules of the Evil Overlord #78. "I will not tell my Legions of Terror
"And he must be taken alive!" The command will be: ``And try to take
him alive if it is reasonably practical.''"
<http://www.eviloverlord.com/>
Chris Travers wrote:
> The problem is pervasive in both proprietary and open source apps
> because the developer sees the RDBMS merely as a simple data store for
> his/her application. The idea that the database might serve several
> apps never occurs to them. Hence it makes real sense to put as much
> logic into the application level as possible because if you put it in
> the database, then that makes more work if you have to switch RDBMS's.
>
Many people do develop that way: the database is just a place to throw
bits and get 'em back later. I don't think we're going to change that
any time soon.
I think in a way, one of our allies is SQLLite, which sounds a little
strange. However, it's very simple, gets the job done (that is, puts
bits on physical storage and retrieves them), and doesn't claim to be
something it's not. MySQL gets on our nerves because they equate their
capabilities with those of PostgreSQL.
SQLLite has also recently replaced MySQL as the "built-in" database
layer for PHP. Hopefully that will cause a lot of people will associate
MySQL with SQLLite rather than PostgreSQL.
Regards,
Jeff Davis
On 8/9/05, Rick Morris <rick@brainscraps.com> wrote: > on the concepts to know this. While http://techdocs.postgresql.org is a > great resource, it is (of course) focused more on many different > technical aspects of PostgreSQL, rather than teaching the overall [...] I've been using PostgreSQL for a while now, and this is the place and the moment when I learned about techdocs.postgresql.org Indeed it is a great a resource. The only problem with it is that I didn't know about its existence until now. Probably I wasn't paying attention ;) but then again -- I think it should be more advertised/linked to. As far as providing documents about more generic concepts -- I think it is a great idea from web visibility point of view. I remember a while back when I was co-developing application which also used MySQL, I used PostgreSQL's documentation to learn SQL. It was later when I learned PostgreSQL, but I think it was important that my first contact with SQL was through PostgreSQL (even though I used it on MySQL back then ;)). These things "stay" and later when I was looking for something SQLish I looked it up in PostgreSQL's docs. And then I was englihtened and started using PostgreSQL. :) I think good documentation, especially basic one is important -- people will return to it, like I did. And at some point they might want to try the real thing. :) As far as convincing people that using relational model is better and that they should switch over -- its hard and sometimes close to impossible. Imagine some huge amount of logs in one table and telling people that given proper tools (good relational database) you could reduce dataset size (normalisation!), automatically pratition the data (inheritance, future table partitioning), while maintaining access to old format of data (views) and giving quicker access to some summaries (materialised views). What will be the result? Normalisation? You mean like instead of one table five tables? No no no, the data will be surely corrupted. And accessing it will be slow and you'll have to write joins like this: select * from tab_a f,tab_b g,tab_c h,tab_d i,tab_e j where f.id=g.someid and ... and besides it will take ages to deploy. Seriousley, I think the best approach to advertise PostgreSQL is through third-party applications. Like SpamAssassin. If PostgreSQL can prove that it will compete easily with any other RDBMS people will have more faith in it. "It is doable, they could do it with SpamAssassin, we can do it with our system, its worth to try!". A good documentation for such cases would be also very valuable. Like "Designing PostgreSQL schema for SpamAssassin" where would be written why some things are created this and that way, so people could learn about PostgreSQLs features and drawbacks (its good to know them from the start!) simply by reading some other projects docs. Regards and good night, Dawid
On August 9, 2005 03:51 pm, Chris Browne wrote:
> One thought suggested by someone outside of "the community" is that
> one of the things that the various "more popular" systems have gotten
> 'right' is the notion of having a "Foo Press" that is responsible for
> deploying books about their systems.
>
> This is true for various values of "Foo" including the members of the
> set {"Microsoft", "Oracle", "MySQL"}.
>
> Contrast with some systems that people hardly ever hear of...
> - There is exactly ONE book available on MaxDB, by No Starch Press
> - There is exactly ONE book available on SQLite, by New Riders
> - APress has ONE book on Firebird...
> - It is not comforting that O'Reilly only has one PostgreSQL book :-(
>
> One "capital idea" might well be to establish a "PostgreSQL Press,"
> whether in cooperation with an existing publisher or otherwise. The
> timing of this would be pretty critical; we're likely at a fairly
> critical point in time for it now, for it to happen or not.
I agree that a centralized clearing house ala postgres community for published works is a very good idea.
I've been writing for a number of years now with my own contacts into a number of publishing firms. I know a number of
you,even the ones doing it in secret, currently have books on the back burner. What's stopping me to publishing is the
samereason that's affecting you i.e. time. The profit motive only kicks in 'after' you've written the book, otherwise
howdoes one survive? I would imagine companies like Oracle etc. have a subsidy system that makes it possible for
authorsto earch income while they write (does anybody know for sure?).
On Tue, 2005-08-09 at 18:09 -0400, Robert Bernier wrote: > I would imagine companies like Oracle etc. have a subsidy system that makes it possible for authors > to earch income while they write (does anybody know for sure?). On my last govt contract, the employees of our prime contractor said they got a $5K bonus for getting a book published. Seems that such publications are good resume fodder when applying for said govt contracts. Yours, Tom
On Tuesday 09 August 2005 18:09, Robert Bernier wrote:
> I would imagine companies like Oracle etc. have a subsidy
> system that makes it possible for authors to earch income while they write
> (does anybody know for sure?).
>
Certainly some publishers will give you an advance on royalties if they have a
good idea of how well a given book will do on the market. It's not a point
to be lost that these "foo press" publishers all came to market some time
after there were already numerous books that all sold well already on the
market for "foo"; I'm sure every one of them has forecast models already in
place for the books they are doing.
--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
> http://techdocs.postgresql.org is > > a great resource, it is (of course) focused more on many different > > technical aspects of PostgreSQL, rather than teaching the overall > [...] > > I've been using PostgreSQL for a while now, and this is the > place and the moment when I learned about > techdocs.postgresql.org Indeed it is a great a resource. The > only problem with it is that I didn't know about its > existence until now. Probably I wasn't paying attention ;) > but then again -- I think it should be more advertised/linked to. I believe Gevik is working on integrating it with the main website, which should make it easier to find. //Magnus
On another note for followup after OSCON 2005... We probably ought to start collecting the slide documents together on <http://techdocs.postgresql.org/>, right? -- let name="cbbrowne" and tld="cbbrowne.com" in String.concat "@" [name;tld];; http://cbbrowne.com/info/linux.html PASCAL is not a language. It was an experiment combining the flexibilty of C with that of a drug-crazed penguin. It is also the 'language' of choice of many CS professors who aren't up to handling REAL programming. Hence, it is not a language.
I have a collection of slides from previous OSCONs on
General Bits/Tidbits. But I have been neglectful (so far) of sending
email to OSCON-05 presenters to request copies or links to
their talks.
Please consider this a request for copies and links to
OSCON-05 presentations. I will post them with the other year's
OSCON presentations on www.varlena.com/GeneralBits/Tidbits.
--elein
--------------------------------------------------------------
elein@varlena.com Varlena, LLC www.varlena.com
(510)655-2584(o) (510)543-6079(c)
PostgreSQL Consulting, Support & Training
PostgreSQL General Bits http://www.varlena.com/GeneralBits/
--------------------------------------------------------------
AIM: varlenallc Yahoo: AElein Skype: varlenallc
--------------------------------------------------------------
I have always depended on the [QA] of strangers.
On Fri, Aug 12, 2005 at 02:56:00PM -0400, Chris Browne wrote:
> On another note for followup after OSCON 2005...
>
> We probably ought to start collecting the slide documents together on
> <http://techdocs.postgresql.org/>, right?
> --
> let name="cbbrowne" and tld="cbbrowne.com" in String.concat "@" [name;tld];;
> http://cbbrowne.com/info/linux.html
> PASCAL is not a language. It was an experiment combining the
> flexibilty of C with that of a drug-crazed penguin. It is also the
> 'language' of choice of many CS professors who aren't up to handling
> REAL programming. Hence, it is not a language.
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings
>
On Friday 12 August 2005 14:56, Chris Browne wrote: > On another note for followup after OSCON 2005... > > We probably ought to start collecting the slide documents together on > <http://techdocs.postgresql.org/>, right? Yep... I'm behind on email, but I have your submission and a few others already on the site. Anyone who hasn't sent one in and wants to please send me an email asap. I'll probably update the main techdocs page tonight or tommorrow once I get mine up there :-P -- Robert Treat Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
All my presentations are on my home page with information on where the talk was given, and the date. --------------------------------------------------------------------------- elein wrote: > I have a collection of slides from previous OSCONs on > General Bits/Tidbits. But I have been neglectful (so far) of sending > email to OSCON-05 presenters to request copies or links to > their talks. > > Please consider this a request for copies and links to > OSCON-05 presentations. I will post them with the other year's > OSCON presentations on www.varlena.com/GeneralBits/Tidbits. > > --elein > -------------------------------------------------------------- > elein@varlena.com Varlena, LLC www.varlena.com > (510)655-2584(o) (510)543-6079(c) > > PostgreSQL Consulting, Support & Training > > PostgreSQL General Bits http://www.varlena.com/GeneralBits/ > -------------------------------------------------------------- > AIM: varlenallc Yahoo: AElein Skype: varlenallc > -------------------------------------------------------------- > I have always depended on the [QA] of strangers. > > > > On Fri, Aug 12, 2005 at 02:56:00PM -0400, Chris Browne wrote: > > On another note for followup after OSCON 2005... > > > > We probably ought to start collecting the slide documents together on > > <http://techdocs.postgresql.org/>, right? > > -- > > let name="cbbrowne" and tld="cbbrowne.com" in String.concat "@" [name;tld];; > > http://cbbrowne.com/info/linux.html > > PASCAL is not a language. It was an experiment combining the > > flexibilty of C with that of a drug-crazed penguin. It is also the > > 'language' of choice of many CS professors who aren't up to handling > > REAL programming. Hence, it is not a language. > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 5: don't forget to increase your free space map settings > > > > ---------------------------(end of broadcast)--------------------------- > TIP 5: don't forget to increase your free space map settings > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
> We need to look at all the things that MySQL is doing right, because
> our technical superiority alone is not going to save us. I've also
> started to think lately that our BSD license may be an even greater
> asset than our feature set. More about that later in a PP rant. :)
> our technical superiority alone is not going to save us. I've also
> started to think lately that our BSD license may be an even greater
> asset than our feature set. More about that later in a PP rant. :)
I think that commercial interest in PostgreSQL is really picking up rather nicely. PostgreSQL is generally accepted by techies (of all varieties) to be superior to MySQL and an increasingly worthy substitute for Oracle, DB2, & SQL Svr in many situations.
Where MySQL has been "superior" is in marketing propaganda. As more commercial companies, with marketing dollars, partner with the Postgres community... I think the PG community will blow by MySQL in the coming eighteen months to become the "Most Advanced" AND the "Most Popular". Additionally, these commercial companies should sponsor ever more full time PG community superstars so the pace of new enterprise-class features, in core PG, continues to quicken.
--Denis Lussier
Chief Architect & Chairman
Denis Lussier wrote: > > We need to look at all the things that MySQL is doing right, because > > our technical superiority alone is not going to save us. I've also > > started to think lately that our BSD license may be an even greater > > asset than our feature set. More about that later in a PP rant. :) > > I think that commercial interest in PostgreSQL is really picking up > rather nicely. PostgreSQL is generally accepted by techies (of all > varieties) to be superior to MySQL and an increasingly worthy > substitute for Oracle, DB2, & SQL Svr in many situations. > > Where MySQL has been "superior" is in marketing propaganda. As more > commercial companies, with marketing dollars, partner with the > Postgres community... I think the PG community will blow by MySQL in > the coming eighteen months to become the "Most Advanced" AND the "Most > Popular". Additionally, these commercial companies should sponsor > ever more full time PG community superstars so the pace of new > enterprise-class features, in core PG, continues to quicken. One point I may have mentioned before is that although many people have voiced concern over MySQL gaining enough developer marketshare to overtake PostgreSQL, I actually see the opposite happening today. MySQL did a number of things right, and in these areas I believe we have already surpassed them. I worked with PostgreSQL 6.5 for a while and I found it a real pain to work with during the prototyping phase. I couldn't just make a chance and test things out, so I did all my prototyping and schema review in MySQL and then once I had my design down the way I wanted it, would reimpliment it in PostgreSQL. One of the things MySQL did right early on was focus on user experience. Today, PostgreSQL is far easier to use (IMO) that MySQL and has a choice of far more powerful front-ends than MySQL. I personally think that MySQL's current market standing is supported primarily by two factors: marketing and market inertia. Instead our market standing is based on a rapid pace of development (far outpacing MySQL) and community involvement. We are not at risk of being relegating to a niche role. As evidence, I might point out that they used to be critical of us because we were adding features too fast (not really noting that the features were parts of the relevant standards). So the question becomes "How do we attack MySQL?" rather than "How do we defend against MySQL?" I think that there are several things that can be done. The first is to help port MySQL-based FOSS projects to PostgreSQL. My company (http://www.metatrontech.com) offers this service for such applications and are working on developing porting environments which could be re-used in this process where database abstraction layers were not used to start with. These porting environments will be modular and released to the community. This is important because the real advantage that a customers get with PostgreSQL is that they can use the database to integrate applications and analyze data across application boundaries. Triggers and views can provide further integration options which will not provide the same level of power in MySQL for the forseable future. Providing customers the option of running apps on PostgreSQL would be a very good start. From there, it will be much easier to make sure that PostgreSQL is the default database manager supported by web hosting provider. The web-hosting/ecommerce market is a very important market for us to attack, not just because of MySQL, but also because the alternative often is MS SQL Server. Attacking MySQL in this market is therefore also moving up to further threaten MS SQL in one of their markets. The second area is relatively obvious and we are doing it anyway. MySQL got away with being light and simple but now they are trying to grow up. This means that they are both tied to bad ways of doing things due to backward compatibility requirements and are outgrowing their simplicity advantage ("Learn MySQL in 10 minutes"). We need to try to help convince people that the additional time spent learning PostgreSQL will be an advantage to them. We particularly need to reach out to FOSS projects and try to help them appreciate the benefits to building on PostgreSQL. I would note that this would be greatly accelerated by the porting of apps from MySQL to PostgreSQL. But the way to do this is to have PostgreSQL advocates get involved in FOSS projects which are MySQL only. Finally, we need to start preparing some community documentation about general database principles (not PostgreSQL specific). We need to turn ourselves into the place that Newbies should go if they want to learn about databases as a whole. I would be happy to help with this but I am not really sure how much time I can devote to that at the moment. Best Wishes, Chris Travers Metatron Technology Consulting
Am Sonntag, 14. August 2005 07:56 schrieb Chris Travers: > The web-hosting/ecommerce market is a very important market for us to > attack, not just because of MySQL, but also because the alternative > often is MS SQL Server. Attacking MySQL in this market is therefore > also moving up to further threaten MS SQL in one of their markets. IMO, the web hosting market is fairly uninteresting. And MySQL knows this, that's why they have invested so much in MaxDB, which gives them an inroad to the "enterprise" market, which has a lower public profile, but much more money.
Peter Eisentraut wrote: >Am Sonntag, 14. August 2005 07:56 schrieb Chris Travers: > > >>The web-hosting/ecommerce market is a very important market for us to >>attack, not just because of MySQL, but also because the alternative >>often is MS SQL Server. Attacking MySQL in this market is therefore >>also moving up to further threaten MS SQL in one of their markets. >> >> > >IMO, the web hosting market is fairly uninteresting. And MySQL knows this, >that's why they have invested so much in MaxDB, which gives them an inroad to >the "enterprise" market, which has a lower public profile, but much more >money. > > > On the surface, sure. But I think that there are a few reasons why it is a market that has been to date critical to MySQL's existing success. The first is that most of the applications which run on web sites (with the exception of content management systems) are directly applicable to other aspects of business. For example, the shopping cart is a simple example of a simple web app with a database backend which, as businesses can grow, can end up being integrated with other applications. Secondly, a very large number of FOSS projects out there run on hosted web sites. As long as we abandon that market to MySQL, we give them an entrenched presence. Every product has a core market on which they are dependent for their core survival. For MySQL, this is (for better or worse) the web hosting market, even though basing a web hosted application on MySQL can be problematic later as integration requirements surface. I therefore suspect that this may also be boosting SQL-Server a bit because most sites offer either MySQL or SQL Server, so customers wanting to be able to move things to a reasonable enterprise platform will choose Windows/SQL-Server over Linux/MySQL. The goal is therefore not to offer small businesses the option to run their web sites on PostgreSQL. The goal is to provide customers hoping to grow to the point that they will bring their hosting inhouse the ability to run all their web apps on a database manager which will continue to meet their needs. This is how such a strategy would help in attaching the SQL Server market. The fact that it would also be a fairly direct threat to MySQL is another side to it. As far as MaxDB, others have noted that the SAPDB codebase is very difficult to maintain and so I expect that this will prevent MaxDB from really being viable in the short run. Longer-term, however, much of this could depend on MySQL AB's income base... Best Wishes, Chris Travers Metatron Technology Consulting
On Tue, Aug 09, 2005 at 09:49:28AM -0400, Rick Morris wrote: > - But, like you said, there are quite a few companies taking a more > serious interest in PostgreSQL. This is different from winning the > single developer. Think about the strategic difference. We all know that > many large companies use PostgreSQL for internal projects. Verio comes > to mind, here in Florida. I liken it to the number of companies that use > FreeBSD instead of Linux (also something you will find at Verio). > FreeBSD will never be as popular as Linux, but there are many companies > who use it extensively because they have found it suits their needs > perfectly; they just don't waste a lot of breath *advertising* that > fact. Companies that use Linux tend to shout it from the rooftops > because it makes good press. (Please... not trying to start a > FreeBSD-vs-Linux war, just noting a strategic similarity). Note that > there is much *less* feature distinction between it and Linux than > between My/Pg, but still, FreeBSD will always be around, because it has > achieved a sort of balance in the market. Individual users tend to use > Linux, but corporate deployments might have 1000 FreeBSD boxes that no > one ever hears about. I think that again is very similar to the > PostgreSQL use cases. But ISTM that these companies that don't mention the fact that they use PostgreSQL (or FreeBSD for that matter) are doing a dis-service to the community. Many, many people default to MySQL (or Linux) because their perception is that's what everyone uses. If it were well known that there are alternatives, people would be much more likely to investigate them and make a more informed decision. While I don't understand most people that choose MySQL knowing the facts, I feel sorry for people that choose MySQL out of ignorance, because they're likely to get bit. -- Jim C. Nasby, Sr. Engineer 512-569-9461 jnasby@pervasive.com
On Tue, Aug 09, 2005 at 11:25:35AM -0700, Chris Travers wrote: > elein wrote: > >Lack of understanding of relational modelling is a big problem. > >People design there databases w/application centric enforcements > >which play well on mysql but violates Date's central rule about > >relational databases: the integrity of the data is defined in the > >database and cannot be circumvented by applications. > > > I hate to sound extremely pessimistic here but I don't really think that > this is the problem, having had this discussion with many people who > should know better or at least have CS degrees (which I don't have). > This also serves to explain the success of MySQL. > > The problem is pervasive in both proprietary and open source apps > because the developer sees the RDBMS merely as a simple data store for > his/her application. The idea that the database might serve several > apps never occurs to them. Hence it makes real sense to put as much > logic into the application level as possible because if you put it in > the database, then that makes more work if you have to switch RDBMS's. > > So MySQL works fine when people see it as basically an SQL interface for > a set of application resources possibly residing on another system. It > is "good enough" for this. Or as I like to say, MySQL is fine if you just want an SQL interface to your filesystem... > But the problem is only masked in this situation. The RDBMS is an > incredibly powerful tool for application integration. However, this is > often not in the best interests of the developers who would rather lock > you into their program. Therefore an app-specific database design makes > economic sense. > > The only people that get hurt by this are the customers. So again, > making connections to businesses so that we get a chance to explain > ourselves is the only way we will be able to compete. I think you're both correct; 90% of people who use databases don't understand how important maintaining the integrity of your data is. Tom Kyte (http://asktom.oracle.com/) harps on this all the time, and there's some articles in that website that provide some good explanations on why you want as much business logic, etc as possible in the database and not in some app server. Of course the real problem is getting this information out to people developing database apps. -- Jim C. Nasby, Sr. Engineer 512-569-9461 jnasby@pervasive.com
On Tue, Aug 09, 2005 at 01:13:54PM -0700, Jeff Davis wrote: > I think in a way, one of our allies is SQLLite, which sounds a little > strange. However, it's very simple, gets the job done (that is, puts > bits on physical storage and retrieves them), and doesn't claim to be > something it's not. MySQL gets on our nerves because they equate their > capabilities with those of PostgreSQL. > > SQLLite has also recently replaced MySQL as the "built-in" database > layer for PHP. Hopefully that will cause a lot of people will associate > MySQL with SQLLite rather than PostgreSQL. Has anyone looked at how easy it would be to migrate an application from SQLLite to PostgreSQL? I'm asking not because I think we need to try and take space away from them, but because a number of sites/projects that start out thinking "oh, this will never grow beyond a few users" end up in a lot of pain when that assumption is proven false. Livejournal is a great example of this. So it would be good if those people had a moderately painless means to migrate to PostgreSQL after they out-grew what SQLLite has to offer. Perhaps it would be worth trying to work with the SQLLite folks on this. -- Jim C. Nasby, Sr. Engineer 512-569-9461 jnasby@pervasive.com
Chris Browne wrote:
>ajs@crankycanuck.ca (Andrew Sullivan) writes:
>
>
>>On Tue, Aug 09, 2005 at 09:49:28AM -0400, Rick Morris wrote:
>>
>>
>>>- Thus, taking the above under advisement, it would seem best not to
>>>fight MySQL on their ground, nor to spend much time on feature
>>>distinction, but to forge more serious relationships with organizations
>>>who have serious needs.
>>>
>>>
>>I think this is a capital idea. It seems this is a place to focus.
>>
>>
>
>Absolutely.
>
>The same would be true of "fighting Oracle on their ground."
>
>One thought suggested by someone outside of "the community" is that
>one of the things that the various "more popular" systems have gotten
>'right' is the notion of having a "Foo Press" that is responsible for
>deploying books about their systems.
>
>This is true for various values of "Foo" including the members of the
>set {"Microsoft", "Oracle", "MySQL"}.
>
>Contrast with some systems that people hardly ever hear of...
> - There is exactly ONE book available on MaxDB, by No Starch Press
> - There is exactly ONE book available on SQLite, by New Riders
> - APress has ONE book on Firebird...
> - It is not comforting that O'Reilly only has one PostgreSQL book :-(
>
>One "capital idea" might well be to establish a "PostgreSQL Press,"
>whether in cooperation with an existing publisher or otherwise. The
>timing of this would be pretty critical; we're likely at a fairly
>critical point in time for it now, for it to happen or not.
>
>
Is there any reason why this could not be a function of the advocacy
group? The revenue generated would thereby be able to go back directly
into the advocacy project :-) I am thinking of publishing libre
documentation for a price, advertising it, and using the revenue to
further promote PostgreSQL.....
I thought about getting into this sort of market but it seems a bit of
trouble for me at the moment and well outside my core business focus.
Best WIshes,
Chris Travers
Metatron Technology Consulting
Attachment
Jim C. Nasby wrote: > > >I think you're both correct; 90% of people who use databases don't >understand how important maintaining the integrity of your data is. Tom >Kyte (http://asktom.oracle.com/) harps on this all the time, and there's >some articles in that website that provide some good explanations on why >you want as much business logic, etc as possible in the database and not >in some app server. > > Part of the problem is that you have a lot of people with different variations of the same opinions, but taking them in different directions. Getting this information to developers is hard because there is a lot of disagreement about how much logic one should put in the database. Personally I have never bought the "Put as much logic into your database as possible." This can *easily* be taken way too far. Review the discussions on pgsql-general about why sending email from the database backend is a bad idea. Can you write a CRM application server in PLPGSQL? Sure. But I am not sure it is a good idea..... What I do is set up a system with the following layers: User Interface UI logic Business Process Logic Database Access/Abstraction Data Presentation Data Maintenance Data Storage The lowest of the three layers should be implimented within the database itself. The top four layers are outside the database. Database Abstraction is simply the idea that if nothing else all your database calls should go through a single module so that if you need to change it later, you can. This also facilitates the development of 3-tier systems and helps people decide *intelligently* what logic to put in the database. The fact of the matter is, your data integrity should be defined independently of your application and the structure of the database should enforce that integrity itself. The ideal database is self-enforcing, self-describing, and allows for information presentation to be independent from storage. Besr WIshes, Chris Travers Metatron Technology Consulting
Chris Travers wrote: > Personally I have never bought the "Put as much logic into your database > as possible." This can *easily* be taken way too far. Review the > discussions on pgsql-general about why sending email from the database > backend is a bad idea. Can you write a CRM application server in > PLPGSQL? Sure. But I am not sure it is a good idea..... We've got a very powerful ERP system that has most of its transactional business logic in pl/pgsql, so I'll respectfullydisagree with you here ;-) We think it makes a powerful showcase for what PostgreSQL can do with even commodity-levelserver hardware. Cheers, Ned -- Ned Lilly President and CEO OpenMFG, LLC 119 West York Street Norfolk, VA 23510 tel. 757.461.3022 email: ned@openmfg.com www.openmfg.com
Jim C. Nasby wrote:
> Has anyone looked at how easy it would be to migrate an application from
> SQLLite to PostgreSQL? I'm asking not because I think we need to try and
> take space away from them, but because a number of sites/projects that
> start out thinking "oh, this will never grow beyond a few users" end up
> in a lot of pain when that assumption is proven false. Livejournal is a
> great example of this. So it would be good if those people had a
> moderately painless means to migrate to PostgreSQL after they out-grew
> what SQLLite has to offer. Perhaps it would be worth trying to work with
> the SQLLite folks on this.
Do you know of any non-standard behavior in SQLLite aside from a missing
feature?
Regards,
Jeff Davis
Jim, > Has anyone looked at how easy it would be to migrate an application from > SQLLite to PostgreSQL? I'm asking not because I think we need to try and > take space away from them, but because a number of sites/projects that > start out thinking "oh, this will never grow beyond a few users" end up > in a lot of pain when that assumption is proven false. Livejournal is a > great example of this. So it would be good if those people had a > moderately painless means to migrate to PostgreSQL after they out-grew > what SQLLite has to offer. Perhaps it would be worth trying to work with > the SQLLite folks on this. The "SQLite Folks" is actually pretty much just Richard. He likes PostgreSQL a lot, and specifically designed SQLite to be compatible with PostgreSQL. Migrating an app (or dual-DB support) is easy, just ask David Wheeler. The main thing we could do for SQLite is offer Richard some PR and other "community" support. It's a terrific little embedded database that just needs to have its profile raised. -- Josh Berkus Aglio Database Solutions San Francisco
chris@metatrontech.com (Chris Travers) writes:
> Chris Browne wrote:
>>ajs@crankycanuck.ca (Andrew Sullivan) writes:
>>>On Tue, Aug 09, 2005 at 09:49:28AM -0400, Rick Morris wrote:
>>>> - Thus, taking the above under advisement, it would seem best not
>>>> to fight MySQL on their ground, nor to spend much time on feature
>>>> distinction, but to forge more serious relationships with
>>>> organizations who have serious needs.
>>>>
>>>I think this is a capital idea. It seems this is a place to focus.
>>
>>Absolutely.
>>
>>The same would be true of "fighting Oracle on their ground."
>>
>>One thought suggested by someone outside of "the community" is that
>>one of the things that the various "more popular" systems have gotten
>>'right' is the notion of having a "Foo Press" that is responsible for
>>deploying books about their systems.
>>
>>This is true for various values of "Foo" including the members of the
>>set {"Microsoft", "Oracle", "MySQL"}.
>>
>>Contrast with some systems that people hardly ever hear of...
>> - There is exactly ONE book available on MaxDB, by No Starch Press
>> - There is exactly ONE book available on SQLite, by New Riders
>> - APress has ONE book on Firebird...
>> - It is not comforting that O'Reilly only has one PostgreSQL book :-(
>>
>>One "capital idea" might well be to establish a "PostgreSQL Press,"
>>whether in cooperation with an existing publisher or otherwise. The
>>timing of this would be pretty critical; we're likely at a fairly
>>critical point in time for it now, for it to happen or not.
> Is there any reason why this could not be a function of the advocacy
> group? The revenue generated would thereby be able to go back
> directly into the advocacy project :-) I am thinking of publishing
> libre documentation for a price, advertising it, and using the
> revenue to further promote PostgreSQL.....
Yes, people are thinking about this, although possibly not yet at the
"Our Own Press" level.
The thing that I would see being particularly valuable about "OOP" is
that it would provide a vehicle to enable deploying "dead tree"
editions of the 'official PostgreSQL documentation set,' which would
provide a way to turn THAT into one of the 'comprehensive guides' that
would suddenly come to "exist" from a publishing perspective.
> I thought about getting into this sort of market but it seems a bit
> of trouble for me at the moment and well outside my core business
> focus.
That's OK; there are a lot of people out there with a number of
distinct foci, so others of us can worry about this particular one
:-).
--
(format nil "~S@~S" "cbbrowne" "cbbrowne.com")
http://cbbrowne.com/info/x.html
The IQ of the group is the lowest IQ of a member of the group divided
by the number of people in the group.
jdavis-pgsql@empires.org (Jeff Davis) writes: > Jim C. Nasby wrote: >> Has anyone looked at how easy it would be to migrate an application >> from SQLLite to PostgreSQL? I'm asking not because I think we need >> to try and take space away from them, but because a number of >> sites/projects that start out thinking "oh, this will never grow >> beyond a few users" end up in a lot of pain when that assumption is >> proven false. Livejournal is a great example of this. So it would >> be good if those people had a moderately painless means to migrate >> to PostgreSQL after they out-grew what SQLLite has to >> offer. Perhaps it would be worth trying to work with the SQLLite >> folks on this. > > Do you know of any non-standard behavior in SQLLite aside from a > missing feature? Well, it in a sense doesn't have data types. The *honest* type that applies to every type is the STRING/TEXT type. -- (format nil "~S@~S" "cbbrowne" "acm.org") http://www3.sympatico.ca/cbbrowne/linuxxian.html First Rule of Computer Security - Only forbid that which can be made impossible. - Facilitate the possible. - Have the wisdom to explain the difference. -- Mark Miller
ned@nedscape.com (Ned Lilly) writes:
> Chris Travers wrote:
>
>> Personally I have never bought the "Put as much logic into your
>> database as possible." This can *easily* be taken way too far.
>> Review the discussions on pgsql-general about why sending email
>> from the database backend is a bad idea. Can you write a CRM
>> application server in PLPGSQL? Sure. But I am not sure it is a
>> good idea.....
>
> We've got a very powerful ERP system that has most of its
> transactional business logic in pl/pgsql, so I'll respectfully
> disagree with you here ;-) We think it makes a powerful showcase for
> what PostgreSQL can do with even commodity-level server hardware.
It seems to me to be a tough call exactly where to stop.
There is considerable *obvious* merit to adding in logic that resides
at the "declarative" level such as the case where constraints provide
somewhat self-documenting data validation.
It seems to me that adding additional such "predicates" comes at a
relatively low cognitive cost.
Foreign key constraints are commonly worthwhile, albeit being
something that has a slightly higher "cognitive cost" as well as
having some potentially negative performance implications. You
mightn't want to implement every FK that is theoretically possible to
implement.
Implementing APIs within the database falls, in my mind, into a more
ambiguous area.
There are numerous good things about making extensive use of pl/pgsql;
you cut down on round trips, and can keep the data validation in an
API that, by being in the DBMS, makes it accessible to ANY accessing
application regardless of what language the application may use.
But it adds an extra layer of logic, and figuring out what is running
where does introduce some "cognitive cost."
Furthermore, if there is a legitimate need for portability between
databases (e.g. - you have an important customer who REALLY wants
Oracle|DB2 support), the cost of using stored procedures efficiently
and quasi-portably goes way up.
For someone to argue that implementing most business logic inside the
DB isn't their favorite idea is something where there needs to be some
room for disagreement :-).
--
output = ("cbbrowne" "@" "ntlug.org")
http://www3.sympatico.ca/cbbrowne/emacs.html
"Few people can be happy unless they hate someother person, nation or
creed." -- Bertrand Russell
On Mon, Aug 15, 2005 at 01:57:10PM -0400, Chris Browne wrote: > ned@nedscape.com (Ned Lilly) writes: > > Chris Travers wrote: > > > >> Personally I have never bought the "Put as much logic into your > >> database as possible." This can *easily* be taken way too far. > >> Review the discussions on pgsql-general about why sending email > >> from the database backend is a bad idea. Can you write a CRM > >> application server in PLPGSQL? Sure. But I am not sure it is a > >> good idea..... > > > > We've got a very powerful ERP system that has most of its > > transactional business logic in pl/pgsql, so I'll respectfully > > disagree with you here ;-) We think it makes a powerful showcase > > for what PostgreSQL can do with even commodity-level server > > hardware. > > It seems to me to be a tough call exactly where to stop. > > There is considerable *obvious* merit to adding in logic that > resides at the "declarative" level such as the case where > constraints provide somewhat self-documenting data validation. > > It seems to me that adding additional such "predicates" comes at a > relatively low cognitive cost. It can get pretty high when the constraints are complicated, and until there's DOMAINs are actually enforced (e.g. on being returned from a function) the way TYPEs are, we've got no good way to handle this. > Foreign key constraints are commonly worthwhile, albeit being > something that has a slightly higher "cognitive cost" as well as > having some potentially negative performance implications. You > mightn't want to implement every FK that is theoretically possible > to implement. > > Implementing APIs within the database falls, in my mind, into a more > ambiguous area. > > There are numerous good things about making extensive use of > pl/pgsql; you cut down on round trips, and can keep the data > validation in an API that, by being in the DBMS, makes it accessible > to ANY accessing application regardless of what language the > application may use. > > But it adds an extra layer of logic, and figuring out what is > running where does introduce some "cognitive cost." It does. The next question is, "what do other options for doing the same thing cost on this scale?" and of course the answer, as with fuzzy, complicated situations as a rule, is "it depends." > Furthermore, if there is a legitimate need for portability between > databases (e.g. - you have an important customer who REALLY wants > Oracle|DB2 support), the cost of using stored procedures efficiently > and quasi-portably goes way up. Is it any more hassle than other options for multiple database support? I guess that comes down to "it depends," too. Cheers, D -- David Fetter david@fetter.org http://fetter.org/ phone: +1 510 893 6100 mobile: +1 415 235 3778 Remember to vote!
Certainly as you get 'farther away' from data storage, it becomes less
reasonable to do stuff in the database and more reasonable to do it
outside. Sending email is an example of something that could well be
done outside the database server. But, here's some things to consider:
- Anything done outside the database is roughly equivalent to cut and
pasting code instead of using functions
- Anything done outside of the database can't be transactionally safe
Let me expand on this with an example:
A bank offers a service to customers where anytime a deposit is made
into their account they'll get an email (BTW, this isn't BS: my bank
does this). There's multiple ways a deposit can hit a bank. You can
mail a deposit to them or see a teller. You can do it at an ATM. You can
get an electronic transfer. Now, though I'm not a banking expert, I'm
pretty certain that these will utilize different systems.
So, a (bad) way to do this would be to have each application (ie:
whatever tellers use) commit the transaction to the database and then
send an email. This means duplication of code. Or worse, a new
application is brought online and the team working on it didn't realize
they had to support sending an email when a deposit is made, so now the
feature that the bank is marketing to it's customers is broken.
A good way to do this would be to make the database responsible for
making sure these emails were sent out any time a deposit was made. This
doesn't mean the database server itself has to send the emails; you
could have a trigger that records a list of deposits that notifications
have to be sent out for. The point is you know that every deposit that
needs an email sent out is being flagged by the only part of the system
that can do that: the database.
Of course, email isn't something that's very transaction-safe, but even
in this example if the database is involved you can do a better job of
ensuring that emails will be sent. If the company email server is
temporarily down the process that runs through the list of deposits that
need an email sent out will stop, but that list is still there. Once the
email server is back the emails will start going out again. If the
emails were processed as transactions came in though, you'd either have
to stop accepting transactions while the email server was down, or the
emails just get lost.
Of course, just because I've come up with an example where it makes
sense to involve the database doesn't mean that it's always the best
thing to do. But ISTM there's a lot of people out there who do
everything they can to keep code out of the database when in fact they
should be doing the exact opposite.
So I view it this way: unless you have a very compelling reason why some
piece of code dealing with your data shouldn't be in the database, it's
a safe bet that it should be in the database. Even in this example,
there's not really much reason why the emails can't be sent from the
database server, assuming you have another machine that handles the
final email delivery to remote systems.
On Mon, Aug 15, 2005 at 01:57:10PM -0400, Chris Browne wrote:
> ned@nedscape.com (Ned Lilly) writes:
> > Chris Travers wrote:
> >
> >> Personally I have never bought the "Put as much logic into your
> >> database as possible." This can *easily* be taken way too far.
> >> Review the discussions on pgsql-general about why sending email
> >> from the database backend is a bad idea. Can you write a CRM
> >> application server in PLPGSQL? Sure. But I am not sure it is a
> >> good idea.....
> >
> > We've got a very powerful ERP system that has most of its
> > transactional business logic in pl/pgsql, so I'll respectfully
> > disagree with you here ;-) We think it makes a powerful showcase for
> > what PostgreSQL can do with even commodity-level server hardware.
>
> It seems to me to be a tough call exactly where to stop.
>
> There is considerable *obvious* merit to adding in logic that resides
> at the "declarative" level such as the case where constraints provide
> somewhat self-documenting data validation.
>
> It seems to me that adding additional such "predicates" comes at a
> relatively low cognitive cost.
>
> Foreign key constraints are commonly worthwhile, albeit being
> something that has a slightly higher "cognitive cost" as well as
> having some potentially negative performance implications. You
> mightn't want to implement every FK that is theoretically possible to
> implement.
>
> Implementing APIs within the database falls, in my mind, into a more
> ambiguous area.
>
> There are numerous good things about making extensive use of pl/pgsql;
> you cut down on round trips, and can keep the data validation in an
> API that, by being in the DBMS, makes it accessible to ANY accessing
> application regardless of what language the application may use.
>
> But it adds an extra layer of logic, and figuring out what is running
> where does introduce some "cognitive cost."
>
> Furthermore, if there is a legitimate need for portability between
> databases (e.g. - you have an important customer who REALLY wants
> Oracle|DB2 support), the cost of using stored procedures efficiently
> and quasi-portably goes way up.
>
> For someone to argue that implementing most business logic inside the
> DB isn't their favorite idea is something where there needs to be some
> room for disagreement :-).
> --
> output = ("cbbrowne" "@" "ntlug.org")
> http://www3.sympatico.ca/cbbrowne/emacs.html
> "Few people can be happy unless they hate someother person, nation or
> creed." -- Bertrand Russell
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com 512-569-9461
On Mon, Aug 15, 2005 at 10:12:01AM -0700, Josh Berkus wrote: > Jim, > > > Has anyone looked at how easy it would be to migrate an application from > > SQLLite to PostgreSQL? I'm asking not because I think we need to try and > > take space away from them, but because a number of sites/projects that > > start out thinking "oh, this will never grow beyond a few users" end up > > in a lot of pain when that assumption is proven false. Livejournal is a > > great example of this. So it would be good if those people had a > > moderately painless means to migrate to PostgreSQL after they out-grew > > what SQLLite has to offer. Perhaps it would be worth trying to work with > > the SQLLite folks on this. > > The "SQLite Folks" is actually pretty much just Richard. He likes PostgreSQL > a lot, and specifically designed SQLite to be compatible with PostgreSQL. > Migrating an app (or dual-DB support) is easy, just ask David Wheeler. > > The main thing we could do for SQLite is offer Richard some PR and other > "community" support. It's a terrific little embedded database that just > needs to have its profile raised. So maybe we should do exactly that. Maybe something on the website: Q: What if I just have a really small app that PGSQL is overkill for? A: Use SQLite. Here's why: ... A feature comparison between the two might be handy as well. Along with some info on how to write code that will be easy to migrate should you need to. I know spare time is a scarce commodity, but perhaps we can get some folks from the SQLite community to help with this. And of course it makes sense to suggest PostgreSQL when SQLite turns out to be too lightweight. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com 512-569-9461
On Aug 15, 2005, at 1:57 PM, Chris Browne wrote: > For someone to argue that implementing most business logic inside the > DB isn't their favorite idea is something where there needs to be some > room for disagreement :-). > I don't disagree but after doing quite a bit of PHP the last few weeks (using Drupal) I see more clearly why most people don't go to the trouble. I can create all kinds of constraints in my database but when I go to save a row that might violate several of them, I'll only get one error back. This won't work in a web form interface where I should provide feedback on all of the errors at once rather than one at a time. So if I want this validation logic to be available at both the application and database level, I have to somehow parse it from the database or create some superset of the specification that will work in the application and create the constraints in the database. Otherwise, I need to maintain the constraints in both places and keep them in sync. John DeSoi, Ph.D. http://pgedit.com/ Power Tools for PostgreSQL
John DeSoi wrote: > On Aug 15, 2005, at 1:57 PM, Chris Browne wrote: > >> For someone to argue that implementing most business logic inside the >> DB isn't their favorite idea is something where there needs to be some >> room for disagreement :-). >> > > I don't disagree but after doing quite a bit of PHP the last few weeks > (using Drupal) I see more clearly why most people don't go to the > trouble. I can create all kinds of constraints in my database but when > I go to save a row that might violate several of them, I'll only get > one error back. This won't work in a web form interface where I should > provide feedback on all of the errors at once rather than one at a > time. So if I want this validation logic to be available at both the > application and database level, I have to somehow parse it from the > database or create some superset of the specification that will work in > the application and create the constraints in the database. Otherwise, > I need to maintain the constraints in both places and keep them in sync. IMHO, this is exactly where there needs to be more work done on application frameworks: automated ways to propagate constraints and business logic back into the application layer. I explored those concepts to a small extent (with code examples) in a couple articles for PHP|Architect. I think it is an area that would involve some serious work, but would bring some serious benefits. Regards, Rick Morris > > John DeSoi, Ph.D. > http://pgedit.com/ > Power Tools for PostgreSQL > > > ---------------------------(end of broadcast)--------------------------- > TIP 9: In versions below 8.0, the planner will ignore your desire to > choose an index scan if your joining column's datatypes do not > match > >
On Tue, Aug 16, 2005 at 10:15:38AM -0400, Rick Morris wrote: > John DeSoi wrote: > >I don't disagree but after doing quite a bit of PHP the last few weeks > >(using Drupal) I see more clearly why most people don't go to the > >trouble. I can create all kinds of constraints in my database but when > >I go to save a row that might violate several of them, I'll only get > >one error back. This won't work in a web form interface where I should > >provide feedback on all of the errors at once rather than one at a > >time. So if I want this validation logic to be available at both the > >application and database level, I have to somehow parse it from the > >database or create some superset of the specification that will work in > >the application and create the constraints in the database. Otherwise, > >I need to maintain the constraints in both places and keep them in sync. > > IMHO, this is exactly where there needs to be more work done on > application frameworks: automated ways to propagate constraints and > business logic back into the application layer. > > I explored those concepts to a small extent (with code examples) in a > couple articles for PHP|Architect. I think it is an area that would > involve some serious work, but would bring some serious benefits. There's at least 3 ways this can happen. You can define the logic/constraints in the application and push them to the database, you can define them in the database and push them to the application, or you can use a seperate framework to drive both. Personally, I'm in favor of #2, because it means you should be able to have any application use the constraints in the database. I think this is something that could possibly be added to PostgreSQL via a pgfoundry project. My initial thought is to provide a means to associate certain constraints/triggers with 'human readable' conditions. So for example, in a table that has username, you could link the unique constraint on username to a message that says 'That username is already in use.' Unfortunately this still doesn't allow for checking multiple constraints at once in the database, and uniqueness can really only be checked by the database at insert/update time. But other constraints could be checked ahead of time. Another possibility is improving on the existing frameworks. Personally, I'm not terribly impressed with the frameworks I've looked at because they seem to divorce themselves from the database too much. They generally put a much greater load on the database because they want to do as much as possible in the application. For example, if you mark a field as being unique, many of them will do a select before trying to insert or update to see if a record already exists. Now you've got the database running 2 queries instead of 1. So far, the best solution I've seen is the XML-based 'datadef' that a friend of mine created at a former job. It was database-centric enough so that it could drive the entire database creation process (including triggers, stored procs, etc) and do it well, while also creating application code (C/C++ in this case). Since it was XML/XSLT based it was pretty easy to add new features, and more importantly, it could support any programming language out there. Everything else I've seen is tied to a specific language, which is a big downside. Unfortunately, he never wanted to release it to the community because he felt the implementation was ugly. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com 512-569-9461
Jim C. Nasby wrote: > On Tue, Aug 16, 2005 at 10:15:38AM -0400, Rick Morris wrote: > >>John DeSoi wrote: >> >>>I don't disagree but after doing quite a bit of PHP the last few weeks >>>(using Drupal) I see more clearly why most people don't go to the >>>trouble. I can create all kinds of constraints in my database but when >>>I go to save a row that might violate several of them, I'll only get >>>one error back. This won't work in a web form interface where I should >>>provide feedback on all of the errors at once rather than one at a >>>time. So if I want this validation logic to be available at both the >>>application and database level, I have to somehow parse it from the >>>database or create some superset of the specification that will work in >>>the application and create the constraints in the database. Otherwise, >>>I need to maintain the constraints in both places and keep them in sync. >> >>IMHO, this is exactly where there needs to be more work done on >>application frameworks: automated ways to propagate constraints and >>business logic back into the application layer. >> >>I explored those concepts to a small extent (with code examples) in a >>couple articles for PHP|Architect. I think it is an area that would >>involve some serious work, but would bring some serious benefits. > > > There's at least 3 ways this can happen. You can define the > logic/constraints in the application and push them to the database, you > can define them in the database and push them to the application, or you > can use a seperate framework to drive both. > > Personally, I'm in favor of #2, because it means you should be able to > have any application use the constraints in the database. Absolutely. That's what I was getting at. > > I think this is something that could possibly be added to PostgreSQL via > a pgfoundry project. My initial thought is to provide a means to > associate certain constraints/triggers with 'human readable' conditions. > So for example, in a table that has username, you could link the unique > constraint on username to a message that says 'That username is already > in use.' Unfortunately this still doesn't allow for checking multiple > constraints at once in the database, and uniqueness can really only be > checked by the database at insert/update time. But other constraints > could be checked ahead of time. One of my ideas involves creating a class for each base datatype, with the possibility for creating additional classes for domains and custom datatypes. Then it becomes easier for the application framework to sort out what types of constraints can apply to a given column. So far, I have only toyed around with this in PHP a little, but I would be happy to share this work. Still, the hard work is in parsing constraint definitions. The information_schema tables/views make this information more accessible, but still, there is a certain amount of crazy reverse-engineering one needs to do. It would be nice eventually for the pgsql modules of any language to be able to derive this information, for example, starting with the pg_meta_data() function in PHP (http://php.net/pg_meta_data). > > Another possibility is improving on the existing frameworks. Personally, > I'm not terribly impressed with the frameworks I've looked at because > they seem to divorce themselves from the database too much. They > generally put a much greater load on the database because they want to > do as much as possible in the application. For example, if you mark a > field as being unique, many of them will do a select before trying to > insert or update to see if a record already exists. Now you've got the > database running 2 queries instead of 1. > Unfortunately. > So far, the best solution I've seen is the XML-based 'datadef' that a > friend of mine created at a former job. It was database-centric enough > so that it could drive the entire database creation process (including > triggers, stored procs, etc) and do it well, while also creating > application code (C/C++ in this case). Since it was XML/XSLT based it > was pretty easy to add new features, and more importantly, it could > support any programming language out there. Everything else I've seen is > tied to a specific language, which is a big downside. Unfortunately, he > never wanted to release it to the community because he felt the > implementation was ugly. I know the feeling ;). That approach has merit, but one possible drawback: the developers/DBAs might circumvent the datadef and make database design changes directly on the DB. Then your application is hosed. I much prefer to develop something that allows the application layer to react automatically to changes in the database design. (I know that is never *completely* possible, but at least in the area of basic constraints and datatypes it would be nice) Regards, Rick Morris
jnasby@pervasive.com ("Jim C. Nasby") writes:
> So far, the best solution I've seen is the XML-based 'datadef' that
> a friend of mine created at a former job. It was database-centric
> enough so that it could drive the entire database creation process
> (including triggers, stored procs, etc) and do it well, while also
> creating application code (C/C++ in this case). Since it was
> XML/XSLT based it was pretty easy to add new features, and more
> importantly, it could support any programming language out
> there. Everything else I've seen is tied to a specific language,
> which is a big downside. Unfortunately, he never wanted to release
> it to the community because he felt the implementation was ugly.
I don't much like that; that adds a layer that can fall out of date,
and offers NOTHING in terms of tools to get things resynchronized.
What would be the "nice to have" thing is some system that allows you
to extract a set of field validation functions from the database.
In effect, you do:
select a.attname, field_validation_function(c.oid, a.attname, 'Perl')
from pg_class c, pg_attribute a where a.attrelid = c.oid and c.relname
= 'my_table');
which returns a list of Perl 'validation functions', one for each
attribute.
Each validation function would take data passed in, validate it
against some Perl-encoded form of the DBMS constraints, and either:
a) Return 0/NULL, indicating that all is OK, or
b) Return a matrix of error messages indicating the failures
It would be neat (would it not? :-)) to allow passing in other names
of languages, e.g. where scripting_language in ('Perl', 'Python',
'Ruby', 'Tcl', 'PHP', 'JavaScript'), where the result would be a set
of functions in those respective languages.
It would probably be ideal for what is to be returned to be, in each
case, the local equivalent to a lambda function so that it could be
transparently embedded inside the user's favorite application
framework rather than forcing some Procrustean naming convention on
them.
If the result was some hunk of XML that could be automatically
transformed into suitable lambda functions, that's OK too, although
there is the demerit that it FORCES a further rewriting process.
I'm not sure what to do about multicolumn constraints, but that's
probably troublesome even in theory :-).
--
output = reverse("moc.enworbbc" "@" "enworbbc")
http://www3.sympatico.ca/cbbrowne/linuxdistributions.html
"Implying that you can build systems without rigourous interface
specification is always a powerful selling technique to the clueless."
-- Paul Campbell, seen in comp.object.corba
Chris Browne wrote:
> jnasby@pervasive.com ("Jim C. Nasby") writes:
>
>>So far, the best solution I've seen is the XML-based 'datadef' that
>>a friend of mine created at a former job. It was database-centric
>>enough so that it could drive the entire database creation process
>>(including triggers, stored procs, etc) and do it well, while also
>>creating application code (C/C++ in this case). Since it was
>>XML/XSLT based it was pretty easy to add new features, and more
>>importantly, it could support any programming language out
>>there. Everything else I've seen is tied to a specific language,
>>which is a big downside. Unfortunately, he never wanted to release
>>it to the community because he felt the implementation was ugly.
>
>
> I don't much like that; that adds a layer that can fall out of date,
> and offers NOTHING in terms of tools to get things resynchronized.
>
> What would be the "nice to have" thing is some system that allows you
> to extract a set of field validation functions from the database.
>
> In effect, you do:
>
> select a.attname, field_validation_function(c.oid, a.attname, 'Perl')
> from pg_class c, pg_attribute a where a.attrelid = c.oid and c.relname
> = 'my_table');
>
> which returns a list of Perl 'validation functions', one for each
> attribute.
>
> Each validation function would take data passed in, validate it
> against some Perl-encoded form of the DBMS constraints, and either:
>
> a) Return 0/NULL, indicating that all is OK, or
>
> b) Return a matrix of error messages indicating the failures
>
> It would be neat (would it not? :-)) to allow passing in other names
> of languages, e.g. where scripting_language in ('Perl', 'Python',
> 'Ruby', 'Tcl', 'PHP', 'JavaScript'), where the result would be a set
> of functions in those respective languages.
Yes! That's exactly the kind of concept I was looking for.
>
> It would probably be ideal for what is to be returned to be, in each
> case, the local equivalent to a lambda function so that it could be
> transparently embedded inside the user's favorite application
> framework rather than forcing some Procrustean naming convention on
> them.
AFAIK each of these has some analogue to the lambda function--even PHP
with create_function(): www.php.net/create_function.
Another 'nice to have' would be a way to provide methods to extract a
value from each column datatype into a suitable variable in the
scripting language. That would be very nice for date columns, arrays, etc...
>
> If the result was some hunk of XML that could be automatically
> transformed into suitable lambda functions, that's OK too, although
> there is the demerit that it FORCES a further rewriting process.
>
> I'm not sure what to do about multicolumn constraints, but that's
> probably troublesome even in theory :-).
On Tue, Aug 16, 2005 at 11:22:05AM -0400, Rick Morris wrote: > Still, the hard work is in parsing constraint definitions. The > information_schema tables/views make this information more accessible, > but still, there is a certain amount of crazy reverse-engineering one > needs to do. It would be nice eventually for the pgsql modules of any > language to be able to derive this information, for example, starting > with the pg_meta_data() function in PHP (http://php.net/pg_meta_data). http://pgfoundry.org/projects/newsysviews/ might make that task a bit easier. > >So far, the best solution I've seen is the XML-based 'datadef' that a > >friend of mine created at a former job. It was database-centric enough > >so that it could drive the entire database creation process (including > >triggers, stored procs, etc) and do it well, while also creating > >application code (C/C++ in this case). Since it was XML/XSLT based it > >was pretty easy to add new features, and more importantly, it could > >support any programming language out there. Everything else I've seen is > >tied to a specific language, which is a big downside. Unfortunately, he > >never wanted to release it to the community because he felt the > >implementation was ugly. > > I know the feeling ;). > > That approach has merit, but one possible drawback: the developers/DBAs > might circumvent the datadef and make database design changes directly > on the DB. Then your application is hosed. I much prefer to develop > something that allows the application layer to react automatically to > changes in the database design. (I know that is never *completely* > possible, but at least in the area of basic constraints and datatypes it > would be nice) Very true. Fortunately I've been able to keep that kind of tweaking on production systems to an absolute minimum everywhere I've worked; the XML datadef file was the defacto standard for what was in the database. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com 512-569-9461