Thread: Decide between Postgresql and Mysql (help of comunity)
Hi, I'm a Postgresql's user and I think that it's very very good and robust. In my work we're confuse between where database is the best choose: Postgresql or Mysql. The Mysql have the reputation that is very fast working in the web but in our application we are estimating many access simultaneous, then I think that the Postgresql is the best choice. Am I right? Our server have 1 GB of RAM, how many users can it support at the same time with this memory? Thanks in advanced Marcos
Marcos wrote: > Hi, > > I'm a Postgresql's user and I think that it's very very good and > robust. > > In my work we're confuse between where database is the best choose: > Postgresql or Mysql. The Mysql have the reputation that is very fast > working in the web but in our application we are estimating many access > simultaneous, then I think that the Postgresql is the best choice. > > Am I right? > > Our server have 1 GB of RAM, how many users can it support at the same > time with this memory? > > Thanks in advanced > > Marcos The RAM/users question depends largely on what the database is used for and what each user is doing in the database. From what I understand, PostgreSQL is designed with stability and reliability as key tenants. MySQL favors performance and ease of use. An example is that, last I checked, MySQL doesn't have an equivalent to PostgreSQL's 'fsync' which helps insure that data is actually written to the disk. This costs performance but increases reliability and crash recovery. HTH Madison -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Madison Kelly (Digimer) -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
On Tue, 2006-03-28 at 09:31, Marcos wrote: > Hi, > > I'm a Postgresql's user and I think that it's very very good and > robust. > > In my work we're confuse between where database is the best choose: > Postgresql or Mysql. The Mysql have the reputation that is very fast > working in the web but in our application we are estimating many access > simultaneous, then I think that the Postgresql is the best choice. > > Am I right? > > Our server have 1 GB of RAM, how many users can it support at the same > time with this memory? This is as much about the code in front of the database as the database itself. You'll want to use an architecture that supports pooled connections (java, php under lighttpd, etc...) and you'll want to look at your read to write ratio. MySQL and PostgreSQL can handle fairly heavy parallel loads. PostgreSQL is generally a much better performer when you need to make a lot of parallel writes. But the bigger question is which one is suited to your application in general. If some major issue in MySQL or PostgreSQL makes it a poor choice for your app, then it doesn't matter how much load it can handle, it's still a poor choice. Generally speaking, MySQL is a poor choice if you're doing things like accounting, where the maths have to be correct. It's quite easy to ask MySQL to do math and get the wrong answer. It also has some serious problems with referential integrity, but most of those can be worked around using innodb tables. But at that point, you're using the same basic storage methods as PostgreSQL uses, i.e. an MVCC storage engine. And now that Oracle has bought Innodb, the availability of that in the future to MySQL is in doubt. There's also the issue of licensing. If you'll be selling copies of your app to customers, you'll be writing a check for each install to MySQL AB. Not so with PostgreSQL. So, what exactly are you planning on doing? Lastly, take a look here: http://sql-info.de/mysql/gotchas.html and here: http://sql-info.de/postgresql/postgres-gotchas.html for a list of the common "gotchas" in both databases. Generally you'll find the PostgreSQL gotchas are of the sort that make you go "oh, that's interesting" and the MySQL gotchas are the kind that make you go "Dear god, you must be kidding me!" But that's just my opinion, I could be wrong.
> So, what exactly are you planning on doing? The application will be a chat for web, the chats will be stored in the server. In a determined interval of time... more or less 2 seconds, the application will be looking for new messages. I believe that it will make many accesses. The write in disc will be constant. Thanks :o) Marcos
On Tue, 2006-03-28 at 13:42, PFC wrote: > > This is as much about the code in front of the database as the database > > itself. You'll want to use an architecture that supports pooled > > connections (java, php under lighttpd, etc...) and you'll want to look > > Well, anybody who uses PHP and cares about performance is already using > lighttpd, no ? > > > MySQL and PostgreSQL can handle fairly heavy parallel loads. > > I'll only speak about MyISAM. MySQL == MyISAM. InnoDB is useless : if you > want transactions, use postgres. I agree with most of what you posted, but I'm not quite sure what you meant here. Innodb in and of itself is a fairly decent MVCC implementation, with, as usual, some limitations (it's rollback performance is HORRIFICLY bad). What really makes innodb useless to me is that there's no real support for proper operation by MySQL itself. If you could force MySQL to only use innodb tables, and to NEVER do the wrong things syntactically, it would be ok. But there are thousands of foot-guns in the MySQL - Innodb combination waiting to take off your toes. Too many to count really. To me, that's what makes innodb so useless, the way MySQL fails to integrate well with it.
> This is as much about the code in front of the database as the database > itself. You'll want to use an architecture that supports pooled > connections (java, php under lighttpd, etc...) and you'll want to look Well, anybody who uses PHP and cares about performance is already using lighttpd, no ? > MySQL and PostgreSQL can handle fairly heavy parallel loads. I'll only speak about MyISAM. MySQL == MyISAM. InnoDB is useless : if you want transactions, use postgres. If you say to yourself "oh yeah, but it would be cool to use a MyISAM table for stuff like hit counters etc"... Is it the job of a SQL database to count hits on the root page of your site ? No. To store user sessions ? No. The job of a SQL database is to efficiently handle data, not to do something that should stay in RAM in the application server process, or at worst, in a memcached record. MySQL + MyISAM has a huge advantage : it can look up data in the index without touching the tables. MySQL handles parallel SELECTs very well. However, throw in some maintenance operation which involves a long query with writes (like a big joined UPDATE) and all access to your website is blocked while the query lasts. This is worsened by the fact that MySQL sucks at complex queries. If all of your updates are done to a few rows, MyISAM is cool, but someday you'll want to do this query which locks a table during one minute... and then you got a problem. Just be very clear about what you want to do, what types of queries you'll want to run in two years... etc.
Marcos wrote: >>So, what exactly are you planning on doing? >> >> > >The application will be a chat for web, the chats will be stored in the >server. In a determined interval of time... more or less 2 seconds, the >application will be looking for new messages. > >I believe that it will make many accesses. The write in disc will be >constant. > > Ok. I would favor PostgreSQL for reasons of ease of future development. However, lets look at what both RDBMS's buy you: MySQL: 1) Possibility of many components for web apps that can be used though the lack of certain features (such as complex updateable views) makes this possibly an issue. 2) Great simple read performance. PostgreSQL: 1) Possibility to integrate any other components later (including those on MySQL via DBI-Link). 2) Fabulous community support (and I am sure fabulous paid support too given the fact that many of those who contribute to the great community support also offer paid support). 3) Better parallel write performance. 4) Greater extensibility, leading to greater flexibility down the road should you want to add in new components without rewriting your front-end. For a simple chat client, you can probably put something together with some Perl/CGI scripts, Jabber, and MySQL or PostgreSQL pretty easily and without much development labor at all. Indeed I would suggest that the RDBMS is, absent other specific concerns, the least of your issues. In other words, both are probably adequate. It is impossible to provide an estimate for capacity though without knowing the app in question, expected query composition, and so forth. Best Wishes, Chris Travers Metatron Technology Consulting
>> So, what exactly are you planning on doing? > > The application will be a chat for web, the chats will be stored in the > server. In a determined interval of time... more or less 2 seconds, the > application will be looking for new messages. > > I believe that it will make many accesses. The write in disc will be > constant. Ah, cool. That's exactly what a database is not designed for xD Try this, I coded this in about 1 hour as a joke last week. http://peufeu.com/demos/xhchat/ It works in firefox and opera, uses xmlhttprequest, and the messages are stored in a dbm database. We have also coded a real HTTP chat. I'll briefly expose the details on-list, but if you want the gory stuff, ask privately. There is a Postgres database for users, authentication, chatrooms and stuff. This database can be modified by a full-blown web application. Of course, messages are not stored in the database. It would be suicidal performance-wise to do so. An asynchronous HTTP server, using select() (lighttpd style) is coded in Python. It is very special-purpose server. It keeps an open connection with the client (browser) and sends messages as they arrive in the chatroom, with no delay. The connection is interrupted only when the client submits a new message via a form, but this is not mandatory. My memories are a bit old, but we benchmarked it at about 4000 messages/second on a low-end server (athlon something). Concurrent connections are unlimited. Disk I/O is zero. I like it. If you store messages in the database, you can hope to be about 10-50 times slower.
> What really makes innodb useless to me is that there's no real support > for proper operation by MySQL itself. If you could force MySQL to only > use innodb tables, and to NEVER do the wrong things syntactically, it > would be ok. But there are thousands of foot-guns in the MySQL That's what I meant actually. And by saying "if you want transactions" I also meant "if you want a database system that will go to great lengths to save your ass and your data instead of helping you shooting yourself in the foot, generally work very well, be reliable, friendly and a pleasure to work with, which means more or less, if you're rational rather than masochistic, then yeah, you should use postgres". > If you could force MySQL to only > use innodb tables, and to NEVER do the wrong things syntactically, it > would be ok. You'd still need to teach it how to hash-join and everything, though. Life sucks when the only join type you have is merge join.
On Tue, Mar 28, 2006 at 09:42:51PM +0200, PFC wrote: > However, throw in some maintenance operation which involves a long > query with writes (like a big joined UPDATE) and all access to your > website is blocked while the query lasts. > This is worsened by the fact that MySQL sucks at complex queries. > > If all of your updates are done to a few rows, MyISAM is cool, but > someday you'll want to do this query which locks a table during one > minute... and then you got a problem. Not to mention that MyISAM loves to eat data. Livejournal suffered at least one major crash due to MyISAM corruption. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
Heh, too quick on the send button... On Tue, Mar 28, 2006 at 09:42:51PM +0200, PFC wrote: > I'll only speak about MyISAM. MySQL == MyISAM. InnoDB is useless : > if you want transactions, use postgres. > If you say to yourself "oh yeah, but it would be cool to use a > MyISAM table for stuff like hit counters etc"... Is it the job of a SQL > database to count hits on the root page of your site ? No. To store user > sessions ? No. The job of a SQL database is to efficiently handle data, > not to do something that should stay in RAM in the application server > process, or at worst, in a memcached record. Actually, it's entirely possible to do stuff like web counters, you just want to do it differently in PostgreSQL. Simply insert into a table every time you have a hit, and then roll that data up periodically. And using MyISAM is no panacea, either. Trying to keep a web counter in a MyISAM table means you'll serialize every web page on that counter update. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
On 3/28/06, Jim C. Nasby <jnasby@pervasive.com> wrote: > Heh, too quick on the send button... > > On Tue, Mar 28, 2006 at 09:42:51PM +0200, PFC wrote: > Actually, it's entirely possible to do stuff like web counters, you just > want to do it differently in PostgreSQL. Simply insert into a table > every time you have a hit, and then roll that data up periodically. > > And using MyISAM is no panacea, either. Trying to keep a web counter in > a MyISAM table means you'll serialize every web page on that counter > update. if you want raw speed, use a sequence for a hit-counter. sequences are wonder-tools and very lightweight. Explain analyze for a sequence nextval on my desktop box reports 47 microseconds. thats 200k sequence updates/second. insert into a table (fsync off/cache write, no keys) is not much slower. PostgreSQL 8.1 saw a lot of performance improvements...but the most important (and least publicized) is the reduced latency of simple queries in high cache enviroments. merlin
On Mar 28, 2006, at 1:57 PM, Madison Kelly wrote: > From what I understand, PostgreSQL is designed with stability and > reliability as key tenants. MySQL favors performance and ease of > use. An From my point of view, mysql favors single-user performance over all else. Get into multiple updaters and you are competing for table locks all the time. Postgres works much better with multiple clients writing to it.
On Mar 28, 2006, at 1:59 PM, Scott Marlowe wrote: > Generally you'll find the PostgreSQL gotchas are of the sort that make > you go "oh, that's interesting" and the MySQL gotchas are the kind > that > make you go "Dear god, you must be kidding me!" > > But that's just my opinion, I could be wrong. I nominate this for "quote of the month". :-)
On Mar 28, 2006, at 11:55 AM, Marcos wrote: > The application will be a chat for web, the chats will be stored in > the > server. In a determined interval of time... more or less 2 seconds, > the > application will be looking for new messages. We bought software for this purpose (phplive). It is based on mysql using isam tables and is written in (surprise!) php. Two of my "favorite" techonologies! :-)
On Tuesday 28 March 2006 14:50, Scott Marlowe wrote: > On Tue, 2006-03-28 at 13:42, PFC wrote: > > > This is as much about the code in front of the database as the database > > > itself. You'll want to use an architecture that supports pooled > > > connections (java, php under lighttpd, etc...) and you'll want to look > > > > Well, anybody who uses PHP and cares about performance is already using > > lighttpd, no ? /flame on if you were *that* worried about performance, you wouldn't be using PHP or *any* interperted language /flame off sorry - couldn't resist it :-)
Gorshkov wrote: > /flame on > if you were *that* worried about performance, you wouldn't be using PHP or > *any* interperted language > /flame off > > sorry - couldn't resist it :-) I hope this was just a joke. You should be sure to clarify - there might be some newbie out there who thinks you are seriouslysuggesting coding major web sites in some old-fashioned compiled language. Craig
On Wednesday 29 March 2006 21:23, Craig A. James wrote: > Gorshkov wrote: > > /flame on > > if you were *that* worried about performance, you wouldn't be using PHP > > or *any* interperted language > > /flame off > > > > sorry - couldn't resist it :-) > > I hope this was just a joke. You should be sure to clarify - there might > be some newbie out there who thinks you are seriously suggesting coding > major web sites in some old-fashioned compiled language. > well yes, it was meant as a joke ..... that's *usually* what a ";-)" means. That being said ..... what *is* the difference between coding a website - major or otherwise - in an "old-fashioned" compiled language and a non-compiled language, except for the amount of hoursepower and memory you require? Old-fashioned doesn't mean bad, inappropriate, or inferior. It's just not the latest-and-greatest, however it's currently defined by the geek fashion police.
This is off-topic for this group so I'll just give a brief reply; I'm happy to carry on more just between the two of us... Gorshkov wrote: > That being said ..... what *is* the difference between coding a website - > major or otherwise - in an "old-fashioned" compiled language and a > non-compiled language, except for the amount of hoursepower and memory you > require? > > Old-fashioned doesn't mean bad, inappropriate, or inferior. It's just not the > latest-and-greatest, however it's currently defined by the geek fashion > police. Our experience coding web sites with C/C++ versus Perl is about a factor of ten in productivity. We only use C/C++ for CPU-intensivecalculations, such as scientific prediction code. Everything else is Perl or Java. I recently re-coded 10,000 lines of C into 650 lines of Perl. Why? String handling, hash tables, and the simplicity ofDBD/DBI. And there was no loss of performance, because the app was strictly I/O bound (that is, Postgres was I/O bound). Sure, the old app may not have been optimal, but we're talking about a factor of 15 reduction in lines of code. That's not "geek fashion", it's good engineering. Pick the best tool for the job, and learn how to use it. Craig
On Wednesday 29 March 2006 22:01, Craig A. James wrote: > This is off-topic for this group so I'll just give a brief reply; I'm happy > to carry on more just between the two of us... > > Gorshkov wrote: > > That being said ..... what *is* the difference between coding a website - > > major or otherwise - in an "old-fashioned" compiled language and a > > non-compiled language, except for the amount of hoursepower and memory > > you require? > > > > Old-fashioned doesn't mean bad, inappropriate, or inferior. It's just not > > the latest-and-greatest, however it's currently defined by the geek > > fashion police. > > Our experience coding web sites with C/C++ versus Perl is about a factor of > ten in productivity. We only use C/C++ for CPU-intensive calculations, > such as scientific prediction code. Everything else is Perl or Java. > > I recently re-coded 10,000 lines of C into 650 lines of Perl. Why? String > handling, hash tables, and the simplicity of DBD/DBI. And there was no > loss of performance, because the app was strictly I/O bound (that is, > Postgres was I/O bound). Sure, the old app may not have been optimal, but > we're talking about a factor of 15 reduction in lines of code. Sounds to me like the C programmers in your past needed to learn how to re-use code and make libraries. That's not a function of the language - that's a function of the programmer. > > That's not "geek fashion", it's good engineering. Pick the best tool for > the job, and learn how to use it. > Thanks for making my point. You choose the best tool for the job, and sometimes it's "old-fashioned". Please remember that - there may be newbies out there who think that if they're not using the latest alpha-beta-zeta version .0006-a-r1, then they must be bad programmers.
Hi, Craig, Craig A. James wrote: > I hope this was just a joke. You should be sure to clarify - there > might be some newbie out there who thinks you are seriously suggesting > coding major web sites in some old-fashioned compiled language. No, but perhaps with a CMS that pregenerates static content, or http://www.tntnet.org/ Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org
cjames@modgraph-usa.com ("Craig A. James") writes: > Gorshkov wrote: >> /flame on >> if you were *that* worried about performance, you wouldn't be using >> PHP or *any* interperted language >> /flame off >> sorry - couldn't resist it :-) > > I hope this was just a joke. You should be sure to clarify - there > might be some newbie out there who thinks you are seriously > suggesting coding major web sites in some old-fashioned compiled > language. Actually, this seems not so bad a point... If people are so interested in micro-managing certain bits of how performance works, then it seems an excellent question to ask why NOT write all the CGIs in C. After all, CGI in C *won't* suffer from the performance troubles associated with repetitively loading in Perl/PHP frameworks (which is why things like FastCGI, mod_perl, and such came about), and you can get a fair level of assurance that the compiled C won't be the performance bottleneck. And yes, it does become natural to ask "why not write CGIs in ASM?" ;-) -- (format nil "~S@~S" "cbbrowne" "acm.org") http://www3.sympatico.ca/cbbrowne/lisp.html "When I was a boy of fourteen, my father was so ignorant I could hardly stand to have the old man around. But when I got to be twenty-one, I was astonished at how much the old man had learned in seven years." -- Mark Twain
On Thu, 2006-03-30 at 11:22, Chris Browne wrote: > cjames@modgraph-usa.com ("Craig A. James") writes: > > > Gorshkov wrote: > >> /flame on > >> if you were *that* worried about performance, you wouldn't be using > >> PHP or *any* interperted language > >> /flame off > >> sorry - couldn't resist it :-) > > > > I hope this was just a joke. You should be sure to clarify - there > > might be some newbie out there who thinks you are seriously > > suggesting coding major web sites in some old-fashioned compiled > > language. > > Actually, this seems not so bad a point... > > If people are so interested in micro-managing certain bits of how > performance works, then it seems an excellent question to ask why NOT > write all the CGIs in C. > > After all, CGI in C *won't* suffer from the performance troubles > associated with repetitively loading in Perl/PHP frameworks (which is > why things like FastCGI, mod_perl, and such came about), and you can > get a fair level of assurance that the compiled C won't be the > performance bottleneck. > > And yes, it does become natural to ask "why not write CGIs in ASM?" > ;-) But as an aside, I've been load testing our web application. We have, in the test farm, two tomcat servers feeding into three jboss servers, feeding into a database farm (oracle and postgresql, doing different things, oracle is the transaction engine, postgresql is the "data collection bucket" so to speak.) Our tomcat servers sit at 10% load, the jboss servers sit at 20 to 40% load, and the Oracle server sits at 100% load. And the thing is, while we can add load balanced tomcat and jboss servers as need be, and get nearly linear scaling from them, we can't do the same for the database. That's going to require vertical scaling. And that, nowadays, is generally the state of web development. It's not the language you're using to write it in, it's how efficiently you're using your database. We can probably tweak the system we're testing now and get more from our databases by adjusting how hibernate hits them, and the types of queries that it's throwing, but in the long run, the bottleneck will always be the database server, because we can throw relatively small amounts of money at the other layers if they happen to be bogging down. Not so much with the database.
>> And yes, it does become natural to ask "why not write CGIs in ASM?" >> ;-) Personally, I'd code it in brainfuck, for aesthetic reasons. > And that, nowadays, is generally the state of web development. It's not > the language you're using to write it in, it's how efficiently you're > using your database. We can probably tweak the system we're testing now > and get more from our databases by adjusting how hibernate hits them, > and the types of queries that it's throwing, but in the long run, the > bottleneck will always be the database server, because we can throw > relatively small amounts of money at the other layers if they happen to > be bogging down. Not so much with the database. So, one wonders why some use 70's languages like Java instead of Lisp or Python, which are slower, but a lot more powerful and faster to develop in... (and don't have hibernate, which is a big bonus) (why do you think I don't like Java ?)
On Thu, Mar 30, 2006 at 11:31:25PM +0200, PFC wrote: > So, one wonders why some use 70's languages like Java instead of > Lisp or Python, which are slower, but a lot more powerful and faster to > develop in... > (and don't have hibernate, which is a big bonus) > (why do you think I don't like Java ?) Python may not have Hibernate, but it has even worse stuff trying to do about the same thing. :-) Anyhow, this is rapidly becoming offtopic for the original thread. /* Steinar */ -- Homepage: http://www.sesse.net/
smarlowe@g2switchworks.com (Scott Marlowe) writes: > And that, nowadays, is generally the state of web development. It's > not the language you're using to write it in, it's how efficiently > you're using your database. Which properly puts my comments in their place :-). More importantly, that seems like a valid statement which has a *wide* scope of effects and side-effects. Including some that ought to put PostgreSQL in a very good place, in that it provides some very good ways of achieving high efficiency. Neat performance thing du jour: Hibernate seems to be the "neat new Java persistence thing." I have been very unimpressed with some of the web frameworks I have seen thus far in their interaction with databases. We use RT (Request Tracker) for tracking tickets, and in its attempt to be "database agnostic," it actually only achieves being MySQL(tm)-specific, because they have an automated query generator that is only good at one style of queries at a time. Coworkers have suggested improved queries that are (on occasion) hundreds or thousands of times faster than what it generates; those improvements fall on deaf ears because they wouldn't work with all the databases. (Well, more precisely, they wouldn't work with MySQL(tm).) There's a home grown flavor of Java persistence mapping; it doesn't seem as heinous as RT's, but it still doesn't make it overly convenient to replace poor queries with more efficient ones. Hibernate has a nifty thing in the form of "Named Queries." It'll often use its own "HQL" to auto-generate SQL, but any time the DBAs come up with something that's nicely tuned, it seems to be highly recommended to generate a "Named Query" for that which allows a Nice Query to be made part of the application without too much weeping and gnashing of teeth on either DBA or developers' sides. A framework that allows you to thereby "soup up" how efficiently you use your database... Hmm... I wonder if that fits into anyone's notable quote? :-). -- (format nil "~S@~S" "cbbrowne" "acm.org") http://www3.sympatico.ca/cbbrowne/lisp.html "When I was a boy of fourteen, my father was so ignorant I could hardly stand to have the old man around. But when I got to be twenty-one, I was astonished at how much the old man had learned in seven years." -- Mark Twain
jython is a full rewrite of python in java and interface naturally with java classes, therefore hibernate ... and is just as easy as python.
Steinar H. Gunderson a écrit :
Steinar H. Gunderson a écrit :
On Thu, Mar 30, 2006 at 11:31:25PM +0200, PFC wrote:So, one wonders why some use 70's languages like Java instead of Lisp or Python, which are slower, but a lot more powerful and faster to develop in...(and don't have hibernate, which is a big bonus)(why do you think I don't like Java ?)Python may not have Hibernate, but it has even worse stuff trying to do about the same thing. :-) Anyhow, this is rapidly becoming offtopic for the original thread. /* Steinar */
On 30.03.2006, at 23:31 Uhr, PFC wrote: > (why do you think I don't like Java ?) Because you haven't used a good framework/toolkit yet? Come on, the language doesn't really matter these days, it's all about frameworks, toolkits, libraries, interfaces and so on. But, nevertheless, this has nothing to do with a decision between PostgreSQL or MySQL. They can both be accessed by a myriad of programming languages, so the decision may (and should) be based on other things. cug -- PharmaLine, Essen, GERMANY Software and Database Development