Thread: MemSQL the "world's fastest database"?
Any thoughts about this? It seems to be a new database system designed from scratch to take advantage of the growth in RAM size (data sets that fit in memory) and the availability of SSD drives. It claims to be "the world's fastest database."
http://www.i-programmer.info/news/84-database/4397-memsql-80000-queries-per-second.html
It's hard to see at a glance if this is a robust system suitable for monetary transactions, or just a fast-but-lossy system that you'd use for social twitter.
Craig
http://www.i-programmer.info/news/84-database/4397-memsql-80000-queries-per-second.html
It's hard to see at a glance if this is a robust system suitable for monetary transactions, or just a fast-but-lossy system that you'd use for social twitter.
Craig
On 06/25/2012 11:25 AM, Craig James wrote: > Any thoughts about this? It seems to be a new database system designed > from scratch to take advantage of the growth in RAM size (data sets that > fit in memory) and the availability of SSD drives. It claims to be "the > world's fastest database." I personally don't put a lot of stock into this. You can get 90k+ TPS from an old PostgreSQL 8.2 install if it's all in memory. High transactional output itself isn't substantially difficult to achieve. I'm also not entirely certain how this is different from something like VoltDB, which also acts as an in-memory database with high TPS throughput. Then there's this from the article: "The key ideas are that SQL code is translated into C++, so avoiding the need to use a slow SQL interpreter, and that the data is kept in memory, with disk read/writes taking place in the background." Besides the nonsense statement that SQL is translated to C++ (Lexical scanners would circumvent even this step, and does that mean you have to literally compile the resulting C++? Ridiculous.) This violates at least the 'D' tenet of ACID. Fine for transient Facebook data, but not going anywhere near our systems. -- Shaun Thomas OptionsHouse | 141 W. Jackson Blvd. | Suite 500 | Chicago IL, 60604 312-444-8534 sthomas@optionshouse.com ______________________________________________ See http://www.peak6.com/email_disclaimer/ for terms and conditions related to this email
Craig James <cjames@emolecules.com> wrote: > It claims to be "the world's fastest database." > [link where they boast of 80,000 tps read-only] 20,000 tps? Didn't we hit well over 300,000 tps in read-only benchmarks of PostgreSQL with some of the 9.2 performance enhancements? -Kevin
On 6/25/12 10:23 AM, Kevin Grittner wrote: > Craig James <cjames@emolecules.com> wrote: > >> It claims to be "the world's fastest database." > >> [link where they boast of 80,000 tps read-only] > > 20,000 tps? Didn't we hit well over 300,000 tps in read-only > benchmarks of PostgreSQL with some of the 9.2 performance > enhancements? Yes. The dirty truth is that there's nothing special, performance-wise, about an "in memory" database except that it doesn't write to disk (or protect your data from power-out). In the early 00's people thought that you could build a database in some fundamentally different way if you started with the tenet that it was 100% in memory. Hence RethinkDB, MySQL InMemory Tabletype, etc. As it turns out, that doesn't change anything; you still need data pages, indexes, sort routines, etc. etc. "Disk" databases don't operate off disk; they get copied to memory, so they're already effectively "in memory". BTW, VoltDB's innovation is not being "in memory" (it can spill to disk), but rather their innovative transactional clustering approach. The new non-relational databases are "fast" on poor hardware (Amazon, developer laptops) by cutting features and optimizing for poor (but common) access patterns. Not by being "in memory", which is just a side effect of not having spill-to-disk code. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
---- Original message ---- >Date: Mon, 25 Jun 2012 12:03:10 -0500 >From: pgsql-performance-owner@postgresql.org (on behalf of Shaun Thomas <sthomas@optionshouse.com>) >Subject: Re: [PERFORM] MemSQL the "world's fastest database"? >To: Craig James <cjames@emolecules.com> >Cc: <pgsql-performance@postgresql.org> > >On 06/25/2012 11:25 AM, Craig James wrote: > >> Any thoughts about this? It seems to be a new database system designed >> from scratch to take advantage of the growth in RAM size (data sets that >> fit in memory) and the availability of SSD drives. It claims to be "the >> world's fastest database." > >I personally don't put a lot of stock into this. You can get 90k+ TPS >from an old PostgreSQL 8.2 install if it's all in memory. High >transactional output itself isn't substantially difficult to achieve. > >I'm also not entirely certain how this is different from something like >VoltDB, which also acts as an in-memory database with high TPS throughput. > >Then there's this from the article: > >"The key ideas are that SQL code is translated into C++, so avoiding the >need to use a slow SQL interpreter, and that the data is kept in memory, >with disk read/writes taking place in the background." > >Besides the nonsense statement that SQL is translated to C++ (Lexical >scanners would circumvent even this step, and does that mean you have to >literally compile the resulting C++? Ridiculous.) This violates at least >the 'D' tenet of ACID. Fine for transient Facebook data, but not going >anywhere near our systems. DB2 on the mainframe (if memory serves), for one, will compile static SQL to machine code. Not that unusual. http://www.mainframegurukul.com/tutorials/database/db2_tutorials/DB2Precompilebind.html http://www.mainframegurukul.com/tutorials/database/db2_tutorials/sample-db2-cobol-compile-jcl.html > >-- >Shaun Thomas >OptionsHouse | 141 W. Jackson Blvd. | Suite 500 | Chicago IL, 60604 >312-444-8534 >sthomas@optionshouse.com > >______________________________________________ > >See http://www.peak6.com/email_disclaimer/ for terms and conditions related to this email > >-- >Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) >To make changes to your subscription: >http://www.postgresql.org/mailpref/pgsql-performance
<gnuoytr@rcn.com> writes:
>> Then there's this from the article:
>>
>> "The key ideas are that SQL code is translated into C++, so avoiding the
>> need to use a slow SQL interpreter, and that the data is kept in memory,
>> with disk read/writes taking place in the background."
>>
>> Besides the nonsense statement that SQL is translated to C++ (Lexical
>> scanners would circumvent even this step, and does that mean you have to
>> literally compile the resulting C++? Ridiculous.) ...
> DB2 on the mainframe (if memory serves), for one, will compile static SQL to machine code. Not that unusual.
Yeah. Actually such techniques go back at least to the fifties (look up
"sort generators" sometime). They are out of fashion now because
(1) the achievable speed difference isn't what it once was, and
(2) programs that execute self-modified code are prone to seriously
nasty security issues. Get any sort of control over the code generator,
and you can happily execute anything you want.
regards, tom lane
On 06/25/2012 01:23 PM, Kevin Grittner wrote: > Craig James<cjames@emolecules.com> wrote: > >> It claims to be "the world's fastest database." > >> [link where they boast of 80,000 tps read-only] > > 20,000 tps? Didn't we hit well over 300,000 tps in read-only > benchmarks of PostgreSQL with some of the 9.2 performance > enhancements? It's 20K TPS on something that MySQL will only do 3.5 TPS. The queries must be much heavier than the ones PostgreSQL can get 200K+ on. We'd have to do a deeper analysis of the actual queries used to know exactly how much heavier though. They might be the type MySQL is usually faster than PostgreSQL on (i.e. ones using simple operations and operators), or they could be ones where PostgreSQL is usually faster than MySQL (i.e. more complicated joins). All I can tell you for sure if that they used a query mix that makes MemSQL look much faster than MySQL. -- Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com
On Sat, Jun 30, 2012 at 10:18 PM, Greg Smith <greg@2ndquadrant.com> wrote: > On 06/25/2012 01:23 PM, Kevin Grittner wrote: >> >> Craig James<cjames@emolecules.com> wrote: >> >>> It claims to be "the world's fastest database." >> >> >>> [link where they boast of 80,000 tps read-only] >> >> >> 20,000 tps? Didn't we hit well over 300,000 tps in read-only >> benchmarks of PostgreSQL with some of the 9.2 performance >> enhancements? > > > It's 20K TPS on something that MySQL will only do 3.5 TPS. The queries must > be much heavier than the ones PostgreSQL can get 200K+ on. We'd have to do > a deeper analysis of the actual queries used to know exactly how much > heavier though. They might be the type MySQL is usually faster than > PostgreSQL on (i.e. ones using simple operations and operators), or they > could be ones where PostgreSQL is usually faster than MySQL (i.e. more > complicated joins). All I can tell you for sure if that they used a query > mix that makes MemSQL look much faster than MySQL. Considering I can build a pgsql 8.4 machine with 256G RAM and 64 Opteron cores and a handful of SSDs or HW RAID that can do REAL 7k to 8k RW TPS right now for well under $10k, 20k TPS on an in memory database isn't all that impressive. I wonder what numbers pg 9.1/9.2 can / will be able to pull off on such hardare?
It sounds like a lot of marketing BS :) But I like the fact that they use modern language like C++. It is a pain to try doing any development on postgresql. Transition to c++ would be nice (I know it's been debated on #hackers a looot).
On 07/01/2012 01:00 AM, Scott Marlowe wrote: > Considering I can build a pgsql 8.4 machine with 256G RAM and 64 > Opteron cores and a handful of SSDs or HW RAID that can do REAL 7k to > 8k RW TPS right now for well under $10k, 20k TPS on an in memory > database isn't all that impressive. Again, their TPS numbers are useless without a contest of how big each transaction is, and we don't know. I can take MemSQL seriously when there's a press release describing how to replicate their benchmark independently. Then it's useful to look at the absolute number. -- Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com