Thread: Postgres vr.s Oracle
Thought I'd point this blog post out to the list: http://www.miketec.org/serendipity/index.php?/archives/7-Oracle-and-Postgres-Redux.html Brian
On Wed, Dec 10, 2008 at 11:08 AM, Brian Hurt <bhurt@janestcapital.com> wrote: > Thought I'd point this blog post out to the list: > http://www.miketec.org/serendipity/index.php?/archives/7-Oracle-and-Postgres-Redux.html I've always heard the performance comparisons violated Oracle agreements. Does that not apply in this case? -- Regards, Richard Broersma Jr. Visit the Los Angeles PostgreSQL Users Group (LAPUG) http://pugs.postgresql.org/lapug
Richard Broersma wrote: > On Wed, Dec 10, 2008 at 11:08 AM, Brian Hurt <bhurt@janestcapital.com> wrote: > > >> Thought I'd point this blog post out to the list: >> http://www.miketec.org/serendipity/index.php?/archives/7-Oracle-and-Postgres-Redux.html >> > > I've always heard the performance comparisons violated Oracle > agreements. Does that not apply in this case? > > > I'm not the person to ask. Brian
> I'm not the person to ask. I suspect that if the legal department in your company saw that they'd ask you to take it down, especially after Oracle contacts them. And I wouldn't be surprised if Oracle contacted them, especially with the results you posted. Thankfully, those of us whose browsers have those materials cached on disk aren't subject to any Oracle agreements. :) -- ----- http://www.globalherald.net/jb01 GlobalHerald.NET, the Smarter Social Network! (tm)
Interesting, but, 8i? It would have been more interesting if it was a somewhat less prehistoric version of Oracle...
bhurt@janestcapital.com (Brian Hurt) writes: > Thought I'd point this blog post out to the list: > http://www.miketec.org/serendipity/index.php?/archives/7-Oracle-and-Postgres-Redux.html There's actually something unfair about the comparison, in an "unfair to Oracle" way, namely that he's using version 8i, which dates back to the PostgreSQL 6.5 days ;-). And vis-a-vis the comments about it possibly being not-permissible to release numbers, it's quite possible that 8i is old enough that when the guy bought his licenses, there wasn't such a clause in the purchase contract. And that's purest speculation, so it could indeed be not-permissible, and the guy may look forward to having lawyers-with-samurai-swords knock on his door ;-). -- let name="cbbrowne" and tld="linuxfinances.info" in name ^ "@" ^ tld;; http://linuxfinances.info/info/internet.html The software isn't finished until the last user is dead.
On Wed, 10 Dec 2008, Chris Browne wrote: > And vis-a-vis the comments about it possibly being not-permissible to > release numbers, it's quite possible that 8i is old enough that when > the guy bought his licenses, there wasn't such a clause in the > purchase contract. Doubt it. 8i shipped in March of 1999. July 30, 1999 was when the infamous "See The Test That Wasn't" article in PC Magazine talked about how Oracle wouldn't allow them to publish the benchmarks they'd done. http://www.infoworld.com/articles/op/xml/00/01/24/000124opfoster.html talks about it (that's where I remember it from, who reads PC Mag?), the commentary there says "Because Oracle's license is not a shrinkwrap per se but an actual signed contract, PC Magazine didn't have a way to challenge the non-disclosure term". While I was poking around looking that up again, I actually found a couple of places what look like legit Oracle contracts are posted at; glancing at these two: http://contracts.onecle.com/bearingpoint/oracle.collab.1995.11.22.shtml http://www.techagreements.com/agreement-preview.aspx?num=23093 Suggests that it's very unlikely the publication of results was within the realm of any contract with them. Mike's earlier blog posting has some choice comments in it too: http://www.miketec.org/serendipity/index.php?/archives/5-Oracle-prices-themselves-out.html "I have one app running on Oracle and three that run on a Postgres database. They run on similar hardware, yet the Oracle server is the one that gives me the performance problems. Postgres gets hammered all day long by the user's poorly written reports and handles it just fine." I get the feeling he's not real concerned about if Oracle gets mad at him. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
On Wed, Dec 10, 2008 at 2:08 PM, Brian Hurt <bhurt@janestcapital.com> wrote: > Thought I'd point this blog post out to the list: > http://www.miketec.org/serendipity/index.php?/archives/7-Oracle-and-Postgres-Redux.html Hmm... I wonder how scientific his benchmarking is and how well his Oracle system was tuned. Because I've done quite a few performance comparisons between Postgres 8.3 and 8.4-dev against a well-tuned Oracle8i instance on Linux, and 8i (from 1999) beats the latest versions of Postgres quite handily on the same hardware. And, while I have received permission from Oracle to publish the result, I haven't had the time to write up the blog entry yet. Outside of simple curiosity, my reason for running the benchmark was simply to show that in terms of performance, Oracle had it right over 10 years ago and that our continual discussions about leaving things to the OS and file system developers (because they know how to manage memory/data better than we do) is pointless. It illustrates that if Postgres ever wants to step into this century and take advantage of newer hardware configurations, we need to accept the fact that PG's inherent design has serious performance-related flaws which need to be addressed sooner rather than later. Similarly, I ran the same tests against Oracle 10g and 11g, and a properly tuned Oracle system is 10-100x faster than Postgres on lots of operations in both OLTP and DSS workloads, but because I didn't expect Postgres to be close to Oracle these days, I went back to comparing against 8i (Standard Edition) just to make my point. Regardless of what some people on this list tell you, one of the main reasons Oracle and other vendors don't like people performing external benchmarks is because the majority of people screw them up. Proper benchmarking is something that takes time to do and you have to have a good amount of experience ensuring the SUT environment is exactly the same for both databases. Similarly, the majority of people don't know how to tune an Oracle system properly, which is why they get bad results. Whether that's the case in this test or not, is unknown. There certainly isn't enough information in that blog entry to determine if the system was tuned properly or whether the tests were even performed correctly, yet it was forwarded on as gospel. Similarly, the benchmark was performed by someone a bit jaded against Oracle, which doesn't bode well for presuming he was unbiased. I can't believe the number of people on these lists who continually forward piss-poor benchmarks just because they think it somehow solidifies their belief that PG is somehow equal to or better than commercial databases in terms of overall performance. Sure, there are cases where PG is faster than Oracle... but they few and far between. If Postgres works well for you, use it. If not, use something else... just please don't push half-baked benchmarks performed by jaded people who are so uninformed that they think Don Burleson is the be-all-end-all of Oracle tuning consultants. -Jonah
On Sat, 2008-12-13 at 13:06 -0500, Jonah H. Harris wrote: > On Wed, Dec 10, 2008 at 2:08 PM, Brian Hurt <bhurt@janestcapital.com> wrote: > > Thought I'd point this blog post out to the list: > > http://www.miketec.org/serendipity/index.php?/archives/7-Oracle-and-Postgres-Redux.html > > Hmm... I wonder how scientific his benchmarking is and how well his > Oracle system was tuned. Because I've done quite a few performance > comparisons between Postgres 8.3 and 8.4-dev against a well-tuned > Oracle8i instance on Linux, and 8i (from 1999) beats the latest > versions of Postgres quite handily on the same hardware. And, while I > have received permission from Oracle to publish the result, I haven't > had the time to write up the blog entry yet. <major snippage> > who are so uninformed that they think Don Burleson is the > be-all-end-all of Oracle tuning consultants. Where is your patch? Joshua D. Drake > > -Jonah > -- PostgreSQL Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997
On Sat, Dec 13, 2008 at 1:09 PM, Joshua D. Drake <jd@commandprompt.com> wrote: > Where is your patch? What's the point? I've had the same arguments and discussion for years and (per you and almost everyone else) anything significant has to be discussed on-list prior to acceptance. It's almost pointless to bring this stuff up anymore. -Jonah
On Sat, 2008-12-13 at 14:09 -0500, Jonah H. Harris wrote: > On Sat, Dec 13, 2008 at 1:09 PM, Joshua D. Drake <jd@commandprompt.com> wrote: > > Where is your patch? > > What's the point? I've had the same arguments and discussion for > years and (per you and almost everyone else) anything significant has > to be discussed on-list prior to acceptance. It's almost pointless to > bring this stuff up anymore. My point is your signal to noise ratio is off. You very well could be correct (in fact I think you probably are) but it is irrelevant because all you do is hand wave. I am sure there are plenty of people in this community that would chew up and digest a valid benchmark from a Oracle/PostgreSQL expert but since we never see them, it is really hard to justify the work it would take to make significant architectural changes. Sincerely, Joshua D. Drake -- PostgreSQL Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997
On Saturday 13 December 2008 13:06:33 Jonah H. Harris wrote: > On Wed, Dec 10, 2008 at 2:08 PM, Brian Hurt <bhurt@janestcapital.com> wrote: > > Thought I'd point this blog post out to the list: > > http://www.miketec.org/serendipity/index.php?/archives/7-Oracle-and-Postg > >res-Redux.html > > Hmm... I wonder how scientific his benchmarking is and how well his > Oracle system was tuned. Because I've done quite a few performance > comparisons between Postgres 8.3 and 8.4-dev against a well-tuned > Oracle8i instance on Linux, and 8i (from 1999) beats the latest > versions of Postgres quite handily on the same hardware. And, while I > have received permission from Oracle to publish the result, I haven't > had the time to write up the blog entry yet. > Curious, but does your "well tuned" system include query hints? I've certainly seen cases where Oracles hinting system allowed them to get to places that Postgres couldn't get to, but without hinting things tend to be much closer (not necessarily even; it still has other advantages like advanced sql syntax) In my mind this is an argument for adding query hinting to postgres, but I hate to even mention that idea :-) > Outside of simple curiosity, my reason for running the benchmark was > simply to show that in terms of performance, Oracle had it right over > 10 years ago and that our continual discussions about leaving things > to the OS and file system developers (because they know how to manage > memory/data better than we do) is pointless. I think you have misjudged the argument, which would be (IMHO) more accurately stated that given this projects resources, we will get better performance by leveraging os/filesystem improvements rather than circumventing them. One such example is a patch for doing auto-alignment of columns at the disk layer for optimal performance. This patch is relatively simple (for this discussion), yet the idea has been around for years; if we can't knock that out, do you really think we have the resources to move towards direct hardware interaction? <snip> > Regardless of what some people on this list tell you, one of the main > reasons Oracle and other vendors don't like people performing external > benchmarks is because the majority of people screw them up. Proper > benchmarking is something that takes time to do and you have to have a > good amount of experience ensuring the SUT environment is exactly the > same for both databases. Similarly, the majority of people don't know > how to tune an Oracle system properly, which is why they get bad > results. Whether that's the case in this test or not, is unknown. Well, there is something to be said for having a database system so complex that properly tuning it can't be done even by experienced users. Postgres is an order of magnitude easier to configure (imho) and we still get beat up for it. -- Robert Treat Conjecture: http://www.xzilla.net Consulting: http://www.omniti.com
On Sat, Dec 13, 2008 at 2:28 PM, Joshua D. Drake <jd@commandprompt.com> wrote: > My point is your signal to noise ratio is off. You very well could be > correct (in fact I think you probably are) but it is irrelevant because > all you do is hand wave. I agree with that, which is why I did it and asked Oracle for permission to publish it. My goal was to point out that the benchmark was far from being authoritative, repeatable, or well-understood and that such things are almost baseless as a valid comparison. > I am sure there are plenty of people in this > community that would chew up and digest a valid benchmark from a > Oracle/PostgreSQL expert but since we never see them, it is really hard > to justify the work it would take to make significant architectural > changes. I likely did get carried away. I hope to find time to write up my findings along with everything needed to repeat the tests soon. -Jonah
On Sat, Dec 13, 2008 at 3:38 PM, Robert Treat <xzilla@users.sourceforge.net> wrote: > Curious, but does your "well tuned" system include query hints? No. > I've certainly seen cases where Oracles hinting system allowed them to get to places that > Postgres couldn't get to, but without hinting things tend to be much closer > (not necessarily even; it still has other advantages like advanced sql > syntax) In my mind this is an argument for adding query hinting to postgres, > but I hate to even mention that idea :-) I've always agreed with adding hints... but as we all know, that's a different topic altogether. > I think you have misjudged the argument, which would be (IMHO) more accurately > stated that given this projects resources, we will get better performance by > leveraging os/filesystem improvements rather than circumventing them. One > such example is a patch for doing auto-alignment of columns at the disk layer > for optimal performance. This patch is relatively simple (for this > discussion), yet the idea has been around for years; if we can't knock that > out, do you really think we have the resources to move towards direct > hardware interaction? I understand the resource limitation. That's not the problem. The problem, from my point of view, is that there's a near-constant rejection against the possibility of using fairly well-known and greatly accepted implementations. Examples are the "Features We Do Not Want" in the TODO, and the "Why don't you use threads, raw devices, async-I/O, <insert your favorite wizz-bang feature here>?" section of the FAQ. For examples, we use threads and async I/O. Threading has been supported by every major OS for how long now? How buggy is it really? A heck of a lot of stuff is written using threads and it runs continuously and under heavy load without a single problem. Asynchronous I/O... Oracle8i supported it in 1999 (and in earlier versions if you knew how to enable it). My point is simply that this isn't new or untested technology, and that we should at least be open to it. But when our own FAQ calls threading buggy, crash-prone, overly complex, and not worth it from a performance standpoint, it makes a lot of people question the amount of Postgres community experience related to performance engineering. Similarly, it makes it difficult for those of us with experience using these technologies to even have discussions about them on-list. > Well, there is something to be said for having a database system so complex > that properly tuning it can't be done even by experienced users. Postgres is > an order of magnitude easier to configure (imho) and we still get beat up for > it. Well, people will always find something to complain about with Postgres, Oracle, and every other thing :) Oracle has tons of options because no workload can be characterized by a small number of knobs. If you know how something needs to work, nine times out of ten Oracle has a way for you to alter the system to make it work that way. Yes, that does make it more complex, and I'm not denying that at all. My point was simply that individuals without extensive experience tuning Oracle and performing scientific benchmarks aren't generally the best people to make valid overall comparisons. My main concern is based on talking to people at PG conferences who put far too much faith and belief into results derived from bad benchmarks. Again, I do not want this thread to be an argument of Oracle vs. Postgres. I just wanted to say that we should look at benchmarks which have some resemblance of scientific validity before simply passing them out or believing in them ourselves. This is open source and believing in our own benchmarketing is just going to hurt us long-term. -Jonah
"Jonah H. Harris" <jonah.harris@gmail.com> writes: > Threading has been supported by every major OS for how long now? How > buggy is it really? A heck of a lot of stuff is written using threads > and it runs continuously and under heavy load without a single > problem. Asynchronous I/O... Oracle8i supported it in 1999 (and in > earlier versions if you knew how to enable it). And it was buggy as hell. In any case 1999 is pretty recent, we support plenty of platforms which predate 1999. Threading is a programming model. If it's convenient to program using it then you use it. If it isn't there are several other ways to multi-thread your program without using OS threads. On modern operating systems (ie, excluding Windows) there's no functional difference between multiple processes with shared memory and threads. All the major operating systems use a single-level scheduler where processes and threads are treated identically. The only difference between them is the programmer convenience of having all memory shared by default instead of having all memory private by default. You pick your programming tools based on what's convenient for the program you're writing. You don't structure your whole program around wanting to use your favourite API whether that's threads or async i/o or whatever. There are plenty of other ways to get the same functionality. > My point is simply that this isn't new or untested technology, and that we > should at least be open to it. But when our own FAQ calls threading buggy, > crash-prone, overly complex, and not worth it from a performance standpoint, > it makes a lot of people question the amount of Postgres community > experience related to performance engineering. There are always people out there with all kinds of wacky ideas. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's PostGIS support!
On Saturday 13 December 2008 23:34:33 Jonah H. Harris wrote: > Again, I do not want this thread to be an argument of Oracle vs. > Postgres. I just wanted to say that we should look at benchmarks > which have some resemblance of scientific validity before simply > passing them out or believing in them ourselves. This is open source > and believing in our own benchmarketing is just going to hurt us > long-term. > Fair points, but one also needs to consider the audience of this list. The information wasn't posted on -performance or -hackers as some scientific validation, it was posted on -advocacy, where even a bad benchmark carries wieght because it is interesting for those of use who do advocacy related work to know what is being discussed even if the exact discussion is not exactly accurate. That said, it is very good that we have people willing to point out those inaccuracies; hopefully that keeps everyone on thier toes when talking with others. -- Robert Treat Conjecture: http://www.xzilla.net Consulting: http://www.omniti.com
Jonah, > Hmm... I wonder how scientific his benchmarking is and how well his > Oracle system was tuned. Because I've done quite a few performance > comparisons between Postgres 8.3 and 8.4-dev against a well-tuned > Oracle8i instance on Linux, and 8i (from 1999) beats the latest > versions of Postgres quite handily on the same hardware. What tests are you running? There are certainly things, performance-wise, which Oracle does better than us. There are also things they do worse. I'll point out that, the last time we had public comparables head-to-head, with *Oracle* doing Oracle's tuning on SpecJAppserver, PostgreSQL was around 90% of Oracle *10* on analogous hardware. In internal tests at Sun I can't quote directly, that comparison held on succeeding generations of hardware I wasn't allowed to publish :-( And I'd match the Sun performance labs' knowledge of Oracle tuning against yours any day. Not, on TPC-E on the other hand, Oracle is way ahead of us ... we were like 50% last I checked, due mostly to the amount of time it takes us to resolve lock conflicts vs. Oracle. Of course, Microsoft is beating Oracle by 50%, so Oracle is not the one to match on that benchmark. --Josh
Josh Berkus wrote: > Not, on TPC-E on the other hand, Oracle is way ahead of us ... we were > like 50% last I checked, due mostly to the amount of time it takes us > to resolve lock conflicts vs. Oracle. Of course, Microsoft is beating > Oracle by 50%, so Oracle is not the one to match on that benchmark. > > --Josh Here is the current TPC-E top 10 http://www.tpc.org/tpce/results/tpce_perf_results.asp where is oracle??? TCP-H http://www.tpc.org/tpch/results/tpch_perf_results.asp TCP-C http://www.tpc.org/tpcc/results/tpcc_perf_results.asp
On Sun, Dec 14, 2008 at 8:38 PM, justin <justin@emproshunts.com> wrote: > Here is the current TPC-E [H, C] top 10 > where is oracle??? Where you should be looking is at the price/performance benchmarks, because that's where Postgres plays. Last time I checked Postgres on a TPC-C, albeit being 100% free, was anywhere from $4.00 to $6.00 per transaction depending on the hardware. Compare that to Oracle's $0.68 or SQL Server's $0.84. Yeah, I expect the normal it's just an industry benchmark, it's not fair, it's not representative of real workloads or real performance response. Or, just for the fun of it, run Postgres on the 100GB TPC-H and let me know what you get for price/performance... then compare that to SQL Server's result from 2006. I do want to caution everyone though. The OSDL-DBT kits are *not* spec-compliant and have several flaws which make the results fairly untrustworthy for comparison purposes. The best TPC-C kit for Postgres I've seen is EnterpriseDB's version of the DBT-2. While it's still not spec-compliant, it fixes several major bugs and includes a more optimized schema. If you want a copy, you could petition them to release their modifications to it. -- Jonah H. Harris, Senior DBA myYearbook.com
On Sun, Dec 14, 2008 at 2:33 PM, Josh Berkus <josh@agliodbs.com> wrote: > What tests are you running? There are certainly things, performance-wise, > which Oracle does better than us. There are also things they do worse. Various benchmarks. So far I've fully performed DBT-2 and DBT-3. I've also been toying with a couple of the OSS benchmarks, but I'm not sure which I'd like to use. > I'll point out that, the last time we had public comparables head-to-head, > with *Oracle* doing Oracle's tuning on SpecJAppserver, PostgreSQL was around > 90% of Oracle *10* on analogous hardware. In internal tests at Sun I can't > quote directly, that comparison held on succeeding generations of hardware I > wasn't allowed to publish :-( Sucks that you weren't able to publish them before leaving :( > And I'd match the Sun performance labs' knowledge of Oracle tuning against > yours any day. Perhaps. I don't know anyone on Sun's performance team, so I can't speak to that directly. Though, I do seem to recall going over the Postgres configuration used on those benchmarks, and that it wasn't as optimized as it could've been. > Not, on TPC-E on the other hand, Oracle is way ahead of us ... we were like > 50% last I checked, due mostly to the amount of time it takes us to resolve > lock conflicts vs. Oracle. Of course, Microsoft is beating Oracle by 50%, > so Oracle is not the one to match on that benchmark. Yeah, Microsoft is doing quite well at TPC-E. -- Jonah H. Harris, Senior DBA myYearbook.com
Josh Berkus <josh@agliodbs.com> writes: > Not, on TPC-E on the other hand, Oracle is way ahead of us ... we were like 50% > last I checked, due mostly to the amount of time it takes us to resolve lock > conflicts vs. Oracle. Of course, Microsoft is beating Oracle by 50%, so Oracle > is not the one to match on that benchmark. We've done essentially no benchmark optimization so on any sufficiently complex benchmark there will undoubtedly be some piece that we basically are not handling correctly at all. The degree to which the problem, which is essentially a bug, causes a slowdown isn't really relevant. Whether it's 50% or 100% or more isn't really interesting. On the flip side Microsoft and Oracle spend a lot of effort into optimizing for the cases the benchmarks cover. Oracle beat Microsoft over the head with updateable views for a long time because it basically broke some of the older benchmarks and made them thousands of times faster according to those benchmarks. It doesn't surprise me that Microsoft is doing it to Oracle now. I'm amazed at how many people in this thread claim to have diagnosed performance problems that we've never heard about on-list though. What do you mean "due mostly to the amount of time it takes us to resolve lock conflicts"?!? Are you sure it isn't due to the cycles we spend buffering RI triggers and checking them one-by-one? 8.4 has some microoptimizations in this area but I'm thinking there are some algorithmic improvements we could make for batch updates. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's RemoteDBA services!
Jonah H. Harris wrote: > On Sun, Dec 14, 2008 at 8:38 PM, justin <justin@emproshunts.com> wrote: > >> Here is the current TPC-E [H, C] top 10 >> where is oracle??? >> > > Where you should be looking is at the price/performance benchmarks, > because that's where Postgres plays. Last time I checked Postgres on > a TPC-C, albeit being 100% free, was anywhere from $4.00 to $6.00 per > transaction depending on the hardware. Compare that to Oracle's $0.68 > or SQL Server's $0.84. Where are you getting the $4 and $6 per transaction for PostgreSql. i just search through this list http://www.tpc.org/tpcc/results/tpcc_results.asp?orderby=dbms it does not contain PostgreSql entry. all of the less than $1.00 per transaction are last the few years. Oracle only just in the last year dropped to below $1.00 it mixed bag from $3 to $52 (back on 2001) > Yeah, I expect the normal it's just an > industry benchmark, it's not fair, it's not representative of real > workloads or real performance response. > First Step in testing and comparing is agree on a Standard that everyone can agree to. Second step test the system without cheating which numerous software including Oracle, MS, and IBM have. > Or, just for the fun of it, run Postgres on the 100GB TPC-H and let me > know what you get for price/performance... then compare that to SQL > Server's result from 2006. > >
On Sun, Dec 14, 2008 at 10:17 PM, justin <justin@emproshunts.com> wrote: > Where are you getting the $4 and $6 per transaction for PostgreSql. i just > search through this list You won't see it there because none of the PG vendors see a reason to spend the time and money officially benchmarking Postgres only for it to place last. Similar to Josh Berkus, I know where it ranks because I (and several other EnterpriseDB developers) ran the tests to try and get Postgres a TPC-C. > Oracle only just in the last year dropped to below $1.00 it mixed bag from > $3 to $52 (back on 2001) Oracle hadn't run price/performance in awhile prior to that due to the cost of the software. Still, Microsoft has had it below $1 since 2005. > First Step in testing and comparing is agree on a Standard that everyone can > agree to. Second step test the system without cheating which numerous > software including Oracle, MS, and IBM have. Cheating? It's an industry standard benchmark. And, for the record, when I compared PG to Oracle on TPC-H, I didn't use Oracle's additional features, I did a one-to-one comparison using the exact same schema and indexes. -Jonah
Jonah H. Harris wrote: > Outside of simple curiosity, my reason for running the benchmark was > simply to show that in terms of performance, Oracle had it right over > 10 years ago and that our continual discussions about leaving things > to the OS and file system developers (because they know how to manage > memory/data better than we do) is pointless. It illustrates that if > Postgres ever wants to step into this century and take advantage of > newer hardware configurations, we need to accept the fact that PG's > inherent design has serious performance-related flaws which need to be > addressed sooner rather than later. Similarly, I ran the same tests > against Oracle 10g and 11g, and a properly tuned Oracle system is > 10-100x faster than Postgres on lots of operations in both OLTP and > DSS workloads, but because I didn't expect Postgres to be close to > Oracle these days, I went back to comparing against 8i (Standard > Edition) just to make my point. 10-100x? I am confused because sometimes I hear that Postgres has bad performance from ex-Oracle users, but in general I hear that Oracle and Postgres have similar performance behavior from people porting applications. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Jonah H. Harris wrote:
You can cheat at industry standards, I see it every day in the electrical field where i spend most of time.
The only way to verify the results is duplicate the test and given these servers cost from $100K to $10million its unlikely these test are verified by a 3rd party running independent hardware. From where i come from for result to be proven someone else must duplicate the results independently with only instructions given by the original tester.
I would like to see them even if PostgreSql comes dead last.On Sun, Dec 14, 2008 at 10:17 PM, justin <justin@emproshunts.com> wrote:Where are you getting the $4 and $6 per transaction for PostgreSql. i just search through this listYou won't see it there because none of the PG vendors see a reason to spend the time and money officially benchmarking Postgres only for it to place last. Similar to Josh Berkus, I know where it ranks because I (and several other EnterpriseDB developers) ran the tests to try and get Postgres a TPC-C.
Does that not prove the point Oracle is over price software compared to the competition.Oracle only just in the last year dropped to below $1.00 it mixed bag from $3 to $52 (back on 2001)Oracle hadn't run price/performance in awhile prior to that due to the cost of the software. Still, Microsoft has had it below $1 since 2005.
First Step in testing and comparing is agree on a Standard that everyone can agree to. Second step test the system without cheating which numerous software including Oracle, MS, and IBM have.Cheating? It's an industry standard benchmark. And, for the record, when I compared PG to Oracle on TPC-H, I didn't use Oracle's additional features, I did a one-to-one comparison using the exact same schema and indexes. -Jonah
You can cheat at industry standards, I see it every day in the electrical field where i spend most of time.
The only way to verify the results is duplicate the test and given these servers cost from $100K to $10million its unlikely these test are verified by a 3rd party running independent hardware. From where i come from for result to be proven someone else must duplicate the results independently with only instructions given by the original tester.
On Sun, Dec 14, 2008 at 11:05 PM, justin <justin@emproshunts.com> wrote: > I would like to see them even if PostgreSql comes dead last. Well, running an official TPC benchmark is an interesting process. First, you have to have a spec-compliant benchmark kit, which we don't. Second, you have to work with a hardware vendor, and none of the vendors are going to work with you if you can't make their hardware look good; they don't want to come in last even if it's due to your software. I know, I talked to three of the major hardware vendors. Lastly, it costs a fair amount of money and time to perform an audited benchmark, which I don't think the community wants to pay for. > Does that not prove the point Oracle is over price software compared to the > competition. That depends on whether you need the performance or not. > The only way to verify the results is duplicate the test and given these > servers cost from $100K to $10million its unlikely these test are verified > by a 3rd party running independent hardware. From where i come from for > result to be proven someone else must duplicate the results independently > with only instructions given by the original tester. Perhaps you should download a full disclosure report of a TPC benchmark. First of all, they're all audited. Second of all, every configuration detail is provided for you to be able to run the test yourself. In fact, if you have a good enough reason to do so (i.e. looking to buy a good amount of software), both Oracle and Microsoft will give you copies of their benchmark kits for evaluation purposes. -- Jonah H. Harris, Senior DBA myYearbook.com
On Sun, Dec 14, 2008 at 10:47 PM, Bruce Momjian <bruce@momjian.us> wrote: > 10-100x? > > I am confused because sometimes I hear that Postgres has bad performance > from ex-Oracle users, but in general I hear that Oracle and Postgres > have similar performance behavior from people porting applications. It depends on the application, whether they had tuned Oracle or not, and whether the performance of their application was really noticeable to begin with. Many times at EDB, we ran into people who had stock, untuned Oracle installs. Personally, I don't think that comparing a tuned Postgres system to an untuned Oracle system is a very fair comparison. Though, there were a couple customer applications that did run faster on Postgres (due to ip4r or Postgres' native integer/floating point data types not present in the customers version of Oracle). The OLTP cases where Oracle starts to outperform Postgres is usually around 25 heavy concurrent sessions. When you start scaling into hundreds of sessions, Oracle really starts to shine. If, however, you're running 10 or so concurrent users, Postgres will generally be faster simply because Oracle isn't optimized for small systems. This statement obviously varies based on what those 10 people are actually doing, because you could likely optimize lots of things using specific Oracle features such as materialized views, more access paths, parallelism, etc. Of course, if an application works equally well on both Oracle and Postgres, I'd say go with Postgres. There's no need to pay for Oracle features you're not going to use. Arrg! I feel like I'm bashing on Postgres when I'm simply trying to defend Oracle from a poorly written benchmark blog. The fact is, both systems are good and one should choose the tool that best fits their need, be that Oracle, Postgres, SQL Server, or (yes, even) MySQL. I know this is the advocacy list, and my only hope was that we have a modicum of science behind a benchmark before putting too much into it. -Jonah
Jonah H. Harris wrote: <blockquote cite="mid:36e682920812142016x21d31a57q41894aa6b20a0108@mail.gmail.com" type="cite"><prewrap="">On Sun, Dec 14, 2008 at 11:05 PM, justin <a class="moz-txt-link-rfc2396E" href="mailto:justin@emproshunts.com"><justin@emproshunts.com></a>wrote: </pre><blockquote type="cite"><pre wrap="">Iwould like to see them even if PostgreSql comes dead last. </pre></blockquote><pre wrap=""> Well, running an official TPC benchmark is an interesting process. First, you have to have a spec-compliant benchmark kit, which we don't. Second, you have to work with a hardware vendor, and none of the vendors are going to work with you if you can't make their hardware look good; they don't want to come in last even if it's due to your software. I know, I talked to three of the major hardware vendors. Lastly, it costs a fair amount of money and time to perform an audited benchmark, which I don't think the community wants to pay for. </pre></blockquote><br /> Thats a problem with testing. Its not cheap or easy. I do this all the time with electricalequipment. The company I work for has 20K in a environmental testing chamber, 55K in power supplies 60K in DMMs which must be certified to NIST costing another 10K every year. This is only for the lab and QC.<br /><br /> So yesi know testing is very expensive but it can be done and should be done independently of the Hardware/Software manufactures. If not how are we to know that this hardware/Software is really a production item. Example of independentlab is Consumer Reports, created to test products without the manufactures influence over the net result as themanufactures were hoisting bogus testing reports on the public. <br /><br /> The more i learn how the tests are run themore suspicious i become its totally black bagged. The test method must be easy to understand be realistic to a realmodel that anyone can run with X time and X money on hand. Since you have stated that is near impossible to do, i wouldsay the testing methods need to be rethought. <br /><br /> Any one can test our product if they went out spend 40K onelectrical lab equipment and has a basic understand of electrical principles. There is no need to have an understandingof ISO 17025 <a class="moz-txt-link-freetext" href="http://en.wikipedia.org/wiki/ISO_17025">http://en.wikipedia.org/wiki/ISO_17025</a><br/><br /><br /><blockquote cite="mid:36e682920812142016x21d31a57q41894aa6b20a0108@mail.gmail.com"type="cite"><pre wrap=""></pre><blockquote type="cite"><prewrap="">Does that not prove the point Oracle is over price software compared to the competition. </pre></blockquote><pre wrap=""> That depends on whether you need the performance or not. </pre></blockquote> That don't make sense???? Its all about costif the other software is cheaper per transaction means you can spend the same amount of money as one would on Oracleon lets say MSSQL then blow by Oracle's performance<br /><blockquote cite="mid:36e682920812142016x21d31a57q41894aa6b20a0108@mail.gmail.com"type="cite"><pre wrap=""> </pre><blockquote type="cite"><prewrap="">The only way to verify the results is duplicate the test and given these servers cost from $100K to $10million its unlikely these test are verified by a 3rd party running independent hardware. From where i come from for result to be proven someone else must duplicate the results independently with only instructions given by the original tester. </pre></blockquote><pre wrap=""> Perhaps you should download a full disclosure report of a TPC benchmark. First of all, they're all audited. Second of all, every configuration detail is provided for you to be able to run the test yourself. In fact, if you have a good enough reason to do so (i.e. looking to buy a good amount of software), both Oracle and Microsoft will give you copies of their benchmark kits for evaluation purposes</pre></blockquote><br /> Used to be work under ISO 9000writing standards and dealing with Auditors. I know how auditing works and ways to make yourself look good and in compliancewhen its completely penciled wiped into shape. Given no 3rd party does completely independent testing i view itas an all around joke. <br /><br /> This is the reason we chose to drop ISO certification as it had nothing to do aboutdoing the job right <br />
On Mon, Dec 15, 2008 at 12:11 AM, justin <justin@emproshunts.com> wrote: > Thats a problem with testing. Its not cheap or easy. ... > So yes i know testing is very expensive but it can be done and should be > done independently of the Hardware/Software manufactures. Well, this is an open source community, and it doesn't have the money or the resources to perform such a benchmark. Similarly, regardless of your opinion regarding TPC database benchmarks, I (and several others) have run these tests ourselves and know for a fact that both Oracle and SQL Server's results aren't fake. Likewise, I (and other PG companies) know where PG stands in comparison. I'm not quite sure where you're coming from, because you obviously haven't performed any of these benchmarks yourself and are therefore a bit uninformed in regard to questioning their validity. -- Jonah H. Harris, Senior DBA myYearbook.com
Jonah H. Harris wrote: > Well, running an official TPC benchmark is an interesting process. > First, you have to have a spec-compliant benchmark kit, which we > don't. Well, then the first order of business would be to write a benchmark kit. I have been thinking for a while that we should make our own maintained version of the DBT+* suite, or whatever other suite is appropriate. And then start running it. The rest of this discussion, while interesting, cannot really lead to any improvements, because none of the results and analyses can be published. Like, you know, "I have these great insights into important problems, but I cannot tell you about them." Is it possible to write a possibly eventually compliant benchmark kit under open-source terms?
On Mon, Dec 15, 2008 at 3:25 AM, Peter Eisentraut <peter_e@gmx.net> wrote: > Well, then the first order of business would be to write a benchmark kit. I > have been thinking for a while that we should make our own maintained > version of the DBT+* suite, or whatever other suite is appropriate. And > then start running it. Agreed. For a start on TPC-C, EnterpriseDB has a closer version of the kit. > The rest of this discussion, while interesting, cannot really lead to any > improvements, because none of the results and analyses can be published. > Like, you know, "I have these great insights into important problems, but I > cannot tell you about them." I will soon tell you, because Oracle gave me permission to publish. > Is it possible to write a possibly eventually compliant benchmark kit under > open-source terms? I believe so. -- Jonah H. Harris, Senior DBA myYearbook.com
"Jonah H. Harris" <jonah.harris@gmail.com> writes: > On Sun, Dec 14, 2008 at 10:47 PM, Bruce Momjian <bruce@momjian.us> wrote: >> 10-100x? >> >> I am confused because sometimes I hear that Postgres has bad performance >> from ex-Oracle users, but in general I hear that Oracle and Postgres >> have similar performance behavior from people porting applications. I think there are problem cases where Postgres really doesn't perform well. That's the only time you're going to see a 10-100x factor anyways. Even if you compared SQL-Lite to Oracle you would expect to see vaguely comparable results unless you're hitting on a case that SQL-Lite just isn't intended to handle. For example there are some queries in the DBT-3 power test which Postgres doesn't optimize well. In some cases Postgres is just entirely lacking the ideal plan type. It's kind of pointless to talk about *how* much slower some random inappropriate plan is than some entirely different plan Oracle uses. Who cares if it's 10x or 100x or even 1000x slower. No amount of optimization is going to fix it unless we address the algorithmic problem anyways. > The OLTP cases where Oracle starts to outperform Postgres is usually > around 25 heavy concurrent sessions. When you start scaling into > hundreds of sessions, Oracle really starts to shine. Postgres is definitely not optimized for systems with many concurrent sessions. In the past we've basically dismissed such systems as misconfigured on the basis that there's not much point in having many more sessions than the number of processors in your system. If you have a big RAID array then it's possible the number of spindles might be more relevant than number of processors. But the point is that there's no performance advantage to configuring more sessions than the system can actually make progress on concurrently rather than queueing up the requests and processing them sequentially. There is a counter-argument to this however. If you have a system with complex multi-request transactions then it's a question of programming convenience. The easiest model to work with is to start a database session and use the database to maintain transaction state -- which is what it's for after all. That's more robust and convenient than using autocommit or otherwise trying to manage short stateless requests. But it does risk leaving you with relatively long-lived sessions which are often idle and forces you to have a large number of backends running. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's Slony Replication support!
Jonah, > Where you should be looking is at the price/performance benchmarks, > because that's where Postgres plays. Last time I checked Postgres on > a TPC-C, albeit being 100% free, was anywhere from $4.00 to $6.00 per > transaction depending on the hardware. Compare that to Oracle's $0.68 > or SQL Server's $0.84. You can't compare DBT2 with TPCC. They're not the same benchmark. --Josh
Josh Berkus <josh@agliodbs.com> writes: > Jonah, > >> Where you should be looking is at the price/performance benchmarks, >> because that's where Postgres plays. Last time I checked Postgres on >> a TPC-C, albeit being 100% free, was anywhere from $4.00 to $6.00 per >> transaction depending on the hardware. Compare that to Oracle's $0.68 >> or SQL Server's $0.84. > > You can't compare DBT2 with TPCC. They're not the same benchmark. Eh? DBT2 is a (partial) implementation of TPC-C. The nature of benchmarks is that small infelicities could invalidate the whole result though and there are definitely infelicities in DBT2's implementation of TPC-C. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL training!
On Mon, Dec 15, 2008 at 3:26 PM, Josh Berkus <josh@agliodbs.com> wrote: > You can't compare DBT2 with TPCC. They're not the same benchmark. Agreed. Somewhere earlier in the thread I mentioned that the OSDL kits are non-spec-compliant, a little buggy, and sometimes give better results than the real tests. -- Jonah H. Harris, Senior DBA myYearbook.com
Gregory Stark escreveu: > Eh? DBT2 is a (partial) implementation of TPC-C. The nature of benchmarks is > that small infelicities could invalidate the whole result though and there are > definitely infelicities in DBT2's implementation of TPC-C. > [I did't read the spec carefully but ...] Could you elaborate what infelicities are? I would be really nice to have some benchmark kit as close as possible from the specs. -- Euler Taveira de Oliveira http://www.timbira.com/
On Mon, Dec 15, 2008 at 6:17 AM, Jonah H. Harris <jonah.harris@gmail.com> wrote: > On Mon, Dec 15, 2008 at 3:25 AM, Peter Eisentraut <peter_e@gmx.net> wrote: >> Well, then the first order of business would be to write a benchmark kit. I >> have been thinking for a while that we should make our own maintained >> version of the DBT+* suite, or whatever other suite is appropriate. And >> then start running it. > > Agreed. For a start on TPC-C, EnterpriseDB has a closer version of the kit. I don't recall turning away patches to make the dbt kits less TPC spec compliant... Regards, Mark
On Mon, Dec 15, 2008 at 12:25 AM, Peter Eisentraut <peter_e@gmx.net> wrote: > Jonah H. Harris wrote: >> >> Well, running an official TPC benchmark is an interesting process. >> First, you have to have a spec-compliant benchmark kit, which we >> don't. > > Well, then the first order of business would be to write a benchmark kit. I > have been thinking for a while that we should make our own maintained > version of the DBT+* suite, or whatever other suite is appropriate. And > then start running it. We're slowing tuning a system for the dbt2 kit in Portland with the hardware from HP. > The rest of this discussion, while interesting, cannot really lead to any > improvements, because none of the results and analyses can be published. > Like, you know, "I have these great insights into important problems, but I > cannot tell you about them." > > Is it possible to write a possibly eventually compliant benchmark kit under > open-source terms? Yes. I believe the only restriction is to not use the kit in a way that competes with the TPC. Using the kits for our developing efforts is ok. Regards, Mark
"Mark Wong" <markwkm@gmail.com> writes: > On Mon, Dec 15, 2008 at 6:17 AM, Jonah H. Harris <jonah.harris@gmail.com> wrote: >> On Mon, Dec 15, 2008 at 3:25 AM, Peter Eisentraut <peter_e@gmx.net> wrote: >>> Well, then the first order of business would be to write a benchmark kit. I >>> have been thinking for a while that we should make our own maintained >>> version of the DBT+* suite, or whatever other suite is appropriate. And >>> then start running it. >> >> Agreed. For a start on TPC-C, EnterpriseDB has a closer version of the kit. > > I don't recall turning away patches to make the dbt kits less TPC spec > compliant... Uhm, why would we send them to you? Are you the maintainer for DBT2 now? I looked recently and couldn't find any upstream source that had been touched any time within the last few years. Where do i find it? -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's RemoteDBA services!
On Tue, Dec 16, 2008 at 12:14 PM, Gregory Stark <stark@enterprisedb.com> wrote: > "Mark Wong" <markwkm@gmail.com> writes: > >> On Mon, Dec 15, 2008 at 6:17 AM, Jonah H. Harris <jonah.harris@gmail.com> wrote: >>> On Mon, Dec 15, 2008 at 3:25 AM, Peter Eisentraut <peter_e@gmx.net> wrote: >>>> Well, then the first order of business would be to write a benchmark kit. I >>>> have been thinking for a while that we should make our own maintained >>>> version of the DBT+* suite, or whatever other suite is appropriate. And >>>> then start running it. >>> >>> Agreed. For a start on TPC-C, EnterpriseDB has a closer version of the kit. >> >> I don't recall turning away patches to make the dbt kits less TPC spec >> compliant... > > Uhm, why would we send them to you? Are you the maintainer for DBT2 now? I > looked recently and couldn't find any upstream source that had been touched > any time within the last few years. Where do i find it? http://sourceforge.net/projects/osdldbt/ Mark has been maintaining the DBT2 kit for a while now - since he was at OSDL. As far as recent updates, he's been focused on updating the results parsing and graphing libraries, which were written in Perl. I've been helping with that a little We've been presenting results -- first some IO tests at Linux Plumbers Conf, and the very beginnings of postgresql tuning results at Pg West. That's Mark, Gabrielle Roth and me. I'll be presenting some of our more recent results, and the donated HP hardware, at FOSDEM. (assuming my talk got accepted! :D) -selena -- Selena Deckelmann Open Source Bridge - http://www.opensourcebridge.org PDXPUG - http://pugs.postgresql.org/pdx Me - http://www.chesnok.com/daily
On Tue, Dec 16, 2008 at 12:29 PM, Selena Deckelmann <selenamarie@gmail.com> wrote: > On Tue, Dec 16, 2008 at 12:14 PM, Gregory Stark <stark@enterprisedb.com> wrote: >> Uhm, why would we send them to you? Are you the maintainer for DBT2 now? I >> looked recently and couldn't find any upstream source that had been touched >> any time within the last few years. Where do i find it? > > http://sourceforge.net/projects/osdldbt/ > > Mark has been maintaining the DBT2 kit for a while now - since he was at OSDL. Correction -- Mark created this kit in 2003. -selena -- Selena Deckelmann Open Source Bridge - http://www.opensourcebridge.org PDXPUG - http://pugs.postgresql.org/pdx Me - http://www.chesnok.com/daily
On Monday 15 December 2008 23:56:08 Jonah H. Harris wrote: > On Mon, Dec 15, 2008 at 3:26 PM, Josh Berkus <josh@agliodbs.com> wrote: > > You can't compare DBT2 with TPCC. They're not the same benchmark. > > Agreed. Somewhere earlier in the thread I mentioned that the OSDL > kits are non-spec-compliant, a little buggy, and sometimes give better > results than the real tests. Could we consolidate everyone's concerns into a project outline, if one is necessary? - Everyone would like to have a TPC-like benchmark kit. - OSDL script are not compliant. - OSDL scripts are outdated/appear unmaintained. - OSDL scripts need POstgreSQL-specific improvements. - EnterpriseDB is apparently maintaining their own fork. What can we do? I found the TPC-C spec on the web, so it's freely available for implementation, it appears.
Peter, > I found the TPC-C spec on the web, so it's freely available for > implementation, it appears. For some definition of "free". It's not open source, and you can't use it however you want. For example, you can't use it to publish a benchmark, and as far as the TPC is concerned, putting it on a wiki is "publishing". This is why I was focussing on SpecJAppserver instead. It has a derivitative wersion (EAStress), which does not have such restrictions. --Josh
On Tuesday 16 December 2008 15:58:15 Peter Eisentraut wrote: > On Monday 15 December 2008 23:56:08 Jonah H. Harris wrote: > > On Mon, Dec 15, 2008 at 3:26 PM, Josh Berkus <josh@agliodbs.com> wrote: > > > You can't compare DBT2 with TPCC. They're not the same benchmark. > > > > Agreed. Somewhere earlier in the thread I mentioned that the OSDL > > kits are non-spec-compliant, a little buggy, and sometimes give better > > results than the real tests. > > Could we consolidate everyone's concerns into a project outline, if one is > necessary? > > - Everyone would like to have a TPC-like benchmark kit. > - OSDL script are not compliant. > - OSDL scripts are outdated/appear unmaintained. > - OSDL scripts need POstgreSQL-specific improvements. > - EnterpriseDB is apparently maintaining their own fork. > Don't forget Jan's TPC-W benchmark, available at http://pgfoundry.org/projects/tpc-w-php/. Not sure if it still works, but Jan claims it did at one point. -- Robert Treat Conjecture: http://www.xzilla.net Consulting: http://www.omniti.com
"Selena Deckelmann" <selenamarie@gmail.com> writes: > http://sourceforge.net/projects/osdldbt/ > > Mark has been maintaining the DBT2 kit for a while now - since he was at OSDL. > > We've been presenting results -- first some IO tests at Linux Plumbers > Conf, and the very beginnings of postgresql tuning results at Pg West. > That's Mark, Gabrielle Roth and me. That SVN repository hasn't been touched in almost two years. It doesn't even work with the current TPC-C datagen without editing the data files. What drives me nuts trying to get these kits running is how they have so many scripts running other scripts and they all depend on having an environment set up with specific users, paths, and environment variables set. Ideally it would just be a simple binary like pgbench which takes command-line options to tell it what to do and can be run without setting up any special environment. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's Slony Replication support!
On Tue, Dec 16, 2008 at 3:38 PM, Gregory Stark <stark@enterprisedb.com> wrote: > > What drives me nuts trying to get these kits running is how they have so many > scripts running other scripts and they all depend on having an environment set > up with specific users, paths, and environment variables set. > > Ideally it would just be a simple binary like pgbench which takes command-line > options to tell it what to do and can be run without setting up any special > environment. Fair enough. We'd love some help. Or maybe patches from the fork of Mark's code. :) Currently, we're focusing on updating the reporting end. -selena -- Selena Deckelmann Open Source Bridge - http://www.opensourcebridge.org PDXPUG - http://pugs.postgresql.org/pdx Me - http://www.chesnok.com/daily
On Tue, Dec 16, 2008 at 12:14 PM, Gregory Stark <stark@enterprisedb.com> wrote: > "Mark Wong" <markwkm@gmail.com> writes: > >> On Mon, Dec 15, 2008 at 6:17 AM, Jonah H. Harris <jonah.harris@gmail.com> wrote: >>> On Mon, Dec 15, 2008 at 3:25 AM, Peter Eisentraut <peter_e@gmx.net> wrote: >>>> Well, then the first order of business would be to write a benchmark kit. I >>>> have been thinking for a while that we should make our own maintained >>>> version of the DBT+* suite, or whatever other suite is appropriate. And >>>> then start running it. >>> >>> Agreed. For a start on TPC-C, EnterpriseDB has a closer version of the kit. >> >> I don't recall turning away patches to make the dbt kits less TPC spec >> compliant... > > Uhm, why would we send them to you? Are you the maintainer for DBT2 now? I > looked recently and couldn't find any upstream source that had been touched > any time within the last few years. Where do i find it? Honestly I would have expected to see them sent to the osdldbt-general mailing list, not to me personally. But I guess that was never advertised in the kit. I don't think I stopped maintaining it, but some people have been sending patches to the list that apply to releases older than what is in the svn repository. That makes it a little hard for me to get it right, especially when they are for databases I don't use. And I admit fault at never announcing that I've move the source code from svn to mercurial to git, which is not on git.postgresql.org. Regards, Mark
On Tue, Dec 16, 2008 at 3:38 PM, Gregory Stark <stark@enterprisedb.com> wrote: > "Selena Deckelmann" <selenamarie@gmail.com> writes: > >> http://sourceforge.net/projects/osdldbt/ >> >> Mark has been maintaining the DBT2 kit for a while now - since he was at OSDL. >> >> We've been presenting results -- first some IO tests at Linux Plumbers >> Conf, and the very beginnings of postgresql tuning results at Pg West. >> That's Mark, Gabrielle Roth and me. > > That SVN repository hasn't been touched in almost two years. It doesn't even > work with the current TPC-C datagen without editing the data files. Please explain further. TPC doesn't have a tpc-c datagen program like they do for the H, W, and E. Otherwise the code in the git repository appears to be working with 8.3.5. > What drives me nuts trying to get these kits running is how they have so many > scripts running other scripts and they all depend on having an environment set > up with specific users, paths, and environment variables set. I agree. I've refactored them a little bit. But it's not that much better. > Ideally it would just be a simple binary like pgbench which takes command-line > options to tell it what to do and can be run without setting up any special > environment. Anything after a the The C, E, H, and newer are not likely to be that simple. The results will be of limited used without physically configuring the disks for example, IMHO. Regards, Mark
On Wed, Dec 17, 2008 at 1:26 AM, Mark Wong <markwkm@gmail.com> wrote: > >> That SVN repository hasn't been touched in almost two years. It doesn't even >> work with the current TPC-C datagen without editing the data files. > > Please explain further. TPC doesn't have a tpc-c datagen program like > they do for the H, W, and E. Sorry, the discrepancy with datagen that I remembered was with DBT3. So uh, where is the current source repository? -- greg
On Tue, Dec 16, 2008 at 5:38 PM, Greg Stark <stark@enterprisedb.com> wrote: > On Wed, Dec 17, 2008 at 1:26 AM, Mark Wong <markwkm@gmail.com> wrote: >> >>> That SVN repository hasn't been touched in almost two years. It doesn't even >>> work with the current TPC-C datagen without editing the data files. >> >> Please explain further. TPC doesn't have a tpc-c datagen program like >> they do for the H, W, and E. > > Sorry, the discrepancy with datagen that I remembered was with DBT3. dbgen always needed editing. The changes are supplied with the kit. > So uh, where is the current source repository? On git.postgresql.org. http://git.postgresql.org/?p=~markwkm/dbt2.git;a=summary And since you bring up dbt3: http://git.postgresql.org/?p=~markwkm/dbt3.git;a=summary Regards, Mark
On Tuesday 16 December 2008 20:19:28 Mark Wong wrote: > And I admit fault at never announcing that > I've move the source code from svn to mercurial to git, which is not > on git.postgresql.org. > that should read "now on", not "not on", right ? -- Robert Treat Conjecture: http://www.xzilla.net Consulting: http://www.omniti.com
On Dec 16, 2008, at 9:10 PM, Robert Treat wrote: > On Tuesday 16 December 2008 20:19:28 Mark Wong wrote: >> And I admit fault at never announcing that >> I've move the source code from svn to mercurial to git, which is not >> on git.postgresql.org. >> > > that should read "now on", not "not on", right ? Yeah, my bad on spelling. Regards, Mark