Thread: Dual core Opterons beating quad core Xeons?
Hi. We are looking at upgrading our primary servers. The final boxes will have 128GB ram, fast disk arrays and 4 CPUs. We currently have some eval units with 8GB ram and crappy disk to let us benchmark CPU choice. One box has 4 3GHz dual core Opterons with 1MB cache, the other box ha 4 3GHz quad core Xeons with 4MB cache. model name : Intel(R) Xeon(R) CPU X7350 @ 2.93GHz cache size : 4096 KB model name : Dual-Core AMD Opteron(tm) Processor 8222 SE cache size : 1024 KB I haven't had a chance to play with the hardware myself yet. The sysadmins have been running some benchmarks themselves though. For every non PG related benchmark they have run, the Xeon wins by around 20%. For pgbench (PG 8.2 running Ubuntu), the Opteron is getting about 6x TPS over the Xeon (3000+ TPS on Opteron vs ~500 on Xeon). Things get a little better for Xeon with PG 8.3 (570-540 TPS). Does this match what other people are seeing or expect, or have we screwed our benchmarks somehow? Is this a PG specific win for Opteron, or will we see similar results with other DBs? Do people see wins for non-PG databases on Xeon, and are they as dramatic as we are seeing for PG on Opteron? With PG 8.2 and 8.3, is it still pretty much limited to 8 cores making 2 of the quad core Xeons redundant or detrimental? I expect we will be running this hardware for 8.2, 8.3 and 8.4. Anyone aware of anything that might change the landscape for 8.4? -- Stuart Bishop <stuart@stuartbishop.net> http://www.stuartbishop.net/
Attachment
Stuart Bishop a écrit : > Hi. > > We are looking at upgrading our primary servers. The final boxes will have > 128GB ram, fast disk arrays and 4 CPUs. > > We currently have some eval units with 8GB ram and crappy disk to let us > benchmark CPU choice. One box has 4 3GHz dual core Opterons with 1MB cache, > the other box ha 4 3GHz quad core Xeons with 4MB cache. > > model name : Intel(R) Xeon(R) CPU X7350 @ 2.93GHz > cache size : 4096 KB > model name : Dual-Core AMD Opteron(tm) Processor 8222 SE > cache size : 1024 KB > > I haven't had a chance to play with the hardware myself yet. The sysadmins > have been running some benchmarks themselves though. > > For every non PG related benchmark they have run, the Xeon wins by around 20%. > > For pgbench (PG 8.2 running Ubuntu), the Opteron is getting about 6x TPS > over the Xeon (3000+ TPS on Opteron vs ~500 on Xeon). Things get a little > better for Xeon with PG 8.3 (570-540 TPS). > > Does this match what other people are seeing or expect, or have we screwed > our benchmarks somehow? > http://tweakers.net/reviews/661/7 as an example You can travel the website for other benchs... (there are about dual and quad core) > Is this a PG specific win for Opteron, or will we see similar results with > other DBs? > > Do people see wins for non-PG databases on Xeon, and are they as dramatic as > we are seeing for PG on Opteron? > > With PG 8.2 and 8.3, is it still pretty much limited to 8 cores making 2 of > the quad core Xeons redundant or detrimental? > > I expect we will be running this hardware for 8.2, 8.3 and 8.4. Anyone aware > of anything that might change the landscape for 8.4? > > -- Cédric Villemain Administrateur de Base de Données Cel: +33 (0)6 74 15 56 53 http://dalibo.com - http://dalibo.org
Attachment
"Stuart Bishop" <stuart@stuartbishop.net> writes: > For pgbench (PG 8.2 running Ubuntu), the Opteron is getting about 6x TPS > over the Xeon (3000+ TPS on Opteron vs ~500 on Xeon). Things get a little > better for Xeon with PG 8.3 (570-540 TPS). There was a problem in the past which affected Xeons. But I thought it had been mostly addressed. Xeons are (or were? these things are always changing) more sensitive to interprocess contention due to their memory architecture though. What are you actually doing in these transactions? Are they read-only? If not is fsync=off (which you don't want if you care about your data but you do if you're trying to benchmark the cpu). Are the crappy disks *identical* crappy disks? If they have different controllers or different drives (or different versions of the OS) then you might be being deceived by write caching on one set and not the other. If they're not read-only transactions and fsync=on then the TPS of 3000+ is not credible and this is likely. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's RemoteDBA services!
On Dec 19, 2007 6:04 AM, Stuart Bishop <stuart@stuartbishop.net> wrote: > Hi. > > We are looking at upgrading our primary servers. The final boxes will have > 128GB ram, fast disk arrays and 4 CPUs. > > We currently have some eval units with 8GB ram and crappy disk to let us > benchmark CPU choice. One box has 4 3GHz dual core Opterons with 1MB cache, > the other box ha 4 3GHz quad core Xeons with 4MB cache. Imagine two scenarios. In one you have an infinite number of hard drives with an infinite amount of battery backed cache, and an infinite I/O bandwidth. In the other you have one disk. Which one is likely to be I/O bound? Yep. So, it's not likely you'll be able to do a realistic benchmark of the CPUs with such a limited disk subsystem... > For pgbench (PG 8.2 running Ubuntu), the Opteron is getting about 6x TPS > over the Xeon (3000+ TPS on Opteron vs ~500 on Xeon). Things get a little > better for Xeon with PG 8.3 (570-540 TPS). pgbench is a mostly I/O bound benchmark. What are your -c, -t and -s settings btw? It's would be much better if you could benchmark something like the real load you'll be running in the future. Are you looking at reporting, transactions, content management, etc...?
On Wed, 19 Dec 2007, Stuart Bishop wrote: > For pgbench (PG 8.2 running Ubuntu), the Opteron is getting about 6x TPS > over the Xeon (3000+ TPS on Opteron vs ~500 on Xeon). Things get a little > better for Xeon with PG 8.3 (570-540 TPS). The 3000+ TPS figure is the correct one for a controller that can cache writes. Around 500TPS is normal for a setup without one. I suspect all you're testing is the difference between the I/O subsystem in the two serves, and it's probaby the case that the Opteron disk subsystem caches writes while the Xeon doesn't. You haven't drawn any useful conclusions comparing Xeons and Opterons yet. Warning: the system with the write caching can easily be doing that incorrectly, in a way that can corrupt your database one day. See http://momjian.us/main/writings/pgsql/sgml/wal-reliability.html for an intro and http://www.westnet.com/~gsmith/content/postgresql/TuningPGWAL.htm for way more detail. If you don't have a real disk setup, you can't use the default pgbench test and expect the results to be useful. The main thing it does is write heavily in a way that makes the disk controller and associated I/O the bottleneck in most cases. The only useful test you can do right now with pgbench is to pass it the -S parameter so that it does only reads instead. That will give you a much better idea how the CPUs compare. You still need to be careful about the database scale relative to the amount of RAM; at some point even the read test will be limited by disk parameters instead of CPU. Take a look at http://www.westnet.com/~gsmith/content/postgresql/pgbench-scaling.htm for a tutorial on using pgbench to quantify read performance. Note that the way I compute the sizes of things in there is a little difficult, one day I'm going to use some of the suggestions at http://andreas.scherbaum.la/blog/archives/282-table-size,-database-size.html to improve that and you should take a look there as well. You'll also need to vary the number of clients a bit. You should see the largest difference between the two servers with around 16 of them (where the Xeon system has a dedicated core for each while the Opteron has 2 clients/core) while a useful spot to compare the maximum throughput of the servers will be around 64 clients. > With PG 8.2 and 8.3, is it still pretty much limited to 8 cores making 2 of > the quad core Xeons redundant or detrimental? Where'd you get the idea 8 cores was a limit? As cores go up eventually you run out of disk or memory bandwidth, but how that plays out is very application dependant and there's no hard line anywhere. > I expect we will be running this hardware for 8.2, 8.3 and 8.4. Anyone aware > of anything that might change the landscape for 8.4? 8.4 is only in the earliest of planning stages right now, nobody knows what that will bring yet. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 19 Dec 2007 13:50:29 -0500 (EST) Greg Smith <gsmith@gregsmith.com> wrote: > > With PG 8.2 and 8.3, is it still pretty much limited to 8 cores > > making 2 of the quad core Xeons redundant or detrimental? > > Where'd you get the idea 8 cores was a limit? As cores go up > eventually you run out of disk or memory bandwidth, but how that > plays out is very application dependant and there's no hard line > anywhere. Actually this is not true. Although I have yet to test 8.3. It is pretty much common knowledge that after 8 cores the acceleration of performance drops with PostgreSQL... This has gotten better every release. 8.1 for example handles 8 cores very well, 8.0 didn't and 7.4 well.... :) Sincerely, Joshua D. Drake - -- The PostgreSQL Company: Since 1997, http://www.commandprompt.com/ Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240 Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate SELECT 'Training', 'Consulting' FROM vendor WHERE name = 'CMD' -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHaWoJATb/zqfZUUQRAgMeAJ9RS7BLAowXpJTbXuufJhIATj9gaACgrH6x LRVDPbyIvn71ANra2yiXmgY= =8QVl -----END PGP SIGNATURE-----
On Dec 19, 2007 12:59 PM, Joshua D. Drake <jd@commandprompt.com> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Wed, 19 Dec 2007 13:50:29 -0500 (EST) > Greg Smith <gsmith@gregsmith.com> wrote: > > > > With PG 8.2 and 8.3, is it still pretty much limited to 8 cores > > > making 2 of the quad core Xeons redundant or detrimental? > > > > Where'd you get the idea 8 cores was a limit? As cores go up > > eventually you run out of disk or memory bandwidth, but how that > > plays out is very application dependant and there's no hard line > > anywhere. > > Actually this is not true. Although I have yet to test 8.3. It is > pretty much common knowledge that after 8 cores the acceleration of > performance drops with PostgreSQL... I thought Tom had played with some simple hacks that got the scaling pretty close to linear for up to 16 cores earlier this year...
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 19 Dec 2007 13:03:32 -0600 "Scott Marlowe" <scott.marlowe@gmail.com> wrote: > > Actually this is not true. Although I have yet to test 8.3. It is > > pretty much common knowledge that after 8 cores the acceleration of > > performance drops with PostgreSQL... > > I thought Tom had played with some simple hacks that got the scaling > pretty close to linear for up to 16 cores earlier this year... > See.. have not tested 8.3 above and 8.2 is better than 8.1 etc... Sincerely, Joshua D. Drake - -- The PostgreSQL Company: Since 1997, http://www.commandprompt.com/ Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240 Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate SELECT 'Training', 'Consulting' FROM vendor WHERE name = 'CMD' -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHaWwCATb/zqfZUUQRApqZAJ92yx3LhMIF2nhI2LKrKAaxK2pqdgCffK9A 22rLNPRHHOaZAcvQTtLmRdA= =GHVK -----END PGP SIGNATURE-----
On Dec 19, 2007 1:07 PM, Joshua D. Drake <jd@commandprompt.com> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Wed, 19 Dec 2007 13:03:32 -0600 > "Scott Marlowe" <scott.marlowe@gmail.com> wrote: > > > > > Actually this is not true. Although I have yet to test 8.3. It is > > > pretty much common knowledge that after 8 cores the acceleration of > > > performance drops with PostgreSQL... > > > > I thought Tom had played with some simple hacks that got the scaling > > pretty close to linear for up to 16 cores earlier this year... > > > > See.. have not tested 8.3 above and 8.2 is better than 8.1 etc... Well, I'm not even sure if those got applied or were just Tom hacking in the basement or, heck, my fevered imagination. :)
Joshua D. Drake wrote: > Actually this is not true. Although I have yet to test 8.3. It is > pretty much common knowledge that after 8 cores the acceleration of > performance drops with PostgreSQL... > > This has gotten better every release. 8.1 for example handles 8 cores > very well, 8.0 didn't and 7.4 well.... :) I agree with the spirit of what you say, but are you overstating things a bit? Benchmarks I see[1] suggest that 8.1.2 scaled pretty reasonably to 16 cores (from the chart on page 9 in the link below). But yeah, 8.0 scaled to maybe 2 cores if you're lucky. :-) Agree with the rest of the things you say, tho. It's getting way better every recent release. [1] http://www.pgcon.org/2007/schedule/attachments/22-Scaling%20PostgreSQL%20on%20SMP%20Architectures%20--%20An%20Update
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 19 Dec 2007 11:14:08 -0800 Ron Mayer <rm_pg@cheapcomplexdevices.com> wrote: > Joshua D. Drake wrote: > > Actually this is not true. Although I have yet to test 8.3. It is > > pretty much common knowledge that after 8 cores the acceleration of > > performance drops with PostgreSQL... > > > > This has gotten better every release. 8.1 for example handles 8 > > cores very well, 8.0 didn't and 7.4 well.... :) > > I agree with the spirit of what you say, but are you overstating > things a bit? My point was :)... which that PDF actually illustrates is the gain between say 2 cores and 8 cores is greater than 8 and 16 and even less when you go beyond 16. > > Benchmarks I see[1] suggest that 8.1.2 scaled pretty reasonably to 16 > cores (from the chart on page 9 in the link below). But yeah, 8.0 > scaled to maybe 2 cores if you're lucky. :-) I really need to check this test out more though because their numbers don't reflect mine. I wonder if that is per connection. Sincerely, Joshua D. Drake - -- The PostgreSQL Company: Since 1997, http://www.commandprompt.com/ Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240 Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate SELECT 'Training', 'Consulting' FROM vendor WHERE name = 'CMD' -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD4DBQFHaXjxATb/zqfZUUQRAhlyAJijNIytenaBH2c5mEivFCT4qRmPAKCiW7Qn 2CDwNUBNd463Kz7G6n68yA== =bnaL -----END PGP SIGNATURE-----
On Wed, 19 Dec 2007, Ron Mayer wrote: > Benchmarks I see[1] suggest that 8.1.2 scaled pretty reasonably to 16 > cores (from the chart on page 9 in the link below). But yeah, 8.0 > scaled to maybe 2 cores if you're lucky. :-) > [1] http://www.pgcon.org/2007/schedule/attachments/22-Scaling%20PostgreSQL%20on%20SMP%20Architectures%20--%20An%20Update Thank you, I was looking for that one but couldn't find it again. Note that those results are using a TPC-C variant, which is not the most CPU intensive of tests out there. It's certainly possible that an application that has more processing to do per transaction (I'm thinking something more in the scientific computing database realm) could scale even better. While I'd expect the bang per buck to go down quite a bit beyond 8 cores, I know I haven't seen any data on what new systems running 8.3 are capable of, and extrapolating performance rules of thumb based on old data is perilous. Bottlenecks shift around in unexpected ways. In that Unisys example, they're running 32-bit single core Xeons circa 2004 with 4MB of *L3* cache and there's evidence that scales >16 processors. Current Xeons are considerably faster and you can get them with 4-8MB of *L2* cache. What does that do to scalability? Beats me. Maybe since the individual CPUs are faster, you bottleneck on something else way before you can use 16 of them usefully. Maybe the much better CPU cache means there's less reliance on the memory bus and they scale better. It depends a lot on the CPU vs. memory vs. disk requirements of your app, which is what I was suggesting before. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
"Joshua D. Drake" <jd@commandprompt.com> writes: > It is pretty much common knowledge that I think we have too much "common knowledge". -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's PostGIS support!
On Dec 19, 2007, at 4:54 PM, Gregory Stark wrote: > "Joshua D. Drake" <jd@commandprompt.com> writes: > >> It is pretty much common knowledge that > > I think we have too much "common knowledge". Yeah. For a lot of folks it's still common knowledge that you should only set shared_buffers to 10% of memory... -- Decibel!, aka Jim C. Nasby, Database Architect decibel@decibel.org Give your computer some brain candy! www.distributed.net Team #1828
Attachment
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 19 Dec 2007 19:51:13 -0600 Decibel! <decibel@decibel.org> wrote: > On Dec 19, 2007, at 4:54 PM, Gregory Stark wrote: > > "Joshua D. Drake" <jd@commandprompt.com> writes: > > > >> It is pretty much common knowledge that > > > > I think we have too much "common knowledge". > > > Yeah. For a lot of folks it's still common knowledge that you should > only set shared_buffers to 10% of memory... Sometimes that's true ;). Joshua D. Drake - -- The PostgreSQL Company: Since 1997, http://www.commandprompt.com/ Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240 Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate SELECT 'Training', 'Consulting' FROM vendor WHERE name = 'CMD' -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHae7UATb/zqfZUUQRAsKAAKCkDNtWarrHT4yDrVn7Bs3GGMRBNACfd2+B 8HDzjIF2OO4aS3AZ7+7muAs= =STaP -----END PGP SIGNATURE-----
"Scott Marlowe" <scott.marlowe@gmail.com> writes:
> Well, I'm not even sure if those got applied or were just Tom hacking
> in the basement or, heck, my fevered imagination. :)
For the record, I hack in the attic ... or what I tell the IRS is my
third-floor office ...
regards, tom lane
On 12/20/07, Tom Lane <tgl@sss.pgh.pa.us> wrote: > "Scott Marlowe" <scott.marlowe@gmail.com> writes: > > Well, I'm not even sure if those got applied or were just Tom hacking > > in the basement or, heck, my fevered imagination. :) > > For the record, I hack in the attic ... or what I tell the IRS is my > third-floor office ... > Awesome band name - 'Hacking in the Attic' :) jan