Thread: Some Performance Advice Needed
Hi, i recently run pgbench against different servers and got some results I dont quite understand. A) EV1: Dual Xenon, 2GHz, 1GB Memory, SCSI 10Krpm, RHE3 B) Dual Pentium3 1.4ghz (Blade), SCSI Disk 10Krmp, 1GB Memory, Redhat 8 C) P4 3.2GHz, IDE 7.2Krpm, 1GBMem, Fedora Core2 All did run only postgres 7.4.6 pgconf settings: max_connections = 100 shared_buffers = 8192 sort_mem = 8192 vacuum_mem = 32768 max_fsm_pages = 200000 max_fsm_relations = 10000 wal_sync_method = fsync wal_buffers = 64 checkpoint_segments = 10 effective_cache_size = 65536 random_page_cost = 1.4 /etc/sysctl.conf shmall and shmmax set to 768mb Runnig PGbench reported A) 220 tps B) 240 tps C) 510 tps Running hdparm reported A) 920mb/s (SCSI 10k) B) 270mb/s (SCSI 10k) C) 1750mb/s (IDE 7.2k) What I dont quite understand is why a P3.2 is twice as fast as a Dual Xenon with SCSI disks, A dual Xenon 2GHz is not faster than a dual P3 1.4Ghz, and the hdparm results also dont make much sense. Has anybody an explanation for that? Is there something I can do to get more performance out of the SCSI disks? Thanks for any advise Alex
On Dec 23, 2004, at 9:27 AM, Alex wrote: > Running hdparm reported > A) 920mb/s (SCSI 10k) > B) 270mb/s (SCSI 10k) > C) 1750mb/s (IDE 7.2k) IDE disks lie about write completion (This can be disabled on some drives) whereas SCSI drives wait for the data to actually be written before they report success. It is quite easy to corrupt a PG (Or most any db really) on an IDE drive. Check the archives for more info. -- Jeff Trout <jeff@jefftrout.com> http://www.jefftrout.com/ http://www.stuarthamm.net/
Alex wrote: > Hi, > i recently run pgbench against different servers and got some results I > dont quite understand. > > A) EV1: Dual Xenon, 2GHz, 1GB Memory, SCSI 10Krpm, RHE3 > B) Dual Pentium3 1.4ghz (Blade), SCSI Disk 10Krmp, 1GB Memory, Redhat 8 > C) P4 3.2GHz, IDE 7.2Krpm, 1GBMem, Fedora Core2 > > Runnig PGbench reported > A) 220 tps > B) 240 tps > C) 510 tps > > Running hdparm reported > A) 920mb/s (SCSI 10k) > B) 270mb/s (SCSI 10k) > C) 1750mb/s (IDE 7.2k) > > What I dont quite understand is why a P3.2 is twice as fast as a Dual > Xenon with SCSI disks, A dual Xenon 2GHz is not faster than a dual P3 > 1.4Ghz, and the hdparm results also dont make much sense. A few things to clear up about the P3/P4/Xeons. Xeons are P4s. Hence, a P4 2ghz will run the same speed as a Xeon 2ghz assuming all other variables are the same. Of course they aren't because your P4 is probably running unregistered memory, uses either a 533mhz or 800mhz FSB compared to the Xeon's shared 400mhz amongs 2 CPUs, running a faster non-smp kernel. Add all those variables up and it's definitely possible for a P4 3.2ghz to run twice as fast as a Dual Xeon 2ghz on a single-thread benchmark. (The corollary here is that in a multi-thread benchmark, the 2X Xeon can only hope to equal your P4 3.2.) P3s are faster than P4s at the same clock rate. By a lot. It's not really that surprising that a P3 1.4 is faster than a P4/Xeon 2.0. I've seen results like this many times over a wide range of applications. The only variable that is throwing off your comparisons are the hard drives. IDE drives have write caching on by default -- SCSI drives have it off. Use: hdparm -W0 /dev/hda to turn it off on the P4 system and rerun the tests then.
Jeff wrote: > > On Dec 23, 2004, at 9:27 AM, Alex wrote: > > >> Running hdparm reported >> A) 920mb/s (SCSI 10k) >> B) 270mb/s (SCSI 10k) >> C) 1750mb/s (IDE 7.2k) > > > > IDE disks lie about write completion (This can be disabled on some > drives) whereas SCSI drives wait for the data to actually be written > before they report success. It is quite > easy to corrupt a PG (Or most any db really) on an IDE drive. Check > the archives for more info. Do we have any real info on this? Specifically which drives? Is SATA the same way? What about SATA-II? I am not saying it isn't true (I know it is) but this is a blanket statement that may or may not be true with newer tech. Sincerely, Joshua D. Drake > > -- > Jeff Trout <jeff@jefftrout.com> > http://www.jefftrout.com/ > http://www.stuarthamm.net/ > > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL
Attachment
>> IDE disks lie about write completion (This can be disabled on some >> drives) whereas SCSI drives wait for the data to actually be written >> before they report success. It is quite >> easy to corrupt a PG (Or most any db really) on an IDE drive. Check >> the archives for more info. > > > Do we have any real info on this? Specifically which drives? Is SATA the > same way? What about SATA-II? > I am not saying it isn't true (I know it is) but this is a blanket > statement that may or may not be > true with newer tech. From my experience with SATA controllers, write caching is controlled via the BIOS.
On Dec 23, 2004, at 4:27 PM, Joshua D. Drake wrote: >> >> IDE disks lie about write completion (This can be disabled on some >> drives) whereas SCSI drives wait for the data to actually be written >> before they report success. It is quite >> easy to corrupt a PG (Or most any db really) on an IDE drive. Check >> the archives for more info. > > Do we have any real info on this? Specifically which drives? Is SATA > the same way? What about SATA-II? > I am not saying it isn't true (I know it is) but this is a blanket > statement that may or may not be > true with newer tech. Scott Marlowe did some tests a while ago on it. They are likely in the archives. Maybe we can get him to pipe up :) -- Jeff Trout <jeff@jefftrout.com> http://www.jefftrout.com/ http://www.stuarthamm.net/
"jd@commandprompt.com ("Joshua D. Drake")" wrote in pgsql.performance:
> Jeff wrote:
>
>>
>> On Dec 23, 2004, at 9:27 AM, Alex wrote:
>>
>>
>>> Running hdparm reported
>>> A) 920mb/s (SCSI 10k)
>>> B) 270mb/s (SCSI 10k)
>>> C) 1750mb/s (IDE 7.2k)
>>
>>
>>
>> IDE disks lie about write completion (This can be disabled on some
>> drives) whereas SCSI drives wait for the data to actually be written
>> before they report success. It is quite
>> easy to corrupt a PG (Or most any db really) on an IDE drive. Check
>> the archives for more info.
>
> Do we have any real info on this? Specifically which drives? Is SATA the
> same way? What about SATA-II?
> I am not saying it isn't true (I know it is) but this is a blanket
> statement that may or may not be
> true with newer tech.
The name hasn't changed, but don't let that give you the wrong
impression because SCSI continues to improve. I only use SCSI drives in
all my servers, and that's because they always seem to outperform SATA and
IDE when there's a multi-user[1] requirement (of course, the choice of OS
[2] is an important factor here too).
Disk fragmentation also plays a role, but can actually become a
hinderance when in a multi-user environment. I find that the caching
algorithm in the OS that I usually choose [2] actually performs extremely
well when more users are accessing data on volumes where the data is
fragmented. I'm told that this is very similar in the Unix environment as
well. Defragmentation makes more sense in a single-user environment
because there are generally a very small number of files being loaded at
one time, and so a user can benefit hugely from defragmentation.
Here's an interesting article (it comes complete with anonymous non-
logical emotion-based reader comments too):
SCSI vs. IDE: Which is really faster?
http://hardware.devchannel.org/hardwarechannel/03/10/20/1953249.shtml?
tid=20&tid=38&tid=49
[1] A somewhat busy web and/or eMail server certainly counts as a multi-
user requirement. Put a database on it where the data isn't being accessed
sequentially, and that can certainly meet the requirements too.
[2] Nearly all my servers run Novell NetWare.
--
Randolf Richardson, pro-active spam fighter - rr@8x.ca
Vancouver, British Columbia, Canada
Sending eMail to other SMTP servers is a privilege.