Re: extremly low memory usage - Mailing list pgsql-performance

Ron wrote:
>> PERC4eDC-PCI Express, 128MB Cache, 2-External Channels
>
> Looks like they are using the LSI Logic MegaRAID SCSI 320-2E
> controller.  IIUC, you have 2 of these, each with 2 external channels?

A lot of people have mentioned Dell's versions of the LSI cards can be
WAY slower than the ones you buy from LSI. Why this is the case? Nobody
knows for sure.

Here's a guess on my part. A while back, I was doing some googling. And
instead of typing "LSI MegaRAID xxx", I just typed "MegaRAID xxx". Going
beyond the initial pages, I saw Tekram -- a company that supposedly
produces their own controllers -- listing products with the exact model
numbers and photos as cards from LSI and Areca. Seemed puzzling until I
read a review about SATA RAID cards where it mentioned Tekram produces
the Areca cards under their own name but using slower components to
avoid competing at the highend with them.

So what may be happening is that the logic circuitry on the Dell PERCs
are the same as the source LSI cards, the speed of the RAID
processor/RAM/internal buffers/etc is not as fast so Dell can shave off
a few bucks for every server. That would mean while a true LSI card has
the processing power to do the RAID calculates for X drives, the Dell
version probably can only do X*0.6 drives or so.


> The 128MB buffer also looks suspiciously small, and I do not see any
> upgrade path for it on LSI Logic's site.  "Serious" RAID controllers
> from companies like Xyratex, Engino, and Dot-hill can have up to 1-2GB

The card is upgradable. If you look at the pic of the card, it shows a
SDRAM DIMM versus integrated RAM chips. I've also read reviews a while
back comparing benchmarks of the 320-2 w/ 128K versus 512K onboard RAM.
Their product literature is just nebulous on the RAM upgrade part. I'm
sure if you opened up the PDF manuals, you could find the exact info


> That 128MB of buffer cache may very well be too small to keep the IO
> rate up, and/or there may be a more subtle problem with the LSI card,
> and/or you may have a configuration problem, but _something(s)_ need
> fixing since you are only getting raw sequential IO of ~100-150MB/s when
> it should be above 500MB/s.

I think it just might be the Dell hardware or the lack of 64-bit IOMMU
on Xeon's. Here's my numbers on 320-1 w/ 128K paired up with Opterons
compared to Jeremiah's.

 >> # time dd if=/dev/zero of=testfile bs=1024 count=1000000
 >> 1000000+0 records in
 >> 1000000+0 records out
 >>
 >> real    0m8.885s
 >> user    0m0.299s
 >> sys     0m6.998s

2x15K RAID1
real    0m14.493s
user    0m0.255s
sys     0m11.712s

6x15K RAID10 (2x 320-1)
real    0m9.986s
user    0m0.200s
sys     0m8.634s


 >> # time dd of=/dev/null if=testfile bs=1024 count=1000000
 >> 1000000+0 records in
 >> 1000000+0 records out
 >>
 >> real    0m1.654s
 >> user    0m0.232s
 >> sys     0m1.415s

2x15K RAID1
real    0m3.383s
user    0m0.176s
sys     0m3.207s

6x15K RAID10 (2x 320-1)
real    0m2.427s
user    0m0.178s
sys     0m2.250s

If all 14 HDs are arranged in a RAID10 array, I'd say there's definitely
something wrong with Jeremiah's hardware.