Re: extremly low memory usage - Mailing list pgsql-performance

On Sun, 2005-08-21 at 16:13 -0400, Ron wrote:
> At 10:54 AM 8/21/2005, Jeremiah Jahn wrote:
> >On Sat, 2005-08-20 at 21:32 -0500, John A Meinel wrote:
> > > Ron wrote:
> > >
> > > Well, since you can get a read of the RAID at 150MB/s, that means that
> > > it is actual I/O speed. It may not be cached in RAM. Perhaps you could
> > > try the same test, only using say 1G, which should be cached.
> >
> >[root@io pgsql]# time dd if=/dev/zero of=testfile bs=1024 count=1000000
> >1000000+0 records in
> >1000000+0 records out
> >
> >real    0m8.885s
> >user    0m0.299s
> >sys     0m6.998s
>
> This is abysmally slow.
>
>
> >[root@io pgsql]# time dd of=/dev/null if=testfile bs=1024 count=1000000
> >1000000+0 records in
> >1000000+0 records out
> >
> >real    0m1.654s
> >user    0m0.232s
> >sys     0m1.415s
>
> This transfer rate is the only one out of the 4 you have posted that
> is in the vicinity of where it should be.
>
>
> >The raid array I have is currently set up to use a single channel. But I
> >have dual controllers in the array. And dual external slots on the card.
> >The machine is brand new and has pci-e backplane.
> >
> So you have 2 controllers each with 2 external slots?  But you are
> currently only using 1 controller and only one external slot on that
> controller?

Sorry, no. I have one dual channel card in the system and two
controllers on the array. Dell PowerVault 220S w/ PERC4eDC-PCI Express

>
>
> > > > Assuming these are U320 15Krpm 147GB HDs, a RAID 10 array of 14 of them
> > > > doing raw sequential IO like this should be capable of at
> > > >  ~7*75MB/s= 525MB/s using Seagate Cheetah 15K.4's
> >BTW I'm using Seagate Cheetah 15K.4's
>
> OK, now we have that nailed down.
>
>
> > > > AFAICT, the Dell PERC4 controllers use various flavors of the LSI Logic
> > > > MegaRAID controllers.  What I don't know is which exact one yours is,
> > > > nor do I know if it (or any of the MegaRAID controllers) are high
> > > > powered enough.
> >
> >PERC4eDC-PCI Express, 128MB Cache, 2-External Channels
>
> Looks like they are using the LSI Logic MegaRAID SCSI 320-2E
> controller.  IIUC, you have 2 of these, each with 2 external channels?
>
> The specs on these appear a bit strange.  They are listed as being a
> PCI-Ex8 card, which means they should have a max bandwidth of 20Gb/s=
> 2GB/s, yet they are also listed as only supporting dual channel U320=
> 640MB/s when they could easily support quad channel U320=
> 1.28GB/s.  Why bother building a PCI-Ex8 card when only a PCI-Ex4
> card (which is a more standard physical format) would've been
> enough?  Or if you are going to build a PCI-Ex8 card, why not support
> quad channel U320?  This smells like there's a problem with LSI's design.
>
> The 128MB buffer also looks suspiciously small, and I do not see any
> upgrade path for it on LSI Logic's site.  "Serious" RAID controllers
> from companies like Xyratex, Engino, and Dot-hill can have up to
> 1-2GB of buffer, and there's sound technical reasons for it.  See if
> there's a buffer upgrade available or if you can get controllers that
> have larger buffer capabilities.
>
> Regardless of the above, each of these controllers should still be
> good for about 80-85% of 640MB/s, or ~510-540 MB/s apiece when doing
> raw sequential IO if you plug 3-4 fast enough HD's into each SCSI
> channel.  Cheetah 15K.4's certainly are fast enough.  Optimal setup
> is probably to split each RAID 1 pair so that one HD is on each of
> the SCSI channels, and then RAID 0 those pairs.  That will also
> protect you from losing the entire disk subsystem if one of the SCSI
> channels dies.
I like this idea, but how exactly does one bond the two channels
together? Won't this cause me to have both an /dev/sdb and an /dev/sdc?


>
> That 128MB of buffer cache may very well be too small to keep the IO
> rate up, and/or there may be a more subtle problem with the LSI card,
> and/or you may have a configuration problem, but _something(s)_ need
> fixing since you are only getting raw sequential IO of ~100-150MB/s
> when it should be above 500MB/s.

It looks like there's a way to add more memory to it.

>
> This will make the most difference for initial reads (first time you
> load a table, first time you make a given query, etc) and for any writes.
>
> Your HW provider should be able to help you, even if some of the HW
> in question needs to be changed.  You paid for a solution.  As long
> as this stuff is performing at so much less then what it is supposed
> to, you have not received the solution you paid for.
>
> BTW, on the subject of RAID stripes IME the sweet spot tends to be in
> the 64KB to 256KB range (very large, very read heavy data mines can
> want larger RAID stripes.).  Only experimentation will tell you what
> results in the best performance for your application.
I think I have them very small at the moment.

>
>
> >I'm not really worried about the writing, it's the reading the reading
> >that needs to be faster.
>
> Initial reads are only going to be as fast as your HD subsystem, so
> there's a reason for making the HD subsystem faster even if all you
> care about is reads.  In addition, I'll repeat my previous advice
> that upgrading to 16GB of RAM would be well worth it for you.

12GB is my max. I may run with it for a while and see.

>
> Hope this helps,
> Ron Peacetree
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
>        choose an index scan if your joining column's datatypes do not
>        match
--
Speak softly and carry a +6 two-handed sword.