Thread: Re: Hardware/OS recommendations for large databases (
Luke Lonergan wrote: > Alan, > > On 11/18/05 8:13 AM, "Alan Stange" <stange@rentec.com> wrote: > > I told you in my initial post that I was observing numbers in > excess of > what you claiming, but you seemed to think I didn't know how to > measure > an IO rate. > > Prove me wrong, post your data. > > I should note too that our system uses about 20% of a single cpu when > performing a table scan at >100MB/s of IO. I think you claimed the > system would be cpu bound at this low IO rate. > > > See above. Here's the output from one iteration of iostat -k 60 while the box is doing a select count(1) on a 238GB table. avg-cpu: %user %nice %sys %iowait %idle 0.99 0.00 17.97 32.40 48.64 Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn sdd 345.95 130732.53 0.00 7843952 0 We're reading 130MB/s for a full minute. About 20% of a single cpu was being used. The remainder being idle. We've done nothing fancy and achieved results you claim shouldn't be possible. This is a system that was re-installed yesterday, no tuning was done to the file systems, kernel or storage array. What am I doing wrong? 9 years ago I co-designed a petabyte data store with a goal of 1GB/s IO (for a DOE lab). And now I don't know what I'm doing, Cheers, -- Alan
Alan, On 11/18/05 9:31 AM, "Alan Stange" <stange@rentec.com> wrote: > Here's the output from one iteration of iostat -k 60 while the box is > doing a select count(1) on a 238GB table. > > avg-cpu: %user %nice %sys %iowait %idle > 0.99 0.00 17.97 32.40 48.64 > > Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn > sdd 345.95 130732.53 0.00 7843952 0 > > We're reading 130MB/s for a full minute. About 20% of a single cpu was > being used. The remainder being idle. Cool - thanks for the results. Is that % of one CPU, or of 2? Was the system otherwise idle? > We've done nothing fancy and achieved results you claim shouldn't be > possible. This is a system that was re-installed yesterday, no tuning > was done to the file systems, kernel or storage array. Are you happy with 130MB/s? How much did you pay for that? Is it more than $2,000, or double my 2003 PC? > What am I doing wrong? > > 9 years ago I co-designed a petabyte data store with a goal of 1GB/s IO > (for a DOE lab). And now I don't know what I'm doing, Cool. Would that be Sandia? We routinely sustain 2,000 MB/s from disk on 16x 2003 era machines on complex queries. - Luke