Thread: Disk tests for a new database server

Disk tests for a new database server

From
Rory Campbell-Lange
Date:
We are testing disk I/O on our new server (referred to in my recent
questions about LVM and XFS on this list) and have run bonnie++ on the
xfs partition destined for postgres; results noted below.

I haven't been able to find many benchmarks showing desirable IO stats.
As far as I can tell the sequential input (around 110000K/sec) looks
good while the sequential output (around 50000K/sec) looks fairly
average.

Advice and comments gratefully received.
Suggested Parameters for running pg_bench would be great!

Thanks,
Rory

The server has a dual core AMD Opteron 270 chip (2000MHz), 6GB of RAM
and an LSI 320-1 card running 4x147GB disks running in a RAID10
configuration. The server has a freshly compiled 2.6.14.3 linux kernel.

partial df output:
    Filesystem            Size  Used Avail Use% Mounted on
    ...
    /dev/mapper/masvg-masdata
                           99G   33M   94G   1% /masdata
    /dev/mapper/masvg-postgres
                           40G   92M   40G   1% /postgres

partial fstab config:
    ...
    /dev/mapper/masvg-masdata /masdata        ext3    defaults        0       2
    /dev/mapper/masvg-postgres /postgres       xfs     defaults        0       2


Version  1.03      ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
mas          13000M 52655  99 52951  10 32246   7 41441  72 113438  12 590.0   0
mas          13000M 49306  83 51967   9 32269   7 42442  70 115427  12 590.5   1
mas          13000M 53449  89 51982  10 32625   7 42819  71 111829  11 638.3   0
mas          13000M 51818  88 51639   9 33127   7 42377  70 108585  11 556.5   0
mas          13000M 48930  90 51750   9 32854   7 41220  71 109813  11 566.2   0
mas          13000M 52148  88 47393   9 35343   7 42871  70 109775  12 582.0   0
mas          13000M 52427  88 53040  10 32315   7 42813  71 112045  12 596.7   0
mas          13000M 51967  87 54004  10 30429   7 46180  76 110973  11 625.1   0
mas          13000M 48690  89 46416   9 35678   7 41429  72 111612  11 627.2   0
mas          13000M 52641  88 52807  10 31115   7 43476  72 110694  11 568.2   0
mas          13000M 52186  88 47385   9 35341   7 42959  72 110963  11 558.7   0
mas          13000M 52092  87 53111  10 32135   7 42636  69 110560  11 562.0   1
mas          13000M 49445  90 47378   9 34410   7 41191  72 110736  11 610.3   0
mas          13000M 51704  88 47699   9 35436   7 42413  69 110446  11 612.0   0
mas          13000M 52434  88 53331  10 32479   7 43229  71 109385  11 620.6   0
mas          13000M 52074  89 53291  10 32095   7 43593  72 109541  11 628.0   0
mas          13000M 48084  88 52624  10 32301   7 40975  72 110548  11 594.0   0
mas          13000M 53019  90 52441  10 32411   7 42574  68 111321  11 578.0   0
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
mas              16  7970  36 +++++ +++  7534  34  7918  36 +++++ +++  4482  22
mas              16  7945  33 +++++ +++  7483  30  7918  42 +++++ +++  4438  20
mas              16  8481  48 +++++ +++  7468  31  7870  39 +++++ +++  4385  23
mas              16  7915  36 +++++ +++  7498  33  7930  41 +++++ +++  4619  23
mas              16  8553  35 +++++ +++  8312  38  8613  37 +++++ +++  4513  24
mas              16  8498  40 +++++ +++  8215  33  8570  43 +++++ +++  4858  26
mas              16  7892  39 +++++ +++  7624  30  5341  28 +++++ +++  4762  22
mas              16  5408  27 +++++ +++  9378  37  8573  41 +++++ +++  4385  21
mas              16  5063  27 +++++ +++  8656  38  5159  27 +++++ +++  4705  24
mas              16  4917  25 +++++ +++  8682  39  5282  28 +++++ +++  4723  22
mas              16  5027  28 +++++ +++  8538  36  5173  29 +++++ +++  4719  23
mas              16  5449  27 +++++ +++  8630  36  5266  28 +++++ +++  5463  27
mas              16  5373  27 +++++ +++  8658  37  5264  26 +++++ +++  4731  22
mas              16  4959  24 +++++ +++  9126  46  5160  26 +++++ +++  4717  24
mas              16  5379  27 +++++ +++  8620  40  5014  27 +++++ +++  4701  21
mas              16  5312  29 +++++ +++  8642  36  7862  42 +++++ +++  4869  24
mas              16  5057  26 +++++ +++  8566  36  5120  28 +++++ +++  4681  21
mas              16  5225  27 +++++ +++  8740  37  5205  28 +++++ +++  4744  21




--
Rory Campbell-Lange
<rory@campbell-lange.net>
<www.campbell-lange.net>

Re: Disk tests for a new database server

From
Steve Poe
Date:
Rory,

While I don't have my specific stats with my from my tests with XFS and
bonnie for our company's db server, I do recall vividly that seq. output
did not increase dramatically until I had 8+ discs in a RAID10
configuration on an LSI card. I was not using LVM. If I had less than 8
discs, seq. output was about equal regardless of file system being uses
(EXT3,JFS,or XFS).

Steve


On Thu, 2005-12-08 at 12:12 +0000, Rory Campbell-Lange wrote:
> We are testing disk I/O on our new server (referred to in my recent
> questions about LVM and XFS on this list) and have run bonnie++ on the
> xfs partition destined for postgres; results noted below.
>
> I haven't been able to find many benchmarks showing desirable IO stats.
> As far as I can tell the sequential input (around 110000K/sec) looks
> good while the sequential output (around 50000K/sec) looks fairly
> average.
>
> Advice and comments gratefully received.
> Suggested Parameters for running pg_bench would be great!
>
> Thanks,
> Rory
>
> The server has a dual core AMD Opteron 270 chip (2000MHz), 6GB of RAM
> and an LSI 320-1 card running 4x147GB disks running in a RAID10
> configuration. The server has a freshly compiled 2.6.14.3 linux kernel.
>
> partial df output:
>     Filesystem            Size  Used Avail Use% Mounted on
>     ...
>     /dev/mapper/masvg-masdata
>                            99G   33M   94G   1% /masdata
>     /dev/mapper/masvg-postgres
>                            40G   92M   40G   1% /postgres
>
> partial fstab config:
>     ...
>     /dev/mapper/masvg-masdata /masdata        ext3    defaults        0       2
>     /dev/mapper/masvg-postgres /postgres       xfs     defaults        0       2
>
>
> Version  1.03      ------Sequential Output------ --Sequential Input- --Random-
>                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
> Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
> mas          13000M 52655  99 52951  10 32246   7 41441  72 113438  12 590.0   0
> mas          13000M 49306  83 51967   9 32269   7 42442  70 115427  12 590.5   1
> mas          13000M 53449  89 51982  10 32625   7 42819  71 111829  11 638.3   0
> mas          13000M 51818  88 51639   9 33127   7 42377  70 108585  11 556.5   0
> mas          13000M 48930  90 51750   9 32854   7 41220  71 109813  11 566.2   0
> mas          13000M 52148  88 47393   9 35343   7 42871  70 109775  12 582.0   0
> mas          13000M 52427  88 53040  10 32315   7 42813  71 112045  12 596.7   0
> mas          13000M 51967  87 54004  10 30429   7 46180  76 110973  11 625.1   0
> mas          13000M 48690  89 46416   9 35678   7 41429  72 111612  11 627.2   0
> mas          13000M 52641  88 52807  10 31115   7 43476  72 110694  11 568.2   0
> mas          13000M 52186  88 47385   9 35341   7 42959  72 110963  11 558.7   0
> mas          13000M 52092  87 53111  10 32135   7 42636  69 110560  11 562.0   1
> mas          13000M 49445  90 47378   9 34410   7 41191  72 110736  11 610.3   0
> mas          13000M 51704  88 47699   9 35436   7 42413  69 110446  11 612.0   0
> mas          13000M 52434  88 53331  10 32479   7 43229  71 109385  11 620.6   0
> mas          13000M 52074  89 53291  10 32095   7 43593  72 109541  11 628.0   0
> mas          13000M 48084  88 52624  10 32301   7 40975  72 110548  11 594.0   0
> mas          13000M 53019  90 52441  10 32411   7 42574  68 111321  11 578.0   0
>                     ------Sequential Create------ --------Random Create--------
>                     -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
> files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
> mas              16  7970  36 +++++ +++  7534  34  7918  36 +++++ +++  4482  22
> mas              16  7945  33 +++++ +++  7483  30  7918  42 +++++ +++  4438  20
> mas              16  8481  48 +++++ +++  7468  31  7870  39 +++++ +++  4385  23
> mas              16  7915  36 +++++ +++  7498  33  7930  41 +++++ +++  4619  23
> mas              16  8553  35 +++++ +++  8312  38  8613  37 +++++ +++  4513  24
> mas              16  8498  40 +++++ +++  8215  33  8570  43 +++++ +++  4858  26
> mas              16  7892  39 +++++ +++  7624  30  5341  28 +++++ +++  4762  22
> mas              16  5408  27 +++++ +++  9378  37  8573  41 +++++ +++  4385  21
> mas              16  5063  27 +++++ +++  8656  38  5159  27 +++++ +++  4705  24
> mas              16  4917  25 +++++ +++  8682  39  5282  28 +++++ +++  4723  22
> mas              16  5027  28 +++++ +++  8538  36  5173  29 +++++ +++  4719  23
> mas              16  5449  27 +++++ +++  8630  36  5266  28 +++++ +++  5463  27
> mas              16  5373  27 +++++ +++  8658  37  5264  26 +++++ +++  4731  22
> mas              16  4959  24 +++++ +++  9126  46  5160  26 +++++ +++  4717  24
> mas              16  5379  27 +++++ +++  8620  40  5014  27 +++++ +++  4701  21
> mas              16  5312  29 +++++ +++  8642  36  7862  42 +++++ +++  4869  24
> mas              16  5057  26 +++++ +++  8566  36  5120  28 +++++ +++  4681  21
> mas              16  5225  27 +++++ +++  8740  37  5205  28 +++++ +++  4744  21
>
>
>
>


Re: Disk tests for a new database server

From
Rory Campbell-Lange
Date:
Hi Steve

On 08/12/05, Steve Poe (spoe@sfnet.cc) wrote:
> Rory,
>
> While I don't have my specific stats with my from my tests with XFS and
> bonnie for our company's db server, I do recall vividly that seq. output
> did not increase dramatically until I had 8+ discs in a RAID10
> configuration on an LSI card. I was not using LVM. If I had less than 8
> discs, seq. output was about equal regardless of file system being uses
> (EXT3,JFS,or XFS).

Thanks for the information. I certainly had not appreciated this fact.

Regards,
Rory

> On Thu, 2005-12-08 at 12:12 +0000, Rory Campbell-Lange wrote:
> > We are testing disk I/O on our new server (referred to in my recent
> > questions about LVM and XFS on this list) and have run bonnie++ on the
> > xfs partition destined for postgres; results noted below.
> >
> > I haven't been able to find many benchmarks showing desirable IO stats.
> > As far as I can tell the sequential input (around 110000K/sec) looks
> > good while the sequential output (around 50000K/sec) looks fairly
> > average.
> >
> > Advice and comments gratefully received.
> > Suggested Parameters for running pg_bench would be great!
> >
> > Thanks,
> > Rory
...
> > Version  1.03      ------Sequential Output------ --Sequential Input- --Random-
> >                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
> > Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
> > mas          13000M 52655  99 52951  10 32246   7 41441  72 113438  12 590.0   0
> > mas          13000M 49306  83 51967   9 32269   7 42442  70 115427  12 590.5   1
> > mas          13000M 53449  89 51982  10 32625   7 42819  71 111829  11 638.3   0
> > mas          13000M 51818  88 51639   9 33127   7 42377  70 108585  11 556.5   0
...
> >                     ------Sequential Create------ --------Random Create--------
> >                     -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
> > files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
> > mas              16  7970  36 +++++ +++  7534  34  7918  36 +++++ +++  4482  22
> > mas              16  7945  33 +++++ +++  7483  30  7918  42 +++++ +++  4438  20
> > mas              16  8481  48 +++++ +++  7468  31  7870  39 +++++ +++  4385  23
> > mas              16  7915  36 +++++ +++  7498  33  7930  41 +++++ +++  4619  23
> > mas              16  8553  35 +++++ +++  8312  38  8613  37 +++++ +++  4513  24
> > mas              16  8498  40 +++++ +++  8215  33  8570  43 +++++ +++  4858  26