Thread: general hardware advice
I am searching what would be the best hardware combination to a new server i have to install, the new server will have a Postgresql 9.0 with a database of about 10gb, the database part it is not the problem for me, in this size almost, the part where i am a bit lost is that i have to share from the same server about 20TB of data with samba or ISCSI (i have to test both ways to share when i have the hardware) because this is to be the file-server of 8 Avid video workstations. I am pretty sure that the Postgresql should be installed in a raid10 configuration wigh pg_xlog in a different physical volume, i know too that the most important things to Postgresql are memory and disks, i will get a raid card with BBU or flash cache to be safe with write caching. About the file-server part the files we are going to move are in the range of 1gb~4gb in mostly a sequential access nature for what i have read of how avid works, so i think the best would be a raid 6 here, but i am not sure what hardware to buy because i have not stored never this quantity of storage, i have read in this mailing list other times about external enclosure boxes to attach storage and i have read raid cards recommendations before, lsi, 3ware or areca seems safe bets but i don't know how to put all the pieces together. I have searched in HP Servers but i can only find HP Smart Array controllers and i don't find a server that sill uses the best of the Smart Array P800 or Smart Array P812, so i don't know what server to buy (it is better to stick with a server vendor or mount my own with hardware parts), what box enclosure to buy and how to connect the two. Could anyone please point me specific hardware parts that he would recommend me? I am studying too the possibility of use an OCZ Vertex 2 Pro with Flashcache or Bcache to use it like a second level filesystem cache, any comments on that please? Thanks, Miguel Angel.
On Sun, Feb 6, 2011 at 11:16 AM, Linos <info@linos.es> wrote: > I am searching what would be the best hardware combination to a new server i > have to install, the new server will have a Postgresql 9.0 with a database > of about 10gb, the database part it is not the problem for me, in this size > almost, the part where i am a bit lost is that i have to share from the same > server about 20TB of data with samba or ISCSI (i have to test both ways to > share when i have the hardware) because this is to be the file-server of 8 > Avid video workstations. > > I am pretty sure that the Postgresql should be installed in a raid10 > configuration wigh pg_xlog in a different physical volume, i know too that > the most important things to Postgresql are memory and disks, i will get a > raid card with BBU or flash cache to be safe with write caching. I'd put all of pgsql on a different card than the file share. > About the file-server part the files we are going to move are in the range > of 1gb~4gb in mostly a sequential access nature for what i have read of how > avid works, so i think the best would be a raid 6 here, but i am not sure > what hardware to buy because i have not stored never this quantity of > storage, i have read in this mailing list other times about external > enclosure boxes to attach storage and i have read raid cards recommendations > before, lsi, 3ware or areca seems safe bets but i don't know how to put all > the pieces together. If the data on this is considered disposable, then you could look at RAID-0 for the fastest performance. Even with SW RAID0 you'd get incredible throughput with a few disks. > I am studying too the possibility of use an OCZ Vertex 2 Pro with Flashcache > or Bcache to use it like a second level filesystem cache, any comments on > that please? My coworkers RAVE over ZFS solaris with flash drives for cache and spinning drives for mass storage.
On Sun, 06 Feb 2011 19:16:23 +0100, Linos <info@linos.es> wrote: > I am searching what would be the best hardware combination to a new > server i have to install, the new server will have a Postgresql 9.0 with > a database of about 10gb, the database part it is not the problem for > me, in this size almost, the part where i am a bit lost is that i have > to share from the same server about 20TB of data with samba or ISCSI (i > have to test both ways to share when i have the hardware) because this > is to be the file-server of 8 Avid video workstations. What is the expected load on the postgresql instance ? Also, for multiple high throughput concurrent streams (as in AVID), filesystem choice is critical.
El 06/02/11 20:05, Scott Marlowe escribió: > On Sun, Feb 6, 2011 at 11:16 AM, Linos<info@linos.es> wrote: >> I am searching what would be the best hardware combination to a new server i >> have to install, the new server will have a Postgresql 9.0 with a database >> of about 10gb, the database part it is not the problem for me, in this size >> almost, the part where i am a bit lost is that i have to share from the same >> server about 20TB of data with samba or ISCSI (i have to test both ways to >> share when i have the hardware) because this is to be the file-server of 8 >> Avid video workstations. >> >> I am pretty sure that the Postgresql should be installed in a raid10 >> configuration wigh pg_xlog in a different physical volume, i know too that >> the most important things to Postgresql are memory and disks, i will get a >> raid card with BBU or flash cache to be safe with write caching. > > I'd put all of pgsql on a different card than the file share. This should not be a problem, i think you have reason here maybe i could use internal disks and 1 raid card for Postgresql. > >> About the file-server part the files we are going to move are in the range >> of 1gb~4gb in mostly a sequential access nature for what i have read of how >> avid works, so i think the best would be a raid 6 here, but i am not sure >> what hardware to buy because i have not stored never this quantity of >> storage, i have read in this mailing list other times about external >> enclosure boxes to attach storage and i have read raid cards recommendations >> before, lsi, 3ware or areca seems safe bets but i don't know how to put all >> the pieces together. > > If the data on this is considered disposable, then you could look at > RAID-0 for the fastest performance. Even with SW RAID0 you'd get > incredible throughput with a few disks. > It is not disposable, all the contrary, i have to take special care of the this files :) >> I am studying too the possibility of use an OCZ Vertex 2 Pro with Flashcache >> or Bcache to use it like a second level filesystem cache, any comments on >> that please? > > My coworkers RAVE over ZFS solaris with flash drives for cache and > spinning drives for mass storage. > i think neither flashcache nor bcache are at the level of zfs with l2arc in ssd but should perform well anyway, what it is the preferred way of use good raid cards with zfs? jbod and configure in raid-z? i have not used zfs still. Regards, Miguel Angel.
El 06/02/11 20:24, Pierre C escribió: > On Sun, 06 Feb 2011 19:16:23 +0100, Linos <info@linos.es> wrote: > >> I am searching what would be the best hardware combination to a new server i >> have to install, the new server will have a Postgresql 9.0 with a database of >> about 10gb, the database part it is not the problem for me, in this size >> almost, the part where i am a bit lost is that i have to share from the same >> server about 20TB of data with samba or ISCSI (i have to test both ways to >> share when i have the hardware) because this is to be the file-server of 8 >> Avid video workstations. > > What is the expected load on the postgresql instance ? > > Also, for multiple high throughput concurrent streams (as in AVID), filesystem > choice is critical. > The load for the Postgresql instance it is to be low, maybe 5 or 6 simultaneous clients with a oltp application, but it is database intensive only when reporting not in the usual work. About the filesystem i was thinking to use xfs, i think would be the best for this use-case in Linux, any advice on this please? Regards, Miguel Angel.
--- On Sun, 2/6/11, Linos <info@linos.es> wrote: > I am studying too the possibility of use an OCZ Vertex 2 > Pro with Flashcache or Bcache to use it like a second level > filesystem cache, any comments on that please? > OCZ Vertex 2 Pro is a lot more expensive than other SSD of comparable performances because it comes with a supercapacitorthat guarantees durability. If you're just using the SSD as a cache, you don't need durability. You could save a lot of money by getting SSDs withoutsupercapacitor, such as Corsair Force.
ah ok, i was not sure about this point, thanks. Regards, Miguel Angel El 06/02/11 21:38, Andy escribió: > > --- On Sun, 2/6/11, Linos<info@linos.es> wrote: > >> I am studying too the possibility of use an OCZ Vertex 2 >> Pro with Flashcache or Bcache to use it like a second level >> filesystem cache, any comments on that please? >> > > OCZ Vertex 2 Pro is a lot more expensive than other SSD of comparable performances because it comes with a supercapacitorthat guarantees durability. > > If you're just using the SSD as a cache, you don't need durability. You could save a lot of money by getting SSDs withoutsupercapacitor, such as Corsair Force. > > > >