Re: most bang for buck with ~ $20,000 - Mailing list pgsql-performance

Well, that's of course really hard to tell. From personal experience in
a read-mostly environment, the subtop woodcrest 5150 (2.6Ghz)
outperforms the top dempsey 5080 (3.7Ghz, in the same system) by quite a
nice margin. But that dempsey already has the faster FB-Dimm memory and
a much wider FSB compared to your 3.06Ghz Xeons.
But if we assume that the 3.7Ghz 5080 is just the extra mhz faster (~
25%), for a single (dual core) 3Ghz Woodcrest you might already be
talking about a 50% improvement in terms of cpu-power over your current
set-up. Of course depending on workload and its scalability etc etc.

In a perfect world, with linear scalability (note, a read-mostly
postgresql can actually do that on a Sun fire T2000 with solaris) that
would yield a 200% improvement when going form 2 to 4 cores. A 70-80%
scaling is more reasonable and would still imply you'd improve more than
150% over your current set-up.
Please note that this is partially based on internal testing and partial
on assumptions and would at least require more real-world testing for a
app more similar to yours.

As soon as we're publishing some numbers on this (and I don't forget),
I'll let you know on the list. That will include postgresql and recent
x86 cpu's on linux and should be ready soon.

Best regards,

Arjen


On 18-8-2006 21:51, Kenji Morishige wrote:
> Thanks Arjen for your reply, this is definitely something to consider. I
> think in our case, we are not too concerned with the tech image as much as if
> the machine will allow us to scale the loads we need. I'm not sure if we
> should worry so much about the IO bandwidth as we are not even close to
> saturating 320MB/s.  I think stability, reliability, and ease-of-use and
> recovery is our main concern at the moment.  I currently am runing a load
> average of about .5 on a dual Xeon 3.06Ghz P4 setup.  How much CPU
> performance improvement do you think the new woodcrest cpus are over these?
>
> -Kenji
>
> On Fri, Aug 18, 2006 at 09:41:55PM +0200, Arjen van der Meijden wrote:
>> Hi Kenji,
>>
>> I'm not sure what you mean by 'something newer'? The intel
>> woodcrest-cpu's are brand-new compared to the amd opterons. But if you
>> need a 4-cpu config (I take it you want 8-cores in that case), Dell
>> doesn't offer much. Whether something new will come, I don't know. I'm
>> not sure when (or if?) a MP-Woodcrest will arrive and/or when Dell will
>> start offering Opteron-servers.
>>
>> Sas has been designed as the successor to SCSI.
>>
>> As I see it, SAS has currently one major disadvantage. Lots of new
>> servers are equipped with SAS-drives, a few nice SAS-raidcontrollers
>> exist, but the availability of external enclosures for SAS is not
>> widespread yet. So your options of going beyond (say) 8 disks per system
>> are a bit limited.
>>
>> There are of course advantages as well. The bus is much wider (you can
>> have 4 lanes of 3Gbps each to an enclosure). You can mix sas and sata
>> disks, so you could have two arrays in the same enclosure, one big
>> storage bin and a very fast array or just use only sata disks on a sas
>> controller. The cabling itself is also much simpler/more flexible
>> (although using a hot-plug enclosure of course shields you mostly from
>> that).
>> But whether its the right choice to make now? I'm not sure. We weren't
>> to fond of investing a lot of money in an end-of-life system. And since
>> we're a tech-website, we also had to worry about our "being modern
>> image", of course ;)
>>
>> The main disadvantage I see in this case is, as said, the limited
>> availability of external enclosures in comparison to SCSI and Fibre
>> Channel. HP currently only offers their MSA50 (for the rather expensive
>> SFF disks) while their MSA60 (normal disks) will not be available until
>> somewhere in 2007 and Dell also only offers one enclosure, the MD1000.
>> The other big players offer nothing yet, as far as I know, while they
>> normally offer several SCSI and/or FC-enclosures.
>> There are also some third-party enclosures (adaptec and promise for
>> instance) available of course.
>>
>> Best regards,
>>
>> Arjen
>>
>> On 18-8-2006 21:07, Kenji Morishige wrote:
>>> Thanks Arjen,
>>> I have unlimited rack space if I really need it.  Is serial/SAS really the
>>> better route to go than SCSI these days? I'm so used to ordering SCSI that
>>> I've been out of the loop with new disk enclosures and disk tech.  I been
>>> trying to price out a HP DL585, but those are considerably more than the
>>> Dells.  Is it worth waiting a few more weeks/months for Dell to release
>>> something newer?
>>>
>>> -Kenji
>>>
>>> On Wed, Aug 09, 2006 at 07:35:22AM +0200, Arjen van der Meijden wrote:
>>>> With such a budget you should easily be able to get something like:
>>>> - A 1U high-performance server (for instance the Dell 1950 with 2x
>>>> Woodcrest 5160, 16GB of FB-Dimm memory, one 5i and one 5e perc raid
>>>> controller and some disks internally)
>>>> - An external SAS direct attached disks storage enclosure full with 15k
>>>> rpm 36GB disks (for instance the MD1000, with 15x 36GB 15k disks)
>>>>
>>>> Going for the dell-solution would set you back "only" (including
>>>> savings) about $13-$14k. HP offers a similar solutions (a HP DL360G5 or
>>>> a DL380G5/DL385 with two MSA50's for instance) which also fit in your
>>>> budget afaik. The other players tend to be (a bit) more expensive, force
>>>> you to go with Fibre Channel or "ancient" SCSI external storage ;)
>>>>
>>>> If you'd like to have a product by a generic vendor, have a look at the
>>>> Adaptec JS50 SAS Jbod enclosure or Promise's Vtrak 300 (both offer 12
>>>> sas/sata bays in 2U) for storage.
>>>>
>>>> If you're limited to only 2U of rack space, its a bit more difficult to
>>>> get maximum I/O in your budget (you have basically space for about 8 or
>>>> 12 3.5" disks (with generic suppliers) or 16 2.5" sff disks (with HP)).
>>>> But you should still be able to have two top-off-the-line x86 cpu's (amd
>>>> opteron 285 or intel woorcrest 5160) and 16GB of memory (even FB Dimm,
>>>> which is pretty expensive).
>>>>
>>>> Best regards,
>>>>
>>>> Arjen van der Meijden
>>>>
>>>>
>>>> On 8-8-2006 22:43, Kenji Morishige wrote:
>>>>> I've asked for some help here a few months ago and got some really
>>>>> helpfull
>>>>> answers regarding RAID controllers and server configuration.  Up until
>>>>> recently I've been running PostgreSQL on a two year old Dual Xeon 3.06Ghz
>>>>> machine with a single channel RAID controller (previously Adaptec 2200S,
>>>>> but
>>>>> now changed to LSI MegaRAID). The 2U unit is from a generic vendor using
>>>>> what
>>>>> I believe is a SuperMicro motherboard.  In the last week after upgrading
>>>>> the
>>>>> RAID controller, the machine has had disk failure and some other issues.
>>>>> I
>>>>> would like to build a very reliable dedicated postgreSQL server that has
>>>>> the
>>>>> ultimate possible performance and reliabily for around $20,000.  The
>>>>> data set
>>>>> size is only currently about 4GB, but is increasing by approximately 50MB
>>>>> daily.  The server also requires about 500 connections and I have been
>>>>> monitoring about 100-200 queries per second at the moment.  I am
>>>>> planning to
>>>>> run FreeBSD 6.1 if possible, but I am open to any other suggestions if it
>>>>> improves performance.
>>>>>
>>>>> I am considering a setup such as this:
>>>>> - At least dual cpu (possibly with 2 cores each)
>>>>> - 4GB of RAM
>>>>> - 2 disk RAID 1 array for root disk
>>>>> - 4 disk RAID 1+0 array for PGDATA
>>>>> - 2 disk RAID 1 array for pg_xlog
>>>>>
>>>>> Does anyone know a vendor that might be able provide such setup?  Any
>>>>> critique in this design? I'm thinking having a 2 channel RAID controller
>>>>> to
>>>>> seperate the PGDATA, root and pg_xlog.
>>>>>
>>>>> Sincerely,
>>>>> Kenji
>>>>>
>>>>> ---------------------------(end of broadcast)---------------------------
>>>>> TIP 2: Don't 'kill -9' the postmaster
>>>>>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
>                http://archives.postgresql.org
>