Re: Help specifying new web server/database machine - Mailing list pgsql-performance

On 09/06/05, Matthew Nuzum (mattnuzum@gmail.com) wrote:
> On 6/9/05, Rory Campbell-Lange <rory@campbell-lange.net> wrote:
> > Disks:
> >
> > I'm somewhat confused here. I've followed the various notes about SATA
> > vs SCSI and it seems that SCSI is the way to go. On a four-slot 1U
> > server, would one do a single RAID10 over 4 disks 10000rpm U320 disks?
> > I would run the database in its own partition, separate from the rest of
> > the OS, possible on LVM. An LSI-Megaraid-2 appears to be the card of
> > choice.

> Can you tell us about your application? How much data will you have,
> what is your ratio of reads to writes, how tollerant to data loss are
> you? (for example, some people load their data in batches and if they
> loose their data its no big deal, others would have heart failure if a
> few transactions were lost)

The application is a web-based prototype system for kids to make their
own galleries based on content found in museums and galleries. They will
link to content provided by curators, and be able to add in their own
material, including movies, sounds and pictures. All the content,
however, will be restricted in size. I also do not intend to store the
movies, sounds or pictures in the database (although I have happily done
the latter in the past).

Up to the data will be uploaded from 3G handsets. The rest will be done
on a per-user, per-pc basis through the web interface.

The service is expected to be used by about 50000 users over 18 months.
Of these around half will be content creators, so will account for say
half a million rows in the main content table and under 2 million rows
in the commentary table. The most used table will probably be a
'history' function required by the contract, tracking use through the
site. I imagine this will account for something like 20 million rows
(with very little data in them).

The main tables will have something like 80% read, 20% write (thumb
suck). The history table will be read by an automated process at 3 in
the morning, to pick up some stats on how people are using the system.

It wouldn't be a problem to very occasionally (once a month) lose a tiny
piece of data (i.e a record). Losing any significant amounts of data is
entirely out of the question.

> If your application is 95% writes then people will suggest drastically
> different hardware than if your application is 95% selects.
>
> Here is an example of one of my servers:
> application is 95+% selects, has 15GB of data (counting indexes), low
> tollerance for data loss, runs on a 1 GHz P3 Compaq server with
> mirrored 35 GB IDE disks and 1.6GB of RAM. Application response time
> is aproximately .1 second to serve a request on a moderately loaded
> server.

Yeah. Maybe the machine I'm speccing up is total overkill for this
project? I'm just worried that if it is a big success, or if we have 400
kids pounding the server at once over high-speed school lines, the thing
will grind to a halt.

Thanks very much for your comments.

Regards,
Rory

--
Rory Campbell-Lange
<rory@campbell-lange.net>
<www.campbell-lange.net>