Re: PostgreSQL alternative to "Oracle Real Application Cluster" - Mailing list pgsql-general

Think about something like this:

A = Data storage.  Probaby SAN or something very reliable and big enough to
hold all the data for images.

B = A database server containing the "Live", real, up-to-the-minute data.

C = An application server used to manage b.

D = A replicated database copy of b.  Read only.

E = One or more web servers, load balancers or caching servers accessible to
your users

+---+        +---+     +---+
| A | ------ | B |-----| C |
+---+        +---+     +---+
 | \
 |  \______________
 |                 \
+---+   +---+     +---+   +---+
| E | - | D |     | E | - | D |
+---+   +---+     +---+   +---+


I hope my diagram is a clear.  This isn't really too difficult a setup.
Imagine for now three database servers.  The master db server (B) is updated
infrequently.  Several database servers out there (D) keep an up-to-date
read only copy of B.  Each of these database servers can feed data to
several web servers.  The webservers are accessible to your users and do
most of the work.

All your image data is stored on a good quality storage medium that is
highly available and fault tolerant.  Each webserver can have read only
access to this data and serve it to requesting web clients directly.

Load can be taken off your storage network and your replica database servers
by using caching servers running something like squid.

No one on this list can tell you what your needs are going to be, but maybe
an example, general purpose cluster might look something like this:

A: One heavy duty SAN.
B: One dual processor XEON server with 4 GB of RAM and fast disk array
C: One application server or an application server cluster suited to your
tasks
D: Two dual processor XEON servers with 4 GB of RAM and fast disk array
E: Two web server clusters consisting of:
E.1: 3 web servers with 1GB RAM and connection to A and able to serve data
from D
E.2: 3 caching servers serving content hosted on E.1
E.3: 1 Load balancer directing requests to E.2 and E.1

So a total configuration like this MIGHT consist of:
1 heavy duty SAN
3 dual processor XEON servers with 4 GB of RAM each and fast disk arrays
13 single processor servers with 1GB of RAM each and HBA connection to SAN
(1 application server (C), 6 web servers (E.1), 6 caching servers (E.2))
2 load balancers

Use any programming language you like.  Perl, Php, Java can all handle a
setup like this.

Store the meta info about your images in the database, store a link to the
image data in the database but store the images on the filesystem (so that
they don't have to be replicated).

The quantity of E.1 and E.2 per each D would be a big variable and need some
testing and tweaking.

Just an idea.  ;-)

--
Matthew Nuzum
www.bearfruit.org
cobalt@bearfruit.org


> -----Original Message-----
> From: Arjen van der Meijden [mailto:acm@tweakers.net]
> Sent: Wednesday, June 18, 2003 9:39 AM
> To: 'Paul Thomas'; 'pgsql-general @ postgresql . org'
> Subject: Re: PostgreSQL alternative to "Oracle Real Application Cluster"
>
> > Paul Thomas wrote:
> >
> > On 18/06/2003 11:19 Hubert Fröhlich wrote:
> > >
> > > What we would like to have is some alternative concept
> > which allows us
> > > to use PostgreSQL on a powerful hardware to get highly
> > performant and
> > > higly available database access on our terabytes.
> > >
> > > a) Does PostgreSQL have some features using a clustered hardware?
> > > b) If no, what could be an alternative hardware concept ?
> > >
> > >
> > > Can anybody give me some advice or some hints to somebody who could
> > > help
> > > us a bit further, some web page ...
> >
> > Have you considered a simple load balancing set-up? You don't
> > mention how
> > you generate the web pages - hopefully not a scripting language if
> > performance matters! If, for  instance, your application is
> > written in
> > Java/JSPs then you could set up several servers running Tomcat and
> > PostgreSQL and load balance them from an Apache web server
> > with mod_jk.
> Where would the 400-4000GB of data go in this setup? On all the distinct
> postgresql-servers? On a single SAN/NAS? (Postgresql doesn't really work
> with that, does it? At least not in a loadbalancing setup).
> And why can't a scriptinglanguage not be used with loadbalancing? I know
> it is hard or impossible to get a connectionpool in such setups, but
> that doesn't mean they can't be used with loadbalancing...
>
> Arjen
>
>