Re: Planning a Large RDBMS - Mailing list pgsql-novice

From Jeff Davis
Subject Re: Planning a Large RDBMS
Date
Msg-id 1339897255.14729.2.camel@jdavis
Whole thread Raw
In response to Planning a Large RDBMS  (Alessandro Gagliardi <alessandro@path.com>)
List pgsql-novice
On Fri, 2012-06-15 at 11:11 -0700, Alessandro Gagliardi wrote:
> We've been using MongoDB for the past couple years and are talking
> about moving to a more traditional (i.e. stable) database system for
> OLTP. I've been using RDBMS for OLAP already, but the demands are
> somewhat different. But we figure, if Facebook can do it, we can. The
> expertise around here is more suited for MySQL, but I, for one, am
> fond of the features in Postgres that you can't find in MySQL. Our
> Mongo system is sharded across 11 logical nodes on Amazon's EC2. Our
> database is in the hundreds of gigabytes (i.e. too big for memory on
> one machine) and we can expect it to exceed a terabyte before too
> long. We have the distinct advantage of being able to plan this from
> the ground up. I've read a bit about partitioning and table spaces,
> but from what I can tell, those solutions still require the database
> to be hosted by a single machine which may not suffice for our
> purposes.

I suggest first identifying what portions of the data are largely
independent versus closely tied together. Then use that to determine how
it makes sense to lay these out across multiple machines. Then, just use
different instances of postgres across the different machines. You'll
need to think carefully about how you interact with those different
instances, but you've already done a lot of that thinking for your
current setup.

One way to think about it is that one postgres instance on one machine
is the unit of consistency.

Regards,
    Jeff Davis



pgsql-novice by date:

Previous
From: Jeff Davis
Date:
Subject: Re: Planning a Large RDBMS
Next
From: Joseph Marlin
Date:
Subject: Impatient warm standby - Recovery spam in pg_log/startup.log