Re: Horizontal scalability/sharding - Mailing list pgsql-hackers

From David E. Wheeler
Subject Re: Horizontal scalability/sharding
Date
Msg-id 2A4846BD-D2BE-46A1-AB25-B5274190A102@justatheory.com
Whole thread Raw
In response to Re: Horizontal scalability/sharding  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
On Sep 1, 2015, at 1:47 PM, Robert Haas <robertmhaas@gmail.com> wrote:

> Admittedly, there are some problems with snapshots here: if you don't
> do anything special about snapshots, then what you have here will be
> "eventually consistent" behavior.  But that might be suitable for some
> environments, such as very loosely coupled system where not all nodes
> are connected all the time.

Given that we’re discussing multi-node architectures here, you should expect that not all nodes will be connected at
anytime. Nodes fail, but the cluster should not. 

> And, for those environments where you do
> need consistent snapshots, we can imagine ways to get that behavior,
> like having the GTM consider the transaction uncommitted until it's
> been logically replicated to every node.

Again, you need a way to deal with nodes going down. I can envision building a cluster with twelve nodes replicated to
eachof three geographically-distributed data centers. Each replication/sync model needs to be able to handle nodes
goingup and down, data centers or racks going up or down, and nodes being added and removed. 

But even with smaller clusters, there’s no way around the fact that no system can guarantee that all nodes will be
availableat all times. 

Best,

David


pgsql-hackers by date:

Previous
From: Greg Stark
Date:
Subject: Re: Unicode mapping scripts cleanup
Next
From: Josh Berkus
Date:
Subject: Re: Horizontal scalability/sharding