Thread: Postgres Clustering

Hey folks,

I have done some googling and found a few things on the matter.  But
am looking for some suggestions from the experts out there.

Got any good pointers for reading material to help me get up to speed
on PostgreSQL clustering?   What options are available?  What are the
issues?  Terminology.  I'm pretty new to the whole data-warehouse
thing.   And once I do all the reading, I'll even be open to product
recommendations :-)

And in particular since I already have heard of this particular
product - are there any opinions on Continuent?

thanks,
-Alan

--
“Mother Nature doesn’t do bailouts.”
         - Glenn Prickett

Alan,

here I'm implementing something similar to the Chord protocol [1] on the application level to partition my data across 6 PostgreSQL servers with N+1 replication. Two up sides on this approch:

1 - When one server is down the load is spread between all the other ones, instead of going only to the replication pair.
2 - Adding one more server is easy, you only have to reallocate aprox. 1/n of your data (n=number of servers).

Good luck there!

Best,
Daniel

Try Cybercluster....

-----Mensaje original-----
De: pgsql-performance-owner@postgresql.org
[mailto:pgsql-performance-owner@postgresql.org] En nombre de Alan McKay
Enviado el: miércoles, 27 de mayo de 2009 13:57
Para: pgsql-performance@postgresql.org; pgsql-general@postgresql.org
Asunto: [PERFORM] Postgres Clustering

Hey folks,

I have done some googling and found a few things on the matter.  But
am looking for some suggestions from the experts out there.

Got any good pointers for reading material to help me get up to speed
on PostgreSQL clustering?   What options are available?  What are the
issues?  Terminology.  I'm pretty new to the whole data-warehouse
thing.   And once I do all the reading, I'll even be open to product
recommendations :-)

And in particular since I already have heard of this particular
product - are there any opinions on Continuent?

thanks,
-Alan

--
“Mother Nature doesn’t do bailouts.”
         - Glenn Prickett

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

On Wed, 27 May 2009, Alan McKay wrote:

> Got any good pointers for reading material to help me get up to speed
> on PostgreSQL clustering?   What options are available?  What are the
> issues?  Terminology.

http://wiki.postgresql.org/wiki/Replication%2C_Clustering%2C_and_Connection_Pooling
is where I keep my notes, which has worked out great for me because then
other people fix mistakes and keep everything current.

> And in particular since I already have heard of this particular
> product - are there any opinions on Continuent?

If what you need is fairly high-level replication where you can handle
some log replication overhead, one of their replicators might work well
for you.  I usually work more with OLTP systems, where write volume is too
high for something running that far outside of the database to perform
well enough.  You might note that Continuent's solutions advertise
"Scaling of read-intensive applications" rather than write ones.

By the way:  cross-posting on these lists is generally frowned upon. It
causes problems for people who reply to you but are aren't on all of the
lists you sent to.  If you're not sure what list something should go on,
just send it to -general rather than cc'ing multiple ones.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

> By the way:  cross-posting on these lists is generally frowned upon. It
> causes problems for people who reply to you but are aren't on all of the
> lists you sent to.  If you're not sure what list something should go on,
> just send it to -general rather than cc'ing multiple ones.

Duly noted!

Thanks for all the input so far folks.

--
“Mother Nature doesn’t do bailouts.”
         - Glenn Prickett

On Wednesday 27 May 2009 12:55:51 Eddy Ernesto Baños Fernández wrote:
> Try Cybercluster....
>
> -----Mensaje original-----
> De: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] En nombre de Alan McKay
> Enviado el: miércoles, 27 de mayo de 2009 13:57
> Para: pgsql-performance@postgresql.org; pgsql-general@postgresql.org
> Asunto: [PERFORM] Postgres Clustering
>
> Hey folks,
>
> I have done some googling and found a few things on the matter. But
> am looking for some suggestions from the experts out there.
>
> Got any good pointers for reading material to help me get up to speed
> on PostgreSQL clustering? What options are available? What are the
> issues? Terminology. I'm pretty new to the whole data-warehouse
> thing. And once I do all the reading, I'll even be open to product
> recommendations :-)
>
> And in particular since I already have heard of this particular
> product - are there any opinions on Continuent?

Continuent works (AFAIK) like pgpool clustering, it sends the same statements to both/all servers in the cluster but it has no insight to the servers beyond this, so if via a direct connection server A becomes out of sync with server B then continuent is oblivious.

Other tools to look at:
- EnterpriseDB's GridSQL
- SLONY
- Command Prompt's PG Replicator

>
> thanks,
> -Alan
>
> --
> “Mother Nature doesn’t do bailouts.”
> - Glenn Prickett
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance

> Continuent works (AFAIK) like pgpool clustering, it sends the same
> statements to both/all servers in the cluster but it has no insight to the
> servers beyond this, so if via a direct connection server A becomes out of
> sync with server B then continuent is oblivious.

So can the same be said for pgpool then?


thanks,
-Alan

--
“Mother Nature doesn’t do bailouts.”
         - Glenn Prickett

On Wednesday 27 May 2009 13:33:55 Alan McKay wrote:
> > Continuent works (AFAIK) like pgpool clustering, it sends the same
> > statements to both/all servers in the cluster but it has no insight to
> > the servers beyond this, so if via a direct connection server A becomes
> > out of sync with server B then continuent is oblivious.
>
> So can the same be said for pgpool then?

Yes

>
>
> thanks,
> -Alan
>
> --
> “Mother Nature doesn’t do bailouts.”
> - Glenn Prickett

Hi,

Le 27 mai 09 à 19:57, Alan McKay a écrit :
> I have done some googling and found a few things on the matter.  But
> am looking for some suggestions from the experts out there.
>
> Got any good pointers for reading material to help me get up to speed
> on PostgreSQL clustering?   What options are available?  What are the
> issues?  Terminology.  I'm pretty new to the whole data-warehouse
> thing.   And once I do all the reading, I'll even be open to product
> recommendations :-)

Depending on your exact needs, which the terminology you're using only
allow to guess about, you might enjoy this reading:
   http://wiki.postgresql.org/wiki/Image:Moskva_DB_Tools.v3.pdf

--
dim

> Depending on your exact needs, which the terminology you're using only allow
> to guess about, you might enjoy this reading:
>  http://wiki.postgresql.org/wiki/Image:Moskva_DB_Tools.v3.pdf

Thanks.  To be honest I don't even know myself what my needs are yet.
I've only been on the job here for a month now.  And one thing I
learned at PGCon last week is that I have a lot of benchmarching work
to do before I can figure out what my needs are!

At this point I'm just trying to read as much as I can on the general
topic of "clustering".    What I am most interested in is load
balancing to be able to scale up when required, and of course to be
able to determine ahead of time when that might be :-)

cheers,
-Alan


--
“Mother Nature doesn’t do bailouts.”
         - Glenn Prickett

We didn't have much luck with Continuent.  They had to make multiple
code level changes to get their product to work correctly with our app
on PG 8.2.  We never did get it successfully implemented.  At this point
I'm stuck with WAL shipping as I can't find anything that fits my
constraints.

Thanks,

Scot Kreienkamp

-----Original Message-----
From: pgsql-general-owner@postgresql.org
[mailto:pgsql-general-owner@postgresql.org] On Behalf Of Alan McKay
Sent: Wednesday, May 27, 2009 1:57 PM
To: pgsql-performance@postgresql.org; pgsql-general@postgresql.org
Subject: [GENERAL] Postgres Clustering

Hey folks,

I have done some googling and found a few things on the matter.  But
am looking for some suggestions from the experts out there.

Got any good pointers for reading material to help me get up to speed
on PostgreSQL clustering?   What options are available?  What are the
issues?  Terminology.  I'm pretty new to the whole data-warehouse
thing.   And once I do all the reading, I'll even be open to product
recommendations :-)

And in particular since I already have heard of this particular
product - are there any opinions on Continuent?

thanks,
-Alan

--
"Mother Nature doesn't do bailouts."
         - Glenn Prickett

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

2009/5/28 Eddy Ernesto Baños Fernández <eebanos@estudiantes.uci.cu>:
> Try Cybercluster....

I looked into that. There is one piece of documentation that is less
than ten pages long. There is no users group, no listserve, no
community that I can discern.

Do you have experience with it and if so could you please share.

Thanks.