Thread: Postgres clustering?
I am interested in finding out about Enterprise scale Postgres installations and clustering, especially on Linux. Essentially I would like to know the possibility that Postgres can store the database data in a central location (ex: on a SAN fiber array) and have a cluster of machines sharing processor/RAM/IO bandwidth to do the application processing. Or perhaps there is another solution similar to what www.emicnetworks.com have developed for MySQL where the 'cluster' seems to be really a set of individual MySQL servers that syncronize their data at a very high speed? I have found from my investigations that the usual Linux cluster tools, like Beowulf/Mosix/openMosix/Alinka/Qlusters dont support Postgres and similar apps because of the shared memory or other issues. The complexity of RDBMS transactions seems to have limited parallel processing in this field to the big (read: EXPENSIVE) guys like Oracle and IBM. Does anyone know of a technique to centralize the Postgres data storage and farm out the processor/RAM/IO needs to a scalable resource? It seems like it should be able to be done, but at the moment its looking to me like the choice looks like this: - Buy bigger and bigger hardware to run Postgres. - Fork out large sums of money and port applications to Oracle or DB2 Anyone have any success with OpenSource tools for this or similar issues? Thanks.
There currently is no solution as I have asked this question quite a few times in the past. The best solution that exists for Postgres is master/slave asynchronous replication which is not what you want. I am very interested in a similar solution and believe that I could get some resources from my company for implementing such a system. Are you interested in helping to develop the functionality for Postgres? Is anybody else in the Postgres world even interested in pursuing this functionality set in more than lip service? Are there any currently active Postgres projects out there trying to achieve this? Is there any activity on the replication issues that appear on the TODO Postgres list at http://developer.postgresql.org/todo.php? If so who is working on them so I can contact them? Feedback on these topics would be appreciated. Keith -----Original Message----- From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of warwick.poole Sent: Thursday, February 26, 2004 2:41 PM To: pgsql-general@postgresql.org Subject: [GENERAL] Postgres clustering? I am interested in finding out about Enterprise scale Postgres installations and clustering, especially on Linux. Essentially I would like to know the possibility that Postgres can store the database data in a central location (ex: on a SAN fiber array) and have a cluster of machines sharing processor/RAM/IO bandwidth to do the application processing. Or perhaps there is another solution similar to what www.emicnetworks.com have developed for MySQL where the 'cluster' seems to be really a set of individual MySQL servers that syncronize their data at a very high speed? I have found from my investigations that the usual Linux cluster tools, like Beowulf/Mosix/openMosix/Alinka/Qlusters dont support Postgres and similar apps because of the shared memory or other issues. The complexity of RDBMS transactions seems to have limited parallel processing in this field to the big (read: EXPENSIVE) guys like Oracle and IBM. Does anyone know of a technique to centralize the Postgres data storage and farm out the processor/RAM/IO needs to a scalable resource? It seems like it should be able to be done, but at the moment its looking to me like the choice looks like this: - Buy bigger and bigger hardware to run Postgres. - Fork out large sums of money and port applications to Oracle or DB2 Anyone have any success with OpenSource tools for this or similar issues? Thanks. ---------------------------(end of broadcast)--------------------------- TIP 8: explain analyze is your friend
----- Original Message ----- From: kbottner@comcast.net <Keith Bottner> To: pgsql-general@postgresql.org Date: 26.02.2004 22:22:02 Subject: Re: [GENERAL] Postgres clustering? > There currently is no solution as I have asked this question quite a few > times in the past. [...] Forgive me if you already get this: I think LinuxLabs has such a thing in its portfolio, called Clusgres. It seems to be a complete (bundled) solution with their cluster products: http://www.linuxlabs.com/clusgres.html Bernd
On Thu, Feb 26, 2004 at 03:22:02PM -0600, Keith Bottner wrote: > I am very interested in a similar solution and believe that I could get some > resources from my company for implementing such a system. Are you interested > in helping to develop the functionality for Postgres? > > Is anybody else in the Postgres world even interested in pursuing this > functionality set in more than lip service? Are there any currently active > Postgres projects out there trying to achieve this? If you want this, then go and support the folks working on the Postgres-R project. They have a design for this sort of thing there, although it does some remarkably tricky things that not everyone thinks will work. (I happen to be among the somewhat optimistic on this front, but I haven't been able to bring any money to the project.) A -- Andrew Sullivan
Thanks Andrew, I will do some digging on the Postgres-R front to see what
their focus is.
Keith
-----Original Message-----
From: pgsql-general-owner@postgresql.org
[mailto:pgsql-general-owner@postgresql.org] On Behalf Of Andrew Sullivan
Sent: Thursday, February 26, 2004 4:36 PM
To: pgsql-general@postgresql.org
Subject: Re: [GENERAL] Postgres clustering?
On Thu, Feb 26, 2004 at 03:22:02PM -0600, Keith Bottner wrote:
> I am very interested in a similar solution and believe that I could
> get some resources from my company for implementing such a system. Are
> you interested in helping to develop the functionality for Postgres?
>
> Is anybody else in the Postgres world even interested in pursuing this
> functionality set in more than lip service? Are there any currently
> active Postgres projects out there trying to achieve this?
If you want this, then go and support the folks working on the Postgres-R
project. They have a design for this sort of thing there, although it does
some remarkably tricky things that not everyone thinks will work. (I happen
to be among the somewhat optimistic on this front, but I haven't been able
to bring any money to the
project.)
A
--
Andrew Sullivan
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)