Thread: PGCluster-II Progress
I was searching info about PgCluster-II yesterday and there is not much information about it. Do can give to me any report of this? Because I need to know the progress of the project. On PgFoundry, only it talks about PgCluster-1.9, but not of the 2.x versions. Who is the PgCluster-II´s developer? Regards "The hurry is enemy of the success: for that reason.......Be patient" Ing. Marcos L. Ortiz Valmaseda Línea Soporte y Despliegue Centro de Tecnologías de Almacenamiento y Análisis de Datos (CENTALAD) Linux User # 418229 PostgreSQL User http://www.postgresql.org http://www.planetpostgresql.org/ http://www.postgresql-es.org/
I had the same question a while a go.
After a lot of googling I found http://www.cybertec.at/english/start_e.html
This seems to be an active replacement for the dead? pg-cluster
HTH
Regards,
Serge Fonville
On Tue, Sep 15, 2009 at 1:29 PM, Marcos Luis Ortiz Valmaseda <mlortiz@uci.cu> wrote:
I was searching info about PgCluster-II yesterday and there is not much information about it.
Do can give to me any report of this? Because I need to know the progress of the project.
On PgFoundry, only it talks about PgCluster-1.9, but not of the 2.x versions.
Who is the PgCluster-II´s developer?
Regards
"The hurry is enemy of the success: for that reason.......Be patient"
Ing. Marcos L. Ortiz Valmaseda
Línea Soporte y Despliegue
Centro de Tecnologías de Almacenamiento y Análisis de Datos (CENTALAD)
Linux User # 418229
PostgreSQL User
http://www.postgresql.org
http://www.planetpostgresql.org/
http://www.postgresql-es.org/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Yeah, the problem here is that CyberCluster is based yet on PostgreSQL 8.1 and is a very old version to use it. I found the developer of PgCluster-II: Atsushi MITANI - mitani@sraw.co.jp "The hurry is enemy of the success: for that reason.......Be patient" Ing. Marcos L. Ortiz Valmaseda Línea Soporte y Despliegue Centro de Tecnologías de Almacenamiento y Análisis de Datos (CENTALAD) Linux User # 418229 PostgreSQL User http://www.postgresql.org http://www.planetpostgresql.org/ http://www.postgresql-es.org/ ----- Mensaje original ----- De: "Serge Fonville" <serge.fonville@gmail.com> Para: "Marcos Luis Ortiz Valmaseda" <mlortiz@uci.cu> CC: "pgsql-hackers" <pgsql-hackers@postgresql.org> Enviados: Martes, 15 de Septiembre 2009 1:32:37 GMT -10:00 Hawai Asunto: Re: [HACKERS] PGCluster-II Progress I had the same question a while a go. After a lot of googling I found http://www.cybertec.at/english/start_e.html This seems to be an active replacement for the dead? pg-cluster HTH Regards, Serge Fonville On Tue, Sep 15, 2009 at 1:29 PM, Marcos Luis Ortiz Valmaseda < mlortiz@uci.cu > wrote: I was searching info about PgCluster-II yesterday and there is not much information about it. Do can give to me any report of this? Because I need to know the progress of the project. On PgFoundry, only it talks about PgCluster-1.9, but not of the 2.x versions. Who is the PgCluster-II´s developer? Regards "The hurry is enemy of the success: for that reason.......Be patient" Ing. Marcos L. Ortiz Valmaseda Línea Soporte y Despliegue Centro de Tecnologías de Almacenamiento y Análisis de Datos (CENTALAD) Linux User # 418229 PostgreSQL User http://www.postgresql.org http://www.planetpostgresql.org/ http://www.postgresql-es.org/ -- Sent via pgsql-hackers mailing list ( pgsql-hackers@postgresql.org ) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Sep 15, 2009 at 7:48 AM, Marcos Luis Ortiz Valmaseda <mlortiz@uci.cu> wrote:
Yeah, AFAICS, PGCluster II is and has been dead for years.
I did an architectural review of PGCluster II while at EnterpriseDB, and it's never going to work from a performance perspective. Unfortunately, the architecture (as it stands) requires coordination of pretty much all locks and semaphores, and will only run on a single system because it requires a nearly-identical shared memory segment for each instance. Unlike Oracle RAC, which was designed to be run on separate nodes and share/coordinate only relevant resources, PGCluster II coordinates almost everything (transaction ids, buffer locks, etc.).
As an example, Oracle doesn't need to communicate with every node in the cluster to generate a new transaction id (SCN in Oracle parlance) because it employs a system based on Lamport timestamps, but PGCluster II has to coordinate a global transaction id across all nodes.
Also, PGCluster II has no concept of shared data/local WAL. Whereas Oracle supports threads of REDO/UNDO generated by the local node, all WAL in PGCluster II has to be coordinated between nodes, which creates a bottleneck on *all* concurrent workloads.
When I first saw the demonstration of PGCluster II, I was in awe. Unfortunately, when I reviewed the architecture, I saw that it was too good to be true. Perhaps it has been rearchitected in private to overcome some of these issues, but I'm not aware of it. All attempts to talk to Atsushi about it were met with no response.
-- Yeah, the problem here is that CyberCluster is based yet on PostgreSQL 8.1 and is a very old version to use it.
I found the developer of PgCluster-II: Atsushi MITANI - mitani@sraw.co.jp
Yeah, AFAICS, PGCluster II is and has been dead for years.
I did an architectural review of PGCluster II while at EnterpriseDB, and it's never going to work from a performance perspective. Unfortunately, the architecture (as it stands) requires coordination of pretty much all locks and semaphores, and will only run on a single system because it requires a nearly-identical shared memory segment for each instance. Unlike Oracle RAC, which was designed to be run on separate nodes and share/coordinate only relevant resources, PGCluster II coordinates almost everything (transaction ids, buffer locks, etc.).
As an example, Oracle doesn't need to communicate with every node in the cluster to generate a new transaction id (SCN in Oracle parlance) because it employs a system based on Lamport timestamps, but PGCluster II has to coordinate a global transaction id across all nodes.
Also, PGCluster II has no concept of shared data/local WAL. Whereas Oracle supports threads of REDO/UNDO generated by the local node, all WAL in PGCluster II has to be coordinated between nodes, which creates a bottleneck on *all* concurrent workloads.
When I first saw the demonstration of PGCluster II, I was in awe. Unfortunately, when I reviewed the architecture, I saw that it was too good to be true. Perhaps it has been rearchitected in private to overcome some of these issues, but I'm not aware of it. All attempts to talk to Atsushi about it were met with no response.
Jonah H. Harris, Senior DBA
myYearbook.com
On Tue, 2009-09-15 at 07:29 -0400, Marcos Luis Ortiz Valmaseda wrote: > I was searching info about PgCluster-II yesterday and there is not > much information about it. > Do can give to me any report of this? Because I need to know the > progress of the project. It is dead. -- Devrim GÜNDÜZ, RHCE Command Prompt - http://www.CommandPrompt.com devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org
Odd, I talked to him a couple of weeks ago and he was working on a new release in preparation for some upcoming talks he was doing ... was working on bringing it up to support 8.3.x ... "But, I'm just prepareing new version of the PGCluster..." Mitani ... any status on this? On Tue, 15 Sep 2009, Devrim G?ND?Z wrote: > On Tue, 2009-09-15 at 07:29 -0400, Marcos Luis Ortiz Valmaseda wrote: >> I was searching info about PgCluster-II yesterday and there is not >> much information about it. >> Do can give to me any report of this? Because I need to know the >> progress of the project. > > It is dead. > -- > Devrim G?ND?Z, RHCE > Command Prompt - http://www.CommandPrompt.com > devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr > http://www.gunduz.org > ---- Marc G. Fournier Hub.Org Hosting Solutions S.A. scrappy@hub.org http://www.hub.org Yahoo:yscrappy Skype: hub.org ICQ:7615664 MSN:scrappy@hub.org
On Tue, 2009-09-15 at 13:16 -0300, Marc G. Fournier wrote: > Odd, I talked to him a couple of weeks ago and he was working on a > new > release in preparation for some upcoming talks he was doing ... was > working on bringing it up to support 8.3.x ... > > "But, I'm just prepareing new version of the PGCluster..." Mitani was probably talking about PGCluster, not PGCluster -II. Mitani? Last time I heard about PGCluster-II was at Anniversary Summit(2006). Oh, and I met with him at France at 2007, and no improvements since then. Regards, -- Devrim GÜNDÜZ, RHCE Command Prompt - http://www.CommandPrompt.com devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org
Hi, 2009/9/16 Marc G. Fournier <scrappy@hub.org>: > > Odd, I talked to him a couple of weeks ago and he was working on a new > release in preparation for some upcoming talks he was doing ... was working > on bringing it up to support 8.3.x ... Yes. He will make a presentation about PGCluster at PGCon 2009 Japan. http://www.postgresql.jp/events/pgcon09j/e/program_2#10 Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
I'm sorry for late my response. I already sent same contents to Marcos, but I did not cc to all of yours. (I'll join pgsql-hackers ML). 1. Statement of PGCluster-II The state of PGCluster-II is re designing. I developed and demonstorated PGCluster-II as full data shared model in 2006. It worked fine. However, perfomance point of view, it was not good at all. Then, I improved it in 2007. Even so, it was absurdly slow. I think that it was required redesigning to improve performanse. The full data shread model is not difficult to build up (I developed the 1st PGCluster-II within 2 weeks), but it is not easy to improve performanse. Then, I chose partial data shared model as my conclusion. I spend two years to decide it. 2. Prospects of PGCluster-II Currentry, I'm developing the cluster server management tools as a part of the latest PGCluster. It will be upcoming release. After that, I will be back to the PGCluster-II project. On comming Nov. 19, The International PostgreSQL Cluster Developper's Meeting (I'm not sure the proper name) will be heldin Tokyo. I'll discuss about a task of PGCluster-II as a part of PostgreSQL's cluster project at the meeting. Clustering is getting more important each day. I'm expecting that PGCluster & PGCluster-II play a role in the solution. Best regards, ------------------- At.Mitani -- original message -- From: Devrim G？ND？Z<devrim@gunduz.org> To: Marc G. Fournier<scrappy@hub.org> Cc: Marcos Luis Ortiz Valmaseda<mlortiz@uci.cu>;mitani<mitani@sraw.co.jp>;pgsql-hackers<pgsql-hackers@postgresql.org> Sent: Tue, 15 Sep 2009 19:20:37 +0300 Subject: Re: [HACKERS] PGCluster-II Progress >On Tue, 2009-09-15 at 13:16 -0300, Marc G. Fournier wrote: >> Odd, I talked to him a couple of weeks ago and he was working on a >> new >> release in preparation for some upcoming talks he was doing ... was >> working on bringing it up to support 8.3.x ... >> >> "But, I'm just prepareing new version of the PGCluster..." > >Mitani was probably talking about PGCluster, not PGCluster -II. Mitani? > >Last time I heard about PGCluster-II was at Anniversary Summit(2006). >Oh, and I met with him at France at 2007, and no improvements since >then. > >Regards, >-- >Devrim G？ND？Z, RHCE >Command Prompt - http://www.CommandPrompt.com >devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr > http://www.gunduz.org >
Excellent news,Mitani. My personal intention if to help you with this work for the redesign of PGCluster-II, or on the codingof it. If we work like a team, we can release this new version of the project more quickly. Do you have any sources repository of this new version? I work fine with CVS and Subversion but if you are working with Git,is better, because is my favorite SCM. Regards, and I'll be waiting your answer. "The hurry is enemy of the success: for that reason.......Be patient" Ing. Marcos L. Ortiz Valmaseda Línea Soporte y Despliegue Centro de Tecnologías de Almacenamiento y Análisis de Datos (CENTALAD) Linux User # 418229 PostgreSQL User http://www.postgresql.org http://www.planetpostgresql.org/ http://www.postgresql-es.org/ ----- Mensaje original ----- De: mitani@sraw.co.jp Para: devrim@gunduz.org, "Marc G. Fournier" <scrappy@hub.org> CC: "pgsql-hackers" <pgsql-hackers@postgresql.org> Enviados: Jueves, 17 de Septiembre 2009 15:13:43 GMT -10:00 Hawai Asunto: Re: [HACKERS] PGCluster-II Progress I'm sorry for late my response. I already sent same contents to Marcos, but I did not cc to all of yours. (I'll join pgsql-hackers ML). 1. Statement of PGCluster-II The state of PGCluster-II is re designing. I developed and demonstorated PGCluster-II as full data shared model in 2006. It worked fine. However, perfomance point of view, it was not good at all. Then, I improved it in 2007. Even so, it was absurdly slow. I think that it was required redesigning to improve performanse. The full data shread model is not difficult to build up (I developed the 1st PGCluster-II within 2 weeks), but it is not easy to improve performanse. Then, I chose partial data shared model as my conclusion. I spend two years to decide it. 2. Prospects of PGCluster-II Currentry, I'm developing the cluster server management tools as a part of the latest PGCluster. It will be upcoming release. After that, I will be back to the PGCluster-II project. On comming Nov. 19, The International PostgreSQL Cluster Developper's Meeting (I'm not sure the proper name) will be heldin Tokyo. I'll discuss about a task of PGCluster-II as a part of PostgreSQL's cluster project at the meeting. Clustering is getting more important each day. I'm expecting that PGCluster & PGCluster-II play a role in the solution. Best regards, ------------------- At.Mitani -- original message -- From: Devrim G？ND？Z<devrim@gunduz.org> To: Marc G. Fournier<scrappy@hub.org> Cc: Marcos Luis Ortiz Valmaseda<mlortiz@uci.cu>;mitani<mitani@sraw.co.jp>;pgsql-hackers<pgsql-hackers@postgresql.org> Sent: Tue, 15 Sep 2009 19:20:37 +0300 Subject: Re: [HACKERS] PGCluster-II Progress >On Tue, 2009-09-15 at 13:16 -0300, Marc G. Fournier wrote: >> Odd, I talked to him a couple of weeks ago and he was working on a >> new >> release in preparation for some upcoming talks he was doing ... was >> working on bringing it up to support 8.3.x ... >> >> "But, I'm just prepareing new version of the PGCluster..." > >Mitani was probably talking about PGCluster, not PGCluster -II. Mitani? > >Last time I heard about PGCluster-II was at Anniversary Summit(2006). >Oh, and I met with him at France at 2007, and no improvements since >then. > >Regards, >-- >Devrim G？ND？Z, RHCE >Command Prompt - http://www.CommandPrompt.com >devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr > http://www.gunduz.org > -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Mitani-San, thank you for this heads up on PGCluster-II. The partial data idea sounds very interesting and I'm looking forward to an inspiring meeting in Tokyo. Kind Regards Markus Wanner