Thread: Postgres-R source code release
Dear Hackers, it has been two years, since I've presented my work on Postgres-R for the first time to a broader audience in Toronto. I continued maintaining that code in my spare time, but to be honest, I didn't have much time for it. As much as I'd like to change that, I now think the best thing for the project itself is to open it up and release the source code. I'd very much like to get others aboard and turn Postgres-R into a real community project again. Of course, the recent core statement about wanting an integrated replication solution also influenced my decision, even though I realize that Postgres-R is way more ambitious path than a log shipping solution. As a first step towards a community project, I've cleaned up the code and tried to comment and document it as good as I can. You are welcome to download the latest patch from http://www.postgres-r.org/downloads/. It expects to be applied against today's CVS HEAD. You'll find a README and a pretty exhaustive TODO file under src/backend/replication/. I plan to write a series of more technical mails following this one, to start the discussion about the internals of Postgres-R and decisions I made. That should give some starting points for those interested it the project. In case you are wondering: I've got married and I'm glad to tell you that I changed my very German sounding name Schiltknecht to a more pronounceable one: my new surname is Wanner. Looking forward to inspiring discussions with fellow hackers. Regards Markus Wanner configure.in | 9 src/Makefile.global.in | 1 src/backend/Makefile | 4 src/backend/access/transam/varsup.c | 23 src/backend/access/transam/xact.c | 115 + src/backend/bootstrap/bootstrap.c | 3 src/backend/catalog/Makefile | 4 src/backend/catalog/catalog.c | 8 src/backend/catalog/index.c | 10 src/backend/commands/Makefile | 4 src/backend/commands/aggregatecmds.c | 25 src/backend/commands/comment.c | 25 src/backend/commands/conversioncmds.c | 25 src/backend/commands/dbcommands.c | 220 ++- src/backend/commands/functioncmds.c | 64 src/backend/commands/indexcmds.c | 9 src/backend/commands/lockcmds.c | 10 src/backend/commands/opclasscmds.c | 75 + src/backend/commands/operatorcmds.c | 24 src/backend/commands/prepare.c | 20 src/backend/commands/proclang.c | 32 src/backend/commands/replicacmds.c | 188 ++ src/backend/commands/schemacmds.c | 29 src/backend/commands/sequence.c | 193 ++ src/backend/commands/tablecmds.c | 44 src/backend/commands/trigger.c | 30 src/backend/commands/tsearchcmds.c | 136 + src/backend/commands/typecmds.c | 69 src/backend/commands/variable.c | 68 src/backend/executor/execMain.c | 651 +++++++++ src/backend/executor/execUtils.c | 9 src/backend/nodes/copyfuncs.c | 14 src/backend/nodes/equalfuncs.c | 12 src/backend/parser/gram.y | 92 + src/backend/parser/keywords.c | 6 src/backend/postmaster/postmaster.c | 349 ++++ src/backend/replication/HISTORY | 33 src/backend/replication/Makefile | 17 src/backend/replication/README | 93 + src/backend/replication/TODO | 161 ++ src/backend/replication/cset.c | 280 +++ src/backend/replication/gc_egcs.c | 580 ++++++++ src/backend/replication/gc_ensemble.c | 823 +++++++++++ src/backend/replication/gc_spread.c | 432 ++++++ src/backend/replication/gc_utils.c | 93 + src/backend/replication/local.c | 878 ++++++++++++ src/backend/replication/manager.c | 1859 ++++++++++++++++++++++++++ src/backend/replication/recovery.c | 1375 +++++++++++++++++++ src/backend/replication/remote.c | 463 ++++++ src/backend/replication/utils.c | 558 +++++++ src/backend/storage/ipc/Makefile | 4 src/backend/storage/ipc/buffer.c | 308 ++++ src/backend/storage/ipc/imsg.c | 369 +++++ src/backend/storage/ipc/ipci.c | 12 src/backend/storage/ipc/procarray.c | 106 + src/backend/storage/lmgr/lock.c | 2 src/backend/storage/lmgr/proc.c | 70 src/backend/tcop/postgres.c | 421 +++++ src/backend/tcop/utility.c | 6 src/backend/utils/cache/syscache.c | 1 src/backend/utils/init/globals.c | 24 src/backend/utils/init/miscinit.c | 6 src/backend/utils/init/postinit.c | 4 src/backend/utils/misc/check_guc | 4 src/backend/utils/misc/guc.c | 70 src/backend/utils/misc/postgresql.conf.sample | 11 src/include/Makefile | 4 src/include/access/transam.h | 4 src/include/access/xact.h | 10 src/include/catalog/indexing.h | 5 src/include/catalog/pg_database.h | 12 src/include/catalog/pg_replication_gcs.h | 60 src/include/commands/dbcommands.h | 2 src/include/commands/replicacmds.h | 22 src/include/commands/sequence.h | 16 src/include/commands/variable.h | 3 src/include/executor/executor.h | 7 src/include/miscadmin.h | 4 src/include/nodes/execnodes.h | 8 src/include/nodes/nodes.h | 1 src/include/nodes/parsenodes.h | 12 src/include/pg_config.h.in | 3 src/include/replication/cset.h | 121 + src/include/replication/gc.h | 215 +++ src/include/replication/jobcache.h | 30 src/include/replication/manager.h | 113 + src/include/replication/messages.h | 51 src/include/replication/recovery.h | 67 src/include/replication/replication.h | 132 + src/include/replication/utils.h | 71 src/include/storage/buffer.h | 63 src/include/storage/imsg.h | 83 + src/include/storage/proc.h | 20 src/include/storage/procarray.h | 8 src/include/utils/guc_tables.h | 4 src/interfaces/ecpg/preproc/preproc.y | 16 src/test/regress/expected/sanity_check.out | 3 src/test/regress/expected/sequence.out | 12 98 files changed, 12777 insertions(+), 68 deletions(-)
On Mon, 2008-07-14 at 17:42 +0200, Markus Wanner wrote: > As much as I'd like to change that, I now think the best thing for the > project itself is to open it up and release the source code. I'd very > much like to get others aboard and turn Postgres-R into a real > community project again. Of course, the recent core statement about > wanting an integrated replication solution also influenced my > decision, even though I realize that Postgres-R is way more ambitious > path than a log shipping solution. > > As a first step towards a community project, I've cleaned up the code > and tried to comment and document it as good as I can. You are welcome > to download the latest patch from > http://www.postgres-r.org/downloads/. That's good news, really. > In case you are wondering: I've got married and I'm glad to tell you > that I changed my very German sounding name Schiltknecht to a more > pronounceable one: my new surname is Wanner. Congrats :) -- Devrim GÜNDÜZ devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org
On Mon, Jul 14, 2008 at 11:42 AM, Markus Wanner <markus.wanner@programmfabrik.de> wrote: > As a first step towards a community project, I've cleaned up the code and > tried to comment and document it as good as I can. You are welcome to > download the latest patch from http://www.postgres-r.org/downloads/. It > expects to be applied against today's CVS HEAD. You'll find a README and a > pretty exhaustive TODO file under src/backend/replication/. Awesome! > In case you are wondering: I've got married Congratulations! -- Jonah H. Harris, Sr. Software Architect | phone: 732.331.1324 EnterpriseDB Corporation | fax: 732.331.1301 499 Thornall Street, 2nd Floor | jonah.harris@enterprisedb.com Edison, NJ 08837 | http://www.enterprisedb.com/
On Mon, Jul 14, 2008 at 05:42:21PM +0200, Markus Wanner wrote: > Dear Hackers, > > it has been two years, since I've presented my work on Postgres-R for > the first time to a broader audience in Toronto. I continued maintaining > that code in my spare time, but to be honest, I didn't have much time > for it. > > As much as I'd like to change that, I now think the best thing for the > project itself is to open it up and release the source code. I'd very > much like to get others aboard and turn Postgres-R into a real community > project again. Of course, the recent core statement about wanting an > integrated replication solution also influenced my decision, even though > I realize that Postgres-R is way more ambitious path than a log shipping > solution. > > As a first step towards a community project, I've cleaned up the code > and tried to comment and document it as good as I can. You are welcome > to download the latest patch from http://www.postgres-r.org/downloads/. > It expects to be applied against today's CVS HEAD. You'll find a README > and a pretty exhaustive TODO file under src/backend/replication/. Would you mind if I were to make a git branch for it on <http://git.postgresql.org/> ? > In case you are wondering: I've got married Congratulations! Cheers, David. -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
David Fetter wrote: > On Mon, Jul 14, 2008 at 05:42:21PM +0200, Markus Wanner wrote: > Would you mind if I were to make a git branch for it on > <http://git.postgresql.org/> ? That's very likely wasted effort, since obviously Markus has got a Monotone tree somewhere ... -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
On Mon, Jul 14, 2008 at 05:35:28PM -0400, Alvaro Herrera wrote: > David Fetter wrote: > > On Mon, Jul 14, 2008 at 05:42:21PM +0200, Markus Wanner wrote: > > > Would you mind if I were to make a git branch for it on > > <http://git.postgresql.org/> ? > > That's very likely wasted effort, since obviously Markus has got a > Monotone tree somewhere ... I'm curious as to your motive behind continual deriding any effort to get git going on branches. Do you have some other alternative in mind? Cheers, David. -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
David Fetter wrote: > On Mon, Jul 14, 2008 at 05:35:28PM -0400, Alvaro Herrera wrote: > > David Fetter wrote: > > > On Mon, Jul 14, 2008 at 05:42:21PM +0200, Markus Wanner wrote: > > > > > Would you mind if I were to make a git branch for it on > > > <http://git.postgresql.org/> ? > > > > That's very likely wasted effort, since obviously Markus has got a > > Monotone tree somewhere ... > > I'm curious as to your motive behind continual deriding any effort to > get git going on branches. Do you have some other alternative in mind? That's not my intention. It's just that the two cases that have popped up are not useful. The with-recursive one because upstream does not use it. If that can be solved, then I have no objection and I think it would be a good thing. This one because it would be messy. I think the way to go here is to have Markus open up his Monotone repo, or convince him to migrate it to Git, but I really doubt that's ever going to happen. -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
On Mon, 2008-07-14 at 14:41 -0700, David Fetter wrote: > On Mon, Jul 14, 2008 at 05:35:28PM -0400, Alvaro Herrera wrote: > > David Fetter wrote: > > > On Mon, Jul 14, 2008 at 05:42:21PM +0200, Markus Wanner wrote: > > > > > Would you mind if I were to make a git branch for it on > > > <http://git.postgresql.org/> ? > > > > That's very likely wasted effort, since obviously Markus has got a > > Monotone tree somewhere ... > > I'm curious as to your motive behind continual deriding any effort to > get git going on branches. Do you have some other alternative in mind? Obviously monotone, but I think the point is, Git gives them nothing Monotone doesn't. Of course, I am personally happy with SVN but hey :P Joshua D. Drake -- The PostgreSQL Company since 1997: http://www.commandprompt.com/ PostgreSQL Community Conference: http://www.postgresqlconference.org/ United States PostgreSQL Association: http://www.postgresql.us/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
On Mon, Jul 14, 2008 at 04:22:47PM -0700, Joshua D. Drake wrote: > On Mon, 2008-07-14 at 14:41 -0700, David Fetter wrote: > > On Mon, Jul 14, 2008 at 05:35:28PM -0400, Alvaro Herrera wrote: > > > David Fetter wrote: > > > > On Mon, Jul 14, 2008 at 05:42:21PM +0200, Markus Wanner wrote: > > > > > > > Would you mind if I were to make a git branch for it on > > > > <http://git.postgresql.org/> ? > > > > > > That's very likely wasted effort, since obviously Markus has got > > > a Monotone tree somewhere ... > > > > I'm curious as to your motive behind continual deriding any effort > > to get git going on branches. Do you have some other alternative > > in mind? > > Obviously monotone, but I think the point is, Git gives them nothing > Monotone doesn't. Apart from being able to interoperate with CVS, you mean? ;) > Of course, I am personally happy with SVN but hey :P You can't have tried a merge in SVN if that's so :P :P Cheers, David. -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
On Jul 14, 2008, at 16:54, David Fetter wrote: >> Of course, I am personally happy with SVN but hey :P > > You can't have tried a merge in SVN if that's so :P :P Those of us who have been doing it for years, in CVS and in SVN, aren't too worried about it. Best, David
On 15/07/2008, David E. Wheeler <david@kineticode.com> wrote: > > > Of course, I am personally happy with SVN but hey :P > > > > You can't have tried a merge in SVN if that's so :P :P > > Those of us who have been doing it for years, in CVS and in SVN, aren't too > worried about it. > Follow the sandal! :D -- Please don't top post, and don't use HTML e-Mail :} Make your quotes concise. http://www.american.edu/econ/notes/htmlmail.htm
Hi, Alvaro Herrera wrote: > I think the > way to go here is to have Markus open up his Monotone repo, or convince > him to migrate it to Git, but I really doubt that's ever going to > happen. He he... good guess ;-) However, as much as I personally like monotone and as much as I dislike git for being a bad rip off of monotone, git certainly has its merits. It has a much larger users base and is faster than monotone for some operations. Having used subversion, CVS and monotone for Postgres-R, I think I'm now giving git a try. I'm about to setup the necessary repositories, but I'm still having a hard time grokking this tool set. (And yes, I'm missing some features compared to monotone. In our case, the most obvious one is 'mtn diff --context' ;-) ) Regards Markus
Hi, David Fetter wrote: > Would you mind if I were to make a git branch for it on > <http://git.postgresql.org/> ? I've set up a git-daemon with the Postgres-R patch here: git://postgres-r.org/repo Since it's a distributed VCS, you should be able to mirror that to git.postgtresql.org somehow (if you figure out how, please tell me!). Please note that I'm still struggling with git and I cannot promise to keep using it. Regards Markus
On Wed, Jul 16, 2008 at 09:35:28PM +0200, Markus Schiltknecht wrote: > Hi, > > David Fetter wrote: >> Would you mind if I were to make a git branch for it on >> <http://git.postgresql.org/> ? > > I've set up a git-daemon with the Postgres-R patch here: > > git://postgres-r.org/repo > > Since it's a distributed VCS, you should be able to mirror that to > git.postgtresql.org somehow (if you figure out how, please tell me!). I've merged the latest Postgres in. Care to see whether it runs? http://git.postgresql.org/?p=~davidfetter/pgr/.git;a=summary > Please note that I'm still struggling with git and I cannot promise > to keep using it. I'm struggling, too, but the cheapness of experimenting is making it easier and easier :) Cheers, David. -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate