Thread: What tools do people use to hack on PostgreSQL?
Hi I'm very excited that my project for implementing read-only queries on PITR slaves was accepted for GSoC, and I'm now trying to work out what tools I'll use for that job. I'd like to be able to create some sort of branches and tags for my own work (only inside my local repository of course). I've considered using git, but I couldn't make the cvs->git gateway work - neither using the postgresql CVS repository directly, nor with a private copy obtained with CVSup. There is also svk, but I think I'd need a svn repo that mirrors the postgresql CVS for that to work. I think Joshua Drake created one once, but I don't now if it is kept up-to-date. What do you guys use for your development work? greetings, Florian Pflug
Florian G. Pflug wrote: > Hi > > I'm very excited that my project for implementing read-only queries > on PITR slaves was accepted for GSoC, Congratulations. > and I'm now trying to work > out what tools I'll use for that job. > > I'd like to be able to create some sort of branches and tags for > my own work (only inside my local repository of course). > > I've considered using git, but I couldn't make the cvs->git gateway > work - neither using the postgresql CVS repository directly, nor with > a private copy obtained with CVSup. > > There is also svk, but I think I'd need a svn repo that mirrors > the postgresql CVS for that to work. I think Joshua Drake created > one once, but I don't now if it is kept up-to-date. > > What do you guys use for your development work? > You can create a CVS mirror very easily (some info on how is below, copied from the buildfarm HOWTO.) The problem will be keeping things in sync with upstream if you commit changes. I'd be tempted to use some other system (svn would work fine, I think) to do your personal checkpointing, if that's what you want to do, so your checkpoint procedure would be something like: cvs update <resolve merge problems> svn ci A lower tech model might just cut a nightly CVS diff and archive it. Normally I don't bother with any of this, I just keep a separate tree for each development which I periodically update but don't do anything else until I'm ready to commit or cut a patch. It depends what you're comfortable with. I haven't very often wished I could roll back. cheers andrew ---------------------- 11. Almost all the bandwidth issues disappear if you use a local CVS repository instead of the one at postgresql.org. The way to do this (or at least the way I did it) is using CVSup. Since building CVSup is non-trivial, the best way to start this is to get a binary package for some system it will run on. In my case this was a Linux system running Fedora Core/1. After a few false starts, I got it working replicating the entire repo at postgresql.org, including the CVSROOT directory. Then I commented out the entries in CVSROOT/loginfo and CVSROOT/commitinfo, and set up the LockDir directive as I wanted it it CVSROOT/config. Then I checked out the CVSROOT module and did that all over again, and checked the module back in. Then to make sure CVSup didn't overwrite those files, I made entries for them in <mirror-home>/sup/repository/refuse. With that done, I was able to change the build config on that machine so that the config variable "cvsrepo" was just the name of the mirror root directory. Everything worked fine. After that I set up an anonymous cvs pserver against the mirror, so that my other machine could also get the source from there instead of from postgresql.org. I did a "cvs login", changed the "cvsrepo" config variable on that machine, and it worked happily too. Finally, I set up a cron job on the mirror machine to update the mirror. The anonymous repository is only updated from the master once every hour, so there is no point in running the cron job more often than that. This should not be too big a deal, as CVSup is extremely efficient, and even doing this so frequently should not incur a lot of bandwidth use. 12. CVSup is not universally available. For example, it does not seem to be available any longer in Fedora Extras, and there are platforms for which it has never been available. However, a similar procedure to the above can be done with rsync, which is pretty universally available. Here is what I did. First I made a repo location, and get an initial repo copy: mkdir -p /home/cvsmirror/pg rsync -avzCH --delete anoncvs.postgresql.org::pgsql-cvs/home/cvsmirror/pg Then remove the sup directory and set up an rsync exclude file: rm -rf /home/cvsmirror/pg/sup echo /sup/ > /home/cvsmirror/pg-excludeecho '/CVSROOT/loginfo*' >> /home/cvsmirror/pg-exclude echo '/CVSROOT/commitinfo*' >> /home/cvsmirror/pg-excludeecho '/CVSROOT/config*' >> /home/cvsmirror/pg-exclude Then edit the CVSROOT as in step 11. The add a job to cron something like this: 43 * * * * rsync -avzCH --delete --exclude-from=/home/cvsmirror/pg-exclude anoncvs.postgresql.org::pgsql-cvs/home/cvsmirror/pg Finally, add a pserver if other local buildfarm member machines need access.
Hi, Florian G. Pflug wrote: > Hi > > I'm very excited that my project for implementing read-only queries > on PITR slaves was accepted for GSoC, and I'm now trying to work > out what tools I'll use for that job. > > I'd like to be able to create some sort of branches and tags for > my own work (only inside my local repository of course). > > I've considered using git, but I couldn't make the cvs->git gateway > work - neither using the postgresql CVS repository directly, nor with > a private copy obtained with CVSup. > > There is also svk, but I think I'd need a svn repo that mirrors > the postgresql CVS for that to work. I think Joshua Drake created > one once, but I don't now if it is kept up-to-date. Yes, it is (the latest visible commit was made 6 hours ago), you can browse sources at: http://projects.commandprompt.com/public/pgsql/browser or do the anonymous checkout with: svn co http://projects.commandprompt.com/public/pgsql/repo/ Regards, -- Alexey Klyukin alexk@commandprompt.com
Alexey Klyukin wrote: > Hi, > > Florian G. Pflug wrote: > > Hi > > > > I'm very excited that my project for implementing read-only queries > > on PITR slaves was accepted for GSoC, and I'm now trying to work > > out what tools I'll use for that job. > > > > I'd like to be able to create some sort of branches and tags for > > my own work (only inside my local repository of course). > > > > I've considered using git, but I couldn't make the cvs->git gateway > > work - neither using the postgresql CVS repository directly, nor with > > a private copy obtained with CVSup. > > > > There is also svk, but I think I'd need a svn repo that mirrors > > the postgresql CVS for that to work. I think Joshua Drake created > > one once, but I don't now if it is kept up-to-date. > > Yes, it is (the latest visible commit was made 6 hours ago), you can > browse sources at: > > http://projects.commandprompt.com/public/pgsql/browser > > or do the anonymous checkout with: > > svn co http://projects.commandprompt.com/public/pgsql/repo/ But if you have a checked out tree, does it work to do an update after the tree has been regenerated? As far as I know, the repo is generated completely every few hours, so it wouldn't surprise me that the checked out copy is not compatible with the new repo. I admit I haven't tried. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Alvaro Herrera wrote: > But if you have a checked out tree, does it work to do an update after > the tree has been regenerated? As far as I know, the repo is generated > completely every few hours, so it wouldn't surprise me that the checked > out copy is not compatible with the new repo. > > I admit I haven't tried. > I have tried and svn up worked without issues. Regards, -- Alexey Klyukin alexk@commandprompt.com
Alexey Klyukin wrote:
> Alvaro Herrera wrote:
>> But if you have a checked out tree, does it work to do an update after
>> the tree has been regenerated? As far as I know, the repo is generated
>> completely every few hours, so it wouldn't surprise me that the checked
>> out copy is not compatible with the new repo.
>>
>> I admit I haven't tried.
>>
> I have tried and svn up worked without issues.
As a note we will be updating this to subversion 1.4 shortly so people
can do svnsync too.
J
>
> Regards,
>
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/
Joshua D. Drake wrote: > Alexey Klyukin wrote: >> Alvaro Herrera wrote: >>> But if you have a checked out tree, does it work to do an update after >>> the tree has been regenerated? As far as I know, the repo is generated >>> completely every few hours, so it wouldn't surprise me that the checked >>> out copy is not compatible with the new repo. >>> >>> I admit I haven't tried. >>> >> I have tried and svn up worked without issues. > > As a note we will be updating this to subversion 1.4 shortly so people > can do svnsync too. Do I read this correctly as "This repository will stay around for a while, and isn't just an experiment that might be stopped tomorrow?". If so, I'll try using it - and lots of thanks for providing that greetings, Florian Pflug
Florian G. Pflug wrote: > Joshua D. Drake wrote: >> Alexey Klyukin wrote: >>> Alvaro Herrera wrote: >>>> But if you have a checked out tree, does it work to do an update after >>>> the tree has been regenerated? As far as I know, the repo is generated >>>> completely every few hours, so it wouldn't surprise me that the checked >>>> out copy is not compatible with the new repo. >>>> >>>> I admit I haven't tried. >>>> >>> I have tried and svn up worked without issues. >> >> As a note we will be updating this to subversion 1.4 shortly so people >> can do svnsync too. > > Do I read this correctly as "This repository will stay around for a while, > and isn't just an experiment that might be stopped tomorrow?". If so, I'll > try using it - and lots of thanks for providing that The subversion repository has been around *a long* time. The only thing that is relatively new is the anonymous checkout and that is just because we were lazy. So yes, you can use it :) Joshua D. Drake > > greetings, Florian Pflug > > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Have you searched our list archives? > > http://archives.postgresql.org > -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/
>> http://projects.commandprompt.com/public/pgsql/browser >> >> or do the anonymous checkout with: >> >> svn co http://projects.commandprompt.com/public/pgsql/repo/ > > But if you have a checked out tree, does it work to do an update after > the tree has been regenerated? As far as I know, the repo is generated > completely every few hours, so it wouldn't surprise me that the checked > out copy is not compatible with the new repo. > > I admit I haven't tried. Not a clue. Anyone try it yet? Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/
Joshua D. Drake wrote: >>> http://projects.commandprompt.com/public/pgsql/browser >>> >>> or do the anonymous checkout with: >>> >>> svn co http://projects.commandprompt.com/public/pgsql/repo/ >> >> But if you have a checked out tree, does it work to do an update after >> the tree has been regenerated? As far as I know, the repo is generated >> completely every few hours, so it wouldn't surprise me that the checked >> out copy is not compatible with the new repo. >> >> I admit I haven't tried. > > Not a clue. Anyone try it yet? git-svn seems to work fine against the SVN repo, apart from the problem with the files deleted in CVS which still show up in SVN. It's only running for about two days though... greetings, Florian Pflug