Thread: What tools do people use to hack on PostgreSQL?

What tools do people use to hack on PostgreSQL?

From
"Florian G. Pflug"
Date:
Hi

I'm very excited that my project for implementing read-only queries
on PITR slaves was accepted for GSoC, and I'm now trying to work
out what tools I'll use for that job.

I'd like to be able to create some sort of branches and tags for
my own work (only inside my local repository of course).

I've considered using git, but I couldn't make the cvs->git gateway
work - neither using the postgresql CVS repository directly, nor with
a private copy obtained with CVSup.

There is also svk, but I think I'd need a svn repo that mirrors
the postgresql CVS for that to work. I think Joshua Drake created
one once, but I don't now if it is kept up-to-date.

What do you guys use for your development work?

greetings, Florian Pflug



Re: What tools do people use to hack on PostgreSQL?

From
Andrew Dunstan
Date:
Florian G. Pflug wrote:
> Hi
>
> I'm very excited that my project for implementing read-only queries
> on PITR slaves was accepted for GSoC,

Congratulations.

> and I'm now trying to work
> out what tools I'll use for that job.
>
> I'd like to be able to create some sort of branches and tags for
> my own work (only inside my local repository of course).
>
> I've considered using git, but I couldn't make the cvs->git gateway
> work - neither using the postgresql CVS repository directly, nor with
> a private copy obtained with CVSup.
>
> There is also svk, but I think I'd need a svn repo that mirrors
> the postgresql CVS for that to work. I think Joshua Drake created
> one once, but I don't now if it is kept up-to-date.
>
> What do you guys use for your development work?
>

You can create a CVS mirror very easily (some info on how is below, 
copied from the buildfarm HOWTO.)

The problem will be keeping things in sync with upstream if you commit 
changes.

I'd be tempted to use some other system (svn would work fine, I think) 
to do your personal checkpointing, if that's what you want to do, so 
your checkpoint procedure would be something like:
 cvs update <resolve merge problems> svn ci

A lower tech model might just cut a nightly CVS diff and archive it.

Normally I don't bother with any of this, I just keep a separate tree 
for each development which I periodically update but don't do anything 
else until I'm ready to commit or cut a patch. It depends what you're 
comfortable with. I haven't very often wished I could roll back.

cheers

andrew

----------------------

11. Almost all the bandwidth issues disappear if you use a local CVS repository
instead of the one at postgresql.org. The way to do this (or at least the
way I did it) is using CVSup. Since building CVSup is non-trivial, the
best way to start this is to get a binary package for some system it will
run on. In my case this was a Linux system running Fedora Core/1. After a
few false starts, I got it working replicating the entire repo at 
postgresql.org, including the CVSROOT directory. Then I commented out the
entries in CVSROOT/loginfo and CVSROOT/commitinfo, and set up the LockDir
directive as I wanted it it CVSROOT/config. Then I checked out the CVSROOT
module and did that all over again, and checked the module back in. Then
to make sure CVSup didn't overwrite those files, I made entries for them
in <mirror-home>/sup/repository/refuse. With that done, I was able to
change the build config on that machine so that the config variable "cvsrepo"
was just the name of the mirror root directory. Everything worked fine.
After that I set up an anonymous cvs pserver against the mirror, so that
my other machine could also get the source from there instead of from
postgresql.org. I did a "cvs login", changed the "cvsrepo" config
variable on that machine, and it worked happily too. Finally, I set up a
cron job on the mirror machine to update the mirror. The anonymous repository 
is only updated from the master once every hour, so there is no point in 
running the cron job more often than that. This should not be too big a deal, 
as CVSup is extremely efficient, and even doing this so frequently should 
not incur a lot of bandwidth use.

12. CVSup is not universally available. For example, it does not seem to
be available any longer in Fedora Extras, and there are platforms for which 
it has never been available. However, a similar procedure to the above can be 
done with rsync, which is pretty universally available. Here is what I did.
First I made a repo location, and get an initial repo copy: mkdir -p /home/cvsmirror/pg rsync -avzCH --delete
anoncvs.postgresql.org::pgsql-cvs/home/cvsmirror/pg
 
Then remove the sup directory and set up an rsync exclude file: rm -rf /home/cvsmirror/pg/sup echo /sup/ >
/home/cvsmirror/pg-excludeecho '/CVSROOT/loginfo*' >> /home/cvsmirror/pg-exclude echo '/CVSROOT/commitinfo*' >>
/home/cvsmirror/pg-excludeecho '/CVSROOT/config*' >> /home/cvsmirror/pg-exclude
 
Then edit the CVSROOT as in step 11.
The add a job to cron something like this: 43 * * * * rsync -avzCH --delete --exclude-from=/home/cvsmirror/pg-exclude
anoncvs.postgresql.org::pgsql-cvs/home/cvsmirror/pg
 
Finally, add a pserver if other local buildfarm member machines need access.





Re: What tools do people use to hack on PostgreSQL?

From
Alexey Klyukin
Date:
Hi,

Florian G. Pflug wrote:
> Hi
> 
> I'm very excited that my project for implementing read-only queries
> on PITR slaves was accepted for GSoC, and I'm now trying to work
> out what tools I'll use for that job.
> 
> I'd like to be able to create some sort of branches and tags for
> my own work (only inside my local repository of course).
> 
> I've considered using git, but I couldn't make the cvs->git gateway
> work - neither using the postgresql CVS repository directly, nor with
> a private copy obtained with CVSup.
> 
> There is also svk, but I think I'd need a svn repo that mirrors
> the postgresql CVS for that to work. I think Joshua Drake created
> one once, but I don't now if it is kept up-to-date.

Yes, it is (the latest visible commit was made 6 hours ago), you can
browse sources at:

http://projects.commandprompt.com/public/pgsql/browser

or do the anonymous checkout with:

svn co http://projects.commandprompt.com/public/pgsql/repo/

Regards,
-- 
Alexey Klyukin                alexk@commandprompt.com


Re: What tools do people use to hack on PostgreSQL?

From
Alvaro Herrera
Date:
Alexey Klyukin wrote:
> Hi,
> 
> Florian G. Pflug wrote:
> > Hi
> > 
> > I'm very excited that my project for implementing read-only queries
> > on PITR slaves was accepted for GSoC, and I'm now trying to work
> > out what tools I'll use for that job.
> > 
> > I'd like to be able to create some sort of branches and tags for
> > my own work (only inside my local repository of course).
> > 
> > I've considered using git, but I couldn't make the cvs->git gateway
> > work - neither using the postgresql CVS repository directly, nor with
> > a private copy obtained with CVSup.
> > 
> > There is also svk, but I think I'd need a svn repo that mirrors
> > the postgresql CVS for that to work. I think Joshua Drake created
> > one once, but I don't now if it is kept up-to-date.
> 
> Yes, it is (the latest visible commit was made 6 hours ago), you can
> browse sources at:
> 
> http://projects.commandprompt.com/public/pgsql/browser
> 
> or do the anonymous checkout with:
> 
> svn co http://projects.commandprompt.com/public/pgsql/repo/

But if you have a checked out tree, does it work to do an update after
the tree has been regenerated?  As far as I know, the repo is generated
completely every few hours, so it wouldn't surprise me that the checked
out copy is not compatible with the new repo.

I admit I haven't tried.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


Re: What tools do people use to hack on PostgreSQL?

From
Alexey Klyukin
Date:
Alvaro Herrera wrote:
> But if you have a checked out tree, does it work to do an update after
> the tree has been regenerated?  As far as I know, the repo is generated
> completely every few hours, so it wouldn't surprise me that the checked
> out copy is not compatible with the new repo.
>
> I admit I haven't tried.
>   
I have tried and svn up worked without issues.

Regards,

-- 
Alexey Klyukin                alexk@commandprompt.com



Re: What tools do people use to hack on PostgreSQL?

From
"Joshua D. Drake"
Date:
Alexey Klyukin wrote:
> Alvaro Herrera wrote:
>> But if you have a checked out tree, does it work to do an update after
>> the tree has been regenerated?  As far as I know, the repo is generated
>> completely every few hours, so it wouldn't surprise me that the checked
>> out copy is not compatible with the new repo.
>>
>> I admit I haven't tried.
>>   
> I have tried and svn up worked without issues.

As a note we will be updating this to subversion 1.4 shortly so people 
can do svnsync too.

J


> 
> Regards,
> 


-- 
      === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive  PostgreSQL solutions since 1997             http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/



Re: What tools do people use to hack on PostgreSQL?

From
"Florian G. Pflug"
Date:
Joshua D. Drake wrote:
> Alexey Klyukin wrote:
>> Alvaro Herrera wrote:
>>> But if you have a checked out tree, does it work to do an update after
>>> the tree has been regenerated?  As far as I know, the repo is generated
>>> completely every few hours, so it wouldn't surprise me that the checked
>>> out copy is not compatible with the new repo.
>>>
>>> I admit I haven't tried.
>>>   
>> I have tried and svn up worked without issues.
> 
> As a note we will be updating this to subversion 1.4 shortly so people 
> can do svnsync too.

Do I read this correctly as "This repository will stay around for a while,
and isn't just an experiment that might be stopped tomorrow?". If so, I'll
try using it - and lots of thanks for providing that

greetings, Florian Pflug



Re: What tools do people use to hack on PostgreSQL?

From
"Joshua D. Drake"
Date:
Florian G. Pflug wrote:
> Joshua D. Drake wrote:
>> Alexey Klyukin wrote:
>>> Alvaro Herrera wrote:
>>>> But if you have a checked out tree, does it work to do an update after
>>>> the tree has been regenerated?  As far as I know, the repo is generated
>>>> completely every few hours, so it wouldn't surprise me that the checked
>>>> out copy is not compatible with the new repo.
>>>>
>>>> I admit I haven't tried.
>>>>   
>>> I have tried and svn up worked without issues.
>>
>> As a note we will be updating this to subversion 1.4 shortly so people 
>> can do svnsync too.
> 
> Do I read this correctly as "This repository will stay around for a while,
> and isn't just an experiment that might be stopped tomorrow?". If so, I'll
> try using it - and lots of thanks for providing that

The subversion repository has been around *a long* time. The only thing 
that is relatively new is the anonymous checkout and that is just 
because we were lazy.

So yes, you can use it :)

Joshua D. Drake


> 
> greetings, Florian Pflug
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
> 
>               http://archives.postgresql.org
> 


-- 
      === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive  PostgreSQL solutions since 1997             http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/



Re: What tools do people use to hack on PostgreSQL?

From
"Joshua D. Drake"
Date:
>> http://projects.commandprompt.com/public/pgsql/browser
>>
>> or do the anonymous checkout with:
>>
>> svn co http://projects.commandprompt.com/public/pgsql/repo/
> 
> But if you have a checked out tree, does it work to do an update after
> the tree has been regenerated?  As far as I know, the repo is generated
> completely every few hours, so it wouldn't surprise me that the checked
> out copy is not compatible with the new repo.
> 
> I admit I haven't tried.

Not a clue. Anyone try it yet?

Joshua D. Drake



-- 
      === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive  PostgreSQL solutions since 1997             http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/



Re: What tools do people use to hack on PostgreSQL?

From
"Florian G. Pflug"
Date:
Joshua D. Drake wrote:
>>> http://projects.commandprompt.com/public/pgsql/browser
>>>
>>> or do the anonymous checkout with:
>>>
>>> svn co http://projects.commandprompt.com/public/pgsql/repo/
>>
>> But if you have a checked out tree, does it work to do an update after
>> the tree has been regenerated?  As far as I know, the repo is generated
>> completely every few hours, so it wouldn't surprise me that the checked
>> out copy is not compatible with the new repo.
>>
>> I admit I haven't tried.
> 
> Not a clue. Anyone try it yet?

git-svn seems to work fine against the SVN repo, apart from the problem with
the files deleted in CVS which still show up in SVN.

It's only running for about two days though...

greetings, Florian Pflug