Thread: [SUGGESTION] CVSync

[SUGGESTION] CVSync

From
"Joel Miller"
Date:
Hello,

I've been trying to get a local mirror of the cvs repository, but my
connection attempts using rsync to cvsup.postgresql.org are always
refused when I try to actually retrieve the pgsql-cvs collection. I tried
to use rsync because cvsup is simply not an option for me.

I'd like to suggest that CVSync (www.cvsync.org) be added as a means for
the public to obtain a local mirror of the cvs repository. It builds on
most every platform (it's written in C, no modula-3 required) and has the
advantage of caching directory scan info to reduce server load -- The
cached scan info is sent to the client, which compares it to its own
cached scan info to decide what to download (You can set up a cron job to
periodically scan the repository directories, or even set up a
sophisticated system that would only scan after commits are made.). It
also sends only the new diffs (optionally compressed) from the ",v" files
that are needed to bring the clients' repository mirror up to date.

Thanks!
Joel



Re: [SUGGESTION] CVSync

From
"Larry Rosenman"
Date:
Joel Miller wrote:
> Hello,
> 
> I've been trying to get a local mirror of the cvs repository, but my
> connection attempts using rsync to cvsup.postgresql.org are always
> refused when I try to actually retrieve the pgsql-cvs collection. I
> tried to use rsync because cvsup is simply not an option for me.
> 
> I'd like to suggest that CVSync (www.cvsync.org) be added as a means
> for the public to obtain a local mirror of the cvs repository. It
> builds on most every platform (it's written in C, no modula-3
> required) and has the advantage of caching directory scan info to
> reduce server load -- The cached scan info is sent to the client,
> which compares it to its own cached scan info to decide what to
> download (You can set up a cron job to periodically scan the
> repository directories, or even set up a sophisticated system that
> would only scan after commits are made.). It also sends only the new
> diffs (optionally compressed) from the ",v" files that are needed to
> bring the clients' repository mirror up to date. 
> 
As a complete out of nowhere suggestion, you might also look 
at csup in FreeBSD's CVS (or the project page), which is a cvsup clone
in C.

LER

-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 512-248-2683                 E-Mail: ler@lerctr.org
US Mail: 430 Valona Loop, Round Rock, TX 78681-3893



Re: [SUGGESTION] CVSync

From
Martijn van Oosterhout
Date:
On Thu, Mar 23, 2006 at 01:32:34PM -0800, Joel Miller wrote:
> Hello,
>
> I've been trying to get a local mirror of the cvs repository, but my
> connection attempts using rsync to cvsup.postgresql.org are always
> refused when I try to actually retrieve the pgsql-cvs collection. I tried
> to use rsync because cvsup is simply not an option for me.

Any particular reason why straight CVS doesn't work for you? Are you
that interested in having the log comment locally?

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

Re: [SUGGESTION] CVSync

From
Tom Lane
Date:
Martijn van Oosterhout <kleptog@svana.org> writes:
> Any particular reason why straight CVS doesn't work for you? Are you
> that interested in having the log comment locally?

Personally, I'd really like to have a local repository copy, because
I spend a *lot* of time with cvsweb etc --- but I'm sure my needs are
several standard deviations away from the mean.  So far I've been
discouraged from setting up a repository by the unreasonable
infrastructure needs of cvsup.  So these alternatives do sound pretty
interesting.

Is csup protocol-compatible with cvsup?  If so people could use it
without Marc having to do anything.  Has anyone got experience with
it --- reliability, performance, etc?
        regards, tom lane


Re: [SUGGESTION] CVSync

From
Andrew Dunstan
Date:
Tom Lane wrote:

>Martijn van Oosterhout <kleptog@svana.org> writes:
>  
>
>>Any particular reason why straight CVS doesn't work for you? Are you
>>that interested in having the log comment locally?
>>    
>>
>
>Personally, I'd really like to have a local repository copy, because
>I spend a *lot* of time with cvsweb etc --- but I'm sure my needs are
>several standard deviations away from the mean.  So far I've been
>discouraged from setting up a repository by the unreasonable
>infrastructure needs of cvsup.  So these alternatives do sound pretty
>interesting.
>
>Is csup protocol-compatible with cvsup?  If so people could use it
>without Marc having to do anything.  Has anyone got experience with
>it --- reliability, performance, etc?
>
>    
>  
>

Tom,

I don't know what unreasonable infrastructure you are referring to. 
Building cvsup is a major pain, but installing and running it isn't, in 
my experience. There's a package in Fedora Extras. Setting up cvsweb 
against my cvsup repo is a fine idea - I wonder why I didn't think of that.

That's not to say that supporting cvsync isn't a good idea too. 
TIMTOWTDI as we perl people like to say.

cheers

andrew



Re: [SUGGESTION] CVSync

From
"Joel Miller"
Date:
It's only partially compatible: Like I mentioned earlier, csup currently
only acts as an alternative cvs checkout client.

Joel


On Thu, Mar 23, 2006 at 3:15 PM, Tom Lane (tgl@sss.pgh.pa.us) wrote:

>Is csup protocol-compatible with cvsup?  If so people could use it
>without Marc having to do anything.  Has anyone got experience with
>it --- reliability, performance, etc?
>





Re: [SUGGESTION] CVSync

From
"Joel Miller"
Date:
Plain old cvs doesn't work for me because I need to get the repository,
not just a checkout of the sources, so I can do diffs, browse old
versions and what-not. I find that it's easier to understand code that's
been around for a while if I can observe how it's changed over time.

Usually, when I have time to these things, it also happens that I am
offline without access to a net connection.

Thanks!
Joel


On Thu, Mar 23, 2006 at 2:50 PM, Martijn van Oosterhout
(kleptog@svana.org) wrote:

>
>Any particular reason why straight CVS doesn't work for you? Are you
>that interested in having the log comment locally?
>




Re: [SUGGESTION] CVSync

From
"Joel Miller"
Date:
Yes, that's one way to solve the problem. I was just trying to suggest
something that would benefit everyone in general while not opening up a
can of worms with regard to increased server load in the long term.

Joel


On Fri, Mar 24, 2006 at 4:34 PM, Kurt Roeckx (kurt@roeckx.be) wrote:

>On Thu, Mar 23, 2006 at 01:32:34PM -0800, Joel Miller wrote:
>> Hello,
>> 
>> I've been trying to get a local mirror of the cvs repository, but my
>> connection attempts using rsync to cvsup.postgresql.org are always
>> refused when I try to actually retrieve the pgsql-cvs collection. I tried
>> to use rsync because cvsup is simply not an option for me.
>
>Which seems to be a permission problem to me.  Maybe the
>permissions should just get changed so that you can use it using
>rsync?
>
>I also find it very handy to have a local mirror of the
>repository.
>
>
>Kurt
>




Re: [SUGGESTION] CVSync

From
Kurt Roeckx
Date:
On Thu, Mar 23, 2006 at 01:32:34PM -0800, Joel Miller wrote:
> Hello,
> 
> I've been trying to get a local mirror of the cvs repository, but my
> connection attempts using rsync to cvsup.postgresql.org are always
> refused when I try to actually retrieve the pgsql-cvs collection. I tried
> to use rsync because cvsup is simply not an option for me.

Which seems to be a permission problem to me.  Maybe the
permissions should just get changed so that you can use it using
rsync?

I also find it very handy to have a local mirror of the
repository.


Kurt



Re: [SUGGESTION] CVSync

From
"Joel Miller"
Date:
[I thought this had gone out to the list when I first sent it, but now I
see that it didn't]

Actually, I've already looked at it very closely (spent a whole day
browsing the latest source code, in fact). The problem is csup still
doesn't support mirroring of the repository ",v" files. It only acts as
an alternative cvs checkout client. It doesn't handle ",v" files at all.

The main developer of csup is looking for help in implementing support
for dealing with the RCS ",v" files but so far it looks like no one is
offering. And I don't think anyone is going to, since cvsync is already
up and running, and so easy to set up and use.

Joel


On Thu, Mar 23, 2006 at 1:59 PM, Larry Rosenman (ler@lerctr.org) wrote:

>Joel Miller wrote:
>> Hello,
>> 
>> I've been trying to get a local mirror of the cvs repository, but my
>> connection attempts using rsync to cvsup.postgresql.org are always
>> refused when I try to actually retrieve the pgsql-cvs collection. I
>> tried to use rsync because cvsup is simply not an option for me.
>> 
>> I'd like to suggest that CVSync (www.cvsync.org) be added as a means
>> for the public to obtain a local mirror of the cvs repository. It
>> builds on most every platform (it's written in C, no modula-3
>> required) and has the advantage of caching directory scan info to
>> reduce server load -- The cached scan info is sent to the client,
>> which compares it to its own cached scan info to decide what to
>> download (You can set up a cron job to periodically scan the
>> repository directories, or even set up a sophisticated system that
>> would only scan after commits are made.). It also sends only the new
>> diffs (optionally compressed) from the ",v" files that are needed to
>> bring the clients' repository mirror up to date. 
>> 
>As a complete out of nowhere suggestion, you might also look 
>at csup in FreeBSD's CVS (or the project page), which is a cvsup clone
>in C.
>
>LER
>
>-- 
>Larry Rosenman                     http://www.lerctr.org/~ler
>Phone: +1 512-248-2683                 E-Mail: ler@lerctr.org
>US Mail: 430 Valona Loop, Round Rock, TX 78681-3893
>





Re: [SUGGESTION] CVSync

From
Neil Conway
Date:
On Thu, 2006-03-23 at 18:15 -0500, Tom Lane wrote:
> Personally, I'd really like to have a local repository copy, because
> I spend a *lot* of time with cvsweb etc --- but I'm sure my needs are
> several standard deviations away from the mean.

I'm actually amazed that anyone does any serious amount of work on
Postgres *without* using cvsup :)

I agree that investigating alternatives would be a good idea: AFAIK
there's no easy way to build cvsup on Linux/AMD64 (without patches and
more pain than I'm willing to endure), so I use cvsup on one machine and
then periodically rsync a copy of that to my main (AMD64) machine.

-Neil




Re: [SUGGESTION] CVSync

From
Tom Lane
Date:
Neil Conway <neilc@samurai.com> writes:
> I agree that investigating alternatives would be a good idea: AFAIK
> there's no easy way to build cvsup on Linux/AMD64 (without patches and
> more pain than I'm willing to endure), so I use cvsup on one machine and
> then periodically rsync a copy of that to my main (AMD64) machine.

Actually, the last time I looked seriously at cvsup, my requirement was
that it build on HPUX, which it did not (well, maybe it would've with
sufficient investment of pain, but that wasn't happening).  I was about
to take another look at whether I couldn't just-install-it on FC4, but
my main FC4 machine is now x86_64, so your remarks are discouraging me
again :-(

This whole discussion reminds me why we've stuck so fervently to
bog-standard ANSI C for Postgres.  There is a payoff for taking
portability seriously.  Too bad the original authors of cvsup were more
interested in using a flavor-of-the-month programming language...
        regards, tom lane


Re: [SUGGESTION] CVSync

From
"Joshua D. Drake"
Date:
> This whole discussion reminds me why we've stuck so fervently to
> bog-standard ANSI C for Postgres.  There is a payoff for taking
> portability seriously.  Too bad the original authors of cvsup were more
> interested in using a flavor-of-the-month programming language...

Is there any reason why we don't allow rsync access to the cvs repo?



> 
>             regards, tom lane
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
>        subscribe-nomail command to majordomo@postgresql.org so that your
>        message can get through to the mailing list cleanly
> 


-- 
            === The PostgreSQL Company: Command Prompt, Inc. ===      Sales/Support: +1.503.667.4564 || 24x7/Emergency:
+1.800.492.2240     Providing the most comprehensive  PostgreSQL solutions since 1997
http://www.commandprompt.com/




Re: [SUGGESTION] CVSync

From
Joe Conway
Date:
Tom Lane wrote:
> Neil Conway <neilc@samurai.com> writes:
> 
>>I agree that investigating alternatives would be a good idea: AFAIK
>>there's no easy way to build cvsup on Linux/AMD64 (without patches and
>>more pain than I'm willing to endure), so I use cvsup on one machine and
>>then periodically rsync a copy of that to my main (AMD64) machine.
> 
> 
> Actually, the last time I looked seriously at cvsup, my requirement was
> that it build on HPUX, which it did not (well, maybe it would've with
> sufficient investment of pain, but that wasn't happening).  I was about
> to take another look at whether I couldn't just-install-it on FC4, but
> my main FC4 machine is now x86_64, so your remarks are discouraging me
> again :-(

FWIW I use the i386 FC4 RPMs on my x86_64 FC4 machine -- works fine.

Joe




Re: [SUGGESTION] CVSync

From
Tom Lane
Date:
"Joshua D. Drake" <jd@commandprompt.com> writes:
> Is there any reason why we don't allow rsync access to the cvs repo?

That'd be Marc's bailiwick not mine ... but rsync from the anoncvs
mirror seems like it couldn't pose any serious security threat.
Marc?
        regards, tom lane


Re: [SUGGESTION] CVSync

From
"Joshua D. Drake"
Date:
Tom Lane wrote:
> "Joshua D. Drake" <jd@commandprompt.com> writes:
>> Is there any reason why we don't allow rsync access to the cvs repo?
> 
> That'd be Marc's bailiwick not mine ... but rsync from the anoncvs
> mirror seems like it couldn't pose any serious security threat.
> Marc?

Well it would sure seem to eliminate alot of the issues :)

Joshua D. Drake

> 
>             regards, tom lane
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
> 
>                http://www.postgresql.org/docs/faq
> 


-- 
            === The PostgreSQL Company: Command Prompt, Inc. ===      Sales/Support: +1.503.667.4564 || 24x7/Emergency:
+1.800.492.2240     Providing the most comprehensive  PostgreSQL solutions since 1997
http://www.commandprompt.com/




Re: [SUGGESTION] CVSync

From
Joel Miller
Date:
Joshua D. Drake wrote:

> Is there any reason why we don't allow rsync access to the cvs repo?
>
The only reason I can possibly think of is server load, comparing the timestamps for every file and directory in a
repositoryfor every rsync session could be taxing if everyone under the sun and all their cousins could do it.
 

CVSync, on the other hand, can cache those timestamps into a file and send it to the client, offloading the comparison
workloadfrom the server to the client where it belongs.
 

Joel

-- 

// file: mice.h
#include "frickin_lasers.h"



Re: [SUGGESTION] CVSync

From
"Marc G. Fournier"
Date:
On Fri, 24 Mar 2006, Tom Lane wrote:

> "Joshua D. Drake" <jd@commandprompt.com> writes:
>> Is there any reason why we don't allow rsync access to the cvs repo?
>
> That'd be Marc's bailiwick not mine ... but rsync from the anoncvs
> mirror seems like it couldn't pose any serious security threat.
> Marc?

I can't think of any reason *not* to .. if someone can, I can easily 
disable, but for now ...

/usr/local/bin/rsync -avzCH --delete anoncvs.postgresql.org::pgsql-cvs .

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664


Re: [SUGGESTION] CVSync

From
"Marc G. Fournier"
Date:
On Fri, 24 Mar 2006, Joel Miller wrote:

> Joshua D. Drake wrote:
>
>> Is there any reason why we don't allow rsync access to the cvs repo?
>> 
> The only reason I can possibly think of is server load, comparing the 
> timestamps for every file and directory in a repository for every rsync 
> session could be taxing if everyone under the sun and all their cousins could 
> do it.
>
> CVSync, on the other hand, can cache those timestamps into a file and send it 
> to the client, offloading the comparison workload from the server to the 
> client where it belongs.

I'm going to look at installing this also ... what does the client side 
need?

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664


Re: [SUGGESTION] CVSync

From
"Joshua D. Drake"
Date:
Marc G. Fournier wrote:
> On Fri, 24 Mar 2006, Tom Lane wrote:
> 
>> "Joshua D. Drake" <jd@commandprompt.com> writes:
>>> Is there any reason why we don't allow rsync access to the cvs repo?
>>
>> That'd be Marc's bailiwick not mine ... but rsync from the anoncvs
>> mirror seems like it couldn't pose any serious security threat.
>> Marc?
> 
> I can't think of any reason *not* to .. if someone can, I can easily 
> disable, but for now ...
> 
> /usr/local/bin/rsync -avzCH --delete anoncvs.postgresql.org::pgsql-cvs .

This appears to work :), although everyone please note the 
destination... you probably don't want the "," ;)

Joshua D. Drake


> 
> ----
> Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
> Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664
> 


-- 
            === The PostgreSQL Company: Command Prompt, Inc. ===      Sales/Support: +1.503.667.4564 || 24x7/Emergency:
+1.800.492.2240     Providing the most comprehensive  PostgreSQL solutions since 1997
http://www.commandprompt.com/




Re: [SUGGESTION] CVSync

From
"Marc G. Fournier"
Date:
On Fri, 24 Mar 2006, Joshua D. Drake wrote:

> Marc G. Fournier wrote:
>> On Fri, 24 Mar 2006, Tom Lane wrote:
>> 
>>> "Joshua D. Drake" <jd@commandprompt.com> writes:
>>>> Is there any reason why we don't allow rsync access to the cvs repo?
>>> 
>>> That'd be Marc's bailiwick not mine ... but rsync from the anoncvs
>>> mirror seems like it couldn't pose any serious security threat.
>>> Marc?
>> 
>> I can't think of any reason *not* to .. if someone can, I can easily 
>> disable, but for now ...
>> 
>> /usr/local/bin/rsync -avzCH --delete anoncvs.postgresql.org::pgsql-cvs .
>
> This appears to work :), although everyone please note the destination... you 
> probably don't want the "," ;)

its actually a "." ... what would a ',' do? :)

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664


Re: [SUGGESTION] CVSync

From
"Joshua D. Drake"
Date:
Marc G. Fournier wrote:
> On Fri, 24 Mar 2006, Joshua D. Drake wrote:
> 
>> Marc G. Fournier wrote:
>>> On Fri, 24 Mar 2006, Tom Lane wrote:
>>>
>>>> "Joshua D. Drake" <jd@commandprompt.com> writes:
>>>>> Is there any reason why we don't allow rsync access to the cvs repo?
>>>>
>>>> That'd be Marc's bailiwick not mine ... but rsync from the anoncvs
>>>> mirror seems like it couldn't pose any serious security threat.
>>>> Marc?
>>>
>>> I can't think of any reason *not* to .. if someone can, I can easily 
>>> disable, but for now ...
>>>
>>> /usr/local/bin/rsync -avzCH --delete anoncvs.postgresql.org::pgsql-cvs .
>>
>> This appears to work :), although everyone please note the 
>> destination... you probably don't want the "," ;)
> 
> its actually a "." ... what would a ',' do? :)

egad... heh... it would create a directory called "," ;) So you probably 
don't want either.

J


> 
> ----
> Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
> Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664
> 


-- 
            === The PostgreSQL Company: Command Prompt, Inc. ===      Sales/Support: +1.503.667.4564 || 24x7/Emergency:
+1.800.492.2240     Providing the most comprehensive  PostgreSQL solutions since 1997
http://www.commandprompt.com/




Re: [SUGGESTION] CVSync

From
Tom Lane
Date:
"Joshua D. Drake" <jd@commandprompt.com> writes:
> Marc G. Fournier wrote:
>> /usr/local/bin/rsync -avzCH --delete anoncvs.postgresql.org::pgsql-cvs .

>> its actually a "." ... what would a ',' do? :)

> egad... heh... it would create a directory called "," ;) So you probably 
> don't want either.

Um, I'm with Marc: where's the "," exactly?  Not that I'm an rsync guru,
but if there's a mistake here I do want to learn from it.
        regards, tom lane


Re: [SUGGESTION] CVSync

From
"Joshua D. Drake"
Date:
Tom Lane wrote:
> "Joshua D. Drake" <jd@commandprompt.com> writes:
>> Marc G. Fournier wrote:
>>> /usr/local/bin/rsync -avzCH --delete anoncvs.postgresql.org::pgsql-cvs .
> 
>>> its actually a "." ... what would a ',' do? :)
> 
>> egad... heh... it would create a directory called "," ;) So you probably 
>> don't want either.
> 
> Um, I'm with Marc: where's the "," exactly?  Not that I'm an rsync guru,
> but if there's a mistake here I do want to learn from it.
> 
>         

Just to be clear. The comma was a mistype. I was stating that you do not 
want to use a period. If you use a period and you are not in the correct 
directory, rsync will remove everything within your working directory. 
It is always better to be explicit with an application that will 
willingly remove your entire directory structure.

I know this, because I accidently did it once when manually pulling the 
archives. I had to restore www.commandprompt.com from backup ;)

Sincerely,

Joshua D. Drake



-- 
            === The PostgreSQL Company: Command Prompt, Inc. ===      Sales/Support: +1.503.667.4564 || 24x7/Emergency:
+1.800.492.2240     Providing the most comprehensive  PostgreSQL solutions since 1997
http://www.commandprompt.com/




Re: [SUGGESTION] CVSync

From
"Marc G. Fournier"
Date:
On Sat, 25 Mar 2006, Tom Lane wrote:

> "Joshua D. Drake" <jd@commandprompt.com> writes:
>> Marc G. Fournier wrote:
>>> /usr/local/bin/rsync -avzCH --delete anoncvs.postgresql.org::pgsql-cvs .
>
>>> its actually a "." ... what would a ',' do? :)
>
>> egad... heh... it would create a directory called "," ;) So you probably
>> don't want either.
>
> Um, I'm with Marc: where's the "," exactly?  Not that I'm an rsync guru,
> but if there's a mistake here I do want to learn from it.

I figure that Joshua just hit the wrong key ... or, at least, I hope that 
is what he did, vs just going blind at a young age ;(

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664


Re: [SUGGESTION] CVSync

From
Tom Lane
Date:
"Joshua D. Drake" <jd@commandprompt.com> writes:
> Just to be clear. The comma was a mistype. I was stating that you do not 
> want to use a period. If you use a period and you are not in the correct 
> directory, rsync will remove everything within your working directory. 

Oh, so you're saying the second argument should be an absolute path?
OK, that's a great clue -- right on par with "don't rm -rf ." ...
        regards, tom lane


Re: [SUGGESTION] CVSync

From
"Joshua D. Drake"
Date:
> I figure that Joshua just hit the wrong key ... or, at least, I hope 
> that is what he did, vs just going blind at a young age ;(

No, not that blind yet :)


> 
> ----
> Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
> Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
> 
>               http://www.postgresql.org/docs/faq
> 


-- 
            === The PostgreSQL Company: Command Prompt, Inc. ===      Sales/Support: +1.503.667.4564 || 24x7/Emergency:
+1.800.492.2240     Providing the most comprehensive  PostgreSQL solutions since 1997
http://www.commandprompt.com/




Re: [SUGGESTION] CVSync

From
"Joshua D. Drake"
Date:
Tom Lane wrote:
> "Joshua D. Drake" <jd@commandprompt.com> writes:
>> Just to be clear. The comma was a mistype. I was stating that you do not 
>> want to use a period. If you use a period and you are not in the correct 
>> directory, rsync will remove everything within your working directory. 
> 
> Oh, so you're saying the second argument should be an absolute path?

Yes, that is exactly what I am saying. Although I typically will use a 
relative path, I always specify a directory name.

> OK, that's a great clue -- right on par with "don't rm -rf ." ...

Yep, just one of those foot guns I prefer not to use ;)

Joshua D. Drake

> 
>             regards, tom lane
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
> 
>                http://archives.postgresql.org
> 


-- 
            === The PostgreSQL Company: Command Prompt, Inc. ===      Sales/Support: +1.503.667.4564 || 24x7/Emergency:
+1.800.492.2240     Providing the most comprehensive  PostgreSQL solutions since 1997
http://www.commandprompt.com/




Re: [SUGGESTION] CVSync

From
Joel Miller
Date:
Marc G. Fournier wrote:

> On Fri, 24 Mar 2006, Joel Miller wrote:
>
>> Joshua D. Drake wrote:
>>
>>> Is there any reason why we don't allow rsync access to the cvs repo?
>>>
>> The only reason I can possibly think of is server load, comparing the 
>> timestamps for every file and directory in a repository for every 
>> rsync session could be taxing if everyone under the sun and all their 
>> cousins could do it.
>>
>> CVSync, on the other hand, can cache those timestamps into a file and 
>> send it to the client, offloading the comparison workload from the 
>> server to the client where it belongs.
>
>
> I'm going to look at installing this also ... what does the client 
> side need?
>
> ----
> Marc G. Fournier           Hub.Org Networking Services 
> (http://www.hub.org)
> Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 
> 7615664
>
To build and install? Just the sources from www.cvsync.org and gcc. I'm 
running OS X, and I needed to specify a useful value for  HASH_TYPE in 
mk/defaults.mk (doing 'make configure' creates that file).

Basically here's all you need to do:   download sources, expand and cd into the folder   make configure   (tweak
HASH_TYPEin ./mk/defaults.mk if needed)   make   make install
 

I created an empty config file in /usr/local/etc/cvsync.conf to keep the 
client app from complaining. You can create individual config files for 
each collection if you want and store them wherever it makes sense for 
you (I keep mine one folder up from the destination). The file format is 
documented in the man pages for cvsync and cvsyncd. I have a commented 
example config file for defining collections on the server side that I 
can send if you want it. You can also specify most parameters on the 
command line as well, both for the client and server.

If you want to take a look at a live deployment of cvsync, check out 
http://openbsd.org/cvsync.html

Hope this helps!
Joel

-- 

// file: mice.h
#include "frickin_lasers.h"



Re: [SUGGESTION] CVSync

From
Alvaro Herrera
Date:
Tom Lane wrote:
> Neil Conway <neilc@samurai.com> writes:
> > I agree that investigating alternatives would be a good idea: AFAIK
> > there's no easy way to build cvsup on Linux/AMD64 (without patches and
> > more pain than I'm willing to endure), so I use cvsup on one machine and
> > then periodically rsync a copy of that to my main (AMD64) machine.
> 
> Actually, the last time I looked seriously at cvsup, my requirement was
> that it build on HPUX, which it did not (well, maybe it would've with
> sufficient investment of pain, but that wasn't happening).  I was about
> to take another look at whether I couldn't just-install-it on FC4, but
> my main FC4 machine is now x86_64, so your remarks are discouraging me
> again :-(

Don't let'em.  I use CVSup on both my ix86 and amd64 machine.  (IIRC the latter
uses a 32bit binary, but it surely works anyway.)

I think these are static binaries, so they would probably work on your
machines unchanged.


Hmm, well, not quite:

$ ldd bin/cvsup        linux-gate.so.1 =>  (0xffffe000)       libz.so.1 => /usr/lib/libz.so.1 (0xb7f5b000)
libXaw.so.7=> /usr/X11R6/lib/libXaw.so.7 (0xb7eff000)       libXmu.so.6 => /usr/X11R6/lib/libXmu.so.6 (0xb7ee9000)
libXext.so.6 => /usr/X11R6/lib/libXext.so.6 (0xb7edb000)       libXt.so.6 => /usr/X11R6/lib/libXt.so.6 (0xb7e8b000)
 libSM.so.6 => /usr/X11R6/lib/libSM.so.6 (0xb7e82000)       libICE.so.6 => /usr/X11R6/lib/libICE.so.6 (0xb7e6a000)
libX11.so.6 => /usr/X11R6/lib/libX11.so.6 (0xb7d9e000)       libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7d64000)
 libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7c2c000)       libXpm.so.4 => /usr/X11R6/lib/libXpm.so.4 (0xb7c16000)
  libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7c12000)       /lib/ld-linux.so.2 (0xb7f71000)
 

(this is the ix86 machine, I don't have the other one handy)

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


Re: [SUGGESTION] CVSync

From
Tom Lane
Date:
Alvaro Herrera <alvherre@commandprompt.com> writes:
> $ ldd bin/cvsup 
>         linux-gate.so.1 =>  (0xffffe000)
>         libz.so.1 => /usr/lib/libz.so.1 (0xb7f5b000)
>         libXaw.so.7 => /usr/X11R6/lib/libXaw.so.7 (0xb7eff000)
>         libXmu.so.6 => /usr/X11R6/lib/libXmu.so.6 (0xb7ee9000)
>         libXext.so.6 => /usr/X11R6/lib/libXext.so.6 (0xb7edb000)
>         libXt.so.6 => /usr/X11R6/lib/libXt.so.6 (0xb7e8b000)
>         libSM.so.6 => /usr/X11R6/lib/libSM.so.6 (0xb7e82000)
>         libICE.so.6 => /usr/X11R6/lib/libICE.so.6 (0xb7e6a000)
>         libX11.so.6 => /usr/X11R6/lib/libX11.so.6 (0xb7d9e000)
>         libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7d64000)
>         libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7c2c000)
>         libXpm.so.4 => /usr/X11R6/lib/libXpm.so.4 (0xb7c16000)
>         libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7c12000)
>         /lib/ld-linux.so.2 (0xb7f71000)

Why in the world is cvsup linked to X libraries?  Surely it does not
need a GUI.
        regards, tom lane


Re: [SUGGESTION] CVSync

From
"Andrew Dunstan"
Date:
Tom Lane said:

> Why in the world is cvsup linked to X libraries?  Surely it does not
> need a GUI.
>

The client has GUI and non-GUI modes. For command-line ops use the -g flag
or unset DISPLAY.

My crontab entry looks like this:

3 * * * * /usr/bin/cvsup -g -r 25 /home/cvsmirror/postgres.cvsup

cheers

andrew




Re: [SUGGESTION] CVSync

From
Alvaro Herrera
Date:
Andrew Dunstan wrote:
> Tom Lane said:
> 
> > Why in the world is cvsup linked to X libraries?  Surely it does not
> > need a GUI.
> 
> The client has GUI and non-GUI modes. For command-line ops use the -g flag
> or unset DISPLAY.

Apparently not only the CVSup authors chose a weird language, they also
wanted to get fancy and wrote a GUI.  It's pretty useless if you ask me.
I always use the -g switch as well, even when using it interactively.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.


Re: [SUGGESTION] CVSync

From
"Joshua D. Drake"
Date:
Alvaro Herrera wrote:
> Andrew Dunstan wrote:
>> Tom Lane said:
>>
>>> Why in the world is cvsup linked to X libraries?  Surely it does not
>>> need a GUI.
>> The client has GUI and non-GUI modes. For command-line ops use the -g flag
>> or unset DISPLAY.
> 
> Apparently not only the CVSup authors chose a weird language, they also
> wanted to get fancy and wrote a GUI.  It's pretty useless if you ask me.
> I always use the -g switch as well, even when using it interactively.
> 

With rsync capability now.... is CVSup even needed?

Joshua D. Drake


-- 
            === The PostgreSQL Company: Command Prompt, Inc. ===      Sales/Support: +1.503.667.4564 || 24x7/Emergency:
+1.800.492.2240     Providing the most comprehensive  PostgreSQL solutions since 1997
http://www.commandprompt.com/




Re: [SUGGESTION] CVSync

From
"Marc G. Fournier"
Date:
On Sat, 25 Mar 2006, Tom Lane wrote:

> Alvaro Herrera <alvherre@commandprompt.com> writes:
>> $ ldd bin/cvsup
>>         linux-gate.so.1 =>  (0xffffe000)
>>         libz.so.1 => /usr/lib/libz.so.1 (0xb7f5b000)
>>         libXaw.so.7 => /usr/X11R6/lib/libXaw.so.7 (0xb7eff000)
>>         libXmu.so.6 => /usr/X11R6/lib/libXmu.so.6 (0xb7ee9000)
>>         libXext.so.6 => /usr/X11R6/lib/libXext.so.6 (0xb7edb000)
>>         libXt.so.6 => /usr/X11R6/lib/libXt.so.6 (0xb7e8b000)
>>         libSM.so.6 => /usr/X11R6/lib/libSM.so.6 (0xb7e82000)
>>         libICE.so.6 => /usr/X11R6/lib/libICE.so.6 (0xb7e6a000)
>>         libX11.so.6 => /usr/X11R6/lib/libX11.so.6 (0xb7d9e000)
>>         libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7d64000)
>>         libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7c2c000)
>>         libXpm.so.4 => /usr/X11R6/lib/libXpm.so.4 (0xb7c16000)
>>         libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7c12000)
>>         /lib/ld-linux.so.2 (0xb7f71000)
>
> Why in the world is cvsup linked to X libraries?  Surely it does not
> need a GUI.

it is possible to build a non-gui version as well, that doesn't require X 
... that's all I ever buld, but, for me, its simple ... cd 
/usr/ports/net/cvsup-without-gui; make install :)



mercury# ldd /usr/local/bin/cvsup
/usr/local/bin/cvsup:        libz.so.2 => /usr/lib/libz.so.2 (0x280fa000)        libutil.so.3 => /usr/lib/libutil.so.3
(0x28107000)       libmd.so.2 => /usr/lib/libmd.so.2 (0x28110000)        libm.so.2 => /usr/lib/libm.so.2 (0x28119000)
    libc.so.4 => /usr/lib/libc.so.4 (0x28134000)
 


----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664


Re: [SUGGESTION] CVSync

From
"Marc G. Fournier"
Date:
On Sat, 25 Mar 2006, Joshua D. Drake wrote:

> Alvaro Herrera wrote:
>> Andrew Dunstan wrote:
>>> Tom Lane said:
>>> 
>>>> Why in the world is cvsup linked to X libraries?  Surely it does not
>>>> need a GUI.
>>> The client has GUI and non-GUI modes. For command-line ops use the -g flag
>>> or unset DISPLAY.
>> 
>> Apparently not only the CVSup authors chose a weird language, they also
>> wanted to get fancy and wrote a GUI.  It's pretty useless if you ask me.
>> I always use the -g switch as well, even when using it interactively.
>> 
>
> With rsync capability now.... is CVSup even needed?

CVSup is on the main repository, rsync is on the anoncvs repository ... 
depends on how fast you want your changes :)

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664


Re: [SUGGESTION] CVSync

From
"Magnus Hagander"
Date:
> > "Joshua D. Drake" <jd@commandprompt.com> writes:
> >> Is there any reason why we don't allow rsync access to the
> cvs repo?
> >
> > That'd be Marc's bailiwick not mine ... but rsync from the anoncvs
> > mirror seems like it couldn't pose any serious security threat.
> > Marc?
>
> I can't think of any reason *not* to .. if someone can, I can
> easily disable, but for now ...
>
> /usr/local/bin/rsync -avzCH --delete
> anoncvs.postgresql.org::pgsql-cvs .

Slightly after this was enabled, all the bittorrent stuff started
falling over on svr4. This has been seen before when the load has
approached a certain level on svr4 - which appears to be severly
overloaded much of time (still on unionfs, I beleive).

Could you please do some monitoring and see if this could be related to
the rsync stuff (I beleive you need to be on the host and not the VM to
do that easily). It seems that it shouldn't be that much load (yet), but
the timing was suspiciously close.

//Magnus


Bittorrent (Was: RE: [SUGGESTION] CVSync )

From
"Marc G. Fournier"
Date:
Does anyone here know how to use the latest version of Bittorrent, from a 
server vs client, perspective?  The one we are currently running is 
*ancient*, and I'd like to get that vServer upgraded and off the 
templates, but without someone that knows how to setup the newer 
bittorrent, well, that's just a scary proposition ;(



On Sun, 26 Mar 2006, Magnus Hagander wrote:

>>> "Joshua D. Drake" <jd@commandprompt.com> writes:
>>>> Is there any reason why we don't allow rsync access to the
>> cvs repo?
>>>
>>> That'd be Marc's bailiwick not mine ... but rsync from the anoncvs
>>> mirror seems like it couldn't pose any serious security threat.
>>> Marc?
>>
>> I can't think of any reason *not* to .. if someone can, I can
>> easily disable, but for now ...
>>
>> /usr/local/bin/rsync -avzCH --delete
>> anoncvs.postgresql.org::pgsql-cvs .
>
> Slightly after this was enabled, all the bittorrent stuff started
> falling over on svr4. This has been seen before when the load has
> approached a certain level on svr4 - which appears to be severly
> overloaded much of time (still on unionfs, I beleive).
>
> Could you please do some monitoring and see if this could be related to
> the rsync stuff (I beleive you need to be on the host and not the VM to
> do that easily). It seems that it shouldn't be that much load (yet), but
> the timing was suspiciously close.
>
> //Magnus
>

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664


Re: [SUGGESTION] CVSync

From
Bruce Momjian
Date:
Andrew Dunstan wrote:
> I don't know what unreasonable infrastructure you are referring to. 
> Building cvsup is a major pain, but installing and running it isn't, in 
> my experience. There's a package in Fedora Extras. Setting up cvsweb 
> against my cvsup repo is a fine idea - I wonder why I didn't think of that.

Using someone else's binary isn't an attractive alternative for open
source users.

--  Bruce Momjian   http://candle.pha.pa.us
 + If your life is a hard drive, Christ can be your backup. +


Re: [SUGGESTION] CVSync

From
Andrew Dunstan
Date:

Bruce Momjian wrote:

>Andrew Dunstan wrote:
>  
>
>>I don't know what unreasonable infrastructure you are referring to. 
>>Building cvsup is a major pain, but installing and running it isn't, in 
>>my experience. There's a package in Fedora Extras. Setting up cvsweb 
>>against my cvsup repo is a fine idea - I wonder why I didn't think of that.
>>    
>>
>
>Using someone else's binary isn't an attractive alternative for open
>source users.
>
>  
>

Really? I haven't built the compiler, or make, or almost anything else 
in the toolset we use on any of the platforms I build on (FC*, Windows, 
Cygwin). So I use binaries built by other peoiple all the time. I bet 
you do too. In any case, cvsup would only be a supplement to cvs, not a 
replacement, and probably only of interest to developers, not users. 
It's not like cvsup is proprietary.

And we support cvsup right now - it's particularly nice with buildfarm.

cheers

andrew