Thread: having problem in rsync'ing cvs
Hi all,<br /><br /> I was rsyncing a fresh copy of CVS repository, and suddenly, midway (around heap.c) rsync complained.When after many retries I couldn't start it, I restarted the OS.<br /><br /> Since it is a VM running insideVista (from where I could ping and browse <a href="http://anoncvs.postgresql.org">anoncvs.postgresql.org</a>), I restartedthe Host OS too.<br /><br />Here's the configuration:<br />Host OS: Windows Vista<br />VM Software: Innotek VirtualBox<br/>Guest OS: Ubuntu 7.10<br /><br />And here's the error I am getting:<br /><br />gurjeet@g-u-v:~/dev/repositories$rsync -avzCH --delete anoncvs.postgresql.org::pgsql-cvs pg_cvsroot/<br /> rsync: getaddrinfo:<a href="http://anoncvs.postgresql.org">anoncvs.postgresql.org</a> 873: Name or service not known<br />rsyncerror: error in socket IO (code 10) at clientserver.c(104) [receiver=2.6.9]<br />gurjeet@g-u-v:~/dev/repositories$<br /><br />Is the rsync daemon on anoncvs down? Is everyone else able to do rsync?<br/><br />Thanks in advance,<br /><br />Best regards,<br />-- <br />gurjeet[.singh]@EnterpriseDB.com<br />singh.gurjeet@{gmail | hotmail | indiatimes | yahoo }.com<br /><br />EnterpriseDB <a href="http://www.enterprisedb.com">http://www.enterprisedb.com</a><br/><br />Mail sent from my BlackLaptop device
On 27/03/2008, Gurjeet Singh <singh.gurjeet@gmail.com> wrote: > Is the rsync daemon on anoncvs down? Is everyone else able to do rsync? > Possibly related; the Postgres git repository at http://repo.or.cz/w/PostgreSQL.git is showing the last commit at 25 hours ago. It's usually a bit more spry than that.
Gurjeet Singh escribió: > Is the rsync daemon on anoncvs down? Is everyone else able to do rsync? I can rsync with no issue. -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
On Wed, Mar 26, 2008 at 7:56 PM, Alvaro Herrera <alvherre@commandprompt.com> wrote:
I attempted again, and it seems to have started responding.... Seems like an intermittent issue.
Thanks much,
Best regards,
--
gurjeet[.singh]@EnterpriseDB.com
singh.gurjeet@{ gmail | hotmail | indiatimes | yahoo }.com
EnterpriseDB http://www.enterprisedb.com
Mail sent from my BlackLaptop device
Gurjeet Singh escribió:I can rsync with no issue.
> Is the rsync daemon on anoncvs down? Is everyone else able to do rsync?
I attempted again, and it seems to have started responding.... Seems like an intermittent issue.
Thanks much,
Best regards,
gurjeet[.singh]@EnterpriseDB.com
singh.gurjeet@{ gmail | hotmail | indiatimes | yahoo }.com
EnterpriseDB http://www.enterprisedb.com
Mail sent from my BlackLaptop device
* Brendan Jurd <direvus@gmail.com> [080326 10:19]: > On 27/03/2008, Gurjeet Singh <singh.gurjeet@gmail.com> wrote: > > Is the rsync daemon on anoncvs down? Is everyone else able to do rsync? > > > > Possibly related; the Postgres git repository at > http://repo.or.cz/w/PostgreSQL.git is showing the last commit at 25 > hours ago. It's usually a bit more spry than that. Oops - no, that's just me... I was recently made aware of this:http://repo.or.cz/w/PostgreSQL.git?a=commit;h=69db64c737012a8d2d6fbcce3ace7136cb2bc85f I started digging around to figure this out on Tuesday. It appears as if the "rsync" mirror of CVS is not always "good". It seems like "long running" CVS operations (like I'm guessing a full tree "tag" of REL8_3_STABLE) aren't mirrored "atomically". Of course, CVS isn't atomic, so we can't really blame it. What appears to have happened is that my rsync caught the rsync mirror when the tree was "not all tagged", so when the fromcvs went about making the new branch on the first appearance of REL8_3_STABLE, it had to remove a bunch of files from the branch because they were *not* tagged with that symbol in CVS (or at least, not presently tagged with that symbol in the rsync mirror of CVS)... I would guess that any incremental CVS mirror/conversion is going to hit this at some random time too. Of course, the risk of hitting it goes up as the frequency of your rsync updates go up. And I just forgot to re-enable my cron after I finished looking at it. BTW, anybody following the GIT mirror, the REL8_3_STABLE branch has been re-wound, you you'll probably have to force update it (git fetch -f) if you only accept fast forward updates on fetches (the default). And if you have patches based on REL8_3_STABLE, you'll need to rebase them too. Of course, seeing as the tree in REL8_3_STABLE mirror was broken, I suspect the set of people using it was 0. a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
On 28/03/2008, Aidan Van Dyk <aidan@highrise.ca> wrote: > And I just forgot to re-enable my cron after I finished looking at it. > Ah, the old "post-maintenance-disabled-cron" gaff. One of my personal favourites. =) I'm not sure that the git repos has fully recovered. There seems to a block of commits missing, between 2008-03-25 13:09 and 2008-03-27 17:24 UTC. Looking at the CVS logs, there was definitely commit action in that timeframe, but none of it is showing up on the git shortlog. Cheers, BJ
* Brendan Jurd <direvus@gmail.com> [080327 16:08]: > I'm not sure that the git repos has fully recovered. There seems to a > block of commits missing, between 2008-03-25 13:09 and 2008-03-27 > 17:24 UTC. > > Looking at the CVS logs, there was definitely commit action in that > timeframe, but none of it is showing up on the git shortlog. OK, I'll take another look at it tonight. a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
> BTW, anybody following the GIT mirror, the REL8_3_STABLE branch has been > re-wound, you you'll probably have to force update it (git fetch -f) if > you only accept fast forward updates on fetches (the default). Thanks! now i can ditch the git.or.cz mirror > And if you have patches based on REL8_3_STABLE, you'll need to rebase > them too. Of course, seeing as the tree in REL8_3_STABLE mirror was > broken, I suspect the set of people using it was 0. > > a.
On Thu, Mar 27, 2008 at 5:34 PM, Alex Hunsaker <badalex@gmail.com> wrote: > > BTW, anybody following the GIT mirror, the REL8_3_STABLE branch has been > > re-wound, you you'll probably have to force update it (git fetch -f) if > > you only accept fast forward updates on fetches (the default). > > Thanks! now i can ditch the git.or.cz mirror > Err oops I was confused, i was talking about git.postgresql.org where REL8_3_STABLE is about 6 weeks old
* Alex Hunsaker <badalex@gmail.com> [080327 19:38]: > On Thu, Mar 27, 2008 at 5:34 PM, Alex Hunsaker <badalex@gmail.com> wrote: > > > BTW, anybody following the GIT mirror, the REL8_3_STABLE branch has been > > > re-wound, you you'll probably have to force update it (git fetch -f) if > > > you only accept fast forward updates on fetches (the default). > > > > Thanks! now i can ditch the git.or.cz mirror > > > > Err oops I was confused, i was talking about git.postgresql.org where > REL8_3_STABLE is about 6 weeks old Ya, Peter E asked me to help look at that for a while ago too. I couldn't figure out why it's not updating either. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
* Brendan Jurd <direvus@gmail.com> [080327 16:36]: > Ah, the old "post-maintenance-disabled-cron" gaff. One of my personal > favourites. =) > > I'm not sure that the git repos has fully recovered. There seems to a > block of commits missing, between 2008-03-25 13:09 and 2008-03-27 > 17:24 UTC. > > Looking at the CVS logs, there was definitely commit action in that > timeframe, but none of it is showing up on the git shortlog. OK, so it should all be valid again. Sorry for any issues I caused to those trying to use that mirror regularly... a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
On 29/03/2008, Aidan Van Dyk <aidan@highrise.ca> wrote: > * Brendan Jurd <direvus@gmail.com> [080327 16:36]: > > > > Looking at the CVS logs, there was definitely commit action in that > > timeframe, but none of it is showing up on the git shortlog. > > OK, so it should all be valid again. > Looks good to me. Thanks Aidan. BJ
* Brendan Jurd <direvus@gmail.com> [080328 11:16]: > On 29/03/2008, Aidan Van Dyk <aidan@highrise.ca> wrote: > > * Brendan Jurd <direvus@gmail.com> [080327 16:36]: > > > > > > Looking at the CVS logs, there was definitely commit action in that > > > timeframe, but none of it is showing up on the git shortlog. > > > > OK, so it should all be valid again. > > > > Looks good to me. Thanks Aidan. And, of course, after "fixing" the missing one, I forgot to enable the cronjob *again*... It's running now again. Really. a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
On Mar 27, 2008, at 10:47 AM, Aidan Van Dyk wrote: > I was recently made aware of this: > http://repo.or.cz/w/PostgreSQL.git? > a=commit;h=69db64c737012a8d2d6fbcce3ace7136cb2bc85f > > I started digging around to figure this out on Tuesday. > > It appears as if the "rsync" mirror of CVS is not always "good". It > seems like "long running" CVS operations (like I'm guessing a full > tree > "tag" of REL8_3_STABLE) aren't mirrored "atomically". Of course, CVS > isn't atomic, so we can't really blame it. > > What appears to have happened is that my rsync caught the rsync mirror > when the tree was "not all tagged", so when the fromcvs went about > making the new branch on the first appearance of REL8_3_STABLE, it had > to remove a bunch of files from the branch because they were *not* > tagged with that symbol in CVS (or at least, not presently tagged with > that symbol in the rsync mirror of CVS)... > > I would guess that any incremental CVS mirror/conversion is going > to hit > this at some random time too. Of course, the risk of hitting it > goes up > as the frequency of your rsync updates go up. Hrm... is there a way to momentarily lock-out access to CVS? What I'm thinking is to have a script that periodically locks CVS access, takes a snapshot of the tree (preferably via a filesystem snapshot), and then unlocks. That snapshot would then be used to drive the mirrors. That would ensure that mirrors always had an atomic view of things. -- Decibel!, aka Jim C. Nasby, Database Architect decibel@decibel.org Give your computer some brain candy! www.distributed.net Team #1828