Re: [SPAM] Re: GIT mirror not updating - Mailing list pgsql-www

Heikki Linnakangas wrote:
> I propose that I will try
> that dummy "branch commit" hack in cvsps, and if it works out, create a
> new GIT repository using that. That would replace the current
> repository, which means that people working against that repository will
> need to use "git-fetch --force" to update their clones, and rebase their
> own branches. It would also be good to contribute the hack into upstream
> cvsps (whatever fork is considered upstream).
>
> I'd like to move to git-cvsimport because:
> - it converts tags
> - I'm more familiar with it than fromcvs/togit.

I tried that, but gave up after a while. I hacked cvsps to create the
dummy commits, but it choked on some files that exist some branches but
not others. It might be fixable, but I'm now convinced that
fromcvs/togit is the simplest solution after all. The lack of tags is
slightly annoying, but not a show-stopper. And we can add them manually
if we really want them, we don't tag that often.

togit normally looks at all the branches, and retrieves all commits from
CVS with date later than the latest commit on any branch in the GIT
repository. I think that's why the back-branches stopped updating after
there was even a single commit on master with later date. I modified it
so that each branch is treated separately. I'm not sure how we ended up
with duplicated history, though.

I have a recipe to fix the repository, and we don't even need to
recreate it from scratch. I'd appreciate that, because I have some
pretty complex stuff in my GIT repository with merges and everything
that could be a lot of work to rebase. This fixes the back-branches, too.

0. Backup

1. Apply the attached patch to togit.

2. Use git-reset to rewind the branches back to the state they were
before breakage. Attached rewind-branches.sh script does that for all
the branches.

3. Run togit.

After that, pulling from a repository that was last updated before Dec
13th (when it was screwed up), you get a nice fast-forward as usual. If
you have already pulled the duplicated history,  you have to use
"git-pull --force" or git-pull will refuse to update because "history
has changed".

It should be safe, but not necessaery, to always use the modified togit
script. After fixing the repository, incremental updates should work
with unmodified togit as well, I think. The patched version is slower,
because it skips already-applied commits at a later stage, but most of
the time is spent in parsing the RCS logs anyway, so it might not be
important.

--
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com
Common subdirectories: fromcvs-9679642582bc/commitset and fromcvs/commitset
Common subdirectories: fromcvs-9679642582bc/t and fromcvs/t
diff -u fromcvs-9679642582bc/togit.rb fromcvs/togit.rb
--- fromcvs-9679642582bc/togit.rb    2008-12-01 02:28:19.000000000 +0200
+++ fromcvs/togit.rb    2008-12-30 11:09:40.000000000 +0200
@@ -31,6 +31,7 @@
     @deleted = []
     @modified = []
     @branchcache = {}
+    @last_dates = {}
     @files = Hash.new{|h,k| h[k] = {}}

     @mark = 0
@@ -47,13 +48,31 @@
       next if type != 'commit'
       branch[/^.*\//] = ""
       @branchcache[branch] = sha
+      @last_dates[branch] = _get_last_date(branch);
+      puts @last_dates[branch]
     end
     @pickupbranches = @branchcache.dup
   end

+  # All commits before this date is skipped when reading the CVS repository.
+  # In incremental mode, should be the date of *last* commit on any branch
+  # in the GIT repository. However, with this patch, we don't assume that
+  # all branches have been updated up to the same point in time. Therefore,
+  # take the *minimum* date of branch tips in GIT.
+  #
+  # In _commit function below, we check that we don't try to commit anything
+  # to a branch with older date the tip commit on the branch. Otherwise
+  # we'd get duplicate commits.
   def last_date
+    return @last_dates.values.min
+    #return @last_dates.values.max # use this for the normal incremental mode
+  end
+
+  def _get_last_date(branch)
+    branch = 'refs/heads/' + branch
+    puts 'requesting ' + branch
     latestref = _command(*%w{git for-each-ref --count=1 --sort=-committerdate
-                                --format=%(refname) refs/heads})
+                                --format=%(refname) } + [branch])
     log = _command('git', 'cat-file', '-p', latestref.strip)
     log.split("\n").each do |line|
       break if line.empty?
@@ -148,7 +167,9 @@
   end

   def commit(author, date, msg, revs)
-    _commit(author, date, msg, revs)
+    if date > @last_dates[@curbranch]
+      _commit(author, date, msg, revs)
+    end
   end

   def merge(branch, author, date, msg, revs)
Only in fromcvs: togit.rb~