Thread: GIT mirror not updating
Looks like the GIT mirror at git.postgresql.org hasn't been updated since Saturday. ? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
On Tue, Dec 16, 2008 at 4:09 PM, Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> wrote: > Looks like the GIT mirror at git.postgresql.org hasn't been updated since > Saturday. I upgraded most of postgresql.org on Sunday (yeah, I have no life), and it seems the new version of git in FreeBSD ports has dropped the convenience binaries such as git-pull. I've update the update-repos script to use 'git pull' instead, and rerun it. If there are other repos, not updated by that script, I'll need more info on them to go fix. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com
Dave Page wrote: > On Tue, Dec 16, 2008 at 4:09 PM, Heikki Linnakangas > <heikki.linnakangas@enterprisedb.com> wrote: > > Looks like the GIT mirror at git.postgresql.org hasn't been updated since > > Saturday. > > I upgraded most of postgresql.org on Sunday (yeah, I have no life), > and it seems the new version of git in FreeBSD ports has dropped the > convenience binaries such as git-pull. It wasn't FreeBSD's fault -- git upstream did it. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
On Tuesday 16 December 2008 18:42:43 Dave Page wrote: > On Tue, Dec 16, 2008 at 4:09 PM, Heikki Linnakangas > > <heikki.linnakangas@enterprisedb.com> wrote: > > Looks like the GIT mirror at git.postgresql.org hasn't been updated since > > Saturday. > > I upgraded most of postgresql.org on Sunday (yeah, I have no life), > and it seems the new version of git in FreeBSD ports has dropped the > convenience binaries such as git-pull. I've update the update-repos > script to use 'git pull' instead, and rerun it. > > If there are other repos, not updated by that script, I'll need more > info on them to go fix. I also needed to upgrade the fromcvs ruby scripts analogously. It's all back to working now.
Heikki Linnakangas wrote:
> Peter Eisentraut wrote:
>> I also needed to upgrade the fromcvs ruby scripts analogously. It's
>> all back to working now.
>
> Oh wow, did you just make the back-branches work too?
Hmm, apparently not. But there was this commit:
commit 90bf6f50d2324131b2868ac97563c73aa36a9974
Author: repo-copy <repo-copy>
Date: Tue Dec 16 19:30:44 2008 +0000
Repo copy files
to the REL8_3_STABLE branch, and similar commits to other branches.
-- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Peter Eisentraut wrote: > I also needed to upgrade the fromcvs ruby scripts analogously. It's all back > to working now. Oh wow, did you just make the back-branches work too? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Heikki Linnakangas wrote:
> Heikki Linnakangas wrote:
>> Peter Eisentraut wrote:
>>> I also needed to upgrade the fromcvs ruby scripts analogously. It's
>>> all back to working now.
>>
>> Oh wow, did you just make the back-branches work too?
>
> Hmm, apparently not. But there was this commit:
>
> commit 90bf6f50d2324131b2868ac97563c73aa36a9974
> Author: repo-copy <repo-copy>
> Date: Tue Dec 16 19:30:44 2008 +0000
>
> Repo copy files
>
> to the REL8_3_STABLE branch, and similar commits to other branches.
Something is wrong with the repository. There seems to be multiple
copies of all commits are in master branch. For example:
$ git-log --grep="Do not try to change" origin/master
commit be4e5c7b547c41d01e7c74dbb0fd805c6b3b6a8f
Author: Michael Meskes <meskes@postgresql.org>
Date: Mon Dec 15 15:34:07 2008 +0000
Do not try to change a const variable.
commit 09fce45b965adffa885f5bf9271d74c255e51202
Author: Michael Meskes <meskes@postgresql.org>
Date: Mon Dec 15 15:34:07 2008 +0000
Do not try to change a const variable.
commit e2da9d8a752101d0b2d4c0ff4e82c9a2e82e4533
Author: Michael Meskes <meskes@postgresql.org>
Date: Mon Dec 15 15:34:07 2008 +0000
Do not try to change a const variable.
Perhaps now is a good time to give up on these scripts, rebuild the
repository using git-cvsimport?
-- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
On Wed, 17 Dec 2008, Heikki Linnakangas wrote: > Perhaps now is a good time to give up on these scripts, rebuild the > repository using git-cvsimport? > My experience with git-cvsimport has been poor. For me it often puts commits on the wrong branches. It uses the first commit to the branch as the branch start time, so a branch will show commits to head as being on the branch until the first commit to the branch. The only cvs->git converter that's really worked for me (does all branches, tags, keyword removal, ...) has been parsecvs [1] which isn't terribly user friendly and is not incremental. [1] http://git.or.cz/gitwiki/InterfacesFrontendsAndTools#head-769e9aa96370a5b3d0356c918f310c177c0799fe Kris Jurka
Kris Jurka wrote: > > > On Wed, 17 Dec 2008, Heikki Linnakangas wrote: > >> Perhaps now is a good time to give up on these scripts, rebuild the >> repository using git-cvsimport? >> > > My experience with git-cvsimport has been poor. For me it often puts > commits on the wrong branches. It uses the first commit to the branch > as the branch start time, so a branch will show commits to head as being > on the branch until the first commit to the branch. I just tried running git-cvsimport on the PG repository, and I can see what you mean. You're right, that's not acceptable. > The only cvs->git converter that's really worked for me (does all > branches, tags, keyword removal, ...) has been parsecvs [1] which isn't > terribly user friendly and is not incremental. > > [1] > http://git.or.cz/gitwiki/InterfacesFrontendsAndTools#head-769e9aa96370a5b3d0356c918f310c177c0799fe "Not incremental" rules that one out, unfortunately. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Heikki Linnakangas wrote: > Kris Jurka wrote: >> On Wed, 17 Dec 2008, Heikki Linnakangas wrote: >> >>> Perhaps now is a good time to give up on these scripts, rebuild the >>> repository using git-cvsimport? >> >> My experience with git-cvsimport has been poor. For me it often puts >> commits on the wrong branches. It uses the first commit to the branch >> as the branch start time, so a branch will show commits to head as >> being on the branch until the first commit to the branch. > > I just tried running git-cvsimport on the PG repository, and I can see > what you mean. You're right, that's not acceptable. I hacked cvsps a little bit, to move the first commit on the branch right after the correct branch point, before writing the output. This is a hack, but seems to work at least for the PG repository. I'm not sure if we want to run a hacked version of cvsps. But then again, the current conversion isn't exactly error-free either, and we're missing tags... -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com diff --git a/cvsps.c b/cvsps.c index 81c6e21..9135641 100644 --- a/cvsps.c +++ b/cvsps.c @@ -149,6 +149,8 @@ static int before_tag(CvsFileRevision * rev, const char * tag); static void determine_branch_ancestor(PatchSet * ps, PatchSet * head_ps); static void handle_collisions(); +static void move_branchpoints(); + int main(int argc, char *argv[]) { debuglvl = DEBUG_APPERROR|DEBUG_SYSERROR|DEBUG_APPMSG1; @@ -224,6 +226,8 @@ int main(int argc, char *argv[]) ps_counter = 0; walk_all_patch_sets(assign_patchset_id); + move_branchpoints(); + handle_collisions(); resolve_global_symbols(); @@ -2689,3 +2693,73 @@ void walk_all_patch_sets(void (*action)(PatchSet *)) action(ps); } } + +/* Given a revision number like A.B.C, remove the last part leaving just + * A.B. + */ +static char *strip_last_part(const char *revstr) +{ + char *s; + + s = strdup(revstr); + *strrchr(s, '.') = '\0'; + return s; +} + +/* For each branch, find the branchpoint, and move the first commit on the + * branch right after the branch point. + */ +static void move_branchpoints(void) +{ + struct hash_entry * he; + + reset_hash_iterator(branch_heads); + while ((he = next_hash_entry(branch_heads))) + { + char *branch = he->he_key; + PatchSet *head_ps = (PatchSet*)he->he_obj; + struct hash_entry * filehe; + int branchpoint = -1; + + /* loop all files on the branch, and find the last patchset on the + * ancestor branch that introduces a revision needed for the branch + */ + reset_hash_iterator(file_hash); + while ((filehe = next_hash_entry(file_hash))) + { + CvsFile *file = (CvsFile *) filehe->he_obj; + char *revstr = (char *)get_hash_object(file->branches_sym, branch); + CvsFileRevision *rev; + + if (revstr == NULL) + continue; + + revstr = strip_last_part(revstr); + + rev = (CvsFileRevision *)get_hash_object(file->revisions, revstr); + debug(DEBUG_STATUS, "branch %s file %s rev %s ps %d", + branch, rev->file->filename, revstr, + rev->post_psm->ps->psid); + /* If it's a new file in this branch, ignore it */ + if (rev->post_psm->pre_rev != NULL && + (branchpoint == -1 || rev->post_psm->ps->psid > branchpoint)) + branchpoint = rev->post_psm->ps->psid; + free(revstr); + } + debug(DEBUG_STATUS, "branch %s branchpoint %d", branch, branchpoint); + + /* + * Move the first patchset in the new branch right after the + * branchpoint. + */ + while(head_ps->psid > branchpoint + 1) + { + PatchSet *prev = list_entry(head_ps->all_link.prev, PatchSet, all_link); + list_del(&head_ps->all_link); + list_add(&head_ps->all_link, prev->all_link.prev); + + head_ps->psid--; + prev->psid++; + } + } +}
On Mon, 2008-12-22 at 19:40 +0200, Heikki Linnakangas wrote: > I hacked cvsps a little bit, to move the first commit on the branch > right after the correct branch point, before writing the output. This is > a hack, but seems to work at least for the PG repository. > > I'm not sure if we want to run a hacked version of cvsps. But then > again, the current conversion isn't exactly error-free either, and we're > missing tags... Do you think this hack (or at least a problem report) would be accepted upstream by the git community? Regards,Jeff Davis
On Thursday 25 December 2008 01:47:49 Jeff Davis wrote: > On Mon, 2008-12-22 at 19:40 +0200, Heikki Linnakangas wrote: > > I hacked cvsps a little bit, to move the first commit on the branch > > right after the correct branch point, before writing the output. This is > > a hack, but seems to work at least for the PG repository. > > > > I'm not sure if we want to run a hacked version of cvsps. But then > > again, the current conversion isn't exactly error-free either, and we're > > missing tags... > > Do you think this hack (or at least a problem report) would be accepted > upstream by the git community? cvsps is not part of git. It's a separate package, that has been unmaintained for a great while, and there are already some forks out there. Which is part of the problem, overall ...
Peter Eisentraut wrote: > On Thursday 25 December 2008 01:47:49 Jeff Davis wrote: >> On Mon, 2008-12-22 at 19:40 +0200, Heikki Linnakangas wrote: >>> I hacked cvsps a little bit, to move the first commit on the branch >>> right after the correct branch point, before writing the output. This is >>> a hack, but seems to work at least for the PG repository. >>> >>> I'm not sure if we want to run a hacked version of cvsps. But then >>> again, the current conversion isn't exactly error-free either, and we're >>> missing tags... >> Do you think this hack (or at least a problem report) would be accepted >> upstream by the git community? > > cvsps is not part of git. It's a separate package, that has been unmaintained > for a great while, and there are already some forks out there. Which is part > of the problem, overall ... Yep. The git people are certainly aware of the problem, there's plenty of discussion about it on the mailing lists. I don't know if this particular hack has been proposed before, I don't think I saw anything identical in the mailing lists, though people have posted cvsps hacks to fix it before. Another problem with git-cvsimport is that only branches that have had any commits to them show up in the GIT repository. This hack won't fix that. If cvsps injected a dummy "branch commit" at the right place, instead of the reordering that my hack does, that might be fixed too. What are we going to do about this? I have the feeling that I have the greatest itch to get the repository fixed. I propose that I will try that dummy "branch commit" hack in cvsps, and if it works out, create a new GIT repository using that. That would replace the current repository, which means that people working against that repository will need to use "git-fetch --force" to update their clones, and rebase their own branches. It would also be good to contribute the hack into upstream cvsps (whatever fork is considered upstream). I'd like to move to git-cvsimport because: - it converts tags - I'm more familiar with it than fromcvs/togit. However, if anyone else is willing to pick up the ball, I'll just shut up and be happy that something gets done. One option is to just leave it as it is, but it's not nice that the history is repeated multiple times. The repository is big enough without that.. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
On Monday 29 December 2008 11:40:41 Heikki Linnakangas wrote: > What are we going to do about this? I have the feeling that I have the > greatest itch to get the repository fixed. I propose that I will try > that dummy "branch commit" hack in cvsps, and if it works out, create a > new GIT repository using that. That would replace the current > repository, which means that people working against that repository will > need to use "git-fetch --force" to update their clones, and rebase their > own branches. It would also be good to contribute the hack into upstream > cvsps (whatever fork is considered upstream). Does your version of cvsps not run into infinite loops on our CVS repository? This is one of the main reasons we are not using it.
Heikki Linnakangas wrote: > Peter Eisentraut wrote: >> On Thursday 25 December 2008 01:47:49 Jeff Davis wrote: >>> On Mon, 2008-12-22 at 19:40 +0200, Heikki Linnakangas wrote: >>>> I hacked cvsps a little bit, to move the first commit on the branch >>>> right after the correct branch point, before writing the output. >>>> This is >>>> a hack, but seems to work at least for the PG repository. >>>> >>>> I'm not sure if we want to run a hacked version of cvsps. But then >>>> again, the current conversion isn't exactly error-free either, and >>>> we're >>>> missing tags... >>> Do you think this hack (or at least a problem report) would be accepted >>> upstream by the git community? >> >> cvsps is not part of git. It's a separate package, that has been >> unmaintained for a great while, and there are already some forks out >> there. Which is part of the problem, overall ... > > Yep. The git people are certainly aware of the problem, there's plenty > of discussion about it on the mailing lists. I don't know if this > particular hack has been proposed before, I don't think I saw anything > identical in the mailing lists, though people have posted cvsps hacks to > fix it before. > > Another problem with git-cvsimport is that only branches that have had > any commits to them show up in the GIT repository. This hack won't fix > that. If cvsps injected a dummy "branch commit" at the right place, > instead of the reordering that my hack does, that might be fixed too. Do we *have* any branches that have had no commits? And in this case, do we *care* that they don't show up? I assume they will show up as soon as we get the first commit? > What are we going to do about this? I have the feeling that I have the > greatest itch to get the repository fixed. I propose that I will try > that dummy "branch commit" hack in cvsps, and if it works out, create a > new GIT repository using that. That would replace the current > repository, which means that people working against that repository will > need to use "git-fetch --force" to update their clones, and rebase their > own branches. It would also be good to contribute the hack into upstream > cvsps (whatever fork is considered upstream). Sounds like a reasonable plan to me. We should be careful to test out all parts before we do it, though, so users don't have to rebase more than once! > I'd like to move to git-cvsimport because: > - it converts tags > - I'm more familiar with it than fromcvs/togit. Can't comment on this. > However, if anyone else is willing to pick up the ball, I'll just shut > up and be happy that something gets done. > > One option is to just leave it as it is, but it's not nice that the > history is repeated multiple times. The repository is big enough without > that.. I'd really like to see us do something about it. I don't think just leaving it as-is is a good solution. //Magnus
Peter Eisentraut wrote: > Does your version of cvsps not run into infinite loops on our CVS repository? > This is one of the main reasons we are not using it. Ah, no, I worked around that by removing REL7_1_BETA tags before the conversion. Sorry, forgot to mention. I haven't investigated why those are causing problems, because I don't care about the 7.1 beta tags much. So the procedure is to: 1. rsync PostgreSQL main repository. 2. Remove offending tags: cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA2 pgsql cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA3 pgsql cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA pgsql 3. Run git-cvsimport. BTW, if we ever convert to git altogether as a project, we'd probably use something else again for the conversion. Many people recommend cvs2git on the git mailing lists. We can't use it for the mirror because it doesn't support incremental updates, but for a one-off conversion it's probably the best option. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Magnus Hagander wrote: > Heikki Linnakangas wrote: >> Another problem with git-cvsimport is that only branches that have had >> any commits to them show up in the GIT repository. This hack won't fix >> that. If cvsps injected a dummy "branch commit" at the right place, >> instead of the reordering that my hack does, that might be fixed too. > > Do we *have* any branches that have had no commits? No, but we will as soon as we branch again. > And in this case, do > we *care* that they don't show up? I assume they will show up as soon as > we get the first commit? I guess we could live with them not showing up until first commit. >> What are we going to do about this? I have the feeling that I have the >> greatest itch to get the repository fixed. I propose that I will try >> that dummy "branch commit" hack in cvsps, and if it works out, create a >> new GIT repository using that. That would replace the current >> repository, which means that people working against that repository will >> need to use "git-fetch --force" to update their clones, and rebase their >> own branches. It would also be good to contribute the hack into upstream >> cvsps (whatever fork is considered upstream). > > Sounds like a reasonable plan to me. We should be careful to test out > all parts before we do it, though, so users don't have to rebase more > than once! Yep. Need to test what happens when new branches are created, at least. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Heikki Linnakangas wrote: > So the procedure is to: > > 1. rsync PostgreSQL main repository. > 2. Remove offending tags: > > cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA2 pgsql > cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA3 pgsql > cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA pgsql Can we permanently hack our repo? It doesn't seem like those tags have a lot of value, after all. (If somebody thinks they do, perhaps we can re-tag.) -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
On Mon, Dec 29, 2008 at 01:14:19PM +0200, Heikki Linnakangas wrote: > Peter Eisentraut wrote: > > BTW, if we ever convert to git altogether as a project, Say, around the time we branch to 8.5? :) Cheers, David. -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
Alvaro Herrera wrote: > Heikki Linnakangas wrote: > >> So the procedure is to: >> >> 1. rsync PostgreSQL main repository. >> 2. Remove offending tags: >> >> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA2 pgsql >> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA3 pgsql >> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA pgsql > > Can we permanently hack our repo? It doesn't seem like those tags have > a lot of value, after all. (If somebody thinks they do, perhaps we can > re-tag.) I don't know. It's not a big deal, either way. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Heikki Linnakangas wrote: > Alvaro Herrera wrote: >> Heikki Linnakangas wrote: >> >>> So the procedure is to: >>> >>> 1. rsync PostgreSQL main repository. >>> 2. Remove offending tags: >>> >>> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA2 pgsql >>> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA3 pgsql >>> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA pgsql >> >> Can we permanently hack our repo? It doesn't seem like those tags have >> a lot of value, after all. (If somebody thinks they do, perhaps we can >> re-tag.) > > I don't know. It's not a big deal, either way. Or if we can figure out *what* is wrong with them, perhaps we can fix it? it is some level of historical information that we'd loose, after all. Granted not very important historical information, but if it's fixable there's no reason not to do that instead. //Magnus
On Monday 29 December 2008 15:25:47 Magnus Hagander wrote: > Heikki Linnakangas wrote: > > Alvaro Herrera wrote: > >> Heikki Linnakangas wrote: > >>> So the procedure is to: > >>> > >>> 1. rsync PostgreSQL main repository. > >>> 2. Remove offending tags: > >>> > >>> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA2 pgsql > >>> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA3 pgsql > >>> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA pgsql > >> > >> Can we permanently hack our repo? It doesn't seem like those tags have > >> a lot of value, after all. (If somebody thinks they do, perhaps we can > >> re-tag.) > > > > I don't know. It's not a big deal, either way. > > Or if we can figure out *what* is wrong with them, perhaps we can fix > it? it is some level of historical information that we'd loose, after > all. Granted not very important historical information, but if it's > fixable there's no reason not to do that instead. I analyzed this back in April; my analysis is in the archives. The short version is that it is probably a bug in cvsps. See also my message to hackers just now about a repository fixup that we *can* do.
On Mon, Dec 29, 2008 at 3:14 AM, Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> wrote: > Peter Eisentraut wrote: > BTW, if we ever convert to git altogether as a project, we'd probably use > something else again for the conversion. Many people recommend cvs2git on > the git mailing lists. We can't use it for the mirror because it doesn't > support incremental updates, but for a one-off conversion it's probably the > best option. > Do you have any sentiments on how good git-cvsserver is? Synopsis: basically git can run as a CVS server, allowing for both read and write access, in addition to being a git repository. Some of the more advanced functionality (tagging, branching) may have to be done with a git client, though. By its own description in the man pages, git cvsserver is 'highly functional'. I don't know what that means, but since nobody is trying to sell me anything it seems like it could be true. One interesting exercise would be to run git with cvsserver and try porting commits via script from Postgres CVS to git cvsserver. If that worked convincingly for a while, perhaps the committers wouldn't mind switching the CVS backend since daily life would not have to change drastically at any one point in time. fdr
On Monday 29 December 2008 19:09:23 Daniel Farina wrote: > One interesting exercise would be to run git with cvsserver and try > porting commits via script from Postgres CVS to git cvsserver. But to do that you would be back at cvsps to extract the actual commits from the CVS repository, which is a major part of the current problem in the first place. And by the time you're done, you'd likely have re-implemented git-cvsimport with an extra level of indirection.
On Mon, Dec 29, 2008 at 10:05 AM, Peter Eisentraut <peter_e@gmx.net> wrote: > On Monday 29 December 2008 19:09:23 Daniel Farina wrote: >> One interesting exercise would be to run git with cvsserver and try >> porting commits via script from Postgres CVS to git cvsserver. > > But to do that you would be back at cvsps to extract the actual commits from > the CVS repository, which is a major part of the current problem in the first > place. And by the time you're done, you'd likely have re-implemented > git-cvsimport with an extra level of indirection. > This is true as long as one is using some extra machinery to convert from vanilla CVS to Git (and thus served through cvsserver). The goal would be to shed those eventually and have CVS users just commit to Git's cvsserver. I only suggested exercising through cvsserver to make sure it works 'as advertised,' for Postgres' purposes, making a convincing argument that anyone who wanted to operate via CVS for a while would not be adversely affected. There's also a benefit for the committers who would *like* to use Git being able to take advantage of its features in that scenario. As-is just fixing cvsps (and cvsimport) doesn't provide an incremental strategy for switching to Git for the purposes of writes to the repository. fdr
Daniel Farina wrote: > There's also a benefit for the committers who would *like* to use Git being > able to take advantage of its features in that scenario. As-is just fixing > cvsps (and cvsimport) doesn't provide an incremental strategy for switching to > Git for the purposes of writes to the repository. That's hardly worth worrying about. There's only a handful of active committers. Coordinating a big-bang switch wouldn't be hard after the decision to convert was made. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:
> Daniel Farina wrote:
>> There's also a benefit for the committers who would *like* to use Git being
>> able to take advantage of its features in that scenario. As-is just fixing
>> cvsps (and cvsimport) doesn't provide an incremental strategy for switching to
>> Git for the purposes of writes to the repository.
> That's hardly worth worrying about. There's only a handful of active
> committers. Coordinating a big-bang switch wouldn't be hard after the
> decision to convert was made.
The problem is to convince those active committers that they want to switch.
regards, tom lane
On Mon, 2008-12-29 at 20:05 +0200, Peter Eisentraut wrote: > On Monday 29 December 2008 19:09:23 Daniel Farina wrote: > > One interesting exercise would be to run git with cvsserver and try > > porting commits via script from Postgres CVS to git cvsserver. > > But to do that you would be back at cvsps to extract the actual commits from > the CVS repository, which is a major part of the current problem in the first > place. And by the time you're done, you'd likely have re-implemented > git-cvsimport with an extra level of indirection. How well does CVS -> SVN work? Maybe we could do CVS -> SVN -> Git. CommandPrompt's SVN repo seems like it's in good shape (has tags, etc.), so maybe this is possible? Regards,Jeff Davis
On Mon, 2008-12-29 at 11:31 -0800, Jeff Davis wrote: > On Mon, 2008-12-29 at 20:05 +0200, Peter Eisentraut wrote: > > On Monday 29 December 2008 19:09:23 Daniel Farina wrote: > > > One interesting exercise would be to run git with cvsserver and try > > > porting commits via script from Postgres CVS to git cvsserver. > > > > But to do that you would be back at cvsps to extract the actual commits from > > the CVS repository, which is a major part of the current problem in the first > > place. And by the time you're done, you'd likely have re-implemented > > git-cvsimport with an extra level of indirection. > > How well does CVS -> SVN work? Maybe we could do CVS -> SVN -> Git. > > CommandPrompt's SVN repo seems like it's in good shape (has tags, etc.), > so maybe this is possible? CVS to SVN is easy and accurate but slow. There is a bug in our transformation script (which has nothing to do with the cvs-svn part) but that is why ours is out of date. I have been meaning to look at tailor to see if it is possible to convert single commits. Right now, we have to transform the entire repo everytime we run. Joshua D. Drake -- PostgreSQL Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company,serving since 1997
On Mon, 2008-12-29 at 11:33 -0800, Joshua D. Drake wrote: > I have been meaning to look at tailor to see if it is possible to > convert single commits. Right now, we have to transform the entire repo > everytime we run. > Oh, nevermind. Incremental updates were already stated as a requirement elsewhere in this thread. Regards,Jeff Davis
On Mon, 2008-12-29 at 11:41 -0800, Jeff Davis wrote: > On Mon, 2008-12-29 at 11:33 -0800, Joshua D. Drake wrote: > > I have been meaning to look at tailor to see if it is possible to > > convert single commits. Right now, we have to transform the entire repo > > everytime we run. > > > > Oh, nevermind. Incremental updates were already stated as a requirement > elsewhere in this thread. Right that is why I have been meaning to look at Tailor. Supposedly it can do it proper. http://progetti.arstecnica.it/tailor/wiki/ Joshua D. Drake > > Regards, > Jeff Davis > > -- PostgreSQL Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company,serving since 1997
-----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160 > Right that is why I have been meaning to look at Tailor. Supposedly it > can do it proper. FWIW, I looked into Tailor some time ago (> 6 months), as well as other conversion tools, including all the ones mentioned in this thread, but none of them worked well, and those that sorta worked (like our current ruby-based one) were *incredibly* slow. I seem to recall it takes > 12 hours for the initial import. I really like the idea of removing those old no-longer-needed tags that were causing some of the problems, and then seeing if we can find a fast, reliable conversion tool. The git community is very helpful as well, we should feel free to enlist their help. By "we" I mean someone else, as I'm on vacation from tomorrow until the 7th. ;) - -- Greg Sabino Mullane greg@turnstep.com PGP Key: 0x14964AC8 200812291833 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -----BEGIN PGP SIGNATURE----- iEYEAREDAAYFAklZXqIACgkQvJuQZxSWSsgiTgCfbt/9eJKbx6FJt0nJPBjgzgYt 2pgAn1ANRPolreroAltsEK4FosYjpNiq =qrQL -----END PGP SIGNATURE-----
Heikki Linnakangas wrote: > I propose that I will try > that dummy "branch commit" hack in cvsps, and if it works out, create a > new GIT repository using that. That would replace the current > repository, which means that people working against that repository will > need to use "git-fetch --force" to update their clones, and rebase their > own branches. It would also be good to contribute the hack into upstream > cvsps (whatever fork is considered upstream). > > I'd like to move to git-cvsimport because: > - it converts tags > - I'm more familiar with it than fromcvs/togit. I tried that, but gave up after a while. I hacked cvsps to create the dummy commits, but it choked on some files that exist some branches but not others. It might be fixable, but I'm now convinced that fromcvs/togit is the simplest solution after all. The lack of tags is slightly annoying, but not a show-stopper. And we can add them manually if we really want them, we don't tag that often. togit normally looks at all the branches, and retrieves all commits from CVS with date later than the latest commit on any branch in the GIT repository. I think that's why the back-branches stopped updating after there was even a single commit on master with later date. I modified it so that each branch is treated separately. I'm not sure how we ended up with duplicated history, though. I have a recipe to fix the repository, and we don't even need to recreate it from scratch. I'd appreciate that, because I have some pretty complex stuff in my GIT repository with merges and everything that could be a lot of work to rebase. This fixes the back-branches, too. 0. Backup 1. Apply the attached patch to togit. 2. Use git-reset to rewind the branches back to the state they were before breakage. Attached rewind-branches.sh script does that for all the branches. 3. Run togit. After that, pulling from a repository that was last updated before Dec 13th (when it was screwed up), you get a nice fast-forward as usual. If you have already pulled the duplicated history, you have to use "git-pull --force" or git-pull will refuse to update because "history has changed". It should be safe, but not necessaery, to always use the modified togit script. After fixing the repository, incremental updates should work with unmodified togit as well, I think. The patched version is slower, because it skips already-applied commits at a later stage, but most of the time is spent in parsing the RCS logs anyway, so it might not be important. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com Common subdirectories: fromcvs-9679642582bc/commitset and fromcvs/commitset Common subdirectories: fromcvs-9679642582bc/t and fromcvs/t diff -u fromcvs-9679642582bc/togit.rb fromcvs/togit.rb --- fromcvs-9679642582bc/togit.rb 2008-12-01 02:28:19.000000000 +0200 +++ fromcvs/togit.rb 2008-12-30 11:09:40.000000000 +0200 @@ -31,6 +31,7 @@ @deleted = [] @modified = [] @branchcache = {} + @last_dates = {} @files = Hash.new{|h,k| h[k] = {}} @mark = 0 @@ -47,13 +48,31 @@ next if type != 'commit' branch[/^.*\//] = "" @branchcache[branch] = sha + @last_dates[branch] = _get_last_date(branch); + puts @last_dates[branch] end @pickupbranches = @branchcache.dup end + # All commits before this date is skipped when reading the CVS repository. + # In incremental mode, should be the date of *last* commit on any branch + # in the GIT repository. However, with this patch, we don't assume that + # all branches have been updated up to the same point in time. Therefore, + # take the *minimum* date of branch tips in GIT. + # + # In _commit function below, we check that we don't try to commit anything + # to a branch with older date the tip commit on the branch. Otherwise + # we'd get duplicate commits. def last_date + return @last_dates.values.min + #return @last_dates.values.max # use this for the normal incremental mode + end + + def _get_last_date(branch) + branch = 'refs/heads/' + branch + puts 'requesting ' + branch latestref = _command(*%w{git for-each-ref --count=1 --sort=-committerdate - --format=%(refname) refs/heads}) + --format=%(refname) } + [branch]) log = _command('git', 'cat-file', '-p', latestref.strip) log.split("\n").each do |line| break if line.empty? @@ -148,7 +167,9 @@ end def commit(author, date, msg, revs) - _commit(author, date, msg, revs) + if date > @last_dates[@curbranch] + _commit(author, date, msg, revs) + end end def merge(branch, author, date, msg, revs) Only in fromcvs: togit.rb~