Bug #1: Home directory is not version controlled

How to reproduce: Modify dotfiles and scripts in the home directory on multiple machines without keeping track of the changes.

What happens:

  • Lots of manual work to synchronize and merge changes.
  • Uncertainty about which changes exist where.
  • Lost work because of minor mistakes or giving up on complex merges.

What should happen: Changes should be reproducible, visible and simple enough to be merged.

How to fix: Use version control.

Workaround:

  1. Fork an existing version controlled home directory.
  2. git clone --recursive git://github.com/your-user/tilde.git
  3. Merge with your existing home directory.
  4. make clean to do miscellaneous cleanup before you
  5. commit and push.
  6. make install to create symbolic links from your home directory to the repository.
  7. clone and pull on any machines which need your changes.

That’s pretty much all there is to this workflow, really. There’s a ton of commands with descriptive tags in .bash_history, configuration for Bash, Vim, Awesome WM, screen layouts, email tools, and much more that you can copy (and criticize) all you want.

Fix Git repository after Subversion conversion

After converting a Git repository from Subversion with svn2git, all was well. At least until I wanted to squash some of the oldest commits with the excellent interactive rebase. Full of fail, I realized that I might have to do some cleanup before going on such a bold expedition. Here’s a couple tricks for “post-processing” a converted repository.

  1. Make a backup! Ideally, do this work on a clone of the original repository, then use a tool like Meld (or simply diff -r repo_backup new_repo) to check that the resulting files are the same as before.
  2. Remove empty commits: git filter-branch --commit-filter 'if [ z$1 = z`git rev-parse $3^{tree}` ]; then skip_commit "$@"; else git commit-tree "$@"; fi' "$@"
  3. Garbage collect and prune: git gc --prune
  4. Start interactive rebase from the first commit: git rebase -i $(git log --format=%H | tail -1)
  5. Squash all commits with empty messages. These are shown as multiple commit IDs on the same line with a space and > between them, like this:
    pick 1111111 >2222222 >3333333 Message

    Just split these up and remove the commits, like this:

    fixup 1111111 nothing
    fixup 2222222 nothing
    pick 3333333 Message

    In Vim, you can do this by repeating the following command until it reports no hits: :%s/^pick \([a-f0-9]\{7\}\) >\([a-f0-9]\{7\}\)/fixup \1 nothing\rpick \2/g

  6. Exit the editor, and the rebase should complete on its own.

Warning, as always: YMMV and RTFM.

cvs2git2svn

After discovering Ohloh, cleaning up and publishing repositories of yore seemed like a good idea. One of them was established back in the CVS newbie days, and contained lots of external binaries – Not the kind of thing you want to version control. Having used CVS, Subversion and Git (in that order), there was only one choice: Interactive rebase with Git. Also, the software was created while at CERN, so it should continue to be hosted there. And they had started a Subversion service in the meantime, so it was time to upgrade as well.

These instructions should fit for any CERN project, and can easily be modified to fit any repository. The usual warnings apply: YMMV and RTFM.

  1. Set some variables to avoid typing: svn_repo=Repository_name
    svn_user=User_with_edit_access
  2. Install the tools: sudo apt-get install cvs2svn git-core git-svn
  3. Create the cvs2git working directory: cvs2git_wd=$(mktemp -dt cvs2git.XXXXXXXXXX)
  4. Copy the contents of the repository (not a working copy) to the working directory: scp -r $svn_user@lxplus.cern.ch:/afs/cern.ch/project/svn/reps/${svn_repo}/* $cvs2git_wd. Don’t worry if /hooks is not copied – You don’t need it. If you don’t have filesystem access to the repository, you can try cvssuck. Be warned: It’s really slow.
  5. Set cvs2git global options:
    1. zcat /usr/share/doc/cvs2svn/examples/cvs2git-example.options.gz > $cvs2git_wd/cvs2git.options
    2. Modify at least ctx.username and author_transforms in $cvs2git_wd/cvs2git.options.
  6. Make the new Git repository: git_wd=$(mktemp -dt git.XXXXXXXXXX) && git init $git_wd
  7. Convert to Git (repeat for each module):
    1. Modify run_options.set_project in $cvs2git_wd/cvs2git.options
    2. Create Git import files: cd $cvs2git_wd && cvs2git --options=cvs2git.options. If you get any warnings or errors you might have to change the options again.
    3. Import to Git: cd $git_wd && cat $cvs2git_wd/cvs2svn-tmp/git-blob.dat $cvs2git_wd/cvs2svn-tmp/git-dump.dat | git fast-import
  8. Make a backup in case the rest goes hairy.
  9. If you need to (which was kind of the point of this exercise), do an interactive rebase from the first commit: git rebase -i $(git log --format=%H | tail -1).
  10. git-svn needs at least one commit to be in the Subversion repository: svn_wd=$(mktemp -dt svn.XXXXXXXXXX) && svn co --username $svn_user svn+ssh://${svn_user}@svn.cern.ch/reps/${svn_repo} $svn_wd && cd $svn_wd && touch .temp && svn add .temp && svn ci -m "git-svn dummy commit"
  11. Convert to Subversion:
    1. Prepare git-svn repository: git2svn_wd=$(mktemp -dt git2svn.XXXXXXXXXX) && git svn clone --username $svn_user svn+ssh://${svn_user}@svn.cern.ch/reps/${svn_repo} $git2svn_wd && cd $git2svn_wd
    2. Get Git commits: git fetch $git_wd
    3. Apply Git commits as master branch: git branch tmp $(cut -b-40 .git/FETCH_HEAD) && git tag -am "Last fetch" last tmp && first_commit=$(git log --format=%H | tail -1) && git checkout $first_commit . && git commit -C $first_commit
    4. Apply Git commits: git rebase master tmp && git branch -M tmp master
    5. Check if this works : git svn dcommit --rmdir --find-copies-harder --dry-run
    6. If it does, you’re good to go: git svn dcommit --rmdir --find-copies-harder

If the last step fails, the easiest way to continue is just to remove all commits from the Subversion repository, fix the Git repository, and restart at step 10.