Martin F. Krafft: Converting a package to Git
Previously, I demonstrated a Debian packaging workflow using Git and
I mentioned the possibility of a follow-up post; well, here it is: you want to
use my workflow (or one that's related) for a package that is currently
maintained with Subversion on
svn.debian.org and you'd like to keep the history
during the conversion.
Make sure to read the previous post before this
one.
I am again using the example of mdadm since its Git packaging repository is in a state of
shambles and I want to restart to get it right and import the history from
the previous Subversion repository.
What better way than to write a blog post as I do so? Well, plenty actually.
This kind of post isn't really made for a blog, and I have
started work on setting up ikiwiki on madduck.net, but it's not yet ready, so I'll stick with the blog
for now. I will make sure that links don't break as I move content over, so
feel free to bookmark this
Importing the package into Git
Thanks to git-svn, the
initial step of getting your package imported into Git is a breeze:
$ git-svn clone --stdlayout --no-metadata \
svn+ssh://svn.debian.org/svn/pkg-mdadm/mdadm mdadm
Sit back and enjoy. If that command exits prematurely with an error such as
the following:
Malformed network data: Malformed network data at /usr/local/bin/git-svn line 1029then you should upgrade to a newer Git version, or have a look here. If your Git does not know --stdlayout then upgrade as well (or use -T trunk -t tags -b branches instead). Sam Vilain notes that it is important to "get the attribution right with the final SVN import - getting the authors map right. I didn't do that. If you look at the repository resulting from the above command, you'll notice strange commit authors, such as madduck@some-unique-uuid-from-svn . git-svn allows you to map these to real names with real email addresses, which ensures that the attributions are good for the whole world to see. When done, switch to the repository and run git-branch -r. As you'll see, git-svn imported all SVN branches and tags as remote branches. You need those if you want to bidirectionally track the Subversion repository, but we are converting, as you may have guessed by the --no-metadata switch above. Therefore, we resort to the Dinosaur method of converting branches to tags, which I'll simplify for mdadm. We also just delete all remote branches after tagging, since mdadm never used branches in the SVN repository. Your mileage may vary.
git branch -r sed -rne 's, *tags/([^@]+)$,\1,p' while read tag; do echo "git tag debian/$tag tags/$ tag ^; git branch -r -d tags/$tag" done git branch -r while read tag; do echo "git branch -r -d $tag" doneIf that seems to work alright, then you can execute the commands. Sam Vilain (again) hints me at git-pack-refs and then to edit .git/packed-refs with an editor. This certainly leaves more room for errors but might be significantly faster.
Cleaning up the SVN references
Even though we passed --no-metadata to git-svn, it did leave some
traces in .git/, which we can now safely remove:
$ git config --remove-section svn-remote.svn $ rm -r .git/svn
Setting things straight
You can skip this section unless you want to know a bit about how to fix up
stuff with Git.
There was actually some nasty tagging errors leading up to the 2.5.6-9
release for etch and I could never be bothered to fix those in SVN,
but now I can (I love Git!):
$ git tag -d debian/2.5.6-10 # never existed $ git tag -f debian/2.5.6-8 2.5.6-8~2 # mistagged $ git checkout -b maint/etch 2.5.6-8 # this is when we diverged $ git apply < /tmp/mdadm-2.5.6-8..2.5.6-9.diff $ git add debian/po/gl.po debian/po/pt.po debian/changelog $ git commit -s $ git tag debian/2.5.6-9Now that that's fixed, there is one other thing to worry about, namely the very last commit to SVN, which obsoletes the repository and points to the Git repository. But that's not all of it. I was also silly enough to include a fix in the same commit. Let's see what Git can do. Since the process of obsoletion involves all but adding a file, we can simply --amend the last commit and provide a new log message:
$ git checkout master $ git rm OBSOLETE debian/OBSOLETE $ git commit --amendNow the repository is in an acceptable state.
Making ends meet
The pkg-mdadm effort on svn.debian.org only
maintained the ./debian/ directory, separate from the upstream code, and boy
was that a bad idea. Just to give one example: think about what's involved in
preparing a Debian-specific patch against the upstream code this has to end,
and we can make it end right here; let's import upstream's code (again not
using his ADSL line, but the upstream branch of the pkg-mdadm Git
repository; see the previous post for
details):
$ git remote add upstream-repo git://git.debian.org/git/pkg-mdadm/mdadm
$ git config remote.upstream-repo.fetch \
+refs/heads/upstream:refs/remotes/upstream-repo/upstream
$ git fetch upstream-repo
$ git checkout -b upstream upstream-repo/master
Now we have two unconnected ancestries in our repository, and it's time to
join them together. The most logical way seems to be to use the last upstream
tag for which we have a Debian tag: 2.6.2.
For this, we branch off the corresponding Debian tag (2.6.2-1) and merge
upstream's 2.6.2 tag into the new branch. This will be a temporary branch
Then, we rebase (remember, nothing has been published yet) the master branch
on top of this temporary branch, before we end that branch's short life. The
Debian tag stays where it is since it describes the state of the repository at
time of the release of 2.6.2-1.
$ git checkout -b tmp/join debian/2.6.2-1 $ git merge mdadm-2.6.2 $ git rebase tmp/join master $ git branch -d tmp/joinIt just so happens that the head of the SVN repository, which is identical to the tip of our master branch, corresponds to Debian release 2.6.2-2, so we tag it:
$ git tag debian/2.6.2-2We are now also "born" in the sense that maintenance in Git has started. Let's mark that point in history. There is no real reason I can foresee for this yet, but nonetheless:
$ git tag -s git-birth
Turning dpatch files into feature branches
We want to turn dpatch files into feature branches and we somehow make it
"proper". We could branch, apply the patch, delete the patch file, checkout
master and delete the patch file there as well, but that appears
"improper" to me at least; so instead, we'll cherry-pick:
$ git checkout -b deb/conffile-location $ debian/patches/01-mdadm.conf-location.dpatch -apply $ git rm debian/patches/01-mdadm.conf-location.dpatch $ git commit -s $ git commit -s $(git ls-files --others --modified)I should quickly intervene to make sure you are following. I am making use of Git's index here. Applying the patch makes the changes in the working tree, but we did not tell Git that we want those to be part of the commit just yet. Instead, we delete the dpatch with git-rm, which automatically registers the deletion with the index. Thus, the first git-commit creates a commit which deletes the dpatch, while the second git-commit creates a commit with all the changes from the dpatch, using git-ls-files to identify new and modified files. But for now, let's move on. We have two commits in the deb/conffile-location branch, and one of those is relevant to the master branch, we cherry-pick it:
$ git cherry-pick deb/conffile-location^If you're confused, let me explain: our goal is to have a number of feature branches, of which master is the one in which most of ./debian/ is maintained. All the branches later come together in the long-living build branch, so deb/conffile-location will never be merged back into master. However, once we applied the dpatch to the feature branch, we can delete it from there and the master branch. By cherry-picking, we "import" the deletion to the master branch. I repeat the same procedure for deb/docs, merging all the documentation-related dpatches, but I'll spare you the details.
and then Git let me down
In the next step, I found I had misunderstood Git merging: I thought Git was
smart, but Linus had his reasons for calling Git the "stupid content tracker"
(more on that later). Read on as I am obsoleting dpatch files that
upstream had merged: 99-*-FIX.dpatch.
For consistency, I wanted to cherry-pick each of the appropriate upstream
commits into the master branch along with deleting the corresponding
dpatch file. Here is one example: 99-monitor-6+10-FIX.dpatch was
obsoleted by upstream's commit 66f8bbb; the -x records the original
commit ID in the log:
$ git cherry-pick -x 66f8bbb $ git rm debian/patches/99-monitor-6+10-FIX.dpatch $ git commit -s -m"remove dpatch obsoleted by $(git rev-parse --short HEAD)"I repeated the procedure for the other dpatch files, removed the dpatch infrastructure, and then went on to merge it all into build to build the package. The build branch is a long-living branch off upstream, but which upstream? I'll fast-forward you past a segfault problem with mdadm, which upstream (thought to have) resolved with commit 23dc1ae after 2.6.3, but he had not yet released 2.6.4. Looking at the commits between 23dc1ae and upstream's HEAD at the time, I decided to include them all and snapshot 4450e59:
$ git fetch upstream-repo $ git checkout upstream $ git merge upstream-repo $ git tag mdadm-2.6.3+200709292116+4450e59 4450e59 $ git checkout master $ git merge --no-commit mdadm-2.6.3+200709292116+4450e59 $ dch -v mdadm-2.6.3+200709292116+4450e59-1 $ git add debian/changelog $ git commit -sAnd then I called poor-mans-gitbuild, which merges master and then deb/* into build. Here is when stuff blew up. I'll make a long story short (read my description of the problem and Linus' answer if you want to know more): I thought Git was smart to identify merges common to both branches and do the right thing, but it turn out that Git does not care at all about commits, it only worries about content and the end result. In our case, unfortunately (or fortunately), the outcome meant a conflict because the upstream branch introduced a simple change (last hunk) in the lines surrounding the patch we cherry-picked, and Git can't handle it. The solution is not to cherry-pick, to cherry-pick all commits touching the context of the dpatch, or to simply merge upstream into all out feature branches. In our case, the first is the easiest solution and since importing dpatch files is a one-time thing (thank $DEITY), I'll leave it at that. Almost. I have spent two days thinking about this more than I should have. And it was this point Linus made which made me appreciate Git even more:
Conflicts aren't bad - they're good. Trying to aggressively resolve them automatically when two branches have done slightly different things in the same area is stupid and just results in more problems. Instead, git tries to do what I don't think anybody else has done: make the conflicts easy to resolve, by allowing you to work with them in your normal working tree, and still giving you a lot of tools to help you see what's going on.
The end
This concludes today's report. Importing the changes from the old Git repo,
tagging and merging the branches is all covered in my previous post, or at least
you'll find enough information there to complete the exercise.
I would like to specifically thank Sam Vilain and Linus Torvalds for their
help in preparing this post, as well as the #git/freenode inhabitants, as
always.
If you are interested in the topic of using version control for distro
packaging, I invite you to join the vcs-pkg mailing list and/or the
#vcs-pkg/irc.oftc.net IRC channel.
Also, if you are interested in Git in general, you can find a list of blog
posts on the Git wiki.
NP: The Police: Zenyatta Mondatta
...at least, in my opinion (which is obviously biased, let's admit that). There are several reasons for this loss:
Unlike some 
When getting this error message:
Having a new home on the web is alwais something really special for me and installing wordpress listening to some good music make me feel quite happy and ready to write a new chapter of my web-life on a new home. I’ve decided to start up a new blog to share my thoughts and opinions on a bigger area of users with the idea to link it to the existing italian one on blogspot (http://andreaveri.blogspot.com), maybe subscribing my feed to community planets such as Ubuntu and Debian one 
….but something is going wrong making it, seems something like the use of a bad syntax or even an ABI breakage on compiling it with the new libtorrent-rasterbar 0.12, I hope to find out a workaround for this; I would like to use linkage again hoping to have it fixed upstream in the near future……and I suggest everyone to browse linkage’s homepage at http://zeflunk.googlepages.com/ and maybe try it out, every istruction on having it installed on Ubuntu can be found at http://code.google.com/p/linkage/wiki/Installation, and yeah…it looks too nice to avoid his use as your main bit-torrent application…
What to say….try it out!
For once, 
I don’t buy everything he says, but in the case of DSA, the part where he says “you cannot have a functional and respectable subgroup if it maintains autonomy like that” is a real problem.
The leadership problem I mentioned is real. And it can theoretically be solved by undelegating one of the problematic side of this DSA-internal dispute. But which one? Given the unwillingness of Joey to discuss the problems, he makes an easy target… which would leave DSA up to Ryan, James and Phil.
But is that a desirable thing? If DSA is perceived as being an “autonomous” group which is not involved in Debian’s main discussions and which is somewhat disconnected from Debian’s day-to-day life, it’s largely due to the behavior of James and Ryan. E-mail communication with them is very difficult as they’ll respond only if they really care about something. And despite the setup of the request tracker, they have barely been able to make proper usage of it… the idea was to use RT tickets to track everything that DSA does but they don’t use it as such. For example, James setup a “wikiadm” group and he never reported anything to the 
I've been meaning to write this for weeks. Hopefully I can still remember
everything.
So Sarah had a c-section the day before we were scheduled to have our
c-section class. We'd both read a bit though, so it wasn't like we were
going in totally uninformed and unprepared..
Firstly, Stanford Hospital's operating room area is rather mind-blowing.
Sarah was transferred from Lucille Packard Children's Hospital to Stanford
because they could get an operating room there sooner. The transfer was
trivial, just roll the whole bed down the corridor, through the magic double
doors into the operating rooms. The doors in question have a whole heap of
video cameras pointing straight down, so they see everyone that gets wheeled
in and out, which I found interesting. Good for records of patient movements
I guess.
Anyway, through those doors is a maze of corridors. There was a light-up
board hanging from the ceiling in the middle of the main corridor with
lights for all of the ORs. I think there were about 20. The illuminated ones
were available. We went down a corridor, and past a few ORs. Outside of each
one is a big stainless-steel scrub area.
We got to "our" OR, and they transferred Sarah onto the operating table.
Because of her heart condition, they wanted to have an arterial line in her
for the duration of the procedure. This is some sort of gadget that goes
into an artery and monitors your blood pressure beat by beat.
The "interesting" thing with the anaesthesiologist was the New Zealand woman
that we'd seen when we had an anaesthesiologist months prior, but the
appointment was with a different anaesthesiologist, and this New Zealand
woman was "observing". At the time, the anaesthesiologist we had the
appointment with told us that she was going to be on vacation at the time of
the delivery, so she wouldn't be attending. So I don't know if the Kiwi
woman got promoted out of "observing" to "doing" in that period, or what,
but she reintroduced herself to us in LPCH prior to moving to Stanford as
being our anaesthesiologist for the delivery.
Anyway, in the OR, they handed her the arterial line, in it's plastic
packaging, and the first thing she asked was whether it came with anything
else to put it in. At this point I figured she hadn't put one of these in
before. (The answer to her question, by the way, was "no")
She then proceeded to have a terrible time getting the line into Sarah's
arm. Normally intravenous lines go into a vein, but this has to go into an
artery. Apparently when you go poking an artery, it spasms. It then becomes
very hard to get it into a moving target. She had a few goes on one arm,
then gave up, and had to move over to the other arm.
Sarah doesn't like needles at the best of times, and getting this honking
great big thing in was no picnic. The bruising on both her wrists afterwards
was phenomenal. She tends to bruise fairly spectacularly whenever they stick
a needle in her. I almost stepped in and asked if it was really necessary
that she have it, because they were having so much trouble getting it in,
and it was clearly hurting Sarah. I think someone else ended up getting it
in, not the anaesthesiologist. Not "that" one anyway. There were heaps of
people in the OR.
Next, came the epidural. Again, we had the inexperienced anaesthesiologist.
This time she said to someone else how she was so nervous because there were
"too many eyes". Not really reassuring. They gave Sarah a numbing injection
first, and then proceeded to poke her in back with the epidural catheter
needle thing.
Sarah was flinching when they did it, and they kept yelling at her not to
move. Apparently it didn't hurt, but it caused discomfort every
time they tried to get it in. Eventually the associate professor of
anaesthesiology stepped in and got it in. Later, they said that Sarah had a
slightly curved spine or something, and that had made it a bit more
complicated.
I'm not begrudging the obviously inexperienced anaesthesiologist for
anything. Stanford is a teaching hospital, and doctors have to get their
experience somehow, but I think that when the patient is conscious, it's
important to come across a little more confident.
So once they got all of that sorted out, they could get her lying down on
the table and get down to business.
Apparently one of the side-effects of the epidural is that it can cause your
body to think it's got hypothermia. Part way through the procedure, Sarah
started shaking uncontrollably. I can't remember whether she complained
about feeling cold or not. She also got a really itchy face. Apparently
that's a side-effect of morphine.
Sarah also doesn't tolerate anaesthetics very well, and started feeling
nauseous part way through the procedure. Sure enough, as has happened
countless times previously post-operatively, she threw up. At least we had
warning and I got a kidney dish in place in time.
That was about all the eventful surgical stuff. They had the usual screen up
from about her neck area, so I couldn't see what they were doing. After
Joshua was delivered, one of the nurses asked me if I wanted to see him, and
took me around from her left shoulder where I'd been sitting to around below
her right foot, to where they had him in one of those perspex portable crib
things. I could see a lot of red out of the corner of my eye, and I could
have seen a lot more if I'd bothered to look, but I was more interested in
seeing my son at that stage.
They let me bring him back around to where I'd been sitting previously, to
show Sarah, but it was hard getting him up to a point where she could see
him properly, and she was partially out of it by this stage. I think she
dozed off for a little bit not long after that.
I forget what time we got into the OR, but they started the operation in
earnest at about 10:00am, and we were out of there by 10:45am if memory
serves. The nurse in charge of the recovery room really didn't want me
there, so I went back to LPCH with one of the nurses to spend some time with
him while he was weighed and measured, and I took some photos, and held him.
This was good, because I got to have a good cry on my own with him.
So overall, in my "I'll never be squeezing a watermelon out of my nether
regions" blokey perspective, having seen what Sarah went through, and how
much discomfort that it looked like she was going through, both in the OR
and afterwards, I'm not convinced c-sections are any less painful than a
vaginal birth (taking the average case of both). I think you're trading some
"short term" (depends on how long your labour is I guess) pain for some
"long term" pain while you recover from your incision. Granted, Sarah wasn't
really laid up all that badly for all that long, but it still didn't look
all that pleasant.
I guess really only the women who have done both delivery methods can really
comment. All I know is I found it very difficult watching Sarah go through
so much pain. So much so that I'm not really looking forward to that again.
Maybe, hopefully, after her 
Intellectually, I mean. According to