Previously, I
demonstrated a Debian packaging workflow using Git and
I mentioned the possibility of a follow-up post; well, here it is: you want to
use my workflow (or one that's related) for a package that is currently
maintained with
Subversion on
svn.debian.org and you'd like to keep the history
during the conversion.
Make sure to read the
previous post before this
one.
I am again using the example of
mdadm since its
Git packaging repository is in a state of
shambles and I want to restart to get it right
and import the history from
the previous
Subversion repository.
What better way than to write a blog post as I do so? Well, plenty actually.
This kind of post
isn't really made for a blog, and I have
started work on setting up
ikiwiki on
madduck.net, but it's not yet ready, so I'll stick with the blog
for now. I will make sure that links don't break as I move content over, so
feel free to bookmark this
Importing the package into Git
Thanks to
git-svn, the
initial step of getting your package imported into Git is a breeze:
$ git-svn clone --stdlayout --no-metadata \
svn+ssh://svn.debian.org/svn/pkg-mdadm/mdadm mdadm
Sit back and enjoy. If that command exits prematurely with an error such as
the following:
Malformed network data: Malformed network data at /usr/local/bin/git-svn
line 1029
then you should upgrade to a newer Git version, or have a look
here. If your Git does not know
--stdlayout then upgrade as well (or use
-T trunk -t tags -b branches
instead).
Sam Vilain notes that it is important to "get the attribution right with the
final SVN import - getting the authors map right. I didn't do that. If you
look at the repository resulting from the above command, you'll notice strange
commit authors, such as madduck@some-unique-uuid-from-svn . git-svn
allows you to map these to real names with real email addresses, which ensures
that the attributions are good for the whole world to see.
When done, switch to the repository and run
git-branch -r. As you'll see,
git-svn imported all SVN branches and tags as remote branches. You need
those if you want to bidirectionally track the Subversion repository, but we
are converting, as you may have guessed by the
--no-metadata switch above.
Therefore, we resort to
the Dinosaur method of converting branches to tags, which I'll simplify for
mdadm. We also just delete all remote branches after tagging, since
mdadm never used branches in the
SVN repository. Your mileage may
vary.
git branch -r sed -rne 's, *tags/([^@]+)$,\1,p' while read tag; do
echo "git tag debian/$tag tags/$ tag ^; git branch -r -d tags/$tag"
done
git branch -r while read tag; do
echo "git branch -r -d $tag"
done
If that seems to work alright, then you can execute the commands.
Sam Vilain (again) hints me at git-pack-refs and then to edit
.git/packed-refs with an editor. This certainly leaves more room for
errors but might be significantly faster.
Cleaning up the SVN references
Even though we passed
--no-metadata to
git-svn, it did leave some
traces in
.git/, which we can now safely remove:
$ git config --remove-section svn-remote.svn
$ rm -r .git/svn
Setting things straight
You can skip this section unless you want to know a bit about how to fix up
stuff with Git.
There was actually some nasty tagging errors leading up to the
2.5.6-9
release for
etch and I could never be bothered to fix those in
SVN,
but now I can (I love Git!):
$ git tag -d debian/2.5.6-10 # never existed
$ git tag -f debian/2.5.6-8 2.5.6-8~2 # mistagged
$ git checkout -b maint/etch 2.5.6-8 # this is when we diverged
$ git apply < /tmp/mdadm-2.5.6-8..2.5.6-9.diff
$ git add debian/po/gl.po debian/po/pt.po debian/changelog
$ git commit -s
$ git tag debian/2.5.6-9
Now that that's fixed, there is one other thing to worry about, namely the
very last commit to
SVN, which obsoletes the repository and points to the
Git repository. But that's not all of it. I was also silly enough to include
a fix in the
same commit. Let's see what Git can do. Since the process of
obsoletion involves all but adding a file, we can simply
--amend the last
commit and provide a new log message:
$ git checkout master
$ git rm OBSOLETE debian/OBSOLETE
$ git commit --amend
Now the repository is in an acceptable state.
Making ends meet
The
pkg-mdadm effort on svn.debian.org only
maintained the
./debian/ directory, separate from the upstream code, and boy
was that a bad idea. Just to give one example: think about what's involved in
preparing a Debian-specific patch against the upstream code this has to end,
and we can make it end right here; let's import upstream's code (again not
using his ADSL line, but the
upstream branch of the
pkg-mdadm Git
repository; see the
previous post for
details):
$ git remote add upstream-repo git://git.debian.org/git/pkg-mdadm/mdadm
$ git config remote.upstream-repo.fetch \
+refs/heads/upstream:refs/remotes/upstream-repo/upstream
$ git fetch upstream-repo
$ git checkout -b upstream upstream-repo/master
Now we have two unconnected ancestries in our repository, and it's time to
join them together. The most logical way seems to be to use the last upstream
tag for which we have a Debian tag:
2.6.2.
For this, we branch off the corresponding Debian tag (
2.6.2-1) and merge
upstream's
2.6.2 tag into the new branch. This will be a temporary branch
Then, we rebase (remember, nothing has been published yet) the master branch
on top of this temporary branch, before we end that branch's short life. The
Debian tag stays where it is since it describes the state of the repository at
time of the release of
2.6.2-1.
$ git checkout -b tmp/join debian/2.6.2-1
$ git merge mdadm-2.6.2
$ git rebase tmp/join master
$ git branch -d tmp/join
It just so happens that the head of the
SVN repository, which is identical
to the tip of our
master branch, corresponds to Debian release
2.6.2-2, so we tag it:
$ git tag debian/2.6.2-2
We are now also "born" in the sense that maintenance in Git has started. Let's
mark that point in history. There is no real reason I can foresee for this
yet, but nonetheless:
$ git tag -s git-birth
Turning dpatch files into feature branches
We want to turn
dpatch files into feature branches and we somehow make it
"proper". We could branch, apply the patch, delete the patch file, checkout
master and delete the patch file there as well, but that appears
"improper" to me at least; so instead, we'll cherry-pick:
$ git checkout -b deb/conffile-location
$ debian/patches/01-mdadm.conf-location.dpatch -apply
$ git rm debian/patches/01-mdadm.conf-location.dpatch
$ git commit -s
$ git commit -s $(git ls-files --others --modified)
I should quickly intervene to make sure you are following. I am making use of
Git's index here. Applying the patch makes the changes in the working tree,
but we did not tell Git that we want those to be part of the commit just yet.
Instead, we delete the
dpatch with
git-rm, which automatically
registers the deletion with the index. Thus, the first
git-commit creates
a commit which deletes the
dpatch, while the second
git-commit
creates a commit with all the changes from the
dpatch, using
git-ls-files to identify new and modified files.
But for now, let's move on. We have two commits in the
deb/conffile-location branch, and one of those is relevant to the
master branch, we cherry-pick it:
$ git cherry-pick deb/conffile-location^
If you're confused, let me explain: our goal is to have a number of feature
branches, of which
master is the one in which most of
./debian/ is
maintained. All the branches later come together in the long-living
build
branch, so
deb/conffile-location will never be merged back into
master. However, once we applied the
dpatch to the feature branch, we
can delete it from there and the
master branch. By cherry-picking, we
"import" the deletion to the
master branch.
I repeat the same procedure for
deb/docs, merging all the
documentation-related
dpatches, but I'll spare you the details.
and then Git let me down
In the next step, I found I had misunderstood Git merging: I thought Git was
smart, but Linus had his reasons for calling Git the "stupid content tracker"
(more on that later). Read on as I am obsoleting
dpatch files that
upstream had merged:
99-*-FIX.dpatch.
For consistency, I wanted to cherry-pick each of the appropriate upstream
commits into the
master branch along with deleting the corresponding
dpatch file. Here is one example:
99-monitor-6+10-FIX.dpatch was
obsoleted by upstream's commit
66f8bbb; the
-x records the original
commit ID in the log:
$ git cherry-pick -x 66f8bbb
$ git rm debian/patches/99-monitor-6+10-FIX.dpatch
$ git commit -s -m"remove dpatch obsoleted by $(git rev-parse --short HEAD)"
I repeated the procedure for the other
dpatch files, removed the
dpatch infrastructure, and then went on to merge it all into
build to
build the package.
The
build branch is a long-living branch off
upstream, but which
upstream? I'll fast-forward you past
a segfault problem with mdadm, which
upstream
(thought to have) resolved with commit 23dc1ae after 2.6.3,
but he had not yet released
2.6.4. Looking at the commits between
23dc1ae and upstream's
HEAD at the time, I decided to include them all
and snapshot
4450e59:
$ git fetch upstream-repo
$ git checkout upstream
$ git merge upstream-repo
$ git tag mdadm-2.6.3+200709292116+4450e59 4450e59
$ git checkout master
$ git merge --no-commit mdadm-2.6.3+200709292116+4450e59
$ dch -v mdadm-2.6.3+200709292116+4450e59-1
$ git add debian/changelog
$ git commit -s
And then I called
poor-mans-gitbuild, which merges
master and then
deb/* into
build. Here is when stuff blew up.
I'll make a long story short (read
my description of the problem and
Linus' answer if you want to know more):
I thought Git was smart to identify merges common to both branches and do the
right thing, but it turn out that Git does not care
at all about commits, it
only worries about content and the end result. In our case, unfortunately
(or fortunately), the outcome meant a conflict because the upstream branch
introduced
a simple change (last hunk)
in the lines surrounding the patch we cherry-picked, and Git can't handle it.
The solution is
not to cherry-pick, to cherry-pick
all commits touching
the context of the
dpatch, or to simply merge
upstream into all out
feature branches. In our case, the first is the easiest solution and since
importing
dpatch files is a one-time thing (thank
$DEITY), I'll leave
it at that.
Almost.
I have spent two days thinking about this more than I should have. And it was
this point Linus made which
made me appreciate Git even more:
Conflicts aren't bad - they're good. Trying to aggressively resolve them
automatically when two branches have done slightly different things in the
same area is stupid and just results in more problems. Instead, git tries to
do what I don't think anybody else has done: make the conflicts easy to
resolve, by allowing you to work with them in your normal working tree, and
still giving you a lot of tools to help you see what's going on.
The end
This concludes today's report. Importing the changes from the old Git repo,
tagging and merging the branches is all covered
in my previous post, or at least
you'll find enough information there to complete the exercise.
I would like to specifically thank Sam Vilain and Linus Torvalds for their
help in preparing this post, as well as the
#git/freenode inhabitants, as
always.
If you are interested in the topic of using version control for distro
packaging, I invite you to join
the vcs-pkg mailing list and/or the
#vcs-pkg/irc.oftc.net IRC channel.
Also, if you are interested in Git in general, you can find a
list of blog
posts on the
Git wiki.
NP:
The Police:
Zenyatta Mondatta
In the last few days, I have replaced the two 20 inch CRT monitors that I have hardly used the last years with two 20
inch TFT displays, and my company (finally) gave me a 19 inch TFT display to accompany my notebook display at work.
Maybe I should take that as a hint that they want to see me in the office more frequently rather than in my home office
which I generously use these days. At home, I built a “new” computer from mainly used parts to drive the two
20 inchers.
I have learned a lot about X in the last days, but spent too much time with it.
I have been running my Desktop with older X and two CRTs for quite some time, but I hardly ever used it due to the
setup of my now ex-flat. Back then, I learned that the “two monitors, one display, one screen, one desktop”
option of X was called Xinerama and statically configured: You write down your display layout in xorg.conf and it uses
it. Configuration changes mean restarting the X server.
KDE notices Xinerama automatically, designates one monitor the “main monitor” and displays its panel there.
You can move windows from one monitor to the other one, but changing virtual desktops always changes what both monitors
display. This is not really what I want, since I often have “static” contents (news reader, mail reader,
chat windows, help screens etc) on one monitor while I do my normal work on the other monitor, which involves changing
virtual desktops.
So I was rather happy when I learned upon the arrival of the external TFT display for the notebook that you can work
without Xinerama. In that case, the X server comes up with one display and two screens, which KDE uses to display two
independent desktops which can be independently configured, but can both be accessed with one mouse and one keyboard.
For me, this means that I finally can independently change virtual desktops on both monitors, but pay the price that I
cannot display a certain virtual desktop on monitor 1 now, and on monitor 2 in five minutes, and that a window opened on
monitor 1 is never going to be shown on monitor 2 without serious interference. This looked ok for me. Additionally, you
can use different font sizes on the different monitors as defaults, which comes in handy if both monitors are blatantly
different in size (which is the case with the notebook and its external monitor).
However, with that setup, KDE seems to be challenged when it comes to saving the configuration. Some things (such as
desktop background and panel config) seem to save fine, desktop independent, other things (such as open windows and
their position) are lost when the session exitst. My daily visit to Kevin&Kell (which used to be in a browser window
that was saved with my session) has badly suffered since then.
Then, a few weeks ago, somebody talked me into trying the X server from experimental. Which is when my dual-desktop
setup ceased working.
The nice people on #debian-x told me that Xinerama is a thing of the past, and is currently in the process of being
replaced by Xrandr 1.2. The removal of “my” two monitor, two screens, two desktop setup is collateral
damage, and if I still want that behavior, the X.org upstream considers this an issue to be handled by the Window
Manager / Desktop Environment now. Gee, thanks. That’s what I call a regression.
After spending half a day cursing and ranting, and even considering to go back to lenny’s (or even etch’s)
X.org, I finally settled on going with the “new” way as I was going to lose my independent desktops sooner
or later anyway. Again, with the friendly help of the people on #debian-x, I found out about a lot of advantages of the
latest X servers.
In most basic cases, the latest X server does not need a configuration file any more. It autodetects your hardware and
chooses whatever modes it finds optimal. It does a pretty decent job in doing so. If one wants to influence how the X
system comes up, once can always write an Xorg.conf file - the configfileless server even writes the “virtual
config file” it uses to the log as a starting point.
Additionally, almost everything regarding screen layout can be reconfigured at run time using the xrandr binary from a
command line. Even a hardware change (such a new monitor plugged into the powered on notebook) is correctly detected.
Cute. This allows me to reconfigure my notebook depending on where it currently is and which external monitor (or
projector) is connected.
This can be perfected if I find out whether X offers a hook to plug a reconfiguration script into that is called when
the X server starts or when the system is waking up from standby or hibernation with the X server running. A lot to do,
and great potential for elegant stuff.
However, while I am talking about hibernation, this is a downside of the new driver: When the system wakes up from
hibernation, the X server sometimes confuses itself enough to become nearly unuseable: I had it become psychotic when
plugging in the external monitor (which can do 1600x1200 points) after waking it up from hibernation at the office. The
internal display can only do 1400x1050, and the box thought that the external monitor only had 1050 y axis pixels,
displaying bit garbage in the lower part of the big monitor, and showing a useable, but unintelligible panel. When
opening any browser or any other application with considerable text amounts in the window, all fonts become unreadable
once one scrolls down. Restarting the X server helps and everyting is fine. But I do not use hibernation to force myself
to log out of X after waking up. After going back to the “one monitor mode”, the X server now thinks that my
virtual desktop is 1600x1200 and only shows 1400x1050, with dead space at the right and lower side of the virtual
desktop. xrandr does not allow me to reisize the FB below 1600x1200. No idea what’s going on here.
In most of the cases, calling xrandr --auto for each output will fix the issues at run time, but sometimes a session
restart is needed. Code needs to stabilize quite a bit before becoming useable in production.
There are a few disadvantages that will remain: Both monitors will change their contents when the virtual desktop is
changed, and both displays will use the same default font size (which might be inappropriate if both displays are of
different size). But I think, when the issues are cleared up, that I can live with these disadvantages.
The “new” desktop system that I built has a Matrox G450 dual head graphics card, and unfortunately, the
Xrandr 1.2 enabled X server from experiemental has shown to be unuseable: Upon some mode changes, the X server goes into
an endless loop and proves itself to be unkillable short of SIGKILL. And if it dies, it sometimes pulls the entire
system with it. There is a corresponding bug in the upstream BTS, but it hasn’t received any noticeable attention by the
upstream developers.
So I might find myself configuring one last Xinerama config before I can finally completly migrate to Xrandr 1.2 on the
desktop as well.
Then I’ll probably need to bug the KDE upstream people to implement a two-desktops-on-a-single-screen feature in
one of their future versions so that I can get my independent-desktop-things back. But that’s a far far away
future.
A couple of nights ago (Tuesday, the 8th) I went to see a live 
I have not been able to put a lot of effort into DAViCal over the last couple of months, since my father was diagnosed with stomach cancer in early September, and he died on 2nd October.
So here it is, finally, including a lot of refactoring work around the handling of DAV/CalDAV REPORT requests and implementation of the DAV::principal-property-search report. This also requires an upgrade to the latest AWL library (0.20), which includes a complete rewrite to the class used for parsing and rendering iCalendar data.
This release is recommended, since you will need some of this stuff to support the upcoming Mozilla Calendar 0.7 release properly.
At this point I have only released the files to 
One of the common criticisms of the war in Iraq is that it was just an excuse to plunder that nations' oil. Of course 
After a very short meeting, the DebConf team agreed on having the next DebConf
the first two weeks of August. That will be August, 2nd to August, 17th. Also,
the Debian Day will be held after DebConf in Buenos Aires, just before the
eigth edition of the Regional Free Software Conference, a very big event with
people coming from different countries in South America. This way, we expect to
reach much more people.
Thanks to Damyan Ivanov for the upload, 
Previously I wrote about 
This coming Thursday I will be at the Parliament and the Internet
conference at Westminster. It will discuss topics including:
...and, really, we nearly never thought that Argentina could win tonight.
South Africa was really too strong for them.
The only domain where Argentinians dominated were scrums but this was much
compensated by their great weakness on line-outs, losing something like 5 or
6 of them.
South Africa thus logical won very easily...and Bryan Habana is now the best
try scorer all along RWCs, along with Jonah Lomu. He is a really great
player, indeed.
Much much impressive South Africa: they'll be by far the favourites of the
final against England, which they beated already during that World Cup. I
said several times that I'm not a deep fan of RSA's play, but I could become
one. Their game improved all along the World Cup and today, they probably
played their best match.
Two remakes next week-end: France-Argentina for the 3rd place and South
Africa-England as final. That will be good matches, no doubt...and I hope
that Bernard Laporte will change the French team to play against Argentina.
"
A few days ago, I was sponsoring some packages. My laptop has something broken and it sometimes shows irreproducible errors, so I used another computer to make the package builds... I did an error. I didn't do it with pbuilder... so the result was a few ftbfs.
