The below is written in the hope it will be helpful for my fellow DDs.
Why using build after push ?
Simple answer: to save time, to always use a clean build environment, to automate more tests.
Real answer: because you are lazy, and tired of always having to type these build commands, and that watching the IRC channel is more fun than watching the build process.
Other less important answers: building takes some CPU time, and makes your computer run slower for other tasks. It is really nice that building doesn t consume CPU cycles on your workstation/laptop, and that a server does the work, not disturbing you while you are working. It is also super nice that it can maintain a Debian repository for you after a successful build, available for everyone to use and test, which would be harder to achieve on your work machine (which may be behind a router doing NAT, or even sometimes turned off, etc.). It s also kind of fun to have an IRC robot telling everyone when a build is successful, so that you don t have to tell them, they can see it and start testing your work.
Install a SID box that can build with cowbuilder
- Setup a SID machine / server.
- Install a build environment with git-buildpackage, pbuilder and cowbuilder (apt-get install all of these).
- Initialize your cowbuilder with: cowbuilder create.
- Make sure that, outside of your chroot, you can do ./debian/rules clean for all of your packages, because that will be called before moving to the cowbuilder chroot. This means you have to install all the build-dependencies involved in the clean process of your packages outside the base.cow of cowbuilder as well. In my case, this means apt-get install openstack-pkg-tools python-setuptools python3-setuptools debhelper po-debconf python-setuptools-git . This part is the most boring one, but remember you can solve these problems when you see them (no need to worry too much until you see a build error).
- Edit /etc/git-buildpackage/gbp.conf, and make sure that under [DEFAULT] you have a line showing builder=git-pbuilder, so that cowbuilder is used by default in the system when using git-buildpackage (and therefore, by Jenkins as well).
Install Jenkins
WARNING: Probably, before really installing, you should read what s bellow (eg: Securing Jenkins).
Simply apt-get install jenkins from experimental (the SID version has some security issues, and has been removed from Wheezy, on a request of the maintainer).
Normally, after installing jenkins, you can access it through:
http://<ip-of-your-server>:8080/
There is no auth by default, so anyone will be able to access your jenkins web GUI and start any script under the jenkins user (sic!).
Jenkins auth
Before doing anything else, you have to enable Jenkins auth, otherwise, you have everything accessible from the outside, meaning that more or less, anyone browsing your jenkins server could be allowed to run any command. It might sound simple, but in fact, Jenkins auth is tricky to activate, and it is very easy to get yourself locked out, with no working web access. So here s the steps:
1. Click on Manage Jenkins then on Configure system
2. Check the enable security checkbox
3. Under security realm select Jenkins s own user database and leave allow users to sign up . Important: leave Anyone can do anything for the moment (otherwise, you will lock yourself out).
5. At the bottom of the screen, click on the SAVE button.
6. On the top right, click to login / create an account. Create yourself an account, and stay logged in.
7. Once logged-in, go back again in the Manage Jenkins -> Configure system , under security.
8. Switch to Project based matrix authentication strategy . Under User/group to add , enter the new login you ve just created, and click on Add .
9. Select absolutely all checkboxes for that user, so that you make yourself an administrator.
10. For the Anonymous user, for Job, check Read, Build and Workspace. For Overall select Read.
11. At the bottom of the screen, hit save again.
Now, anonymous (eg: not logged-in) users should be able to see all projects, and be able to click on the build now button. Note that if you lock yourself out, the way to fix is to turn off Jenkins, edit config.xml, remove the useSecurity thing, and all what is in authorizationStragegy and securityRealm , then restart Jenkins. I had to do that multiple times until I had it right (as it isn t really obvious you got to leave Jenkins completely insecure when creating a new user).
Securing Jenkins: Proxy Jenkins through apache to use it over SSL
When doing a quick $search-engine search, you will see lots of tutorials to use apache as a proxy, which seems to be the standard way to run Jenkins. Add the following to /etc/apache2/sites-available/default-ssl:
ProxyPass / http://localhost:8080/
ProxyPassReverse / http://localhost:8080/
ProxyRequests Off
Then perform the following commands on the shell:
htpasswd -c /etc/apache2/jenkins_htpasswd <your-jenkins-username>
a2enmod proxy
a2enmod proxy_http
a2enmod ssl
a2ensite default-ssl
a2dissite default
apache2ctl restart
Then disable access to the port 8080 of jenkins from outside:
iptables -I INPUT -d <ip-of-your-server> -p tcp --dport 8080 -j REJECT
Of course, this doesn t mean you shouldn t take the steps to activate Jenkins own authentication, which is disabled by default (sic!).
Build a script to build packages in a cowbuilder
I thought it was hard. In fact it was not. All together, this was kind of fun to hack. Yes hack. What I did yet another kind of 10km ugly shell script. The way to use it is simply: build-openstack-pkg <package-name>. On my build server, I have put that script in /usr/bin, so that it is accessible from the default path.Ugly, but it does the job!
Jenkins build script for openstack
At the end of the script, scan_repo() generates the necessary files for a Debian repository to work under /home/ftp. I use pure-ftpd to serve it. /home/ftp must be owned by jenkins:jenkins so that the build script can copy packages in it.
This build script is by no mean state of the art, and in fact, it s quite hack-ish (so I m not particularly proud of it, but it does its job ). If I am showing it in this blog post, it is just to show an example of what can be done. It is left as an exercise to the reader to create another build script adapted to its own needs, and write something cleaner and more modular.
Dependency building
Let s say that you are using the Built-Using: field, and that package B needs to be built if package A changes. Well, Jenkins can be configured to do that. Simply edit the configuration of project B (you will find it, it s easy ).
My use case: In my case, for building Glance, Heat, Keystone, Nova, Quantum, Cinder and Ceilometer, which are all components of Openstack, I have written a small (about 500 lines) library of shell functions, and an also small (90 lines) Makefile, which are packaged in openstack-pkg-tools (so Nova, Glance, etc. all build-depends on openstack-pkg-tools). The shell functions are included in each maintainer scripts (debian/*.config and debian/*.postinst mainly) to avoid having some pre-depends that would break debconf flow. The Makefile of openstack-pkg-tools is included in debian/rules of each packages.
In such a case, trying to manage the build process by hand is boring and time consuming (spending your time watching the build process of package A, so that you can manually start the build of package B, then wait again ). But it is also error prone: it is easy to do a mistakes in the build order, you can forget to dpkg -i the new version of package A, etc.
But that s not it. Probably at some point, you will want Jenkins to rebuild everything. Well, that s easy to do. Simply create a dummy project, and have other project to build after that one. The build steps could simply be: echo Dummy project as a shell script (I m not even sure that s needed ).
Configuring git to start a build on each push
In Jenkins, pass your mouse over the Build now URL. Well, we just need to wget that URL in your Alioth repository. A small drawing is better than long explanation:
for i in ls /git/openstack ; do
echo "wget -q --no-check-certificate \
https://<ip-of-your-server>/job/$ PROJ_NAME /build?delay=0sec \
-O /dev/null" >/git/openstack/$ i /hooks/post-receive \
&& chmod 0770 /git/openstack/$ i /hooks/post-receive;
done
The chmod 0770 is necessary if you don t want every Alioth users to have access to your Jenkins server web interface and see an eventual htpassword protection that you can add to your jenkins box (I m not covering that, but it is fairly easy to add such protection). Note that all of the members of your Alioth group will then have access to this post-receive hook, containing the password of your htaccess, so you must trust everyone in your Alioth group to not do nasty things with your Jenkins.
Bonus point: IRC robot
If you would like to see the result of your build published on IRC, Jenkins can do that. Click on Manage Jenkins , then on Manage Plugins . Then click on available and check the box in front of IRC plugin . Go at the bottom of the screen and click on Add . Then check the box to restart Jenkins automatically. Once it is restarted, go again under Manage jenkins then Configure system . Select the IRC Notification and configure it to join the network and the channel you want. Click on Advanced to select the IRC nick name of your bot, and make sure you change the port (by default jenkins has 194, when IRC normally uses 6667). Be patient when waiting for the IRC robot to connect / disconnect, this can take some time.
Now, for each jenkins Job, you can tick the IRC Notification option.
Doing piuparts after build
One nice thing with automated builds, is that most of the time, you don t need to wait starring at them. So you can add as many tests as you want, the Jenkins IRC robot will anyway let you know sooner or later the result of your build. So adding piuparts tests in the build script seems the correct thing to do. Though that is still on my todo, so maybe that will be for my next blog post.
This is my monthly summary of my free software related activities. If you re among the people who 
... has been 
I was going to start this series with explaining how I did the remote set-up, but instead I will share something that happened today.
One of the first things you want to do when putting a server directly connected to the Internet is some filtering. You don't want to have an application listening on the network by mistake, so a simple netfilter firewall is a good way to ensure you are only accepting connections on ports you explicitly allowed.
I have been a long-time user of 
Wow, my first "Debian day" in 2013, but it's a bit of a misnomer
because I didn't do any Debian work tonight.
I spent some time poking at 
As the first and third quantile are close together we can assume that the majority of the work is done by a few, especially since the second quantile is 5. This is supported by the histogram below, where the x axis is the number of recorded messages and y is the number of developers.
I've only helped
"politically" here and there with this over the years, and I'm
happy to see it live. Your thanks for this should go to the blog
editors---Francesca Ciceri and Ana Guerrero Lopez---and to DSA for
making it real. A proper delegation for the editors is pending and
I'm confident the next DPL will pick it up.
Disclaimer
I'm no symbol expert and dealing with symbol files for first time from
the time I started packaging. What I did here is depending on some
suggestions I got in #debian-mentors.
If you think what I did was wrong please enlighten me :-).
Problem
Recently 2 of my library packages pugixml and ctpp2 got accepted
into Debian archive and when buildd tried to build them on remaining
architectures other than one for which I uploaded (amd64) builds
failed. This was expected as symbols file which I generated was for
amd64. As usual I got 2 serious bug reports
There is no common way to express license information in each source
file at the moment. Some people embed license information in each
file, others keep it in README or another file at the top of the
source tree.
Worse, there is no common syntax to express the license information
in a machine-parseable way. If we had this, we could have tools that,
for example, tell you if you're trying to merge code that has an
incompatible license.
Obviously, this kind of thing can never work perfectly. People
keep inventing new licenses, and it is not possible for a computer
program to fully understand any license. It is not clear humans
can do that, either. However, it would be possible to do it to a
number of well-known licenses, which would help most of the time.
A classic 20/80 situation.
I would like to suggest a syntax, similar to Emacs's "Hey Emacs"
modelines, for embedding a summary of the license, or licenses,
for a source file, in such a way that it can be programmatically
extracted and parsed and analysed. With this syntax, one could then
write a tool to ensure that all files in a project have the same
license, or that all licenses are compatible with the project's
overall license.
Such a tool will, of course, rely on heuristics and assumptions.
For example, it needs to assume the machine-parseable license
summary is correct, and rely on a ruleset on what licenses are
compatible with what licenses. Things can go wrong. That's life.
Remember, this is aiming at doing the 20% of the work that will
work 80% of the time, not perfection.
I don't have a tool written, but I have a suggestion for the syntax.
According to the update 
Given the recent discussions on