Search Results: "ag"

12 February 2012

Iustin Pop: perf stats for doing nothing

Perf stats for "doing nothing" I've recently discovered the perf Linux tool. I heard that oprofile was deprecated and that there is a new tool, and I noted down to try it sometime. Updated: more languages, fixed typos, more details, some graphs. Apologies if this shows twice in your feed. The problem with perf stats is that I hate bloat, or even perceived bloat. Even when it doesn't affect me in any way, the concept of wasted cycles makes me really sad. You probably can guess where this is going I said, well, let's see what perf says about a simple "null" program. Surely doing nothing should be just a small number of instructions, right? Note: I think that perf also records kernel-side code, because the lowest I could get was about ~50K instructions for starting a null program in assembler that doesn't use libc and just executes the syscall asm instruction. However, these ~50K instructions are noise the moment you start to use more high-level languages. Yes, this is expected, but the I was still shocked. And there's lots of delta between languages I'd expected to behave somewhat identical. Again, this is not important in the real world. At all. They are just numbers, and probably the noise (due to short runtime) has lots of influence on the resulting numbers. And I might have screwed up the measurements somehow. Test setup Each program was the equivalent of 'exit 0' in the appropriate form for the language. During the measurements, the machine was as much as possible idle (single-user mode, measurements run at real-time priority, etc.). For compiled languages, -O2 was used. For scripts, a simple #!/path/to/interpreter (without options, except in the case of Python, see below) was used. Each program/script was run 500 times (perf's -r 500) and I've checked that the variations were small ( 0.80% on the metrics I used). You can find all the programs I've used at http://git.k1024.org/perf-null.git/, the current tests are for the tag version perf-null-0.1. The raw data for the below tables/graphs is at log-4. Results Compiled languages
Language Cycles Instructions
asm 63K 51K
c-dietlibc 74K 57K
c-libc-static 177K 107K
c-libc-shared 506K 300K
c++-static 178K 107K
c++-dynamic 1,750K 1,675K
haskell-single 2,229K 1,338K
haskell-threaded 2,629K 1,522K
ocaml-bytecode 3,271K 2,741K
ocaml-native 1,042K 666K
Going from dietlibc to glibc doubles the number of instructions, and for libc going from static to dynamic linking again roughly doubles it. I didn't manage to compile a program dynamically-linked against dietlibc. C++ is interesting. Linked statically, it is in the same ballpark as C, but when linked dynamically, it executes an order of magnitude (!) more instructions. I would guess that the initialisation of the standard C++ library is complex? Haskell, which has a GC and quite a complex runtime, executes slightly less instructions than C++, but uses more cycles. Not bad, given the capabilities of the runtime. The two versions of the Haskell program are with the single-threaded runtime and with the multi-threaded one; not much difference. A fully statically-linked Haskell binary (not recommended usually) goes below 1M instructions, but not by much. OCaml is a very nice surprise. The bytecode runtime is a bit slow to startup, but the (native) compiled version is quite fast to start: only 2 number of instructions and cycles compared to C, for an advanced language. And twice as fast as Haskell . Nice! Shells
Language Cycles Instructions
dash 766K 469K
bash 1,680K 1,044K
mksh 1,258K 942K
mksh-static 504K 322K
So, dash takes ~470K instructions to start, which is way below the C++ count and a bit higher than the C one. Hence, I'd guess that dash is implemented in C . Next, bash is indeed slower on startup than dash, and by slightly more than 2 (both instructions and cycles). So yes, switching /bin/sh from bash to dash makes sense. I wasn't aware of mksh, so thanks for the comments. It is, in the static variant, more efficient that dash, by about 1.5 . However, the dynamically linked version doesn't look too great (dash is also dynamically linked; I would guess a statically-linked dash "beats" mksh-static). Text processing I've added perl here (even though it's a 'full' language) just for comparison; it's also in the next section.
Language Cycles Instructions
mawk 849K 514K
gawk 1,363K 980K
perl 2,946K 2,213K
A normal spread. I knew the reason why mawk is Priority: required is that it's faster than gawk, but I wouldn't have guessed it's almost twice as fast. Interpreted languages Here is where the fun starts
Language Cycles Instructions
lua 5.1 1,947K 1,485K
lua 5.2 1,724K 1,335K
lua jit 1,209K 803K
perl 2,946K 2,213K
tcl 8.4 5,011K 4,552K
tcl 8.5 6,888K 6,022K
tcl 8.6 8,196K 7,236K
ruby 1.8 7,013K 6,128K
ruby 1.9.3 35,870K 35,022K
python 2.6 -S 11,752K 10,247K
python 2.7 -S 11,438K 10,198K
python 3.2 -S 29,003K 27,409K
pypy -S 21,106K 10,036K
python 2.6 25,143K 21,989K
python 2.7 47,325K 50,217K
python 2.7 -O 47,341K 50,185K
python 3.2 113,567K 124,133K
python 3.2 -O 113,424K 124,133K
pypy 90,779K 68,455K
The numbers here are not quite what I expected. There's a huge delta between the fastest (hi Lua!) and the slowest (bye Python!). I wasn't familiar with Lua, so I tested it thanks to the comments. It is, I think, the only language which actually improves from one version to the next (bonus points), and where the JIT version also make is faster. In context, lua jit starts faster than C++. Perl is the one that goes above C++'s instructions count, but not by much. From the point of view of the system, a Perl 'hello world' is only about 1.3 -1.6x slower than a C++ one. Not bad, not bad. Next category is composed of TCL and Ruby, both of which had older versions 2-3 slower than Perl, but whose most recent versions are even more slower. TCL has an almost constant slowdown across versions (5M, 6.9M, 8.2M cycles), but Ruby seems to have taken a significant step backwards: 1.9.3 is 5 slower than 1.8. I wonder why? As for TCL, I didn't expect it to be slower to startup than Perl; good to know. Last category is Python. Oh my. If you run perf stat python -c 'pass' you get some unbelievable numbers, like 50M instructions to do, well, nothing. Yes, it has a GC, yes, it does import modules at runtime, but still On closer investigation, the site module and the imports it does do eat a lot of time. Running a simpler python -S brings it back to a more reasonable 10M instructions, which is in-line with the other interpreted languages. However, even with the -S taken into account, Python also slows down across versions: a tiny improvement from 2.6 to 2.7, but (like Ruby) a 3 slowdown from 2.7 to 3.2. Trying the optimised version (-O) doesn't help at all. Trying pypy, which was based on Python 2.7, makes it around 2 slower to startup (both with and without -S). So in the interpreted languages, it seems only Lua is trying to improve, the rest of the languages are piling up bloat with every version. Note: I should have tried multiple perl versions too. Java Java is in its own category; you guess why , right? GCJ was version 4.6, whereas by java below I mean OpenJDK Runtime Environment (IcedTea6 1.11) (6b24-1.11-4).
Language Cycles Instructions
null-gcj 97,156K 74,576K
java -jamvm 85,535K 80,102K
java -server 147,174K 136,803K
java -zero 132,967K 124,977K
java -cacao 229,799K 205,312K
Using gcj to compile to native code (not sure whether that's native-native or something else) results in a binary that uses less than 100M cycles to start, but the jamvm VM is faster than that (85M cycles). Not bad for java! Python 3.2 is slower to startup yes, I think the world has gone crazy. However, the other VMs are a few times slower: server (the default one) is ~150M cycles, and cacao is ~230M cycles. Wow. The other thing about java is that it was the only one that couldn't be put nicely in a file that you just exec (there is binfmt_misc indeed, but that doesn't allow different Java classes to use different Java VMs, so I don't count this), as opposed to every single other thing I tested here. Someone didn't grow on Unix? Comparative analysis Since there are almost 4 orders of magnitude difference between all the things tested here, a graph of cycles or instructions is not really useful. However, cycles/instruction, branches percentage and branches miss-predicted percentage can be. Hence first the cycles/instructions: Cycles/instruction Pypy is jumping out of the graph here, with the top value of over 2 cycles/instruction. Lua JIT is also bigger than Lua non-JIT, so maybe there's something to this (mostly joking, two data points don't make a series). On the other hand, Python wins as best cycles/instruction (0.91). Lots of ILP, to get below 1? Java gets, irrespective of VM, consistently near 1.0-1.1. C++ gets very different numbers between static linking (1.666) and dynamic linking (1.045), whereas C has basically identical numbers. mksh also has a difference between dynamic and static linking. Hmm Ruby, TCL and Python have consistent values across versions. And that's about what I can see from that graph. Next up, percentage of branches out of total instructions and percentage of branches missed: Branch statistics Note that the two lines shouldn't really be on the same graph; for the branch %, the 100% is the total instructions count, but for the branch miss %, the 100% is the total branch count. Anyway. There are two low-value outliers: So it seems the gcj libraries are well optimised? I'm not familiar enough with this topic, but on the graph it does indeed stand out. On the other end, mksh-static has a high branch miss percentage: 11.60%, which jumps clearly ahead of all the others; this might be why it has a high cycles/instruction count, due to all the stalls in misprediction; one has to wonder why it confuses the branch predictor? I find it interesting that the overall branch count is very similar across languages, both when most of the cost is in the kernel (e.g. asm) and when the user-space cost heavily over-weighs the kernel (e.g. Java). The average is 20.85%, minimum is 17.46%, max 22.93%, standard deviation (if I used gnumeric correctly) is just 0.01. This seems a bit suspicious to me . On the other hand, the mispredicted branches percentage varies much more: from a measly 2.82% to 11.60% (5x difference). Summary So to recap, counting just instructions: Overall, you get roughly three orders of magnitude slower startup between a plain C program using dietlibc and Python. And all, to do basically nothing. On the other hand, I learned some interesting things while doing it, so it wasn't quite for nothing .

Gregor Herrmann: RC bugs 2012/05-06

I was at FOSDEM over the last weekend, so here's my report for two weeks now:

Neil Williams: Multi-Arch progress

With dpkg from experimental and the new zlib upload in unstable, I've now got a partial Multi-Arch install. There are more packages necessary, particularly related to how -dev packages can exist and how a cross compiler gets built/installed in Multi-Arch world. One bug #659588 in libglib2.0-0 so far but that's quite good seeing as it's been all but impossible to test the Multi-Arch changes in existing packages until now.
$ dpkg --print-foreign-architectures

armel

i386



$ dpkg -l   cut -c -80   grep armel grep -v cross

ii  gcc-4.6-base:armel                   4.6.2-14                     

ii  libc6:armel                          2.13-26                    

ii  libdatrie1:armel                     0.2.5-3                

ii  libffi5:armel                        3.0.10-3                 

ii  libgcc1:armel                        1:4.6.2-14                 

ii  libgmp10:armel                       2:5.0.4+dfsg-1         

ii  libgmpxx4ldbl:armel                  2:5.0.4+dfsg-1        

ii  libgomp1:armel                       4.6.2-14         

ii  libmpc-dev:armel                     0.9-4              

ii  libmpc2:armel                        0.9-4          

ii  libmpfr-dev:armel                    3.1.0-3         

ii  libmpfr4:armel                       3.1.0-3       

ii  libpcre3:armel                       8.12-4                

ii  libpixman-1-0:armel                  0.24.4-1    

ii  libpng12-0:armel                     1.2.46-4     

ii  libpopt0:armel                       1.16-3        

ii  libpwl5:armel                        0.11.2-6    

ii  libselinux1:armel                    2.1.0-4.1      

ii  libstdc++6:armel                     4.6.2-14       

ii  libthai0:armel                       0.1.16-3     

ii  linux-libc-dev:armel                 3.2.4-1         

ii  zlib1g:armel                         1:1.2.6.dfsg-1

There are more packages which can be installed on amd64 for i386 as #659588 only affects Multi-Arch versions which cannot execute compiled binaries from the foreign architecture.

Further progress, inside a test chroot, involves using gcc-4.7 from experimental but even then, libc6-dev is not installable as a Multi-Arch package. That's the current blocker for toolchain stuff.


ii  gcc-4.7-base:i386               4.7-20120210-1

ii  libgcc1:i386                    1:4.7-20120210-1


Once we have libc6-dev:armel installable on i386 and amd64, work on cross-building Emdebian can be considered again. It's been a long time.

Richard Hartmann: FACTA

You know ACTA and related topics have reached the mainstream when the German news show devotes a segment of several minutes(!) to ACTA, complete with background information about the sleazy backroom dealings, and the various political sunday talk shows debate controversially about it, as well. In a round of five 60-plus aged people, only one defenced ACTA; the rest was sceptic or dead against it. The several tens of thousands of people who walked the streets of Germany yesterday (I couldn't; I am ill) really managed to stir up Germany's political landscape. This. is. awesome. Now if only the population at large would understand the inherent differences between physical counterfeiting, physical theft, and copyright infrigement. That's where we all come in: Inform people about the background and actual issues. Now, more than ever, there's a chance that they will be willing to listen.

Christian Perrier: 2012 update 9 for Debian Installer localization

Status for D-I level 1 (core D-I files): Status for D-I level 2 (packages that have localized material that may appear during default installs, such as iso-codes, tasksel, etc.): Status for D-I level 3 (packages that have localized material that may appear during non-default installs, such as win32-loader) Full 100% completeness (hall of fame) for 18 languages: Bulgarian, Czech, German, Esperanto, Spanish, Persian, French, Indonesian, Italian, Japanese, Kazakh, Dutch, Polish, Portuguese, Russian, Slovak, Turkish Simplified Chinese

11 February 2012

Jonathan McDowell: On arranging meetings

I've been spending a lot more time recently in meetings. Mostly things I should actually be at. And in general if it's something I think is reasonable I'll try to be there. In an effort to help with this I actually keep my work calendar up to date. Given that I'm running Linux on my laptop and the corporate standard is Exchange this requires a little bit of effort on my part (the Thunderbird Provider for Microsoft Exchange and Android support for talking to Exchange are helpful with this). Sometimes it seems like I shouldn't bother. I spent this week at a conference, and marked my calendar to indicate I was out of the office. I think I had at least 3 meeting requests, all for things that would actually have been appropriate for me to go to. Last week I managed to be booked for 7 hours of meetings from 7am until noon. That included a 30 minute window where I was triple booked. The thing is, I'm really not that busy in terms of meetings - you can usually find a spot when I'm free on any given day unless I'm actually not in the office. If you bother to check my calendar, that is. Another problem I have is the times people like to book meetings at. Booking a technical meeting at 9am isn't going to get me at my best. Equally doing so at 5pm is likely to have me clock watching to make sure I don't miss my bus and/or train. Also I seem to work with far too many people who don't eat lunch and book hour long meetings at midday or 1pm. I understand sometimes that's the only time you can get everyone into a room together, but at least bloody ask and explain the need rather than just sending out a meeting invite. Finally, book meetings of a realistic length. There are some people who invite me to things and cause me to add another 30 minutes on the end, because I know it always overruns. It's not all bad. I have a VP who always runs a meeting to time, and never seems to call one for spurious reasons. I've also worked with a program manager who will organize the meeting so that if you're only there for one point on the agenda that'll get dealt with near the beginning so it doesn't take up more of your time that it needs to. Funnily enough I'm much more likely to go to things both of these people arrange. Disclaimer: In the unlikely event anyone I work with who invites me to meetings is reading this, I might be talking about you, but everything I mention has been done by more than one person, so I'm not thinking about anyone in particular for each point.

Eugene V. Lyubimkin: multiarch (and) hacks

I experienced situations of writing substantial amounts of code (feature branches) for hours, days and even weeks only to find later that the written code can be thrown to the bin, either because of hidden design problem(s) or too much negative implications outweighting the positive implications of the change.

After reading some recent multiarch threads on debian-devel@ and seeing what hacks are seriously being proposed to implement only to keep the thing going, I now think that the low-level part of the Debian multiarch implementation proposal is no less broken than its high-level part, and the whole proposal is a one big hack which requires and will require more subhacks. Some will benefit from the added functionality, but all, both maintainers and users will suffer from drawbacks.

How two paragraphs above are related? We still have the time to revert the changes and say "sorry, it was a nice idea but the software world isn't ready".

Stefano Zacchiroli: mutt-notmuch 0.2

My mutt-notmuch hack seems to be a quite popular way to integrate Mutt with notmuch. As a nice consequence, my (indexed!) inbox attracts patches from mutt-notmuch users eager to improve it. Collecting some of them, I've just tagged mutt-notmuch 0.2 with the following changes: Many thanks to Scott Barker, Christine Spang, David Newgas, and Ryan Kavanagh for the above patches. While I was at it, I've also moved mutt-notmuch repository to Gitorious. Git self-hosting is nice, but either you move to something like gitolite (which I didn't have time to setup and tune ATM) or you're stuck without merge requests which are quite nice. (Why not Github? Because.) If you're using mutt-notmuch you might also be interested in the discussion of libnotmuch support in mutt. I'd love to see that landing in mutt and be able to throw away mutt-notmuch entirely, but that seems a bit premature as of yet.

Russell Coker: Magic entries for BTRFS and Software RAID

I ve just discovered that the magic database for the file(1) command in Debian/Unstable has no support for Linux Software RAID and that it s support for BTRFS is lacking (no reporting of space used, number of devices, or the UUID). Below is my first draft of a change to fix these problems. I would appreciate it if someone with a big-endian system could test these out and let me know how they go, I suspect that I will have to change the lelong types to long but I m not sure. 4096 lelong 0xa92b4efc Linux Software RAID
>4100 lelong x version 1.2 (%d)
>4112 belong x UUID=%8x:
>4116 belong x \b%8x:
>4120 belong x \b%8x:
>4124 belong x \b%8x
>4128 string x name=%s
>4168 lelong x level=%d
>4188 lelong x disks=%d 0 lelong 0xa92b4efc Linux Software RAID
>4 lelong x version 1.1 (%d)
>16 belong x UUID=%8x:
>20 belong x \b%8x:
>24 belong x \b%8x:
>28 belong x \b%8x
>32 string x name=%s
>72 lelong x level=%d
>92 lelong x disks=%d # BTRFS
0 10040 string _BHRfS_M BTRFS Filesystem
>0x1012b string >\0 label "%s",
>0 10090 lelong x sectorsize %d,
>0 10094 lelong x nodesize %d,
>0 10098 lelong x leafsize %d,
>0 10020 belong x UUID=%8x-
>0 10024 beshort x \b%4x-
>0 10026 beshort x \b%4x-
>0 10028 beshort x \b%4x-
>0x1002a beshort x \b%4x
>0x1002c belong x \b%8x,
>0 10078 lequad x %lld/
>0 10070 lequad x \b%lld bytes used,
>0 10088 lequad x %lld devices Related posts:
  1. Label vs UUID vs Device Someone asked on a mailing list about the issues related...
  2. Starting with BTRFS Based on my investigation of RAID reliability [1] I have...
  3. Software vs Hardware RAID Should you use software or hardware RAID? Many people claim...

John Goerzen: Shell Scripts For Preschoolers

It probably comes as no surprise to anybody that Jacob has had a computer since he was 3. Jacob and I built it from spare parts, together. It may come as something of a surprise that it has no graphical interface, and Jacob uses the command line and loves it and did even before he could really read. A few months ago, I wrote about the fun Jacob had with speakers and a microphone, and posted a copy of the cheat sheet he has with his computer. Lately, Jacob has really enjoyed playing with the speech synthesizer both trying to make it say real words and nonsense words. Sometimes he does that for an hour. I was asked for a copy of the scripts I wrote. They are really simple. I gave them names that would be easy for a preschooler to remember and spell, even if they conflicted with existing Unix/Linux commands. I put them in /usr/local/bin, which occurs first on the PATH, so it doesn t matter if they conflict. First, for speech systhesis, /usr/local/bin/talk:
#!/bin/bash
echo "Press Ctrl-C to stop."
espeak -v en-us -s 150
 espeak comes from the espeak package. It seemed to give the most consistenly useful response. Now, on to the sound-related programs. Here s /usr/local/bin/ssl, the sound steam locomotive . It starts playing a train sound if one isn t already playing:
#!/bin/bash
pgrep mpg321 > /dev/null mpg321 -q /usr/local/trainsounds/main.mp3 &
sl "$@"
 And then there s /usr/local/bin/record:
#!/bin/bash
cd $HOME/recordings
echo "Now recording. Press Ctrl-C to stop."
DATE= date +%Y-%m-%dT%H-%M-%S
FILENAME="$DATE-$$.wav"
chmod a-w *.wav
exec arecord -c 1 -f S16_LE -c 1 -r 44100 "$FILENAME"
 This simply records in a timestamped file. Then, its companion, /usr/local/bin/play. Sorry about the indentation; for whatever reason, it is being destroyed by the blog, but you get the idea.
#!/bin/bash
case "$1" in
train)
mpg321 /usr/local/trainsounds/main.mp3
;;
song)
/usr/bin/play /usr/local/trainsounds/traindreams.flac
;;
*)
cd $HOME/recordings
exec aplay ls -tr tail -n 1
;;
esac
 So, Jacob can run just play , which will play back his most recent recording. As something of a bonus, the history of recordings is saved for us to listen to later. If he types play train , there is the sound of a train passing. And, finally, play song plays Always a Train in My Dreams by Steve Gillette (I heard it on the radio once and bought the CD). Some of these commands kick off sound playing in the background, so here is /usr/local/bin/bequiet:
#!/bin/bash
killall mpg321 &> /dev/null
killall play &> /dev/null
killall aplay &> /dev/null
killall cw &> /dev/null

10 February 2012

Bartosz Feński: Livin on the edge

zenbook ~ # uname -r
3.3.0-rc3-fenio
zenbook ~ # Let s see how many sudden shutdowns will be on that kernel ;)

Raphaël Hertzog: People behind Debian: Ana Beatriz Guerrero L pez, member of the Debian KDE team

If you met Ana, you ll easily remember her. She has a great and pronounced Spanish accent :-) I m glad that the existence of the Debian Women project helped her to join Debian because she has been doing a great job. From KDE packaging to publicity/marketing work, her interests shifted over the years but this allowed her to stay very involved. As she explains it very well, Debian is big enough so that you can stop doing something which is no longer fun for you, and still find something new to do in another part of Debian! Read on to learn more about Ana, the KDE team, Debian s participation to the Google Summer of Code, and more. Raphael: Who are you? Ana: I m Ana Guerrero L pez and I m in my early 30s. I was born and raised in the wonderful city of Sevilla, Spain and I live in Lyon, France. I share my life with another Debian Developer and my paid work is doing Debian support and integration, so you won t be surprised to read that Debian is a big part of my life. Raphael: How did you start contributing to Debian? Ana: Although I knew about the existence of Linux since 1997 or so, I didn t really start using Linux until the summer 2001 when I finally got a computer on my own and an Internet link at home. In the beginning, I was using Mandrake in a dual boot with Windows and later around 2003, I happily moved to only using Debian and ditching the Windows partition. Once settled as a Debian user, I knew anybody could help improve the distribution but I hesitated to join mostly due to two reasons, my perception of Debian was the one of a very elitist and aggressive club and who wants to join this kind of cult^wproject? And even if I wanted to join, I did not know how to get started. By the summer of 2004, the Debian Women project started, it made me seeing Debian as a more welcoming project, and I started maintaining my first packages. The following summer 2005, I attended akademy 2005 (the annual KDE conference) where I had the pleasure to meet there some of the people from the KDE team and this really made a difference for me. Christopher Martin and Adeodato Sim , with the help of other people, have started the maintenance of KDE as a team a few months before and by that time most of the KDE modules where under the maintenance umbrella of the team. This was a very good move since it allowed easily to share the KDE maintenance in a more coordinated way and also eased having non-DDs, like me at that time, to join in and help.
The Debian Women project started, it made me seeing Debian as a more welcoming project.
Raphael: You re part of the Debian KDE team. What s your role in the team and what are your plans for Wheezy? Ana: Nowadays, I am not as active in the KDE team as I used to be in the past. The KDE 3 to KDE 4 transition was quite tiring and changes on the KDE side like the successive marketing renames, the shorter 6 months schedule (it used to be at least 9) or the uncoordinated KDE releases mostly burnt me out. Currently, I am mostly working in helping others to get started within the team, some small fixes here and there, and helping with the uploads: an upload of the full KDE suite to the archive requires some building power and upload bandwidth not everybody have. For Wheezy, with the tentative freeze date in June, the plan is to try to ship the latest possible point release of the KDE 4.8 series. The first release of the series, 4.8.0 was released a couple of weeks ago and while writing these lines, the packaging work for 4.8 hasn t started yet. The next move for the team is getting 4.7.4 in unstable, currently sitting in experimental.
For Wheezy, [ ] the plan is to try to ship the latest possible point release of the KDE 4.8 series.
Besides the KDE packages, there is some software which users perceive as KDE, such as amarok, digikam, etc., which are not part of KDE but fall under its umbrella. These other programs have their own maintainers and their updates depend greatly in the availability of them. For the KDE office suite, we have right now KOffice in the archive. KOffice got a fork some time ago named Calligra and we should replace KOffice by Calligra in the archive before the release of Wheezy. Sadly there isn t yet a final release of Calligra to use. My personal goal for Wheezy was to finish the removal of all the remaining packages depending on KDE 3 and Qt 3 that Squeeze still contained. The removal of the KDE 3 libraries and all the packages using them was quickly achieved after the release of Squeeze. The removal of Qt 3 soon showed that it was task harder than expected since some popular packages (sometimes not in the Debian archive, e.g. third-party scientific software) depend on it, and also Qt 3 is a requirement for LSB compatibility. Right now, Qt 3 has been orphaned for 9 months and nobody has shown any interest in adopting it. Raphael: KDE, much like GNOME, has been forked by people who were unhappy by the direction that the project has taken since version 4 (cf Trinity). What s your personal opinion on KDE 4.x and what s the position of the Debian KDE team concerning this fork? Ana: I use KDE 4 on my laptop and I think it is a solid desktop environment and platform. However I am finding it less and less attractive for me. On one side, my usage of the computer has been slightly changing and on the other side, I do not like how the new developments in KDE are evolving, things like plasmoids or activities are not attractive for me. I have switched my other 2 systems to awesome although I continue to use mainly a bunch of KDE applications: dolphin, konsole, kate, juk, kmix, etc. So you might say my desktop environment is an awesome KDE. Regarding the Trinity project, a lot of users complained very loudly when KDE 3 got replaced by KDE 4 in testing/unstable, so I find quite laudable the decision of some users to act instead and try to continue with a forked development of KDE 3. However the Trinity team seems to be about 3 persons (funny for a project named Trinity :) ) while KDE 3 is big. In perspective, it does not look that big because KDE 4 is even larger, but it is still too much for such small team. In addition those developers need to maintain Qt3 that has been end-of-lifed years ago by Nokia/Trolltech . So my guess is that sooner or later the project will fade away. Nobody from the KDE team is interested in Trinity and in case someone wants to package it for Debian, they would have to make a new team. For the reasons mentioned above: Qt3 maintenance and reduced upstream group, this would be a bad idea. My advice if you do not like KDE 4 and you miss KDE 3, would be taking a look at razor-qt based on Qt4 and quite similar to KDE 3. I read they have plans to port it to Qt4, but frankly that could take some years same it took to the KDE project for KDE 4.0.0 ;-) Raphael: You used to maintain news.debian.net, a WordPress blog dedicated to Debian, but you stopped a while ago. A few months later you started to maintain a Debian page on Google+. Why did you stop the blog and what s your goal with the Google+ page? Ana: I blogged about the reasons I started news.debian.net. In short, I thought Debian needed a better system to publish news, something like a blog. I first tried to suggest the idea to the press/publicity team but they weren t interested, so I started the project alone. IMHO the blog worked quite well and I was feeling like it should be made official. I talked about this with some people but at the time I wasn t pushing it because I had other priorities and I knew pushing it to become official would need some extra time and energy. Stefano decided to start the discussion about making news.debian.net official (that s moving it to a debian.org domain) in its own initiative. After the public discussion and some private exchange of emails with DSA, the situation became frustrating and I decided to close news.debian.net after the release of Squeeze. Later, during DebConf, an officer from the press team announced they were launching a blog and I asked Stefano if he could try to have a discussion about this to see if it could still somehow fit my ideas, and maybe contributing myself, but nobody from the press team answered Stefano s email and the blog hasn t started yet either. Irony that communication didn t work when wanting to improve communication. About the Google+ page, everyday I follow what is going in Debian and quite often I find things I want to share. I do not want to clutter my own profiles with Debian stuff or have people following me because of that, so I decided to create the Debian page when Google+ made them available. I like the fact that people can follow that without having an account in Google+ although they can not comment anonymously. I am not happy about the fact that Google+ is a closed platform but hopefully the data will become easier to export in the near future. Right now, there are some services that provides RSS feeds of Google+ pages if you want to follow the page and you are not in Google+ (or I could setup one if several people ask me). Raphael: Last year you helped to manage Debian s participation to the Google Summer of Code. How did it went? Is there something that you can improve for this year? Ana: I think last year we managed to have people in Debian more aware about what the students were doing. That also helped students to get more feedback and therefore get to know more people in the project and get more integrated. Students were sending periodic public reports available to everybody interested in the status of the projects and some of them also held their own sessions in DebConf. We still failed to start looking for mentors early enough and to give them information about how the GSoC worked and how they could have a successful project. Having good projects in Debian is harder than in other projects because the GSoC mostly promotes having students started in Open Source *coding* for a project, while Debian is more a project about integrating software and we overall do not have so many parts that has to be coded. My personal goal for this year is to try getting the projects earlier to attract good students from the very beginning, even if that means we have less projects than in other years. Raphael: What motivates you to continue to contribute year after year? Ana: Three things. I like improving the OS I use, I like the friends I have made while working in Debian through the years and because I have fun. Also Debian is quite a big project, so if you become tired or burn out working in some area, you always can easily find interesting things to do somewhere else. Raphael: Is there someone in Debian that you admire for their contributions? Ana: Adeodato Sim , he is now in a long leave from the project, but it is one of those persons who made a difference in the project in his job in the release team some years ago. Aur lien Jarno because of his tireless work in (e)glibc and porting of several architectures. I also have special admiration for all those people who have been very active in the project for more than 7-8 years because I know it is not always easy to combine it with real life.
Thank you to Ana for the time spent answering my questions. I hope you enjoyed reading her answers as I did. Note that older interviews are indexed on wiki.debian.org/PeopleBehindDebian.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook .

7 comments Liked this article? Click here. My blog is Flattr-enabled.

Dominique Dumont: Dpkg: edit debian/control description without worrying about leading . and

Hello In Debian package, the debian/control file has a specific syntax where each line must begin with a space and paragraphs are separated by a single . . If you want to follow DEP-5 syntax for debian/copyright files, you have a similar problem when comes the time to fill the license text for creative-common license: it s easy to miss a space or a dot. Let s say I want to add a new section in the description of my package. Here s the current description: 
Description: module to automate definition of a DBIx::Class::Schema
 DBIx::Class::Schema::Loader is an extension to DBIx::Class that automates the
 definition of a DBIx::Class::Schema by scanning table schemas and setting up
 columns and primary keys appropriately. It supports MySQL, PostgreSQL, SQLite
 and DB2.
 .
 Bare table definitions are fairly straightforward, but relationship creation
 is somewhat heuristic, especially with respect to choosing relationship types
 and names, as well as join types. The relationships generated by this module
 will probably never be as well-defined as hand-generated ones.
I want to add This package is awesome but I don t want to be bothered by leading dots and white spaces. First, I go in the package directory, then check the syntax with cme: 
$ cme check dpkg
Then create a loop directory, where the dpkg content will be mapped to a fuse directory: 
$ mkdir loop
$ cme fusefs dpkg -fuse-dir loop/
Mounting config on loop/ in background.
Use command 'fusermount -u loop/' to unmount
$ cat loop/control/binary/libdbix-class-schema-loader-perl/Description
DBIx::Class::Schema::Loader is an extension to DBIx::Class that automates the
definition of a DBIx::Class::Schema by scanning table schemas and setting up
columns and primary keys appropriately. It supports MySQL, PostgreSQL, SQLite
and DB2.
Bare table definitions are fairly straightforward, but relationship creation
is somewhat heuristic, especially with respect to choosing relationship types
and names, as well as join types. The relationships generated by this module
will probably never be as well-defined as hand-generated ones.
Then edit loop/control/binary/libdbix-class-schema-loader-perl/Description with your favorite editor. For the sake of this blog, I will use echo to edit this file: 
$ echo -e "\nThis module is awesome" >> loop/control/binary/libdbix-class-schema-loader-perl/Description
Now umount the loop (note that the actual debian/control is not changed before this step): 
fusermount -u loop
Now the result: 
$ cat debian/control
[...]
Description: module to automate definition of a DBIx::Class::Schema
 DBIx::Class::Schema::Loader is an extension to DBIx::Class that automates the
 definition of a DBIx::Class::Schema by scanning table schemas and setting up
 columns and primary keys appropriately. It supports MySQL, PostgreSQL, SQLite
 and DB2.
 .
 Bare table definitions are fairly straightforward, but relationship creation
 is somewhat heuristic, especially with respect to choosing relationship types
 and names, as well as join types. The relationships generated by this module
 will probably never be as well-defined as hand-generated ones.
 .
 This module is awesome
Here s another example to fill the license text for a license while respecting DEP-5 syntax, (done after the cme command): 
# creating this dir will create a new DEP-5
# standalone license section with name CC-BY-30
mkdir loop/copyright/License/CC-BY-30
# fill the text part of the  license
wget http://creativecommons.org/licenses/by/3.0/legalcode -O - \
   html2text -style pretty > loop/copyright/License/CC-BY-30/text
$ fusermount -u loop
$ cat debian/copyright
[...]
License: CC-BY-30
 .
 .
 .
                                Creative_Commons
 .
 .
 Creative Commons Legal Code
 .
 .
 Attribution 3.0 Unported
[...]
And that s it. (note that cme command is new and is available only from version 2.004 of Config::Model) Of course, there s more than a awy to do it: you can also run cme edit dpkg and add or edit license text in the GUI (or in an editor launched by the GUI). cme will take care of the syntax details. All the best

Alexander Reichle-Schmehl: Release Critical Bug report for Week 06

The bug webinterface of the Ultimate Debian Database currently knows about the following release critical bugs:
In Total:1340
Affecting Wheezy:777
Wheezy only:122
Remaining to be fixed in Wheezy:655
Of these 655 bugs, the following tags are set:
Pending in Wheezy:41
Patched in Wheezy:86
Duplicates in Wheezy:45
Can be fixed in a security Update:21
Contrib or non-free in Wheezy:16
Claimed in Wheezy:0
Delayed in Wheezy:6
Otherwise fixed in Wheezy:49
Ignoring all the above (multiple tags possible) 447 bugs need to be fixed by Debian Contributors to get Debian 7.0 Wheezy released. However, with the view of the Release Managers, 687 need to be dealt with for the release to happen. Please see Interpreting the release critical bug statistics for an explanation of the different numbers.

Alexander Reichle-Schmehl: Release Critical Bug report for Week 06

The bug webinterface of the Ultimate Debian Database currently knows about the following release critical bugs:
In Total:1340
Affecting Wheezy:777
Wheezy only:122
Remaining to be fixed in Wheezy:655
Of these 655 bugs, the following tags are set:
Pending in Wheezy:41
Patched in Wheezy:86
Duplicates in Wheezy:45
Can be fixed in a security Update:21
Contrib or non-free in Wheezy:16
Claimed in Wheezy:0
Delayed in Wheezy:6
Otherwise fixed in Wheezy:49
Ignoring all the above (multiple tags possible) 447 bugs need to be fixed by Debian Contributors to get Debian 7.0 Wheezy released. However, with the view of the Release Managers, 687 need to be dealt with for the release to happen. Please see Interpreting the release critical bug statistics for an explanation of the different numbers.

Michal Čihař: Book about phpMyAdmin 3.4

With quite usual delay after release, book Mastering phpMyAdmin 3.4 for Effective MySQL Management has been published. Unlike the previous edition, I was not doing technical review here, but it is anyway special book for me - the cover photo is coming from me, what is actually for the very first time this has happened. As usual with Packt opensource books, phpMyAdmin project receives some money for each sold copy of the book, so you're welcome to buy it :-).

Filed under: English Phpmyadmin 2 comments Flattr this!

Russell Coker: Starting with BTRFS

Based on my investigation of RAID reliability [1] I have determined that BTRFS [2] is the Linux storage technology that has the best potential to increase data integrity without costing a lot of money. Basically a BTRFS internal RAID-1 should offer equal or greater data protection than RAID-6. As BTRFS is so important and so very different to any prior technology for Linux it s not something that can be easily deployed in the same way as other filesystems. It is possible to easily switch between filesystems such as Ext4 and XFS because they work in much the same way, you have a single block device which the filesystem uses to create a single mount-point. While BTRFS supports internal RAID so it may have multiple block devices and it may offer multiple mountable filesystems and snapshots. Much of the functionality of Linux Software RAID and LVM is covered by BTRFS. So the sensible way to deploy BTRFS is to give it all your storage and not make use of any other RAID or LVM. So I decided to do a test installation. I started with a Debian install CD that was made shortly before the release of Squeeze (it was first to hand) and installed with BTRFS for the root filesystem, I then upgraded to Debian/Unstable to get the latest kernel as BTRFS is developing rapidly. The system failed on the first boot after upgrading to Unstable because the /etc/fstab entry for the root filesystem had the FSCK pass number set to 1 which wasn t going to work as no FSCK program has been written. I changed that number to 0 and it then worked. The initial install was on a desktop system that had a single IDE drive and a CD-ROM drive. For /boot I used a degraded RAID-1 and then after completing the installation I removed the CD-ROM drive and installed a second hard drive, after that it was easy to add the other device to the RAID-1. Then I tried to add a new device to the BTRFS group with the command btrfs device add /dev/sdb2 /dev/sda2 and was informed that it can t do that to a mounted filesystem! That will decrease the possibilities for using BTRFS on systems with hot-swap drives, I hope that the developers regard it as a bug. Then I booted with an ext3 filesystem for root and tried the btrfs device add /dev/sdb2 /dev/sda2 again but got the error message btrfs: sending ioctl 5000940a to a partition! which is not even found by Google. The next thing that I wanted to do was to put a swap file on BTRFS, the benefits for having redundancy and checksums on swap space seem obvious and other BTRFS features such as compression might give a benefit too. So I created a file by using dd to take take from /dev/zero, ran mkswap on it and then tried to run swapon. But I was told that the file has holes and can t be used. Automatically making zero blocks into holes is a useful feature in many situations, but not in this case. So far my experience with BTRFS is that all the basic things work (IE storing files, directories, etc). But the advanced functions I wanted from BTRFS (mirroring and making a reliable swap space) failed. This is a bit disappointing, but BTRFS isn t described as being ready for production yet. Related posts:
  1. Discovering OS Bugs and Using Snapshots I m running Debian/Unstable on an EeePC 701, I ve got an...
  2. Reliability of RAID ZDNet has an insightful article by Robin Harris predicting the...
  3. How I Partition Disks Having had a number of hard drives fail over the...

C.J. Adams-Collier: NIST::NVD::Store::SQLite3

I published an SQLite3 storage back-end to NIST::NVD on the CPAN. It s pretty quick. About as fast as the DB_File one, but without the down side of being tied to DB_File. It shouldn t be too difficult to re-factor this code to any DBI-based database. MariaDB anyone? I know it works on Debian. The nightly CPAN test results should come back shortly and I ll find out how well it works on other platforms.

9 February 2012

Matthew Garrett: Is GPL usage really declining?

Matthew Aslett wrote about how the proportion of projects released under GPL-like licenses appears to be declining, at least as far as various sets of figures go. But what does that actually mean? In absolute terms, GPL use has increased - any change isn't down to GPL projects transitioning over to liberal licenses. But an increasing number of new projects are being released under liberal licenses. Why is that?

The figures from Black Duck aren't a great help here, because they tell us very little about the software they're looking at. FLOSSmole is rather more interesting. I pulled the license figures from a few sites and found the following proportion of GPLed projects:

RubyForge: ~30%
Google Code: ~50%
Launchpad: ~70%

I've left the numbers rough because there's various uncertainties - should proprietary licenses be included in the numbers, is CC Sharealike enough like the GPL to count it there, that kind of thing. But what's clear is that these three sites have massively different levels of GPL use, and it's not hard to imagine why. They all attract different types of developer. The RubyForge figures are obviously going to be heavily influenced by Ruby developers, and that (handwavily) implies more of a bias towards web developers than the general developer population. Launchpad, on the other hand, is going to have a closer association with people with an Ubuntu background - it's probably more representative of Linux developers. Google Code? The 50% figure is the closest to the 56.8% figure that Black Duck give, so it's probably representative of the more general development community.

The impression gained from this is that the probability of you using one of the GPL licenses is influenced by the community that you're part of. And it's not a huge leap to believe that an increasing number of developers are targeting the web, and the web development community has never been especially attached to the GPL. It's not hard to see why - the benefits of the GPL vanish pretty much entirely when you're never actually obliged to distribute the code, and while Affero attempts to compensate from that it also constrains your UI and deployment model. No matter how strong a believer in Copyleft you are, the web makes it difficult for users to take any advantage of the freedoms you'd want to offer. It's as easy not to bother.
So it's pretty unsurprising that an increase in web development would be associated with a decrease in the proportion of projects licensed under the GPL.

This obviously isn't a rigorous analysis. I have very little hard evidence to back up my assumptions. But nor does anyone who claims that the change is because the FSF alienated the community during GPLv3 development. I'd be fascinated to see someone spend some time comparing project type with license use and trying to come up with a more convincing argument.

(Raw data from FLOSSmole: Howison, J., Conklin, M., & Crowston, K. (2006). FLOSSmole: A collaborative repository for FLOSS research data and analyses. International Journal of Information Technology and Web Engineering, 1(3), 17 26.)

comment count unavailable comments

Lior Kaplan: Debian packaging for beginners @ FOSDEM

After not attending FOSDEM for a few years, this year I decided to attend and also give a talk about Debian packaging for beginners , a replay of a talk given by Gergely Nagy in Debconf11 (video). As the per distribution devrooms were replaced a few years ago with the cross-distributions rooms, I thought it might be a good chance to have this kind of introduction talk to help people started contributing to Debian (or derived distributions). The talk wasn t meant to replace reading the documentation (new maintainer s guide, developer s references and the debian policy), but it s a good start for those who want some hands-on experience. The idea is to start with a just the upstream directory and progress trough the various errors and warnings we get during the build process using dpkg-buildpackage. The outcome is of course very basic, but enough to get people ready to do things on their own, including various QA tests on the package (e.g. lintian and debdiff). The presentation covers the important points of the 3 main files in the debian directory: control, changelog and rules. It addition it holds some information about the various tools one can used to test the packages. I hope to make another version of the presentation to be more standalone than just having the main points during the talk itself.
Filed under: Debian GNU/Linux, FOSDEM

Next.

Previous.