I've been spending a lot more time recently in meetings. Mostly things I should actually be at. And in general if it's something I think is reasonable I'll try to be there. In an effort to help with this I actually keep my work calendar up to date. Given that I'm running Linux on my laptop and the corporate standard is Exchange this requires a little bit of effort on my part (the Thunderbird Provider for Microsoft Exchange and Android support for talking to Exchange are helpful with this). Sometimes it seems like I shouldn't bother. I spent this week at a conference, and marked my calendar to indicate I was out of the office. I think I had at least 3 meeting requests, all for things that would actually have been appropriate for me to go to. Last week I managed to be booked for 7 hours of meetings from 7am until noon. That included a 30 minute window where I was triple booked. The thing is, I'm really not that busy in terms of meetings - you can usually find a spot when I'm free on any given day unless I'm actually not in the office. If you bother to check my calendar, that is. Another problem I have is the times people like to book meetings at. Booking a technical meeting at 9am isn't going to get me at my best. Equally doing so at 5pm is likely to have me clock watching to make sure I don't miss my bus and/or train. Also I seem to work with far too many people who don't eat lunch and book hour long meetings at midday or 1pm. I understand sometimes that's the only time you can get everyone into a room together, but at least bloody ask and explain the need rather than just sending out a meeting invite. Finally, book meetings of a realistic length. There are some people who invite me to things and cause me to add another 30 minutes on the end, because I know it always overruns. It's not all bad. I have a VP who always runs a meeting to time, and never seems to call one for spurious reasons. I've also worked with a program manager who will organize the meeting so that if you're only there for one point on the agenda that'll get dealt with near the beginning so it doesn't take up more of your time that it needs to. Funnily enough I'm much more likely to go to things both of these people arrange. Disclaimer: In the unlikely event anyone I work with who invites me to meetings is reading this, I might be talking about you, but everything I mention has been done by more than one person, so I'm not thinking about anyone in particular for each point.

While looking for something to use as a system-unique fall-back when a TPM is not available, I looked at /sys/devices/virtual/dmi/id/product_uuid (same as dmidecode s System Information / UUID ), but was disappointed when, under KVM, the file was missing (and running dmidecode crashes KVM *cough*). However, after a quick check, I noticed that KVM supports the -uuid option to set the value of /sys/devices/virtual/dmi/id/product_uuid. Looks like libvirt supports this under capabilities / host / uuid in the XML, too.

host# kvm -uuid 12345678-ABCD-1234-ABCD-1234567890AB ...
host# ssh localhost ...
...
guest# cat /sys/devices/virtual/dmi/id/product_uuid
12345678-ABCD-1234-ABCD-1234567890AB

2012, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

If you met Ana, you ll easily remember her. She has a great and pronounced Spanish accent I m glad that the existence of the Debian Women project helped her to join Debian because she has been doing a great job. From KDE packaging to publicity/marketing work, her interests shifted over the years but this allowed her to stay very involved. As she explains it very well, Debian is big enough so that you can stop doing something which is no longer fun for you, and still find something new to do in another part of Debian! Read on to learn more about Ana, the KDE team, Debian s participation to the Google Summer of Code, and more. Raphael: Who are you? Ana: I m Ana Guerrero L pez and I m in my early 30s. I was born and raised in the wonderful city of Sevilla, Spain and I live in Lyon, France. I share my life with another Debian Developer and my paid work is doing Debian support and integration, so you won t be surprised to read that Debian is a big part of my life. Raphael: How did you start contributing to Debian? Ana: Although I knew about the existence of Linux since 1997 or so, I didn t really start using Linux until the summer 2001 when I finally got a computer on my own and an Internet link at home. In the beginning, I was using Mandrake in a dual boot with Windows and later around 2003, I happily moved to only using Debian and ditching the Windows partition. Once settled as a Debian user, I knew anybody could help improve the distribution but I hesitated to join mostly due to two reasons, my perception of Debian was the one of a very elitist and aggressive club and who wants to join this kind of cult^wproject? And even if I wanted to join, I did not know how to get started. By the summer of 2004, the Debian Women project started, it made me seeing Debian as a more welcoming project, and I started maintaining my first packages. The following summer 2005, I attended akademy 2005 (the annual KDE conference) where I had the pleasure to meet there some of the people from the KDE team and this really made a difference for me. Christopher Martin and Adeodato Sim , with the help of other people, have started the maintenance of KDE as a team a few months before and by that time most of the KDE modules where under the maintenance umbrella of the team. This was a very good move since it allowed easily to share the KDE maintenance in a more coordinated way and also eased having non-DDs, like me at that time, to join in and help.

The Debian Women project started, it made me seeing Debian as a more welcoming project.

Raphael: You re part of the Debian KDE team. What s your role in the team and what are your plans for Wheezy? Ana: Nowadays, I am not as active in the KDE team as I used to be in the past. The KDE 3 to KDE 4 transition was quite tiring and changes on the KDE side like the successive marketing renames, the shorter 6 months schedule (it used to be at least 9) or the uncoordinated KDE releases mostly burnt me out. Currently, I am mostly working in helping others to get started within the team, some small fixes here and there, and helping with the uploads: an upload of the full KDE suite to the archive requires some building power and upload bandwidth not everybody have. For Wheezy, with the tentative freeze date in June, the plan is to try to ship the latest possible point release of the KDE 4.8 series. The first release of the series, 4.8.0 was released a couple of weeks ago and while writing these lines, the packaging work for 4.8 hasn t started yet. The next move for the team is getting 4.7.4 in unstable, currently sitting in experimental.

For Wheezy, [ ] the plan is to try to ship the latest possible point release of the KDE 4.8 series.

Besides the KDE packages, there is some software which users perceive as KDE, such as amarok, digikam, etc., which are not part of KDE but fall under its umbrella. These other programs have their own maintainers and their updates depend greatly in the availability of them. For the KDE office suite, we have right now KOffice in the archive. KOffice got a fork some time ago named Calligra and we should replace KOffice by Calligra in the archive before the release of Wheezy. Sadly there isn t yet a final release of Calligra to use. My personal goal for Wheezy was to finish the removal of all the remaining packages depending on KDE 3 and Qt 3 that Squeeze still contained. The removal of the KDE 3 libraries and all the packages using them was quickly achieved after the release of Squeeze. The removal of Qt 3 soon showed that it was task harder than expected since some popular packages (sometimes not in the Debian archive, e.g. third-party scientific software) depend on it, and also Qt 3 is a requirement for LSB compatibility. Right now, Qt 3 has been orphaned for 9 months and nobody has shown any interest in adopting it. Raphael: KDE, much like GNOME, has been forked by people who were unhappy by the direction that the project has taken since version 4 (cf Trinity). What s your personal opinion on KDE 4.x and what s the position of the Debian KDE team concerning this fork? Ana: I use KDE 4 on my laptop and I think it is a solid desktop environment and platform. However I am finding it less and less attractive for me. On one side, my usage of the computer has been slightly changing and on the other side, I do not like how the new developments in KDE are evolving, things like plasmoids or activities are not attractive for me. I have switched my other 2 systems to awesome although I continue to use mainly a bunch of KDE applications: dolphin, konsole, kate, juk, kmix, etc. So you might say my desktop environment is an awesome KDE. Regarding the Trinity project, a lot of users complained very loudly when KDE 3 got replaced by KDE 4 in testing/unstable, so I find quite laudable the decision of some users to act instead and try to continue with a forked development of KDE 3. However the Trinity team seems to be about 3 persons (funny for a project named Trinity ) while KDE 3 is big. In perspective, it does not look that big because KDE 4 is even larger, but it is still too much for such small team. In addition those developers need to maintain Qt3 that has been end-of-lifed years ago by Nokia/Trolltech . So my guess is that sooner or later the project will fade away. Nobody from the KDE team is interested in Trinity and in case someone wants to package it for Debian, they would have to make a new team. For the reasons mentioned above: Qt3 maintenance and reduced upstream group, this would be a bad idea. My advice if you do not like KDE 4 and you miss KDE 3, would be taking a look at razor-qt based on Qt4 and quite similar to KDE 3. I read they have plans to port it to Qt4, but frankly that could take some years same it took to the KDE project for KDE 4.0.0 Raphael: You used to maintain news.debian.net, a WordPress blog dedicated to Debian, but you stopped a while ago. A few months later you started to maintain a Debian page on Google+. Why did you stop the blog and what s your goal with the Google+ page? Ana: I blogged about the reasons I started news.debian.net. In short, I thought Debian needed a better system to publish news, something like a blog. I first tried to suggest the idea to the press/publicity team but they weren t interested, so I started the project alone. IMHO the blog worked quite well and I was feeling like it should be made official. I talked about this with some people but at the time I wasn t pushing it because I had other priorities and I knew pushing it to become official would need some extra time and energy. Stefano decided to start the discussion about making news.debian.net official (that s moving it to a debian.org domain) in its own initiative. After the public discussion and some private exchange of emails with DSA, the situation became frustrating and I decided to close news.debian.net after the release of Squeeze. Later, during DebConf, an officer from the press team announced they were launching a blog and I asked Stefano if he could try to have a discussion about this to see if it could still somehow fit my ideas, and maybe contributing myself, but nobody from the press team answered Stefano s email and the blog hasn t started yet either. Irony that communication didn t work when wanting to improve communication. About the Google+ page, everyday I follow what is going in Debian and quite often I find things I want to share. I do not want to clutter my own profiles with Debian stuff or have people following me because of that, so I decided to create the Debian page when Google+ made them available. I like the fact that people can follow that without having an account in Google+ although they can not comment anonymously. I am not happy about the fact that Google+ is a closed platform but hopefully the data will become easier to export in the near future. Right now, there are some services that provides RSS feeds of Google+ pages if you want to follow the page and you are not in Google+ (or I could setup one if several people ask me). Raphael: Last year you helped to manage Debian s participation to the Google Summer of Code. How did it went? Is there something that you can improve for this year? Ana: I think last year we managed to have people in Debian more aware about what the students were doing. That also helped students to get more feedback and therefore get to know more people in the project and get more integrated. Students were sending periodic public reports available to everybody interested in the status of the projects and some of them also held their own sessions in DebConf. We still failed to start looking for mentors early enough and to give them information about how the GSoC worked and how they could have a successful project. Having good projects in Debian is harder than in other projects because the GSoC mostly promotes having students started in Open Source *coding* for a project, while Debian is more a project about integrating software and we overall do not have so many parts that has to be coded. My personal goal for this year is to try getting the projects earlier to attract good students from the very beginning, even if that means we have less projects than in other years. Raphael: What motivates you to continue to contribute year after year? Ana: Three things. I like improving the OS I use, I like the friends I have made while working in Debian through the years and because I have fun. Also Debian is quite a big project, so if you become tired or burn out working in some area, you always can easily find interesting things to do somewhere else. Raphael: Is there someone in Debian that you admire for their contributions? Ana: Adeodato Sim , he is now in a long leave from the project, but it is one of those persons who made a difference in the project in his job in the release team some years ago. Aur lien Jarno because of his tireless work in (e)glibc and porting of several architectures. I also have special admiration for all those people who have been very active in the project for more than 7-8 years because I know it is not always easy to combine it with real life.

---

Thank you to Ana for the time spent answering my questions. I hope you enjoyed reading her answers as I did. Note that older interviews are indexed on wiki.debian.org/PeopleBehindDebian.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook .

7 comments Liked this article? Click here. My blog is Flattr-enabled.

With quite usual delay after release, book Mastering phpMyAdmin 3.4 for Effective MySQL Management has been published. Unlike the previous edition, I was not doing technical review here, but it is anyway special book for me - the cover photo is coming from me, what is actually for the very first time this has happened. As usual with Packt opensource books, phpMyAdmin project receives some money for each sold copy of the book, so you're welcome to buy it :-).

Filed under: English Phpmyadmin 2 comments Flattr this!

Matthew Aslett wrote about how the proportion of projects released under GPL-like licenses appears to be declining, at least as far as various sets of figures go. But what does that actually mean? In absolute terms, GPL use has increased - any change isn't down to GPL projects transitioning over to liberal licenses. But an increasing number of new projects are being released under liberal licenses. Why is that?

The figures from Black Duck aren't a great help here, because they tell us very little about the software they're looking at. FLOSSmole is rather more interesting. I pulled the license figures from a few sites and found the following proportion of GPLed projects:

RubyForge: ~30%
Google Code: ~50%
Launchpad: ~70%

I've left the numbers rough because there's various uncertainties - should proprietary licenses be included in the numbers, is CC Sharealike enough like the GPL to count it there, that kind of thing. But what's clear is that these three sites have massively different levels of GPL use, and it's not hard to imagine why. They all attract different types of developer. The RubyForge figures are obviously going to be heavily influenced by Ruby developers, and that (handwavily) implies more of a bias towards web developers than the general developer population. Launchpad, on the other hand, is going to have a closer association with people with an Ubuntu background - it's probably more representative of Linux developers. Google Code? The 50% figure is the closest to the 56.8% figure that Black Duck give, so it's probably representative of the more general development community.

The impression gained from this is that the probability of you using one of the GPL licenses is influenced by the community that you're part of. And it's not a huge leap to believe that an increasing number of developers are targeting the web, and the web development community has never been especially attached to the GPL. It's not hard to see why - the benefits of the GPL vanish pretty much entirely when you're never actually obliged to distribute the code, and while Affero attempts to compensate from that it also constrains your UI and deployment model. No matter how strong a believer in Copyleft you are, the web makes it difficult for users to take any advantage of the freedoms you'd want to offer. It's as easy not to bother.
So it's pretty unsurprising that an increase in web development would be associated with a decrease in the proportion of projects licensed under the GPL.

This obviously isn't a rigorous analysis. I have very little hard evidence to back up my assumptions. But nor does anyone who claims that the change is because the FSF alienated the community during GPLv3 development. I'd be fascinated to see someone spend some time comparing project type with license use and trying to come up with a more convincing argument.

(Raw data from FLOSSmole: Howison, J., Conklin, M., & Crowston, K. (2006). FLOSSmole: A collaborative repository for FLOSS research data and analyses. International Journal of Information Technology and Web Engineering, 1(3), 17 26.)

comments

SNMP stands for Simple Network Management Protocol. It allows a manager to query information from an agent. A popular use of SNMP is to retrieve network interface counters to plot a bandwidth graph. While the S in SNMP stands for simple, SNMP can be quite difficult to deal with. However, it is still the de facto standard for retrieving metrics in an heterogeneous environment. Net-SNMP is a suite of SNMP applications including an agent. Out of the box, this agent exports a lot of information but if something is missing, there are several ways to extend it. Which ones? tl;dr: in my opinion, extending Net-SNMP should be done with either of those methods:

• using extend directive;
• using pass_persist directive;
• using AgentX protocol.

SNMP in a nutshell The variables accessible with SNMP through the agent are organized in a tree. Each variable is typed and associated to an object identifier (or OID). For example, .1.3.6.1.2.1.31.1.1.1.1.2 is the OID of the name of the second network card. Usually, SNMP is operated over UDP, port 161. For the purpose of this article, we consider a manager can issue three kinds of requests:

GET

retrieve a variable.

GETNEXT

retrieve the next variable, in lexical order of their OID. This operation enables an SNMP manager to walk through all available variables, for example to discover the list of interfaces of an agent. This operation is what makes an SNMP agent difficult to implement but this is also a critical feature.

SET

request the modification of a variable.

Since such an OID may be difficult to handle by humans, available variables are described by a MIB module which is a file that should be parsed to get a mapping between OID and variable names and types. For example, the above OID can also be referred as IF-MIB::ifName.2. Those MIB modules are described using a subset of ASN.1 defined in RFC 2578. A MIB module allows to group several variables into a conceptual table. For example, IF-MIB::ifDescr, IF-MIB::ifType, IF-MIB::ifSpeed are columns of IF-MIB::ifTable. Each row is associated to an index. In the case of IF-MIB::ifTable, it is the interface index. Net-SNMP includes snmptable allowing one to display such a table in a natural manner¹:

$ snmptable localhost IF-MIB::ifTable
SNMP table: IF-MIB::ifTable
 ifIndex ifDescr           ifType ifMtu   ifSpeed
       1      lo softwareLoopback 16436  10000000
       2    eth0   ethernetCsmacd  1500 100000000
       3    eth1   ethernetCsmacd  1500  10000000
       4     br0   ethernetCsmacd  1500         0

Variables inside a table are called columnar objects while those outside are scalar objects. SNMP, as a protocol, does not care of this distinction since it does not know the concept of MIB modules. Other useful tools from Net-SNMP includes snmpget to get a specific variable, snmpwalk to discover available variables (with GETNEXT operation) from an OID and snmptranslate to translate between object names and OID.

$ snmpget localhost IF-MIB::ifDescr.2
IF-MIB::ifDescr.2 = STRING: eth0
$ snmpwalk localhost IF-MIB::ifDescr
IF-MIB::ifDescr.1 = STRING: lo
IF-MIB::ifDescr.2 = STRING: eth0
IF-MIB::ifDescr.3 = STRING: eth1
IF-MIB::ifDescr.4 = STRING: br0
$ snmpwalk -On public localhost .1.3.6.1.2.1.2.2.1.2
.1.3.6.1.2.1.2.2.1.2.1 = STRING: lo
.1.3.6.1.2.1.2.2.1.2.2 = STRING: eth0
.1.3.6.1.2.1.2.2.1.2.3 = STRING: eth1
.1.3.6.1.2.1.2.2.1.2.4 = STRING: br0
$ snmptranslate -On IF-MIB::ifDescr.3
.1.3.6.1.2.1.2.2.1.2.3

Extending Net-SNMP ethtool -S allows us to access various statistics from a network card, like the number of octets received and transmitted or the number of collisions. While some of those statistics are defined in IF-MIB, in RMON-MIB and in EtherLike-MIB, some of them are not available and very specific. Therefore, we would like to export those statistics, unmodified, through SNMP. To the best of my knowledge, no MIB module exists for such an usage. The first task is to write one. It can be called ETHTOOL-MIB. It only contains one table indexed by the interface index and the name of the statistic. Here is the expected output:

$ snmpwalk localhost ETHTOOL-MIB::ethtoolStat.2
ethtoolStat.2.'align_errors' = Counter64: 0
ethtoolStat.2.'broadcast' = Counter64: 25
ethtoolStat.2.'multicast' = Counter64: 345
ethtoolStat.2.'rx_errors' = Counter64: 0
ethtoolStat.2.'rx_missed' = Counter64: 0
ethtoolStat.2.'rx_packets' = Counter64: 262011
ethtoolStat.2.'tx_aborted' = Counter64: 0
ethtoolStat.2.'tx_errors' = Counter64: 0
ethtoolStat.2.'tx_multi_collisions' = Counter64: 0
ethtoolStat.2.'tx_packets' = Counter64: 296170
ethtoolStat.2.'tx_single_collisions' = Counter64: 0
ethtoolStat.2.'tx_underrun' = Counter64: 0
ethtoolStat.2.'unicast' = Counter64: 261641

I have coded various implementations of this MIB module and made them available in a GitHub repository.

With arbitrary commands The extend directive allows the agent to execute an arbitrary command and to provide its output and status through NET-SNMP-EXTEND-MIB module. If we want to export the statistics for eth0, we could add something like this in snmpd.conf, using a custom script to format the output of ethtool -S in a way that is more suitable for our use:

# Export eth0 statistics
extend eth0-names  /full/path/to/ethtool-stats eth0 names
extend eth0-values /full/path/to/ethtool-stats eth0 values

The result can be retrieved through NET-SNMP-EXTEND-MIB:: nsExtendOutput2Table:

$ snmpwalk localhost NET-SNMP-EXTEND-MIB::nsExtendOutput2Table
nsExtendOutLine."eth0-names".1 = STRING: tx_packets
nsExtendOutLine."eth0-names".2 = STRING: rx_packets
nsExtendOutLine."eth0-names".3 = STRING: tx_errors
nsExtendOutLine."eth0-names".4 = STRING: rx_errors
nsExtendOutLine."eth0-names".5 = STRING: rx_missed
nsExtendOutLine."eth0-names".6 = STRING: align_errors
nsExtendOutLine."eth0-names".7 = STRING: tx_single_collisions
nsExtendOutLine."eth0-names".8 = STRING: tx_multi_collisions
nsExtendOutLine."eth0-values".1 = STRING: 246309
nsExtendOutLine."eth0-values".2 = STRING: 223941
nsExtendOutLine."eth0-values".3 = STRING: 0
nsExtendOutLine."eth0-values".4 = STRING: 0
nsExtendOutLine."eth0-values".5 = STRING: 0
nsExtendOutLine."eth0-values".6 = STRING: 0
nsExtendOutLine."eth0-values".7 = STRING: 0
nsExtendOutLine."eth0-values".8 = STRING: 0

It is also possible to configure such a command using SET requests. For example, if we want to add statistics for eth2, we could issue the following command:

$ snmpset -m +NET-SNMP-EXTEND-MIB localhost \
>    'nsExtendStatus."eth2-names"'  = createAndGo \
>    'nsExtendCommand."eth2-names"' = /full/path/to/ethtool-stats \
>    'nsExtendArgs."eth2-names"'    = 'eth2 names' \
>    'nsExtendStatus."eth2-values"'  = createAndGo \
>    'nsExtendCommand."eth2-values"' = /full/path/to/ethtool-stats \
>    'nsExtendArgs."eth2-values"'    = 'eth2 values'
nsExtendStatus."eth2-names" = INTEGER: createAndGo(4)
nsExtendCommand."eth2-names" = STRING: /full/path/to/ethtool-stats
nsExtendArgs."eth2-names" = STRING: eth2 names
nsExtendStatus."eth2-values" = INTEGER: createAndGo(4)
nsExtendCommand."eth2-values" = STRING: /full/path/to/ethtool-stats
nsExtendArgs."eth2-values" = STRING: eth2 values
$ snmpgetnext -m +NET-SNMP-EXTEND-MIB localhost \
>    'nsExtendOutLine."eth2-names"'
nsExtendOutLine."eth2-names".1 = STRING: tx_packets

This can be very convenient but also a huge security risk. You should disable this functionality or enable it only for some SNMPv3 user. Using extend does not allow to implement an arbitrary MIB module. If the cache is not disabled, extend is a good solution for various simple needs, like providing the content of a one-line file or the output of some Nagios plugin. However, exporting anything tabular, like interface statistics, is not a good fit.

With pass-through scripts Net-SNMP allows one to extend the agent by delegating some sub-tree to a script defined by the pass_persist directive. Such a script will receive requests on its standard input with a very simple line-oriented protocol described in the manual page of snmpd.conf. Here is an example of interaction:

   PING
   PONG
   get
   .1.3.6.1.4.1.39178.100.1.1.1.2.2.109.117.108.116.105.99.97.115.116
   .1.3.6.1.4.1.39178.100.1.1.1.2.2.109.117.108.116.105.99.97.115.116
   counter64
   223941
   getnext
   .1.3.6.1.4.1.39178.100.1.1.1.2.2
   .1.3.6.1.4.1.39178.100.1.1.1.2.2.97.108.105.103.110.95.101.114
   counter64
   0

The protocol is simple enough to implement it from scratch in any scripting language. However, in Perl, you may prefer to use SNMP:: ExtensionSNMP:: PassPersist extension from S bastien Aperghis-Tramoni. You can find the complete example on Github.

use SNMP::Extension::PassPersist;
my $extsnmp = SNMP::Extension::PassPersist->new(
    backend_collect => \&update_tree
    );
$extsnmp->run;
sub update_tree  
    my @interfaces = </sys/class/net/*>;
    foreach my $interface (@interfaces)  
        $interface =~ s/^.*\///;
        open(ETHTOOL, "ethtool -S $interface 2>/dev/null  ") or next;
        while (<ETHTOOL>)  
            /^\s+(\w+): (\d+)$/ or next;
            my $name  = $1;
            my $value = int($2);
            my $oid   = oid_compute($interface, $name);
            $extsnmp->add_oid_entry($oid, "counter", $value);
         
        close(ETHTOOL);
     
 

With Python, snmp-passpersist module, from Nicolas Agius, provides a similar interface. Look at the complete example on GitHub. pass_persist allows you to implement an arbitrary MIB module with honest performances. Here are some important things to know about this directive:

• Unless you have a recent version of Net-SNMP, 64bit types are converted to 32bit equivalents. The counters may overflow quickly.
• While handling a request, you are not allowed to pause (for example, to fetch a remote information): the agent would become unresponsive. If you need slow-to-get data or data from the agent, grab them in a separate thread and ensure you have them beforehand.

With AgentX protocol AgentX is a protocol defined in RFC 2741 allowing a master agent to be extended by independent sub-agents. For snmpd to become a master agent, you just need to add master agentx in snmpd.conf. Here are some of the benefits of an AgentX sub-agent over pass_persist:

1. No configuration is needed for the master agent to accept an additional sub-agent. A sub-agent registers to the master agent the MIB modules (or part of them) it wants to take care of.
2. A sub-agent is decoupled from the master agent. It can run with a different identity or be integrated into another daemon to export its internal metrics, send traps or allow remote configuration through SNMP.
3. AgentX protocol can be carried over TCP. Sub-agents can therefore run on a foreign host or in a jailed environment.
4. 64bit types are fully supported. Traps are also supported.

Perl and Python Net-SNMP provides a Perl module, NetSNMP::agent, to write a sub-agent using AgentX protocol. This module is not as convenient as the module for pass_persist described above since it does not provide a convenient layer to put required information into some datastore. Basically, it mimics the available C API. Here are a few examples of its use:

• An implementation of ETHTOOL-MIB I have done in the same way as the pass_persist version. My Perl skills are a bit rusty and the code has many rough edges.
• An implementation of a sub-agent for Redis servers.
• An implementation of BRIDGE-MIB shipped with Net-SNMP.

There is no similar binding for Python but there is an agentx extension using ctypes. It provides a more high-level view. I did not write an example for this one but you can look at the example provided by the author.

C with new API To write a sub-agent in C, you have to choose between two API:

1. the API inherited from UCD-SNMP also known as the traditional API;
2. the new API developed for Net-SNMP 5.x.

With both API, an handler is registered to take care of one or several sub-trees. With the traditional API, the handler is invoked with very little information. It is then up to you to locate the appropriate variable by mapping the index part of the OID to the objects you want to export. It is quite simple for scalar objects but columnar objects with complex indexes are a pain with such an API. The new API provides various helpers. Some of them allow you to copy objects into an array or a list and let Net-SNMP library do the hard work of serving them. Another helper will provide methods to serve objects without putting them in a special structure but by providing an iterator. Some of those helpers can also help you to cache some variables. On top of this new API, Net-SNMP comes with mib2c, a tool that will help you convert a MIB module into C code: after a few questions, you will get a skeleton C code you will have to complete with the logic to retrieve real objects. A generated file also provides directions on what to do (with some magic command to have a step-by-step cookbook). I was able to implement ETHTOOL-MIB by just running it through mib2c, choosing the table container helper with cache and adding some code in ethtoolStatTable_data_access.c. You can look at the result on Github. Here is a stripped-down version² of what has been modified:

/* Socket for ioctl */
skfd = socket(AF_INET, SOCK_DGRAM, 0);
/* Iterate through all interfaces */
getifaddrs(&ifap);
for (ifa = ifap; ifa != NULL; ifa = ifa->ifa_next)  
    /* Grab statistics name and values */
    strcpy(ifr.ifr_name, ifa->ifa_name);
    drvinfo.cmd         = ETHTOOL_GDRVINFO;
    ifr.ifr_data        = (caddr_t) &drvinfo;
    ioctl(skfd, SIOCETHTOOL, &ifr);
    n_stats             = drvinfo.n_stats;
    strings->cmd        = ETHTOOL_GSTRINGS;
    strings->string_set = ETH_SS_STATS;
    strings->len        = n_stats;
    ifr.ifr_data        = (caddr_t) strings;
    ioctl(skfd, SIOCETHTOOL, &ifr);
    stats->cmd          = ETHTOOL_GSTATS;
    stats->n_stats      = n_stats;
    ifr.ifr_data        = (caddr_t) stats;
    ioctl(skfd, SIOCETHTOOL, &ifr);
    ifIndex = if_nametoindex(ifa->ifa_name);
    /* Iterate through statistics */
    for (i = 0; i < n_stats; i++)  
        /* Declare the appropriate index */
        strncpy(ethtoolStatName,
                (char *)&strings->data[i * ETH_GSTRING_LEN],
                ETH_GSTRING_LEN);
        ethtoolStatName[sizeof(ethtoolStatName) - 1] = '\0';
        ethtoolStatName_len = strlen(ethtoolStatName);
        /* Append this new variable to the container. */
        rowreq_ctx = ethtoolStatTable_allocate_rowreq_ctx();
        ethtoolStatTable_indexes_set(rowreq_ctx,
                    ifIndex,
                    ethtoolStatName, ethtoolStatName_len);
        rowreq_ctx->data.ethtoolStat.high = stats->data[i] >> 32;
        rowreq_ctx->data.ethtoolStat.low = (uint32_t)stats->data[i];
        CONTAINER_INSERT(container, rowreq_ctx);
     
 

See? It is pretty easy. I have only enumerated all the objects I wanted to expose. I didn t even have to convert statistics names to an OID, mib2c built a ethtoolStatTable_indexes_set() for this purpose. OpenHPI project provides an extensive documentation to write a sub-agent this way. Now, on the downside, to implement a single table, I have 14 generated files whose code style is usually unfit to integrate into an existing project : some people, me included, do not like automatically generated code, all the more when it is so verbose. The solution would be to use the new API without mib2c; unfortunately, while the documentation exists, all provided examples rely exclusively on mib2c tool. Unless you are able to grasp the new API without it, I would restrict its use to two cases:

• Writing a new module for inclusion into Net-SNMP. Since most modules are now written with the help of mib2c, there is no coding style problem.
• Writing a standalone sub-agent for some MIB. As a standalone project, you won t run into coding style problems either.

For other uses, let s have a look at the traditional API.

C with traditional API The traditional API is way simpler and even if it is quite old, it is easier to find associated examples. Here is how to initialize the agent, again, in a stripped-down version (the complete version is on GitHub):

static oid ethtool_oid[] =  1, 3, 6, 1, 4, 1, 39178, 100, 1 ;
static struct variable3 ethtool_vars[] =  
   1, ASN_COUNTER64, RONLY, ethtool_stat, 3,  1, 1, 2 
 ;
int main()
 
  netsnmp_enable_subagent();
  snmp_disable_log();
  snmp_enable_stderrlog();
  init_agent("ethtoolAgent");
  REGISTER_MIB("ethtoolStatTable", ethtool_vars,
               variable3, ethtool_oid);
  init_snmp("ethtoolAgent");
  while (1)
    agent_check_and_process(1);
 

The REGISTER_MIB() macro is used to register the provided root OID (ethtool_oid) to the master agent. It also registers handlers associated to various sub-OID through the ethtool_vars[] array. Each member of this array features the following fields:

• magic is an integer allowing one to discriminate OID handled by the same handler. For a table, this allows to handle several columns with the same handler since they share a common index.
• type is the type of the variable that will be returned.
• acl tells if the object is read-only (RONLY) or read-write (RWRITE).
• findVar is the handler associated to the object. Its task will be to find the appropriate object and return its value.
• The two last fields are the relative OID this entry is responsible for along with its length.

The prototype of ethtool_stat() handler function is the following:

static u_char*
ethtool_stat(struct variable *vp, oid *name, size_t *length,
             int exact, size_t *var_len, WriteMethod **write_method);

This function should return the value of the requested variable or NULL if no appropriate object was found. The value can be statically stored since it will be copied. Here is the description of the arguments:

• struct variable *vp is the structure the handle was associated with, except that vp->name is now a full OID, not a relative one. You can access vp->magic to determine which column or which scalar to handle if this handler is common to several objects.
• oid *name and size_t *length describe the OID requested. If you want to return another OID (for a GETNEXT request for example), you can modify them. name points to an OID array large enough to fit any OID you want to return.
• int exact tells if the requested OID should be exactly matched (case of GET or SET) or not (GETNEXT).
• size_t *var_len is used to tell the size of the returned object.
• WriteMethod **write_method is only used if the variable is modifiable. When a SET request is received, the API will call the handler with exact set to 1 and expects to get a pointer to the function that will handle the write operation³.

To implement this function, you are on your own. When relying on externally fetched data, you need to handle a cache yourself. I have used a red-black tree for this purpose (with code stolen from OpenBSD). This also enables an easy implementation of the GETNEXT operation. Here are three projects using this API to provide SNMP support:

• Keepalived is a VRRP daemon and a monitoring daemon for LVS clusters. I have added a complete SNMP support. The most interesting files are core/snmp.c, check/check_snmp.c and vrrp/vrrp_snmp.c. It features a tight integration into Keepalived event loop that should be reusable by other projects. It includes read-only support of all data structures (configuration, state and statistics), ability to send traps on events and a write support for some values.
• lldpd is an implementation of 802.1AB (LLDP) which is a protocol allowing an equipment to advertise itself to its neighbors on Ethernet level. 802.1AB comes with a huge MIB module and many extensions. lldpd provides a fairly complete read-only implementation of this MIB module. Some tables have complex indexes. Have a look at agent.c. There is currently no write support and no traps but lldpd features privilege separation and some code allows it to still use an Unix socket to speak with the master agent.
• Asterisk, an open source PBX, includes an AgentX sub-agent allowing one to grab some metrics. Look at res/snmp/agent.c.

Other solutions The manual page for snmpd.conf explores other ways to extend Net-SNMP agent under the section Extending agent functionality :

• exec and sh are deprecated. Use extend instead.
• pass is a bit simpler than pass_persist. The command is run once for each request. Such a command can easily be turned into a pass_persist script. Moreover, Net-SNMP will not answer any request while a pass command is running.
• proxy redirects requests to another SNMP agent. Unless you have some code that can only act as a complete SNMP agent, it is better to use AgentX instead. For programs using Net-SNMP API, turning an agent into an AgentX sub-agent is just one line of code. Since a sub-agent involves less code than a complete agent, it is more efficient and more reliable.
• smux has been superseeded by AgentX.
• perl and dlmod allow one to load and execute external code inside the running agent. Unless you have important performance requirements, it is better to use a sub-agent instead: if some bug happens, only the sub-agent will be affected, not the master agent. A sub-agent sticks more closely to the Unix philosophy: write programs that do one thing and do it well . However, Net-SNMP FAQ says:

Implementing the module in C within the main agent (directly or via dlmod) is probably the most efficient and reliable, closely followed by embedded perl (or python) extensions. These have the advantage of minimal overheads between the code implementing the MIB module, and the agent framework, and no inter-process communication issues.

---

1. Additional arguments for snmp* are usually required to specify a version and a community (for SNMPv1 and SNMPv2c) or a user (for SNMPv3). In all examples, I do not specify anything. I assume that ~/.snmp/snmp.conf contains the appropriate bits. Moreover, some outputs are truncated.
2. No allocations, no variable declarations and no error checkings. Do not do this at home!
3. If you want to look at an example of how to handle a SET request, look at the
   SNMP support for Keepalived. A SET operation is more complex because SNMP ensures that the
   SET operation is atomic across all provided variables. It uses a three-staged commit. The traditional API is very inefficient for this kind of operation because the object has to be located again for each step.

As you might have noticed from my previous posts, we've moved phpMyAdmin code to github. Now I'm going to describe some things which might be useful for you if you are about do to similar switch. While using git, moving to another hosting provider should be pretty straightforward. Just add another remote, push code there and it should be done. On the other side you probably have dozens of things in your infrastructure which you need to adjust. So the first thing to do is write down what all services are connected to your current git repositories. Let me name some which might be worth checking:

• continuous integration server
• snapshots generating
• demo server (in case you're providing something like we do)
• commit notifications
• various statistics services such as cia.vc
• website generating
• references in wiki, website and documetation

Once you think you have remembered all important things (the less important will probably show up anyway, but majority of things should work), you're ready to make the move. I've decided to make the move in few steps. First push all data to new location, what can take some time. I'll get in more details to that later. In the same time I asked all contributors to give me their login information, so that I can allow them access to new repositories. Once all recently active developers were migrated, it was time to push all remaining commits to new git repositories and make the switch for real. Pushing git repo to another location, should be pretty easy. On the other side if you have many branches, it get's slightly more complex, I've ended up with following shell snippet (pushing all branches present in origin to github remote):

git branch -r   grep origin/   grep -v HEAD   sed 's@.*/@@'   while read b ; do git checkout $b ; git push github $b:$b ; done

Please ensure that you check output of this, because you may hit network problems somewhere in the middle and you end up with few branches than you expect. As the code is pretty much idempotent, you can safely run it several times until there is nothing to push. You should also push all tags to new location:

git push --tags github

Okay, we've all data on right place, so let's switch all our users to new location:

git remote set-url origin git@github.com:phpmyadmin/phpmyadmin.git # read/write
git remote set-url origin git://github.com/phpmyadmin/phpmyadmin.git # read only

Of course everybody has to do this manually. Next good thing is to let people know when they are using wrong repo (which will stay there for some time). Unfortunately there is AFAIK no way to warn them on pull, so let's warn at least on push:

$ cat > hooks/update
#!/bin/sh
echo "phpMyAdmin git repositories have moved to https://github.com/phpmyadmin"
exit 1

I think this is pretty much all. You can find some more bits in our Git migration wiki page. PS: Thanks to github for offering us hosting and sorry for breaking their branch displaying page by too many divergent branches.

Filed under: Coding English Phpmyadmin 4 comments Flattr this!

This is the second entry in my series about squeeze release critical bug squashing. In response to my last blog post it was asked whether this is proper release critical bug squashing. Indeed there haven't been any patches or upload involved in this, only BTS handling, but this doesn't mean that these bugs weren't considered to be affecting squeeze. You can see this effort currently as weeding out the "wrong" bugs so that the list gets more useful and actually be able to ask maintainers to address the real issues. You can at least see in this graph that the blue line is going down constantly since the year change instead of rising up like before. And I hope I will be able to keep it below the green line for a while still. Also thanks to the release-team and ftpmasters that it was possible to keep the massbugs about waf binary blob not being preferred source for modification out of affecting squeeze and ignore it for the current stable release the required changes for those would rather be a fair bit intrusive for a stable update.

• 637213: doesn't fail in squeeze
• 648989: doesn't fail in squeeze
• 584568: most stuff (BTS, PTS, ...) use maintainer information from unstable only
• 624464: most stuff (BTS, PTS, ...) use maintainer information from unstable only
• 632106: most stuff (BTS, PTS, ...) use maintainer information from unstable only
• 632108: most stuff (BTS, PTS, ...) use maintainer information from unstable only
• 632262: most stuff (BTS, PTS, ...) use maintainer information from unstable only
• 632264: most stuff (BTS, PTS, ...) use maintainer information from unstable only
• 632265: most stuff (BTS, PTS, ...) use maintainer information from unstable only
• 632266: most stuff (BTS, PTS, ...) use maintainer information from unstable only
• 595298: squeeze has linux 2.6.32
• 613732: doesn't fail in squeeze
• 616694: doesn't fail in squeeze
• 620264: doesn't fail in squeeze
• 620573: doesn't fail in squeeze
• 621440: libdb4.8 not obsolete in squeeze
• 624593: doesn't fail in squeeze
• 626868: doesn't fail in squeeze
• 628475: libdb4.8 not obsolete in squeeze
• 629838: boost1.46 not in squeeze
• 628475: boost1.46 not in squeeze
• 631623: doesn't fail in squeeze
• 639990: doesn't fail in squeeze
• 632763: no libevent 2.0 in squeeze
• 627000: not affecting squeeze
• 658216: doesn't fail in squeeze
• 654870: doesn't fail in squeeze
• 655115: this is explicitly about testing
• 657092: working in squeeze
• 655710: doesn't fail in squeeze
• 644691: no nautilus 3 in squeeze

I am glad that I managed to keep it up and even have a nice margin in case I can't put any effort into it some day but still have more bugs squashed than days there are in the year so far. Currently I am at 60 bugs in 39 days. This gives a warm feeling. :) Enjoy!

/debian permanent link Comments: 3 Flattr this

Part of our efforts to reduce power consumption is to identify processes which keep waking up the disk even when the computer is idle. This already resulted in a few bug reports (and some fixes, too), but we only really just began with this. Unfortunately there is no really good tool to trace file access events system-wide. powertop claims to, but its output is both very incomplete, and also wrong (e. g. it claims that read accesses are writes). strace gives you everything you do and don t want to know about what s going on, but is per-process, and attaching strace to all running and new processes is cumbersome. blktrace is system-wide, but operates at a way too low level for this task: its output has nothing to do any more with files or even inodes, just raw block numbers which are impossible to convert back to an inode and file path. So I created a little tool called fatrace ( file access trace , not fat race ) which uses fanotify, a couple of /proc lookups and some glue to provide this. By default it monitors the whole system, i. e. all mounts (except the virtual ones like /proc, tmpfs, etc.), but you can also tell it to just consider the mount of the current directory. You can write the log into a file (stdout by default), and run it for a specified number of seconds. Optional time stamps and PID filters are also provided.

$ sudo fatrace
rsyslogd(967): W /var/log/auth.log
notify-osd(2264): O /usr/share/pixmaps/weechat.xpm
compiz(2001): R device 8:2 inode 658203
[...]

It shows the process name and pid, the event type (Rread, Write, Open, or Close), and the path. Sometimes its not possible to determine a path (usually because it s a temporary file which already got deleted, and I suspect mmaps as well), in that case it shows the device and inode number; such programs then need closer inspection with strace. If you run this in gnome-terminal, there is an annoying feedback loop, as gnome-terminal causes a disk access with each output line, which then causes another output line, ad infinitum. To fix this, you can either redirect output to a file (-o /tmp/trace) or ignore the PID of gnome-terminal (-p pidof gnome-terminal ). So to investigate which programs are keeping your disk spinning, run something like

  $ sudo fatrace -o /tmp/trace -s 60

and then do nothing until it finishes. My next task will be to write an integration program which calls fatrace and powertop, and creates a nice little report out of that raw data, sorted by number of accesses and process name, and all that. But it might already help some folks as it is right now. The code lives in bzr branch lp:fatrace (web view), you can just run make and sudo ./fatrace. I also uploaded a package to Ubuntu Precise, but it still needs to go through the NEW queue. I also made a 0.1 release, so you can just grab the release tarball if you prefer. Have a look at the manpage and --help, it should be pretty self-explanatory.

I've just updated phpMyAdmin repos on github and enabled notifications hooks there, so the earlier announced switch to github is done. All you need to do is point your repositories to pull/push to github, for main repository it can be done using:

git remote set-url origin git@github.com:phpmyadmin/phpmyadmin.git

If you are using just read only access then use following:

git remote set-url origin git://github.com/phpmyadmin/phpmyadmin.git

For other repositories just replace last part of the URL with repository name (they have not changed). Everything should work as before, pushes should be now faster, because all notifications are done in background on github and they don't block pushing.

Filed under: English Phpmyadmin 2 comments Flattr this!

As I announced on debian-devel, Guillem Jover uploaded a snapshot of dpkg s multiarch branch to experimental (version 1.16.2~wipmultiarch). Beware: There will
likely be some small interface changes between this version and the version that will be released later in unstable (possibly in the output of dpkg --get-selections, dpkg --list, maybe other commands). multiarch allows you to install packages from different architectures on the same machine. This can be useful if your computer can run programs from 2 architectures (eg. x86 CPU supporting i386 and amd64), or if you often need to cross-compile software and thus need the libraries of your target architecture. Test dpkg with multiarch support If you want to test multiarch support in dpkg, install the package from experimental (apt-get install dpkg/experimental assuming you have experimental in your sources.list). Then you can add a supplementary architecture to your system by doing sudo dpkg --add-architecture <arch> (e.g. i386 if you are on amd64, and vice-versa). APT will automatically pick up the new architecture and start downloading the Packages file for the new architecture (it uses dpkg --print-foreign-architectures to know about them). From there on you can install packages from the foreign architectures with apt-get install foo:<arch> . Many packages will not be installable because some of their dependencies have not yet been updated to work with in a multiarch world (libraries must be installed in a multiarch-compliant path so as to be co-installable, and then marked Multi-Arch: same ). Other dependencies might need to be marked Multi-Arch: foreign . See wiki.debian.org/Multiarch/Implementation for more HOWTO-like explanations. Now is a good time to see if you can install the foreign packages that you could need in such a setup and to help to convert the required libraries. You can also read Cyril Brulebois article which quickly shows how to hunt for the problematic packages which have not been converted to multiarch (in his sample, ucf is not ready. Since it s an Architecture: all package which can run on any architecture, it means that it s lacking a Multi-Arch: foreign field). Report bugs If you discover any bug in dpkg s multiarch implementation, please report it to the Bug Tracking System (against dpkg with the version 1.16.2~wipmultiarch ). If you notice important libraries or packages which are not yet multiarch ready, please open wishlist bug reports requesting the conversion and point the maintainers towards the wiki page linked above. Even better, prepare patches and submit those with your bug reports. Again, you can follow the lead of Cyril Brulebois who filed 6 bugs! Review the multiarch implementation If you re a C programmer and have some good knowledge of dpkg (or are willing to learn more of it), we would certainly benefit from more eyes reviewing the multiarch branch. If you want to discuss some design issues of the multiarch implementation in dpkg (or have questions related to your review), please get in touch via debian-dpkg@lists.debian.org. The latest version of the branch is pu/multiarch/master in Guillem s personal repository. I have my own version of the branch (pu/multiarch/full) which is usually a snapshot of Guillem s branch with my own submitted fixes.

$ git clone git://git.debian.org/dpkg/dpkg.git
$ cd dpkg
$ git remote add guillem git://git.hadrons.org/git/debian/dpkg/dpkg.git
$ git remote add buxy git://git.debian.org/~hertzog/dpkg.git
$ git fetch guillem && git fetch buxy

If you followed the instructions above, the relevant branches are thus guillem/pu/multiarch/master and buxy/pu/multiarch/full. Both branches are regularly rebased on top of master where Guillem merges progressively the commits from the multi-arch branch as his review progresses. Thank you in advance for your help bringing multiarch in shape for Debian Wheezy,

6 comments Liked this article? Click here. My blog is Flattr-enabled.

Everyone agrees that backups are generally a good thing. But it seems that there is a lot less agreement about how backups should work. Here is a list of 5 principles of backup software that seem to get ignored most of the time: (1/5) Backups should not be Application Specific It s quite reasonable for people to want to extract data from a backup on a different platform. Maybe someone will want to extract data a few decades after the platform becomes obsolete. I believe that vendors of backup software have an ethical obligation to make it possible for customers to get their data out with minimal effort regardless of the circumstances. Often when writing a backup application there will be good reasons for not using the existing formats for data storage (tar, cpio, zip, etc). But ideally any data store which involves something conceptually similar to a collection of files in one larger file will use one of those formats. There have been backward compatible extensions to tar and zip for SE Linux contexts and for OS/2 EAs the possibility of extending archive file formats with no consequence other than warnings on extraction with an unpatched utility has been demonstrated. For a backup which doesn t involve source files (EG the contents of some sort of database) then it should be in a format that can be easily understood and parsed. Well designed XML is generally a reasonable option. Generally the format should involve plain text that is readable and easy to understand which is optionally compressed with a common compression utility (pkzip is a reasonable choice). (2/5) Data Store Formats should be Published For every data store there should be public documentation about it s format to allow future developers to write support for it. It really isn t difficult to release some commented header files so that people can easily determine the data structures. This includes all data stores including databases and filesystems. If I suddenly find myself with a 15yo image of a NTFS filesystem containing a proprietary database I should be able to find official header files for the version of NTFS and the database server in question so I can decode the data if it s important enough. When an application vendor hides the data formats it gives the risk of substantial data loss at some future time. Imposing such risk on customers to try and prevent them from migrating to a rival product is unethical. (3/5) Backups should be forward and backward compatible It is entirely unreasonable for a vendor to demand that all their users install the latest versions of their software. There are lots of good reasons for not upgrading which includes hardware not supporting new versions of the OS, lack of Internet access to perform the upgrade, application compatibility, and just liking the way the old version works. Even for the case of a critical security fix it should be possible to restore data without applying the fix. For any pair of versions of software that are only separated by a few versions it should be possible to backup data from one and restore to the other. Even if the data can t be used directly (EG a backup of AMD64 programs that is restored on an i386 system) it should still be accessible. If a new version of the software doesn t support the ancient file formats then it should be possible for the users to get a slightly older version which talks to both the old and new versions. Backups made on 64bit systems running the latest development version of Linux and on 10yo 32bit proprietary Unix systems are interchangeable. Admittedly Unix is really good at preserving file format compatibility, but there is no technical reason why other systems can t do the same. Source code to cpio, tar, and gnuzip, is freely available! Apple TimeMachine fails badly in this regard, even a slightly older version of Mac OS can t do a restore. It is however nice that most of the TimeMachine data is a tree of files which could be just copied to another system. (4/5) Backup Software should not be Dropped Sony Ericsson has made me hate them even more by putting the following message on their update web site: The Backup and Restore app will be overwritten and cannot be used to restore data. Check out Android Market for alternative apps to back up and restore your data, such as MyBackup. So if you own a Sony Ericsson phone and it is lost, stolen, or completely destroyed and all you have is a backup made by the Sony Ericsson tool then the one thing you absolutely can t do is to buy a new Sony Ericsson phone to restore the data. I believe that anyone who releases backup software has an ethical obligation to support restoring to all equivalent systems. How difficult would it be to put a new free app in the Google Market that has as it s sole purpose recovering old Sony Ericsson backups onto newer phones? It really can t be that difficult, so even if they don t want to waste critical ROM space by putting the feature in all new phones they can make it available to everyone who needs it. When compared to the cost of developing a new Android release for a series of phones the cost of writing such a restore program would be almost nothing. It is simply mind-boggling that Sony Ericsson go against their own commercial interests in this regard. Surely it would make good business sense to be able to sell replacements for all the lost and broken Sony Ericsson phones, but instead customers who get burned by broken backups are given an incentive to buy a product from any other vendor. (5/5) The greater the control over data the greater the obligation for protecting it If you have data stored in a simple and standard manner (EG the /DCIM directory containing MP4 and JPEG files that is on the USB accessible storage in every modern phone) then IMHO it s quite OK to leave customers to their own devices in terms of backups. Typical users can work out that if they don t backup their pictures then they risk losing them, and they can work out how to do it. My Sony Ericsson phones have data stored under /data (settings for Android applications) which is apparently only accessible as root. Sony Ericsson have denied me root access which prevents me running backup programs such as Titanium Backup, therefore I believe that they have a great obligation to provide a way of making a backup of this data and restoring it on a new phone or a phone that has been updated. To just provide phone upgrade instructions which tell me that my phone will be entirely wiped and that I should search the App Market for backup programs is unacceptable. I believe that there are two ethical options available to Sony Ericsson at this time, one is to make it easy to root phones so that Titanium Backup and similar programs can be used, and the other option is to release a suitable backup program for older phones. Based on experience I don t expect Sony Ericsson to choose either option. Now it is also a bad thing for the Android application developers to make it difficult or impossible to backup their data. For example the Wiki for one Android game gives instructions for moving the saved game files to a new phone which starts with root your phone . The developers of that game should have read the Wiki, realised that rooting a phone for the mundane task of transferring saved game files is totally unreasonable, and developed a better alternative. The best thing for developers to do is to allow the users to access their own data in the most convenient manner. Then it becomes the user s responsibility to manage it and they can concentrate on improving their application. Why Freedom is Important Installing CyanogenMod on my Galaxy S was painful, but having root access so I can do anything I want is a great benefit. If phone vendors would do the right thing then I could recommend that other people use the vendor release, but it seems that vendors can be expected to act unethically. So I can t recommend that anyone use an un-modded Android phone at any time. I also can t recommend ever buying a Sony Ericsson product, not even when it s really cheap. Google have done a great thing with their Data Liberation Front [1]. Not only are they providing access to the data they store on our behalf (which is a good thing) but they have a mission statement that demands the same behavior from other companies they make it an issue of competitive advantage! So while Sony Ericsson and other companies might not see a benefit in making people like me stop hating them, failing to be as effective in marketing as Google is a real issue. Data Liberation is something that should be discussed at board elections of IT companies. Keep in mind the fact that ethics are not just about doing nice things, they are about establishing expectations of conduct that will be used by people who deal with you in future. Sony Ericsson has shown that I should expect that they will treat the integrity of my data with contempt and I will keep this in mind every time I decline an opportunity to purchase their products. Google has shown that they consider the protection of my data as an important issue and therefore I can be confident when using and recommending their services that I won t get stuck with data that is locked away. While Google has demonstrated that corporations can do the right thing, the vast majority of evidence suggests that we should never trust a corporation with anything that we might want to retrieve when it s not immediately profitable for the corporation. Therefore avoiding commercial services for storing important data is the sensible thing to do.

• [1] http://www.dataliberation.org/

Related posts:

1. Galaxy S vs Xperia X10 and Android Network Access Galaxy S Review I ve just been given an indefinite loan...
2. Standardising Android Don Marti wrote an amusing post about the lack of...
3. On Burning Platforms Nokia is in the news for it s CEO announcing that...

Yet another FOSDEM is behind us and I'd like to thank all people organizing it. It was a great event as usual. This year there were some changes - the conference grew and there was an extra building. This is great, but on the other side, there were more tracks to follow and occasionally I wanted to be in four places at once, what is of course not manageable. Combined with quite freezing weather (well it was still much warmer than it is now in Prague), moving from one side of campus to another was not that comfortable as in last years, but there is not much man can do with that. And the biggest change for me - I did not manage beer event this year. We enjoyed great team dinner on Friday evening and while it ended, I was too lazy to move to crowded beer event and rather enjoyed bed in my hotel.

Filed under: Debian English Phpmyadmin Suse 0 comments Flattr this!

For the past few years the anti-spam system I run has been based on a simplified version of something I previously ran commercially. Although the code is similar in intent there were both explicit feature removals, and simplifications made. Last month I re-implimented domain-blacklisting - because a single company keeps ignoring requests to remove me. So LinkedIn.com if you're reading this:

• I've never had an account on your servers.
• I find your junk mail annoying.
• I suspect I'll join your site/service when hell freezes over.

I've also implemented TLD-blacklisting which has been useful.

TLD-blacklisting in my world is not about blocking mail from foo@bar.ph (whether in the envelope sender, or the from: header), instead it is about matching the reverse DNS of the connecting client. If I recieve a connection from 1.2.3.4 and the reverse DNS of that IP address matches, say, /\.sa$/i then I default to denying it. My real list is longer, and handled via files:

steve@steve:~$ ls /srv/_global_/blacklisted/tld/ -C
ar  br  cn  eg  hr  in  kr  lv  mn  np  ph  ro  sg  tg  ua  ve  zw
aw  cc  cy  gm  hu  is  kz  ma  my  nu  pk  rs  sk  th  ug  vn
be  ch  cz  gr  id  it  lk  md  mz  nz  pl  ru  su  tr  uy  ws
bg  cl  ec  hk  il  ke  lt  mk  no  om  pt  sa  sy  tw  uz  za

On average I'm rejecting about 2500 messagse a day at SMTP-time, and 30 messages, or so, hit my SPAM folder after being filtered with CRM114 after being accepted for delivery. (They are largely from @hotmail and @yahoo, along with random compromised machines. The amount of times I see a single mail from a host with RDNS mysql.example.org is staggering.). (Still looking forward to the development of Haraka, a node.js version of qpsmtpd.) ObQuote: "Mr. Mystery Guest? Are you still there? " - Die Hard

Yesterday we did held phpMyAdmin team meeting and we discussed several things for project future. One of discussed things was to move our repositories to github for easier collaboration and management of private repositories. After some discussions we've decided to go for it. As I write this blog, all repositories are being pushed there and once all developers will get their access rights there, we will move completely. But even right now, you can fork us on github, the code is already there and will be updated obsessionally until we fully move there.

Filed under: English Phpmyadmin 0 comments Flattr this!

Again, as usual in last few years, I'm spending first weekend in February in Brussels, where FOSDEM is happening. This year we've again decided to do make this team meeting for phpMyAdmin, so people from five countries and three continents came to one conference to discuss future development and other stuff. But of course this is not only thing I'm going to do here. I came with openSUSE folks, where we've brought lot of beer, some DVDs and hardware to show. You're welcome to check it out. And of course there is about 430 talks to visit during weekend :-).

Filed under: Debian English Phpmyadmin Suse 0 comments Flattr this!

I am currently working on a Qt-based app that needs to communicate through the serial port. Apart from all the benefits that a normal lib with a serial port implementation will bring in this case, having a Qt-based serial port lib would make even more sense, as it should be as multiplatform as possible and use the signal/slot mechanism. Also it should have a DFSG-compatible license, so I can package it for Debian, of course :-)

So I have found two libs which seemed to have the above mentioned features: QExtSerialPort and QSerialPort.

QExtSerialPort seems to be the most recommended lib in the web. It features polled and signal-based functionality; it uses Qt's standard types inheriting QIODevice. But it does not states the license in any file within the source code. The original project page at SourceForge says it's in public domain. And the newer project page at Google code says it's under the new BSD license. I have asked in the mailing list for a clarification. So far nothing has changed (although in further threads the authors showed some willing to change this). And then I got to the point of finding a bug, but I don't want to spend time to track it down and make a patch without a clear license.

QSerialPort it's another lib with more or less the same features as QExtSerialPort. It's main LICENSE file says it's under the LGPL2, but licensecheck will say that the present files are LGPL3. Also, on reviewing the code, I found some minor stuff that could be improved. Well, I could contact the author and see if [s]he would receive the patches... but his site seems down. And I could not find a real-person's name in the code so far :-/

So I made a last attempt to try to get QExtSerialPort in a suitable license. If it doesn't suceed, I think I'll have to start writing one myself. The downside: I only use Linux, so there will be no multiplatform features unless someone else contributes it. Of course, if you have another option or any idea to share, I'll be happy to know it :-)

By the way, this should be my first post on Planet Debian in english, so hello planet!

My last post missed an important thing about GHC 7.4's handling of encodings for FileName. It can in fact be safe to use FilePath to write a command like rm. This is because GHC internally uses a special encoding for FilePath data, that is documented to allow "arbitrary undecodable bytes to be round-tripped through it". (It seems to do this by encoding the undecodable bytes as very high unicode code points.) So, when presented with a filename that cannot be decoded using utf-8 (or whatever the system encoding is), it still handles it, and using the resulting FilePath will in fact operate on the right file. Whew! Moral of the story is that if you're going to be using GHC 7.4 to read or write filenames from a pipe, or a file, you need to arrange for the Handle you're reading or writing to use this special encoding too. I use this to set up my Handles:

import System.IO
import GHC.IO.Encoding
import GHC.IO.Handle
fileEncoding :: Handle -> IO ()
fileEncoding h = hSetEncoding h =<< getFileSystemEncoding

Even if you're only going to write a FilePath to stdout, you need to do this. Otherwise, your program will crash on some filenames! This doesn't seem quite right to me, but I hesitate to file a bug report. (And this is not a new problem in GHC anyway.) If I did, it would have this testcase:

# touch "me "
# LANG=C ghc
Prelude> :m System.Directory
Prelude System.Directory> mapM_ putStrLn =<< getDirectoryContents "."
me*** Exception: <stdout>: hPutChar: invalid argument (invalid character)

Since git-annex reads lots of filenames from git commands and other places, I had to deal with this extensively. Unfortunatly I have not found a way to read Text from a Handle using the fileSystemEncoding. So I'm stuck with slow Strings. But, it does seem to work now.

---

PS: I found a bug in GHC 7.4 today where one of those famous Haskell immutable values seems to get well, mutated. Specifically a [FilePath] that is non-empty at the top of a function ends up empty at the bottom. Unless IO is done involving it at the top. Really. Hope to develop a test case soon. Happily, the code that triggered it did so while working around a bug in GHC that is fixed in 7.4. Language bugs.. gotta love em.

I'm going to deliver this talk at fosdem 2012, room H.1301 (CrossDistribution Devroom) at 16:30 on Sat. If you are interested, please come by. In particular I'd like to talk with all the developers out there that are using our work (of edos fame) and to discuss with them future plans to migrate their programs to the new generation of mancoosi - powered QA tools. Scroll down to get the slides . fosdem link : http://fosdem.org/2012/schedule/event/distro_qa_tools

Abstract FOSS distributions are increasingly over pressure to deliver stable releases including the most up to date upstream software. Time-based release strategies has exacerbated this problem putting even more pressure on QA teams. The recently concluded Mancoosi project has developed a number of tools to automatically analyse large packages collections for a range of problems, from installability checks to speculative analysis of software repositories. In this talk I'll present four command line tools to identify and correct potential problems as soon as possible during the release cycle. In particular :

• Debcheck: This tools helps to identify all broken packages within a repository and provides a detailed explanation of the problem. This can be used to prevent shipping releases that contain packages that cannot be installed because of missing or malformed dependencies.
• Buildcheck: Given a Sources file and a set of binary repositories, this tool identifies those source packages that cannot be compiled because their build dependencies cannot be satisfied.
• Outdated: This tool identifies those broken packages that need special attention because of outdated meta-data.
• Challenged: This tool performs a speculative analysis of the repository to identify those packages that, if upgraded to a specific version, would break a large number of other packages in the repository. This tool would be particularly useful during the upgrade of a specific component to evaluate its impact on the software archive.

Most of our tools support both rpm (version 4 and 5) and deb based distributions. The mancoosi team.

Attachment	Size
main.pdf	504.46 KB

First of all, take this blog post as a rant from developer. I don't expect that this will change anything in how people ask me questions. Anyway before you write me email with some questions, please check following easy bits:

1. Have you tried to write your question into Google?
2. Most projects have documentation, have you looked at it?
3. There are mailing lists or discussion forums, do you really need to write directly to me?
4. Writing subject like "!important" or ALL IN CAPS will work other way than you probably expect.
5. Yes I have more email addresses, that does not mean you should write to all of them.

If you do follow these basic rules, you might get more help than usual. If you are interested in knowing more, there is famous How To Ask Questions The Smart Way which will tell you more information than I did in this rant.

Filed under: English Gammu Phpmyadmin 0 comments Flattr this!