Setting up basic systemd-network was recently described by Joachim, and the post inspired me to try it as well. The twist is that in my case I need a bridge for my KVM with Lava server and arm/aarch64 qemu system emulators...For background, qemu/kvm support a few ways to provide network to guests. The default is user networking, which requires no privileges, but is slow and based on ancient SLIRP code. The other common option is tap networking, which is fast, but complicated to set up. Turns out, with networkd and qemu bridge helper, tap is easy to set up.

$ for file in /etc/systemd/network/*; do echo $file; cat $file; done
/etc/systemd/network/eth.network
[Match]
Name=eth1
[Network]
Bridge=br0

/etc/systemd/network/kvm.netdev
[NetDev]
Name=br0
Kind=bridge

/etc/systemd/network/kvm.network
[Match]
Name=br0
[Network]
DHCP=yes

Diverging from Joachims simple example, we replaced "DHCP=yes" with "Bridge=br0". Then we proceed to define the bridge (in the kvm.netdev) and give it an ip via dhcp in kvm.network. From the kvm side, if you haven't used the bridge helper before, you need to give the helper permissions (setuid root or cap_net_admin) to create a tap device to attach on the bridge. The helper needs an configuration file to tell what bridge it may meddle with.

# cat > /etc/qemu/bridge.conf <<__END__
allow br0
__END__
# setcap cap_net_admin=ep /usr/lib/qemu/qemu-bridge-helper

Now we can start kvm with bridge networking as easily as with user networking:

$ kvm -m 2048 -drive file=jessie.img,if=virtio -net bridge -net nic,model=virtio -serial stdio

The manpages systemd.network(5) and systemd.netdev(5) do a great job explaining the files. Qemu/kvm networking docs are unfortunately not as detailed.

The UDD bugs interface currently knows about the following release critical bugs:

• In Total: 1168
  • Affecting Jessie: 274 That's the number we need to get down to zero before the release. They can be split in two big categories:
    • Affecting Jessie and unstable: 224 Those need someone to find a fix, or to finish the work to upload a fix to unstable:
      • 30 bugs are tagged 'patch'. Please help by reviewing the patches, and (if you are a DD) by uploading them.
      • 12 bugs are marked as done, but still affect unstable. This can happen due to missing builds on some architectures, for example. Help investigate!
      • 182 bugs are neither tagged patch, nor marked done. Help make a first step towards resolution!
    • Affecting Jessie only: 50 Those are already fixed in unstable, but the fix still needs to migrate to Jessie. You can help by submitting unblock requests for fixed packages, by investigating why packages do not migrate, or by reviewing submitted unblock requests.
      • 2 bugs are in packages that are unblocked by the release team.
      • 48 bugs are in packages that are not unblocked.

Graphical overview of bug stats thanks to azhag:

Mikkel Rasmussen:

If you look at the idea of "The Kitchen of Tomorrow" as IKEA thought about it is the core idea is that cooking is slavery. It's the idea that technology can free us from making food. It can do it for us. It can recognise who we are, we don't have to be tied to the kitchen all day, we don't have to think about it. Now if you're an anthropologist, they would tell you that cooking is perhaps one of the most complicated things you can think about when it comes to the human condition. If you think about your own cooking habits they probably come from your childhood, the nation you're from, the region you're from. It takes a lot of skill to cook. It's not so easy. And actually, it's quite fun to cook. there's also a lot of improvisation. I don't know if you ever tried to come home to a fridge and you just look into the fridge: oh, there's a carrot and some milk and some white wine and you figure it out. That's what cooking is like it's a very human thing to do.

Therefore, if you think about it, having anything that automates this for you or decides for you or improvises for you is actually not doing anything to help you with what you want to do, which is that it's nice to cook. More generally, if you make technology for example that has at its core the idea that cooking is slavery and that idea is wrong, then your technology will fail. Not because of the technology, but because it simply gets people wrong. This happens all the time. You cannot swing a cat these days without hitting one of those refrigerator companies that make smart fridges. I don't know you've ever seen them, like a "intelligent fridge". There's so many of them that there is actually a website called "Fuck your internet fridge" by a guy who tracks failed prototypes on intelligent fridges. Why? Because the idea is wrong. Not the technology, but the idea about who we are - that we do not want the kitchen to be automated for us. We want to cook. We want Japanese knives. We want complicated cooking. And so what we are saying here is not that technology is wrong as such. It's just you need to base it especially when you are innovating really big ideas on something that's a true human insight. And cooking as slavery is not a true human insight and therefore the prototypes will fail.

(I hereby nominate "internet fridge" as the term to describe products or ideas that whilst technologically sound is based on fundamentally flawed anthropology.) Hearing "I hate X" and thinking that simply removing X will provide real value to your users is short-sighted, especially when you don't really understand why humans are doing X in the first place.

I ve many distributed Linux instances across several clouds, be them global, such as Amazon or Digital Ocean, or regional clouds such as TeutoStack or Enter. Probably many of you are facing the same issue: having a consistent UNIX identity across all multiple instances. While in an ideal world LDAP would be a perfect choice, letting LDAP open to the wild Internet is not a great idea. So, how to solve this issue, while being secure? The trick is to use the new NSS module for SecurePass. While SecurePass has been traditionally used into the operating system just as a two factor authentication, the new beta release is capable of holding extended attributes , i.e. arbitrary information for each user profile. We will use SecurePass to authenticate users and store Unix information with this new capability. In detail, we will:

• Use PAM to authenticate the user via RADIUS
• Use the new NSS module for SecurePass to have a consistent UID/GID/ .

SecurePass and extended attributes The next generation of SecurePass (currently in beta) is capable of storing arbitrary data for each profile. This is called Extended Attributes (or xattrs) and -as you can imagine- is organized as key/value pair. You will need the SecurePass tools to be able to modify users extended attributes. The new releases of Debian Jessie and Ubuntu Vivid Vervet have a package for it, just:

# apt-get install securepass-tools

ERRATA CORRIGE: securepass-tools hasn t been uploaded to Debian yet, Alessio is working hard to make the package available in time for Jessie though.

For other distributions or previous releases, there s a python package (PIP) available. Make sure that you have pycurl installed and then:

# pip install securepass-tools

While SecurePass tools allow local configuration file, we highly recommend for this tutorial to create a global /etc/securepass.conf, so that it will be useful for the NSS module. The configuration file looks like:

[default]
app_id = xxxxx
app_secret = xxxx
endpoint = https://beta.secure-pass.net/

Where app_id and app_secrets are valid API keys to access SecurePass beta. Through the command line, we will be able to set UID, GID and all the required Unix attributes for each user:

# sp-user-xattrs user@domain.net set posixuid 1000

While posixuid is the bare minimum attribute to have a Unix login, the following attributes are valid:

• posixuid UID of the user
• posixgid GID of the user
• posixhomedir Home directory
• posixshell Desired shell
• posixgecos Gecos (defaults to username)

Install and Configure NSS SecurePass In a similar way to the tools, Debian Jessie and Ubuntu Vivid Vervet have native package for SecurePass:

# apt-get install libnss-securepass

For previous releases of Debian and Ubuntu can still run the NSS module, as well as CentOS and RHEL. Download the sources from: https://github.com/garlsecurity/nss_securepass Then:

./configure
make
make install (Debian/Ubuntu Only)

For CentOS/RHEL/Fedora you will need to copy files in the right place:

/usr/bin/install -c -o root -g root libnss_sp.so.2 /usr/lib64/libnss_sp.so.2
ln -sf libnss_sp.so.2 /usr/lib64/libnss_sp.so

The /etc/securepass.conf configuration file should be extended to hold defaults for NSS by creating an [nss] section as follows:

[nss]
realm = company.net
default_gid = 100
default_home = "/home"
default_shell = "/bin/bash"

This will create defaults in case values other than posixuid are not being used. We need to configure the Name Service Switch (NSS) to use SecurePass. We will change the /etc/nsswitch.conf by adding sp to the passwd entry as follows:

$ grep sp /etc/nsswitch.conf
 passwd:  files sp

Double check that NSS is picking up our new SecurePass configuration by querying the passwd entries as follows:

$ getent passwd user
 user:x:1000:100:My User:/home/user:/bin/bash

$ id user
 uid=1000(user)  gid=100(users) groups=100(users)

Using this setup by itself wouldn t allow users to login to a system because the password is missing. We will use SecurePass authentication to access the remote machine. Configure PAM for SecurePass On Debian/Ubuntu, install the RADIUS PAM module with:

# apt-get install libpam-radius-auth

If you are using CentOS or RHEL, you need to have the EPEL repository configured. In order to activate EPEL, follow the instructions on http://fedoraproject.org/wiki/EPEL Be aware that this has not being tested with SE-Linux enabled (check off or permissive). On CentOS/RHEL, install the RADIUS PAM module with:

# yum -y install pam_radius

Note: as per the time of writing, EPEL 7 is still in beta and does not contain the Radius PAM module. A request has been filed through RedHat s Bugzilla to include this package also in EPEL 7 Configure SecurePass with your RADIUS device. We only need to set the public IP Address of the server, a fully qualified domain name (FQDN), and the secret password for the radius authentication. In case of the server being under NAT, specify the public IP address that will be translated into it. After completion we get a small recap of the already created device. For the sake of example, we use secret as our secret password. Configure the RADIUS PAM module accordingly, i.e. open /etc/pam_radius.conf and add the following lines:

radius1.secure-pass.net secret 3
radius2.secure-pass.net secret 3

Of course the secret is the same we have set up on the SecurePass administration interface. Beyond this point we need to configure the PAM to correct manage the authentication. In CentOS, open the configuration file /etc/pam.d/password-auth-ac; in Debian/Ubuntu open the /etc/pam.d/common-auth configuration and make sure that pam_radius_auth.so is in the list.

auth required   pam_env.so
auth sufficient pam_radius_auth.so try_first_pass
auth sufficient pam_unix.so nullok try_first_pass
auth requisite  pam_succeed_if.so uid >= 500 quiet
auth required   pam_deny.so

Conclusions Handling many distributed Linux poses several challenges, from software updates to identity management and central logging. In a cloud scenario, it is not always applicable to use traditional enterprise solutions, but new tools might become very handy. To freely subscribe to securepass beta, join SecurePass on: http://www.secure-pass.net/open
And then send an e-mail to info@garl.ch requesting beta access.

Two weeks ago I left NYC for a day trip to Norfolk, Virginia, to attend xTupleCon 2014. For those who don t know, xTuple is an incredibly reknown open source Enterprise Resource Planning (ERP) software. If you ve been following this blog, you might recall that during my participation in the Google Summer of Code 2014, I wrote a beta JSCommunicator extension for xTuple (to see how I went about doing that, look up Kickstarting the JSCommunicator-xTuple extension). Now, I wasn t flying down to Virginia on the eve of my first grad school midterms (gasp!) for fun - although I will admit, I enjoyed myself a fair amount. My GSoC mentor, Daniel Pocock was giving a talk about WebRTC and JSCommunicator at xTupleCon and invited me to participate. So that was the first perk of going to xTupleCon, I got to finally meet my GSoC mentor in person! During the presentation, Daniel provided a high level expanation of WebRTC and how it works. WebRTC (Web Real Time Communications) enables real time transmission of audio, video and data streams through browser-to-browser applications. This is one of the many perks of WebRTC; it doesn t require installation of plugins, making its use less vulnerable to security breaches. Websockets are used for the signalling channel and the SIP or XMPP can be used as the signaling protocol - in JSCommunicator, we used SIP. What was also done in JSCommunicator and can be done in other applications is to use a library to implement the signaling and SIP stack. JSCommunicator uses (and I highly reccomend) JsSIP. The basic SIP architecture is comprised of peers and a SIP Proxy Server that supports SIP over Websockets transport: If you have users behind NAT, you ll also need a TURN server for relay: In both cases, setup is not too difficult, particularly if using reSIProcate which offers both the SIP proxy and the TURN server. Daniel Pocock has an excellent post on how to setup and configure your SIP proxy and TURN server. With regard to JSCommunicator, it is a generic telephone in HTML5 which can easily be embedded in any web site or web app. Almost every aspect of JSCommunicator is easily customizable. More about JSCommunicator setup and architecture is detailed in a previous post. The JSCommunicator-xTuple extension can be installed in xTuple as an npm package (xtuple-jscommunicator). It is still at a very beta - or even pre-beta - stage and there are various limitations; the configuration must be hard-coded and dialing is done manually as opposed to clicking on a contact. Some of these limitations are features are on the wish list for future work. For example, some ideas for the next version of the extension are a click to dial from CRM records functionality and to bring up CRM records for an incomming call. Additionaly, the SIP proxy could be automatically installed with the xTuple server installation if desired. We closed the presentation with a live demo during which I made a video call from a JSCommunicator instance embedded in freephonebox.net to the JSCommunicator xTuple extension running on Daniel s laptop. Despite the occasionally iffy hotel Wifi, the demo was a hit - at one point I even left the conference room and took a quick walk around the hotel and invited other xTupleCon attendees to say hello to those in the room. The audience s reception was more enthusiastic than I anticipated, giving way to a pretty extensive Q&A session. It s great to see more and more people interested in WebRTC, I can t emphasize enough what a useful and versatile tool it is. Here s an action shot of part of the xTuple WebRTC presentation:

I joined the board of directors of the Free Software Foundation a couple of weeks ago. I've been travelling a bunch since then, so haven't really had time to write about it. But since I'm currently waiting for a test job to finish, why not?

It's impossible to overstate how important free software is. A movement that began with a quest to work around a faulty printer is now our greatest defence against a world full of hostile actors. Without the ability to examine software, we can have no real faith that we haven't been put at risk by backdoors introduced through incompetence or malice. Without the freedom to modify software, we have no chance of updating it to deal with the new challenges that we face on a daily basis. Without the freedom to pass that modified software on to others, we are unable to help people who don't have the technical skills to protect themselves.

Free software isn't sufficient for building a trustworthy computing environment, one that not merely protects the user but respects the user. But it is necessary for that, and that's why I continue to evangelise on its behalf at every opportunity.

However.

Free software has a problem. It's natural to write software to satisfy our own needs, but in doing so we write software that doesn't provide as much benefit to people who have different needs. We need to listen to others, improve our knowledge of their requirements and ensure that they are in a position to benefit from the freedoms we espouse. And that means building diverse communities, communities that are inclusive regardless of people's race, gender, sexuality or economic background. Free software that ends up designed primarily to meet the needs of well-off white men is a failure. We do not improve the world by ignoring the majority of people in it. To do that, we need to listen to others. And to do that, we need to ensure that our community is accessible to everybody.

That's not the case right now. We are a community that is disproportionately male, disproportionately white, disproportionately rich. This is made strikingly obvious by looking at the composition of the FSF board, a body made up entirely of white men. In joining the board, I have perpetuated this. I do not bring new experiences. I do not bring an understanding of an entirely different set of problems. I do not serve as an inspiration to groups currently under-represented in our communities. I am, in short, a hypocrite.

So why did I do it? Why have I joined an organisation whose founder I publicly criticised for making sexist jokes in a conference presentation? I'm afraid that my answer may not seem convincing, but in the end it boils down to feeling that I can make more of a difference from within than from outside. I am now in a position to ensure that the board never forgets to consider diversity when making decisions. I am in a position to advocate for programs that build us stronger, more representative communities. I am in a position to take responsibility for our failings and try to do better in future.

People can justifiably conclude that I'm making excuses, and I can make no argument against that other than to be asked to be judged by my actions. I hope to be able to look back at my time with the FSF and believe that I helped make a positive difference. But maybe this is hubris. Maybe I am just perpetuating the status quo. If so, I absolutely deserve criticism for my choices. We'll find out in a few years.

comments

Even though the Wikipedia defines gamer as someone who partakes in interactive gaming, such as (predominantly) video games or board games , this doesn t really gets close to that term means socially at the moment. Going back to Wikipedia, we find that the video game subculture is a form of new media subculture that has been influenced by video games , so it might be quite accurate to define gamers as members of that subculture. You will find that most of the uses of the term gamer in the social networks and in the blogosphere refer to that. Please notice that, even though it is quite likely that most of the gamers play video games, the other way round does not need to be true and, in fact, it isn t. Not everyone who plays video games belongs to the video game subculture, shares their point of view, their values and aesthetics, or even know about it. Kind of like what happens with the word hacker . Not everyone who hacks around with a computer belongs to the hacker subculture. Mostly everyone who has access to the technology plays video games now. From babies and kids to grandparents. And people play them in every possible technological system around, not only on video game consoles or personal computers, but alse on mobile phones, tablets, web browsers. And many of those people who use different kind of technologies to play video games are not gamers. Not in the sense of belonging to the video game subculture. It is important to acknowledge that: that the video game subculture does not have the monopoly over video games or the video game developing industry anymore. As you can imagine, all this rand doesn t come from nowhere. During the last months, we have been witnessing a fight between some conservative core members of the video game subculture and people who want to bring some fresh air into the sociocultural elements of that subculture. Namely, that women shouldn t be discriminated inside it. As every time that a women raises her voice to complain about anything in the Internet, they have been subjected to insults, attacks, rape and death threats, etc. I m talking about something called #GamerGate, and even though I m not going to get into it, I will provide some URLs in case someone might be interested. Please acknowledge that not all the points of view might be represented in this list (in fact, they are not, as I won t be promoting in my blog things that I severely disagree with), so search the web for more information if you want to get that.

• Gamergate: A Scandal Erupts in the Video-Game Community
• What death threats have to do with video games
• Why #Gamergaters Piss Me The F*** Off
• Actress Felicia Day Opens Up About GamerGate Fears, Has Her Private Details Exposed Minutes Later
• Felicia Day speaks candidly about #GamerGate, immediately gets attacked
• VGS- #GamerGate: Threats of Mass Shooting and Rape for Journalistic Integrity- Interview with Brianna Wu + Jennifer Allaway
• Gamergate can t stop being about harassment
• Brianna Wu and the human cost of Gamergate: every woman I know in the industry is scared
• Rape and death threats are terrorizing female gamers. Why haven t men in tech spoken out?
• Why GamerGate Can t Continue
• The only guide to Gamergate you will ever need to read
• Gamergate s anti-woman agenda made clear: Actor Felicia Day threatened for speaking up
• Gloria Steinem: Gamergate is making the web especially hostile and dangerous
• #GamerGate Has Finally Made My Head Explode
• Gamergate Should Stop Lying to Journalists and Itself
• #GamerGate: the misogynist movement blighting the video games industry
• #Gamergate is really about terrorism: Why Bill Maher should be vilifying the gaming community, too
• Women Are Being Driven Offline : Feminist Anita Sarkeesian Terrorized for Critique of Video Games
• The Hashtags and Death Threats Behind #GamerGate, Gaming s Latest Controversy
• Anita Sarkeesian on GamerGate: We Have a Problem and We re Going to Fix This
• Tropes vs Women in Video Games by Anita Sarkeesian
• Anita Sarkeesian, Feminist Frequency XOXO Festival (2014)
• Is GamerGate About Media Ethics or Harassing Women? Harassment, the Data Shows
• What Is #GamerGate and Why Are Women Being Threatened About Video Games?
• Gamergate: Female gamers and developers threatened with rape, murder as online row rages
• Feminist Critics of Video Games Facing Threats in GamerGate Campaign
• Gamergate: when outsiders become the oppressors
• The Future Of The Culture Wars Is Here, And It s Gamergate
• On GamerGate: A letter from the editor
• Pew Research about Online Harassment
• Pew Poll: People Believe Online Gaming is the Least Welcoming Space for Women
• Pew: Gaming Is Least Welcoming Online Space For Women
• A Narrative of #GamerGate and Examination of Claims of Collusion with 4chan
• The End of Gamers
• Nobody Wins the GamerGate Civil War
• A GamerGate recognition guide
• Visual timeline of the #GamerGate events

I ve never been a gamer myself, meaning part of the subculture I mentioned. At some point I was probably closer tho the core values they had then than I am now. In any case, video games have already consolidated themselves as an important part of current culture, entertainment, education and socialization, and are definitely here to stay. That will probably mean that the percentage of gamers (members of the video game subculture) will become smaller. as the number of non-gamer video game players keeps raising.

Just a friendly reminder: If your package is not in unstable (and reasonably bug free) by Sunday, it's not in Jessie. I am not doing full stats as I am unsure about the diff format at the moment, but in week 43, we had 284 bugs for Squeeze and 468 for Wheezy. (282 + 468) / 2 = 376; so we are a bit better off than on average. Still, here's to hoping this freeze will be shorter. The UDD bugs interface currently knows about the following release critical bugs:

• In Total: 1193
  • Affecting Jessie: 319 That's the number we need to get down to zero before the release. They can be split in two big categories:
    • Affecting Jessie and unstable: 240 Those need someone to find a fix, or to finish the work to upload a fix to unstable:
      • 20 bugs are tagged 'patch'. Please help by reviewing the patches, and (if you are a DD) by uploading them.
      • 22 bugs are marked as done, but still affect unstable. This can happen due to missing builds on some architectures, for example. Help investigate!
      • 198 bugs are neither tagged patch, nor marked done. Help make a first step towards resolution!
    • Affecting Jessie only: 79 Those are already fixed in unstable, but the fix still needs to migrate to Jessie. You can help by submitting unblock requests for fixed packages, by investigating why packages do not migrate, or by reviewing submitted unblock requests.
      • 0 bugs are in packages that are unblocked by the release team.
      • 79 bugs are in packages that are not unblocked.

About a month ago, I received an upstream bugreport that the nbd-server wouldn't build on Solaris and its derivatives. This was because nbd-server uses the d_type field of struct dirent, which is widely implemented (in Linux and FreeBSD, at least), but not part of POSIX and therefore not implemented on Solaris (which tends to be more conservative about implementing new features). The bug reporter pointed towards a blog post by a Solaris user who had written something he calls "adirent", meant to work around the issue by implementing something that would wrap readdir() so that it would inject a stat() call when needed. While that approach works, it seems a bit strange to add a function which wraps readdir to become portable. After all, readdir() does not always return the file type in d_type, not even on systems that do implement it. One example in which this is true is XFS; if one runs readdir() on a directory on an XFS filesystem, then everything will have DT_UNKNOWN as its filetype, indicating that you need to run stat() after all. As such, I think a better approach is to use that fact so that things will just work on systems where d_type isn't available. The GNU autotools even have a test for it (AC_STRUCT_DIRENT_D_TYPE), which makes things easier. In the case of NBD, I've added that to configure.ac, and then added a touch of preprocessor magic to reuse the infrastructure for dealing with DT_UNKNOWN which is already there:

#ifdef HAVE_STRUCT_DIRENT_D_TYPE
#define NBD_D_TYPE de->d_type
#else
#define NBD_D_TYPE 0
#define DT_UKNOWN 0
#define DT_REG 1
#endif

(...opendir(), readdir(), ...)

switch(NBD_D_TYPE)  
    case DT_UNKNOWN:

(...call stat(), figure out if it is a file...)

    case DT_REG:

(...we know it is a file...)

    default:

(...we know it is not a file...) this seems cleaner to me than using a wrapper, and has the additional advantage that the DT_UNKNOWN code path could receive some more testing.

Petter posted yesterday about Listadmin, the quick way to moderate mailman lists. Petter: THANKS. I am a fan of automatization. But, yes, I had never thouguht of doing this. Why? Don't know. But this is way easier than using the Web interface for Mailman:

$ listadmin 
fetching data for conoc_des@my.example.org ... nothing in queue
fetching data for des_polit_pub@my.example.org ... nothing in queue
fetching data for econ_apl@my.example.org ... nothing in queue
fetching data for educ_ciencia_tec@my.example.org ... nothing in queue
fetching data for est_hacend_sec_pub@my.example.org ... 
[1/1] ============== est_hacend_sec_pub@my.example.org ======
From:     sender@example.org                                            
Subject:  Invitaci n al Taller Insumo Producto                          
Reason:   El cuerpo del mensaje es demasiado grande: 777499    Spam? 0  
Approve/Reject/Discard/Skip/view Body/Full/jump #/Undo/Help/Quit ? a
Submit changes? [yes] 
fetching data for fiscal_fin@my.example.org ... nothing in queue
fetching data for historia@my.example.org ... nothing in queue
fetching data for industrial@my.example.org ... nothing in queue
fetching data for medio_amb@my.example.org ... nothing in queue
fetching data for mundial@my.example.org ... nothing in queue
fetching data for pol_des@my.example.org ... nothing in queue
fetching data for sec_ener@my.example.org ... nothing in queue
fetching data for sec_prim@my.example.org ... nothing in queue
fetching data for trab_tec@my.example.org ... nothing in queue
fetching data for urb_reg@my.example.org ... nothing in queue
fetching data for global@my.example.org ... nothing in queue

I don't know how in many years of managing several mailing lists I never thought about this! I'm echoing this, as I know several of my readers run mailman as well, and might not be following Planet Debian.

If you ever had to moderate a mailman list, like the ones on alioth.debian.org, you know the web interface is fairly slow to operate. First you visit one web page, enter the moderation password and get a new page shown with a list of all the messages to moderate and various options for each email address. This take a while for every list you moderate, and you need to do it regularly to do a good job as a list moderator. But there is a quick alternative, the listadmin program. It allow you to check lists for new messages to moderate in a fraction of a second. Here is a test run on two lists I recently took over:

% time listadmin xiph
fetching data for pkg-xiph-commits@lists.alioth.debian.org ... nothing in queue
fetching data for pkg-xiph-maint@lists.alioth.debian.org ... nothing in queue
real    0m1.709s
user    0m0.232s
sys     0m0.012s
%

In 1.7 seconds I had checked two mailing lists and confirmed that there are no message in the moderation queue. Every morning I currently moderate 68 mailman lists, and it normally take around two minutes. When I took over the two pkg-xiph lists above a few days ago, there were 400 emails waiting in the moderator queue. It took me less than 15 minutes to process them all using the listadmin program. If you install the listadmin package from Debian and create a file ~/.listadmin.ini with content like this, the moderation task is a breeze:

username username@example.org
spamlevel 23
default discard
discard_if_reason "Posting restricted to members only. Remove us from your mail list."
password secret
adminurl https:// domain /mailman/admindb/ list 
mailman-list@lists.example.com
password hidden
other-list@otherserver.example.org

There are other options to set as well. Check the manual page to learn the details. If you are forced to moderate lists on a mailman installation where the SSL certificate is self signed or not properly signed by a generally accepted signing authority, you can set a environment variable when calling listadmin to disable SSL verification:

PERL_LWP_SSL_VERIFY_HOSTNAME=0 listadmin

If you want to moderate a subset of the lists you take care of, you can provide an argument to the listadmin script like I do in the initial screen dump (the xiph argument). Using an argument, only lists matching the argument string will be processed. This make it quick to accept messages if you notice the moderation request in your email. Without the listadmin program, I would never be the moderator of 68 mailing lists, as I simply do not have time to spend on that if the process was any slower. The listadmin program have saved me hours of time I could spend elsewhere over the years. It truly is nice free software. As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b. Update 2014-10-27: Added missing 'username' statement in configuration example. Also, I've been told that the PERL_LWP_SSL_VERIFY_HOSTNAME=0 setting do not work for everyone. Not sure why.

Last night I mostly patched my local copy of less to build and link against the PCRE regular expression library. I've wanted to do that for a while, and reading Raymond Chen's blog post last night made me try it out. The patch was small and pretty neat, and I'm familiar with GNU less having patched it in the past. But it doesn't contain tests. Test cases are hard. Many programs, such as less, are used interactively which makes writing a scaffold hard. Other programs suffer from a similar fate - I'm not sure how you'd even test a web browser such as Firefox these days - mangleme would catch some things, eventually, but the interactive stuff? No clue. In the past MySQL had a free set of test cases, but my memory is that Oracle locked them up. SQLite is famous for its decent test coverage. But off the top of my head I can't think of other things. As a topical example there don't seem to be decent test-cases for either bash or openssl. If it compiles it works, more or less. I did start writing some HTTP-server test cases a while back, but that was just to automate security attacks. e.g. Firing requests like:

GET /../../../etc/passwd HTTP/1.0
GET //....//....//....//etc/passwd HTTP/1.0
etc

(It's amazing how many toy HTTP server components included in projects and products don't have decent HTTP-servers.) I could imagine that being vaguely useful, especially because it is testing the protocol-handling rather than a project-specific codebase. Anyway, I'm thinking writing test cases for things is good, but struggling to think of a decent place to start. The project has to be:

• Non-interactive.
• Open source.
• Widely used - to make it a useful contribution.
• Not written in some fancy language.
• Open to receiving submissions.

Comments welcome; but better yet why not think about the test-coverage of any of your own packages and projects...?

I ve previously covered running LAVA on ARM devices, now that the packages are in Debian. I ve also covered setting up the home lab, including the difficulty in obtaining the PDU and relying on another machine to provide USB serial converters with inherent problems of needing power to keep the same devices assigned to the same ser2net ports. There have been ideas about how to improve the situation. Conferences are a prime example setting up a demo involving LAVA means bringing a range of equipment, separate power bricks, separate network switches (with power bricks), a device of some kind to connect up the USB serial converters (and power brick) and then the LAVA server (with SATA drive and power brick) that is without the actual devices and their cables and power. Each of those power cables tend to be a metre long, with networking and serial, it quickly becomes a cable spaghetti. Ideas around this also have application inside larger deployments, so the hardware would need to daisy-chain to provide services to a rack full of test devices. The objective is a single case providing network, power and serial connectivity to a number of test devices over a single power input and network uplink. Naturally, with a strong free software and open development bias, the unit will be Open Hardware running Debian, albeit with a custom Beaglebone Linux kernel. It s a Test Automation Controller, so we re using the name OpenTAC. Progress Open hardware ARM device running Debian to automate tests on 4 to 8 devices, initially aimed at LAVA support for Linaro engineers. Power distribution, serial console, network and optional GPIO extensions. The design involves:

• A Beaglebone Black (revC)
  • USB hotplug support required, certainly during development.
• Custom PCB connected as a Beaglebone Cape, designed by Andy Simpkins.
• Base board provides 4 channels:
  • 5V Power delivered over USB
  • Ethernet standard Cat5, no LEDs
  • Serial connectivity
    • RS232
    • UART
  • GPIO
• Internal gigabit network switch
• Space for a board like a CubieTruck (with SATA drive) to act as LAVA server
• Daughter board:
  • Same basic design as the base board, providing another 4 channels, equivalent to the base channels. When the daughter board is fitted, a second network switch would be added instead of the CubieTruck.
• Power consumption measurement per channel
  • queries made via the Beaglebone Black over arbitrary time periods, including during the test itself.
• The GPIO lines can be used to work around issues with development boards under test, including closing connections which may be required to get a device to reboot automatically, without manual intervention.
• Serial connections to test devices can be isolated during device power-cycles this allows for devices which pull power over the serial connection. (These are typically hardware design issues but the devices still need to be tested until the boards can be modified or replaced.)
• Thermal control, individual fan control via the Beaglebone Black.
• 1U case rackable or used alone on the desk of developers.
• Software design:
  • lavapdu backend module for PDU control (opentac.py) & opentac daemon on the BBB
    • telnet opentac-01 3225
  • ser2net for serial console control
    • telnet opentac-01 4000

The initial schematics are now complete and undergoing design review. A lot of work remains

A new maintenance release of littler is available now. The main change are a few updates and extensions to the examples provided along with littler. Several of those continue to make use of the wonderful docopt package by Edwin de Jonge. Carl Boettiger and I are making good use of these littler examples, particularly to install directly from CRAN or GitHub, in our Rocker builds of R for Docker (about which we should have a bit more to blog soon too). Full details for the littler release are provided as usual at the ChangeLog page. The code is available via the GitHub repo, from tarballs off my littler page and the local directory here. A fresh package has gone to the incoming queue at Debian; Michael Rutter will probably have new Ubuntu binaries at CRAN in a few days too. Comments and suggestions are welcome via the mailing list or issue tracker at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

I m on my way home from D sseldorf where I attended the LinuxCon Europe and Linux Plumber conferences. I was quite surprised how huge LinuxCon was, there were about 1.500 people there! Certainly much more than last year in New Orleans. Containers (in both LXC and docker flavors) are the Big Thing everybody talks about and works with these days; there was hardly a presentation where these weren t mentioned at all, and (what felt like) half of the presentations were either how to improve these, or how to use these technologies to solve problems. For example, some people/companies really take LXC to the max and try to do everything in them including tasks which in the past you had only considered full VMs for, like untrusted third-party tenants. For example there was an interesting talk how to secure networking for containers, and pretty much everyone uses docker or LXC now to deploy workloads, run CI tests. There are projects like fleet which manage systemd jobs across an entire cluster of containers (distributed task scheduler) or like project-builder.org which auto-build packages from each commit of projects. Another common topic is the trend towards building/shipping complete (r/o) system images, atomic updates and all that goodness. The central thing here was certainly Stateless systems, factory reset, and golden images which analyzed the common requirements and proposed how to implement this with various package systems and scenarios. In my opinion this is certainly the way to go, as our current solution on Ubuntu Touch (i. e. Ubuntu s system-image) is far too limited and static yet, it doesn t extend to desktops/servers/cloud workloads at all. It s also a lot of work to implement this properly, so it s certainly understandable that we took that shortcut for prototyping and the relatively limited Touch phone environment. On Plumbers my main occupations were mostly the highly interesting LXC track to see what s coming in the container world, and the systemd hackfest. On the latter I was again mostly listening (after all, I m still learning most of the internals there..) and was able to work on some cleanups and improvements like getting rid of some of Debian s patches and properly run the test suite. It was also great to sync up again with David Zeuthen about the future of udisks and some particular proposed new features. Looks like I m the de-facto maintainer now, so I ll need to spend some time soon to review/include/clean up some much requested little features and some fixes. All in all a great week to meet some fellows of the FOSS world a gain, getting to know a lot of new interesting people and projects, and re-learning to drink beer in the evening (I hardly drink any at home :-P). If you are interested you can also see my raw notes, but beware that there are mostly just scribbling. Now, off to next week s Canonical meeting in Washington, DC!

A new release 0.1.0 of the RPushbullet package (interfacing the neat Pushbullet service) landed on CRAN today. It brings a number of goodies relative to the first release 0.0.2 of a few months ago:

• pushing of files is now supported thanks to a nice pull request bu Mike Birdgeneau
• a default device can be designated in the ~/.rpushbullet.json file or options
• initialization has been rewritten to use recpients which can be indices, device names or, if missing entirely, the (new) default device
• alternatively, email is supported as another recipient option in which case the Pushbullet service will send an email to the give address
• pbGetDevices() now returns a proper S3 object with corresponding print() and summary() methods
• the documentation regarding package initialization, and setting of key, devices, etc has been expanded
• more examples has been added to the documentation
• various minor cleanups, fixes, corrections throughout

There is a whole boat load of more wickedness in the Pushbullet API so if anybody feels compelled to add it, fire off pull requests at GitHub. More details about the package are at the RPushbullet webpage and the RPushbullet GitHub repo. Courtesy of CRANberries, there is also a diffstat report for this release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

I've just recently built the large bulk of VMs that we use for first semester teaching. This year that was 112. We use the same general approach for these as our others: get a generic base image up and running, with just enough configuration complete so a puppet client starts up; get it talking to our master; let puppet take it from there. There are pragmatic balances between how much we do in the kickstart versus how much we do in puppet, but also when we build a new VM from scratch versus when we clone an existing image, and how specialisation we do in the clone image. Unfortunately this year we ended up in a situation where our clone image wouldn't talk to our puppet master out of the box, due to some changes we'd made to our master set up since the clone image was prepared. We didn't really have enough time to re-clone the entire set of VMs from a fixed base image, and instead needed to fix them whilst up. However we couldn't rely on puppet to do that, since they wouldn't talk to the puppet master. We needed to manually reset the puppet client state per VM and then re-establish a trust relationship with the correct master (which is not the default master hostname in our environment anymore). Luckily, we deploy a local account with a known passphrase via the kickstart, which also has sudo access, as an interim measure before puppet strips it back out again and sets up proper LDAP and Kerberos authentication. So we can at least get into the boxes. However logging into 112 VMs by hand is not a particularly pleasant task. In the past I might have tried to achieve this using something like clusterssh but this year I decided to give ansible a try instead. Ansible started life, I believe, as a tool that would let you run arbitrary commands on remote hosts, including navigating ssh and sudo as required, without needing any agent software on the remote end. It has since seemed to grow into an enterprise product in its own right, seemingly in competition with puppet, chef, cfengine et al. Looking at the Ansible website now I'd be rather put off by just how "enterprisey" it has become - much as I am by the puppet website, if I'm honest - but if you persevere past the webinars, testimonials, etc. etc., you can find yourself to the documentation, and running an arbitrary command is as simple as

• defining a list of hosts
• running an ansible command line referencing some or all of those hosts

The hosts file format is simple

[somehosts]
host1
host2
...
[otherhosts]
host3

The command line can be a little bit more complex, especially if you need to use one username for ssh, another for sudo, and you don't want to use ssh key auth:

ansible -i ./hostsfile somehosts -k -u someuser \
    --sudo -K -a 'puppet agent --onetime --no-daemonize --verbose 

"all" would work where I've used somehosts in the example above. So there you go: using one configuration management system to bootstrap another. I'm sure I've reserved myself a special place in hell for this.

Hi everybody, We have gotten some inquires about how Elektra s merge functionality compares to the merge functionality built into git: git merge-file. I am glad to say that Elektra outperforms git s merge functionality in the same ways it outperforms diff3 when applied to configuration files. Obviously, git s merge functionality does a much better job with source-code as that is not the goal of elektra. In that previous example, I showed that because diff3 is lined based, where Elektra is not (unless you mount a file using the line plugin). The example I used before, and I will go over again, is using smb.conf and in-line comments. Many of our storage plug-ins understand the difference between comments and actual configuration data. So if a configuration file has an inline comment like so:
max log size = 10000 ; Controls the size of the log file (in KiB)
we can compare the actual Keys, value pairs between versions max log size = 10000 and deal with the comments separately. As a result, if we have a base:
max log size = 1000 ; Size in KiB

Ours:
max log size = 10000 ; Size in KiB

Theirs:
max log size = 1000 ; Controls the size of the log file (in KiB)

The result using elektra-merge would be:
max log size = 10000 ; Controls the size of the log file (in KiB)
Just like diff3, git merge-file can only compare these lines as lines, and thus there is a conflict. When running git merge-file smb.conf.ours smb.conf.base smb.conf.theirs we get the following output showing a conflict:
<<<<<<< smb.conf.ours
max log size = 2000 ; Size in KiB
=======
max log size = 1000 ; Controls the size of the log file (in KiB)
>>>>>>> smb.conf.theirs
This really shows the strength of Elektra s plugin system and why it makes merge an obvious use-case of Elektra. I hope this example makes it clear why using Elektra s merge functionality is advantageous over git s merge functionality. Once again I would like to stress the importance of quality storage plug-ins for Elektra. The more quality plugins we have the more powerful Elektra can be. If you are interested in plugins and would like to help us by adding functionality to Elektra by creating a new plug-in be sure to read my basic tutorial on how to do so. Sincerely,
Ian Donnelly