After returning from a vacation, I went over my RTL bugs backlog. I missed helping with checking a few patches, so I got left with verifying the fixes done by the KACST guys on master. Last week was dedicated for verifications of the bug fixes, even found a regression and reverted the commit after discussion with the author. Yesterday, I took my time with pushing these fixes also to the 4-0 branch, having them available as soon as possible instead of waiting for the 4.1 release. Total of 7 bugs got pushed, see https://wiki.documentfoundation.org/RTL_Bugs#4.0.3 for the list. I notified the developers, so they ll know their changes are available to the public sooner than expected (and let the people enjoy their work earlier). All these pushes make sure that master doesn t have any RTL support superiority over the current stable branch. In one of the cases I had to do some follow up about the commit, finding another relevant commit. This was merged with a trick Eike taught me a while ago for squashing commits with git rebase. Fridrich quickly helped with verifying that the squash stills looks good, letting me push another dependent fix on top of it.
Filed under: LibreOffice

This package is my pure C test harness that understands the TAP protocol. It's akin to prove, except written in C. It also provides a TAP library for C and shell scripts, which is useful even if you don't want to use the harness. I was going to just do a small release with no major changes, just some cleanup and merging of changes from elsewhere, but I do hate making tiny releases and I'd been meaning to get back to doing some larger development for a while. So this release has several major changes, including one backward-incompatible one. The backward-incompatible change is that, to specify a list of tests to run, one now needs to use the -l option to runtests. That's because the default interpretation of the command line is now that it is a list of tests to run and summarize. This is more consistent with other test harnesses, such as prove. Also in this release, runtests will look for the test name as given if it can't find it by appending -t or .t, which allows one to pass the actual name of the test executable on the command line (with or without the -o option). This makes command-line completion less annoying. There are a few other minor fixes to display: output is flushed after each test executable completes even if the output isn't a terminal, which produces better display when teeing to a log file, and partial status of tests that use lazy plans now show the total test count as "?" to match the behavior of prove. I also fixed a few errors when displaying a test abort to a terminal. You can get the latest version from the C TAP Harness distribution page.

This is basically a link-post to the Debian Project Leader election email discussions on GMANE s blog-style interface to debian-vote. After only 3 days of the 21, there s already a pageful, so if I don t start collecting links now, I ll probably miss some. Right or wrong, I ve grouped these into three topics: The Job

1. Why do you think you are a good candidate for DPL (10 Mar 2013)
2. How do you plan to represent Debian externally? (10 Mar 2013)
3. about a DPL board (12 Mar 2013)
4. DPL term duration (12 Mar 2013)
5. Work balance and traveling (12 Mar 2013)
6. trying to do awesome and risking to fail (11 Mar 2013)
7. To Lucas: how do you plan to push your ideas (12 Mar 2013)
8. All candidates quotes for the press if you win (13 Mar 2013)

Money

1. using debian funds for Debian s hardware infrastru (12 Mar 2013)
2. Usage of Debian s Money (12 Mar 2013)
3. Debian s relationship with money and the economy (12 Mar 2013)

Project Management

1. getting new people to Debian (10 Mar 2013)
2. Free Software challenges and Debian role (11 Mar 2013)
3. Development and technical issues and challenges (10 Mar 2013)
4. Are there problematic infrastructure or processes in Debian? (12 Mar 2013)
5. to Moray: encourage teams to take interns (11 Mar 2013)

So, what do you think are the key points or differences? Leave me a comment, or get involved in the discussions. Campaigning ends and voting begins 30/31 March.

A bug I've recently seen in production code gave me the idea for this blog post. Probably smarter people already wrote better things on this topic, so this is mostly for myself, to better summarise my own thoughts. Corrections are welcome, please leave a comment! Let's say we have a somewhat standard API in Python or C++:

• a function (init_t) to create/initialise our data type, which returns an object of type t
• the object then has some properties or methods that we can call

Signalling failures to initialise the object can be done in two ways: either by raising an exception, or by returning a null/None result. There are advantages and disadvantages to both:

• if using exceptions, then you must be sure to catch and handle all possible exceptions that the initialisation function can raise (otherwise more of your code can be aborted than you want)
• if using None as return value, then you must check that the value is correct before using it, otherwise you get an exception or null pointer dereference

The None model can create latent bugs, for example in the following code:

ok = True
for arg in input_list:
  t = init_t(arg)
  if t is None:
    ok = False
    continue
  t.process()
return ok

The presence of the continue statement there is critical. Removing it, or moving it after some other statements which work with t will result in a bug. So using value-based returns forces us to introduce (manually) control points, without having the possibility to validate the model by the compiler (e.g. in C++). So it would seem that this kind of latent bugs pushes us to use the exception model, with its drawbacks. Let's look at how this interface would be implemented in (IMHO) idiomatic Haskell (where the a and b types represent the input and output types):

initT :: a -> Maybe T
processT :: T -> b
my_fn =
  
 case initT arg of
   Nothing -> -- handle failure
   Just v -> processT v

Yes, this can be written better, but it's beside the main point. The main point is that by introducing a wrapper type around our main type (T), we are forced via the type system to handle the failure case. We can't simply pass the result of initT to a function which accepts T, because it won't type check. And, no matter what we do with the result value, there are no exceptions involved here, so we only have to think about types/values, and not control flow changes. In effect, types become automatically-validated control-flow instructions. Or so it looks to me . So using types properly, we can avoid the exception-vs-return-value debate, and have all advantages without the disadvantages of either. If that is the case, why isn't this technique used more in other languages? At least in statically typed languages, it would be possible to implement it (I believe), via a C++ template, for example. In Python, you can't actually apply it, as there's no static way of enforcing the correct decapsulation. I was very saddened to see that Google's Go language, which is quite recent, has many examples where initialisation functions return a tuple err, value = , separating the actual value from the error, making it not safer than Python. It might be that polymorphic types are not as easy to work with, or it might be the lack of pattern matching. In any case, I don't think this was the last time I've seen a null pointer dereference (or the equivalent AttributeError: 'NoneType' object has no attribute ). Sadly You can even go further in Haskell and introduce more control flow structure via wrapper types. Please bear another contrived example: an HTML form that gets some input data from the user, validates it, saves it to the database, and echoes it back to the user. Without types, you would have to perform these steps manually, and ensure they are kept in the correct order when doing future modifications. With types, you only have to design the types correctly and export only smart contructors (but not the plain ones):

module Foo ( ValidatedValue
           , validateValue
           , RecordId
           , CommittedValue
           , commitValue
           , buildFeedbackForm
           ) where
data ValidatedValue a = ValidatedValue a
validateValue :: a -> Maybe (ValidatedValue a)
data RecordId =  
data CommittedValue a = CommittedValue a RecordId
commitValue :: ValidatedValue a -> ComittedValue a
buildFeedbackForm :: CommittedValue a -> HTMLDoc

From these types, it follows more or less that the only correct workflow is:

1. get a value from the user
2. validate it
3. commit it, getting a transaction/record ID
4. send the response to the user

In other words:

handleFeedbackForm = buildFeedbackForm . commitValue . validateValue

There are still issues here, e.g. the type a is completely hidden behind the wrapper types, and we can't recover some basic properties (even if we use newtype, unless we use GeneralizedNewtypeDeriving). But it does offer a way to improve control flow safety. And that is my 0.02 currency unit for today.

...we stopped the singing. This may be irony:

No More Google Reader
from Google Operating System by Alex Chitu
via What's Hot in Google Reader

but the topic at hand remains serious. This is a stark reminder of why it's important to be able to get your data out of cloud services at a moment's notice and why you should always keep local backups of important data. Thankfully, Google is quite fair about this, as opposed to a lot of other large players. (As a side note, export of full data should be doable, but come with another layer of security like a grace period, password confirmation, verification email, or other means. Imagine someone gains temporary access to your account and exports all your digital life at once. And yes, I know how that relates to IMAP.) Google Reader has felt like a second class citizen for quite some time now, the removal of features to push Google+ and the slow updates for Android were two signs that didn't forebode well (Vim's spelling correction does not know "forbode", how weird is that)(turns out I'm simply stupid, see comments; thanks Jim). I can see how this move may make sense from Google's POV even though I am sad to see an old companion go. Anyway, the stats page of two of my accounts show how sadly I will miss Google Reader in two simple lines:

Since January 24, 2010 you have read a total of 59,253 items.
Since February 20, 2011 you have read a total of 215,285 items.

Farewell, my friend. PS: Maybe those petitions will sway them, but I won't get my hopes up. PS: Speaking of Google, I found this gem today; TMI.

While working on fixing a few encoding problems that I managed to introduce to the BTS almost half a year ago, I took a side bit of coding, and introduced libravatar support to the BTS. Every e-mail now has an avatar to the right which should correspond to the sender. Libravatar is a federated service, which means that if you control your domain, you can serve your own icons. It also automatically falls back to gravatar, so if you're using that service, things should "just work". Hopefully this will be primarily amusing, and people won't abuse it. More importantly, but much less fun, the double encoding problems (where mails would get double-encoded if any of the headers contained non-us-ascii text), and mojibake wontfix icon ( ) should be fixed now. If you see any additional cases of this, please report them to owner@bugs.debian.org.

sage@guug This January, I held an updated version of my "Gitify your life" talk at sage@guug, which is the Munich chapter of the German Unix User Group. Once again, it was quite successful and turnout was about three times that of normal attendance. Afterwards, I was asked if I would be willing to hold the same talk at the German Unix User Group's Spring Meeting in 2014 as the programme for 2013 has been finalized for quite some time, already. Obviously, I said I would be willing to do that and will submit the talk once registration opens. Open Source Days By the way, I am writing this post from a hotel room in Copenhagen. Tomorrow, there will be a world first... "Gitify your life" will not only be held as a talk, it will be expanded into a full three hour workshop (with the option of extending it if there's interest as the room will be free after my workshop). Having never held a workshop of this scope, I am really looking forward to this experience. Besides enabling me to get people up and running with their personal data, this will allow me to see where new users are having issues with vcsh. I keep hearing of people not getting their heads around the concept easily, but no matter how many promises of updates to my docs I get while helping them, no one really follows through with that promise once they know how to use vcsh. Maybe I will find someone willing to follow through and/or simply gather enough material for docs 2.0. Your place? As an aside, and I will probably try to put a real announcement together as well, if you know of a conference, user group or other place where a lightning talk, a full talk, or a workshop about one or all of these topics would fit, please let me know. I will gladly submit proposals to or accept invitations from most places as long as I can spread this message. Especially if the venue happens to be reasonably close to Southern Germany, covers travel cost, or if they can guarantee video recording of the talk. Joey Hess has speculated if the talk isn't really called "git naked" as I seem to keep on giving that particular talk at venues which don't have video recordings. Please help me make Joey happy... ...with a video recording, I mean.

Hi everyone! Today I make an announcement I thought I would never ever make. But things changed. Discussion about this has a long history, starting as a non-serious suggestion at DesktopSummit 2011, continued with people on IRC, but it was decided back then that it wouldn t be worth the effort. This has changed too, and a small team has formed to work on it. We hereby announce Tanglu, a new Debian-based-Linux distribution. Why do we need another one? Let me explain the concepts of that distro: Tanglu will be based on Debian Testing and follow the Debian development closely. It will have a 6-months release-cycle and it s target audience are Linux desktop users. We will make installing and setting up the distro as easy as possible. Tanglu will be usable for both developers of upstream software and the average Linux user and Linux newbie. This is possible because in our opinion developers and users don t have different needs for a desktop system. Both kinds of users like a polished desktop which just works . We will, hwever, not apply any kind of fancy modification on upstream software, we will basically just distribute what upstream created, so users can get an almost pure GNOME and KDE experience. Tanglu is designed to be able to solve the issue that Debian is frozen for a long time and Debian Developers can t make new upstream versions available for testing easily. During a Debian freeze, DDs can upload their software to the current Tanglu development version and later start the new Debian cycle with already tested packages from Tanglu. The delta between Tanglu and Debian should be kept as minimal as possible. However, Tanglu is not meant as experimental distribution for Debian, so please upload experimental stuff to Experimental. Only packages good enough for a release should go into Tanglu. Ideally, Tanglu and Debian should be working well together in mixed environments, where you for example have Debian servers and multiple Tanglu desktops with the new software, targeted at desktop user. Since the differences between Tanglu and Debian should not be very high, administering both systems should be very easy (if you know Debian). Tanglu will be an open project, driven by community. At the beginning of each cycle, people can make suggestions for release goals they want to implement (similar to Fedora, but without FESCo). These proposals are discussed in public and are rejected if there are major technical concerns. If consensus about a certain proposal is lacking, a vote is done for it. The proposal can be accepted with absolute majority. If this does not happen, the proposal is postponed for the next release, where people can vote for it again. If nobody wants that function, it is rejected. In general, decisions made by Debian are superior and have to be followed. We don t think we know every package and every software better than the original upstream. That s why it makes much sense to rely on feedback from others and to have a community-based and peer-reviewed distribution, instead of secretly developing stuff and dumping it on the community. Tanglu will have a highly predictable set of features, defined at the beginning of a cycle, so you will know what you can expect from the next release as soon as possible and plan for it. Tanglu will make it easy to deploy applications for it. It will contain a software-center, similar to what Ubuntu has. We will also try to establish a solution for a Linux-AppCenter , a place for Linux applications, which will be open not only for Tanglu, but can be implemented in any other distribution too. Possible income will flow back into development of the platform. Now, let s answer the FQA (Future Questions Asked): Why don t you contribute to Debian directly and create yet another distribution? First of all, we contribute to Debian And for me, I can say that I will contribute to Debian even more. The point is that Debian can not cover all possible use-cases, and with Tanglu we want to make a distro which solves this. You might ask why we have to create a new distro for that, instead of creating improvements inside Debian? Creating a new distro allows us to do stuff we can never do in Debian. For example, we will include proprietary firmware in that distro, we will make installations of proprietary stuff possible easily (but don t ship with it by default) and we will have a time-based release cycle. These are already things which are a no-go for Debian, and that s fine. We don t want Debian to support these cases, as it is already a great distribution. We want to offer a distro as close to Debian as possible, but with a few modifications for use-cases which are not covered by Debian itself. Of course we will participate in DEX. If Debian Developers contribute to Tanglu, freezes will take even longer! This is an often-heard concern, it comes up on every mailinglist discussion about continuing development while freeze. I would disagree here, packaging new upstream stuff is not slowing down testing and improving of packages in Testing. Also, Tanglu is an offer for Debian developers to participate (we will sync privileges for their packages) we don t expect anyone to work on it, but as we think DDs know their packages best, we will make it possible for them to participate without extra barriers. We hope that Tanglu can add value to Debian and that Debian cycles can start with better-tested packages. You said you are a small team you cannot develop a whole distribution with it! Let s put that to the test! All people working on this are well aware of the issue that the project can not survive without much community-involvement on the long run. But we see a chance that many people are interested in it and that there is a high demand for it. At the beginning, we will just start with a small set of packages. We will also sync many packages from Ubuntu, to reduce workload. For example, it is planned to use the Ubuntu-Kernel and KDE packaging. By doing this, we keep the workload at the beginning low. We also reduce duplicate work with that. We even have some possible sponsors for the new distribution. But nothing is set in stone yet, so just wait for it to happen. Why not participate in Arch, OpenSUSE $other_distro? These are not Debian . I know, it sounds odd, but if you like the Debian way of doing things, you want to use a Debian-based distribution. There is nothing wrong with OpenSUSE. And Debian has issues too. But we want to be close to Debian and use it s tools and way of doing things. I hate you!!! You are doing it wrong!! The project is useless!
Well, that s fine. But there is no reason for hating us. If you dislike our idea, there are basically two options: First, you hate us but the project is successful. In that case, you have been wrong with hate, as there are definitely people who liked the project and contributed to it. Second, you hate us and we fail. In this case, there is no reason for hate, as the project will just vanish and you don t have to worry about it. So hating it would ve been just a big waste of energy. Also keep in mind that forking is a way to keep development healthy and to adapt software to new use-cases which it didn t target before. And we are not introducing incompatibilities here (like e.g. writing our own display server could). Instead, we want to stay close to Debian and reuse as much code as possible. Which desktop will you use? Everyone can add a new desktop to Tanglu, as long as the desktop-environment is present in Debian. Long term, we will have to offer Linux-newbies a default flavour, probably by setting a default download on the website. But as long as there is a community for a given desktop-environment, the desktop is considered as supported. At the beginning, we will focus on KDE, as many people have experience with it. But adding vanilla GNOME is planned too. Can you say something about the software used in Tanglu? Yes, but this is still in flow, so I can t promise final decisions here. On the server, side, we will try to use the Dak for repository management, as soon as we have enough server capacity. We will also use the standard Debian repository GUI and basically reuse most of the software there, to diverge less from Debian. The distribution itself will use a Linux Kernel from Ubuntu and systemd as the primary init system, as well as the login manager logind. It will be based on current Debian Testing with some fresh packages from Unstable and Experimental. We might also use the Ubuntu driver packages and KDE packaging. We expect to have a very rough start with the first release, but there will be enough time to polish Tanglu. Nice idea! How can I help? Well, you can help with basically anything at time from writing manuals, over designing logos and pages to administering a webserver and create packages. We are at an early stage of development at the moment, but we wanted to go public with it as soon as possible, to include the community and receive feedback so we can make that distro community-centric from the beginning. Most of the infrastructure is currently in progress too. So, if you want to get started with Tanglu, subscribe to our mailinglist tanglu-devel and write a mail to it, intruducing yourself. We can then include you in the loop. Generally, if you want to get access to our machines, a trusted GPG-signature will help a lot. If you want to talk to us, join #tanglu-devel or Freenode! Most discussions are currently happening there. And that s it! Tanglu will be awesome!

Some other projects of mine will develop a bit slower because I am now involved in Tanglu. But nothing will stop, and there is some pretty cool stuff coming for both GNOME and KDE (and I still have to implement DEP-11 for Debian).

As most people that are using that funny grey on black thing with the blinking white box AKA a commandline, I do have a heavily configured shell. zsh in my case. Some while ago my old configuration for it started to annoy me and so I set out to redo it. With the basic requirement to be flexible and able to adjust to various zsh versions (back as far as Lenny). I ve set out and looked how others do it and took the (I hope) best bits and put them together in a way I like it. Maybe others want to pick from me, so here I describe a little, maybe someone is interested First - the config is completly inside .zsh. The only thing directly in my $HOME is a symlink from $HOME/.zshenv to $HOME/.zsh/zshenv.home. All in one place, nicer this way. The zshenv just sets ZDOTDIR, so zsh then uses the .zsh itself. Next I am using modules . Well, actually everything is splitted out in small(er) files, so its way easier to maintain. And to overwrite stuff wherever I need to. So my .zshrc just loads those module files - and then goes and checks if it finds the same file inside a series of subdirectories. Those subdirectories are based on the hostname, the kernel name, the username, the domain name and the name of the distribution of the system. And a number of combinations of those. That system allows me, for example, to have a set of aliases defined only on Debian systems (like those to do with apt-get & co). Or define some extra variables if I login to the host franck, domain debian.org. Or whatever more. I m also on the way of doing something similar for all my dotfiles. While I was working at that I finally got fed up with the fact that my prompt, which is based on Phil! s wasn t doing what I wanted it on various of the hosts I access. Thankfully someone pointed me at a change to the grml zsh config: Frank Terbeck rewrote their prompt to make use of the zstyle system. Now that got me going, I like the idea of having the whole prompt easily configurable and changeable. My prompt always has shown slight differences depending on where the shell runs, which meant I had to carefully craft PS1 for every such change, and when one changes something one had to check all the places to adjust them too. No longer. I took Franks work and extended it for me. I wanted to still have a design matching my old prompt (minus colors, I changed color themes anyways), but with full flexibility the zstyle system can offer here. Now I am at a point where

• One zstyle call can change the information or order in my prompt.
• The prompt automagically leaves out information when the terminal width gets under a limit,
• I can have whatever extra information items in the prompt, even if the prompt never defined a default for it. Including running neccessary functions to update those extra items.
• the prompt switches between color and black/white mode with one zstyle change,
• the prompt draws lines differently, depending on the capabilities of the terminal.

As an example, on the ftp-master host my prompt, while otherwise being the same as everywhere else, contains one extra item - displaying the current status of the archive (dinstall). If you want to try it, the setup file is available, but note that I don t give any warranty. :) Works for me . I load it in my Prompts module, while having the prompt setup file in my function path. The 4 commented lines, my testcase, are a nice example of a custom token, though a real life one, the setup for the ftp-master prompt with the one extra item can be found over here. Comments? Bugfixes? Enhancements? Update: Got asked for an example how to use my prompt to fake another. So here is what you need to do to use my prompt setup to imitate the prompt clint, as delivered with zsh:

autoload promptinit && promptinit
zstyle ':prompt:ganneff' vcs_info true
# Alternatively you can set whatever you like it to be.
zstyle ':prompt:ganneff' set_vcs_info_defaults true
zstyle ':prompt:ganneff' colors true
## no right prompt
zstyle ':prompt:ganneff:right:setup' use-rprompt false
## color of brackets depending on variable
zstyle ':prompt:ganneff:*:items:openbracket' pre '$ $ SSH_CLIENT+"$ PR_YELLOW " :-"$ PR_RED " '
zstyle ':prompt:ganneff:*:items:openbracket' post '$ PR_NO_COLOR '
zstyle ':prompt:ganneff:*:items:closebracket' pre '$ $ SSH_CLIENT+"$ PR_YELLOW " :-"$ PR_RED " '
zstyle ':prompt:ganneff:*:items:closebracket' post '$ PR_NO_COLOR '
zstyle ':prompt:ganneff:*:items:openanglebracket' pre '$ $ SSH_CLIENT+"$ PR_YELLOW " :-"$ PR_RED " '
zstyle ':prompt:ganneff:*:items:openanglebracket' post '$ PR_NO_COLOR '
zstyle ':prompt:ganneff:*:items:closeanglebracket' pre '$ $ SSH_CLIENT+"$ PR_YELLOW " :-"$ PR_RED " '
zstyle ':prompt:ganneff:*:items:closeanglebracket' post '$ PR_NO_COLOR '
## extra date format, and its color
zstyle ':prompt:ganneff:*:items:date' token '%D %a %y/%m/%d %R %Z '
zstyle ':prompt:ganneff:*:items:date' pre '$ PR_CYAN '
## pts. %l instead of %y, also color again
zstyle ':prompt:ganneff:*:items:pts' token '%l'
zstyle ':prompt:ganneff:*:items:pts' pre '$ PR_GREEN '
zstyle ':prompt:ganneff:*:items:pts' post '$ PR_NO_COLOR '
## host ends with :
zstyle ':prompt:ganneff:*:items:host' token '%m:'
## and a different style for the rc level
zstyle ':prompt:ganneff:*:items:rc' token '%(?..[%?%1v] )'
zstyle ':prompt:ganneff:*:items:rc' pre ''
## change color for other parts that differ to my default
zstyle ':prompt:ganneff:*:items:user' pre '$ PR_GREEN '
zstyle ':prompt:ganneff:*:items:host' pre '$ PR_GREEN '
zstyle ':prompt:ganneff:*:items:at' pre '$ PR_GREEN '
zstyle ':prompt:ganneff:*:items:path' pre '$ PR_GREEN '
zstyle ':prompt:ganneff:*:items:history' pre ''
## Show the shell level in a different way
zstyle ':prompt:ganneff:*:items:shell-level' pre ''
zstyle ':prompt:ganneff:*:items:shell-level' token 'zsh%(2L./$SHLVL.) '
## And history is just a bold number. Lazy here, not using pre and post
zstyle ':prompt:ganneff:*:items:history' token '%B%h%b '
## And the last part in prompt, a # or % depending on privileges
## Using standard zsh tokens for bold
zstyle ':prompt:ganneff:*:items:privileges' pre '%B'
zstyle ':prompt:ganneff:*:items:privileges' post '%b'
## Now there are two parts not defined by default, so lets create them
## Both are simple existing variables, no need for extra precmd functions
## system info
zstyle ':prompt:ganneff:extra:ostype' pre '$ PR_CYAN '
zstyle ':prompt:ganneff:extra:ostype' post '$ PR_NO_COLOR '
zstyle ':prompt:ganneff:extra:ostype' token "$ MACHTYPE /$ OSTYPE /$(uname -r)"
## zsh version
zstyle ':prompt:ganneff:extra:zshvers' pre '$ PR_CYAN '
zstyle ':prompt:ganneff:extra:zshvers' post '$ PR_NO_COLOR '
zstyle ':prompt:ganneff:extra:zshvers' token "$ ZSH_VERSION "
zstyle ':prompt:ganneff:left:full:setup' items \
    openbracket date closebracket openbracket pts closebracket openbracket ostype closebracket \
    openbracket zshvers closebracket newline openanglebracket user at host \
    path closeanglebracket newline \
    shell-level space history rc space vcs privileges space
prompt ganneff

And if you now tell me Uh, thats so much more than just saying /prompt clint/ , then you are right. But then: This way is flexible. As I wrote above, my prompt on the ftpmaster host is different. Actually, my prompt on all machines inside the domain debian.org differs, by having

zstyle ':prompt:ganneff:*:items:host' pre '$ PR_YELLOW '

in the Prompts definition for domain:debian.org. And thats enough to show me im on a .debian.org machine by a yellow hostname, instead of my default red. On the ftpmaster host I have the additional few lines: # Want one more piece in my prompt here, dinstall status zstyle :prompt:ganneff:left:full:setup items \ ulcorner line openparentheses user at host pts closeparentheses line history \ line dinstall line shell-level line flexline openparentheses path closeparentheses line urcorner newline \ llcorner line rc openparentheses time closeparentheses line vcs line change-root pipe space

zstyle ':prompt:ganneff:extra:dinstall' pre '$ PR_CYAN '
zstyle ':prompt:ganneff:extra:dinstall' post '$ PR_NO_COLOR '
zstyle ':prompt:ganneff:extra:dinstall' token '$DINSTALL'
zstyle ':prompt:ganneff:extra:dinstall' precmd jj_update_dinstall
zmodload zsh/mapfile
jj_update_dinstall ()  
    DINSTALL="$ $ (z)$ (f)mapfile[/srv/ftp.debian.org/web/dinstall.status] [2] [3,99] "
 

And woo, thats simple. I think. (There ought to be a way to just easily add the item at a defined place in the items zstyle, but meh, too lazy to look).

If you re a Google Reader user, you probably read in one of your subscriptions that Google is pulling the plug on Google Reader. It is yet another demonstration of why putting data in the cloud isn t so much of a nice idea: the service you rely on may well disappear some day, with all the data it contains. Sure Google, in its extreme goodness, allows you to take out the Google Reader data. Or does it?
These are what you ll get from Google Takeout for Reader:

• followers.json, following.json: both files contain similar data, that I suspect correspond to Buzz subscriptions (yet another dead service). Each friends item contains some information about your friend , and a stream identifier for their activity (I guess), as well as a few websites urls. For instance Tim Bray s stream is user/05198174665841271019/state/com.google/broadcast . What the hell do I do with that? Fortunately, he has websites, but not all my friends have. Thankfully, I haven t really been using this feature, so there s almost nothing in these files.
• liked.json, starred.json, shared.json, shared-by-followers.json: all have the same structure, and contain items you liked, starred, shared, or that the people you follow shared (yeah, that file is badly named). Each item contains an url (or so I hope), and the corresponding content (yay). shared-by-followers.json however doesn t contain more than the items the people you follow actively shared: it doesn t contain their feeds (and I m pretty sure I read more from Tim Bray than the two links he shared)
• subscriptions.xml: Essentially, a list of RSS feed urls, with no content ; nothing from Tim Bray here, but now that I think about it, I think I was only following his Buzz feed, so that went away with Buzz without me noticing.

Interestingly, while looking into shared-by-followers.json, I found urls that would correspond to friend streams. For instance, Tim Bray s is http://www.google.com/reader/public/atom/user/05198174665841271019/state/com.google/broadcast. But it s useless: all it displays is permission denied . As for subscriptions, one of the strengths of Google Reader is that it allowed to search though past items, which means a big part of the interesting data is the archived items. But that s not part of the take out . Sure, you have the feed urls, but most RSS feeds contain a limited amount of items, not the entire history of items for the given feed. So, history is more or less lost. Except if I star, like or share all items in all my subscriptions and take out again. So much goodness. It could have been worse, though.

Review: Higher Order Perl, by Mark Jason Dominus

Publisher:	Morgan Kaufmann
Copyright:	2005
ISBN:	1-55860-701-3
Format:	Trade paperback
Pages:	563

Disclaimer: The author and I both used to be (technically still are, but the newsgroup is dead) moderators of comp.lang.perl.moderated, and I knew him electronically from answering questions on the Perl Usenet newsgroups many years ago. Perl is, in general construction, a typical imperative language in the style of C, with object-oriented capabilities bolted (somewhat awkwardly) on. Most Perl code looks remarkably like C code with more powerful built-in data types and no memory management. A minority of Perl programs embrace object-oriented techniques and remind one of C++ or Java (or Python). But Perl also has powerful capabilities borrowed from functional programming languages and the Lisp tradition, even though many Perl programmers rarely use them and aren't familiar with them. Higher Order Perl focuses on those capabilities and how to use them effectively, starting with callbacks and uses of function pointers and moving into recursion and analysis of recursive functions, iterators, currying, and parsers. It concludes with a fully-worked example of constructing a declarative programming system using the techniques developed earlier in the book. Higher Order Perl is a programming book for an intermediate to advanced Perl programmer, already a rare topic. This is not a look at how to apply Perl to another area of programming, or a cookbook of techniques. It's an attempt to help a Perl programmer think about and use the language differently, and it follows through on that. It's refreshing and rare to read a programming technique book that's targeted at the practicing expert. (I think the size of the audience is often too small for publishers to target it.) Even the most experienced Perl programmer is probably going to learn something fundamental from this book, not just interesting trivia around the edges. On the negative side, though, I found the book a bit too focused on computer science and mathematics problems, particularly in the choice of examples and sample scripts. (And I say this as someone who has a master's degree in computer science with a software theory focus.) This is a bit hard to avoid for topics like recursion, where problems like computing Fibonacci numbers are classic, but throughout the book I struggled to focus past feelings of "but why would I ever do that"? The extended discussion of the Newton-Raphson method is the most memorable; I'm not sure that's a problem many Perl programmers would have and need a higher-order technique for. There's also a lot of discussion of recursion analysis and transformations between recursive and iterative expressions of problems, which is ground I remember well from my degree but which I've rarely had any practical use for in day-to-day programming. This is not a uniform problem, though, just a tendency. There are some great examples that I think are more in the mainstream of Perl problems, including a reinvention of File::Find that shows how to add more flexibility, a web spider, and a great discussion on how to construct a conventional queryable database on top of a flat file (a topic that's near and dear to my heart). And then there's the chapter on infinite streams. Dominus presents a method of using closures to create a version of an interator: a sort of infinite linked list that can keep generating additional elements. He presents this in several contexts, but one of them is log parsing, and that turned out to be exactly the solution that I needed for a problem I was working on while reading this book. I've written about this elsewhere, but this was a wonderful idea that helped me think about both Perl and a major application area in a completely new way, and I wrote an application using that knowledge that would have taken me much longer using different techniques and would have been much less fun. So, for me, this chapter was more than worth the entire book, and blankets the rest of it in a delighted feeling. Other people may or may not have that experience. I think it will depend on whether one of these techniques hits home for you the way that one did for me. This is a book with some idiosyncracies, and some sections that may drag. Having lots of fully-worked examples is a major plus, but some of those are so comprehensive that one can get a bit lost in the details. That particularly hit me with the last couple of chapters on parsing and on the example declarative programming application. Quite a bit of that text involved reinventing a recursive descent parser (another very computer science example), when I'm not quite sure why one wouldn't use one of the existing parser generators on CPAN for practical purposes, and inventing a lot of new syntax to try to make parser Perl code more readable. But one certainly can't complain that Dominus omits necessary details, and there is some appeal in watching an experienced programmer work through a problem from analysis to implementation. But, despite the idiosyncracies, I recommend this book to any experienced Perl programmer who wants to expand their view of the language. The techniques here closures, higher-order functions, iterators and streams, and formal parsers are powerful and underused in the Perl community. You may want to pick and choose which sections you pay close attention to, but I think everyone will find something of interest here. In the weeks since I read this book, my opinion of it has only grown. And I can't tell you just how much I loved the infinite stream concept. Rating: 8 out of 10

Last week I had another birthday, which was nice. I'm now all mature, and everything. Honest. I received a few surprise gifts from friends and strangers alike, which was pretty good. Other than that I didn't do too much. This weekend I'm going to be using "airbnb" to spend the weekend in Dundee with my partner who is regularly commuting between Edinburgh and Perth/Dundee, to work in various hospitals. With all the commuting time she's not had too much time to explore the actual city, and I've only been there once before so I'm sure it will be a fun weekend. The templer static site generator got a little bit of pimping on LWN.net the other day, thanks to Martin Michlmayr, although embarassingly I seem to have read the article and repeated the content in the conclusion, and duplicated that in my own comment. Ooops. Beyond that I've done little coding recently, although I suspect now that nodejs has had a stable release I might do something interesting soon. I don't want to dwell on the failure of Sim City - because I don't run windows and couldn't have tried it even if I wanted to - but I'm pondering the idea of a persistant grid-space where different items can be placed. I've not tried anything browser-based before, but the popularity of things like minecraft make me wonder if you had an "infinite grid" where folk could store "stuff", and scroll around in a browser you might be able to do interesting things. Starting small, with a 100x100 grid, and some kind of updated play-by-mail turfwars/drug-war like experience should be simple. But then again enthusiasm is easy to generate until you start working out how you'd interface with the server and what kind of client you'd need. Now to enjoy some 21 year old whisky and call it a night..

Today, during Blackhat Europe, Offensive Security announced the availability of Kali Linux 1.0, which aims to be the most advanced, robust, and stable penetration testing distribution to date. It is the successor of Backtrack Linux. Kali s choice of Debian Kali s release is a significant event in the security auditing and penetration testing field, and I m proud to see that Debian was retained as the best distribution to create this new product. Here s what Mati Aharoni of Offensive Security told me:

Debian provides a reliable base to build a new distribution and yet can easily be customized to add bleeding edge features, thanks to the unstable and experimental distributions.

Kali s development policies Even though Kali was prepared in secret, from now on Kali s development happens in the open in public git repositories. There are repositories for all the packages that have been created (or forked) as well as for the ISO images creation script. Debian packages are maintained with git-buildpackage, pristine-tar and the associated helper tools, making it easy to integrate the latest changes of Debian. Kali packaged several hundreds tools that relate to their field and they intend to contribute those which are DFSG-free back to Debian. Kali s technical infrastructure In the last year, I have been working within the Kali team to setup large parts of their infrastructure as a proper Debian derivative. Kali s main ISO images are built with live-build. All the bugfixes that I contributed to Debian Live were the direct result of my work for Kali. The git repositories are managed with gitolite. The package repositories are built with reprepro. The build daemons use rebuildd and sbuild. The (push) mirrors are synchronized with the same tools than Debian (based on rsync), but there s also a central server which redirects to a mirror close to you (and which is used by default everywhere). This one runs mirrorbrain (and not Rapha l Geissert s redirector). The ARM build daemons (armel/armhf) run on machines powered by Calxeda s Highbank (4 cores, 4 GB RAM) that work pretty well. Even better, Offensive Security is willing to dedicate one node of this cluster for Debian s own usage. The future This first release is not an end. It s only the start of a journey. Not all applications have been packaged yet and there s lot of work left to integrate everything in Debian. I m really looking forward to continue my collaboration with the Kali team as this has been one of the most interesting project I ever had as a Debian consultant. And also one of the few where I could really contribute something back to Debian.

9 comments Liked this article? Click here. My blog is Flattr-enabled.

I've just released, to code.liw.fi, version 1.20130313 of cliapp, my Python framework for Unix-like command line programs. It contains the typical stuff such programs need to do, such as parsing the command line for options, and iterating over input files. Version 1.20130313

• Add cliapp.Application.compute_setting_values method. This allows the application to have settings with values that are computed after configuration files and the command line are parsed.
• Cliapp now logs the Python version at startup, to aid debugging.
• cliapp.runcmd now logs much less during execution of a command. The verbose logging was useful while developing pipeline support, but has now not been useful for months.
• More default settings and options have an option group now, making --help output prettier.
• The --help output and the output of the help subcommand now only list summaries for subcommands. The full documentation for a subcommand can be seen by giving the name of the subcommand to help.
• Logging setup is now more overrideable. The setup_logging method calls setup_logging_handler_for_syslog, setup_logging_handler_for_syslog, or setup_logging_handler_to_file, and the last one calls setup_logging_format and setup_logging_timestamp to create the format strings for messages and timestamps. This allows applications to add, for example, more detailed timestamps easily.
• The process and system CPU times, and those of the child processes, and the process wall clock duration, are now logged when the memory profiling information is logged.
• Subcommands added with add_subcommand may now have aliases. Subcommands defined using Application class methods named cmd_* cannot have aliases.
• Settings and subcommands may now be hidden from --help and help output. New option --help-all and new subcommand help-all show everything.
• cliapp(5) now explains how --generate-manpage is used. Thanks to Enrico Zini for the suggestion.
• New function cliapp.ssh_runcmd for executing a command remotely over ssh. The function automatically shell-quotes the argv array given to it so that arguments with spaces and other shell meta-characters work over ssh.
• New function cliapp.shell_quote quotes strings for passing as shell arguments.
• cliapp.runcmd now has a new keyword argument: log_error. If set to false, errors are not logged. Defaults to true.

Bug fixes:

• The process title is now set only if /proc/self/comm exists. Previously, on the kernel in Debian squeeze (2.6.32), setting the process title would fail, and the error would be logged to the terminal. Reported by William Boughton.
• A setting may no longer have a default value of None.

I've just got back from Linaro Connect Asia 2013 in Hong Kong. This was my first time back in Hong Kong since 1995 when I left after living there for 10 years. It was really interesting to see how much the place had changed, as well as how much it hadn't! I started the week by giving a talk together with Stefano Stabellini Introducing Xen on ARM to a full house. The slides can be found on the event page or google docs. There is a video on youtube but due to the cameras being setup for a fish-bowl style panel it is unfortunately mostly of my backside. The reception was excellent and the talk seeded a large number of hallway track conversations with all the many folks who are interested in Xen on the ARM platform. The day after our talk Lars Kurth (Xen.org community manager) and Mark Heath (VP of XenServer Engineering, and my boss) gave a keynote (video) announced Citrix's intention to join the Linaro Enterprise Group (AKA "LEG") on the 1st of April. This is pretty exciting for me since it means I'll get to work much more closely with the Linaro and ARM communities and spend even more of my time hacking on Xen on ARM! At some point Stefano and myself were also video interviewed by Charbax of http://armdevices.net/ but it doesn't appear to have gone online yet. Lars and Mark also gave interviews which are online already. One nice thing about Connect is that the afternoons are left free of talks so I managed to find time in the hackrooms to implement SMP host support and 64-bit domain 0 support for Xen on ARM as well as getting multiarch cross building working for upstream Xen with Wookey's kind assistance (no patches for this yet since I still need to clean them up). It was also useful to catch up with the ARM kernel developers and KVM guys to see what they are up to and share ideas etc. Finally right at the end Stefano and myself showed Xen on ARM at Demo Friday. We had Xen running on the Arndale Board, ARM Chromebook and 64-bit ARMv8 using the emulators (no hardware being available yet). All in all I had an excellent (if tiring) week, there was loads of interest in Xen on ARM and in Citrix joining LEG and everyone was very welcoming. I'm really looking forward to the next Connect in Dublin. Outside of Connect I managed to find time to visit Sham Shui Po at look at all the gadgets but managed to resist buying anything (Stefano got a Thinkpad X230, which was enough retail therapy for both of us!). I caught up with my sister (who lives there again), her boyfriend and my parents (who happened to be visiting her that week). I also managed have some drinks with some old school friends, most of whom I've not seen for nearly twenty years, but it was just like old times ;-). I'm off again to Hong Kong next week with my family to visit my sister (yes, there was some bad planning involved here...). This time I plan to do all the toursity type stuff that I never did while I lived there. I've also got tickets for the Hong Kong Rubgy Sevens and hopefully I'll manage catch up with some more old friends. I probably won't be able to resist buying gadgets this time around though.

WebAuth is the site-wide web authentication system that we use at Stanford. After a lot of time focusing on things at work other than coding, it's been my primary job for the past nine months or so, which has been a lovely change. I wasn't planning on doing another 4.4 release and instead focusing on 4.5.0 (which is well underway and adds significant new features to multifactor support), but I kept finding bugs, including several that were quite embarassing. So this is another bug-fix release, and hopefully the last one before 4.5.0. WebAuthTrustAuthzIdentity enabled WebAuthDoLogout in the same scope (ever since it was added in 4.4.0). It's horribly embarassing to have been writing C for more than 20 years and still manage to omit a break in a switch statement. Benjamin Coddington found another bug in authorization identity handling where if the user changed their authorization identity to match their authentication identity, their authentication would be rejected. We now detect that case and just discard the authorization identity if it's the same as the authentication identity. There are multiple fixes in the mod_webauth logging of bad app tokens: expired app tokens are now logged with a proper message (and at info level rather than error), empty app tokens (created internally and therefore seen by subqueries before fixup to delete expired app tokens) are now just ignored, and invalid app tokens don't result in spewing binary strings into the Apache error log. mod_webauthldap has, since the beginning, refused to add more than 127 values of a multivalued LDAP attribute to the environment. We ran into that limit with entitlements, so it's now been removed. This runs the risk of overflowing the environnment, but I did some calculations, and it looks like you're going to have to try really hard and have impressively large multivalued attributes to create that problem. If it actually becomes a problem for someone, I'll add a configuration option. There's also a syntax fix to the default WebLogin error template and a variety of minor bug fixes, mostly around error handling conditions, to correct problems caught by clang --analyze. (The current master branch gets a completely clean bill of health from clang --analyze using clang 3.0.) You can get the latest release from the official WebAuth distribution site or from my WebAuth distribution pages.

On Saturday I had the opportunity to teach another one-day master class on Rcpp. The class had been organized by Jared Lander, and organized very well I might add. The weekend started with a slight disappointment. I had taken Friday off, and hoped to reach NY by early afternoon to join JJ there, and to spend the afternoon with the RStan team. However, the tail end of last week's snowstorm made it such that we both got to Columbia's stats department closer to 6pm rather than 1pm, and half the team had left. Dang. Very frustrating travel experience. We salvaged the evening by gabbing over a cold beverage or two, before sharing some sacred New York pizza with Wes McKinney and Jared. The class itself on Saturday went quite well. With JJ on deck, we were able to have every participant log into an EC2-hosted instance of RStudio Server, which worked very well for usage examples of Rcpp. It has been almost a year since I last taught the class, and many exciting things--such as Rcpp attributes, added by JJ himself--have appeared, which made it extra fun. Participants were rather kind with praise. Either they really liked it, or they really are hard-nosed New Yorkers who manage to lie to my face without me noticing. We ended the day with some hard-earned cold beverages, followed by some dinner at Sylvia's (as tweeted by Jared) followed by more drinks. Ended up a little past my usual bedtime, but I managed to get out and enjoy a lovely 6.5 miles run across Central Park the next morning before leaving town. All in all, a very nice weekend, the travel horror of Friday notwithstanding. And who know, maybe we'll just do it again another time...

Conrad released a first bug-fix release 3.800.1 of Armadillo earlier today. This has been wrapped up in release 0.3.800.1 of RcppArmadillo as usual. This release also contains a very nice function sample() (contributed by Christian Gunning) which provides sampling (with or without replacement) at the C++ level modeled after what we are used to in R itself. We also refactored the unit tests into just two compilation units to speed testing up a little. The summary of the main changes follows:

Changes in RcppArmadillo version 0.3.800.1 (2013-03-12)

• Upgraded to Armadillo release Version 3.800.1 (Miami Beach)
  • workaround for a bug in ATLAS 3.8 on 64 bit systems
  • faster matrix-vector multiply for small matrices
• Added new sample() function and tests contributed by Christian Gunning
• Refactored unit testing code for faster unit test performance

Courtesy of CRANberries, there is also a diffstat report for the most recent release As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If anyone wants to check out some new Hy stuff, check out the REPL 3.0 at hy.pault.ag Also, check out the GitHub Repo, and star it, or consider contributing to it. Hang tight, this is technical Hopefully you re a rockn Pythonista already. If not, hang on! Right, so, here s the Hython we re working with:

;;;; testing.hy
(import-from sunlight openstates)
(defn get-legislators [state]
  "Get some Legislators from a state"
  (kwapply (.legislators openstates)  "state" state ))
(defn print-legislator-count [state]
  "Print the Legislative count for a state"
  (print (len (get-legislators state))))

And a Python script to run it:

#!/usr/bin/env python
import hy
import testing
# import pdb; pdb.set_trace()
testing.print_legislator_count("ma")

Which outputs (correctly): 198. Now, let s try and debug this sucker: I added import pdb; pdb.set_trace() to the top of the Pythonic script (after the imports), to drop into a pdb shell:

> /home/tag/tmp/invoke.py(8)()
-> testing.print_legislator_count("ma")
(Pdb) 

Here s where we are:

(Pdb) l
  3     import hy
  4     import testing
  5     
  6     import pdb; pdb.set_trace()
  7     
  8  ->  testing.print_legislator_count("ma")
[EOF]

OK. Let s step into it.

(Pdb) s
--Call--
> /home/tag/tmp/testing.hy(12)print_legislator_count()
-> (defn print-legislator-count [state]

Whoh! Right! We ve just invoked Lisp. Let s step through the Lisp code:

> /home/tag/tmp/testing.hy(14)print_legislator_count()
-> (print (len (get-legislators state))))
(Pdb) 
--Call--
> /home/tag/tmp/testing.hy(7)get_legislators()
-> (defn get-legislators [state]
(Pdb) 
> /home/tag/tmp/testing.hy(9)get_legislators()->[ (SNIP) ]
-> (kwapply (.legislators openstates)  "state" state ))
(Pdb) l
  4     (import-from sunlight openstates)
  5     
  6     
  7     (defn get-legislators [state]
  8       "Get some Legislators from a state"
  9  ->    (kwapply (.legislators openstates)  "state" state ))
 10     
 11     
 12     (defn print-legislator-count [state]
 13       "Print the Legislative count for a state"
 14       (print (len (get-legislators state))))
(Pdb) 

Rockn . This worked hilariously well.

-> (kwapply (.legislators openstates)  "state" state ))
(Pdb) state
'ma'

Whoh, right there. Look at that! Python s pdb has access to Hy s bits! Neat :)

(Pdb) openstates
(Pdb) s

And. just to prove it s actually Python :)

(Pdb) s
198
--Return--
> /home/tag/tmp/testing.hy(14)print_legislator_count()->None
-> (print (len (get-legislators state))))
(Pdb) s
--Return--
> /home/tag/tmp/invoke.py(8)()->None
-> testing.print_legislator_count("ma")

Whoo! From Python to Lisp and back! Now, for fun, here s some pudb action: And, now, some bpython voodoo: Hopefully this is the start of something wicked neat! :)