Solarized A while back I was looking for some information on the web, and happened upon a blog post about the subject. I don't remember what I was looking for, but on the same blog, there was a screen shot of what I then learned was the Solarized theme. This caught my eye that I decided to try it myself ASAP. Up until last year, I've been using for many years the 'black on light yellow' xterm scheme. This is good during the day, but too strong during night, so on some machines I switched to 'white on black', but this was not entirely satisfying. The solarized theme promises to have consistent colours over both light and dark background, which would help to make my setups finally consistent, and extends to a number of programs. Amongst these, there are themes for mutt on both light and dark backgrounds using only 16 colours. This was good, as my current hand-built theme is based on 256 colours, and this doesn't work well in the Linux console. So I tried changing my terminal to the custom colours, played with it for about 10 minutes, then decided that its contrast is too low, bordering on unreadable. I switch to another desktop where I still had open an xterm using white-on-black, and this being at night my eyes immediately go 'no no no too high contrast'. In about ten minutes I got so used to it that the old theme was really really uncomfortable. There was no turning back now Interestingly, the light theme was not that much better than black-on-light-yellow, as that theme is already pretty well behaved. But I still migrated for consistency. Programs/configs Starting from the home page and the internet, I found resources for:

• Vim and Emacs (for which I use the debian package elpa-solarized-theme).
• Midnight Commander, for which I currently use peel's theme, although I'm not happy with it; interestingly, the default theme almost works on 16-custom-colours light terminal scheme, but not quite on the dark one.
• Mutt, which is both in the main combined repository but also on the separate one. I'm not really happy with mutt's theme either, but that seems mostly because I was using a quite different theme before. I'll try to improve what I feel is missing over time.
• dircolors; I found this to be an absolute requirement for good readability of ls --color, as the defaults are too bad
• I also took the opportunity to unify my git diff and colordiff theme, but this was not really something that I found and took 'as-is' from some repository; I basically built my own theme.

16 vs 256 colours The solarized theme/configuration can be done in two ways:

• by changing the Xresources/terminal 16 basic colours to custom RGB values, or:
• by using approximations from the fixed 256 colours available in the xterm-256color terminfo

Upstream recommends the custom ones, as they are precisely tuned, instead of using the approximated ones; honestly I don't know if they would make a difference. It's too bad upstream went silent a few years back, as technically it's possible to override also colours above 16 in the 256-colour palette, but in any case, each of the two options has its own cons:

• using customised 16-colour means that all terminal programs get the new colours scheme, even if they were designed (colour-wise) based on the standard values; this makes some things pretty unreadable (hence the need to fix dircolors), but at least somewhat consistent.
• using 256-colour palette, unchanged programs stay the same, but now they look very different than the programs that were updated to solarized; note thought I haven't tested this, but that's how I understand things would be.

So either way it's not perfect. Desktop-wide consistency Also not perfect is that for proper consistent look, many more programs would have to be changed; but I don't see that happening in today's world. I've seen for example 3 or 4 Midnight Commander themes, but none of them were actually in the spirit of solarized, even though they were tweaked for solarized. Even between vim and emacs, which both have one canonical solarized theme, the look is close but not really the same (looking at the markdown source for this blog post: URLs, headers and spelling mistakes are all different), but this might be due not necessarily the theme itself. So no global theme consistency (I'd wish), but still, I find this much better on the eyes and not lower on readability after getting used to it. Thanks Ethan!

Passwords are used everywhere in our modern life. Between your email account and your bank card, a lot of critical security infrastructure relies on "something you know", a password. Yet there is little standard documentation on how to generate good passwords. There are some interesting possibilities for doing so; this article will look at what makes a good password and some tools that can be used to generate them. There is growing concern that our dependence on passwords poses a fundamental security flaw. For example, passwords rely on humans, who can be coerced to reveal secret information. Furthermore, passwords are "replayable": if your password is revealed or stolen, anyone can impersonate you to get access to your most critical assets. Therefore, major organizations are trying to move away from single password authentication. Google, for example, is enforcing two factor authentication for its employees and is considering abandoning passwords on phones as well, although we have yet to see that controversial change implemented. Yet passwords are still here and are likely to stick around for a long time until we figure out a better alternative. Note that in this article I use the word "password" instead of "PIN" or "passphrase", which all roughly mean the same thing: a small piece of text that users provide to prove their identity.

What makes a good password? A "good password" may mean different things to different people. I will assert that a good password has the following properties:

• high entropy: hard to guess for machines
• transferable: easy to communicate for humans or transfer across various protocols for computers
• memorable: easy to remember for humans

High entropy means that the password should be unpredictable to an attacker, for all practical purposes. It is tempting (and not uncommon) to choose a password based on something else that you know, but unfortunately those choices are likely to be guessable, no matter how "secret" you believe it is. Yes, with enough effort, an attacker can figure out your birthday, the name of your first lover, your mother's maiden name, where you were last summer, or other secrets people think they have. The only solution here is to use a password randomly generated with enough randomness or "entropy" that brute-forcing the password will be practically infeasible. Considering that a modern off-the-shelf graphics card can guess millions of passwords per second using freely available software like hashcat, the typical requirement of "8 characters" is not considered enough anymore. With proper hardware, a powerful rig can crack such passwords offline within about a day. Even though a recent US National Institute of Standards and Technology (NIST) draft still recommends a minimum of eight characters, we now more often hear recommendations of twelve characters or fourteen characters. A password should also be easily "transferable". Some characters, like & or !, have special meaning on the web or the shell and can wreak havoc when transferred. Certain software also has policies of refusing (or requiring!) some special characters exactly for that reason. Weird characters also make it harder for humans to communicate passwords across voice channels or different cultural backgrounds. In a more extreme example, the popular Signal software even resorted to using only digits to transfer key fingerprints. They outlined that numbers are "easy to localize" (as opposed to words, which are language-specific) and "visually distinct". But the critical piece is the "memorable" part: it is trivial to generate a random string of characters, but those passwords are hard for humans to remember. As xkcd noted, "through 20 years of effort, we've successfully trained everyone to use passwords that are hard for human to remember but easy for computers to guess". It explains how a series of words is a better password than a single word with some characters replaced. Obviously, you should not need to remember all passwords. Indeed, you may store some in password managers (which we'll look at in another article) or write them down in your wallet. In those cases, what you need is not a password, but something I would rather call a "token", or, as Debian Developer Daniel Kahn Gillmor (dkg) said in a private email, a "high entropy, compact, and transferable string". Certain APIs are specifically crafted to use tokens. OAuth, for example, generates "access tokens" that are random strings that give access to services. But in our discussion, we'll use the term "token" in a broader sense. Notice how we removed the "memorable" property and added the "compact" one: we want to efficiently convert the most entropy into the shortest password possible, to work around possibly limiting password policies. For example, some bank cards only allow 5-digit security PINs and most web sites have an upper limit in the password length. The "compact" property applies less to "passwords" than tokens, because I assume that you will only use a password in select places: your password manager, SSH and OpenPGP keys, your computer login, and encryption keys. Everything else should be in a password manager. Those tools are generally under your control and should allow large enough passwords that the compact property is not particularly important.

Generating secure passwords We'll look now at how to generate a strong, transferable, and memorable password. These are most likely the passwords you will deal with most of the time, as security tokens used in other settings should actually never show up on screen: they should be copy-pasted or automatically typed in forms. The password generators described here are all operated from the command line. Password managers often have embedded password generators, but usually don't provide an easy way to generate a password for the vault itself. The previously mentioned xkcd cartoon is probably a common cultural reference in the security crowd and I often use it to explain how to choose a good passphrase. It turns out that someone actually implemented xkcd author Randall Munroe's suggestion into a program called xkcdpass:

    $ xkcdpass
    estop mixing edelweiss conduct rejoin flexitime

In verbose mode, it will show the actual entropy of the generated passphrase:

    $ xkcdpass -V
    The supplied word list is located at /usr/lib/python3/dist-packages/xkcdpass/static/default.txt.
    Your word list contains 38271 words, or 2^15.22 words.
    A 6 word password from this list will have roughly 91 (15.22 * 6) bits of entropy,
    assuming truly random word selection.
    estop mixing edelweiss conduct rejoin flexitime

Note that the above password has 91 bits of entropy, which is about what a fifteen-character password would have, if chosen at random from uppercase, lowercase, digits, and ten symbols:

    log2((26 + 26 + 10 + 10)^15) = approx. 92.548875

It's also interesting to note that this is closer to the entropy of a fifteen-letter base64 encoded password: since each character is six bits, you end up with 90 bits of entropy. xkcdpass is scriptable and easy to use. You can also customize the word list, separators, and so on with different command-line options. By default, xkcdpass uses the 2 of 12 word list from 12 dicts, which is not specifically geared toward password generation but has been curated for "common words" and words of different sizes. Another option is the diceware system. Diceware works by having a word list in which you look up words based on dice rolls. For example, rolling the five dice "1 4 2 1 4" would give the word "bilge". By rolling those dice five times, you generate a five word password that is both memorable and random. Since paper and dice do not seem to be popular anymore, someone wrote that as an actual program, aptly called diceware. It works in a similar fashion, except that passwords are not space separated by default:

    $ diceware
    AbateStripDummy16thThanBrock

Diceware can obviously change the output to look similar to xkcdpass, but can also accept actual dice rolls for those who do not trust their computer's entropy source:

    $ diceware -d ' ' -r realdice -w en_orig
    Please roll 5 dice (or a single dice 5 times).
    What number shows dice number 1? 4
    What number shows dice number 2? 2
    What number shows dice number 3? 6
    [...]
    Aspire O's Ester Court Born Pk

The diceware software ships with a few word lists, and the default list has been deliberately created for generating passwords. It is derived from the standard diceware list with additions from the SecureDrop project. Diceware ships with the EFF word list that has words chosen for better recognition, but it is not enabled by default, even though diceware recommends using it when generating passwords with dice. That is because the EFF list was added later on. The project is currently considering making the EFF list be the default. One disadvantage of diceware is that it doesn't actually show how much entropy the generated password has those interested need to compute it for themselves. The actual number depends on the word list: the default word list has 13 bits of entropy per word (since it is exactly 8192 words long), which means the default 6 word passwords have 78 bits of entropy:

    log2(8192) * 6 = 78

Both of these programs are rather new, having, for example, entered Debian only after the last stable release, so they may not be directly available for your distribution. The manual diceware method, of course, only needs a set of dice and a word list, so that is much more portable, and both the diceware and xkcdpass programs can be installed through pip. However, if this is all too complicated, you can take a look at Openwall's passwdqc, which is older and more widely available. It generates more memorable passphrases while at the same time allowing for better control over the level of entropy:

    $ pwqgen
    vest5Lyric8wake
    $ pwqgen random=78
    Theme9accord=milan8ninety9few

For some reason, passwdqc restricts the entropy of passwords between the bounds of 24 and 85 bits. That tool is also much less customizable than the other two: what you see here is pretty much what you get. The 4096-word list is also hardcoded in the C source code; it comes from a Usenet sci.crypt posting from 1997. A key feature of xkcdpass and diceware is that you can craft your own word list, which can make dictionary-based attacks harder. Indeed, with such word-based password generators, the only viable way to crack those passwords is to use dictionary attacks, because the password is so long that character-based exhaustive searches are not workable, since they would take centuries to complete. Changing from the default dictionary therefore brings some advantage against attackers. This may be yet another "security through obscurity" procedure, however: a naive approach may be to use a dictionary localized to your native language (for example, in my case, French), but that would deter only an attacker that doesn't do basic research about you, so that advantage is quickly lost to determined attackers. One should also note that the entropy of the password doesn't depend on which word list is chosen, only its length. Furthermore, a larger dictionary only expands the search space logarithmically; in other words, doubling the word-list length only adds a single bit of entropy. It is actually much better to add a word to your password than words to the word list that generates it.

Generating security tokens As mentioned before, most password managers feature a way to generate strong security tokens, with different policies (symbols or not, length, etc). In general, you should use your password manager's password-generation functionality to generate tokens for sites you visit. But how are those functionalities implemented and what can you do if your password manager (for example, Firefox's master password feature) does not actually generate passwords for you? pass, the standard UNIX password manager, delegates this task to the widely known pwgen program. It turns out that pwgen has a pretty bad track record for security issues, especially in the default "phoneme" mode, which generates non-uniformly distributed passwords. While pass uses the more "secure" -s mode, I figured it was worth removing that option to discourage the use of pwgen in the default mode. I made a trivial patch to pass so that it generates passwords correctly on its own. The gory details are in this email. It turns out that there are lots of ways to skin this particular cat. I was suggesting the following pipeline to generate the password:

    head -c $entropy /dev/random   base64   tr -d '\n='

The above command reads a certain number of bytes from the kernel (head -c $entropy /dev/random) encodes that using the base64 algorithm and strips out the trailing equal sign and newlines (for large passwords). This is what Gillmor described as a "high-entropy compact printable/transferable string". The priority, in this case, is to have a token that is as compact as possible with the given entropy, while at the same time using a character set that should cause as little trouble as possible on sites that restrict the characters you can use. Gillmor is a co-maintainer of the Assword password manager, which chose base64 because it is widely available and understood and only takes up 33% more space than the original 8-bit binary encoding. After a lengthy discussion, the pass maintainer, Jason A. Donenfeld, chose the following pipeline:

    read -r -n $length pass < <(LC_ALL=C tr -dc "$characters" < /dev/urandom)

The above is similar, except it uses tr to directly to read characters from the kernel, and selects a certain set of characters ($characters) that is defined earlier as consisting of [:alnum:] for letters and digits and [:graph:] for symbols, depending on the user's configuration. Then the read command extracts the chosen number of characters from the output and stores the result in the pass variable. A participant on the mailing list, Brian Candler, has argued that this wastes entropy as the use of tr discards bits from /dev/urandom with little gain in entropy when compared to base64. But in the end, the maintainer argued that reading "reading from /dev/urandom has no [effect] on /proc/sys/kernel/random/entropy_avail on Linux" and dismissed the objection. Another password manager, KeePass uses its own routines to generate tokens, but the procedure is the same: read from the kernel's entropy source (and user-generated sources in case of KeePass) and transform that data into a transferable string.

Conclusion While there are many aspects to password management, we have focused on different techniques for users and developers to generate secure but also usable passwords. Generating a strong yet memorable password is not a trivial problem as the security vulnerabilities of the pwgen software showed. Furthermore, left to their own devices, users will generate passwords that can be easily guessed by a skilled attacker, especially if they can profile the user. It is therefore essential we provide easy tools for users to generate strong passwords and encourage them to store secure tokens in password managers.

Note: this article first appeared in the Linux Weekly News.

There is/was a somewhat controversial book by Osho which I read long back Sambhog se Samadhi Ki Aur or the English version From Sex to Superconsciousness While I can t say I understand or understood it all, read it about a decade back, the main point shared in the book was that if you are able to achieve bliss/orgasm during sex, you might be able to have a glimpse of super-consciousness. I had to share the above context as I had gone to a meetup couple of weeks back had gone to a meetup where a friend, Dr. Swati Shome is attempting to write an educational book for teenagers to talk about sex. I did help her a bit in the past I tried to share some of the concerns I had as my generation didn t have any guidance from parents or teachers. Most of us were left to our own devices which is similar to today s children as well with the exception that they have the web. You look at both the books, both written in Pune (my home-town) and both talking about the same subject but from so different a view-point. If you see the comments on the meetup page, it really pains to see people s concerns. I don t know if there is any solution to the widespread ignorance, myth-making etc. and hence felt a bit sad . Sharing a small clip I had seen few months back. Just to give a bit of context, the law as has been shared as passed in 2015 has happened after the 2012 Delhi Gang Rape. A part of it is also that the Indian society still frowns upon live-in relationships so in part it may also be a push-back from the conservatives. After all the BJP, a right of the center party has been in power for 2.5 years now so it s possible that they were part of it. As I don t have enough knowledge of what the actual case was, who were the litigants and the defendants, the lawyers and the judge involved, I cannot further speculate. If somebody has more info. or link please pass it on. It would be interesting to know if it was a single bench ruling or 3-5 judge bench. The yin-yan symbol I had shared becomes a bit more apt as in quite a few cultures, including Indian and Japanese, the two are seen as parts of the same coin. One life-giving, the other life-taking or not even taking but converting into something else. That came few days later when I was reading an article about sleep. The purpose of sleep, is to forget . It was slightly strange and yet interesting article. What disturbed me though, was the bit about the mouse being killed and his brain being sliced. I tried to find many a justification for it, but none I could have peace with. And the crux of that is because the being, the creature s wilful consent hasn t been taken. In nature s eyes humans and mice are one and the same. We don t get any special passes due to the fact that we are human. A natural disaster doesn t care whether you are small or big, fat or strong, mouse or wo/man, coward or brave. It s sheer luck and after disaster preparedness that people and animals get saved or not. I thought quite a bit that instead of animals being used for scientific experiments, why don t we use actual humans. While I m sure PETA supporters probably may have spear-headed this idea for a long time, but it doesn t mean I can t come to this realization by myself. After all, it s not about pandering to a group but rather what I think is right. Passing the baton to humans does have its own knotty problems though. For any such kind of endeavour, people s participation and wilful consent would be needed. While humans can and do give wilful consent, it is difficult problem as you don t know the situation in which that consent has been taken. We all know about Organ trafficking . Many people especially from lower economic background may be enticed and cheated with the whole economics for science. In most Indian middle and higher-middle classes religion plays a part even though with death the body is cremated and is supposed to scatter among the Pancha Mahaboota, the five elements. I, for one have no hang-ups if some scientist were to slice my brain to find something, provided I m dead or for that matter any part of the body. If more people thought like that, probably we wouldn t have to specially grow and then kill lab mice and guinea-pigs to test out theories. Possibly medical innovations would probably be a lot faster than now. Ironically, most medical innovations have happened during wars and continues to do so till date. Comments, ideas, suggestions and criticisms all are welcome.
Filed under: Miscellenous Tagged: #Death, #Innocence, #Medical Innovatiion, #Medicine, #Murder, #PETA, #Sex, #sleep, #war, education, exploitation, nature

I was an happy user of rxvt-unicode until I got a laptop with an HiDPI display. Switching from a LoDPI to a HiDPI screen and back was a pain: I had to manually adjust the font size on all terminals or restart them. VTE is a library to build a terminal emulator using the GTK+ toolkit, which handles DPI changes. It is used by many terminal emulators, like GNOME Terminal, evilvte, sakura, termit and ROXTerm. The library is quite straightforward and writing a terminal doesn t take much time if you don t need many features. Let s see how to write a simple one.

A simple terminal Let s start small with a terminal with the default settings. We ll write that in C. Another supported option is Vala.

#include <vte/vte.h>
int
main(int argc, char *argv[])
 
    GtkWidget *window, *terminal;
    /* Initialise GTK, the window and the terminal */
    gtk_init(&argc, &argv);
    terminal = vte_terminal_new();
    window = gtk_window_new(GTK_WINDOW_TOPLEVEL);
    gtk_window_set_title(GTK_WINDOW(window), "myterm");
    /* Start a new shell */
    gchar **envp = g_get_environ();
    gchar **command = (gchar *[]) g_strdup(g_environ_getenv(envp, "SHELL")), NULL  ;
    g_strfreev(envp);
    vte_terminal_spawn_sync(VTE_TERMINAL(terminal),
        VTE_PTY_DEFAULT,
        NULL,       /* working directory  */
        command,    /* command */
        NULL,       /* environment */
        0,          /* spawn flags */
        NULL, NULL, /* child setup */
        NULL,       /* child pid */
        NULL, NULL);
    /* Connect some signals */
    g_signal_connect(window, "delete-event", gtk_main_quit, NULL);
    g_signal_connect(terminal, "child-exited", gtk_main_quit, NULL);
    /* Put widgets together and run the main loop */
    gtk_container_add(GTK_CONTAINER(window), terminal);
    gtk_widget_show_all(window);
    gtk_main();
 

You can compile it with the following command:

gcc -O2 -Wall $(pkg-config --cflags --libs vte-2.91) term.c -o term

And run it with ./term:

More features From here, you can have a look at the documentation to alter behavior or add more features. Here are three examples.

Colors You can define the 16 basic colors with the following code:

#define CLR_R(x)   (((x) & 0xff0000) >> 16)
#define CLR_G(x)   (((x) & 0x00ff00) >>  8)
#define CLR_B(x)   (((x) & 0x0000ff) >>  0)
#define CLR_16(x)  ((double)(x) / 0xff)
#define CLR_GDK(x) (const GdkRGBA)  .red = CLR_16(CLR_R(x)), \
                                    .green = CLR_16(CLR_G(x)), \
                                    .blue = CLR_16(CLR_B(x)), \
                                    .alpha = 0  
vte_terminal_set_colors(VTE_TERMINAL(terminal),
    &CLR_GDK(0xffffff),
    &(GdkRGBA)  .alpha = 0.85  ,
    (const GdkRGBA[]) 
        CLR_GDK(0x111111),
        CLR_GDK(0xd36265),
        CLR_GDK(0xaece91),
        CLR_GDK(0xe7e18c),
        CLR_GDK(0x5297cf),
        CLR_GDK(0x963c59),
        CLR_GDK(0x5E7175),
        CLR_GDK(0xbebebe),
        CLR_GDK(0x666666),
        CLR_GDK(0xef8171),
        CLR_GDK(0xcfefb3),
        CLR_GDK(0xfff796),
        CLR_GDK(0x74b8ef),
        CLR_GDK(0xb85e7b),
        CLR_GDK(0xA3BABF),
        CLR_GDK(0xffffff)
 , 16);

While you can t see it on the screenshot¹, this also enables background transparency.

Miscellaneous settings VTE comes with many settings to change the behavior of the terminal. Consider the following code:

vte_terminal_set_scrollback_lines(VTE_TERMINAL(terminal), 0);
vte_terminal_set_scroll_on_output(VTE_TERMINAL(terminal), FALSE);
vte_terminal_set_scroll_on_keystroke(VTE_TERMINAL(terminal), TRUE);
vte_terminal_set_rewrap_on_resize(VTE_TERMINAL(terminal), TRUE);
vte_terminal_set_mouse_autohide(VTE_TERMINAL(terminal), TRUE);

This will:

• disable the scrollback buffer,
• not scroll to the bottom on new output,
• scroll to the bottom on keystroke,
• rewrap content when terminal size change, and
• hide the mouse cursor when typing.

Update the window title An application can change the window title using XTerm control sequences (for example, with printf "\e]2;$ title \a"). If you want the actual window title to reflect this, you need to define this function:

static gboolean
on_title_changed(GtkWidget *terminal, gpointer user_data)
 
    GtkWindow *window = user_data;
    gtk_window_set_title(window,
        vte_terminal_get_window_title(VTE_TERMINAL(terminal))?:"Terminal");
    return TRUE;
 

Then, connect it to the appropriate signal, in main():

g_signal_connect(terminal, "window-title-changed", 
    G_CALLBACK(on_title_changed), GTK_WINDOW(window));

Final words I don t need much more as I am using tmux inside each terminal. In my own copy, I have also added the ability to complete a word using ones from the current window or other windows (also known as dynamic abbrev expansion). This requires to implement a terminal daemon to handle all terminal windows with one process, similar to urxvtcd. While writing a terminal from scratch ² suits my need, it may not be worth it. evilvte is quite customizable and can be lightweight. Consider it as a first alternative. Honestly, I don t remember why I didn t pick it. UPDATED: evilvte has not seen an update since 2014. Its GTK+3 support is buggy. It doesn t support the latest versions of the VTE library. Therefore, it s not a good idea to use it. You should also note that the primary goal of VTE is to be a library to support GNOME Terminal. Notably, if a feature is not needed for GNOME Terminal, it won t be added to VTE. If it already exists, it will likely to be deprecated and removed.

---

1. Transparency is handled by the composite manager (Compton, in my case).
2. For some definition of scratch since the hard work is handled by VTE.

On Friday night I attended a talk by Sherry Turkle called Reclaiming Conversation: The Power of Talk in a Digital Age . Here are my notes.

---

Turkle is an anthropologist who interviews people from different generations about their communication habits. She has observed cross-generational changes thanks to (a) the proliferation of instant messaging apps such as WhatsApp and Facebook Messenger; and (b) fast web searching from smartphones. Her main concern is that conversation is being trivialised. Consider six or seven college students eating a meal together. Turkle s research has shown that the etiquette among such a group has shifted such that so long as at least three people are engaged in conversation, others at the table feel comfortable turning their attention to their smartphones. But then the topics of verbal conversation will tend away from serious issues you wouldn t talk about your mother s recent death if anyone at the table was texting. There are also studies that purport to show that the visibility of someone s smartphone causes them to take a conversation less seriously. The hypothesis is that the smartphone is a reminder of all the other places they could be, instead of with the person they are with. A related cause of the trivialisation of conversation is that people are far less willing to make themselves emotionally vulnerable by talking about serious matters. People have a high degree of control over the interactions that take place electronically (they can think about their reply for much longer, for example). Texting is not open-ended in the way a face-to-face conversation is. People are unwilling to give up this control, so they choose texting over talking. What is the upshot of these two respects in which conversation is being trivialised? Firstly, there are psycho-social effects on individuals, because people are missing out on opportunities to build relationships. But secondly, there are political effects. Disagreeing about politics immediately makes a conversation quite serious, and people just aren t having those conversations. This contributes to polarisation. Note that this is quite distinct from the problems of fake news and the bubbling effects of search engine algorithms, including Facebook s news feed. It would be much easier to tackle fake news if people talked about it with people around them who would be likely to disagree with them. Turkle understands connection as a capacity for solitude and also for conversation. The drip feed of information from the Internet prevents us from using our capacity for solitude. But then we fail to develop a sense of self. Then when we finally do meet other people in real life, we can t hear them because we just use them to try to establish a sense of self. Turkle wants us to be more aware of the effects that our smartphones can have on conversations. People very rarely take their phone out during a conversation because they want to escape from that conversation. Instead, they think that the phone will contribute to that conversation, by sharing some photos, or looking up some information online. But once the phone has come out, the conversation almost always takes a turn for the worse. If we were more aware of this, we would have access to deeper interactions. A further respect in which the importance of conversation is being downplayed is in the relationships between teachers and students. Students would prefer to get answers by e-mail than build a relationship with their professors, but of course they are expecting far too much of e-mail, which can t teach them in the way interpersonal contact can. All the above is, as I said, cross-generational. Something that is unique to millenials and below is that we seek validation for the way that we feel using social media. A millenial is not sure how they feel until they send a text or make a broadcast (this makes them awfully dependent on others). Older generations feel something, and then seek out social interaction (presumably to share, but not in the social media sense of share ). What does Turkle think we can do about all this? She had one positive suggestion and one negative suggestion. In response to student or colleague e-mails asking for something that ought to be discussed face-to-face, reply I m thinking. And you ll find they come to you. She doesn t want anyone to write empathy apps in response to her findings. For once, more tech is definitely not the answer. Turkle also made reference to the study reported here and here and here.

Last week, while working on CodeWorld, via a sequence of yak shavings, I ended up creating a nicely small library that provides Control.Applicative.Succs, a new applicative functor. And because I am trying to keep my Haskell karma good¹, I wanted to actually prove that my code fulfills the Applicative and Monad laws. This led me to inserted writing long comments into my code, filled with lines like this:

The second Applicative law:
  pure (.) <*> Succs u us <*> Succs v vs <*> Succs w ws
= Succs (.) [] <*> Succs u us <*> Succs v vs <*> Succs w ws
= Succs (u .) (map (.) us) <*> Succs v vs <*> Succs w ws
= Succs (u . v) (map ($v) (map (.) us) ++ map (u .) vs) <*> Succs w ws
= Succs (u . v) (map (($v).(.)) us ++ map (u .) vs) <*> Succs w ws
= Succs ((u . v) w) (map ($w) (map (($v).(.)) us ++ map (u .) vs) ++ map (u.v) ws)
= Succs ((u . v) w) (map (($w).($v).(.)) us ++ map (($w).(u.)) vs ++ map (u.v) ws)
= Succs (u (v w)) (map (\u -> u (v w)) us ++ map (\v -> u (v w)) vs ++ map (\w -> u (v w)) ws)
= Succs (u (v w)) (map ($(v w)) us ++ map u (map ($w) vs ++ map v ws))
= Succs u us <*> Succs (v w) (map ($w) vs ++ map v ws)
= Succs u us <*> (Succs v vs <*> Succs w ws)

Honk if you have done something like this before! I proved all the laws, but I was very unhappy. I have a PhD on something about Haskell and theorem proving. I have worked with Isabelle, Agda and Coq. Both Haskell and theorem proving is decades old. And yet, I sit here, and tediously write manual proofs by hand. Is this really the best we can do? Of course I could have taken my code, rewritten it in, say, Agda, and proved it correct there. But (right now) I don t care about Agda code. I care about my Haskell code! I don t want to write it twice, worry about copying mistakes and mismatchs in semantics, and have external proofs to maintain. Instead, I want to prove where I code, and have the proofs checked together with my code! Then it dawned to me that this is, to some extent, possible. The Haskell compiler comes with a sophisticated program transformation machinery, which is meant to simplify and optimize code. But it can also be used to prove Haskell expressions to be equivalent! The idea is simple: Take two expressions, run both through the compiler s simplifier, and check if the results are the same. If they are, then the expressions are, as far as the compiler is concerned, equivalent. A handful of hours later, I was able to write proof tasks like

app_law_2 = (\ a b (c::Succs a) -> pure (.) <*> a <*> b <*> c)
        === (\ a b c -> a <*> (b <*> c))

and others into my source file, and the compiler would tell me happily:

[1 of 1] Compiling Successors       ( Successors.hs, Successors.o )
GHC.Proof: Proving getCurrent_proof1  
GHC.Proof: Proving getCurrent_proof2  
GHC.Proof: Proving getCurrent_proof3  
GHC.Proof: Proving ap_star  
GHC.Proof: Proving getSuccs_proof1  
GHC.Proof: Proving getSuccs_proof2  
GHC.Proof: Proving getSuccs_proof3  
GHC.Proof: Proving app_law_1  
GHC.Proof: Proving app_law_2  
GHC.Proof: Proving app_law_3  
GHC.Proof: Proving app_law_4  
GHC.Proof: Proving monad_law_1  
GHC.Proof: Proving monad_law_2  
GHC.Proof: Proving monad_law_3  
GHC.Proof: Proving return_pure  
GHC.Proof proved 15 equalities

This is how I want to prove stuff about my code! Do you also want to prove stuff about your code? I packaged this up as a GHC plugin in the Haskell library ghc-proofs (not yet on Hackage). The README of the repository has a bit more detail on how to use this plugin, how it works, what its limitations are and where this is heading. This is still only a small step, but finally there is a step towards low threshold program equivalence proofs in Haskell.

My WordPress blog got hacked two days ago and now twice today. This morning I purged MySQL and restored a good backup from three days ago, changed all DB and WordPress passwords (both the old and new ones were long and autogenerated ones), but not even an hour after the redeploy the hack was back. (It can still be seen on Planet Debian and Planet Ubuntu. Neither the Apache logs nor the Journal had anything obvious, nor were there any new files in global or user www directories, so I m a bit stumped how this happened. Certainly not due to bruteforcing a password, that would both have shown in the logs and also have triggered ban2fail, so this looks like an actual vulnerability. I upgraded to WordPress 4.7.1 a few days ago, and apparently 4.7.2 fixes a few vulnerabilities, although all of them don t sound like they would match my situation. jessie-backports is still at 4.7.1, so I missed that update. But either way, all WordPress blogs hosted on my server are down for the time being. I took this as motivation to finally migrate to something more robust. WordPress has tons of features that I never need, and also a lot of overhead (dynamic generation, MySQL, its own user/passwords, etc.). I had a look around, and it seems Hugo and Blogofile are nice contenders no privileges, no database, outputting static files, input is Markdown (so much nicer to type than HTML!), and maintaining your blog in git and previewing the changes on my local laptop are straightforward. I happened to try Hugo first, and like it enough to give it an extended try you have plenty of themes to choose from and they are straightforward to customize, so I don t need to spend a lot of time learning and crafting CSS. I ran the WordPress to Hugo Exporter, and it produced remarkable results fairly usable HTML Markdown and metadata conversion, it keeps all the original URLs, and it s painless to use. Nicely done! So here it is, on to a much more secure server now! \o/

I ve got some little scripts and a template here to automatically create Vagrant boxes from cutting edge Debian testing daily snapshots (netinstall ISO image) using HashiCorp s Packer. To create Vagrant boxes with these, you first need a running binary of Packer. There is a Debian package available if that s also your working environment, but Packer is going to be introduced into the stable branch with the upcoming Stretch release itself. However, Ubuntu already has it, and some other derivatives, too. And there are prebuild binaries available from the developer s site which run fine out-of-the-box (you just have to put the single binary somewhere into you $PATH, or expand that to find it). The JSON template should run with any Packer which is available for any of the different systems. Vagrant itself isn t needed to build the box with Packer, but Virtualbox is of course needed to pre bake the machine image within a virtual machine. In Debian the base binaries of Virtualbox are in the contrib archive section, so that source might be added to /etc/apt/sources.list, if haven t already. The scripts have been tested to run with 5.1.10, and I haven t seen that any late versions are demanded in particular, but of course heavily outdated versions might not work properly. Packer installs the guest additions ISO file for Virtualbox into the virtual machine (and the shipped provisioning script then builds them inside). For that, the Debian package which ships that (which is in non-free) is recognized if it is installed, and then could be used by Packer. When the ISO isn t available in the places which Packer checks on the host the builder then automatically downloads the corresponding ISO from http://download.virtualbox.org/virtualbox. When the tarball with the scripts is unpacked, just do make create and the process should run through, if Packer and Virtualbox are available. If your environment doesn t have GNU Make nor wget you might want to copy the relevant lines from the Makefile and run it manually. But if it does, just do it like this:

/tmp/debian-testing-vagrantbox$ make create
virtualbox-iso output will be in this color.
==> virtualbox-iso: Downloading or copying Guest additions
    virtualbox-iso: Downloading or copying: file:///usr/share/virtualbox/VBoxGuestAdditions.iso
==> virtualbox-iso: Downloading or copying ISO
    virtualbox-iso: Downloading or copying: http://cdimage.debian.org/cdimage/daily-builds/daily/arch-latest/amd64/iso-cd/debian-testing-amd64-netinst.iso
    virtualbox-iso: Download progress: 10%
 ... 
    virtualbox-iso: Download progress: 96%
==> virtualbox-iso: Starting HTTP server on port 8219
==> virtualbox-iso: Creating virtual machine...
==> virtualbox-iso: Creating hard drive...
==> virtualbox-iso: Creating forwarded port mapping for communicator (SSH, WinRM, etc) (host port 2885)
==> virtualbox-iso: Starting the virtual machine...
==> virtualbox-iso: Waiting 10s for boot...
==> virtualbox-iso: Typing the boot command...
==> virtualbox-iso: Waiting for SSH to become available...

The Virtualbox window then pops up and the build process continues within the virtual machine for a while. You might want to file a Github issue when there s a problem on your machine, please! (please include the tail of your packer.log) The Packer template (debian-testing-vagrant.json) is described in the documentation of the virtualbox-iso builder. A preseeding script for the Debian Installer (preseed.cfg) is also included which gets injected into the virtual build environment during the build process. The creation progress of the Debian base installation could be easily monitored since the Virtualbox window is fully shown during the Packer run (if you loose your mouse pointer by clicking inside that window, do <Right>+<Control> to escape). For good performance, a fast internet connection is needed since a whole base system must be downloaded if that s available the whole automated process very only takes a couple of minutes to complete on a non-SSD machine. When Packer has finished and a fresh box is created (the size is about 690 MB), it then could be used with Vagrant. Just add the new box with:

/tmp/debian-testing-vagrantbox$ vagrant box add stretch-preview debian-testing-vagrant.box
==> box: Box file was not detected as metadata. Adding it directly...
==> box: Adding box 'stretch-preview' (v0) for provider: 
    box: Unpacking necessary files from: file:///tmp/debian-testing-vagrantbox/debian-testing-vagrant.box
==> box: Successfully added box 'stretch-preview' (v0) for 'virtualbox'!

It then could be initialized within a random working directory with:

/tmp/myproject$ vagrant init stretch-preview
A  Vagrantfile  has been placed in this directory. You are now
ready to  vagrant up  your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
 vagrantup.com  for more information on using Vagrant.

After that, you could launch the virtual box with:

/tmp/myproject$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'stretch-preview'...
==> default: Matching MAC address for NAT networking...
==> default: Setting the name of the VM: myproject_default_1486321215067_75270
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: 
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default: 
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Mounting shared folders...
    default: /vagrant => /tmp/myproject
/tmp/myproject$

Then you can SSH into it by doing (touch is used here only to point to the shared folder):

/tmp/myproject$ touch hello!
/tmp/myproject$ vagrant ssh
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
/usr/bin/xauth:  file /home/vagrant/.Xauthority does not exist
vagrant@packer-virtualbox-iso-1486319595:~$ $ cat /etc/debian_version
9.0
vagrant@packer-virtualbox-iso-1486319595:~$ ls /vagrant/
hello!  Vagrantfile
vagrant@packer-virtualbox-iso-1486319595:~$ exit
logout
Connection to 127.0.0.1 closed.
/tmp/myproject$ vagrant halt
==> default: Attempting graceful shutdown of VM...
/tmp/myproject$

If you haven t worked with Vagrant before, maybe this is appealing. The user experience is somehwhat different from working with a chroot. Packer makes it very convenient to keep freshly created boxes for it coming. Have fun!

My most loathed feature of Go is the mandatory use of GOPATH: I do not want to put my own code next to its dependencies. Hopefully, this issue is slowly starting to be accepted by the main authors. In the meantime, you can workaround this problem with more opinionated tools (like gb) or by crafting your own Makefile. For the later, you can have a look at Filippo Valsorda s example or my own take which I describe in more details here. This is not meant to be an universal Makefile but a relatively short one with some batteries included. It comes with a simple Hello World! application.

Project structure For a standalone project, vendoring is a must-have¹ as you cannot rely on your dependencies to not introduce backward-incompatible changes. Some packages are using versioned URLs but most of them aren t. There is currently no standard tool to handle vendoring. My personal take is to vendor all dependencies with Glide. It is a good practice to split an application into different packages while the main one stay fairly small. In the hellogopher example, the CLI is handled in the cmd package while the application logic for printing greetings is in the hello package:

.
  cmd/
    hello.go
    root.go
    version.go
  glide.lock (generated)
  glide.yaml
  vendor/ (dependencies will go there)
  hello/
    root.go
    root_test.go
  main.go
  Makefile
  README.md

Down the rabbit hole Let s take a look at the various features of the Makefile.

GOPATH handling Since all dependencies are vendored, only our own project needs to be in the GOPATH:

PACKAGE  = hellogopher
GOPATH   = $(CURDIR)/.gopath
BASE     = $(GOPATH)/src/$(PACKAGE)
$(BASE):
    @mkdir -p $(dir $@)
    @ln -sf $(CURDIR) $@

The base import path is hellogopher, not github.com/vincentbernat/hellogopher: this shortens imports and makes them easily distinguishable from imports of dependency packages. However, your application won t be go get-able. This is a personal choice and can be adjusted with the $(PACKAGE) variable. We just create a symlink from .gopath/src/hellogopher to our root directory. The GOPATH environment variable is automatically exported to the shell commands of the recipes. Any tool should work fine after changing the current directory to $(BASE). For example, this snippet builds the executable:

.PHONY: all
all:   $(BASE)
    cd $(BASE) && $(GO) build -o bin/$(PACKAGE) main.go

Vendoring dependencies Glide is a bit like Ruby s Bundler. In glide.yaml, you specify what packages you need and the constraints you want on them. Glide computes a glide.lock file containing the exact versions for each dependencies (including recursive dependencies) and download them in the vendor/ folder. I choose to check into the VCS both glide.yaml and glide.lock files. It s also possible to only check in the first one or to also check in the vendor/ directory. A work-in-progress is currently ongoing to provide a standard dependency management tool with a similar workflow. We define two rules²:

GLIDE = glide
glide.lock: glide.yaml   $(BASE)
    cd $(BASE) && $(GLIDE) update
    @touch $@
vendor: glide.lock   $(BASE)
    cd $(BASE) && $(GLIDE) --quiet install
    @ln -sf . vendor/src
    @touch $@

We use a variable to invoke glide. This enables a user to easily override it (for example, with make GLIDE=$GOPATH/bin/glide).

Using third-party tools Most projects need some third-party tools. We can either expect them to be already installed or compile them in our private GOPATH. For example, here is the lint rule:

BIN    = $(GOPATH)/bin
GOLINT = $(BIN)/golint
$(BIN)/golint:   $(BASE) #  
    go get github.com/golang/lint/golint
.PHONY: lint
lint: vendor   $(BASE) $(GOLINT) #  
    @cd $(BASE) && ret=0 && for pkg in $(PKGS); do \
        test -z "$$($(GOLINT) $$pkg   tee /dev/stderr)"   ret=1 ; \
     done ; exit $$ret

As for glide, we let the user a chance to override which golint executable to use. By default, it uses a private copy. But a user can use its own copy with make GOLINT=/usr/bin/golint. In , we have the recipe to build the private copy. We simply issue go get³ to download and build golint. In , the lint rule executes golint on each package contained in the $(PKGS) variable. We ll explain this variable in the next section.

Working with non-vendored packages only Some commands need to be provided with a list of packages. Because we use a vendor/ directory, the shortcut ./... is not what we expect as we don t want to run tests on our dependencies⁴. Therefore, we compose a list of packages we care about:

PKGS = $(or $(PKG), $(shell cd $(BASE) && \
    env GOPATH=$(GOPATH) $(GO) list ./...   grep -v "^$(PACKAGE)/vendor/"))

If the user has provided the $(PKG) variable, we use it. For example, if they want to lint only the cmd package, they can invoke make lint PKG=hellogopher/cmd which is more intuitive than specifying PKGS. Otherwise, we just execute go list ./... but we remove anything from the vendor directory.

Tests Here are some rules to run tests:

TIMEOUT = 20
TEST_TARGETS := test-default test-bench test-short test-verbose test-race
.PHONY: $(TEST_TARGETS) check test tests
test-bench:   ARGS=-run=__absolutelynothing__ -bench=.
test-short:   ARGS=-short
test-verbose: ARGS=-v
test-race:    ARGS=-race
$(TEST_TARGETS): test
check test tests: fmt lint vendor   $(BASE)
    @cd $(BASE) && $(GO) test -timeout $(TIMEOUT)s $(ARGS) $(PKGS)

A user can invoke tests in different ways:

• make test runs all tests;
• make test TIMEOUT=10 runs all tests with a timeout of 10 seconds;
• make test PKG=hellogopher/cmd only runs tests for the cmd package;
• make test ARGS="-v -short" runs tests with the specified arguments;
• make test-race runs tests with race detector enabled.

Tests coverage go test includes a test coverage tool. Unfortunately, it only handles one package at a time and you have to explicitely list the packages to be instrumented, otherwise the instrumentation is limited to the currently tested package. If you provide too many packages, the compilation time will skyrocket. Moreover, if you want an output compatible with Jenkins, you ll need some additional tools.

COVERAGE_MODE    = atomic
COVERAGE_PROFILE = $(COVERAGE_DIR)/profile.out
COVERAGE_XML     = $(COVERAGE_DIR)/coverage.xml
COVERAGE_HTML    = $(COVERAGE_DIR)/index.html
.PHONY: test-coverage test-coverage-tools
test-coverage-tools:   $(GOCOVMERGE) $(GOCOV) $(GOCOVXML) #  
test-coverage: COVERAGE_DIR := $(CURDIR)/test/coverage.$(shell date -Iseconds)
test-coverage: fmt lint vendor test-coverage-tools   $(BASE)
    @mkdir -p $(COVERAGE_DIR)/coverage
    @cd $(BASE) && for pkg in $(PKGS); do \ #  
        $(GO) test \
            -coverpkg=$$($(GO) list -f '  join .Deps "\n"  ' $$pkg   \
                    grep '^$(PACKAGE)/'   grep -v '^$(PACKAGE)/vendor/'   \
                    tr '\n' ',')$$pkg \
            -covermode=$(COVERAGE_MODE) \
            -coverprofile="$(COVERAGE_DIR)/coverage/ echo $$pkg   tr "/" "-" .cover" $$pkg ;\
     done
    @$(GOCOVMERGE) $(COVERAGE_DIR)/coverage/*.cover > $(COVERAGE_PROFILE)
    @$(GO) tool cover -html=$(COVERAGE_PROFILE) -o $(COVERAGE_HTML)
    @$(GOCOV) convert $(COVERAGE_PROFILE)   $(GOCOVXML) > $(COVERAGE_XML)

First, we define some variables to let the user override them. We also require the following tools (in ):

• gocovmerge merges profiles from different runs into a single one;
• gocov-xml converts a coverage profile to the Cobertura format;
• gocov is needed to convert a coverage profile to a format handled by gocov-xml.

The rules to build those tools are similar to the rule for golint described a few sections ago. In , for each package to test, we run go test with the -coverprofile argument. We also explicitely provide the list of packages to instrument to -coverpkg by using go list to get a list of dependencies for the tested package and keeping only our owns.

Final result While the main goal of using a Makefile was to work around GOPATH, it s also a good place to hide the complexity of some operations, notably around test coverage. The excerpts provided in this post are a bit simplified. Have a look at the final result for more perks!

---

1. In Go, vendoring is about both bundling and dependency management. As the Go ecosystem matures, the bundling part (fixed snapshots of dependencies) may become optional but the vendor/ directory may stay for dependency management (retrieval of the latest versions of dependencies matching a set of constraints).
2. If you don t want to automatically update glide.lock when a change is detected in glide.yaml, rename the target to deps-update and make it a phony target.
3. There is some irony for bad mouthing go get and then immediately use it because it is convenient.
4. I think ./... should not include the vendor/ directory by default. Dependencies should be trusted to have run their own tests in the environment they expect them to succeed. Unfortunately, this is unlikely to change.

Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you re interested in Java, Games and LTS topics, this might be interesting for you. Debian Games

• In January 2017 we had the last chance to get new upstream releases into the next stable release of Debian 9 aka Stretch. Hence I packaged new versions of pygame-sdl2, renpy, fife, unknown-horizons, redeclipse and redeclipse-data and also backported Red Eclipse to Jessie.
• I uploaded fifechan to unstable and applied an upstream patch to fix a segmentation fault (#852247) in Unknown Horizons.
• Package cleanups and improvements: freeorion (#843538), I enabled support for mips64el again; I tidied up gtkatlantic, powermanga, lincity-ng, opencity and tecnoballz; I applied a patch from Reiner Herrmann to make the build of netpanzer reproducible (#827150); In spring I changed the build-dependency of asciidoc to asciidoc-base (#850387) although it turned out later that this wasn t strictly needed. I also removed ConvertUTF8 related code from spring because it might be non-free. I don t think this is necessarily true but I didn t want to argue with Lintian in this case.
• I sponsored a new upstream release of pentobi for Juhani Numminen.
• I backported minetest 0.4.5 to jessie-backports and fixed #851114, which I think was not really an issue since we already provide the font sources in Debian and Minetest depends on the respective package.
• I triaged RC bug #847812 in pysolfc, provided a patch and reassigned the issue to src:pillow. Apparently this affected a lot more 32 bit applications written in Python.

Debian Java

• For Debian Java the situation was very similar but I didn t want to break too many reverse-dependencies right before the freeze. So I chose rather harmless packages and updated objenesis, syncany and jackson-dataformat-xml. At first I couldn t build objenesis 2.5 from source but after some consultation on the debian-java list, it became clear that the test failure was probably an upstream bug. Henri Tremblay kindly issued a new release of Objenesis which addressed the build failure.
• Package updates and bugfixes: sweethome3d (#849885), I confirmed a build-failure in java-gnome (#849669) and investigated another issue in jackson-module-afterburner (#849931) which was later removed from Debian. I fixed an RC bug in commons-daemon (#849649). I also released new revisions of libhamcrest-java (#846116), libjide-oss-java, libjchart2d-java, activemq, osgi-annotations, jsemver, openjpa and xmlbeans (#820839,#822091).
• I prepared security updates for groovy and groovy2 in Jessie. (#853233, #853235)

Debian LTS This was my eleventh month as a paid contributor and I have been paid to work 12,75 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following:

• From 16. January until 22. January I was in charge of our LTS frontdesk. I triaged security issues in imagemagick, wordpress, hesiod, opus, mysql-5.5, netbeans, groovy and zoneminder.
• DLA-779-1. Issued a security update for Tomcat 7 fixing 1 CVE and a regression when running Tomcat with SecurityManager enabled.
• DLA-761-2. Issued a regression update for python-bottle. (Debian bug #850176).
• DLA-781-1 and DLA-781-2. Issued a security update for Asterisk fixing 2 CVE after I had prepared the package last month. Later Brad Barnett discovered a regression when using SIP communication and provided assistance with debugging the issue. I corrected this one in DLA-781-2.
• DLA-792-1. Issued a security update for libphp-swiftmailer fixing 1 CVE.
• DLA-793-1. Issued a security update for opus fixing 1 CVE.
• DLA-794-1. Issued a security update for groovy fixing 1 CVE.
• DLA-797-1. Issued a security update for mysql-5.5 fixing 10 CVE. The update was prepared by Lars Tangvald.
• DLA-813-1. Issued a security update for wordpress fixing 9 CVE.

Misc

• In xarchiver (#850103) I added binutils to the list of suggested packages, in iftop (#850040) I applied a patch from Brian Russell and I packaged a new upstream release of mediathekview, a Java application to watch and download broadcasts from German television stations. I had to make some major packaging changes because the build system switched from Ant to Gradle but there were fewer issues than expected.

FTP assistant This month I only marked 146 packages for accept and rejected 25 packages. I only sent 3 emails to maintainers asking questions. Nevertheless I could pass a big mark. All in all I accepted more than 10000 packages now! Debian LTS This was my thirty-first month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. This month my all in all workload has been 12.75h. During that time I did uploads of

• [DLA 805-1] bind9 security update for three CVEs
• [DLA 806-1] zoneminder security update for one CVE

Unfortunately the upload of jasper had to be postponed, as there is no upstream fix for most of the open CVEs yet.
I also suggested to mark th slum-llnl CVE as , as the patch would be too invasive. Further I did another week of frontdesk work. Last but not least I took care of about 140 items of the TODO list[1]. Ok, it was not that much work, but the enormous number is impressing :-). I also had a look at [2] and filed bugs against two packages. Within hours the maintainers responded to that bugs, clarified everything to mark the CVEs as not-affected and nobody has to care about them anymore. This is a good example of how the knowledge of the maintainer can help the security teams! So, if you have some time left, have a look at [3] and take care of something. [1] https://security-tracker.debian.org/tracker/status/todo
[2] https://security-tracker.debian.org/tracker/status/unreported
[3] https://security-tracker.debian.org/tracker Other stuff This month I sponsored a new round of sidedoor and printrun. After advocating Dara Adib to become Debian Maintainer, I hope my activities as sponsor can be reduced again :-). Further I uploaded another version of setserial, but as you can see in #850762 it does not seem to satisfy everybody. I also uploaded new upstream versions of duktape and pipexec. As I didn t do any DOPOM in December I adopted two packages in January: pescetti and salliere. I dedicate those uploads to my aunt Birgit, who was a passionate bridge player. You will never be forgotten.

The soft freeze has been on going for almost a month now and the full stretch freeze will start tomorrow night (UTC). It has definitely been visible in the number of unblock requests that we have received so far. Fortunately, we are no where near the rate of the jessie freeze. At the moment, all unblock requests are waiting for the submitter (either for a clarification or an upload). Looking at stretch at a glance (items are in no particular order):

• Secure boot support is progressing too slowly.
• The openssl transition is still on-going and badly needs closure.
• The MySQL to MariaDB transition is done.
• The gcc-5 removal almost complete missing only 2 packages.
• About 150 RC bugs (in key packages) need to be handled
• We are missing a stretch-backport suite so DSA can start their upgrade tests
• The release checklist have a few items that could be completed now.

Secure boot support Currently, we are blocked on two items:

• We do not have signing done yet for the boot packages (not even manual signing).
• Our shim is not yet signed, so no hardware would be trusting our boot chain out of the box.

After they are done, we are missing a handful of uploads to provide a signed bootloader etc. plus d-i and some infrastructure bits need to be updated. At the moment, we are waiting for a handful of key people/organisations to move on their part. As such, there is not a lot you can do to assist here (unless you are already involved in the work).
On the flip side, if both of these items are resolved soon, there is a good chance that we can support secure boot in stretch.See bug#820036 and blockers for more information on the remaining items. Where can you help with the release? At the moment, the best you can do is to:

• Test (packages, upgrades, etc.) and report bugs
• File bugs against release-notes for issues that should be documented
• Fix RC bugs (please see the next section)

Release Critical Bug report The UDD bugs interface currently knows about the following release critical bugs:

• In Total: 1148 (Including 193 bugs affecting key packages)
  • Affecting stretch: 294 (key packages: 158)
    That s the number we need to get down to zero before the release. They can be split in two big categories:
    • Affecting stretch and unstable: 232 (key packages: 134)
      Those need someone to find a fix, or to finish the work to upload a fix to unstable:
      • 30 bugs are tagged patch . (key packages: 21)
        Please help by reviewing the patches, and (if you are a DD) by uploading them.
      • 17 bugs are marked as done, but still affect unstable. (key packages: 5)
        This can happen due to missing builds on some architectures, for example. Help investigate!
      • 185 bugs are neither tagged patch, nor marked done. (key packages: 108)
        Help make a first step towards resolution!
    • Affecting stretch only: 62 (key packages: 24)
      Those are already fixed in unstable, but the fix still needs to migrate to stretch. You can help by submitting unblock requests for fixed packages, by investigating why packages do not migrate, or by reviewing submitted unblock requests.
      • 36 bugs are in packages that are unblocked by the release team. (key packages: 14)
      • 26 bugs are in packages that are not unblocked. (key packages: 10)

Filed under: Debian, Release-Team

A major new update of the RPushbullet package is now on CRAN. RPushbullet interfacing the neat Pushbullet service for inter-device messaging, communication, and more. It lets you easily send alerts like the one to the to your browser, phone, tablet, ... -- or all at once. This release owes a lot to Seth Wenchel who was instrumental in driving several key refactorings. We now use the curl package instead of relying on system() calls to the binary. We also switched from RJSONIO to jsonlite. A new helper function to create the required resourcefile was added, and several other changes were made as detailed below in the extract from the NEWS.Rd file.

Changes in version 0.3.0 (2017-02-03)

• The curl binary use was replaced by use of the curl package; several new helper functions added (PRs #30, #36 by Seth closing #29)
• Use of RJSONIO was replaced by use of jsonlite (PR #32 by Seth closing #31)
• A new function pbSetup was added to aid creating the resource file (PRs #34, #37 by Seth and Dirk)
• The package intialization was refactored so that non-loading calls such as RPushbullet::pbPost(...) now work (#33 closing #26)
• The test suite was updated and extended
• The Travis script was updated use run.sh
• DESCRIPTION, README.md and other files were updated for current R CMD check standards
• Deprecated parts such as 'type=address' were removed, and the documentation was updated accordingly.
• Coverage support was added (in a 'on-demand' setting as automated runs would need a Pushbullet API token)

Courtesy of CRANberries, there is also a diffstat report for this release. More details about the package are at the RPushbullet webpage and the RPushbullet GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Today, I have released the first beta for Tails 3.0, that will be the first version of Tails based on Debian 9 (Stretch). Our automated test suite pretends it works pretty well and matches our safety expectations. I'm inclined to trust it. But as we learned after porting Tails to Squeeze, Wheezy and Jessie: quick, exploratory testing of pre-releases will not identify all the remaining regressions. So this time I'm trying to change this narrative a bit. I have committed to provide security updates for the 3.0~ series, just like we do for stable versions of Tails. This was the only missing bit to make me feel comfortable asking my fellow Tails contributors to upgrade to Tails 3.0~beta1 for their daily usage. I hope this helps us release a great Tails 3.0 on June 13 and a better Debian Stretch too: the more early users of Tails based on Stretch, the more chances they identify a few annoying regressions in Stretch before it's called stable :) For details, see the official announcement. Next step: FOSDEM. And then, back to organizing an event that aims at improving both social and technical aspects of Debian (to be announced in about a week, stay tuned); because the way we get organized and how power is distributed matter.

It's that time of the year :-) And FOSDEM is fun. But this year I won't be going; there was a scheduling conflict, and I didn't really have anything new to present (although I probably could have shifted around priorities to get something). But FOSDEM 2017 also means there's a year since FOSDEM 2016, where I presented Nageru, my live video mixer. And that's been a pretty busy year, so I thought I'd do a live cap from high up above. First of all, Nageru has actually been used in production we did Solskogen and Fyrrom, and both gave invaluable input. Then there have been some non-public events, which have also been useful. The Nageru that's in git right now is evolved considerably from the 1.0.0 that was released last year. diffstat shows 19660 insertions and 3543 deletions; that's counting about 2500 lines of vendored headers, though. Even though I like deleting code much more than adding it, the doubling (from ~10k to ~20k lines) represents a significant amount of new features: 1.1.x added support for non-Intel GPUs. 1.2.x added support for DeckLink input cards (through Blackmagic's proprietary drivers), greatly increasing hardware support, and did a bunch of small UI changes. 1.3.x added x264 support that's strong enough that Nageru has really displaced VLC as my go-to tool for just video-signal-to-H.264-conversion (even though it feels overkill), and also added hotplug support. 1.4.x added multichannel audio support including support for MIDI controllers, and also a disk space indicator (because when you run out of disk during production without understanding that's what happens, it really sucks), and brought extensive end-user documentation. And 1.5.x, in development right now, will add HDMI/SDI output, which, like all the previous changes, requires various rearchitecting and fixing. Of course, there are lots of things that haven't changed as well; the basic UI remains the same, including the way the theme (governing the look-and-feel of the finished video stream) works. The basic design has proved sound, and I don't think I would change a lot if I were to design something like 1.0.0 again. As a small free software project, you have to pick your battles, and I'm certainly glad I didn't start out doing something like network support (or a distributed architecture in general, really). So what's for the next year of Nageru? It's hard to say, and it will definitely depend on the concrete needs of events. A hot candidate (since I might happen to need it) is chroma keying, although good keying is hard to get right and this needs some research. There's also been some discussion around other concrete features, but I won't name them until a firm commitment has been made; priorities can shift around, and it's important to stay flexible. So, enjoy FOSDEM! Perhaps I'll return with a talk in 2018. In the meantime, I'll preparing the stream for the 2017 edition of Fyrrom, and I know for sure there will be more events, more features and more experiences to be had. And, inevitably, more bugs. :-)

Debian LTS November marked the 21st month I contributed to Debian LTS under the Freexian umbrella. I had 8 hours allocated which I used for:

• the first half of a LTS front desk week
• updating icedove 45.6.0 resulting in DLA-782-1 fixing 8 CVEs
• releasing DLA-783-1 for XEN, the actual update was provided by credativ
• testing the bind9 update prepared by Thorsten Alteholz
• fixing 8 CVEs in imagemagick resulting in DLA-807-1.
• work on recent qemu CVEs

Other Debian stuff

• Usual bunch of libvirt and related uploads
• Uploaded git-buildpackage 0.8.10 to 0.8.12.1 to experimental and unstable fixing (among other things) a long standing bug when using multiple tarballs with filters and pristine-tar as well as making generated orig tarballs reproducible so one gets identical tarballs even without pristine-tar.
• Ran a gbp import-dsc of unstable and filed bugs for cases where pristine-tar would not import the package. Started to look into git-apply errors.

Some other Free Software activites

• libplanfahr: switched the example to python3 and made it parse arguments without date as "today":

  $ ./run python examples/trip-query.py --when=21:00 Essen Gelsenkirchen
  Loaded provider de_db
  Start: Essen Hbf
  End: Gelsenkirchen Hbf
  Trip #1
         Start:     Essen Hbf
         Departure: 2017-02-02 21:18
         Delay:     0
         End:       Gelsenkirchen Hbf
         Arrival:   2017-02-02 21:26
         Delay:     0
         Switches:  0
  Trip #2
         Start:     Essen Hbf
         Departure: 2017-02-02 21:22
         Delay:     0
         End:       Gelsenkirchen Hbf
         Arrival:   2017-02-02 21:33
         Delay:     0
         Switches:  0
  Trip #3
         Start:     Essen Hbf
         Departure: 2017-02-02 21:44
         Delay:     0
         End:       Gelsenkirchen Hbf
         Arrival:   2017-02-02 21:52
         Delay:     0
         Switches:  0
  

• Proposed a workaround to rbvmomi to massively speedup cloning under certain conditions when using CachedOVFDeployer
• Proposed a fix to unbreak ansible's zypper module on first installations
• Made ausroller use git-buildpackage from pypi on non Debian based distros
• Made further progess on the Merkur board clones

Changes

• gitk: colour of remote refs
• https-everywhere: Update Debian rules
• check-all-the-things: release, security fix, reset terminal modes, allow running removed checkers, regression fix (quote fix), TODO items (for deep-text-corrector, sblint, decopy, Coala Bear & dangerous Python checks)
• devscripts: fix grep-excuses warning
• autorevision: build correctness, tarball reproducibility (due to gzip & AUTHORS.txt)
• whowatch: parameter correctness (for NULL, int), spelling/grammar/typos, https, remove dead links & restore terminal status on exit
• spelling dictionaries:
  • codespell: recod, undefuned, normaly, uner, buid, stength, privide, decstiption(s), revrese, lightweigh, multible, asbtraction, resulution, meatadata, seriuos, transalte, interractive, timestan, partiton, databas, claculate, patern, substiution, freee, operatation(s), ambigous, clasified, coypright, previlege, gental, bacup & detction
  • lintian: recod, undefuned, uner, buid, stength, privide, dimentional, decstiption(s), revrese, lightweigh, asbtraction, resulution, FTBS, meatadata, seriuos, transalte, interractive, timestan, partiton, databas, claculate, patern, substiution, freee, operatation(s), previlege, gental, bacup & detction
• Debian timeline: latest bug milestones
• Debian bits: FOSDEM draft fixes
• Planet Debian derivatives: add Netrunner & update BlankOn logo
• Debian buildd website: plain text logs (1 2 3)
• Debian uscan multi-redirector: add github package.json/cmake checker & remove gitorious
• Debian packages website: update manual pages patch to use https, merge patches (1 2)
• Debian website: fix security Makefiles for DLAs (for 2017, 2016, 2015, 2014), derivatives (wording/formatting fixes 1, 2), build by default), link fixes (for CD brokenness, stability, gitweb to cgit)
• Debian wiki pages: AlejandroRios, ChrootOnAndroid, CopyrightReviewTools, CrossCompiling, CrossGrading, DebianBootstrap, DebianDay, (2017), DebianEvents (Asia, oc) DebianScience, DebianWiki/Browsing, dedup.debian.net, Derivatives/Census (AstraLinux, BlankOn, Netrunner, SalentOS, SerbianLinux, Symbiosis, Template), EmbeddedCodeCopies, ExternalEntities, Fonts, fr/GraphicsCard, fr/Intel HD Graphics, FrontPage, gobby.debian.org, HP/ProLiant, Ideas, InstallingDebianOn, TurrisOmnia (1 2), InterWikiMap, josch/notes, LVM, Merchandise/BoFDebConf10, Mobile (1 2), NewArchiveSections, PaulWise/InterestingSoftware, PortTemplate, ppc64el, PressCoverage2017, PressCoverage, redmine, ReproducibleBuilds/History, Services/Debian Packages, SponsorChecklist, Sprints (2017/DebianMed2017 (1 2)), SystemBuildTools, systemd, Teams/Dpkg/FAQ, ThisWeekInDebian, WhyDebian, Year
• File Formats wiki pages: ACE, File_identification_software, File_command
• Debian package uploads: autorevision 1.20-1, check-all-the-things 2017.01.15 & openchange 1:2.2-6+deb8u1

Issues

• Security migration for check-all-the-things
• GPL violations due to embedded copies of Liberation Fonts in emscripten, freedink, gargoyle-free, gcstar, manaplus, minetest, pcs & zygrib
• Outdated embedded unicode-data copies in boost1.62 & boost1.63
• Crash in liferea
• Regression in coreutils date
• Missing conffile removal in dnssec-trigger & xtightvncviewer
• gitorious redirector usage in fgrun & spectacle
• Annoying issues in hexchat autorejoin, totem & a podcast & gdb messages
• Cosmetic issues in parcimonie menu, gitk colours & aptitude debtags
• Features in duck, apt-file, needrestart, packages.debian.org & aptitude
• An already implemented feature in needrestart

Review

• Spam: report spam in 6 Debian bugs, 132 Debian mails and one LWN comment
• Patches: point out extraneous whitespace changes in a codespell PR
• Debian packages: reviewed and sponsored fsprotect 1.0.7
• Debian wiki: RecentChanges for the month, feedback on InstallDebianWithoutLiveCD article
• Debian screenshots:
  • Approved screenshots of ngraph-gtk lightdm-gtk-greeter-settings kalgebra qtile patat toilet sagemath (2) w3c-markup-validator pentobi pentobi-kde-thumbnailer zygrib (2) sauerbraten kmail aiscm (5) codeblocks lxqt libreoffice-help-es inkscape synaptic thunar gimp midori filezilla libreoffice pluma qupzilla gnome-2048 simplescreenrecorder gerbv keepassx gcstar confclerk onionshare blockout2 subuser dosbox zsnes dosemu lmarbles klickety geda-gattrib katomic kiki-the-nano-bot freedm frogatto (2)
  • Approved removal of screenshots: seaview screenshot contained advertising, mupen64plus/mupen64plus-audio-all were weird screenshots with multiple OS windows overlaid & mkgmap was a screenshot of the OSM wiki instead of the software
  • Rejected screenshots: gccgo looked like a Thai mobile phone screenshot, python-stem was a screenshot for OPENDIME USB, seaview was downloaded from elsewhere and not an authentic screenshot & fbreader contained non-free ebook contents

Administration

• Debian: reboot 1 non-responsive VM, redirect 2 users to support channels, redirect 1 contributor to xkb upstream, redirect 1 potential contributor, redirect 1 bug reporter to mirror team, ping 7 folks about restarting processes with upgraded libs, manually restart the sectracker process due to upgraded libs, restart the package tracker process due to upgraded libs, investigate failures connecting to the XMPP service, investigate /dev/shm issue on abel.d.o, clean up after rename of the fedmsg group.
• Debian mentors: lintian/security updates & reboot
• Debian packages: deploy 2 contributions to the live server
• Debian wiki: unblacklist 1 IP address, whitelist 10 email addresses, disable 18 accounts with bouncing email, update email for 2 accounts with bouncing email, reported 1 Debian member as MIA, redirect 1 user to support channels, add 4 domains to the whitelist.
• Reproducible builds: rescheduled Debian pyxplot:amd64/unstable for themill.
• Openmoko: security updates & reboots.

Debian derivatives

• Send the annual activity ping mail.
• Happy new year messages on IRC, forward to the list.
• Note that SerbianLinux does not provide source packages.
• Expand URL shortener on SerbianLinux page.
• Invite PelicanHPC, Netrunner, DietPi, Hamara Linux (on IRC), BitKey to the census.
• Add research publications link to the census template
• Fix Symbiosis sources.list
• Enquired about SalentOS downtime
• Fixed and removed some 404 BlankOn links (blog, English homepage)
• Fixed changes to AstraLinux sources.list
• Welcome Netrunner to the census

Sponsors I renewed my support of Software Freedom Conservancy. The openchange 1:2.2-6+deb8u1 upload was sponsored by my employer. All other work was done on a volunteer basis.

It is a new month, and time to publish the next chapter in Hacker Noir. This is chapter 2, titled "Development setup phase". I hope you enjoy it. Feedback via email, irc, identi.ca, twitter are welcome. Or come talk to me at FOSDEM if you're there.

I got a new computer and wondered... How can I test it? One of those innocent questions that brings hours and hours of work and questionning...

A new desktop: Intel NUC devices After reading up on Jeff Atwood's blog and especially his article on the scooter computer, I have discovered a whole range of small computers that could answer my need for a faster machine in my office at a low price tag and without taking up too much of my precious desk space. After what now seems like a too short review I ended up buying a new Intel NUC device from NCIX.com, along with 16GB of RAM and an amazing 500GB M.2 hard drive for around 750$. I am very happy with the machine. It's very quiet and takes up zero space on my desk as I was able to screw it to the back of my screen. You can see my review of the hardware compatibility and installation report in the Debian wiki. I wish I had taken more time to review the possible alternatives - for example I found out about the amazing Airtop PC recently and, although that specific brand is a bit too expensive, the space of small computers is far and wide and deserves a more thorough review than just finding the NUC by accident while shopping for laptops on System76.com...

Reviving the Stressant project But this, and Atwood's Is Your Computer Stable? article, got me thinking about how to test new computers. It's one thing to build a machine and fire it up, but how do you know everything is actually really working? It is common practice to do a basic stress test or burn-in when you get a new machine in the industry - how do you proceed with such tests? Back in the days when I was working at Koumbit, I wrote a tool exactly for that purpose called Stressant. Since I am the main author of the project and I didn't see much activity on it since I left, I felt it would be a good idea to bring it under my personal wing again, and I have therefore moved it to my Gitlab where I hope to bring it back to life. Parts of the project's rationale are explained in an "Intent To Package" the "breakin" tool (Debian bug #707178), which, after closer examination, ended up turning into a complete rewrite. The homepage has a bit more information about how the tool works and its objectives, but generally, the idea is to have a live CD or USB stick that you can just plugin into a machine to run a battery of automated tests (memtest86, bonnie++, stress-ng and disk wiping, for example) or allow for interactive rescue missions on broken machines. At Koumbit, we had Debirf-based live images that we could boot off the network fairly easily that we would use for various purposes, although nothing was automated yet. The tool is based on Debian, but since it starts from boot, it should be runnable on any computer. I was able to bring the project back to life, to a certain extent, by switching to vmdebootstrap instead of debirf for builds, but that removed netboot support. Also, I hope that Gitlab could provide with an autobuilder for the images, but unfortunately there's a bug in Docker that makes it impossible to mount loop images in Docker images (which makes it impossible to build Docker in Docker, apparently).

Should I start yet another project? So there's still a lot of work to do in this project to get it off the ground. I am still a bit hesitant in getting into this, however, for a few reasons:

1. It's yet another volunteer job - which I am trying to reduce for health and obvious economic reasons. That's a purely personal reason and there isn't much you can do about it.
2. I am not sure the project is useful. It's one thing to build a tool that can do basic tests on a machine - I can probably just build an live image for myself that will do everything I need - it's another completely different thing to build something that will scale to multiple machines and be useful for more various use cases and users.

(A variation of #1 is how everything and everyone is moving to the cloud. It's become a common argument that you shouldn't run your own metal these days, and we seem to be fighting an uphill economic battle when we run our own datacenters, rack or even physical servers these days. I still think it's essential to have some connexion to metal to be autonomous in our communications, but I'm worried that focusing on such a project is another of my precious dead entreprises... ) Part #2 is obviously where you people come in. Here's a few questions I'd like to have feedback on:

1. (How) do you perform stress-testing of your machines before putting them in production (or when you find issues you suspect to be hardware-related)?
2. Would a tool like breakin or stressant be useful in your environment?
3. Which tools do you use now for such purposes?
4. Would you contribute to such a project? How?
5. Do you think there is room for such a project in the existing ecology of projects) or should I contribute to an existing project?

Any feedback here would be, of course, greatly appreciated.