Solarized A while back I was looking for some information on the web, and happened upon a blog post about the subject. I don't remember what I was looking for, but on the same blog, there was a screen shot of what I then learned was the Solarized theme. This caught my eye that I decided to try it myself ASAP. Up until last year, I've been using for many years the 'black on light yellow' xterm scheme. This is good during the day, but too strong during night, so on some machines I switched to 'white on black', but this was not entirely satisfying. The solarized theme promises to have consistent colours over both light and dark background, which would help to make my setups finally consistent, and extends to a number of programs. Amongst these, there are themes for mutt on both light and dark backgrounds using only 16 colours. This was good, as my current hand-built theme is based on 256 colours, and this doesn't work well in the Linux console. So I tried changing my terminal to the custom colours, played with it for about 10 minutes, then decided that its contrast is too low, bordering on unreadable. I switch to another desktop where I still had open an xterm using white-on-black, and this being at night my eyes immediately go 'no no no too high contrast'. In about ten minutes I got so used to it that the old theme was really really uncomfortable. There was no turning back now Interestingly, the light theme was not that much better than black-on-light-yellow, as that theme is already pretty well behaved. But I still migrated for consistency. Programs/configs Starting from the home page and the internet, I found resources for:

• Vim and Emacs (for which I use the debian package elpa-solarized-theme).
• Midnight Commander, for which I currently use peel's theme, although I'm not happy with it; interestingly, the default theme almost works on 16-custom-colours light terminal scheme, but not quite on the dark one.
• Mutt, which is both in the main combined repository but also on the separate one. I'm not really happy with mutt's theme either, but that seems mostly because I was using a quite different theme before. I'll try to improve what I feel is missing over time.
• dircolors; I found this to be an absolute requirement for good readability of ls --color, as the defaults are too bad
• I also took the opportunity to unify my git diff and colordiff theme, but this was not really something that I found and took 'as-is' from some repository; I basically built my own theme.

16 vs 256 colours The solarized theme/configuration can be done in two ways:

• by changing the Xresources/terminal 16 basic colours to custom RGB values, or:
• by using approximations from the fixed 256 colours available in the xterm-256color terminfo

Upstream recommends the custom ones, as they are precisely tuned, instead of using the approximated ones; honestly I don't know if they would make a difference. It's too bad upstream went silent a few years back, as technically it's possible to override also colours above 16 in the 256-colour palette, but in any case, each of the two options has its own cons:

• using customised 16-colour means that all terminal programs get the new colours scheme, even if they were designed (colour-wise) based on the standard values; this makes some things pretty unreadable (hence the need to fix dircolors), but at least somewhat consistent.
• using 256-colour palette, unchanged programs stay the same, but now they look very different than the programs that were updated to solarized; note thought I haven't tested this, but that's how I understand things would be.

So either way it's not perfect. Desktop-wide consistency Also not perfect is that for proper consistent look, many more programs would have to be changed; but I don't see that happening in today's world. I've seen for example 3 or 4 Midnight Commander themes, but none of them were actually in the spirit of solarized, even though they were tweaked for solarized. Even between vim and emacs, which both have one canonical solarized theme, the look is close but not really the same (looking at the markdown source for this blog post: URLs, headers and spelling mistakes are all different), but this might be due not necessarily the theme itself. So no global theme consistency (I'd wish), but still, I find this much better on the eyes and not lower on readability after getting used to it. Thanks Ethan!

There is/was a somewhat controversial book by Osho which I read long back Sambhog se Samadhi Ki Aur or the English version From Sex to Superconsciousness While I can t say I understand or understood it all, read it about a decade back, the main point shared in the book was that if you are able to achieve bliss/orgasm during sex, you might be able to have a glimpse of super-consciousness. I had to share the above context as I had gone to a meetup couple of weeks back had gone to a meetup where a friend, Dr. Swati Shome is attempting to write an educational book for teenagers to talk about sex. I did help her a bit in the past I tried to share some of the concerns I had as my generation didn t have any guidance from parents or teachers. Most of us were left to our own devices which is similar to today s children as well with the exception that they have the web. You look at both the books, both written in Pune (my home-town) and both talking about the same subject but from so different a view-point. If you see the comments on the meetup page, it really pains to see people s concerns. I don t know if there is any solution to the widespread ignorance, myth-making etc. and hence felt a bit sad . Sharing a small clip I had seen few months back. Just to give a bit of context, the law as has been shared as passed in 2015 has happened after the 2012 Delhi Gang Rape. A part of it is also that the Indian society still frowns upon live-in relationships so in part it may also be a push-back from the conservatives. After all the BJP, a right of the center party has been in power for 2.5 years now so it s possible that they were part of it. As I don t have enough knowledge of what the actual case was, who were the litigants and the defendants, the lawyers and the judge involved, I cannot further speculate. If somebody has more info. or link please pass it on. It would be interesting to know if it was a single bench ruling or 3-5 judge bench. The yin-yan symbol I had shared becomes a bit more apt as in quite a few cultures, including Indian and Japanese, the two are seen as parts of the same coin. One life-giving, the other life-taking or not even taking but converting into something else. That came few days later when I was reading an article about sleep. The purpose of sleep, is to forget . It was slightly strange and yet interesting article. What disturbed me though, was the bit about the mouse being killed and his brain being sliced. I tried to find many a justification for it, but none I could have peace with. And the crux of that is because the being, the creature s wilful consent hasn t been taken. In nature s eyes humans and mice are one and the same. We don t get any special passes due to the fact that we are human. A natural disaster doesn t care whether you are small or big, fat or strong, mouse or wo/man, coward or brave. It s sheer luck and after disaster preparedness that people and animals get saved or not. I thought quite a bit that instead of animals being used for scientific experiments, why don t we use actual humans. While I m sure PETA supporters probably may have spear-headed this idea for a long time, but it doesn t mean I can t come to this realization by myself. After all, it s not about pandering to a group but rather what I think is right. Passing the baton to humans does have its own knotty problems though. For any such kind of endeavour, people s participation and wilful consent would be needed. While humans can and do give wilful consent, it is difficult problem as you don t know the situation in which that consent has been taken. We all know about Organ trafficking . Many people especially from lower economic background may be enticed and cheated with the whole economics for science. In most Indian middle and higher-middle classes religion plays a part even though with death the body is cremated and is supposed to scatter among the Pancha Mahaboota, the five elements. I, for one have no hang-ups if some scientist were to slice my brain to find something, provided I m dead or for that matter any part of the body. If more people thought like that, probably we wouldn t have to specially grow and then kill lab mice and guinea-pigs to test out theories. Possibly medical innovations would probably be a lot faster than now. Ironically, most medical innovations have happened during wars and continues to do so till date. Comments, ideas, suggestions and criticisms all are welcome.
Filed under: Miscellenous Tagged: #Death, #Innocence, #Medical Innovatiion, #Medicine, #Murder, #PETA, #Sex, #sleep, #war, education, exploitation, nature

I was an happy user of rxvt-unicode until I got a laptop with an HiDPI display. Switching from a LoDPI to a HiDPI screen and back was a pain: I had to manually adjust the font size on all terminals or restart them. VTE is a library to build a terminal emulator using the GTK+ toolkit, which handles DPI changes. It is used by many terminal emulators, like GNOME Terminal, evilvte, sakura, termit and ROXTerm. The library is quite straightforward and writing a terminal doesn t take much time if you don t need many features. Let s see how to write a simple one.

A simple terminal Let s start small with a terminal with the default settings. We ll write that in C. Another supported option is Vala.

#include <vte/vte.h>
int
main(int argc, char *argv[])
 
    GtkWidget *window, *terminal;
    /* Initialise GTK, the window and the terminal */
    gtk_init(&argc, &argv);
    terminal = vte_terminal_new();
    window = gtk_window_new(GTK_WINDOW_TOPLEVEL);
    gtk_window_set_title(GTK_WINDOW(window), "myterm");
    /* Start a new shell */
    gchar **envp = g_get_environ();
    gchar **command = (gchar *[]) g_strdup(g_environ_getenv(envp, "SHELL")), NULL  ;
    g_strfreev(envp);
    vte_terminal_spawn_sync(VTE_TERMINAL(terminal),
        VTE_PTY_DEFAULT,
        NULL,       /* working directory  */
        command,    /* command */
        NULL,       /* environment */
        0,          /* spawn flags */
        NULL, NULL, /* child setup */
        NULL,       /* child pid */
        NULL, NULL);
    /* Connect some signals */
    g_signal_connect(window, "delete-event", gtk_main_quit, NULL);
    g_signal_connect(terminal, "child-exited", gtk_main_quit, NULL);
    /* Put widgets together and run the main loop */
    gtk_container_add(GTK_CONTAINER(window), terminal);
    gtk_widget_show_all(window);
    gtk_main();
 

You can compile it with the following command:

gcc -O2 -Wall $(pkg-config --cflags --libs vte-2.91) term.c -o term

And run it with ./term:

More features From here, you can have a look at the documentation to alter behavior or add more features. Here are three examples.

Colors You can define the 16 basic colors with the following code:

#define CLR_R(x)   (((x) & 0xff0000) >> 16)
#define CLR_G(x)   (((x) & 0x00ff00) >>  8)
#define CLR_B(x)   (((x) & 0x0000ff) >>  0)
#define CLR_16(x)  ((double)(x) / 0xff)
#define CLR_GDK(x) (const GdkRGBA)  .red = CLR_16(CLR_R(x)), \
                                    .green = CLR_16(CLR_G(x)), \
                                    .blue = CLR_16(CLR_B(x)), \
                                    .alpha = 0  
vte_terminal_set_colors(VTE_TERMINAL(terminal),
    &CLR_GDK(0xffffff),
    &(GdkRGBA)  .alpha = 0.85  ,
    (const GdkRGBA[]) 
        CLR_GDK(0x111111),
        CLR_GDK(0xd36265),
        CLR_GDK(0xaece91),
        CLR_GDK(0xe7e18c),
        CLR_GDK(0x5297cf),
        CLR_GDK(0x963c59),
        CLR_GDK(0x5E7175),
        CLR_GDK(0xbebebe),
        CLR_GDK(0x666666),
        CLR_GDK(0xef8171),
        CLR_GDK(0xcfefb3),
        CLR_GDK(0xfff796),
        CLR_GDK(0x74b8ef),
        CLR_GDK(0xb85e7b),
        CLR_GDK(0xA3BABF),
        CLR_GDK(0xffffff)
 , 16);

While you can t see it on the screenshot¹, this also enables background transparency.

Miscellaneous settings VTE comes with many settings to change the behavior of the terminal. Consider the following code:

vte_terminal_set_scrollback_lines(VTE_TERMINAL(terminal), 0);
vte_terminal_set_scroll_on_output(VTE_TERMINAL(terminal), FALSE);
vte_terminal_set_scroll_on_keystroke(VTE_TERMINAL(terminal), TRUE);
vte_terminal_set_rewrap_on_resize(VTE_TERMINAL(terminal), TRUE);
vte_terminal_set_mouse_autohide(VTE_TERMINAL(terminal), TRUE);

This will:

• disable the scrollback buffer,
• not scroll to the bottom on new output,
• scroll to the bottom on keystroke,
• rewrap content when terminal size change, and
• hide the mouse cursor when typing.

Update the window title An application can change the window title using XTerm control sequences (for example, with printf "\e]2;$ title \a"). If you want the actual window title to reflect this, you need to define this function:

static gboolean
on_title_changed(GtkWidget *terminal, gpointer user_data)
 
    GtkWindow *window = user_data;
    gtk_window_set_title(window,
        vte_terminal_get_window_title(VTE_TERMINAL(terminal))?:"Terminal");
    return TRUE;
 

Then, connect it to the appropriate signal, in main():

g_signal_connect(terminal, "window-title-changed", 
    G_CALLBACK(on_title_changed), GTK_WINDOW(window));

Final words I don t need much more as I am using tmux inside each terminal. In my own copy, I have also added the ability to complete a word using ones from the current window or other windows (also known as dynamic abbrev expansion). This requires to implement a terminal daemon to handle all terminal windows with one process, similar to urxvtcd. While writing a terminal from scratch ² suits my need, it may not be worth it. evilvte is quite customizable and can be lightweight. Consider it as a first alternative. Honestly, I don t remember why I didn t pick it. UPDATED: evilvte has not seen an update since 2014. Its GTK+3 support is buggy. It doesn t support the latest versions of the VTE library. Therefore, it s not a good idea to use it. You should also note that the primary goal of VTE is to be a library to support GNOME Terminal. Notably, if a feature is not needed for GNOME Terminal, it won t be added to VTE. If it already exists, it will likely to be deprecated and removed.

---

1. Transparency is handled by the composite manager (Compton, in my case).
2. For some definition of scratch since the hard work is handled by VTE.

The Gitlab.com database incident of 2017/01/31 and the resulting data loss reminded everyone (at least for the next days) how it s easy to lose data, even when you think all your systems are safe. Being really interested by the process of backing up data, I read with interest the report (kudos to the Gitlab company for being so transparent about it) and I was soooo excited to find the following sentence:

Regular backups seem to also only be taken once per 24 hours, though team-member-1 has not yet been able to figure out where they are stored. According to team-member-2 these don t appear to be working, producing files only a few bytes in size.

Whoa, guys! I m so sorry for you about the data loss, but from my point of view I was so excited to find a big FOSS company publicly admitting and communicating about a perfect use case for the Backup Checker project, a Free Software I ve been writing these last years. Data loss: nobody cares before, everybody cries after Usually people don t care about the backups. It s a serious business for web hosters and the backup team from big companies but otherwise and in other places, nobody cares. Usually everybody agrees about how backups are important but few people make them or install an automatized system to create backups and the day before, nobody verifies they are usable. The reason is obvious: it s totally boring, and in some cases e.g for large archives, difficult. Because verifying backups is boring for humans, I launched the Backup Checker project in order to automatize this task. Backup Checker offers a wide range of features, checking lots of different archives (tar. gz,bz2,xz , zip, tree of files and offer lots of different tests (hash sum, size equal, smaller/greater than , unix rights, ,). Have a look at the official documentation for a exhaustive list of features and possible tests. Automatize the controls of your backups with Backup Checker Checking your backups means to describe in a configuration file how a backup should be, e.g a gzipped database dump. You usually know about what size the archive is going to be, what the owner and the group owner should be. Even easier, with Backup Checker you can generate this list of criterias from an actual archive, and remove uneeded criterias to create a template you can re-use for different kind of archives. Ok, 2 minutes of your time for a real word example, I use an existing database sql dump in an tar.gz archive to automatically create the list describing this backup:

$ backupchecker -G database-dump.tar.gz
$ cat database-dump.list
[archive]
mtime  1486480274.2923253
[files]
database.sql  =7854803 uid 1000 gid 1000 owner chaica group chaica mode 644 type f mtime 1486480253.0

Now, just remove parameters too precise from this list to get a backup template. Here is a possible result:

[files]
database.sql  >6m uid 1000 gid 1000 mode 644 type f

We define here a template for the archive, meaning that the database.sql file in the archive should have a size greater than 6 megabytes, be owned by the user with the uid of 1000 and the group with a gid of 1000, this file should have the mode 644 and be a regular file. In order to use a template instead of the complete list, you also need to remove the sha512 from the .conf file. Pretty easy hmm? Ok, just for fun, lets replicate the part of the Gitlab.com database incident mentioned above and write an archive with an empty sql dump inside an archive:

$ touch /tmp/database.sql && \
tar zcvf /tmp/database-dump.tar.gz /tmp/database.sql && \
cp /tmp/database-dump.tar.gz .

Now we launch Backup Checker with the previously created template. If you didn t change the name of database-dump.list file, the command should only be:

$ backupchecker -C database-dump.conf
$ cat a.out 
WARNING:root:1 file smaller than expected while checking /tmp/article-backup-checker/database-dump.tar.gz: 
WARNING:root:database.sql size is 0. Should have been bigger than 6291456.

The automatized controls of Backup Checker trigger a warning in the log file. The empty sql dump has been identified inside the archive. A step further As you could read in this article, verifying some of your backups is not a time consuming task, given the fact you have a FOSS project dedicated to this task, with an easy way to realize a template of your backups and to use it. This article provided a really simple example of such a use case, the Backup Checker has lots of features to offer when verifying your backups. Read the official documentation for more complete descriptions of the available possibilities. Data loss, especially for projets storing user data is always a terrible event in the life of an organization. Lets try to learn from mistakes which could happen to anyone and build better backup systems. More information about the Backup Checker project

• The Backup Checker project on Github
• The official documentation of the Backup Checker project on Readthedocs.

When WordPress originally announced their latest security update, there were three security fixes. While all security updates can be serious, they didn t seem too bad. Shortly after, they updated their announcement with a fourth and more serious security problem. I have looked after the Debian WordPress package for a while. This is the first time I have heard people actually having their sites hacked almost as soon as this vulnerability was announced. If you are running WordPress 4.7 or 4.7.1, your website is vulnerable and there are bots out there looking for it. You should immediately upgrade to 4.7.2 (or, if there is a later 4.7.x version to that). There is now updated Debian wordpress version 4.7.2 packages for unstable, testing and stable backports. For stable, you are on a patched version 4.1 which doesn t have this specific vulnerability (it was introduced in 4.7) but you should be using 4.1+dfsg-1+deb8u12 which has the fixes found in 4.7.1 ported back to 4.1 code.

Last weekend, as a result of my addiction to buying random microcontrollers to play with, I received some Maple Minis. I bought the Baite clone direct from AliExpress - so just under 3 each including delivery. Not bad for something that s USB capable, is based on an ARM and has plenty of IO pins. I m not entirely sure what my plan is for the devices, but as a first step I thought I d look at getting GnuK up and running on it. Only to discover that chopstx already has support for the Maple Mini and it was just a matter of doing a ./configure --vidpid=234b:0000 --target=MAPLE_MINI --enable-factory-reset ; make. I d hoped to install via the DFU bootloader already on the Mini but ended up making it unhappy so used SWD by following the same steps with OpenOCD as for the FST-01/BusPirate. (SWCLK is D21 and SWDIO is D22 on the Mini). Reset after flashing and the device is detected just fine: And GPG is happy: While GnuK isn t the fastest OpenPGP smart card implementation this certainly seems to be one of the cheapest ways to get it up and running. (Plus the fact that chopstx already runs on the Mini provides me with a useful basis for other experimentation.)

I m investigating the creation of VM images for different virtualisation solutions. Among the target platforms is a destop as a service platform based on an OpenStack public cloud. We ve been working with bootstrap-vz for creating VMs for Vagrant+VirtualBox so I wanted to test its use for OpenStack. There are already pre-made images available, including official Debian ones, but I like to be able to re-create things instead of depending on some external magic (which also means to be able to optimize, customize and avoid potential MitM, of course). It appears that bootstrap-vz can be used with cloud-init provided that some bits of config are specified. In particular the cloud_init plugin of bootstrap-vz requires a metadata_source set to NoCloud, ConfigDrive, OpenStack, Ec2 . Note we explicitely spell it OpenStack and not Openstack as was mistakenly done in the default Debian cloud images (see https://bugs.debian.org/854482). The following snippet of manifest provides the necessary bits :

---
name: debian- system.release - system.architecture - %Y %m %d 
provider:
  name: kvm
  virtio_modules:
  - virtio_pci
  - virtio_blk
bootstrapper:
  workspace: /target
  # create or reuse a tarball of packages
  tarball: true
system:
  release: jessie
  architecture: amd64
  bootloader: grub
  charmap: UTF-8
  locale: en_US
  timezone: UTC
volume:
  backing: raw
  partitions:
    #type: gpt
    type: msdos
    root:
      filesystem: ext4
      size: 4GiB
    swap:
      size: 512MiB
packages:
  # change if another mirror is closer
  mirror: http://ftp.fr.debian.org/debian/
plugins:
  root_password:
    password: whatever
  cloud_init:
    username: debian
    # Note we explicitely spell it 'OpenStack' and not 'Openstack' as done in the default Debian cloud images (see https://bugs.debian.org/854482)
    metadata_sources: NoCloud, ConfigDrive, OpenStack, Ec2
  # admin_user:
  #   username: Administrator
  #   password: Whatever
  minimize_size:
    # reduce the size by around 250 Mb
    zerofree: true

I ve tested this with the bootstrap-vz version in stretch/testing (0.9.10+20170110git-1) for creating jessie/stable image, which were booted on the OVH OpenStack public cloud. YMMV. Hope this helps

Recently some of my coworkers and I experienced an issue with using the upper left touchpad button on our Dell Latitude E7470 and similar laptops (E5xxx from the current generation). Some time in January we could no longer hold down this button and select text with the touchpad. Using the left button below the touchpad still worked. This hit my coworker running Fedora and myself running Debian/stretch. So I first thought that it's likely a libinput issue (same version in Debian/stretch and Fedora and I recently pulled that in as an update), somehow blacklisting the upper left key because it's connected to the trackpoint. So I filled #99594 upstream. While this was not very helpful at first, and according to Peter very unlikely to be related to libinput, another coworker using Debian/jessie found this issue to hit him when he upgraded the backports kernel in use from 4.8 to 4.9. That finally led to the conclusion that it's a bug in the Linux alps driver, which is already fixed in 4.10 and probably 4.9.6. Until the Debian kernel team pulls in a fresh 4.9 point release I'm using 4.10-rc6 from experimental. For Debian/jessie + backports kernel user it might be more convenient to just stay at 4.8 in case this issue annoys you. Kudos to Peter, Benjamin, TW and WW for the help in locating the origin of this issue! Lessons learned:

• I should've started with the painful downgrade of xorg and libinput via snapshot.d.o before opening the bugreport.
• A lot more of the touchpad related hardware support is nowadays in the kernel and not in the xorg layer. Either that was just my personal historic misunderstanding, or it was different 10 years ago.
• There is an interesting set of slides from Benjamin related to debuging input device issues.

Last week, while working on CodeWorld, via a sequence of yak shavings, I ended up creating a nicely small library that provides Control.Applicative.Succs, a new applicative functor. And because I am trying to keep my Haskell karma good¹, I wanted to actually prove that my code fulfills the Applicative and Monad laws. This led me to inserted writing long comments into my code, filled with lines like this:

The second Applicative law:
  pure (.) <*> Succs u us <*> Succs v vs <*> Succs w ws
= Succs (.) [] <*> Succs u us <*> Succs v vs <*> Succs w ws
= Succs (u .) (map (.) us) <*> Succs v vs <*> Succs w ws
= Succs (u . v) (map ($v) (map (.) us) ++ map (u .) vs) <*> Succs w ws
= Succs (u . v) (map (($v).(.)) us ++ map (u .) vs) <*> Succs w ws
= Succs ((u . v) w) (map ($w) (map (($v).(.)) us ++ map (u .) vs) ++ map (u.v) ws)
= Succs ((u . v) w) (map (($w).($v).(.)) us ++ map (($w).(u.)) vs ++ map (u.v) ws)
= Succs (u (v w)) (map (\u -> u (v w)) us ++ map (\v -> u (v w)) vs ++ map (\w -> u (v w)) ws)
= Succs (u (v w)) (map ($(v w)) us ++ map u (map ($w) vs ++ map v ws))
= Succs u us <*> Succs (v w) (map ($w) vs ++ map v ws)
= Succs u us <*> (Succs v vs <*> Succs w ws)

Honk if you have done something like this before! I proved all the laws, but I was very unhappy. I have a PhD on something about Haskell and theorem proving. I have worked with Isabelle, Agda and Coq. Both Haskell and theorem proving is decades old. And yet, I sit here, and tediously write manual proofs by hand. Is this really the best we can do? Of course I could have taken my code, rewritten it in, say, Agda, and proved it correct there. But (right now) I don t care about Agda code. I care about my Haskell code! I don t want to write it twice, worry about copying mistakes and mismatchs in semantics, and have external proofs to maintain. Instead, I want to prove where I code, and have the proofs checked together with my code! Then it dawned to me that this is, to some extent, possible. The Haskell compiler comes with a sophisticated program transformation machinery, which is meant to simplify and optimize code. But it can also be used to prove Haskell expressions to be equivalent! The idea is simple: Take two expressions, run both through the compiler s simplifier, and check if the results are the same. If they are, then the expressions are, as far as the compiler is concerned, equivalent. A handful of hours later, I was able to write proof tasks like

app_law_2 = (\ a b (c::Succs a) -> pure (.) <*> a <*> b <*> c)
        === (\ a b c -> a <*> (b <*> c))

and others into my source file, and the compiler would tell me happily:

[1 of 1] Compiling Successors       ( Successors.hs, Successors.o )
GHC.Proof: Proving getCurrent_proof1  
GHC.Proof: Proving getCurrent_proof2  
GHC.Proof: Proving getCurrent_proof3  
GHC.Proof: Proving ap_star  
GHC.Proof: Proving getSuccs_proof1  
GHC.Proof: Proving getSuccs_proof2  
GHC.Proof: Proving getSuccs_proof3  
GHC.Proof: Proving app_law_1  
GHC.Proof: Proving app_law_2  
GHC.Proof: Proving app_law_3  
GHC.Proof: Proving app_law_4  
GHC.Proof: Proving monad_law_1  
GHC.Proof: Proving monad_law_2  
GHC.Proof: Proving monad_law_3  
GHC.Proof: Proving return_pure  
GHC.Proof proved 15 equalities

This is how I want to prove stuff about my code! Do you also want to prove stuff about your code? I packaged this up as a GHC plugin in the Haskell library ghc-proofs (not yet on Hackage). The README of the repository has a bit more detail on how to use this plugin, how it works, what its limitations are and where this is heading. This is still only a small step, but finally there is a step towards low threshold program equivalence proofs in Haskell.

... but that doesn't mean the work is over. One big job that needs to happen after the conference is to review and release the video recordings that were made. With several hundreds of videos to be checked and only a handful of people with the ability to do so, review was a massive job that for the past three editions took several months; e.g., in 2016 the last video work was done in July, when the preparation of the 2017 edition had already started. Obviously this is suboptimal, and therefore another solution was required. After working on it for quite a while (in my spare time), I came up with SReview, a video review and transcoding system written in Perl. An obvious question that could be asked is why I wrote yet another system, though, and did not use something that already existed. The short answer to that is "because what's there did not exactly do what I wanted to". The somewhat longer answer also involves the fact that I felt like writing something from scratch. The full story, however, is this: there isn't very much out there, and what does exist is flawed in some ways. I am aware of three other review systems that are or were used by other conferences:

1. A bunch of shell scripts that were written by the DebConf video team and hooked into the penta database. Nobody but DebConf ever used it. It allowed review via an NFS share and a webinterface, and required people to watch .dv files directly from the filesystem in a media player. For this and other reasons, it could only ever be used from the conference itself. If nothing else, that final limitation made it impossible for FOSDEM to use it, but even if that wasn't the case it was still too basic to ever be useful for a conference the size of FOSDEM.
2. A review system used by the CCC "voc" team. I've never actually seen it in use, but I've heard people describe it. It involves a complicated setup of Samba servers, short MPEG transport stream segments, a FUSE filesystem, and kdenlive, which took someone several days to set up as an experiment back at DebConf15. Critically, important parts of it are also not licensed as free software, which to me rules it out for a tool in support of FOSDEM. Even if that wasn't the case, however, I'm still not sure it would be ideal; this system requires intimate knowledge of how it works from its user, which makes it harder for us to crowdsource the review to the speaker, as I had planned to.
3. Veyepar. This one gets many things right, and we used it for video review at DebConf from DebConf14 onwards, as well as FOSDEM 2014 (but not 2015 or 2016). Unfortunately, it also gets many things wrong. Most of these can be traced back to the fact that Carl, as he freely admits, is not a programmer; he's more of a sysadmin type who also manages to cobble together a few scripts now and then. Some of the things it gets wrong are minor issues that would theoretically be fixable with a minimal amount of effort; others would be more involved. It is also severely underdocumented, and so as a result it is rather tedious for someone not very familiar with the system to be able to use it. On a more personal note, veyepar is also written in the wrong language, so while I might have spent some time improving it, I ended up starting from scratch.

Something all these systems have in common is that they try to avoid postprocessing as much as possible. This only makes sense; if you have to deal with loads and loads of video recordings, having to do too much postprocessing only ensures that it won't get done... Despite the issues that I have with it, I still think that veyepar is a great system, and am not ashamed to say that SReview borrows many ideas and concepts from it. However, it does things differently in some areas, too:

1. A major focus has been on making the review form be as easy to use as possible. While there is still room for improvement (and help would certainly be welcome in that area from someone with more experience in UI design than me), I think the SReview review form is much easier to use than the veyepar one (which has so many options that it's pretty hard to understand sometimes).
2. SReview assumes that as soon as there are recordings in a given room sufficient to fill all the time that a particular event in that room was scheduled for, the whole event is available. It will then generate a first rough cut, and send a notification to the speaker in question, as well as the people who organized the devroom. The reviewer will then almost certainly be required to request a second (and possibly third or fourth) cut, but I think the advantage of that is that it makes the review workflow be more intuitive and easier to understand.
3. Where veyepar requires one or more instances of per-state scripts to be running (which will then each be polling the database and just start a transcode or cut or whatever script as needed), SReview uses a single "dispatch" script, which needs to be run once for the whole system (if using an external scheduler) or once per core that may be used (if not using an external scheduler), and which does all the database polling required. The use of an external scheduler seemed more appropriate, given that things like gridengine exist; gridengine is a job scheduler which allows one to submit a job to be ran on any node in a cluster, along with the resources that this particular job requires, and which will then either find an appropriate node to run the job on, or will put the job in a "pending" state until the required resources can be found. This allows me to more easily add extra encoding capacity when required, and allows me to also do things like allocate less resources to a particular part of the whole system, even while jobs are already running, without necessarily needing to abort jobs that might be using those resources.

The system seems to be working fine, although there's certainly still room for improvement. I'm thinking of using it for DebConf17 too, and will therefore probably work on improving it during DebCamp. Additionally, the experience of using it for FOSDEM 2017 has given me ideas of where to improve it further, so it can be used more easily by different parties, too. Some of these have been filed as issues against a "1.0" milestone on github, but others are only newly formed in my gray matter and will need some thinking through before they can be properly implemented. Certainly, it looks like this will be something that'll give me quite some fun developing further. In the mean time, if you're interested in the state of a particular video of FOSDEM 2017, have a look at the video overview page, which lists all talks along with their review/transcode status. Also, if you were a speaker or devroom organizer at FOSDEM 2017, please check your mailbox and review your talk! With your help, we should hopefully be able to release all our videos by the end of the week. Update (2017-02-06 17:18): clarified my position on the qualities of some of the other systems after feedback from people who were a bit disappointed by my description of them... and which was fair enough. Apologies. Update (2017-02-08 16:26): Fixes to the c3voc stuff after feedback from them.

I ve got some little scripts and a template here to automatically create Vagrant boxes from cutting edge Debian testing daily snapshots (netinstall ISO image) using HashiCorp s Packer. To create Vagrant boxes with these, you first need a running binary of Packer. There is a Debian package available if that s also your working environment, but Packer is going to be introduced into the stable branch with the upcoming Stretch release itself. However, Ubuntu already has it, and some other derivatives, too. And there are prebuild binaries available from the developer s site which run fine out-of-the-box (you just have to put the single binary somewhere into you $PATH, or expand that to find it). The JSON template should run with any Packer which is available for any of the different systems. Vagrant itself isn t needed to build the box with Packer, but Virtualbox is of course needed to pre bake the machine image within a virtual machine. In Debian the base binaries of Virtualbox are in the contrib archive section, so that source might be added to /etc/apt/sources.list, if haven t already. The scripts have been tested to run with 5.1.10, and I haven t seen that any late versions are demanded in particular, but of course heavily outdated versions might not work properly. Packer installs the guest additions ISO file for Virtualbox into the virtual machine (and the shipped provisioning script then builds them inside). For that, the Debian package which ships that (which is in non-free) is recognized if it is installed, and then could be used by Packer. When the ISO isn t available in the places which Packer checks on the host the builder then automatically downloads the corresponding ISO from http://download.virtualbox.org/virtualbox. When the tarball with the scripts is unpacked, just do make create and the process should run through, if Packer and Virtualbox are available. If your environment doesn t have GNU Make nor wget you might want to copy the relevant lines from the Makefile and run it manually. But if it does, just do it like this:

/tmp/debian-testing-vagrantbox$ make create
virtualbox-iso output will be in this color.
==> virtualbox-iso: Downloading or copying Guest additions
    virtualbox-iso: Downloading or copying: file:///usr/share/virtualbox/VBoxGuestAdditions.iso
==> virtualbox-iso: Downloading or copying ISO
    virtualbox-iso: Downloading or copying: http://cdimage.debian.org/cdimage/daily-builds/daily/arch-latest/amd64/iso-cd/debian-testing-amd64-netinst.iso
    virtualbox-iso: Download progress: 10%
 ... 
    virtualbox-iso: Download progress: 96%
==> virtualbox-iso: Starting HTTP server on port 8219
==> virtualbox-iso: Creating virtual machine...
==> virtualbox-iso: Creating hard drive...
==> virtualbox-iso: Creating forwarded port mapping for communicator (SSH, WinRM, etc) (host port 2885)
==> virtualbox-iso: Starting the virtual machine...
==> virtualbox-iso: Waiting 10s for boot...
==> virtualbox-iso: Typing the boot command...
==> virtualbox-iso: Waiting for SSH to become available...

The Virtualbox window then pops up and the build process continues within the virtual machine for a while. You might want to file a Github issue when there s a problem on your machine, please! (please include the tail of your packer.log) The Packer template (debian-testing-vagrant.json) is described in the documentation of the virtualbox-iso builder. A preseeding script for the Debian Installer (preseed.cfg) is also included which gets injected into the virtual build environment during the build process. The creation progress of the Debian base installation could be easily monitored since the Virtualbox window is fully shown during the Packer run (if you loose your mouse pointer by clicking inside that window, do <Right>+<Control> to escape). For good performance, a fast internet connection is needed since a whole base system must be downloaded if that s available the whole automated process very only takes a couple of minutes to complete on a non-SSD machine. When Packer has finished and a fresh box is created (the size is about 690 MB), it then could be used with Vagrant. Just add the new box with:

/tmp/debian-testing-vagrantbox$ vagrant box add stretch-preview debian-testing-vagrant.box
==> box: Box file was not detected as metadata. Adding it directly...
==> box: Adding box 'stretch-preview' (v0) for provider: 
    box: Unpacking necessary files from: file:///tmp/debian-testing-vagrantbox/debian-testing-vagrant.box
==> box: Successfully added box 'stretch-preview' (v0) for 'virtualbox'!

It then could be initialized within a random working directory with:

/tmp/myproject$ vagrant init stretch-preview
A  Vagrantfile  has been placed in this directory. You are now
ready to  vagrant up  your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
 vagrantup.com  for more information on using Vagrant.

After that, you could launch the virtual box with:

/tmp/myproject$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'stretch-preview'...
==> default: Matching MAC address for NAT networking...
==> default: Setting the name of the VM: myproject_default_1486321215067_75270
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: 
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default: 
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Mounting shared folders...
    default: /vagrant => /tmp/myproject
/tmp/myproject$

Then you can SSH into it by doing (touch is used here only to point to the shared folder):

/tmp/myproject$ touch hello!
/tmp/myproject$ vagrant ssh
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
/usr/bin/xauth:  file /home/vagrant/.Xauthority does not exist
vagrant@packer-virtualbox-iso-1486319595:~$ $ cat /etc/debian_version
9.0
vagrant@packer-virtualbox-iso-1486319595:~$ ls /vagrant/
hello!  Vagrantfile
vagrant@packer-virtualbox-iso-1486319595:~$ exit
logout
Connection to 127.0.0.1 closed.
/tmp/myproject$ vagrant halt
==> default: Attempting graceful shutdown of VM...
/tmp/myproject$

If you haven t worked with Vagrant before, maybe this is appealing. The user experience is somehwhat different from working with a chroot. Packer makes it very convenient to keep freshly created boxes for it coming. Have fun!

Warning: Spoilers below. Rant below. Much angry, MANY ALL-CAPS. You've been warned! So, today we went to see "La La Land", because I've heard good things about it, and because I do enjoy good musicals. And because of this, I wrote this post, instead of what I originally had in mind (related to kernel configuration). Was it a good movie? Definitely yes. Was it a good musical? So and so. Did I like the ending? HELL NO, over and over NO. The movie itself was much better than I expected. I don't read plot details in advance nor real reviews, so I expected more of a musical, and less of a good plot. But the movie had a very good plot. Two young people, striving to fulfil their artistic dreams, fall in love, and they fight through-sometimes helping, sometimes hindering each other until, finally, each gets their own breakthrough, etc. The choice of actress was spot on halfway through the movie, I was thinking that I can't imagine the same plot played by a different actress. Of course many other actresses could have played the part, but Emma Stone played so well, I have trouble seeing the same character with the same always half-happy, half-sad attitude. The choice of actor was I think OK at first I was in doubt, but he played also well. Or maybe it was just that I couldn't identify with him at first. Not that I identify well with artists in general The dance scenes were OK, and the singing good, but as I said, the musical part was secondary to the actual struggles of the characters. The movie itself was, technically, very well done; a lot of filming was in bars/clubs/locations with difficult lighting, and the shooting was very good. They also had a scene on a pier, looking towards the ocean and the setting sun, and the characters walking towards the beach so heavily back-lighted, and I kept thinking "If I get only one shot this perfectly exposed and colour correct(ed), I'm happy". So high notes here. Back to the plot. The story of how she and him fought their own struggles was very nicely told. Tick-tack, up and down (hope and rejection), leaning on the other to get morale back, is a captivating story. The cliff-hanger at the pre-end with her career, the going back home, the last minute save, all very well told. So at this stage, I would have given the movie a 9/10. And I was happy. Then we have the usual "one character has to go away to a far away country for a long time", except in this case it was just 4 months. And they have the usual discussion "what do we do with our relation, where do we take it", and she says "I will always love you", to which he replies "And I will too" (or equivalent). In my mind, this means they'll have to survive during the break, they'll have to also survive through his touring months/years, but in the end love will be stronger. Because this is what the movie told us until now, that she made it because of him, and he made it because of her. Neither of them would have been this strong without the other (he wouldn't have picked up the invitation from his old pal, she wouldn't have gone to the final audition request nor write the play which got her the audition/recognition). Estimated movie ending: awesome. And then something happens. The timeline jumps 5 years in the future (as expected), and she is famous, married (WITH SOMEONE ELSE) and happy mother of a 3-year old. Through fate, she and her husband enter the club of Sebastian (as he also fulfilled his dream), she and Sebastian see each other, he plays their song, during which we're served a re-run of the movie but in stupid "everything goes well" style (all bad events eliminated), in which it is she and Sebastian who enter the club (which belongs now to somebody else), and then we're back in real time, song ends, she and her husband leave, but before that she and Sebastian exchange one last smile, THE END. And I'm sitting there, not believing my eyes. WHAT THE? So I get home, not write this post for four hours to calm down, but I can't. Because this doesn't make sense. AT ALL. What does the internet say? Quoting from this CNN article, written exactly today. The director says:

"That ending was there from the get-go," [director Damien Chazelle] told CNN in a recent interview. "I think I just have a thing about love stories where the lovers don't wind up together at the end; I find it very romantic."

Huh, excuse me?

"I think there's a reason why most of the greatest love stories in history don't end with happily ever after," Chazelle said. "To me, if you're telling a story about love, love has to be bigger than the characters." Chazelle sees Mia and Sebastian's love as a "third character" and something that "lives on." "[The ending gives] you that sense that even if the relationship itself might be over in practical terms, the love is not over," he said. "The love lasts, and I think that's just a beautiful kind of thing."

OH FOR THE LOVE OF. This is a wishy-washy explanation that tries to approach the thing from the artistic side. No, this is bullshit, because of multiple things. Let me try to roll back and explain what I think was the intention.

1. An earlier fight between Mia and Sebastian points to the fact that they're both very dedicated to their careers, and this means it's hard for them to stay together if they both chase their dream. He has to be on tour, and she has to rehearse for her play, so they won't see each other for at least two weeks (in this instance). Later, she calls him and leaves a message that she hasn't seen him in a while (complex scene which ends in another fight, which is very well done). So we see the conflict that seems to say "You can't have a relation of equals; one party has to give up their dream". While this might be partially true in the real world, I don't go to movies to see the real world.
2. After the year-long window into their life, I can't think that either Sebastian or Mia can be really successful without the other; because they are so alike, so passionate about their dreams, that a normal person wouldn't be able to understand and push the other when they need. However, the ending show both Mia and Sebastian quite successful, so one has to wonder: did they make it alone? Sebastian seems so (we don't see a partner for him), Mia unclear, likely not. How did Sebastian get through? What did Mia find in her husband?
3. This is very one-sided, since I'm a man, so bear with me: Sebastian helped Mia through her tough time. Once she got the breakthrough (and they split), she found somebody else, and I have to wonder in what circumstances they met. In the sense that maybe her husband only knew "successful Mia" and not "struggling/aspiring Mia". Her husband seems completely oblivious to all the eye contact between Mia and Sebastian in the club, seems to know Sebastian/about Sebastian not. How deep is their relation?
4. This is still one sided, sorry. When they break up (before Mia leaves for Paris), Sebastian asks "so where do we go from here?". Mia says "Nowhere". He asks once more, she rejects him again. So after one year of mad love and cries and happy moments, he gives up over two sentences? He's been following his dream (proper Jazz) in spite of all downturns in life until then, but he gives up on his real love over this? It doesn't make sense; trying to identify my self with the character, I can't reconcile this scene at all, unless he didn't really love her.

So no, I don't see them ending apart as romantic. I see it as the director is saying "You can't have both love and your [career] dreams. Choose either.", and he gives the "love" fake ending in the mini-re-roll of the movie, and the "career" wrong ending in the actual ending. And worse, he does it by negating significant parts of the character development done until now. Moreover, this conclusion is wrong. Wrong because this is a movie, and if movies don't manage to make you dream that you can achieve all, if movies tell you "choose either", then all is lost. Their love is not a separate character; them struggling to find each other in the successful phase of their life, learning to adapt to the new "he" and "she", would be the third thing. As it was shown, their love is simply a young love, that can't really survive the changes in life; they each said "I'll love you forever", but with this ending it sounds more "I'll cherish the memory of young you forever". Or differently said, it sounded like a cheap excuse to use when ending their relationship, in order to not negate the relationship itself. My version of the movie is another half hour long. It explains how Sebastian get over the "only jazz is pure old jazz" and manages to build a successful business around his old-style-but-modern jazz, instead of the pop-style jazz of the touring band (while thinking about her). It explains how Mia becomes a successful actress and gets over her first/second movies (while thinking about him), because one movie doesn't make one really successful (that reminds me: 3 year old child after 5 year forward-jump? when/how did her career go?). Hell, make it even more bitter show how their correspondence starts strong but becomes more and more sporadic over time, dying after the first 2 years. Show how both of them try other relations, and not find the same spark that they had before. And then, after they have matured, they meet again. And, just like the first time, they fall for each other, once again. She for his music, him for her passion for acting/for acting itself. She finds that him naming his club after her suggestion is oh-so-grown-up-and-sweet, he is happy that she finally grew into what he saw in her from the beginning. And he sings their song once more. But no. I'm not an artist, so I can only get the "die hope die die die love because I can" version. I still recommend the movie, but not the "after 5 years" scenes. Also, I didn't get time to bike today nor yesterday, so all you really get here is an ANGRY RANT. Because while I drink the coffee black and the tea without sugar, I like my happy endings, DAMN IT.

My most loathed feature of Go is the mandatory use of GOPATH: I do not want to put my own code next to its dependencies. Hopefully, this issue is slowly starting to be accepted by the main authors. In the meantime, you can workaround this problem with more opinionated tools (like gb) or by crafting your own Makefile. For the later, you can have a look at Filippo Valsorda s example or my own take which I describe in more details here. This is not meant to be an universal Makefile but a relatively short one with some batteries included. It comes with a simple Hello World! application.

Project structure For a standalone project, vendoring is a must-have¹ as you cannot rely on your dependencies to not introduce backward-incompatible changes. Some packages are using versioned URLs but most of them aren t. There is currently no standard tool to handle vendoring. My personal take is to vendor all dependencies with Glide. It is a good practice to split an application into different packages while the main one stay fairly small. In the hellogopher example, the CLI is handled in the cmd package while the application logic for printing greetings is in the hello package:

.
  cmd/
    hello.go
    root.go
    version.go
  glide.lock (generated)
  glide.yaml
  vendor/ (dependencies will go there)
  hello/
    root.go
    root_test.go
  main.go
  Makefile
  README.md

Down the rabbit hole Let s take a look at the various features of the Makefile.

GOPATH handling Since all dependencies are vendored, only our own project needs to be in the GOPATH:

PACKAGE  = hellogopher
GOPATH   = $(CURDIR)/.gopath
BASE     = $(GOPATH)/src/$(PACKAGE)
$(BASE):
    @mkdir -p $(dir $@)
    @ln -sf $(CURDIR) $@

The base import path is hellogopher, not github.com/vincentbernat/hellogopher: this shortens imports and makes them easily distinguishable from imports of dependency packages. However, your application won t be go get-able. This is a personal choice and can be adjusted with the $(PACKAGE) variable. We just create a symlink from .gopath/src/hellogopher to our root directory. The GOPATH environment variable is automatically exported to the shell commands of the recipes. Any tool should work fine after changing the current directory to $(BASE). For example, this snippet builds the executable:

.PHONY: all
all:   $(BASE)
    cd $(BASE) && $(GO) build -o bin/$(PACKAGE) main.go

Vendoring dependencies Glide is a bit like Ruby s Bundler. In glide.yaml, you specify what packages you need and the constraints you want on them. Glide computes a glide.lock file containing the exact versions for each dependencies (including recursive dependencies) and download them in the vendor/ folder. I choose to check into the VCS both glide.yaml and glide.lock files. It s also possible to only check in the first one or to also check in the vendor/ directory. A work-in-progress is currently ongoing to provide a standard dependency management tool with a similar workflow. We define two rules²:

GLIDE = glide
glide.lock: glide.yaml   $(BASE)
    cd $(BASE) && $(GLIDE) update
    @touch $@
vendor: glide.lock   $(BASE)
    cd $(BASE) && $(GLIDE) --quiet install
    @ln -sf . vendor/src
    @touch $@

We use a variable to invoke glide. This enables a user to easily override it (for example, with make GLIDE=$GOPATH/bin/glide).

Using third-party tools Most projects need some third-party tools. We can either expect them to be already installed or compile them in our private GOPATH. For example, here is the lint rule:

BIN    = $(GOPATH)/bin
GOLINT = $(BIN)/golint
$(BIN)/golint:   $(BASE) #  
    go get github.com/golang/lint/golint
.PHONY: lint
lint: vendor   $(BASE) $(GOLINT) #  
    @cd $(BASE) && ret=0 && for pkg in $(PKGS); do \
        test -z "$$($(GOLINT) $$pkg   tee /dev/stderr)"   ret=1 ; \
     done ; exit $$ret

As for glide, we let the user a chance to override which golint executable to use. By default, it uses a private copy. But a user can use its own copy with make GOLINT=/usr/bin/golint. In , we have the recipe to build the private copy. We simply issue go get³ to download and build golint. In , the lint rule executes golint on each package contained in the $(PKGS) variable. We ll explain this variable in the next section.

Working with non-vendored packages only Some commands need to be provided with a list of packages. Because we use a vendor/ directory, the shortcut ./... is not what we expect as we don t want to run tests on our dependencies⁴. Therefore, we compose a list of packages we care about:

PKGS = $(or $(PKG), $(shell cd $(BASE) && \
    env GOPATH=$(GOPATH) $(GO) list ./...   grep -v "^$(PACKAGE)/vendor/"))

If the user has provided the $(PKG) variable, we use it. For example, if they want to lint only the cmd package, they can invoke make lint PKG=hellogopher/cmd which is more intuitive than specifying PKGS. Otherwise, we just execute go list ./... but we remove anything from the vendor directory.

Tests Here are some rules to run tests:

TIMEOUT = 20
TEST_TARGETS := test-default test-bench test-short test-verbose test-race
.PHONY: $(TEST_TARGETS) check test tests
test-bench:   ARGS=-run=__absolutelynothing__ -bench=.
test-short:   ARGS=-short
test-verbose: ARGS=-v
test-race:    ARGS=-race
$(TEST_TARGETS): test
check test tests: fmt lint vendor   $(BASE)
    @cd $(BASE) && $(GO) test -timeout $(TIMEOUT)s $(ARGS) $(PKGS)

A user can invoke tests in different ways:

• make test runs all tests;
• make test TIMEOUT=10 runs all tests with a timeout of 10 seconds;
• make test PKG=hellogopher/cmd only runs tests for the cmd package;
• make test ARGS="-v -short" runs tests with the specified arguments;
• make test-race runs tests with race detector enabled.

Tests coverage go test includes a test coverage tool. Unfortunately, it only handles one package at a time and you have to explicitely list the packages to be instrumented, otherwise the instrumentation is limited to the currently tested package. If you provide too many packages, the compilation time will skyrocket. Moreover, if you want an output compatible with Jenkins, you ll need some additional tools.

COVERAGE_MODE    = atomic
COVERAGE_PROFILE = $(COVERAGE_DIR)/profile.out
COVERAGE_XML     = $(COVERAGE_DIR)/coverage.xml
COVERAGE_HTML    = $(COVERAGE_DIR)/index.html
.PHONY: test-coverage test-coverage-tools
test-coverage-tools:   $(GOCOVMERGE) $(GOCOV) $(GOCOVXML) #  
test-coverage: COVERAGE_DIR := $(CURDIR)/test/coverage.$(shell date -Iseconds)
test-coverage: fmt lint vendor test-coverage-tools   $(BASE)
    @mkdir -p $(COVERAGE_DIR)/coverage
    @cd $(BASE) && for pkg in $(PKGS); do \ #  
        $(GO) test \
            -coverpkg=$$($(GO) list -f '  join .Deps "\n"  ' $$pkg   \
                    grep '^$(PACKAGE)/'   grep -v '^$(PACKAGE)/vendor/'   \
                    tr '\n' ',')$$pkg \
            -covermode=$(COVERAGE_MODE) \
            -coverprofile="$(COVERAGE_DIR)/coverage/ echo $$pkg   tr "/" "-" .cover" $$pkg ;\
     done
    @$(GOCOVMERGE) $(COVERAGE_DIR)/coverage/*.cover > $(COVERAGE_PROFILE)
    @$(GO) tool cover -html=$(COVERAGE_PROFILE) -o $(COVERAGE_HTML)
    @$(GOCOV) convert $(COVERAGE_PROFILE)   $(GOCOVXML) > $(COVERAGE_XML)

First, we define some variables to let the user override them. We also require the following tools (in ):

• gocovmerge merges profiles from different runs into a single one;
• gocov-xml converts a coverage profile to the Cobertura format;
• gocov is needed to convert a coverage profile to a format handled by gocov-xml.

The rules to build those tools are similar to the rule for golint described a few sections ago. In , for each package to test, we run go test with the -coverprofile argument. We also explicitely provide the list of packages to instrument to -coverpkg by using go list to get a list of dependencies for the tested package and keeping only our owns.

Final result While the main goal of using a Makefile was to work around GOPATH, it s also a good place to hide the complexity of some operations, notably around test coverage. The excerpts provided in this post are a bit simplified. Have a look at the final result for more perks!

---

1. In Go, vendoring is about both bundling and dependency management. As the Go ecosystem matures, the bundling part (fixed snapshots of dependencies) may become optional but the vendor/ directory may stay for dependency management (retrieval of the latest versions of dependencies matching a set of constraints).
2. If you don t want to automatically update glide.lock when a change is detected in glide.yaml, rename the target to deps-update and make it a phony target.
3. There is some irony for bad mouthing go get and then immediately use it because it is convenient.
4. I think ./... should not include the vendor/ directory by default. Dependencies should be trusted to have run their own tests in the environment they expect them to succeed. Unfortunately, this is unlikely to change.

Here is how I managed to extend my OpenVPN setup on my Linode VPS to include IPv6 traffic. This ensures that clients can route all of their traffic through the VPN and avoid leaking IPv6 traffic, for example. It also enables clients on IPv4-only networks to receive a routable IPv6 address and connect to IPv6-only servers (i.e. running your own IPv6 broker).

Request an additional IPv6 block The first thing you need to do is get a new IPv6 address block (or "pool" as Linode calls it) from which you can allocate a single address to each VPN client that connects to the server. If you are using a Linode VPS, there are instructions on how to request a new IPv6 pool. Note that you need to get an address block between /64 and /112. A /116 like Linode offers won't work in OpenVPN. Thankfully, Linode is happy to allocate you an extra /64 for free.

Setup the new IPv6 address If your server only has an single IPv4 address and a single IPv6 address, then a simple DHCP-backed network configuration will work fine. To add the second IPv6 block on the other hand, I had to change my network configuration (/etc/network/interfaces) to this:

auto lo
iface lo inet loopback
allow-hotplug eth0
iface eth0 inet dhcp
    pre-up iptables-restore /etc/network/iptables.up.rules
iface eth0 inet6 static
    address 2600:3c01::xxxx:xxxx:xxxx:939f/64
    gateway fe80::1
    pre-up ip6tables-restore /etc/network/ip6tables.up.rules
iface tun0 inet6 static
    address 2600:3c01:xxxx:xxxx::/64
    pre-up ip6tables-restore /etc/network/ip6tables.up.rules

where 2600:3c01::xxxx:xxxx:xxxx:939f/64 (bound to eth0) is your main IPv6 address and 2600:3c01:xxxx:xxxx::/64 (bound to tun0) is the new block you requested. Once you've setup the new IPv6 block, test it from another IPv6-enabled host using:

ping6 2600:3c01:xxxx:xxxx::1

OpenVPN configuration The only thing I had to change in my OpenVPN configuration (/etc/openvpn/server.conf) was to change:

proto udp

to:

proto udp6

in order to make the VPN server available over both IPv4 and IPv6, and to add the following lines:

server-ipv6 2600:3c01:xxxx:xxxx::/64
push "route-ipv6 2000::/3"

to bind to the right V6 address and to tell clients to tunnel all V6 Internet traffic through the VPN. In addition to updating the OpenVPN config, you will need to add the following line to /etc/sysctl.d/openvpn.conf:

net.ipv6.conf.all.forwarding=1

and the following to your firewall (e.g. /etc/network/ip6tables.up.rules):

# openvpn
-A INPUT -p udp --dport 1194 -j ACCEPT
-A FORWARD -m state --state NEW -i tun0 -o eth0 -s 2600:3c01:xxxx:xxxx::/64 -j ACCEPT
-A FORWARD -m state --state NEW -i eth0 -o tun0 -d 2600:3c01:xxxx:xxxx::/64 -j ACCEPT
-A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT

in order to ensure that IPv6 packets are forwarded from the eth0 network interface to tun0 on the VPN server. With all of this done, apply the settings by running:

sysctl -p /etc/sysctl.d/openvpn.conf
ip6tables-apply
systemctl restart openvpn.service

Testing the connection Now connect to the VPN using your desktop client and check that the default IPv6 route is set correctly using ip -6 route. Then you can ping the server's new IP address:

ping6 2600:3c01:xxxx:xxxx::1

and from the server, you can ping the client's IP (which you can see in the network settings):

ping6 2600:3c01:xxxx:xxxx::1002

Once both ends of the tunnel can talk to each other, you can try pinging an IPv6-only server from your client:

ping6 ipv6.google.com

and then pinging your client from an IPv6-enabled host somewhere:

ping6 2600:3c01:xxxx:xxxx::1002

If that works, other online tests should also work.

Couple of weeks ago, either on some mailing list, on IRC or somewhere else, somebody mentioned that people always put higher amount of airline expenditure for self when asking for sponsorship. Now last year, between sending the application and getting the approval for sponsorship, there was 3 months of difference between the two. Now if you put up an application for sponsorship like I did last year, I had added 10% to the cost of flight tickets of the cheapest prevailing prices at that point in time on skyscanner or any of the meta-search-engines were showing me at that point in time. I was sceptical whether the amount that I had put for the to and fro tickets would be enough or not. Strangely, I was lucky enough to get my ticket around the new estimated price. I would have to mention though that there were only 2 tickets left at the new price and if I had waited just a few hours more, those tickets would have gone too and all other tickets were around 25% more than before. The only reasons I could fathom are a. Luck, pure and simple. b. Going at the end of the tourism season This was evident as I was able to book my extended stay at any hostel just 2 days before my stay at UCT (University of Cape Town) was over. Was corroborated by hostel staff, shop-owners as well as whatever info. I found on the web before and during my stay in SA. c. South Africa being more lenient than probably Canada is giving and processing visas. While looking at the third point, thought I better check world tourism rankings and saw the Wikipedia page for it. Interestingly, South Africa seems to have a slight edge over Canada when it comes to statistics and hence strengthens my assumptions that probably more people apply for SA than Canada as they know the possibility of more people making through visa processing. It would stand to be logical that more people would apply for a tourist or similar short-term resident visas if they know they have a good chance going through. While researching on the topic I also came across/ hunted to find the hardest places to get a Visa for and was surprised to find India being lasted therein.Coincidentally, that site also has a UK domain.It does burst the bubble in Incredible India a little bit. As a newbie who had no clue I knew I was probably a victim of Information Asymmetry where the airlines have much more information about travel trends, ongoing trends at Airports, Politics and Economics of Countries, Price of Crude Oil, Profit, Competition and probably many more factors that I haven t taken into account which decide fares. While researching on the topic, one of the interesting finds I had while trying to figure the above is that Airlines didn t pass on fuel savings to their customers. Now I don t know whether this was the same around the world or was this only in UK. I am shocked that British (and by definition EU, as UK was part of EU at that time) travellers or consumer groups didn t file a suit in the court of law as reading the above smells of anticompetitive behaviour. The most shocking statement was this Average fares to Spain rose by 10 per cent over the same period. Telegraph, UK. In order to lessen this information asymmetry a bit, I used google flights and its data of the past 2 months to see how the fares have been changing to have some insight of where the fares might end up. I know google is hated by one and all, but in this instance I couldn t find any comparable site which does this kind of thing. As can be seen in the graph, the tickets had started relatively cheap from around INR 65k ish to around 80k ish at this point in time. That is a jump of around 30% in the last couple of months. All of these flights have a layover somewhere in Europe and taking a second flight from there to Canada. The one which didn t show much of action is the direct plan between BOM YUL and back but then this seems to be a premium service . Taking a direct flight from BOM YUL is north of INR 90k/- which doesn t make much sense unless one is fond of spending 13+ hours in flight. Definitely not my cup of tea. With layovers it makes the experience a bit more bearable. While the real action is probably 3 or bit more months away, its interesting to see how things are panning out at least on airline price tickets and the dynamics involved therein. Even with all the above attempts at finding the answer, I m no closer to figuring out to estimate airline ticket prices when window is largish 3 months in making. Any ideas anybody? If the previous jump is any indication, then 10-15% escalation bit might not hack this time around. Any strategies that people could advise while trying to put a ball-park figure.
Filed under: Miscellenous Tagged: #air-travel, #Airfares, #Competition, #Price Estimation, #Sponsorship

A new version of the nanotime package for working with nanosecond timestamps is now on CRAN. nanotime uses the RcppCCTZ package for (efficient) high(er) resolution time parsing and formatting, and the bit64 package for the actual integer64 arithmetic. This release adds an improved default display format always showing nine digits of fractional seconds. It also changes the print() method to call format() first, and we started to provide some better default Ops methods. These fixes were suggested by Matt Dowle. We also corrected a small issue which could lead to precision loss in formatting as pointed out by Leonardo Silvestri.

Changes in version 0.1.1 (2017-02-04)

• The default display format now always shows nine digits (#10 closing #9)
• The default print method was updated to use formated output, and a new new converter as.integer64 was added
• Several 'Ops' method are now explicitly defined allowing casting of results (rather than falling back on bit64 behaviour)
• The format routine is now more careful about not loosing precision (#13 closing #12)

We also have a diff to the previous version thanks to CRANberries. More details and examples are at the nanotime page; code, issue tickets etc at the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you re interested in Java, Games and LTS topics, this might be interesting for you. Debian Games

• In January 2017 we had the last chance to get new upstream releases into the next stable release of Debian 9 aka Stretch. Hence I packaged new versions of pygame-sdl2, renpy, fife, unknown-horizons, redeclipse and redeclipse-data and also backported Red Eclipse to Jessie.
• I uploaded fifechan to unstable and applied an upstream patch to fix a segmentation fault (#852247) in Unknown Horizons.
• Package cleanups and improvements: freeorion (#843538), I enabled support for mips64el again; I tidied up gtkatlantic, powermanga, lincity-ng, opencity and tecnoballz; I applied a patch from Reiner Herrmann to make the build of netpanzer reproducible (#827150); In spring I changed the build-dependency of asciidoc to asciidoc-base (#850387) although it turned out later that this wasn t strictly needed. I also removed ConvertUTF8 related code from spring because it might be non-free. I don t think this is necessarily true but I didn t want to argue with Lintian in this case.
• I sponsored a new upstream release of pentobi for Juhani Numminen.
• I backported minetest 0.4.5 to jessie-backports and fixed #851114, which I think was not really an issue since we already provide the font sources in Debian and Minetest depends on the respective package.
• I triaged RC bug #847812 in pysolfc, provided a patch and reassigned the issue to src:pillow. Apparently this affected a lot more 32 bit applications written in Python.

Debian Java

• For Debian Java the situation was very similar but I didn t want to break too many reverse-dependencies right before the freeze. So I chose rather harmless packages and updated objenesis, syncany and jackson-dataformat-xml. At first I couldn t build objenesis 2.5 from source but after some consultation on the debian-java list, it became clear that the test failure was probably an upstream bug. Henri Tremblay kindly issued a new release of Objenesis which addressed the build failure.
• Package updates and bugfixes: sweethome3d (#849885), I confirmed a build-failure in java-gnome (#849669) and investigated another issue in jackson-module-afterburner (#849931) which was later removed from Debian. I fixed an RC bug in commons-daemon (#849649). I also released new revisions of libhamcrest-java (#846116), libjide-oss-java, libjchart2d-java, activemq, osgi-annotations, jsemver, openjpa and xmlbeans (#820839,#822091).
• I prepared security updates for groovy and groovy2 in Jessie. (#853233, #853235)

Debian LTS This was my eleventh month as a paid contributor and I have been paid to work 12,75 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following:

• From 16. January until 22. January I was in charge of our LTS frontdesk. I triaged security issues in imagemagick, wordpress, hesiod, opus, mysql-5.5, netbeans, groovy and zoneminder.
• DLA-779-1. Issued a security update for Tomcat 7 fixing 1 CVE and a regression when running Tomcat with SecurityManager enabled.
• DLA-761-2. Issued a regression update for python-bottle. (Debian bug #850176).
• DLA-781-1 and DLA-781-2. Issued a security update for Asterisk fixing 2 CVE after I had prepared the package last month. Later Brad Barnett discovered a regression when using SIP communication and provided assistance with debugging the issue. I corrected this one in DLA-781-2.
• DLA-792-1. Issued a security update for libphp-swiftmailer fixing 1 CVE.
• DLA-793-1. Issued a security update for opus fixing 1 CVE.
• DLA-794-1. Issued a security update for groovy fixing 1 CVE.
• DLA-797-1. Issued a security update for mysql-5.5 fixing 10 CVE. The update was prepared by Lars Tangvald.
• DLA-813-1. Issued a security update for wordpress fixing 9 CVE.

Misc

• In xarchiver (#850103) I added binutils to the list of suggested packages, in iftop (#850040) I applied a patch from Brian Russell and I packaged a new upstream release of mediathekview, a Java application to watch and download broadcasts from German television stations. I had to make some major packaging changes because the build system switched from Ant to Gradle but there were fewer issues than expected.

It has been a quiet start to the year due to work keeping me very busy. Most of my spare time (when not sitting shattered on the sofa) was spent resurrecting my old website from backups. My son had plenty of visitors coming to visit as well, which prompted me to restart work on my model railway in the basement. Last year I received a whole heap of track, and also a tunnel formation from a friend at work. I managed to finish the supporting structure for the tunnel, and connect one end of it to the existing track layout. The next step (which will be a bit harder) is to connect the other end of the tunnel into the existing layout. The basement is one of the favourite things for me to keep my son and his friends occupied when there is a visit. The railway and music studio are very popular with the little guests. Debian

• Packaged latest Gramps 4.2.5 release for Debian so that it will be part of the Stretch release.
• Package latest abcmidi release so it too would be part of Stretch. The upstream author had changed his website, so it took a while to locate a tarball.
• Tested my latest patches to convert Cree.py to Qt5, but found another Qt4 Qt5 change to take into account (SIGNAL function). I ran out of time to fully investigate that one, before Creepy was booted out of testing again. I am seriously considering the removal of Cree.py from Debian, as the upstream maintainer does not seem very active any more, and I am a little tired of being upstream for a project that I don t actually use myself. It was only because it was a reverse dependency of osm-gps-map that I originally got involved.
• Started preparing a Gramps 5.2.5 backport for Jessie, but found that the tests I enabled in unstable were failing in the Jessie build. I need to investigate this further.

Ubuntu

• Announced the Ubuntu Studio 16.02.2 point release date on the Ubuntu Studio mailing lists asking for testers. The date subsequently got put back to February the 9th.
• Upgraded my Ubuntu Studio machine from Wily to Xenial.

Other

• Resurrected my old Drupal Gammon One Name Study website. I used Drupal VM to get the site going again, before transferring it to the new webhost. It was originally a Drupal 7 site, and I did not have the required versions of Ansible & Vagrant on my Ubuntu Xenial machine, so the process was quite involved. I will blog about that separately, as it may be a useful lesson for others. As part of that, I started on a backport of vagrant, but found a bug which I need to follow up on.
• Also managed to extract my old WordPress blog posts from the same machine that had the failed Drupal instance, and import them into this blog. I also learnt some stuff in that process that I will blog about at some point.

Plan status from last month & update for next month Debian Before the 5th February 2017 Debian Stretch hard freeze I hope to:

• Convert Cree.py from QT4 to QT5 to allow it to re-enter Stretch. Failed

For the Debian Stretch release:

• Keep an eye on the Release Critical bugs list, and see if I can help fix any.

Generally:

• Finish the Gramps 5.2.5 backport for Jessie.
• Package all the latest upstream versions of my Debian packages, and upload them to Experimental to keep them out of the way of the Stretch release.
• Begin working again on all the new stuff I want packaged in Debian.

Ubuntu

• Finish the ubuntustudio-lightdm-theme, ubuntustudio-default-settings transition including an update to the ubuntustudio-meta packages. Still to do (actually started today)
• Reapply to become a Contributing Developer. Still to do
• Start working on an Ubuntu Studio package tracker website so that we can keep an eye on the status of the packages we are interested in. Started
• Start testing & bug triaging Ubuntu Studio packages. Still to do
• Test Len s work on ubuntustudio-controls Still to do

Other

• Try and resurrect my old Gammon one-name study Drupal website from a backup and push it to the new GoONS Website project. Done
  
• Give JMRI a good try out and look at what it would take to package it. In progress
• Also look at OpenPLC for simulating the relay logic of real railway interlockings (i.e. a little bit of the day job at home involving free software fun!).

FTP assistant This month I only marked 146 packages for accept and rejected 25 packages. I only sent 3 emails to maintainers asking questions. Nevertheless I could pass a big mark. All in all I accepted more than 10000 packages now! Debian LTS This was my thirty-first month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. This month my all in all workload has been 12.75h. During that time I did uploads of

• [DLA 805-1] bind9 security update for three CVEs
• [DLA 806-1] zoneminder security update for one CVE

Unfortunately the upload of jasper had to be postponed, as there is no upstream fix for most of the open CVEs yet.
I also suggested to mark th slum-llnl CVE as , as the patch would be too invasive. Further I did another week of frontdesk work. Last but not least I took care of about 140 items of the TODO list[1]. Ok, it was not that much work, but the enormous number is impressing :-). I also had a look at [2] and filed bugs against two packages. Within hours the maintainers responded to that bugs, clarified everything to mark the CVEs as not-affected and nobody has to care about them anymore. This is a good example of how the knowledge of the maintainer can help the security teams! So, if you have some time left, have a look at [3] and take care of something. [1] https://security-tracker.debian.org/tracker/status/todo
[2] https://security-tracker.debian.org/tracker/status/unreported
[3] https://security-tracker.debian.org/tracker Other stuff This month I sponsored a new round of sidedoor and printrun. After advocating Dara Adib to become Debian Maintainer, I hope my activities as sponsor can be reduced again :-). Further I uploaded another version of setserial, but as you can see in #850762 it does not seem to satisfy everybody. I also uploaded new upstream versions of duktape and pipexec. As I didn t do any DOPOM in December I adopted two packages in January: pescetti and salliere. I dedicate those uploads to my aunt Birgit, who was a passionate bridge player. You will never be forgotten.

The soft freeze has been on going for almost a month now and the full stretch freeze will start tomorrow night (UTC). It has definitely been visible in the number of unblock requests that we have received so far. Fortunately, we are no where near the rate of the jessie freeze. At the moment, all unblock requests are waiting for the submitter (either for a clarification or an upload). Looking at stretch at a glance (items are in no particular order):

• Secure boot support is progressing too slowly.
• The openssl transition is still on-going and badly needs closure.
• The MySQL to MariaDB transition is done.
• The gcc-5 removal almost complete missing only 2 packages.
• About 150 RC bugs (in key packages) need to be handled
• We are missing a stretch-backport suite so DSA can start their upgrade tests
• The release checklist have a few items that could be completed now.

Secure boot support Currently, we are blocked on two items:

• We do not have signing done yet for the boot packages (not even manual signing).
• Our shim is not yet signed, so no hardware would be trusting our boot chain out of the box.

After they are done, we are missing a handful of uploads to provide a signed bootloader etc. plus d-i and some infrastructure bits need to be updated. At the moment, we are waiting for a handful of key people/organisations to move on their part. As such, there is not a lot you can do to assist here (unless you are already involved in the work).
On the flip side, if both of these items are resolved soon, there is a good chance that we can support secure boot in stretch.See bug#820036 and blockers for more information on the remaining items. Where can you help with the release? At the moment, the best you can do is to:

• Test (packages, upgrades, etc.) and report bugs
• File bugs against release-notes for issues that should be documented
• Fix RC bugs (please see the next section)

Release Critical Bug report The UDD bugs interface currently knows about the following release critical bugs:

• In Total: 1148 (Including 193 bugs affecting key packages)
  • Affecting stretch: 294 (key packages: 158)
    That s the number we need to get down to zero before the release. They can be split in two big categories:
    • Affecting stretch and unstable: 232 (key packages: 134)
      Those need someone to find a fix, or to finish the work to upload a fix to unstable:
      • 30 bugs are tagged patch . (key packages: 21)
        Please help by reviewing the patches, and (if you are a DD) by uploading them.
      • 17 bugs are marked as done, but still affect unstable. (key packages: 5)
        This can happen due to missing builds on some architectures, for example. Help investigate!
      • 185 bugs are neither tagged patch, nor marked done. (key packages: 108)
        Help make a first step towards resolution!
    • Affecting stretch only: 62 (key packages: 24)
      Those are already fixed in unstable, but the fix still needs to migrate to stretch. You can help by submitting unblock requests for fixed packages, by investigating why packages do not migrate, or by reviewing submitted unblock requests.
      • 36 bugs are in packages that are unblocked by the release team. (key packages: 14)
      • 26 bugs are in packages that are not unblocked. (key packages: 10)

Filed under: Debian, Release-Team