Japanese authors have a tendency to commit suicide, it seems. I have read Ryunosuke Akutagawa ( , at 35), Yukio Mishima ( , at 45), and also Osamu Dazai ( , at 39). Their end often reflects in their writings, and one of these examples is the book I just finished, No Longer Human. Considered as Dazai s master piece, and with Soseki s Kokoro the best selling novels in Japan. The book recounts the life of Oba Yozo, from childhood to the end in a mental hospital. The early years, described in the first chapter ( Memorandum ), are filled with the feeling of differentness, alienation from the rest, and Oba starts his way of living by playing the clown, permanently making jokes. The Second Memorandom spans the time to university, where he drops out, tries to become a painter, indulges in alcohol, smoking and prostitutes, leading to a suicide attempt together with a married woman, but he survived. The first part of the Third Memorandom sees a short recovering due to his relationship with a woman. He stops drinking and works as cartoonist, but in the last part his drinking pal from university times shows up again and they return into an ever increasing vicious drinking. Eventually he is separated from his wife, and confined to a mental hospital. Very depressing to read, but written in a way that one cannot stop reading. The disturbing thing about this book is that, although the main actor conceives many bad actions, we feel somehow attached to him and feel pity for him. It is somehow a exercise how circumstances and small predispositions can make a huge change in our lives. And it warns us that each one of us can easily come to this brink.

Just finished my first book of Jonas Jonasson, a Swedish journalist and author. Most famous for his book The Hundred-Year-Old Man Who Climbed Out the Window and Disappeared, but author of two others. The one I read was The Girl Who Saved the King of Sweden, which strange enough became in German Die Analphabetin die rechnen konnte (The analphabet who could compute). The story recounts the countless turns the life of Nombeko Mayeki, a black girl born in Soweto as latrine cleaner, who manages to save the Swedish king as well as most of the world from an atomic desaster by first getting driven over by a drunkard of South African nuclear bomb engineer, then meeting a clique of three Chinese sisters excelling in faking antiquities, and two Mossad agents. With the (unwilling) help of those agents she escapes to Sweden (including the atomic bomb) where she meets twins of a psychotic father who brought them up as one child so that the spare one can eradicate the Swedish monarchy. After many twists and setbacks, including several meetings with the Chinese premier Hu Jintao, she finally manages to get rid of the atomic bomb, get her undercover twin a real identity, and set up a proper life ah, and not to forget, save the King of Sweden! A fast paced, surprisingly funny and lovely story about how little things can change our lives completely.

A while back, I found myself in need of two webservers that would terminate TLS (with different rules). I wanted to run some custom code I d written (which uses TLS peer authentication), and also nginx on port 443. The best way I figured out how to do this was to write a tool to sit on port 443, and parse TLS Client Hello packets, and dispatch to the correct backend depending on the SNI name. SNI, or Server Name Indication allows the client to announce (yes over cleartext!) what server it s looking for, similar to the HTTP Host header. Sometimes, like in the case above, the Host header won t work, since you ve already done a TLS handshake by the time you figure out who they re looking for. I also spun the Client Hello parser out into its own importable package, just in case someone else finds themselves in this same boat. The code s up on github.com/paultag/sniff!

The procps developers are happy to announce that version 3.3.12 of procps was released today. This version has a mixture of bug fixes and enhancements. This unfortunately means another API bump but we are hoping this will be fixed with the new library API coming soon. procps is developed on gitlab and the new version of procps can be found at https://gitlab.com/procps-ng/procps/tree/newlib procps 3.3.12 can be found at https://gitlab.com/procps-ng/procps/tags/v3.3.12 From the NEWS file, procps 3.1.12 has the following:

• build: formerly optional enable-oomem unconditional
• free: man document rewritten for shared Debian #755233
• free: interpret intervals in non-locale way Debian #692113
• kill: report error if cannot kill process Debian #733172
• library: refine calculation of cached memory
• library: find tty quicker Debian #770215
• library: eliminate threads display inconsistencies Redhat #1284091
• pidof: check cmd if space found in argv0
• pmap: fixed detail parsing on long mapping lines
• pmap: fix occasional incorrect memory usage values Redhat #1262864
• ps: sort by cgroup Debian #692279
• ps: display control group name with -o cgname
• ps: fallback to attr/current for context Debian #786956
• ps: enabled broken thcount option Redhat #1174313
• tests: conditionally add prctl Debian #816237
• top: displays the 3 new linux-4.5 RES memory fields
• top: man page memory fields corrected + new narrative
• top: added display of CGNAME (control group name)
• top: is now more responsive to cpus brought online
• top: namespace cols use suppressible zero

We are hoping this will be the last one to use the old API and the new format API ( imaginatively called newlib ) will be used in subsequent releases. Feedback for this and any other version of procps can be sent to either the issue tracker or the development email list.

I have updated the openpht repository with builds of OpenPHT 1.6.2 for Debian/sid for both amd64 and i386 architecture. For those who have forgotten it, OpenPHT is the open source fork of Plex Home Theater that is used on RasPlex, see my last post concerning OpenPHT for details. The repository also contains packages (source and amd64/i386) for shairplay which is necessary for building and running OpenPHT. sid and testing For sid use the following lines: You can also grab the binary for amd64 directly here for amd64 and i386, you can get the source package with Note that if you only get the binary deps, you also need libshairplay0 from amd64 or i386. The release file and changes file are signed with my official Debian key 0x860CDC13. jessie Builds for Debian stable release jessie are available directly from the github project page of OpenPHT Now be ready for enjoying the next movie!

11. Why use a genetic algorithm instead of a backpropagation algorithm to train a neural net ? 12. What is the difference between a support vector machine (SVM) and an artificial neural network (ANN) ? 13. What s a Naives Bayes classifier ? 14. What is the difference between supervised and unsurpervised learning 15. What is Continue reading Tech questions 11-16: Artificial intelligence

A year ago, I wrote:

Instead of keeping substantial tabs open until I have read all of them, or losing them in the jungle of browser bookmarks, I have written a script that collects them into a file per month, and turns them into markdown files for my blog.

That script turned out to be quirky and overengineered, so much so that I stopped using it myself. I've now rethought my approach, and downscaled it: instead of saving a copy of each page locally, I can blog a reference to https://archive.org or https://archive.is. I do not need to autogenerate a description from the site itself. The result has been a nicely minimal set of changes to staticsite that resulted in a new version where adding a link to a monthly collection is as easy as typing ssite new -a links. As long as I'll remember to rebuild the site 3 weeks from now, a new post should automagically appear in my blog.

concurrent-output is a more meaty Haskell library than the ones I've covered so far. Its interface is simple, but there's a lot of complexity under the hood. Things like optimised console updates, ANSI escape sequence parsing, and transparent paging of buffers to disk. It developed out of needing to display multiple progress bars on the console in git-annex, and also turned out to be useful in propellor. And since it solves a general problem, other haskell programs are moving toward using it, like shake and stack. Next: twenty years of free software -- part 12 propellor

Background: I ve been relearning Rust (more about that in a separate post, some time later), and in doing so, I chose to implement the low-level parts of git (I ll touch the why in that separate post I just promised). Disclaimer: It s friday. This is not entirely(?) a serious post. So, I was looking at Documentation/technical/index-format.txt, and saw:

32-bit number of index entries.

What? The index/staging area can t handle more than ~4.3 billion files? There I was, writing Rust code to write out the index.

try!(out.write_u32::<NetworkOrder>(self.entries.len()));

(For people familiar with the byteorder crate and wondering what NetworkOrder is, I have a use byteorder::BigEndian as NetworkOrder) And the Rust compiler rightfully barfed:

error: mismatched types:
 expected  u32 ,
    found  usize  [E0308]

And there I was, wondering: mmmm should I just add as u32 and silently truncate or hey what does git do? And it turns out, git uses an unsigned int to track the number of entries in the first place, so there is no truncation happening. Then I thought but what happens when cache_nr reaches the max? Well, it turns out there s only one obvious place where the field is incremented. What? Holy coffin nails, Batman! No overflow check? Wait a second, look 3 lines above that:

ALLOC_GROW(istate->cache, istate->cache_nr + 1, istate->cache_alloc);

Yeah, obviously, if you re incrementing cache_nr, you already have that many entries in memory. So, how big would that array be?

        struct cache_entry **cache;

So it s an array of pointers, assuming 64-bits pointers, that s ~34.3 GB. But, all those cache_nr entries are in memory too. How big is a cache entry?

struct cache_entry  
        struct hashmap_entry ent;
        struct stat_data ce_stat_data;
        unsigned int ce_mode;
        unsigned int ce_flags;
        unsigned int ce_namelen;
        unsigned int index;     /* for link extension */
        unsigned char sha1[20];
        char name[FLEX_ARRAY]; /* more */
 ;

So, 4 ints, 20 bytes, and as many bytes as necessary to hold a path. And two inline structs. How big are they?

struct hashmap_entry  
        struct hashmap_entry *next;
        unsigned int hash;
 ;
struct stat_data  
        struct cache_time sd_ctime;
        struct cache_time sd_mtime;
        unsigned int sd_dev;
        unsigned int sd_ino;
        unsigned int sd_uid;
        unsigned int sd_gid;
        unsigned int sd_size;
 ;

Woohoo, nested structs.

struct cache_time  
        uint32_t sec;
        uint32_t nsec;
 ;

So all in all, we re looking at 1 + 2 + 2 + 5 + 4 32-bit integers, 1 64-bits pointer, 2 32-bits padding, 20 bytes of sha1, for a total of 92 bytes, not counting the variable size for file paths. The average path length in mozilla-central, which only has slightly over 140 thousands of them, is 59 (including the terminal NUL character). Let s conservatively assume our crazy repository would have the same average, making the average cache entry 151 bytes. But memory allocators usually allocate more than requested. In this particular case, with the default allocator on GNU/Linux, it s 156 (weirdly enough, it s 152 on my machine). 156 times 4.3 billion 670 GB. Plus the 34.3 from the array of pointers: 704.3 GB. Of RAM. Not counting the memory allocator overhead of handling that. Or all the other things git might have in memory as well (which apparently involves a hashmap, too, but I won t look at that, I promise). I think one would have run out of memory before hitting that integer overflow. Interestingly, looking at Documentation/technical/index-format.txt again, the on-disk format appears smaller, with 62 bytes per file instead of 92, so the corresponding index file would be smaller. (And in version 4, paths are prefix-compressed, so paths would be smaller too). But having an index that large supposes those files are checked out. So let s say I have an empty ext4 file system as large as possible (which I m told is 2^60 bytes (1.15 billion gigabytes)). Creating a small empty ext4 tells me at least 10 inodes are allocated by default. I seem to remember there s at least one reserved for the journal, there s the top-level directory, and there s lost+found ; there apparently are more. Obviously, on that very large file system, We d have a git repository. git init with an empty template creates 9 files and directories, so that s 19 more inodes taken. But git init doesn t create an index, and doesn t have any objects. We d thus have at least one file for our hundreds of gigabyte index, and at least 2 who-knows-how-big files for the objects (a pack and its index). How many inodes does that leave us with? The Linux kernel source tells us the number of inodes in an ext4 file system is stored in a 32-bits integer. So all in all, if we had an empty very large file system, we d only be able to store, at best, 2^32 22 files And we wouldn t even be able to get cache_nr to overflow. while following the rules. Because the index can keep files that have been removed, it is actually possible to fill the index without filling the file system. After hours (days? months? years? decades?*) of running

seq 0 4294967296   while read i; do touch $i; git update-index --add $i; rm $i; done

One should be able to reach the integer overflow. But that d still require hundreds of gigabytes of disk space and even more RAM.

• At the rate it was possible to add files to the index when I tried (yeah, I tried), for a few minutes, and assuming a constant rate, the estimate is close to 2 years. But the time spent reading and writing the index file increases linearly with its size, so the longer it d run, the longer it d take.

Ok, it s actually much faster to do it hundreds of thousand files at a time, with something like:

seq 0 100000 4294967296   while read i; do j=$(seq $i $(($i + 99999))); touch $j; git update-index --add $j; rm $j; done

At the rate the first million files were added, still assuming a constant rate, it would take about a month on my machine. Considering reading/writing a list of a million files is a thousand times faster than reading a list of a billion files, assuming linear increase, we re still talking about decades, and plentiful RAM. Fun fact: after leaving it run for 5 times as much as it had run for the first million files, it hasn t even done half more One could generate the necessary hundreds-of-gigabytes index manually, that wouldn t be too hard, and assuming it could be done at about 1 GB/s on a good machine with a good SSD, we d be able to craft a close-to-explosion index within a few minutes. But we d still lack the RAM to load it. So, here is the open question: should I report that integer overflow? Wow, that was some serious procrastination. Edit: Epilogue: Actually, oops, there is a separate integer overflow on the reading side that can trigger a buffer overflow, that doesn t actually require a large index, just a crafted header, demonstrating that yes, not all integer overflows are equal.

I've spent the past days updating several of my websites to be "responsive". Mostly that means I open the site in firefox then press Ctrl-alt-m to switch to mobile-view. Once I have the mobile-view I then fix the site to look good in small small space. Because my general design skills are poor I've been fixing most sites by moving to bootstrap, and ensuring that I don't use headers/footers that are fixed-position. Beyond the fixes to appearances I've also started rationalizing the domains, migrating content across to new homes. I've got a provisional theme setup at steve.fi, and I've moved my blog over there too. The plan for blog-migration went well:

• Setup a redirect to from https://blog.steve.org.uk to https://blog.steve.fi/
• Replace the old feed with a CGI script which outputs one post a day, telling visitors to update their feed.
  • This just generates one post, but the UUID of the post has the current date in it. That means it will always be fresh, and always be visible.
• Updated the template/layout on the new site to use bootstrap.

The plan was originally to setup a HTTP-redirect, but I realized that this would mean I'd need to keep the redirect in-place forever, as visitors would have no incentive to fix their links, or update their feeds. By adding the fake-RSS-feed, pointing to the new location, I am able to assume that eventually people will update, and I can drop the dns record for blog.steve.org.uk entirely - Already google seems to have updated its spidering and searching shows the new domain already.

Just over a month ago I ordered a new Nexus 6P [1]. I ve had it for over a month now and it s time to review it and the Samsung Galaxy S5 Mini I also bought. Security The first noteworthy thing about this phone is the fingerprint scanner on the back. The recommended configuration is to use your fingerprint for unlocking the phone which allows a single touch on the scanner to unlock the screen without the need to press any other buttons. To unlock with a pattern or password you need to first press the power button to get the phone s attention. I have been considering registering a fingerprint from my non-dominant hand to reduce the incidence of accidentally unlocking it when carrying it or fiddling with it. The phone won t complete the boot process before being unlocked. This is a good security feature. Android version 6 doesn t assign permissions to apps at install time, they have to be enabled at run time (at least for apps that support Android 6). So you get lots of questions while running apps about what they are permitted to do. Unfortunately there s no allow for the duration of this session option. A new Android feature prevents changing security settings when there is an overlay running . The phone instructs you to disable overlay access for the app in question but that s not necessary. All that is necessary is for the app to stop using the overlay feature. I use the Twilight app [2] to dim the screen and use redder colors at night. When I want to change settings at night I just have to pause that app and there s no need to remove the access from it note that all the web pages and online documentation saying otherwise is wrong. Another new feature is to not require unlocking while at home. This can be a convenience feature but fingerprint unlocking is so easy that it doesn t provide much benefit. The downside of enabling this is that if someone stole your phone they could visit your home to get it unlocked. Also police who didn t have a warrant permitting search of a phone could do so anyway without needing to compel the owner to give up the password. Design This is one of the 2 most attractive phones I ve owned (the other being the sparkly Nexus 4). I think that the general impression of the appearance is positive as there are transparent cases on sale. My phone is white and reminds me of EVE from the movie Wall-E. Cables This phone uses the USB Type-C connector, which isn t news to anyone. What I didn t realise is that full USB-C requires that connector at both ends as it s not permitted to have a data cable with USB-C at the device and and USB-A at the host end. The Nexus 6P ships with a 1M long charging cable that has USB-C at both ends and a ~10cm charging cable with USB-C at one end and type A at the other (for the old batteries and the PCs that don t have USB-C). I bought some 2M long USB-C to USB-A cables for charging my new phone with my old chargers, but I haven t yet got a 1M long cable. Sometimes I need a cable that s longer than 10cm but shorter than 2M. The USB-C cables are all significantly thicker than older USB cables. Part of that would be due to having many more wires but presumably part of it would be due to having thicker power wires for delivering 3A. I haven t measured power draw but it does seem to charge faster than older phones. Overall the process of converting to USB-C is going to be a lot more inconvenient than USB SuperSpeed (which I could basically ignore as non-SuperSpeed connectors worked). It will be good when laptops with USB-C support become common, it should allow thinner laptops with more ports. One problem I initially had with my Samsung Galaxy Note 3 was the Micro-USB SuperSpeed socket on the phone being more fiddly for the Micro-USB charging plug I used. After a while I got used to that but it was still an annoyance. Having a symmetrical plug that can go into the phone either way is a significant convenience. Calendars and Contacts I share most phone contacts with my wife and also have another list that is separate. In the past I had used the Samsung contacts system for the contacts that were specific to my phone and a Google account for contacts that are shared between our phones. Now that I m using a non-Samsung phone I got another Gmail account for the purpose of storing contacts. Fortunately you can get as many Gmail accounts as you want. But it would be nice if Google supported multiple contact lists and multiple calendars on a single account. Samsung Galaxy S5 Mini Shortly after buying the Nexus 6P I decided that I spend enough time in pools and hot tubs that having a waterproof phone would be a good idea. Probably most people wouldn t consider reading email in a hot tub on a cruise ship to be an ideal holiday, but it works for me. The Galaxy S5 Mini seems to be the cheapest new phone that s waterproof. It is small and has a relatively low resolution screen, but it s more than adequate for a device that I ll use for an average of a few hours a week. I don t plan to get a SIM for it, I ll just use Wifi from my main phone. One noteworthy thing is the amount of bloatware on the Samsung. Usually when configuring a new phone I m so excited about fancy new hardware that I don t notice it much. But this time buying the new phone wasn t particularly exciting as I had just bought a phone that s much better. So I had more time to notice all the annoyances of having to download updates to Samsung apps that I ll never use. The Samsung device manager facility has been useful for me in the past and the Samsung contact list was useful for keeping a second address book until I got a Nexus phone. But most of the Samsung apps and 3d party apps aren t useful at all. It s bad enough having to install all the Google core apps. I ve never read mail from my Gmail account on my phone. I use Fetchmail to transfer it to an IMAP folder on my personal mail server and I d rather not have the Gmail app on my Android devices. Having any apps other than the bare minimum seems like a bad idea, more apps in the Android image means larger downloads for an over-the-air update and also more space used in the main partition for updates to apps that you don t use. Not So Exciting In recent times there hasn t been much potential for new features in phones. All phones have enough RAM and screen space for all common apps. While the S5 Mini has a small screen it s not that small, I spent many years with desktop PCs that had a similar resolution. So while the S5 Mini was released a couple of years ago that doesn t matter much for most common use. I wouldn t want it for my main phone but for a secondary phone it s quite good. The Nexus 6P is a very nice phone, but apart from USB-C, the fingerprint reader, and the lack of a stylus there s not much noticeable difference between that and the Samsung Galaxy Note 3 I was using before. I m generally happy with my Nexus 6P, but I think that anyone who chooses to buy a cheaper phone probably isn t going to be missing a lot.

• [1] https://etbe.coker.com.au/2016/06/01/i-just-ordered-a-nexus-6p/
• [2] https://play.google.com/store/apps/details?id=com.urbandroid.lux

A year ago I ve blogged about people requesting CVE without letting upstream know. On the other hand, per requests from Debian, I m working on improving PHP upstream CVE request process. For the last few release this means I ask the security list members which issues they think should have CVE and ask for them in parallel to the release being made (usually in the space between the release being tagged publicly and is actually announced). In the last week, I ve encountered a case where a few CVE were assigned to old PHP issues without any public notice. The fixes for these issues have been published a year ago (August 2015). And I find out about these assignment through warning published by the distributions (mostly Debian, which I m close to). Sometimes things fall between the chairs, and it s perfectly OK to ask for CVE to make sure security issues do get attention even if time has passed. But after the issues (and fixes) are public, I don t see a reason to do so without making the request itself public as well. And even if the request wasn t public, at least notify upstream so this info can be added to the right places. Most of these bug were found out when I started to add sequential number into the CVE search after getting an a notice from Debian for two of the issues.

• CVE-2015-8873 for PHP #69793
• CVE-2015-8874 for PHP #66387
• CVE-2015-8876 for PHP #70121
• CVE-2015-8877 for PHP #70064
• CVE-2015-8878 for PHP #70002
• CVE-2015-8879 for PHP #69975
• CVE-2015-8880 for PHP aa8cac57 (Dec 2015)

And while working on processing these issues for PHP, I also notice they weren t updated for libGD where appropriate (including recent issues). Beyond keeping the eco-system up to date, another aspect of publicity is getting opinions from other parties. For example, in the case of CVE-2015-8879, RedHat doesn t agree with the classification of the bug as security. To give an example of a way things should happen is the request of CVE for PHP 5.5.34 in April, in which the Gentoo security team asked for assignment without upstream knowledge, Debian representative CCs upstream (Thanks!) and also asks CVE for issues covered by Ubuntu, to which the Ubuntu guy then adds some details. I hope this blog post will reach the anonymous people behind these CVE requests, and also the people assigning them. Without transparency and keeping things in synchronization, the idea of having a centralized location for security warning is not going to accomplish its goals.
Filed under: Debian GNU/Linux, PHP

Introduction NX is a software suite which implements very efficient compression of the X11 protocol. This increases performance when using X applications over a network, especially a slow one. NX (v3) has been originally developed by NoMachine and has been Free Software ever since. Since NoMachine obsoleted NX (v3) some time back in 2013/2014, the maintenance has been continued by a versatile group of developers. The work on NX (v3) is being continued under the project name "nx-libs". History Until January 2015, nx-libs was more mainly a redistribution approach of the original NX (v3) software. We (we as in mainly a group of X2Go developers) kept changes applied to the original NoMachine sources as minimal as possible. We also kept the original files and folders structure. Patches had been maintained via the quilt utility on top of a Git repository, the patches had always been kept separate. That was the 3.5.0.x series of nx-libs. In January 2015, the characteristics of nx-libs as maintained by the X2Go project between 2011 and 2015 changed. A decision was reached:

• Drop quilt patching approach
• Apply all already used patches to the code via Git
• Clean-up the complete source code tree
• Start backporting code from X.org
• Add new features, upgrade API/ABI and X11 extensions

This effort is now about to be released as "nx-libs 3.6.0.0". Contributors Since 2011, the nx-libs code base has to a great extent been maintained in the context of the X2Go Project [1]. Qindel Group joining in... In 2014, developers from the Qindel Group [2] joined the nx-libs maintenance. They found X2Go's work on nx-libs and suggested joining forces as best as possible on hacking nx-libs. The Arctica Project comming up... Starting in January 2015, the development on the 3.6.x branch of the project was moved into a new project called the Arctica Project [3]. Development Funding by Qindel In September 2015, a funding project was set up at Qindel. Since then, the Qindel group greatly supports the development of nx-libs 3.6.x monetarily and with provided man power. The funding project officially is a cooperation between Qindel and DAS-NETZWERKTEAM (business run by Mike Gabriel, long term maintainer of nx-libs). The funding is split into two subprojects and lasts until August 2017:

• Work on nx-libs (coordinated by DAS-NETZWERKTEAM, Mike Gabriel)
• Work on multimedia support in remote graphical sessions (coordinated by Guangzhou Nianguan Electronics Technology Co.,Ltd)

The current nx-libs development effort is coordinated in the context of the Arctica Project (by Mike Gabriel), with use cases in Arctica, X2Go and TheQVD (VDI product worked on at Qindel) in mind. People involved There are various people involved in nx-libs 3.6.x maintenance and development, some of them shall be explicitly named here (in alphabetical order of surnames):

• Mario Becroft (long term NX contributor)
• Nicolas Arenas Alonso (funding project coordinator at Qindel)
• Mike DePaulo (CVE hunter)
• Salvador Fandi o (developer, code reviewer, provided by Qindel)
• Mike Gabriel (development coordination, developer, packager)
• The GZNGET FLOSS Team (Arctica Project developer, advisory)
• Jordi Marqu s (Arctica Project infrastructure maintainer)
• Mihai Moldovan (developer, code reviewer)
• Ulrich Sibiller (developer, code reviewer)
• Vadim Troshchinskiy (developer, code reviewer, provided by Qindel)

Some other people have contributed, but have left the project already. Thanks for your work on nx-libs:

• Fernando Carvajal (contributor, provided by Qindel)
• Reinhard Tartler (contributor to X2Go)

A big thanks to everyone involved!!! Special thanks go to Stefan Baur for employing Mihai Moldovan and handling all the bureaucracy, so that Mihai can work on this project and get funded for his work. Achievements of nx-libs 3.5.99.0 We are very close to our self-defined release goal 3.6.0. The below tasks have already been (+/-) completed for version 3.5.99.0:

• CVE security audit (complete)
• remove unused code (+/- complete)
• no bundled non-X11 libraries anymore (complete)
• complete imake cleanup (+/- complete)
• replace as many libNX_X - libraries by X.org's libX - libraries (complete, only remaining library: libNX_X11)
• support for iOS (nxproxy, complete)
• Unix file socket communication for nxproxy -C <-> nxproxy -S connections (complete)
• allow Unix file sockets as channel endpoints (complete)
• new RandR based Xinerama extension (+/- complete, more QA needed)
• nxcomp protocol clean-up (complete)
• RandR Xserver extension upgrade to protocol version 1.5 (complete, from version 1.2)
• Composite Xserver extension upgrade to proto version 0.4 (complete, from version 0.2)
• various ABI/API updates (with respect to X.org), so that extension backporting should be easy now (+/- complete)

In a previous blog post [4], the code reduction in nx-libs has already been discussed. With this announcement, we want to give a status update about our effort of shrinking the nx-libs code base:

    [mike@minobo nx-libs (3.6.x)]$ cloc --match-f '.*\.(c cpp h)' .
        1896 text files.
        1896 unique files.                                          
        7430 files ignored.
    http://cloc.sourceforge.net v 1.60  T=5.88 s (307.3 files/s, 143310.5 lines/s)
    -------------------------------------------------------------------------------
    Language                     files          blank        comment           code
    -------------------------------------------------------------------------------
    C                              958          68574          74891         419638
    C/C++ Header                   730          25683          33957         130418
    C++                            120          17007          11721          61292
    -------------------------------------------------------------------------------
    SUM:                          1808         111264         120569         611348
    -------------------------------------------------------------------------------

The previous statistics had these sums in the last line. First the nx-libs 3.5.0.x code tree (where we came from):

    -------------------------------------------------------------------------------
    SUM:                          5614         329279         382337        1757743
    -------------------------------------------------------------------------------

Then the nx-libs 3.6.x status as it was on May 9th 2016:

    -------------------------------------------------------------------------------
    SUM:                          1922         118581         126783         662635
    -------------------------------------------------------------------------------

Another 50.000 lines of code have been removed over the past two months. Work pending for the final 3.6.0 release goal

• allow embedding of nxproxy into other windows (work pending)
• Fix the Composite extension in nx-X11 Xserver (work pending)
• nxcomp logging clean-up (work pending)
• optimizing documentation: how to tune NX connections (work pending)
• XServer extension upgrades:
  • XInput 1.3 -> 1.4 (work pending)
  • Xfixes 3.0 -> 5.0 (work in progress)
  • Damage 1.0 -> 1.1 (work pending)
  • Xres 1.0 -> 2.0 (work pending)
  • Sync 1.0 -> 3.0 (work pending)
  • Render 0.10 -> 0.11 (work pending)
• Drop as many X11 related proto headers as possible

Known Issues There are several open issues on the nx-libs Github project space, see https://github.com/ArcticaProject/nx-libs/issues. Testing the nx-libs 3.5.99.0 We are currently working on provisioning release builds and nightly builds of nx-libs for various recent Linux distributions. Please stay tuned and watch Mike Gabriel's blog [5]. We already have nightly builds of nx-libs for Debian and Ubuntu [6], but there are more to come soon. Until now, please use the build recipes provided in the README.md file of the nx-libs source tree [7]. References

• [1] http://wiki.x2go.org
• [2] http://www.qindel.com
• [3] https://arctica-project.org
• [4] https://sunweavers.net/blog/node/39
• [5] https://sunweavers.net
• [6] https://sunweavers.net/blog/node/20
• [7] https://github.com/ArcticaProject/nx-libs/blob/3.6.x/README.md

With more than 10 million users, the Scratch online community is the largest online community where kids learn to program. Since it was created, a central goal of the community has been to promote remixing the reworking and recombination of existing creative artifacts. As the video above shows, remixing programming projects in the current web-based version of Scratch is as easy is as clicking on the see inside button in a project web-page, and then clicking on the remix button in the web-based code editor. Today, close to 30% of projects on Scratch are remixes. Remixing plays such a central role in Scratch because its designers believed that remixing can play an important role in learning. After all, Scratch was designed first and foremost as a learning community with its roots in the Constructionist framework developed at MIT by Seymour Papert and his colleagues. The design of the Scratch online community was inspired by Papert s vision of a learning community similar to Brazilian Samba schools (Henry Jenkins writes about his experience of Samba schools in the context of Papert s vision here), and a comment Marvin Minsky made in 1984:

Adults worry a lot these days. Especially, they worry about how to make other people learn more about computers. They want to make us all computer-literate. Literacy means both reading and writing, but most books and courses about computers only tell you about writing programs. Worse, they only tell about commands and instructions and programming-language grammar rules. They hardly ever give examples. But real languages are more than words and grammar rules. There s also literature what people use the language for. No one ever learns a language from being told its grammar rules. We always start with stories about things that interest us.

In a new paper titled Remixing as a pathway to Computational Thinking that was recently published at the ACM Conference on Computer Supported Collaborative Work and Social Computing (CSCW) conference, we used a series of quantitative measures of online behavior to try to uncover evidence that might support the theory that remixing in Scratch is positively associated with learning. Of course, because Scratch is an informal environment with no set path for users, no lesson plan, and no quizzes, measuring learning is an open problem. In our study, we built on two different approaches to measure learning in Scratch. The first approach considers the number of distinct types of programming blocks available in Scratch that a user has used over her lifetime in Scratch (there are 120 in total) something that can be thought of as a block repertoire or vocabulary. This measure has been used to model informal learning in Scratch in an earlier study. Using this approach, we hypothesized that users who remix more will have a faster rate of growth for their code vocabulary. Controlling for a number of factors (e.g. age of user, the general level of activity) we found evidence of a small, but positive relationship between the number of remixes a user has shared and her block vocabulary as measured by the unique blocks she used in her non-remix projects. Intriguingly, we also found a strong association between the number of downloads by a user and her vocabulary growth. One interpretation is that this learning might also be associated with less active forms of appropriation, like the process of reading source code described by Minksy. The second approach we used considered specific concepts in programming, such as loops, or event-handling. To measure this, we utilized a mapping of Scratch blocks to key programming concepts found in this paper by Karen Brennan and Mitchel Resnick. For example, in the image below are all the Scratch blocks mapped to the concept of loop . We looked at six concepts in total (conditionals, data, events, loops, operators, and parallelism). In each case, we hypothesized that if someone has had never used a given concept before, they would be more likely to use that concept after encountering it while remixing an existing project. Using this second approach, we found that users who had never used a concept were more likely to do so if they had been exposed to the concept through remixing. Although some concepts were more widely used than others, we found a positive relationship between concept use and exposure through remixing for each of the six concepts. We found that this relationship was true even if we ignored obvious examples of cutting and pasting of blocks of code. In all of these models, we found what we believe is evidence of learning through remixing. Of course, there are many limitations in this work. What we found are all positive correlations we do not know if these relationships are causal. Moreover, our measures do not really tell us whether someone has understood the usage of a given block or programming concept.However, even with these limitations, we are excited by the results of our work, and we plan to build on what we have. Our next steps include developing and utilizing better measures of learning, as well as looking at other methods of appropriation like viewing the source code of a project.

This blog post and the paper it describes are collaborative work with Sayamindu Dasgupta, Andr s Monroy-Hern ndez, and William Hale. The paper is released as open access so anyone can read the entire paper here. This blog post was also posted on Sayamindu Dasgupta s blog and on Medium by the MIT Media Lab.

My first Haskell program, and the only software I've written that was inspired by living in a particular place, git-annex has received the lion's share of my time for five years. It was written just to solve my own problem, but in a general way, that turned out to be useful in lots of other situations. So over the first half a year or so, it started attracting some early adopters who made some very helpful suggestions. Then I did the git-annex assistant kickstarter, and started blogging about each day I worked on it. Four years of funding and seven hundred and twenty one posts later, the git-annex devblog is still going. So, I won't talk about technical details in this post, they've all been covered. One thing I wondered when starting git-annex -- besides whether I would be able to write it in Haskell at all -- was would that prevent it from getting many patches. I count roughly 65 "thanks" messages in the changelog, so it gets perhaps one patch contributed per month. It's hard to say if that's a lot or a little. Part of git-annex is supporting various cloud storage systems via "special remotes". Of those not written by me, only 1 was contributed in Haskell. Compare with 13 that use the plugin system that lets other programming languages be used. The other question about using Haskell is, did it make git-annex a better program. I think it did. The strong type system did prevent plenty of bugs, although there have still been some real howlers. The code is still not taking full advantage of the power of Haskell's type system, on the other hand it uses many Haskell libraries that do leverage the type system more. I've done more small and large refactorings of git-annex than on any other program I've written, because the strong types and referential transparency makes refactoring easier and safer in Haskell. And the code has turned out to be much more flexible, for all its static types, than the kind of code I was writing before. Examples include building the git-annex assistant, which uses the rest of git-annex as a library, and making git-annex run actions concurrently, thanks to there being no global variables to complicate things (and excellent support for concurrency and parallelism in Haskell). So: Glad I wrote it, glad I used Haskell for it, estatic that many other people have found it useful, and astounded that I've been funded to work on it for four years. Next: twenty years of free software -- part 8 github-backup

Debconf streaming and kudos to the video team

With http://debconf16.debconf.org/ being in South Africa, a lot of people (like me) probably weren't able to attend and are missing the cheese and wine party, mao games and general socialization that is happening there.

One thing we don't have to miss, however, are the talks: as usual the video team https://wiki.debconf.org/wiki/DebConf16/Videoteam is doing a great job recording and https://debconf16.debconf.org/video/live-streaming/ all talks so that people can still participate a bit from their home.

What they do, however, requires a lot of manpower, so if you are attending Debconf please consider volunteering https://wiki.debconf.org/wiki/Videoteam/roles_for_volunteers to help: from my experience last year they are very nice people who are welcoming towards new contributors and they have periodical training sessions to help people getting started with the various tasks. More informations about video team meetings and training session are in the topic of the IRC channel, #debconf-video@OFTC.

I don't think there are cookies involved (which just proves that the video team isn't evil), but you may get a t-shirt and you will get a warm fuzzy feeling of having helped people around the world.

@Debian #debconf

Today was day one of the DebConf16 Open Festival and I attended the open hardware panel, part of the talk about Code For South Africa, shirish's experiences and the DebConf new folks session. The open hardware panel was a wide ranging discussion between bdale, Andy and indiebio. bdate talked about the experiences he has had with his rocketry hardware. bdale said "Make concious decisions about what you are buying", referencing a case where he investigated, found a GPL violation and didn't buy. Various people care about openness of different layers of the hardware. Off-the-shelf products are very strongly integrated, which is great for makers but means that people who care about lower layers like CPU micro-architecture aren't able to participate. Andy said "We are just beginning to come out of the shareware stage [of open hardware]". bdale mentioned the companies who do hardware production as a service from design files. Later in the pub some folks mentioned j-core, an open re-implementation of SuperH processors. I missed most of the code4sa talk unfortunately, but it was about government services and open data. shirish covered his journey through life to Debian. His youth, how satellite TV and knowledge of the outside world came to India around the time of the Iraq war. His experience accessing the Internet for the first time, uncensored vs the usual censorship in India's media. His experiences of Windows 95 viruses and crashes. He learned of PCTwist Linux through a magazine cover. His initial install was not a success but eventually managed to break through and install a desktop, but experienced network and other issues. Eventually he encountered Ubuntu and began contributing bug reports. His experiences there led him to Debian. He began blogging about Debian. In the last few years he and others have been going around the country doing mini-DebConfs at institutes around India. The first question was predictably about having a DebConf in India and how shirish might like to get more involved. DebConf in India sounds like a possibility some day and shirish was thinking about getting involved in publicity, marketting and the Debian installer. The DebConf new folks session was a great intro to DebConf for folks new to the community. There were some quite excellent touches added to this year's version of the event by indiebio and Rhonda. I also got some things done. Usual spam reporting. Reviewed wiki RecentChanges. Talked to the chromium-bsu/MacPorts maintainer about AX_CHECK_GL brokenness. Filed Debian wishlist bug #829292 asking to update autoconf-archive. Redirected a Hurd porterbox request to the exodar admin and quickly found out I was wrong to do that, rectified. Then we found out the LDAP sync to exodar was broken. Replied to someone who intends to sell Debian pre-installs. Thanked BunsenLabs folks for joining the derivatives list. Applied reproducible builds patch for cats from Chris Lamb. Heard about awesome new terminal-mode screensaver. Moo! Prepared a blog post about check-all-the-things.

Review: Coming Home, by Jack McDevitt

Series:	Alex Benedict #7
Publisher:	Ace
Copyright:	November 2014
Printing:	November 2015
ISBN:	0-425-26088-7
Format:	Mass market
Pages:	356

Coming Home is a direct sequel to Firebird, the first time McDevitt has done that in this series. You therefore don't want to start here, although the nature of the sequel doesn't require that you remember Firebird in that much detail. The mystery of the disappearing starships was understood in Firebird but not resolved. The title advertises that as a major theme in this book, but it progresses very slowly. There's more media bickering and various factional attempts to draw Alex (and, to a lesser extent, Chase) into the controversies. McDevitt does a good job writing popular media, the strange position of public intellectual and talk-show favorite, and the way this filters into popular arguments. But with Alex trying to take a nuanced and unsure position and with a lot of talk but little action, it's not the most compelling reading. Coming Home holds to form in balancing a mystery and a second plot. Since the starship problem is understood, it can't be the central mystery of the book. That role is taken by the discovery of communication device from the very early days of space flight, which takes Alex and Chase to Earth for the first time in this series (at least that I can recall). This time, the search is for a legendary trove of historical artifacts that was moved from a space flight museum in Florida when the ocean rose to cover the state. From there, its location was lost in the middle of a general economic collapse called the Time of Troubles that destroyed most of Earth's governmental systems (and, apparently rather more importantly to McDevitt, the space program). Anyone who has gotten this far in the series will know the standard problem with McDevitt's futures: they're indistinguishable from the 1960s except that they have flying cars. There is a tiny break from that tradition here, since climate change has clearly happened to Earth, covering Florida with the ocean and shifting the major cities north. But the sense of deep history that McDevitt is trying for in this series doesn't work: I just don't believe as much time has gone by as the story claims. He does offer an explanation for why technology has been stagnant for apparently millennia, but it's just a contention that science ran out of more things to discover due to authorial fiat and is now just a matter of engineering and step-wise refinement. (I think the science behind the disappearing starships happening in this very same book undermines that contention considerably.) Even if one can put that aside, Earth is, well, boring. This is partly an intriguing stylistic choice by McDevitt: Earth is intended to be just one more world. It might have a longer history than many other human-occupied worlds, but history is so long everywhere that this only matters to a few people like Alex and Chase. The choice makes sense, but it doesn't make a good story. And there are other things that I flatly didn't believe, such as the supposed isolation of Earth from the communication network of Alex's home world of Rimway for... no apparent reason other than that different communication networks don't talk to each other except via, essentially, letters. Even if one is willing to ignore the mysterious failure of forward progress in technology, this makes no social or engineering sense given the capabilities already shown in the series. I found the main mystery at best mildly interesting. McDevitt does a good job showing the research and investigation process, with all its tedium and dead ends, but I wasn't as invested in the search as he wanted me to be. The endless space boosterism started getting on my nerves, as it so often does in science fiction of a certain type, and I had a much harder time swallowing McDevitt's Earth than his fictional Rimway. I will give him credit for a surprisingly affecting conclusion of the main mystery, but the missing starship subplot putters to an undistinguished end. I think this series is running out of steam. It's become increasingly formulaic, and the characters, like McDevitt's future technology, have stopped developing. I was hoping the reunion foreshadowed since Firebird would shake things up, but at least in this book it doesn't. Maybe it will in subsequent books, but I'm starting to question whether I really want to keep reading. Rating: 5 out of 10

11. Why use a genetic algorithm instead of a backpropagation algorithm to train a neural net ? 12. What is the difference between a support vector machine (SVM) and an artificial neural network (ANN) ? 13. What s a Naives Bayes classifier ? 14. What is the difference between supervised and unsurpervised learning 15. What is Continue reading Tech questions 11-16: Artificial intelligence