Search Results: "will"

21 November 2016

Mike Gabriel: Please Welcome D0n1elT to the FLOSS World

TL;DR; If you run a FLOSS development project and you notice D0n1elT appearing on your IRC channel, please give him a warm welcome. D0n1elT is a young man highy talented in various FLOSS related topics already. He probably needs some guidance at the beginning and I hope he won't be too shy to ask for it. But you can be sure: your channel has been joined by someone you should consider as a future resource. The Long Story During the last two weeks I had the great pleasure of supervising a fine young man (very young, still, indeed) in all sorts of IT topics. This young man turned out to be so skilled and interested in various FLOSS related areas, I really want to introduce him to all of you. The young man's real name is Daniel Teichmann. On IRC he may appear under his nick: D0n1elT. His GnuPG Fingerprint is: 6C6E 7F8F F7E8 B22E FC76 E9F7 8A79 028F DA56 7C6C. Daniel goes to a local school here in Nothern Germany, near where I live. He attends the 9th grade at his school, and as common for students of his age and grade, practical training was scheduled for the last two weeks. Daniel had originally applied for practical training at some other business near his place of living (which is quite far off from the school, actually). However, that company cancelled his training position two work days before the training was supposed to start. Daniel's teacher rang me up and asked for help. He advertised Daniel as someone who is far advanced in IT topics compared to his co-students. "He even writes his own programs (in Java and C++)." Spontaneously, Andreas Buchholz (CEO of LOGO EDV-Systeme GmbH) and I decided to accept Daniel as a trainee. Without having met him, with no application interview beforehand. The deal was: Daniel comes to Andreas business location in Kiel (40-50km away from Daniel's place of living) and I (working as freelancer for LOGO on a regular basis) do the supervising part. On day one and two, as a warm-up, Daniel installed a Debian Edu Main Server, worked himself through GOsa, LDAP, SSH, GnuPG, Jabber and IRC and configured two routers. All topics were new to him and I could hardly think of new tasks to give to him. As means of communication we set up a Jabber account, then an IRC account (as backup). However, it turned out that Daniel really got a hang of IRC over the next couple of days, so we used that as primary communication channel. Daniel had already programmed various projects in Java (whereas I have never touched Java, so far :-( ). He has written plugins for Minecraft servers. He knows well how to implement object oriented coding models. His coding style looks very good and clean (esp. for someone who has never head a nitpicking code reviewer). He started coding at the age of 9. Instead of diving into Java (where I would not have been of much help, anyway) I decided to provide him with some really basic and Unix-like knowledge: Bash scripting. I wanted to see how he handles another "language" and how he applies his Java knowledge to a lower level, syntactically weaker language. Guess what, he managed that assignment very well. Working on Impressive Display At Daniel's school we run substitute teacher info screens based on a fancy Bash script, named impressive-display, and the impressive PDF viewer. The Impressive Display tool is available in Debian testing/unstable under the same name. So over the next couple of days we worked on Impressive Display. Daniel contributed so many new code passages, conceptual ideas and security concerns, that I decided to make him co-copyright holder. Every change contributed by him received intensive testing before committing to Git. While working on Impressive Display, collaborating with Daniel via Git was a mere pleasure. In his spare time Daniel likes watching Github tutorials. Quite extraordinary. The result is a new major release of Impressive Display: Version 0.3.1 (bumped up from 0.2.3). We added the feature of handling info screen farms based on PXE boot images. It is now possible to configure as many different info screens as needed within the same PXE bootable chroot. Furthermore, Impressive Display now has a PDF presentation (written in LaTeX Beamer) that documents how to setup your own info screens. The PDF presentation is the default PDF that comes up when you start Impressive Display directly after installation. Investigating other Realms We also took a deeper look at remote desktop stuff, one of my most favourite topics. By that impulse Daniel set up his first Vserver machine at some hosting provider. He figured out how to run X2Go Server on that machine with an XFCE desktop. Next step was to run the irssi instance from his notebook inside a screen session on the Vserver. Some days later, Daniel PM'ed me: "I have an IRC bouncer now...". Quintessence It was a great pleasure meeting this young, highly curious and already highly skilled young man over the past two weeks. Daniel, it was an asset to me working with you. You are such a fast learner when it comes to getting accustomed to new working environments, it is amazing. I cannot deny having observed the tendency of preferring rather geeky tools. I was highly delighted, that What's-That and Facebook are nothing that rocks you so much. Unfortunately, all of the above makes you quite unique and non-mainstream among people of your age. My wish for you (and the FLOSS world) is that you start getting in touch with other (FLOSS) developers, maybe of your age, maybe older, and that you (if this is what you want) become an asset to the world of Free Software. The Free Software world can be a world where technical, political and spiritual work become one with friendship among people. Take care and farewell! I am sure, we will meet again. light+love Mike Gabriel (aka sunweaver on IRC and debian.org)

Reproducible builds folks: Reproducible Builds: week 82 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday November 13 and Saturday November 19 2016: Media coverage Elsewhere in Debian Documentation update Packages reviewed and fixed, and bugs filed Reviews of unreproducible packages 43 package reviews have been added, 4 have been updated and 12 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been updated: 4 issue types have been added: Weekly QA work During our reproducibility testing, some FTBFS bugs have been detected and reported by: strip-nondeterminism development disorderfs development debrebuild development debrebuild is new tool proposed by HW42 and josch (see #774415: "From srebuild sbuild-wrapper to debrebuild"). debrepatch development debrepatch is a set of scripts that we're currently developing to make it easier to track unapplied patches. We have a lot of those and we're not always sure if they still work. The plan is to set up jobs to automatically apply old reproducibility patches to newer versions of packages and notify the right people if they don't apply and/or no longer make the package reproducible. debpatch is a component of debrepatch that applies debdiffs to Debian source packages. In other words, it is to debdiff(1) what patch(1) is to diff(1). It is a general tool that is not specific to Reproducible Builds. This week, Ximin Luo worked on making it more "production-ready" and will soon submit it for inclusion in devscripts. reprotest development Ximin Luo significantly improved reprotest, adding presets and auto-detection of which preset to use. One can now run e.g. reprotest auto . or reprotest auto $pkg_$ver.dsc instead of the long command lines that were needed before. He also made it easier to set up build dependencies inside the virtual server and made it possible to specify pre-build dependencies that reprotest itself needs to set up the variations. Previously one had to manually edit the virtual server to do that, which was not very usable to humans without an in-depth knowledge of the building process. These changes will be tested some more and then released in the near future as reprotest 0.4. tests.reproducible-builds.org Misc. This week's edition was written by Chris Lamb, Holger Levsen, Ximin Luo and reviewed by a bunch of Reproducible Builds folks on IRC.

Steve Kemp: Detecting fraudulent signups?

I run a couple of different sites that allow users to sign-up and use various services. In each of these sites I have some minimal rules in place to detect bad signups, but these are a little ad hoc, because the nature of "badness" varies on a per-site basis. I've worked in a couple of places where there are in-house tests of bad signups, and these usually boil down to some naive, and overly-broad, rules: Some places penalise users based upon location too: At the moment I've got a simple HTTP-server which receives a JSON post of a new users' details, and returns "200 OK" or "403 Forbidden" based on some very very simple critereon. This is modeled on the spam detection service for blog-comments server I use - something that is itself becoming less useful over time. (Perhaps time to kill that? A decision for another day.) Unfortunately this whole approach is very reactive, as it takes human eyeballs to detect new classes of problems. Code can't guess in advance that it should block usernames which could collide with official ones, for example allowing a username of "admin", "help", or "support". I'm certain that these systems have been written a thousand times, as I've seen at least five such systems, and they're all very similar. The biggest flaw in all these systems is that they try to classify users in advance of them doing anything. We're trying to say "Block users who will use stolen credit cards", or "Block users who'll submit spam", by correlating that behaviour with other things. In an ideal world you'd judge users only by the actions they take, not how they signed up. And yet .. it is better than nothing. For the moment I'm continuing to try to make the best of things, at least by centralising the rules for myself I cut down on duplicate code. I'll pretend I'm being cool, modern, and sexy, and call this a micro-service! (Ignore the lack of containers for the moment!)

20 November 2016

Steinar H. Gunderson: Nageru documentation

Even though the World Chess Championship takes up a lot of time these days, I've still found some time for Nageru, my live video mixer. But this time it doesn't come in form of code; rather, I've spent my time writing documentation. I spent some time fretting over what technical solution I wanted. I explicitly wanted end-user documentation, not developer documentation I rarely find HTML-rendered versions of every member function in a class the best way to understand a program anyway. Actually, on the contrary: Having all sorts of syntax interwoven in class comments tends to be more distracting than anything else. Eventually I settled on Sphinx, not because I found it fantastic (in particular, ReST is a pain with its bizarre variable punctuation-based syntax), but because I'm convinced it has all the momentum right now. Just like git did back in the day, the fact that the Linux kernel has chosen it means it will inevitably grow a quite large ecosystem, and I won't be ending up having to maintain it anytime soon. I tried finding a balance between spending time on installation/setup (only really useful for first-time users, and even then, only a subset of them), concept documentation (how to deal with live video in general, and how Nageru fits into a larger ecosystem of software and equipment) and more concrete documentation of all the various features and quirks of Nageru itself. Hopefully, most people will find at least something that's not already obvious to them, without drowning in detail. You can read the documentation at https://nageru.sesse.net/doc/, or if you want to send patches, the right place to patch is the git repository.

19 November 2016

Keith Packard: AltOS-Lisp-2

Updates to Altos Lisp I wrote a few days ago about a tiny lisp interpreter I wrote for AltOS Really, it's almost "done" now, I just wanted to make a few improvements Incremental Collection I was on a walk on Wednesday when I figured out that I didn't need to do a full collection every time; a partial collection that only scanned the upper portion of memory would often find plenty of free space to keep working for a while. To recap, the heap is in two pieces; the ROM piece and the RAM piece. The ROM piece is generated during the build process and never changes afterwards (hence the name), so the only piece which is collected is the RAM piece. Collection works like: 
chunk_low = heap base
new_top = heap base
For all of the heap
    Find the first 64 live objects above chunk_low
    Compact them all to new_top
    Rewrite references in the whole heap for them
    Set new_top above the new locations
    Set chunk_low above the old locations
top = new_top
The trick is to realize that there's really no need to start at the bottom of the heap; you can start anywhere you like and compact stuff, possibly leaving holes below that location in the heap. As the heap tends to have long-lived objects slowly sift down to the beginning, it's useful to compact objects higher than that, skipping the compaction process for the more stable area in memory. Each time the whole heap is scanned, the top location is recorded. After that, incremental collects happen starting at that location, and when that doesn't produce enough free space, a full collect is done. The collector now runs a bunch faster on average now. Binary Searches I stuck some linear searches in a few places in the code, the first was in the collector when looking to see where an object had moved to. As there are 64 entries, the search is reduced from 32 to 6 compares on average. The second place was in the frame objects, which hold the list of atom/value bindings for each lexical scope (including the global scope). These aren't terribly large, but a binary search is still a fine plan. I wanted to write down here the basic pattern I'm using for binary searches these days, which avoids some of the boundary conditions I've managed to generate in the past: 
int find (needle)  
    int l = 0;
    int r = count - 1;
    while (l <= r)  
        int m = (l + r) >> 1;
        if (haystack[m] < needle)
            l = m + 1;
        else
            r = m - 1;
     
    return l;
With this version, the caller can then check to see if there's an exact match, and if not, then the returned value is the location in the array where the value should be inserted. If the needle is known to not be in the haystack, and if the haystack is large enough to accept the new value: 
void insert(needle)  
    int l = find(needle);
    memmove(&haystack[l+1],
        &haystack[l],
        (num - l) * sizeof (haystack[0]));
    haystack[l] = needle;
Similarly, if the caller just wants to know if the value is in the array: 
bool exists(needle)  
    int l = find(needle);
    return (l < count && haystack[l] == needle);
Call with Current Continuation Because the execution stack is managed on the heap, it's completely trivial to provide the scheme-like call with current continuation, which constructs an object which can be 'called' to transfer control to a saved location: 
> (+ "hello " (call/cc (lambda (return) (setq boo return) (return "foo "))) "world")
"hello foo world"
> (boo "bar ")
"hello bar world"
> (boo "yikes ")
"hello yikes world"
One thing I'd done previously is dump the entire state of the interpreter on any error, and that included a full stack trace. I adopted that code for printing of these continuation objects: 
boo
[
    expr:   (call/cc (lambda (return) (set (quote boo) return) (return "foo ")))
    state:  val
    values: (call/cc
             [recurse...]
             )
    sexprs: ()
    frame:   
]
[
    expr:   (+ "hello " (call/cc (lambda (return) (set (quote boo) return) (return "foo "))) "world")
    state:  formal
    values: (+
             "hello "
             )
    sexprs: ((call/cc (lambda (return) (set (quote boo) return) (return "foo ")))
             "world"
             )
    frame:   
]
The top stack frame is about to return from the call/cc spot with a value; supply a value to 'boo' and that's where you start. The next frame is in the middle of computing formals for the + s-expression. It's found the + function, and the "hello " string and has yet to get the value from call/cc or the value of the "world" string. Once the call/cc "returns", that value will get moved to the values list and the sexpr list will move forward one spot to compute the "world" value. Implementing this whole mechanism took only a few dozen lines of code as the existing stack contexts were already a continuation in effect. The hardest piece was figuring out that I needed to copy the entire stack each time the continuation was created or executed as it is effectively destroyed in the process of evaluation. I haven't implemented dynamic-wind yet; when I did that for nickle, it was a bit of a pain threading execution through the unwind paths. Re-using Frames I decided to try and re-use frames (those objects which hold atom/value bindings for each lexical scope). It wasn't that hard; the only trick was to mark frames which have been referenced from elsewhere as not-for-reuse and then avoid sticking those in the re-use queue. This reduced allocations even further so that for simple looping or tail-calling code, the allocator may never end up being called. How Big Is It? I've managed to squeeze the interpreter and all of the rest of the AltOS system into 25kB of Cortex-M0 code. That leaves space for the 4kB boot loader and 3kB of flash to save/restore the 3kB heap across resets. Adding builtins to control timers and GPIOs would make this a reasonable software load for an Arduino; offering a rather different programming model for those with a taste for adventure. Modern ARM-based Arduino boards have plenty of flash and ram for this. It might be interesting to get this running on the Arduino Zero; there's no real reason to replace the OS either; porting the lisp interpreter into the bare Arduino environment wouldn't take long.

17 November 2016

Reproducible builds folks: Reproducible Builds: week 81 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday November 6 and Saturday November 12 2016: Media coverage Matthew Garrett blogged about Tor, TPMs and service integrity attestation and how reproducible builds are the base for systems integrity. The Linux Foundation announced renewed funding for us as part of the Core Infrastructure Initiative. Thank you! Outreachy updates Maria Glukhova has been accepted into the Outreachy winter internship and will work with us the Debian reproducible builds team. To quote her words 
siamezzze: I've been accepted to #outreachy winter internship - going to
work with Debian reproducible builds team. So excited about that! <3
Debian
Toolchain development and fixes dpkg: debrebuild: Bugs filed Chris Lamb: Daniel Shahaf: Niko Tyni: Reiner Herrman: Reviews of unreproducible packages 136 package reviews have been added, 5 have been updated and 7 have been removed in this week, adding to our knowledge about identified issues. 3 issue types have been updated: Weekly QA work During of reproducibility testing, some FTBFS bugs have been detected and reported by: diffoscope development A new version of diffoscope 62~bpo8+1 was uploaded to jessie-backports by Mattia Rizzolo. Meanwhile in git, Ximin Luo greatly improved speed by fixing a O(n2) lookup which was causing diffs of large packages such as GCC and glibc to take many more hours than was necessary. When this commit is released, we should hopefully see full diffs for such packages again. Currently we have 197 source packages which - when built - diffoscope fails to analyse. buildinfo.debian.net development tests.reproducible-builds.org Debian: reproducible-builds.org website F-Droid was finally added to our list of partner projects. (This was an oversight and they had already been working with us for some time.) Misc. This week's edition was written by Ximin Luo and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

Gunnar Wolf: Book presentation by @arenitasoria: Hacker ethics, security and surveillance

At the beginning of this year, Irene Soria invited me to start a series of talks on the topic of hacker ethics, security and surveillance. I presented a talk titled Cryptography and identity: Not everything is anonymity. The talk itself is recorded and available in archive.org (sidenote: I find it amazing that Universidad del Claustro de Sor Juana uses archive.org as their main multimedia publishing platform!) But as part of this excercise, Irene invited me to write a chapter for a book covering the series. And, yes, she delivered! So, finally, we will have the book presentation: I know, not everybody following my posts (that means... Only those at or near Mexico City) will be able to join. But the good news: The book, as soon as it is presented, will be published under a CC BY-SA license. Of course, I will notify when it is ready.

Mart n Ferrari: Replacing proprietary Play Services with MicroG

For over a year now, I have been using CyanogenMod in my Nexus phone. At first I just installed some bundle that brought all the proprietary Google applications and libraries, but later I decided that I wanted more control over it, so I did a clean install with no proprietary stuff. This was not so great at the beginning, because the base system lacks the geolocation helpers that allow you to get a position in seconds (using GSM antennas and Wifi APs). And so, every time I opened OsmAnd (a great maps application, free software and free maps), I would have to wait minutes for the GPS radio to locate enough satellites. Shortly after, I found about the UnifiedNLP project that provided a drop-in replacement for the location services, using pluggable location providers. This worked great, and you could choose to use the Apple or Mozilla on-line providers, or off-line databases that you could download or build yourself. This worked well for most of my needs, and I was happy about it. I also had F-droid for installing free software applications, DAVdroid for contacts and calendar synchronisation, K9 for reading email, etc. I still needed some proprietary apps, but most of the stuff in my phone was Free Software. But sadly, more and more application developers are buying into the vendor lock-in that is Google Play Services, which is a set of APIs that offer some useful functionality (including the location services), but that require non-free software that is not part of the AOSP (Android Open-Source Project). Mostly, this is because they make push notifications a requirement, or because they want you to be able to buy stuff from the application. This is not limited to proprietary software. Most notably, the Signal project refuses to work without these libraries, or even to distribute the pre-compiled binaries on any platform that is not Google Play! (This is one of many reason why I don't recommend Signal to anybody). And of course, many very useful services that people use every day require you to install proprietary applications, which care much less about your choices of non-standard platforms. For the most part, I had been able to just get the package files for these applications1 from somewhere, and have the functionality I wanted. Some apps would just work perfectly, others would complain about the lack of Play Services, but offer a degraded experience. You would not get notifications unless you re-opened the application, stuff like that. But they worked. Lately, some of the closed-source apps I sometimes use stopped working altogether. So, tired of all this, I decided to give the MicroG project a try.

MicroG MicroG is a direct descendant of UnifiedNLP and the NOGAPPS project. I had known about it for a while, but the installation procedures always put me off. LWN published an article about them recently, and so I decided to spend a good chunk of time making it work. This blog post is to help making this a bit less painful. Some prerequisites:
  • No Gapps installed.
  • Rooted phone (at least for the mechanism I describe here).
  • Working ADB with root access to the phone.
  • UnifiedNLP needs to be uninstalled (MicroG carries its own version of it).

Signature spoofing The main problem with the installation, is that MicroG needs to pretend to be the original Google bundle. It has to show the same name, but most importantly, it has to spoof its cryptographic signatures so other applications do not realise it is not the real thing. OmniROM and MarshROM (other alternative firmwares for Android) provide support for signature spoofing out of the box. If you are running these, go ahead and install MicroG, it will be very easy! Sadly, the CyanogenMod refused allowing signature spoofing, citing security concerns2, so most users will have to go the hard way. The options for enabling this feature are basically two: either patch some core libraries H4xx0r style, or use the "Xposed framework". Since I still don't really understand what this Xposed thing is, and it is one of these projects that distributes files on XDA forums, I decided to go the dirty way.

Patching the ROM Note: this method requires a rooted phone, java, and adb. The MicroG wiki links to three different projects for patching your ROM, but turns out that two of them would not work at all with CyanogenMod 11 and 13 (the two versions I tried), because the system is "odexed" (whatever that means, the Android ecosystem is really annoying). I actually upgraded CM to version 13 just to try this, so a couple of hours went there, with no results. The project that did work is Haystack by Lanchon, which seems to be the cleaner and better developed of the three. Also the one with the most complex documentation. In a nutshell, you need to download a bunch of files from the phone, apply a series of patches to it, and then reupload it. To obtain the files and place them in the maguro (the codename for my phone) directory: 
$ ./pull-fileset maguro
Now you need to apply some patches with the patch-fileset script. The patches are located in the patches directory: 
$ ls patches
sigspoof-core
sigspoof-hook-1.5-2.3
sigspoof-hook-4.0
sigspoof-hook-4.1-6.0
sigspoof-hook-7.0
sigspoof-ui-global-4.1-6.0
sigspoof-ui-global-7.0
The patch-fileset script takes these parameters: 
patch-fileset <patch-dir> <api-level> <input-dir> [ <output-dir> ]
Note that this requires knowing your Android version, and the API level. If you don't specify an output directory, it will append the patch name to the input directory name. Note that you should also check the output of the script for any warnings or errors! First, you need to apply the patch that hooks into your specific version of Android (6.0, API level 23 in my case): 
$ ./patch-fileset patches/sigspoof-hook-4.1-6.0 23 maguro
>>> target directory: maguro__sigspoof-hook-4.1-6.0
[...]
*** patch-fileset: success
Now, you need to add the core spoofing module: 
$ ./patch-fileset patches/sigspoof-core 23 maguro__sigspoof-hook-4.1-6.0
>>> target directory: maguro__sigspoof-hook-4.1-6.0__sigspoof-core
[...]
*** patch-fileset: success
And finally, add the UI elements that let you enable or disable the signature spoofing: 
$ ./patch-fileset patches/sigspoof-ui-global-4.1-6.0 23 maguro__sigspoof-hook-4.1-6.0__sigspoof-core
>>> target directory: maguro__sigspoof-hook-4.1-6.0__sigspoof-core__sigspoof-ui-global-4.1-6.0
[...]
*** patch-fileset: success
Now, you have a bundle ready to upload to your phone, and you do that with the push-fileset script: 
$ ./push-fileset maguro__sigspoof-hook-4.1-6.0__sigspoof-core__sigspoof-ui-global-4.1-6.0
After this, reboot your phone, go to settings / developer settings, and at the end you should find a checkbx for "Allow signature spoofing" which you should now enable.

Installing MicroG Now that the difficult part is done, the rest of the installation is pretty easy. You can add the MicroG repository to F-Droid and install the rest of the modules from there. Check the installation guide for all the details. Once all the parts are in place, and after a last reboot, you should find a MicroG settings icon that will check that everything is working correctly, and give you the choice of which components to enable. So far, other applications believe this phone has nothing weird, I get quick GPS fixes, push notifications seem to work... Not bad at all for such a young project! Hope this is useful. I would love to hear your feedback in the comments!
  1. Which is a pretty horrible thing, having to download from fishy sites because Google refuses to let you use the marketplace without their proprietary application. I used to use a chrome extension to trick Google Play into believing your phone was downloading it, and so you could get the APK file, but now that I have no devices running stock Android, that does not work any more.
  2. Actually, I am still a bit concerned about this, because it is not completely clear to me how protected this is against malicious actors (I would love to get feedback on this!).
3 comments

16 November 2016

Shirish Agarwal: The long tail in a common s man journey to debconf16 2

This is an extension of part 1 which I shared few days ago. This would be a longish one so please bear. First of all somebody emailed me this link so in the future a layover at Doha Airport will be a bit expensive from before, approx INR 700/- added to the ticket costs Moving on, Let me share an experience I shared one of the last few days I had while I was in Cape Town
Singer singing some great oldies from 60's , 70's till 90's.

Singer singing some great oldies from 60 s , 70 s till 90 s.

I had booked a place near Long Street, Cape Town using Bernelle s help. What I had not known at that time that near Long Street there are free walking tours every couple of hours. I took part in all the tours and those were nice experiences. Where they start the walk, there was the gentleman pictured above. I was amazed by this gentleman s rich voice. He strummed lot of classics from the 60 s, 70 s till the 90 s . I had two coffees and thought I was at a premium rock concert. It was a bitter-sweet experience for me because I could see that he has such prodigious talent and still he had to struggle to survive to make ends meet. I did my 2 bit but wish I could have done something more. Side note Before I forget there is one trick of feh which I use to view images without it getting very high-resolution (especially on my low-end systems) [shirish@debian] - [/run/user/1000/gvfs/mtp:host=%5Busb%3A001%2C006%5D/Card/DCIM/Camera] - [4621]
[$] feh -g 1350x1000 .
 This actually makes it far far easier to traverse through the 1000 odd photos of the trip that I have in my personal archive without doing any sort of conversion methodology. Btw, it took me time but finally was able to create an album at gallery.debconf.org . Haven t been able to upload photos as came across an error which I have shared at https://lists.debconf.org/lurker/message/20161113.215659.fce58823.en.html Moving on, here s the funny story/experience I wanted to share could have been arrested ;) What happened was this. This is from the Doha Airport. I had seen big buggies (ones similar to golf carts) which was ferrying people from end of the concourse to the other. I had been walking the whole day and even with the horizontal escalators and everything, it takes a toll. I was half-tired, half-sleepy and saw a buggy stationed. From behind it looked like the buggies I had seen. As there was no place to park my behind there, I entered into the buggy and sat there. Around 15-20 minutes later a Doha cop in another buggy came to me and asked me if something had happened ? I had no clue what he was talking about. He told/shared/asked me in friendly tone whether I had committed a crime or wanted to report a crime. When I replied in negative to both, he asked then why I was sitting there. I replied it was for stretching my legs and it was the buggy which was being used to transport people from A. to B. He gently told me I had entered into the wrong one and it was actually a cop buggy. I couldn t believe it. He did go his own way as he saw I was dead-tired. After 10-15 minutes, half-believingly I came out of the buggy and to my shock the gentleman was right. There was nothing to do but solder on to find a spot in this big airport. I shared this with few friends and family and managed to elicit few laughs hence sharing. The somewhat sad one was I had met a couple with a baby. Now as shared before, Most Airports including the Doha Airport is Air-Conditioned/Climate-Controlled and is probably in mid-20 s so it was more than cold for me. The couples with the baby were from Asian sub-continent. From their clothes and the way they were, they were not very well off. I do remember them sharing that they had a death in the family and hence were going. I didn t know at that point in time that there was something called bereavement fares and if they were able to take opportunity of those tickets. But this is besides the point . The issue was that their baby had been running a high-fever and the A/C was making matters worse. I had seen a pharmacy but no clinic in the airport. It was much later I came across http://dohahamadairport.com/airport-guide/facilities-services/medical-emergencies but as can be seen on the web-page it doesn t tell whether the services are chargeable or not. I assume it would be paid, although in some of the developed/industrialized countries it is rumoured not to be for simple ailments such as the baby was going through. Have no idea if that s true or not. I also don t know how it equates with travel insurance as well as most travel insurance is also supposed to help you in situations like these. I was concerned as it was a baby and babies as all know are very very fragile. If anybody has an idea or had similar experience would like to know specifically related to International Airport environment as it has transit issues unlike in domestic airports where I don t think it would be a bit more easy. Now coming to my own inadequacies/lack of foresight which I had mentioned I will share, I had asked/queried and got to lead a Debian-installation workshop on the Open Debian Day. I had done a few earlier and had installed it a few times on my system and for my friends, relatives and some clients. The only bad experiences I had were to do with UEFI but even those in the jessie releases had got resolved quite a bit, so was pretty confident. The day before the Installfest was to happen, Mensah Nyarko Yaa Dufie (one full name) of Ghana approached me to install Debian on her system. I had some older version of the Debian DVD either 8.1 or 8.3 and had known that 8.5 had been released just a few days back. Had seen pretty fast internet (as far as downloading Debian DVD) is concerned hence asked her to wait a bit while I downloaded the newest image. I sha256summed it to make sure that the image was bit-to-bit perfect. Now I hadn t bought a pen drive/disk from India as I was under the impression that in such conferences, pen drives should not be an issue. I had asked Bernelle privately before via e-mail as well and she had assured me that some pen-drives would be available. She gave me a handful of HP pen drives. The pen drives as we came know during our usage were somewhat flaky. It would pop out/lose connection even with the slightest nudge to the lappy. Somehow I was able to transfer the image to the usb disk. As people say hindsight is 50:50 maybe it was not such a smart move on my part to download the big DVD image and maybe I should have got the netinstall iso . Be careful, the link I have just shared is of the old version, if you have good web link and want to try the newest stable netinstall head to cdimage.debian.org . Apart from that goof-up I dunno (still) of anyway to know if a copy from an .iso image to usb was successful or not and did it do correctly I did the following command sudo dd if=/path_to/debian-dvd.iso of=/dev/usb-mount-point which is usually /dev/sdb on all of my systems . Her system was a brand new HP (don t remember the model details) which she had bought just a few weeks/months before debconf. We tried a few times but it failed at installing the boot-loader stage. I asked Ritesh Raj Saraff (a friend and DD) and while he had some ideas, none of them worked. Ritesh later pointed out Steve McIntyre and shared he is part of the Debian-Installer team. At that point in time, I had no clue who Steve McIntyre was otherwise I probably would not have approached him. He quickly acquiesced to my request and shared that he would be there for the workshop. With load of my mind little bit, I apologized to mensah and asked her to be at the workshop the following day. I had no clue what was wrong at this point in time, whether it was the iso image in the usb disk or a UEFI issue. This also wasn t good for my confidence but as somebody from the Debian-Installer team was there, I was somewhat relaxed. Next day, some more people came for the Installfest. While I had made 2-3 copies, clearly it was not enough as more people came. I was in a frenzy and asked Deven Bansod, Keerthana Krishnan, Prabaharan Jaminy (the whole GSOC and Outreachy attendees) to volunteer to help out in making more iso images on usb disks. I introduced mensah to Steve McIntyre and we tried 2-3 times to get debian installed on the system but it didn t move from the same place. Ritesh shared that dd had a memory leak and hence cat was a better way to do it. So we did

$ cat debian.iso > /dev/sdb and soon other machines had debian sporting on their desktops. But mensa s lappy wouldn t get move from the boot-loader stage. Suddenly Steve had the bright idea (light bulb moment) that maybe the .iso is corrupted/usb disk is bad or something is incomplete. We started on another usb disk. Now this is where I have a query While I don t want to compare, in Ubuntu there was an image self-checking mechanism where probably behind the scenes (backend) the checksums published in a file are compared with checksums generated by apps. which are on the .iso image. While it does extend your time, the end result is you know if there is some issue on the decompressed image on the usb disk. AFAIK we don t have anything similar. The only two things I know is the wiki page and of course the various checksums of the image as shared at http://cdimage.debian.org/debian-cd/8.6.0/amd64/iso-cd/ or http://cdimage.debian.org/debian-cd/8.6.0/amd64/iso-dvd/ If anybody knows of any movement or a bug in the BTS which I can follow for the above issue please let me know. This time Steve was able to install it without any issues. I asked him whether he had to make some specific FAT/Ex-FAT/NTFS partitions as some new UEFI-based lappies need one or more but he replied in the negative. While mensa did get her debian install, the GUI didn t come while command-prompt was available. Then Steve added backports to the sources.list, got the new kernel, new Intel/Nvidia drivers (think it was one of those hybrid models IIRC) and she was able to boot into GNOME-Debian. I didn t saw any bug-reports about checksumming state of the applications before installation but did couple of reports about badblocks support and memory checking and from action on both bug-reports it is also need of the hour (although the earlier one has been marked as won t fix :(). In this whole thing, I liked/appreciated the way Steve handled things, I intuitively understood/knew that he wasn t just part of the Debian-installer team but someone better. I can t explain it but it was there. A little investigation in the evening and it turned out that he had been Debian Project Leader for two consecutive years (2008 and 2009) . In hindsight it probably was a good thing I didn t know that before otherwise I probably wouldn t have interacted with him and it would have been my loss. To have been the DPL and still being so humble while technically being so proficient, I was amazed and didn t know what to make of it. Here i.e. in India, if somebody wins even the mohalla elections (neighbourhood elections) the person carries a big chip on her/is shoulder not just till he is on the seat but even beyond, and here was an example of a previous DPL asking time from one of the developers in a video if it s possible in the next couple of days. Lastly,last week have able to report 2 bugs upstream. The first one is of youtube-dl . It s somewhat complicated hence will not go there atm. The second and more surprising one was from nano our esteemed text-editor- Hopefully the bug will be fixed once a new version comes.
Filed under: Miscellenous Tagged: #buggy, #cop, #Debconf16, #doha airport, #Installfest, #nano, #singer, #youtube-dl, travel

Bits from Debian: Debian Contributors Survey 2016

The Debian Contributor Survey launched last week! In order to better understand and document who contributes to Debian, we (Mathieu ONeil, Molly de Blanc, and Stefano Zacchiroli) have created this survey to capture the current state of participation in the Debian Project through the lense of common demographics. We hope a general survey will become an annual effort, and that each year there will also be a focus on a specific aspect of the project or community. The 2016 edition contains sections concerning work, employment, and labour issues in order to learn about who is getting paid to work on and with Debian, and how those relationships affect contributions. We want to hear from as many Debian contributors as possible whether you've submitted a bug report, attended a DebConf, reviewed translations, maintain packages, participated in Debian teams, or are a Debian Developer. Completing the survey should take 10-30 minutes, depending on your current involvement with the project and employment status. In an effort to reflect our own ideals as well as those of the Debian project, we are using LimeSurvey, an entirely free software survey tool, in an instance of it hosted by the LimeSurvey developers. Survey responses are anonymous, IP and HTTP information are not logged, and all questions are optional. As it is still likely possible to determine who a respondent is based on their answers, results will only be distributed in aggregate form, in a way that does not allow deanonymization. The results of the survey will be analyzed as part of ongoing research work by the organizers. A report discussing the results will be published under a DFSG-free license and distributed to the Debian community as soon as it's ready. The raw, disaggregated answers will not be distributed and will be kept under the responsibility of the organizers. We hope you will fill out the Debian Contributor Survey. The deadline for participation is: 4 December 2016, at 23:59 UTC. If you have any questions, don't hesitate to contact us via email at:

Russ Allbery: Review: The Philosopher Kings

Review: The Philosopher Kings, by Jo Walton
Series: Thessaly #2
Publisher: Tor
Copyright: June 2015
ISBN: 0-7653-3267-1
Format: Hardcover
Pages: 345
Despite the cliffhanger at the end of The Just City, The Philosopher Kings doesn't pick up immediately afterwards. Argh. It's a great book (as I'm about to describe), but I really wanted to also read that book that happened in between. Still, this is the conclusion to the problem posed in The Just City, and I wouldn't recommend reading this book on its own (or, really, either book separate from the other). Despite the unwanted gap, and another change at the very start of the book that I won't state explicitly since it's a spoiler but that made me quite unhappy (despite driving the rest of the plot), this is much closer to the book that I wanted to read. Walton moves away from the SF philosophical question that drove much of the second half of The Just City in favor of going back to arguments about human organization, the nature of justice, choices between different modes of life, and the nature of human relationships. Those were the best parts of The Just City, and they're elaborated here in some fascinating ways that wouldn't have been possible in the hothouse environment of the previous book. I also thought Apollo was more interesting here than in the previous book. Still somewhat infuriating, particularly near the start, but I felt like I got more of what Walton was trying for, and more of what Apollo was attempting to use this existence to understand. And, once the plot hits its stride towards the center of the book, I started really liking Apollo. I guess it took a book and a half for him to mature enough to be interesting. A new viewpoint character, Arete, gets most of the chapters in this book, rather than following the pattern of The Just City and changing viewpoint characters every chapter. Her identity is a spoiler for The Just City, so I'll leave that a mystery. She's a bit more matter-of-fact and observational than Maia, but she does that thing that I love in Walton's characters: take an unexpected, fantastic situation, analyze and experiment with it, and draw some practical and matter-of-fact conclusions about how to proceed. I think that's the best way to describe this entire series: take a bunch of honest, thoughtful, and mostly good people, put them into a fantastic situation (at first Plato's Republic, a thought experiment made real, and then some additional fantasy complexities), and watch them tackle that situation like reasonable human beings. There is some drama, of course, because humans will disagree and will occasionally do awful, or just hurtful, things to each other. But the characters try to defuse the drama, try to be thoughtful and fair and just in their approach, and encourage change, improvement, and forgiveness in others. I don't like everyone in these books, but the vast majority of them are good people (and the few who aren't stand out), and there's something satisfying in reading about them. And the philosophical debate is wonderful throughout this book (which I'm not saying entirely because the characters have a similar reaction to a newly-introduced philosophical system as I did as a reader, although that certainly helps). I'm not saying much about the plot since so much would spoil the previous book. But Walton adds some well-done complexities and complications, and while I was dubious about them at the start of the book, I definitely came around. I enjoyed watching the characters reinvent some typical human problems, but still come at them from a unique and thoughtful angle and come up with some novel solutions. And the ending took me entirely by surprise, in a very good way. It's better than the best ending I could have imagined for the book, providing some much-needed closure and quite a bit of explanation. (And, thankfully, does not end on another cliffhanger; in fact, I'm quite curious to see what the third book is going to tackle.) Recommended, including the previous book, despite the bits that irritated me. Followed by Necessity. Rating: 9 out of 10

15 November 2016

Antoine Beaupr : The Turris Omnia router: help for the IoT mess?

The Turris Omnia router is not the first FLOSS router out there, but it could well be one of the first open hardware routers to be available. As the crowdfunding campaign is coming to a close, it is worth reflecting on the place of the project in the ecosystem. Beyond that, I got my hardware recently, so I was able to give it a try.

A short introduction to the Omnia project The Turris Omnia Router The Omnia router is a followup project on CZ.NIC's original research project, the Turris. The goal of the project was to identify hostile traffic on end-user networks and develop global responses to those attacks across every monitored device. The Omnia is an extension of the original project: more features were added and data collection is now opt-in. Whereas the original Turris was simply a home router, the new Omnia router includes:
  • 1.6GHz ARM CPU
  • 1-2GB RAM
  • 8GB flash storage
  • 6 Gbit Ethernet ports
  • SFP fiber port
  • 2 Mini-PCI express ports
  • mSATA port
  • 3 MIMO 802.11ac and 2 MIMO 802.11bgn radios and antennas
  • SIM card support for backup connectivity
Some models sold had a larger case to accommodate extra hard drives, turning the Omnia router into a NAS device that could actually serve as a multi-purpose home server. Indeed, it is one of the objectives of the project to make "more than just a router". The NAS model is not currently on sale anymore, but there are plans to bring it back along with LTE modem options and new accessories "to expand Omnia towards home automation". Omnia runs a fork of the OpenWRT distribution called TurrisOS that has been customized to support automated live updates, a simpler web interface, and other extra features. The fork also has patches to the Linux kernel, which is based on Linux 4.4.13 (according to uname -a). It is unclear why those patches are necessary since the ARMv7 Armada 385 CPU has been supported in Linux since at least 4.2-rc1, but it is common for OpenWRT ports to ship patches to the kernel, either to backport missing functionality or perform some optimization. There has been some pressure from backers to petition Turris to "speedup the process of upstreaming Omnia support to OpenWrt". It could be that the team is too busy with delivering the devices already ordered to complete that process at this point. The software is available on the CZ-NIC GitHub repository and the actual Linux patches can be found here and here. CZ.NIC also operates a private GitLab instance where more software is available. There is technically no reason why you wouldn't be able to run your own distribution on the Omnia router: OpenWRT development snapshots should be able to run on the Omnia hardware and some people have installed Debian on Omnia. It may require some customization (e.g. the kernel) to make sure the Omnia hardware is correctly supported. Most people seem to prefer to run TurrisOS because of the extra features. The hardware itself is also free and open for the most part. There is a binary blob needed for the 5GHz wireless card, which seems to be the only proprietary component on the board. The schematics of the device are available through the Omnia wiki, but oddly not in the GitHub repository like the rest of the software.

Hands on I received my own router last week, which is about six months late from the original April 2016 delivery date; it allowed me to do some hands-on testing of the device. The first thing I noticed was a known problem with the antenna connectors: I had to open up the case to screw the fittings tight, otherwise the antennas wouldn't screw in correctly. Once that was done, I simply had to go through the usual process of setting up the router, which consisted of connecting the Omnia to my laptop with an Ethernet cable, connecting the Omnia to an uplink (I hooked it into my existing network), and go through a web wizard. I was pleasantly surprised with the interface: it was smooth and easy to use, but at the same time imposed good security practices on the user. Install wizard performing automatic updates For example, the wizard, once connected to the network, goes through a full system upgrade and will, by default, automatically upgrade itself (including reboots) when new updates become available. Users have to opt-in to the automatic updates, and can chose to automate only the downloading and installation of the updates without having the device reboot on its own. Reboots are also performed during user-specified time frames (by default, Omnia applies kernel updates during the night). I also liked the "skip" button that allowed me to completely bypass the wizard and configure the device myself, through the regular OpenWRT systems (like LuCI or SSH) if I needed to. The Omnia router about to rollback to latest snapshot Notwithstanding the antenna connectors themselves, the hardware is nice. I ordered the black metal case, and I must admit I love the many LED lights in the front. It is especially useful to have color changes in the reset procedure: no more guessing what state the device is in or if I pressed the reset button long enough. The LEDs can also be dimmed to reduce the glare that our electronic devices produce. All this comes at a price, however: at \$250 USD, it is a much higher price tag than common home routers, which typically go for around \$50. Furthermore, it may be difficult to actually get the device, because no orders are being accepted on the Indiegogo site after October 31. The Turris team doesn't actually want to deal with retail sales and has now delegated retail sales to other stores, which are currently limited to European deliveries.

A nice device to help fight off the IoT apocalypse It seems there isn't a week that goes by these days without a record-breaking distributed denial-of-service (DDoS) attack. Those attacks are more and more caused by home routers, webcams, and "Internet of Things" (IoT) devices. In that context, the Omnia sets a high bar for how devices should be built but also how they should be operated. Omnia routers are automatically upgraded on a nightly basis and, by default, do not provide telnet or SSH ports to run arbitrary code. There is the password-less wizard that starts up on install, but it forces the user to chose a password in order to complete the configuration. Both the hardware and software of the Omnia are free and open. The automatic update's EULA explicitly states that the software provided by CZ.NIC "will be released under a free software licence" (and it has been, as mentioned earlier). This makes the machine much easier to audit by someone looking for possible flaws, say for example a customs official looking to approve the import in the eventual case where IoT devices end up being regulated. But it also makes the device itself more secure. One of the problems with these kinds of devices is "bit rot": they have known vulnerabilities that are not fixed in a timely manner, if at all. While it would be trivial for an attacker to disable the Omnia's auto-update mechanisms, the point is not to counterattack, but to prevent attacks on known vulnerabilities. The CZ.NIC folks take it a step further and encourage users to actively participate in a monitoring effort to document such attacks. For example, the Omnia can run a honeypot to lure attackers into divulging their presence. The Omnia also runs an elaborate data collection program, where routers report malicious activity to a central server that collects information about traffic flows, blocked packets, bandwidth usage, and activity from a predefined list of malicious addresses. The exact data collected is specified in another EULA that is currently only available to users logged in at the Turris web site. That data can then be turned into tweaked firewall rules to protect the overall network, which the Turris project calls a distributed adaptive firewall. Users need to explicitly opt-in to the monitoring system by registering on a portal using their email address. Turris devices also feature the Majordomo software (not to be confused with the venerable mailing list software) that can also monitor devices in your home and identify hostile traffic, potentially leading users to take responsibility over the actions of their own devices. This, in turn, could lead users to trickle complaints back up to the manufacturers that could change their behavior. It turns out that some companies do care about their reputations and will issue recalls if their devices have significant enough issues. It remains to be seen how effective the latter approach will be, however. In the meantime, the Omnia seems to be an excellent all-around server and router for even the most demanding home or small-office environments that is a great example for future competitors.
Note: this article first appeared in the Linux Weekly News.

Keith Packard: AltOS-Lisp

A Tiny Lisp for AltOS I took a bit of a diversion over the last week or so when I wondered how small a lisp interpreter I could write, and whether I could fit that into one of the processors that AltOS runs on. It turns out, you can write a tiny lisp interpreter that fits in about 25kB of ram with a 3kB heap for dynamic data. I decided to target our ChaosKey boards; they're tiny, and I've got a lot of them. That processor offers 28kB of usable flash space (after the 4kB boot loader) and 6kB of ram with the processor running at a steaming 48MHz. I'm not at all sure this is useful, but I always enjoy doing language implementations, and this one presented some 'interesting' challenges: Iterative Compacting Allocator I'm betting someone has built one of these before, but I couldn't find one, so I wrote my own. The basic strategy is to walk the heap to find a subset of the active objects which are allocated sequentially in memory with only unused storage between them. These objects are then compacted in-place, and then the heap is walked again to update all references to the moved objects. Then, the process is restarted to find another subset and move them. By looking for these subsets starting at the bottom of the heap, and working upwards towards the top, the whole heap can be compacted into a contiguous chunk at the bottom of memory. Allocation involves moving a pointer along at the top of active memory; when it gets to the top of the heap, collect and see if there's space now. As always, the hardest part was to make sure all active memory was tied down. The second hardest part was to make sure that all active pointers were updated after any allocation, in case a collect moved the underlying object. That was just bookkeeping, but did consume much of the development time. One additional trick was to terminate the recursion during heap walking by flagging active cons cell locations in a global bitmap and then walking that separately, iterating until that bitmap is empty. Nested lambdas form another recursion which should probably get the same approach, but I haven't done that yet. An unexpected "benefit" of the tiny heap is that the collector gets called a lot, so any referencing bugs will have a good chance of being uncovered in even a short program execution. ROM-able Lisp Instead of implementing all of the language in C, I wanted to be able to implement various pieces in Lisp itself. Because of the complex nature of the evaluation process, adding things like 'let' or even 'defun' turn out to be dramatically simpler in Lisp. However, I didn't want to consume bunches of precious RAM to hold these basic functions. What I did was to create two heaps, one in ROM and the other in RAM. References are be tagged as to which heap they're in. 16-bit Values Lisp programs use a pile of references. Using a full 32 bits for each one would mean having a lot less effective storage. So, instead, I use an offset from the base of the heap. The top bit of the offset is used to distinguish between the ROM heap and the RAM heap. I needed a place to store type information, so I settled on using the bottom two bits of the references. This allows for four direct type values. One of these values is used to indicate an indirect type, where the type is stored in the first byte of the object. The direct types are:
ValueType
0Cons cell
114-bit int
2String
3Other
With 2 tag bits, the allocator needs to work in 32-bit units as the references couldn't point to individual bytes. Finally, I wanted 0 to be nil, so I add four to the offsets within the heaps. The result is that the ROM and RAM heaps can each cover up to 32k - 4 bytes. Note that ints are not stored in the heap; instead they are immediate values stored in 14 bits, providing a range of -8192 to 8191. One can imagine wanting more range in ints at some point. Heap-based Evaluator A simple lisp implementation uses the fact that eval is re-entrant and do the operation on the C stack: 
val eval(val exprs)  
    val vals;
    while (exprs)  
        vals = append(vals, eval(car(exprs)));
        exprs = exprs->cdr;
     
    return execute (car(vals), cdr(vals));
This makes things really simple and provides for a clean framework for implementing various bits of lisp, including control flow and macros. However, it rapidly consumes all of the available memory for a stack, while also requiring separate bookkeeping for the in-use memory in each frame. I replaced this design with one which keeps the lisp stack on the heap, and then performs eval with a state machine with all state stored in global variables so that the memory manager can reference them directly. Each eval operation is performed in a separate 'stack' context, which holds the entire eval state except for the current value, which lives in a separate global variable and is used to pass values out of one stack frame and into another. When the last stack context is finished, the evaluation terminates and the value is returned to the caller. There are nine states in the state machine, each of which is implemented in a separate function, making the state machine a simple matter of pulling the current state from the top of the stack and invoking the associated function: 
while (ao_lisp_stack)  
    if (!(*evals[ao_lisp_stack->state])()   ao_lisp_exception)  
        ao_lisp_stack_clear();
        return AO_LISP_NIL;
     
 
return ao_lisp_v;
Because there's no C recursion involved, catching exceptions is a simple matter of one test at this level. Primitives like progn, while, cond and eval all take special magic in the state machine to handle; getting all of that working took several attempts before I found the simple loop shown above. Lexical Scoping The last time I did a lisp interpreter, I implemented dynamic scoping. Atoms were all global and had values associated directly with them. Evaluating a lambda started by saving all of the existing global values for the parameter atoms and then binding the new values. When finished, the previous values would be restored. This is almost correct, but provides surprising results for things like: 
> (setq baz 1)
> (def foo (lambda (bar) (+ baz bar)))
> (def bletch (lambda (baz) (foo baz)))
> (bletch 2)
4
The value that foo gets for 'baz' is 2 instead of 1 under dynamic scoping, which most people find surprising. This time, I was determined to use lexical scoping, and it turned out to be surprisingly easy. The first trick was to separate the atoms from their 'value'; each atom can have a different value in different lexical scopes. So, each lexical scope gets a 'frame' object, those contain the value for each atom defined in that scope. There's a global scope which holds all of the globally defined values (like baz, foo and bletch above). Each frame points to its enclosing scope, so you can search upwards to find the right value. The second trick was to realize that the lexical scope of a lambda is the scope in which the lambda itself is evaluated, and that the evaluation of a lambda expression results in a 'function' object, which contains the lambda and its enclosing scope: 
> (def foo (lambda (bar bletch)
       ((lambda (baz)
          (+ baz bar))
        bletch)))
> (foo 2 3)
5
In this case, the inner lambda in foo can 'see' the value of bar from the enclosing lambda. More subtly, even if the inner lambda were executed multiple times, it would see the same baz, and could even change it. This can be used to implement all kinds of craziness, including generators: 
> (defun make-inc (add)
  ((lambda (base)
     (lambda ()
       (progn
     (setq base (+ base add))
     base)))
   0)
  )
> (setq plus2 (make-inc 2))
> (plus2)
2
> (plus2)
4
The current implementation of each frame is a simple array of atom/value pairs, with a reference to the parent frame to form the full scope. There are dramatically faster implementations of this same concept, but the goal here was small and simple. A Tiny Allocator Optimization With eval consuming heap space for stacks, frames and argument lists, the interpreter was spending a lot of time in the collector. As a simple optimization, I added some free lists for stack frames and cons cells. Stack frames are never referenced when they're finished, so they can always go on the free list. Cons cells used to construct argument lists for functions are usually free. Builtin functions have a bit which indicates whether they might hold on to a reference to the argument list. Interpreted lambdas can't get the list while nlambdas, lexprs and macros do. Each lambda execution creates a new frame, and while it would be possible to discover if that frame 'escapes' the lambda, I decided to not attempt to cache free ones yet. Save and Restore To make the lisp interpreter more useful in tiny computers, I added the ability to save and restore the entire heap to flash. This requires leaving enough space in the flash to preserve the heap, further constraining the amount of flash available for the application. Code All of this code is in the 'lisp' branch of my AltOS repository: AltOS The lisp interpreter is independent from the rest of AltOS and could be re-purposed for another embedded operating system. It runs fine on ChaosKey hardware, and also on the STM32F042 Nucleo-32 board There's also a test framework which runs on Linux, and is how I developed almost all of the code. That's in the src/test directory in the above repository, and is called 'ao_lisp_test'. Towers of Hanoi Here's an implementation of the classic recursive Towers of Hanoi game; it shows most of the current features of the language. 
;
; Towers of Hanoi
;
; Copyright   2016 Keith Packard <keithp@keithp.com>
;
; This program is free software; you can redistribute it and/or modify
; it under the terms of the GNU General Public License as published by
; the Free Software Foundation, either version 2 of the License, or
; (at your option) any later version.
;
; This program is distributed in the hope that it will be useful, but
; WITHOUT ANY WARRANTY; without even the implied warranty of
; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
; General Public License for more details.
;
; ANSI control sequences
(defun move-to (col row)
  (patom "\033[" row ";" col "H" nil)
  )
(defun clear ()
  (patom "\033[2J" nil)
  )
(defun display-string (x y str)
  (move-to x y)
  (patom str)
  )
; Here's the pieces to display
(setq stack '("*" "**" "***" "****" "*****" "******" "*******"))
(setq top (+ (length stack) 3))
;
; Here's all of the stacks of pieces
; This is generated when the program is run
;
(setq stacks nil)
; Display one stack, clearing any
; space above it
(defun display-stack (x y clear stack)
  (cond ((= 0 clear)
     (cond (stack (progn
            (display-string x y (car stack))
            (display-stack x (1+ y) 0 (cdr stack))
            )
              )
           )
     )
    (t (progn
         (display-string x y "          ")
         (display-stack x (1+ y) (1- clear) stack)
         )
       )
    )
  )
; This should probably be included in the rom image...
(defun length (list)
  (cond (list (1+ (length (cdr list))))
    (0)
    )
  )
; Position of the top of the stack on the screen
; Shorter stacks start further down the screen
(defun stack-pos (y stack)
  (- y (length stack))
  )
; Display all of the stacks, spaced 20 columns apart
(defun display-stacks (x y stacks)
  (cond (stacks (progn
          (display-stack x 0 (stack-pos y (car stacks)) (car stacks))
          (display-stacks (+ x 20) y (cdr stacks)))
        )
    )
  )
; Display all of the stacks, then move the cursor
; out of the way and flush the output
(defun display ()
  (display-stacks 0 top stacks)
  (move-to 1 21)
  (flush)
  )
; Reset stacks to the starting state, with
; all of the pieces in the first stack and the
; other two empty
(defun reset-stacks ()
  (setq stacks (list stack nil nil))
  (length stack)
  )
; more functions which could usefully
; be in the rom image
(defun min (a b)
  (cond ((< a b) a)
    (b)
    )
  )
(defun nth (list n)
  (cond ((= n 0) (car list))
    ((nth (cdr list) (1- n)))
    )
  )
; Replace a stack in the list of stacks
; with a new value
(defun replace (list pos member)
  (cond ((= pos 0) (cons member (cdr list)))
    ((cons (car list) (replace (cdr list) (1- pos) member)))
    )
  )
; Move a piece from the top of one stack
; to the top of another
(defun move-piece (from to)
  (let ((from-stack (nth stacks from))
    (to-stack (nth stacks to))
    (piece (car from-stack)))
    (setq from-stack (cdr from-stack))
    (setq to-stack (cons piece to-stack))
    (setq stacks (replace stacks from from-stack))
    (setq stacks (replace stacks to to-stack))
    (display)
    (delay 100)
    )
  )
; The implementation of the game
(defun _hanoi (n from to use)
  (cond ((= 1 n)
     (progn
      (move-piece from to)
      nil)
     )
    (t
     (progn
      (_hanoi (1- n) from use to)
      (_hanoi 1 from to use)
      (_hanoi (1- n) use to from)
      )
     )
    )
  )
; A pretty interface which
; resets the state of the game,
; clears the screen and runs
; the program
(defun hanoi ()
  (setq len (reset-stacks))
  (clear)
  (_hanoi len 0 1 2)
  )

Russ Allbery: Review: The Broken Kingdoms

Review: The Broken Kingdoms, by N.K. Jemisin
Series: Inheritance #2
Publisher: Orbit
Copyright: November 2010
Printing: September 2011
ISBN: 0-316-04395-8
Format: Mass market
Pages: 395
The Broken Kingdoms is a fairly direct sequel to The Hundred Thousand Kingdoms and depends heavily on the end of that book. It had been a long time since I'd read the previous book (about five years), and I looked up plot summaries to remind myself what happened. It turned out that I probably didn't have to do that; the explanation does come when it's critical. But this book will definitely spoil the end of The Hundred Thousand Kingdoms. Oree is an artist who sells her work to tourists in Shadow, the city beneath the World Tree. It's a good enough living, particularly for a blind immigrant from Nimaro, the area settled by the survivors of the destruction of Maro. Oree is not strictly entirely blind, since she can see magic, but that's not particularly helpful in daily life. She's content to keep that quiet, along with her private paintings that carry a strange magic not found in her public trinkets. One of the many godlings who inhabit Shadow is Oree's former lover, so she has some connection to the powerful of the city. But she prefers her quiet life until, that is, she finds a man at sunrise in a pile of muck and takes him home to clean him up. A man who she ends up taking care of, despite the fact that he never speaks to her, and despite his total lack of desire or apparent capability to take care of himself or avoid any danger. Not that it seems to matter, since he comes back to life every time he dies. If you've read The Hundred Thousand Kingdoms, you have a pretty good guess at who the man Oree calls Shiny actually is. But that discovery is not the core plot of this book. Someone is killing the godlings. They're not immortal, although they don't age, but killing them should require immense power or the intervention of the Three, the gods who run the metaphysical universe of this series. Neither of those seem to be happening, and still godlings are being murdered. Nahadoth is not amused: the humans and godlings have one month to find the killer before he does something awful to all of them. Then Shiny somehow kills a bunch of priests of Itempas, and the Order is after both him and Oree. Desperate, she turns to her former boyfriend and the godlings for help, and is pulled into the heart of a dark conspiracy. The Broken Kingdoms adds a few new elements to Jemisin's world-building, although it never quite builds up to the level of metaphysics of the previous book. But it's mostly a book about Oree: her exasperated care of Shiny, her attempts to navigate her rapidly complicating life, and her determination to do the right thing for her friends. It's the sort of book that pits cosmic power and grand schemes against the determined inner morality of a single person who is more powerful than she thinks she is. That part of the story I liked quite a lot. Shiny, and Oree's complicated relationship with Shiny, I wasn't as fond of. Oree treats him like a broken and possibly healing person, which is true, but he's also passively abusive in his dark moping. Jemisin tries very hard throughout the book to help the reader try to grasp a bit of what must be going through Shiny's head, and she does succeed at times, but I never much cared for what I found there. And neither Nahadoth nor Yeine, when they finally make their appearance, are very likable. (Yeine in particular I found deeply disappointing and not up to her level of ethics in the first book.) Oree is still quite capable of carrying the story single-handed, and I did like her godling friends. But I felt like the ending required liking Shiny a lot more than I did, or being a lot more sympathetic to Nahadoth and Yeine than I was, and it left a bad taste in my mouth. I enjoyed reading about Oree, but I felt like this story gave her a remarkably depressing ending. This book is also structured with a long middle section where everything seems to get more and more horrible and the antagonists are doing awful things. It's a common structural way to build tension that I rarely like. Even knowing that there's doubtless an upward arc and protagonist triumph coming, those sections are often unpleasant and difficult to read through, and I had that reaction here. The Broken Kingdoms is less of a weird romance than The Hundred Thousand Kingdoms (although there is some romance), so you may enjoy it more if you thought that angle was overdone. It does have some interesting world-building, particularly at the godling level, and Lil is one of my favorite characters. I think Oree got a raw deal from the story and would have preferred a different ending, but I'm not sorry I read it. Followed by The Kingdoms of Gods. Rating: 7 out of 10

13 November 2016

Andrew Cater: MiniDebconf Cambridge, ARM, 13/11/16 - Day 4 post 3

Just about to watch Daniel Silverstone demonstrating his new FPGA board.

Mystorm - open hardware - growing out of Icestorm. Programmable hardware - toolchains - and showing the hardware.

The Debian video team have advised that they are about to start the transcoding process - videos will be available soonish :)

http://video.debconf.org/minidebconf-low.webm


Petter Reinholdtsen: Coz profiler for multi-threaded software is now in Debian

The Coz profiler, a nice profiler able to run benchmarking experiments on the instrumented multi-threaded program, finally made it into Debian unstable yesterday. Llu s Vilanova and I have spent many months since I blogged about the coz tool in August working with upstream to make it suitable for Debian. There are still issues with clang compatibility, inline assembly only working x86 and minimized JavaScript libraries. To test it, install 'coz-profiler' using apt and run it like this:
coz run --- /path/to/binary-with-debug-info
This will produce a profile.coz file in the current working directory with the profiling information. This is then given to a JavaScript application provided in the package and available from a project web page. To start the local copy, invoke it in a browser like this:
sensible-browser /usr/share/coz-profiler/viewer/index.htm
See the project home page and the USENIX ;login: article on Coz for more information on how it is working.

Daniel Pocock: Are all victims of French terrorism equal?

Some personal observations about the terrorist atrocities around the world based on evidence from Wikipedia and other sources The year 2015 saw a series of distressing terrorist attacks in France. 2015 was also the 30th anniversary of the French Government's bombing of a civilian ship at port in New Zealand, murdering a photographer who was on board at the time. This horrendous crime has been chronicled in various movies including The Rainbow Warrior Conspiracy (1989) and The Rainbow Warrior (1993). The Paris attacks are a source of great anxiety for the people of France but they are also an attack on Europe and all civilized humanity as well. Rather than using them to channel more anger towards Muslims and Arabs with another extended (yet ineffective) state of emergency, isn't it about time that France moved on from the evils of its colonial past and "drains the swamp" where unrepentant villains are thriving in its security services? Fran ois Hollande and S gol ne Royal. Royal's brother G rard Royal allegedly planted the bomb in the terrorist mission to New Zealand. It is ironic that Royal is now Minister for Ecology while her brother sank the Greenpeace flagship. If Fran ois and S gol ne had married (they have four children together), would G rard be the president's brother-in-law or terrorist-in-law? The question has to be asked: if it looks like terrorism, if it smells like terrorism, if the victim of that French Government attrocity is as dead as the victims of Islamic militants littered across the floor of the Bataclan, shouldn't it also be considered an act of terrorism? If it was not an act of terrorism, then what is it that makes it differ? Why do French officials refer to it as nothing more than "a serious error", the term used by Prime Minister Manuel Valls during a recent visit to New Zealand in 2016? Was it that the French officials felt it was necessary for Libert , galit , fraternit ? Or is it just a limitation of the English language that we only have one word for terrorism, while French officials have a different word for such acts carried out by those who serve their flag? If the French government are sincere in their apology, why have they avoided releasing key facts about the atrocity, like who thought up this plot and who gave the orders? Did the soldiers involved volunteer for a mission with the code name Op ration Satanique, or did any other members of their unit quit rather than have such a horrendous crime on their conscience? What does that say about the people who carried out the orders? If somebody apprehended one of these rogue employees of the French Government today, would they be rewarded with France's highest honour, like those tourists who recently apprehended an Islamic terrorist on a high-speed train? If terrorism is such an absolute evil, why was it so easy for the officials involved to progress with their careers? Would an ex-member of an Islamic terrorist group be able to subsequently obtain US residence and employment as easily as the French terror squad's commander Louis-Pierre Dillais? When you consider the comments made by Donald Trump recently, the threats of violence and physical aggression against just about anybody he doesn't agree with, is this the type of diplomacy that the US will practice under his rule commencing in 2017? Are people like this motivated by a genuine concern for peace and security, or are these simply criminal acts of vengence backed by political leaders with the maturity of schoolyard bullies?

12 November 2016

John Goerzen: Morning in the Skies

IMG_8515 This is morning. Time to fly. Two boys, happy to open the hangar door and get the plane ready. It s been a year since I passed the FAA exam and became a pilot. Memories like these are my favorite reminders why I did. It is such fun to see people s faces light up with the joy of flying a few thousand feet above ground, of the beauty and freedom and peace of the skies. I ve flown 14 different passengers in that time; almost every flight I ve taken has been with people, which I enjoy. I ve heard wow or beautiful so many times, and said it myself even more times. IMG_6083 I ve landed in two state parks, visited any number of wonderful small towns, seen historic sites and placid lakes, ascended magically over forests and plains. I ve landed at 31 airports in 10 states, flying over 13,000 miles. airports Not once have I encountered anyone other than friendly, kind, and outgoing. And why not? After all, we re working around magic flying carpet machines, right? IMG_7867_bw (That s my brother before a flight with me, by the way) Some weeks it is easy to be glum. This week has been that way for many, myself included. But then, whether you are in the air or on the ground, if you pay attention, you realize we still live in a beautiful world with many wonderful people. And, in fact, I got a reminder of that this week. Not long after the election, I got in a plane, pushed in the throttle, and started the takeoff roll down a runway in the midst of an Indiana forest. The skies were the best kind of clear blue, and pretty soon I lifted off and could see for miles. Off in the distance, I could see the last cottony remnants of the morning s fog, lying still in the valleys, surrounding the little farms and houses as if to give them a loving hug. Wow. Sometimes the flight is bumpy. Sometimes the weather doesn t cooperate, and it doesn t happen at all. Sometimes you can fly across four large states and it feels as smooth as glass the whole way. Whatever happens, at the end of the day, the magic flying carpet machine gets locked up again. We go home, rest our heads on our soft pillows, and if we so choose, remember the beauty we experienced that day. Really, this post is not about being a pilot. This post is a reminder to pay attention to all that is beautiful in this world. It surrounds us; the smell of pine trees in the forest, the delight in the faces of children, the gentle breeze in our hair, the kind word from a stranger, the very sunrise. I hope that more of us will pay attention to the moments of clear skies and wind at our back. Even at those moments when we pull the hangar door shut. IMG_20160716_093627

10 November 2016

Ritesh Raj Sarraf: apt-offline 1.7.2 released

I am happy to announce the release of apt-offline 1.7.2. This has turned out in time for the next release of Debian, i.e. Debian Stretch. A long standing cosmetic issue in CLI of the progress bar total item count has been fixed. There are also a bunch of other bug fixes, for which the specifics are present in the git logs. Also, in this release, we've tried to catch-up on the Graphical Interface, adding the GUI equivalent of the features, that were added to apt-offline in the recent past. In 1.7.1, we added the changelog option, and in this release, we've added the GUI equivalent of it. The 'set' command had had many new options. So in this release the 'set' command's GUI has those options added. I hope you like this release. Please test and file bug reports in time so that we have a bug free version for Debian Stretch. Future tasks now will focus on porting the GUI to PyQt5 first and then apt-offline to Python3. Any help is welcome. You can download apt-offline from the github page or the alioth page. For Debian users, the package will show up in the repository soon. PS: What is apt-offline ? 
Description: offline APT package manager
 apt-offline is an Offline APT Package Manager.
 .
 apt-offline can fully update and upgrade an APT based distribution without
 connecting to the network, all of it transparent to APT.
 .
 apt-offline can be used to generate a signature on a machine (with no network).
 This signature contains all download information required for the APT database
 system. This signature file can be used on another machine connected to the
 internet (which need not be a Debian box and can even be running windows) to
 download the updates.
 The downloaded data will contain all updates in a format understood by APT and
 this data can be used by apt-offline to update the non-networked machine.
 .
 apt-offline can also fetch bug reports and make them available offline.

Categories:

Keywords:

Like:

9 November 2016

Russ Allbery: Some thoughts on the US elections

I apparently am not going to get anything done today until I write this. Some thoughts, in no particular order. I don't have any profound conclusions. I'm honestly pretty upset. And pretty scared. But we have to talk to each other. And we have to listen to each other. And we have to persaude each other. And we have to be willing to be persuaded. And please go tell someone this morning that you have their back.

Next.

Previous.