Search Results: "xam"

24 October 2013

Daniel Pocock: Final report on GSoC 2013 projects

Google Summer of Code finished recently. This is the first year that I have participated as a mentor for the Debian Project. Its a big responsibility to be part of the Debian team and to be one of the Debian team members representing Debian at the GSoC Mentor Summit. Birthdays all round This has been a particularly important year for the Debian Project, as the project celebrated our 20th birthday recently, on 16 August 2013, one of the final days of DebConf13 in Switzerland. GSoC celebrates its 10th year in 2014, with a generous 10% boost in the student stipend to mark the occasion. Google: don't be evil Google's "Don't be evil" approach to business has always been an interesting point for discussion. In my view, there are few cases of absolute good or absolute evil that we can universally classify and agree on. At a more pragmatic level, various people have commented that GSoC is a recruiting program for Google: some have even cited this as a reason not to participate. This is a more interesting point for discussion. The results of the program are not exclusive to the headhunters at Google. Anybody can browse through the Debian GSoC weekly student reports from each student to find out exactly what the students were up to and try to recruit them. Mentors and the projects they belong to have not been forced to sign any wide-ranging non-disclosure agreements or non-compete agreements with Google. In one recent example, the xWiki open source project even went as far as setting up a new office in Romania and employing some former GSoC students on a permanent basis.
Assorted images from Iasi, Romania, where xWiki set up an office employing former GSoC students How does Google benefit then? Well, there appear to be several possibilities:
  • Generating a huge amount of goodwill and recognition for the value that they place on the development of the next generation of software engineers
  • Stimulating the expansion of high quality free software projects, many of which they use directly or indirectly for their internal projects
  • Access to private evaluations prepared by the mentors. These evaluations don't take more than an hour for the mentors to complete but they do serve to give Google's headhunters a slight headstart over anybody else who is scouring the web for the names of graduates who completed GSoC
Overall, it appears to fit the definition of a "win-win" scenario: yes, Google gets stuff out of it, but the benefit is not exclusive to Google and projects like Debian are winners too. My initial approach I started by reviewing some of these materials: One particular concept that I took note of was the need to give students some opportunity to be innovative and original in their project proposals. In other words, it has been suggested that students should not be given a precise specification: rather, they should be given some very general concepts or goals and asked to suggest some funky, innovative new idea. The wisdom of this approach varies. If your free software project has some very specific gap that you need to fill, you may only be interested in taking a student who can fill that gap and you may not want to waste the time of other applicants. On the other hand, if you do precisely document this gap you want filled, you may get a bunch of very similar proposals from students and it may be more difficult to distinguish which of them is the most desirable candidate. Furthermore, if your expectations are too rigid, you may not benefit from a top-gun coming in and contributing some really exciting piece of work that you hadn't even thought of. In the end, I opted for providing more generic project briefs and looking at how students responded. The briefs that I published are available here. Coding tasks during the selection process There is no doubt about it, coding tasks are an important part of ensuring students are capable of development work before they are selected. They are also very important for us as mentors. It is not uncommon for some of us to be completely out of touch with the capabilities of other developers. The coding tasks completed by the students during the selection phase allow us to get a feel for their capabilities and set reasonable expectations for how much they can achieve during the course of the project. To simplify the management of coding tasks and engage the students in the Debian community at large, I decided to create tasks as wishlist items in the Debian bug tracking system, such as this dynalogin bug report. The prospective GSoC students were asked to put their name in the bug tracker to take ownership of the task they would complete. Student project proposals For the real-time communications project, I felt all the students slightly underestimated the complexity of this field and would have to be extremely lucky to complete everything they hoped to. After all, this is a field that the free software movement has been wrestling with for years without gaining the upper hand. For somebody with good knowledge of the projects in this space, it is possible to guide the students towards meaningful and achievable subprojects. Nonetheless, the proposals were useful for me in understanding which part of the topics were most attractive to the students. For the other project areas, the student proposals were slightly more specific. Fabian's appeared to be the most specific and well considered proposal for the one-time-password project, in fact, it appeared very much like a project that he could work through from start to finish. Debian teamwork Anybody looking at the GSoC web site will notice that there is a certain amount of overhead in joining the program, promotion, payment administration and optional participation in the GSoC Mentor Summit. As a large organisation, Debian established a dedicated SoC administration team who looked after the GSoC and Outreach Program for Women administrivia. As a developer and mentor, I found this arrangement was highly effective and allowed me to focus over 99% of my efforts on the front-line work, selecting students and supporting them through their projects. Project results One of the more controversial issues during the Debian 7 (wheezy) release cycle was the inclusion of the Mumble voice conferencing software. Although the software has some serious issues, it was eventually escalated to the technical committee and allowed to remain in Debian, partly due to lack of any alternative. One of the first results from Catalin's project was the creation of an alternative, the new reConServer project, based on the librecon conversation manager API from the reSIProcate SIP stack. reConServer is a significantly more generic solution than Mumble, as it allows any SIP peer to participate in an audio conference. It supports TURN for NAT traversal too. Catalin then went on to make the first working implementation of a scheme for passing context information from a web site to SIP WebRTC call as described in the IETF draft. All work has been integrated into release tarballs and packaging. Fabian worked through the latest standards for one-time passwords, extending the oath-toolkit and dynalogin projects to implement mutual two-factor authentication with just about any arbitrary crypto suite using one-time passwords. This work is yet to be merged into an official release of dynalogin as it requires co-ordination with the oath-toolkit release and package updates. However, there is a compelling demonstration in his video from DebConf13 and the source is available under Fabian's github account. In the post-Snowden world, the work from both of these student projects will be hugely valuable to people wanting to tighten up their computer security and communication security practices. The development process Deciding to mentor more than one project turned out to be a worthwhile decision. At first, this might seem like a big risk but it is manageable with support from co-mentors. The upside is that different students work at different speeds, they have different code styles, they even work at different times of the day or week and mentoring more than one student/project allows the mentor to gain a better appreciation of how the students differ than if mentoring only a single project. For Catalin's project, it became obvious at an early stage that he would benefit from having a dedicated server to run the VoIP applications on public IP addresses and low-latency networks while testing them. I spun up a virtual machine with IPv4 and IPv6 in my Xen Cloud Platform and this was extremely helpful for both of us. Throughout the project, I took a hands-off approach and often left the students to their own devices. I gave some suggestions to Catalin about the object-oriented design and this was enough for him to fill in the gaps and write all the code by himself. Fabian's project had fewer ambiguities and he was able to use existing code as a model more frequently than Catalin. It is possible that the students may have produced more code if I had thought through the designs myself and given them definite blue-prints. However, I feel that this would have made it harder for me to appreciate their own abilities and test the limits of their design skills. DebConf13 participation One of the highlights for the students was their participation in DebConf13. Both students presented a session at the conference showcasing some of their work. The DebConf video team produces high quality videos of all sessions at the conference and this in turn provides some great evidence of the students' capabilities and what GSoC adds to Debian and the wider free software universe. I would encourage other mentors to actively contemplate ways to involve their students in conferences during or shortly after the completion of their project work. Looking ahead Google has recently confirmed that Google Summer of Code will be repeated in 2014. It is the 10th anniversary of the program and to celebrate, students will get 10% extra pay. Thank you There are many people to thank for the success of these projects and the wider success of the Debian GSoC team:
  • The Debian GSoC and OPW organisation admin team
  • Google and the Google Open Source Programs Office
  • All the co-mentors who assisted on these projects with me: Simon Josefsson, Luke Faraone, Sylvain Berfini, Eloy Coto and Jes s P rez Rubio
  • The students themselves, C t lin Constantin U urelu and Fabian Gr nbichler
  • All the other very capable and motivated students who applied, submitted code samples but were not selected to participate: many of these students could have also completed a successful project and turning them away is one of the most difficult tasks for mentors in a volunteer organisation like Debian

Russ Allbery: Review: Applied Cryptography

Review: Applied Cryptography, by Bruce Schneier
Publisher: John Wiley & Sons
Copyright: 1996
ISBN: 0-471-11709-9
Format: Trade paperback
Pages: 621
This is the next entry in the series of Russ reading books that he bought years ago and never got around to reading. Thankfully, this time, the book has aged somewhat better. This review is for the second edition of Applied Cryptography, published in 1996. Given how important computer security has become, and how central cryptography is to computer security, one might think that the passage of 17 years would make a book effectively obsolete. This turns out not to be the case. Yes, Rijndael (the current AES standard and the most widely-used block cipher), Camellia (the up-and-comer in the block cipher world), and the SHA-2 hash postdate this book and aren't discussed. Yes, there have been some further developments in elliptic-curve public-key cryptography. And yes, much of the political information in this book, as well as the patent situation for public-key cryptosystems, is now mostly of historical interest. But a surprising amount of this book still applies directly. Partly that's because much of Applied Cryptography is focused on general principles, classes of algorithms, and cryptographic protocols, and those do not change quickly. A rather amazing number of techniques and protocols still in widespread use today originated in the 1970s. Block ciphers, stream ciphers, cipher modes, public-key systems, signature systems, hash functions, key exchange, secret splitting, key management, and other, similar topics have not changed substantially. And even in the area of specific algorithms, DES is still in use (unfortunately) and its analysis is valuable for understanding any block cipher. RC4, Triple DES, CAST, RSA, MD4, MD5, and SHA-1 are still part of the cryptography landscape, and Schneier was already warning against using MD5 in 1996. The source code included in this book has, thankfully, been made obsolete by widely-available, high-quality free implementations of all common ciphers, available for any modern operating system. But the math, the explanations, and much of the cryptanalysis is still quite relevant. While it contains a comprehensive introduction to cryptographic concepts, Applied Cryptography is structured more like a reference manual than a tutorial. The first two chapters provide some foundations. Then, a wide variety of protocols, from the common to the completely esoteric, are discussed across the subsequent four chapters. There are four chapters on the basics of ciphers, another chapter on the mathematics, and then the longest part of the book: twelve chapters that provide a tour of all major and several interesting minor protocols that existed in 1996. These are organized into block ciphers, stream ciphers (and pseudo-random number generators), hash functions, and public-key systems. An entire chapter is devoted to the history and analysis of DES, and I think that was my favorite chapter of the book. Not only is it worth understanding how DES works, but it also provides a comprehensive introduction to the complexities of block cipher design, the politics of interactions with the NSA, and a fascinating history of early computer cryptosystems. Finally, Applied Cryptography closes with chapters on real-world systems and on politics. The politics section is mostly a historical curiosity, as is much of the chapter on real-world systems. Schneier discusses some of the PKCS and X.509 protocols here but doesn't use SSL as one of his examples despite its specification predating this book (perhaps the most glaring omission of the book). But he does discuss Kerberos (which is of personal interest to me), and it was useful to see it analyzed in the broader context of this sort of book. Those who read Schneier's blog regularly, or who have read any of his other books, will know that he's concise, incisive, and very readable, even with difficult material. I think he does err on the side of compactness this is not a book that handholds the reader or explains something more than once but there's a lot of ground to cover, so I can't really blame him. Expect to have to read the more difficult parts several times, and (at least on a first read) expect to just skip past some of the most complex sections as probably not worth the effort. But I think Schneier does a great job structuring the book so that one comes away with a high-level impression and overview sufficient to make informed guesses about capabilities and relative difficulty even if one doesn't have the patience to work through the details. I suspect most readers will want to skim at least part of this book. Unless you're willing to do a lot of background reading to understand the math, or a lot of careful study to follow the detailed descriptions and flaws in often-obscure algorithms, the algorithm discussions start to blend together. I found myself skipping a lot of the math and focusing on the basic descriptions and the cryptanalysis, particularly for the obscure ciphers. Many of the algorithms are also highly opaque and of obscure benefit; I doubt I will ever care about blind signatures or oblivious transfer, and I struggled to wrap my mind around the details of bit commitment (although it's apparently important in zero knowledge proofs, which in turn are relevant to authentication). But this is not the sort of book where you have to read every word. It's very well-structured, provides clear resynchronization points for readers who have had enough of a particular topic, and marks the topics that are more obscure or of dubious general usefulness. The most useful parts of this book, in my opinion, are the basic conceptual introductions to each class of cryptographic algorithm or protocol, and those chapters are excellent. I've been working in applied computer security, at the application and deployment level, for years, and this is the first time that I've felt like I really understood the difference between a block cipher and a stream cipher, or the implications of block cipher mode selection. Applied Cryptography is also a fascinating exercise in adjusting one's intuition for where the complexity lies. It's a memorable experience to move from the chapters on block ciphers, full of S-boxes and complex mixing and magic numbers, into the far more straightforward mathematical analysis of stream ciphers. Or the mathematical simplicity of RSA (although quite a bit of complexity is lurking on the cryptanalysis side, much of which is only touched on here). Of more historical interest, but still quite impressive, is that Applied Cryptography doubles as a comprehensive literature review. Not only does it cover nearly every algorithm in use at the time, it discusses the cryptography literature on nearly every topic with brief summaries of results and lines of investigation. The references section is a stunning 66 pages of small print featuring 1,653 references, and those are mentioned and put in context throughout the book. I'm not personally interested in chasing those lines of research further, and of course 17 years means there are many new papers that are as important to read today, but it's obvious that this book would have been an amazing and invaluable map to the research territory when it was first published. There are now newer books on this topic that you should consider if you're purchasing a book today. Cryptography Engineering in particular is in my to-read pile, and I'm interested to see how much of this book it could replace, although I believe it's lighter on the mathematical analysis and details. But I think Applied Cryptography still has a place on the bookshelf of any computer security professional. Schneier is succinct, detailed, straightforward, and comprehensive, at least for 1996, and I know I'll be reaching for this book the next time I forget the details of CFB mode and then get hopelessly confused by the Wikipedia article on the topic. Rating: 8 out of 10

23 October 2013

Rog rio Brito: Assorted random notes

This post is just a bunch of random notes, but I think that things are going well, this time.

Btrfs Just started playing with btrfs after converting a 2TB disk with about 1.3TB of data from ext4 to btrfs. This particular filesystem contains my backup data and I hope that btrfs is able to live up to high standards. I will probably write a detailed review with my impressions, but suffice to say that it is working fine and I hope that I will be happy for a long time to come. My other filesystems, for the moment, will be ext4.

Coursera Downloader Thanks to John Lehmann's generosity, now coursera-dl has a new home: a github organization of its own!

youtube-dl Some people may have noticed, others may not, but when downloading videos from Youtube, they apparently are getting more aggressive with the use of Dynamic Adaptive Streaming over HTTP (also called DASH) and, as a result, some (perhaps going to be all in the near future?) of the videos may not be available in the resolution/formats that you used to like (like me, with format number 35). By the way, one thing that is interesting with youtube videos provided via DASH is that they are available in different streams: one for the video and another for the audio. What does this mean in practical terms for users of youtube-dl? Well, if you wanted to download videos in resolutions like the 480p (format 35) that I mentioned, then you will probably have to change your way of doing things, until a more automated solution is in place. You will have to download both the audio and the video and, then, "combine" them (that is, multiplex them) to create one "normal" video file with both the audio and the video. I usually do this via: ffmpeg -i audio.m4a -i video.mp4 -vcodec copy -acodec copy combined.mp4 If you prefer having a Matroska container instead of an mp4 container (which, BTW, results in smaller muxing overhead), then you can use the command line: mkvmerge -o combined.mkv audio.m4a video.mp4 Oh, those m4a and mp4 extensions are a new addition that I just sent upstream (in the past, both would have been named with an extension of mp4). As, an aside, I like formats 135 for video and 140 for audio, for the reasons that I mentioned in a comments to issue 1612:
Otherwise, to download 480p videos (which I do for lectures and so on with other projects of mine, like edx-dl) I have to call youtube-dl twice: once for format 135 and another for format 140, since the old (?) format 35 files are much smaller than the lower resolution 360p files (due to the former being encoded in High profile vs. the latter being encoded in Constrained Baseline profile).
While this is unfortunate for some, this is a good thing for others: I once had a blind user of youtube-dl asking me if he could avoid downloading the whole video just to extract the audio, so that he save on bandwidth. Well, now this is possible. Expect a new version of youtube-dl to be uploaded soon to Debian unstable.

Music It is so nice to see the music that I like getting better. It was particularly pleasant to see Tarja Turunen being joined on stage by Floor Jansen. In fact, it is my impression that the once female-fronted bands are getting more and more into collaborations and side-projects and they are, in many ways, getting more refined. Examples of this includes Floor, Tarja, Anneke van Giersbergen, Sharon den Adel, Kari Ruesl tten and many others that I can't remember right now. Of course, some thanks to people that take the time to film, and upload/share the videos so that other people not at these events can watch them is highly appreciated and they usually are treated as anonymous, but, of course, this should be fixed.

22 October 2013

Francesca Ciceri: Improving collaboration skills

[...] this is a great example of overcoming something that we call "problem A, problem B" paradox.
So you've got a group of people coming together, and they're all trying to solve the same problem (problem A) and they're really passionate about it. And then, you've got this problem B thing, which is that you got a bunch of people with different views.
And they may seem to be conflicting, and it seems like a paradox: the same differences that we need to solve the problem are getting in our way. But if you think about it as a paradox, you can get through it.
You think: OK, we've got all these differences. You have to identify the differences that actually are going to move us forward and those that are getting in our way.
And the differences that are getting in your way, set them aside. Push them off to the side and focus on the ones that are helping you move forward.
Kathryn Jablokow Well, I noticed that at first everybody was saying "Yes, but this is my idea".
And as we got closer and closer together, it became "Yes, and".
Jack Matson
(Quotes from a lecture about "Collaboration skills", part of the Creativity, Innovation and Change course on Coursera) IMO this is an important lesson to learn if you want to work effectively in a community project like Debian.
We talk much about diversity, and the added value of diversity in our project.
But diversity also means that to converge towards consensus is difficult and requires a specific set of best practices. A little thing like saying "Yes and" instead of "Yes but" could be of help. Words definitely matter.

21 October 2013

Julian Andres Klode: python-apt 0.9 released

I released python-apt 0.9. This completely removes support for the old API from the code base (it was disabled for the entirety of 0.8 in Debian, and in Ubuntu since saucy). Highlights:
Filed under: Debian

Aigars Mahinovs: Moved to Mezzanine

After my server that has hosted my blog for some years had given out its last breath (second motherboard failure), I decited it was time for a change. And not just server change, but also change in the blog engine itself. As I now focus on Python and Django almost exclusively at work, it felt logical to use some kind of Django-based blog or CMS. I tried django-cms and mezzanine and ... Mezzanine is so fast and simple, that I simply stopped looking. After simply following the tutorial and creating a skeleton project, I had a ready-to go site with all the CMS features, incuding a blog. I just had to change a few settings to have the blog module be the home page of the site, change site settings for the title and Google Analytics settings and such and tweak the theme a bit to my liking. This was my first real exposure to a Bootstrap design. I must say - it is very simple to understand and modify if your needs fit within its limits. For example, I wanted to remove the left sidebar and expand the main content block to fill that. All I had to do was to remove the div element with class "left-sidebar span-2" and change the class of the main content part from "span-7" to "span-9". To do that I simply copied the templates/base.html file from mezzanine default templates and modified it. The information from django-debug-toolbar showed me what files were used in rendering the page. But the feature that really got me hooked was the Wordpress import. Using a simple management command I was able to feed into Mezzanine instance the XML export file from Wordpress. It created blog posts, categories, comments, static pages and even redirects from Wordpress permalinks to Mezzanine permalinks. It was not flawless - there were a few issues: After that was done it was a relatively straightforward process of picking up the code and data and deploying it to a Django-friendly hosting service. There is a plenty of good competition out there, most now offer a simple one-click Django installation, so I just created a simple Django skeleton via their web interface and then replace what they generated with my app while keeping their settings as local_settings.py . I should probably write a bit more details about the process. After I create a custom fabric file for this. It is quite a strange feeling to have a Mezzanine blog that responds faster from a shared server half a continent away, compared to a Wordpress on dedicated server in the same room There are a few features that I am still missing - most notably draft post autosave. That has bitten me hard while writing this post :P Also a Twitter digest post feature. But on the bright side - that is a great motivation to write such features, preferably in a portable way that other people can use too :)

Aigars Mahinovs: Moved to Mezzanine

After my server that has hosted my blog for some years had given out its last breath (second motherboard failure), I decited it was time for a change. And not just server change, but also change in the blog engine itself. As I now focus on Python and Django almost exclusively at work, it felt logical to use some kind of Django-based blog or CMS. I tried django-cms and mezzanine and ... Mezzanine is so fast and simple, that I simply stopped looking. After simply following the tutorial and creating a skeleton project, I had a ready-to go site with all the CMS features, incuding a blog. I just had to change a few settings to have the blog module be the home page of the site, change site settings for the title and Google Analytics settings and such and tweak the theme a bit to my liking. This was my first real exposure to a Bootstrap design. I must say - it is very simple to understand and modify if your needs fit within its limits. For example, I wanted to remove the left sidebar and expand the main content block to fill that. All I had to do was to remove the div element with class "left-sidebar span-2" and change the class of the main content part from "span-7" to "span-9". To do that I simply copied the templates/base.html file from mezzanine default templates and modified it. The information from django-debug-toolbar showed me what files were used in rendering the page. But the feature that really got me hooked was the Wordpress import. Using a simple management command I was able to feed into Mezzanine instance the XML export file from Wordpress. It created blog posts, categories, comments, static pages and even redirects from Wordpress permalinks to Mezzanine permalinks. It was not flawless - there were a few issues: After that was done it was a relatively straightforward process of picking up the code and data and deploying it to a Django-friendly hosting service. There is a plenty of good competition out there, most now offer a simple one-click Django installation, so I just created a simple Django skeleton via their web interface and then replace what they generated with my app while keeping their settings as local_settings.py . I should probably write a bit more details about the process. After I create a custom fabric file for this. It is quite a strange feeling to have a Mezzanine blog that responds faster from a shared server half a continent away, compared to a Wordpress on dedicated server in the same room There are a few features that I am still missing - most notably draft post autosave. That has bitten me hard while writing this post :P Also a Twitter digest post feature. But on the bright side - that is a great motivation to write such features, preferably in a portable way that other people can use too :)

19 October 2013

Dirk Eddelbuettel: New BH release 1.51.1-3

A new release of the BH package is now on CRAN and its mirrors. BH provides (a subset of) the Boost library for C++, particularly the (large) parts delivered as pure template headers not requiring linking. When this package is installed, an R package developer can deploy it at build-time via a simple LinkingTo: BH declaration (which, despite the title affects only compilation, not linking; don't ask...). This frees developers from having to include the (sizeable) Boost headers in their packages, and provides (parts of) Boost as part of the R build system. A short example of using this BH package with Rcpp is provided in this Rcpp Gallery post; a number of other Boost-related posts are also available. This release expands the scope of the package by a quite bit as shown in the NEWS entry:
Changes in version 1.51.0-3 (2013-10-19)
The other change is that I am now acting as maintainer taking over from Jay who has headed the initial creation and first releases after he, Mike and I had talked about this for way too long without actually doing anything about it. Thanks for getting everything rolling, Jay! Comments and suggestions are welcome via the mailing list or issue tracker available via the package page at R-Forge.

16 October 2013

Carl Chenet: Brebis 0.7, the fully automated backup checker

Follow me on Identi.ca or Twitter The new version of Brebis Chevillette 0.7 was published two days ago. Brebis is a fully automated backup checker. Brebis does not create backups. It only performs controls on them. Brebis is able to detect a corrupted archive compressed with different formats, automatically checking the integrity of your archives and lots of parameters of the files inside these archives or inside a tree of files and provide detailed reports if unexpected states are detected. Check out the list of the supported features. What s new? The main features of this new version are: Complete example of installing, configuring and running Brebis How to install from your package manager Brebis is available in Debian, FreeBSD, Archlinux and Frugalware. Use your favorite package manager to install it. How to install from the sources 
# wget http://brebisproject.org/attachments/download/8/brebis-0.7.tar.gz
# tar zxvf brebis-0.7.tar.gz && cd brebis-0.7
# python3.3 setup.py install --install-scripts=/usr/bin
# mkdir /etc/brebis
What you need before running brebis Our goal is to check on a regular basis the archive /backups/monthly-backup.tar.gz. Brebis needs two files in order to correctly run: a configuration file with general parameters and the list of files with all their parameters. Don t be afraid, both of theses files will be generated by Brebis with the following command: 
# brebis -G /backups/monthly-backup.tar.gz
# ls /backups/
monthly-backup.tar.gz monthly-backup.conf monthly-backup.list
All the files inside your archives have been detected and their names and parameters (uid, gid, mode, md5, ) have been written in the file monthly-backup.list. More information about those files. How to configure Brebis Lets save /backups/monthly-backup.list in /etc/brebis: # mv /backups/monthly-backup.list /etc/brebis For /backups/monthly-backup.conf, we just modify the files_list parameters: 
[main]
name=monthly-backup
type=archive
path=/backups/monthly-backup.tar.gz
files_list=/etc/brebis/monthly-backup.list
We also save monthly-backup.conf in /etc/brebis: # mv /backups/monthly-backup.conf /etc/brebis Use Brebis Using Brebis is as simple as: # brebis -c /etc/brebis/ -l /var/log/brebis.log If there is not any difference between the current state of your backup and your list of file, the logfile /var/log/brebis.log will be empty. If any difference happens, they will be reported in /var/log/brebis.log. Show time. I m modifying the md5 hash sum of a file in /etc/brebis/monthly-backup.list, during the next run of brebis, I get the following content in brebis.log: 
WARNING:root:1 file with unexpected hash while checking
/backups/monthly-backup.tar.gz:
WARNING:root:toto/titi hash is ce4f8cacd8fc702bdd03531b9447818b.
Should have been ce4f8cacd8fc702bdd03531b94478184.
As you can read, Brebis reports a difference between what was expected and what it checks. Now it s easy to do the same for any backup you want to check on a regular basis. Just add something like the next line in /etc/crontab: 01 00 * * * root brebis -c /etc/brebis/ -l /var/log/brebis.log A brebis run will happen every day at midnight to check your backup. Every configuration files in the directory /etc/brebis/ will be inspected. Feedback about Brebis We at the Brebis Project welcome any feedback about Brebis. Feel free to subscribe to the Brebis-users mailing list, by Twitter or email me directly at carl.chenet@brebisproject.org 
Official website: http://www.brebisproject.org
Mailing-list: http://lists.sourceforge.net/lists/listinfo/brebis-users

11 October 2013

Petter Reinholdtsen: Oslo community mesh network - with NUUG and Hackeriet at Hausmania

Wireless mesh networks are self organising and self healing networks that can be used to connect computers across small and large areas, depending on the radio technology used. Normal wifi equipment can be used to create home made radio networks, and there are several successful examples like Freifunk and Athens Wireless Metropolitan Network (see wikipedia for a large list) around the globe. To give you an idea how it work, check out the nice overview of the Kiel Freifunk community which can be seen from their dynamically updated node graph and map, where one can see how the mesh nodes automatically handle routing and recover from nodes disappearing. There is also a small community mesh network group in Oslo, Norway, and that is the main topic of this blog post. I've wanted to check out mesh networks for a while now, and hoped to do it as part of my involvement with the NUUG member organisation community, and my recent involvement in the Freedombox project finally lead me to give mesh networks some priority, as I suspect a Freedombox should use mesh networks to connect neighbours and family when possible, given that most communication between people are between those nearby (as shown for example by research on Facebook communication patterns). It also allow people to communicate without any central hub to tap into for those that want to listen in on the private communication of citizens, which have become more and more important over the years. So far I have only been able to find one group of people in Oslo working on community mesh networks, over at the hack space Hackeriet at Husmania. They seem to have started with some Freifunk based effort using OLSR, called the Oslo Freifunk project, but that effort is now dead and the people behind it have moved on to a batman-adv based system called meshfx. Unfortunately the wiki site for the Oslo Freifunk project is no longer possible to update to reflect this fact, so the old project page can't be updated to point to the new project. A while back, the people at Hackeriet invited people from the Freifunk community to Oslo to talk about mesh networks. I came across this video where Hans J rgen Lysglimt interview the speakers about this talk (from youtube): I mentioned OLSR and batman-adv, which are mesh routing protocols. There are heaps of different protocols, and I am still struggling to figure out which one would be "best" for some definitions of best, but given that the community mesh group in Oslo is so small, I believe it is best to hook up with the existing one instead of trying to create a completely different setup, and thus I have decided to focus on batman-adv for now. It sure help me to know that the very cool Serval project in Australia is using batman-adv as their meshing technology when it create a self organizing and self healing telephony system for disaster areas and less industrialized communities. Check out this cool video presenting that project (from youtube): According to the wikipedia page on Wireless mesh network there are around 70 competing schemes for routing packets across mesh networks, and OLSR, B.A.T.M.A.N. and B.A.T.M.A.N. advanced are protocols used by several free software based community mesh networks. The batman-adv protocol is a bit special, as it provide layer 2 (as in ethernet ) routing, allowing ipv4 and ipv6 to work on the same network. One way to think about it is that it provide a mesh based vlan you can bridge to or handle like any other vlan connected to your computer. The required drivers are already in the Linux kernel at least since Debian Wheezy, and it is fairly easy to set up. A good introduction is available from the Open Mesh project. These are the key settings needed to join the Oslo meshfx network:
SettingValue
Protocol / kernel modulebatman-adv
ESSIDmeshfx@hackeriet
Channel / Frequency11 / 2462
Cell ID02:BA:00:00:00:01
The reason for setting ad-hoc wifi Cell ID is to work around bugs in firmware used in wifi card and wifi drivers. (See a nice post from VillageTelco about "Information about cell-id splitting, stuck beacons, and failed IBSS merges! for details.) When these settings are activated and you have some other mesh node nearby, your computer will be connected to the mesh network and can communicate with any mesh node that is connected to any of the nodes in your network of nodes. :) My initial plan was to reuse my old Linksys WRT54GL as a mesh node, but that seem to be very hard, as I have not been able to locate a firmware supporting batman-adv. If anyone know how to use that old wifi access point with batman-adv these days, please let me know. If you find this project interesting and want to join, please join us on IRC, either channel #oslohackerspace or #nuug on irc.freenode.net. While investigating mesh networks in Oslo, I came across an old research paper from the university of Stavanger and Telenor Research and Innovation called The reliability of wireless backhaul mesh networks and elsewhere learned that Telenor have been experimenting with mesh networks at Gr nerl kka in Oslo. So mesh networks are also interesting for commercial companies, even though Telenor discovered that it was hard to figure out a good business plan for mesh networking and as far as I know have closed down the experiment. Perhaps Telenor or others would be interested in a cooperation? Update 2013-10-12: I was just told by the Serval project developers that they no longer use batman-adv (but are compatible with it), but their own crypto based mesh system.

10 October 2013

Russ Allbery: Review: 2312

Review: 2312, by Kim Stanley Robinson
Publisher: Orbit
Copyright: 2012
ISBN: 0-316-19280-5
Format: Kindle
Pages: 563
Swan Er Hong is a native of Mercury, granddaughter of one of the most influential women in the solar system. Alex, known as the Lion of Mercury, has just died as the story opens, and Swan is mourning. But she's also discovering that something else has been going on, something that Alex had not confided in her. Something that involves an inspector from the asteroids, a scientist from Titan, and a message Alex left in the event of her death, asking Swan to convey something personally to another scientist on Io. And then Mercury is attacked and the situation becomes much more complicated. 2312 is, like most Robinson novels, built on top of monumental world-building. The date is given away by the title; the setting is a solar system full of terraforming, habitats, and explosive human colonization. There are human settlements on every world that could possibly sustain them, including a terraformed Mars and a vast in-progress terraforming project on Venus. Mercury, Song's home, supports the city known as Terminator, a shielded habitat that travels around the planet on rails, pushed forward by the thermal expansion of metal at sunrise. Humans have scattered across the solar system and changed everything they've touched, even the asteroids. Many of those have been repurposed as long-haul shuttles: self-contained habitats on carefully-designed routes on which people live for months or years until they get to their destinations. Parts of this book are gorgeous. One of Robinson's strengths is creating images of the vastness of space, science, and human endeavor that feel plausible and inspiring at the same time. The setting of 2312 frees him to go wild with mega construction projects, and he takes full advantage. Parts of the book inspire the same sense of awe as the Hoover Dam or the Great Wall of China, transplanted into space and backed with neat scientific ideas to ponder. (Even if the more skeptical will note certain human habitation hazards, such as prolonged weightlessness and radiation, are mostly hand-waved away.) Unusually for a Robinson novel, 2312 also features an interesting and vibrant protagonist. Usually I can barely remember Robinson's protagonists five minutes after I finish the book, but Swan is so impulsive, erratic, and emotionally intense that I doubt I'll forget her. Her past is stuffed with outrageous risks and intense creativity: having songbird neurons transplanted into her brain, ingesting extraterrestrial bacteria, designing habitats in the asteroid belt, and creating art on the plains of Mercury. As the story gets underway, she meets the oversized and mellow Wahram; at first, she finds him boring, but then they're thrown together by one of the best set pieces of the book, and both they and the reader discover they work well together. Wahram serves as a straight man and as a stand-in for the reader's opinion, showing Swan all the more clearly in contrast. All this sounds quite promising. Unfortunately, what makes 2312 an ambitious failure rather than an excellent story is the story part. Robinson has built an amazing world, but he doesn't seem to know what to do with it. His economy, at least outside of Earth (and more on that in a moment), is essentially post-scarcity. Few of the characters we meet seem to have any concept of fiscal or practical limitation. They head across the solar system on a moment's notice, take vacations in habitats, or head to favorite spots for the view. It felt like Robinson was trying to write a Culture story. Unlike Banks, however, he struggles with generating a plot. 2312 is, in structure, a murder mystery. The characters witness a crime, survive it, and then intend to investigate it, and the crime appears to grow and link to fault lines in their society. This could have worked, but it has two problems: their society is only barely coherent enough for the reader to perceive the potential fault lines, and Robinson seems to have no idea how to construct a mystery plot. There is no sense of coherence to Swan and Wahram's approach, no sense of a gathering of clues, and very little sense of drama. All of the plot revelations are either dropped in Swan's lap by another character at a convenient moment or generated by Swan refusing to take the plot of the book seriously. The characters do essentially no meaningful investigation on-camera and show almost no investment in the outcome of the plot. When the climax comes, it's all weirdly forgettable. There are also large sections of the book that appear to have nothing to do with the rest of the plot, and that's where the unfortunate interval on Earth comes in. Robinson takes advantage of the scenes on Earth to do a bit of alienation and shows how foreign and strange and stifling Earth feels to someone who grew up outside of its atmosphere. Parts of this work, but he puts the plot on hold to do it. And parts of it do not work at all. The most glaring example is Swan and Wahram's bizarre bit of attempted charity in Africa, which comes across as stunningly high-handed and arrogant. This could be in character, particularly for Swan (who is not long on empathy), but, if so, the book doesn't signal to the reader that it should be read that way. Instead, there are some side (or snide) comments that seem to indicate Robinson knows nothing about the economic arc of Africa from the past twenty years. And when their absurd, botched, condescending charity plan fails for all the obvious reasons, the characters, and apparently the novel, throw up their hands and write Earth off as a stagnant lost cause that can't accept the imposition of a good idea and go back to the plot, never apparently caring about Earth again. Almost as frustrating is the way that these interludes are tied back into the story, which is usually through Swan getting ridiculously lucky on her random encounter rolls. It felt like whenever Robinson needed to make progress in the plot, Swan would just accidentally run into exactly the right person or situation to bring up the next plot point or to have some investigation make sense. (Not that Swan usually figured this out. Normally, the inspector explains it to her.) The author's finger was planted so firmly on the scales that it destroyed my suspension of disbelief and made a mockery of the idea that the characters were actually investigating anything. 2312 is built around a skeleton of a plot, but the lack of engagement with it, the lack of tension and emotion, the way the next developments are generally narrated to the protagonists and the reader, and the repeated use of random encounters to steer it left me without much reason to care. Robinson tries a few twists, but since the story never felt committed to its plot anyway, those twists feel less like planned complications and more like another random veer in the road. It didn't help that the final outcome was more prosaic and forgettable than the book had been implying it would be. There is a lot to like here. I'm very pleased to see Robinson finally write a memorable protagonist, and he's very good at both world-building and set pieces. But it needed a plot, and it needed a more coherent and complete cultural backdrop for its characters. Without both, it fell back into the typical Robinson trap: gorgeous moments separated by a whole lot of boring, and an overall impression of a construction tour rather than a story. There are bits of it I loved and still remember, particularly Song and Wahram in the tunnels of Mercury (which is possibly the best extended characterization Robinson has ever written). But the book as a whole is a mess, and I can't recommend wading through it for the good parts. Rating: 5 out of 10

8 October 2013

Daniel Kahn Gillmor: Unaccountable surveillance is wrong

As I mentioned earlier, the information in the documents released by Edward Snowden show a clear pattern of corporate and government abuse of the information networks that are now deeply intertwined with the lives of many people all over the world. Surveillance is a power dynamic where the party doing the spying has power over the party being surveilled. The surveillance state that results when one party has "Global Cryptologic Dominance" is a seriously bad outcome. The old saw goes "power corrupts, and absolute power corrupts absolutely". In this case, the stated goal of my government appears to be absolute power in this domain, with no constraint on the inevitable corruption. If you are a supporter of any sort of a just social contract (e.g. International Principles on the Application of Human Rights to Communications Surveillance), the situation should be deeply disturbing. One of the major sub-threads in this discussion is how the NSA and their allies have actively tampered with and weakened the cryptographic infrastructure that everyone relies on for authenticated and confidential communications on the 'net. This kind of malicious work puts everyone's communication at risk, not only those people who the NSA counts among their "targets" (and the NSA's "target" selection methods are themselves fraught with serious problems). The US government is supposed to take pride in the checks and balances that keep absolute power out of any one particular branch. One of the latest attempts to simulate "checks and balances" was the President's creation of a "Review Group" to oversee the current malefactors. The review group then asked for public comment. A group of technologists (including myself) submitted a comment demanding that the review group provide concrete technical details to independent technologists. Without knowing the specifics of how the various surveillance mechanisms operate, the public in general can't make informed assessments about what they should consider to be personally safe. And lack of detailed technical knowledge also makes it much harder to mount an effective political or legal opposition to the global surveillance state (e.g. consider the terrible Clapper v. Amnesty International decision, where plaintiffs were denied standing to sue the Director of National Intelligence because they could not demonstrate that they were being surveilled). It's also worth noting that the advocates for global surveillance do not themselves want to be surveilled, and that (for example) the NSA has tried to obscure as much of their operations as possible, by over-classifying documents, and making spurious claims of "national security". This is where the surveillance power dynamic is most baldly in play, and many parts of the US government intelligence and military apparatus has a long history of acting in bad faith to obscure its activities. The people who have been operating these surveillance systems should be ashamed of their work, and those who have been overseeing the operation of these systems should be ashamed of themselves. We need to better understand the scope of the damage done to our global infrastructure so we can repair it if we have any hope of avoiding a complete surveillance state in the future. Getting the technical details of these compromises in the hands of the public is one step on the path toward a healthier society. PostscriptLest I be accused of optimism, let me make clear that fixing the technical harms is necessary, but not sufficient; even if our technical infrastructure had not been deliberately damaged, or if we manage to repair it and stop people from damaging it again, far too many people still regularly accept ubiquitous private (corporate) surveillance. Private surveillance organizations (like Facebook and Google) are too often in a position where their business interests are at odds with their users' interests, and powerful adversaries can use a surveillance organization as a lever against weaker parties. But helping people to improve their own data sovereignty and to avoid subjecting their friends and allies to private surveillance is a discussion for a separate post, i think.Tags: cryptography, nsa

7 October 2013

Vincent Fourmond: Release 0.7 of ctioga2

Quite a bit of work has gone into ctioga2 since my last post. Among the most interesting features, ctioga2 now features:
The latter feature proved actually instrumental in a research project under way in my lab (which is why I wrote it in the first place). It is demonstrated in the image above, made using this code.I've also started the gnuplot versus ctioga2 gallery I was speaking about in my last post, where I try to reproduce most of gnuplot's examples using ctioga2; it can be found here. While I don't like so much the look of the gnuplot graphs, and I try to keep the look of ctioga2's as close as possible to that look, it proved invaluable in detecting small bugs, fixing small glitches and providing inspiration for new features.In the meantime, my todo-list for ctioga2 doesn't seem to get any smaller. Let's hope I'll implement them fast enough ! Keep posted...

5 October 2013

Ian Campbell: qcontrol: support for x86 devices

Until now qcontrol has mostly only supported only ARM (kirkwood) based devices (upstream has a configuration example for the HP Media Vault too, but I don't know if it is used). Debian bug #712191 asked for at least some basic support for x86 based devices. The mostly don't use the QNAP PIC used on the ARM devices so much of the qcontrol functionality is irrelevant but at least some of them do have a compatible A125 LCD. Unfortunately I don't have any x86 QNAP devices and I've been unable to figure out a way to detect that we are running on an QNAP box as opposed to any random x86 box so I've not been able to implement the hardware auto-detection used on ARM to configure qcontrol for the appropriate device at installation time. I don't want to include a default configuration which tries to drive an LCD on some random serial port since I have no way of knowing what will be on the other end or what the device might do if sent random bytes of the LCD control protocol. So I've implemented debconf prompting for the device type which is used only if auto-detection fails, so it shouldn't change anything for existing users on ARM. You can find this in version 0.5.2-3~exp1 in experimental (see DebianExperimental on the Debian wiki for how to use experimental). Currently the package only knows about the existing set of ARM platforms and a default "unknown" platform, which has an empty configuration. If you have a QNAP device (ARM or x86) which is not currently supported then please install the package from experimental and tailor /etc/qcontrol.conf for you platform (e.g. by uncommenting the a125 support and giving it the correct serial port). Then send me the result along with the device's name. If the device is an ARM one please also send me the contents of /proc/cpuinfo too so I can implement auto-detection. If you know how to detect a particular x86 QNAP device programmatically (via DMI decoding, PCI probing, sysfs etc, but make sure it is 100% safe on non-QNAP platforms) then please do let me know.

Vincent Sanders: If I have a style, I am not aware of it.

I wish I had known about that quote from Michael Graves before now. I would have perhaps had an answer to some recent visitors to makespace.

There are regular scheduled visits to makespace where people can come and view our facilities, and perhaps start the process of becoming a member if they decide they like what they see.

Varnishing a folding chair using a stool as a stand
However, we sometimes get people who just turn up at the door. If a member feels charitable they may choose to give a short tour rather than just turn the person away. I happened to be in one Friday afternoon recently varnishing a folding chair when two such people rang the doorbell. There were few other members about and because watching varnish dry was dull I decided to be helpful and do a quick tour.

I explained that they really ought to return for a scheduled event for a proper tour, gave the obligatory minimal safety briefing, and showed them the workshops and tools. During the tour it was mentioned they were attending a certain local higher education establishment and were interested in makespace as an inexpensive studio.

Before they left I was asked what I was working on. I explained that I had been creating stools and chairs from plywood. At this point the conversation took a somewhat surreal turn, one of them asked, well more demanded, who my principle influence had been in designing with plywood.

When I said that I had mainly worked from a couple of Google image searches they were aghast and became quite belligerent. They both insisted I must have done proper research and my work was obviously influenced by Charles and Ray Eames and Arne Jacobsen and surely I intended to cite my influences in my design documents.

My admission that I had never even heard the names before and had no design documents seemed to lead to a distinctly condescending tone as they explained that all modern plywood design stemmed from a small number of early 20th century designers and any competent designers research would have revealed that.

At this point in proceedings I was becoming a bit put out that my good deed of showing off the workshop had not gone unpunished. I politely explained that I designed simply by generating a requirement in my head, maybe an internet search to see what others had done, measuring real things to get dimensions and then a great deal of trial and error.

I was then abruptly informed that my "design process was completely invalid and there were well established ways to design furniture correctly and therefore my entire design was invalid" and that I was wasting time and material. I thanked them for their opinion and showed them out, safe and well before anyone gets any ideas.

I put the whole incident out of my mind until I finished writing up the final folding chair post the other day. It struck me that perhaps I had been unknowingly influenced by these designers. It was certainly true I had generated ideas from the hundreds of images my searches had revealed.

I did some research and it turns out that from the 1930s to the 1950s there were a string of designers using plywood in novel ways from the butterfly stool by Sori Yanagi through formed curvy chairs by Alvar Aalto and eero saarinen.

While these designers produced some wonderfully iconic and contemporary furniture I think that after reviewing my initial notes that two more modern designers Christian Desile and Leo Salom probably influenced me more directly. Though I did not reference their designs beyond seeing the images along with hundreds of others, certainly nothing was directly copied.

And there in lies an often repeated observation: no one creates anything without being influenced by their environment. The entire creative process of several billion ape descendants (or Golgafrincham telephone sanitisers if you prefer) is based on the simple process of copying, combining and transforming what is around us.

Isaac Newton by Sir Godfrey Kneller [Public domain], via Wikimedia Commons
I must accept that certain individuals at points in history have introduced radical improvements in their field, people like Socrates Galileo Leonardo Newton Einstein. However, even these outstanding examples were enlightened enough to acknowledge those that came before. Newtons quotation "If I have seen further it is by standing on the shoulders of giants" pretty much sums it up.

In my case I am privileged enough to live in a time where my environment has grown to the size of the world thanks to the internet. My influences, and therefore what I create, is that much richer but at the same time it means that my influence on others is similarly diminished.

I have joined the maker community because I want to create. The act of creation teaches me new skills in a physical, practical way and additionally I get to exercise my mind using new techniques or sometimes things I had forgotten I already knew. I view this as an extension of my previous Open Source software work, adding a physical component to a previously purely mental pursuit.

But importantly I like that my creations might provide inspiration for someone else. To improve those chances in the wider world I force myself to follow a few basic rules:
Release
Possibly one of the hardest things for any project. I carefully avoided the word finish here because my experience leads me to the conclusion that I always want to improve my designs.

But it is important to get to a point in a project where you can say "that is good enough to share", this is more common in software but it really applies to any project.

Share
If your aim here is to improve your society with your contribution sharing your designs and information is important. I think there is nothing better than someone else taking one of your designs and using it and perhaps improving on it, remember that is what you probably did in one way or another better to make it less hard for that to happen.

Ensure your design files are appropriately licensed and they are readily accessible. I personally lean towards the more generally accessible open source licences like MIT but the decision is ultimately yours.

Licencing is important, especially in the current copyright happy society. I know it sounds dull and no one takes that seriously, right? Sorry, but yes they do and it is better for you to be clear from the start, especially if there is a software component to your project. Oh one personal plea, use an existing well known licence, the world simply does not need another one!

Write about it
The blog posts about the things I have made sometimes take almost as much time as the creation. The clear recording on my thoughts in written and photographic form often gives me more inspiration for improvements or other projects.

If someone else gets pleasure from the telling then that can only be good. If you do not do this then your voice cannot be heard and you wasted an opportunity to motivate others.

Feedback
If you do manage to get feedback on your creation, read it. You may disagree or not be interested for the current project but the feedback process is important. In software this often manifests as bug reports, in more physical projects this often becomes forum or blog comments.

Just remember that you need a thick skin for this, the most vocal members of any society are the minority with inflammatory opinion, the silent majority are by definition absent but there is still useful feedback out there.

Create again
By this I simply mean that once you are satisfied yourself on a project move on to the next. This may sound a little obvious but once you have some creative momentum it is much easier to keep going project to project than in you leave time between.

Also do not have too many projects ongoing, by all means have a couple so you do not get stuck waiting for materials or workshop time but more than three and four and you will never be able to release any of them.
Those students were perhaps somewhat misguided in how they stated their opinions, but they are correct that in the world in which we find ourselves we are all influenced. Though contrary to received wisdom those influences are more likely to be from the internet and our peers in the global maker society than historical artists.

Steve Kemp: I understand volunterering is hard

The tail end of this week was mostly spoiled by the discovery that libbeanstalkclient-ruby was not included in Wheezy. Apparently it was removed because the maintainer had no time, and there were no reverse dependencies - #650308. Debian maintainers really need to appreciate that no official dependencies doesn't mean a package is unused. Last year I needed to redesign our companies monitoring software, because we ran out of options that scaled well. I came up with the (obvious) solution: I chose beanstalkd for my central queue precisely because it was packaged for Debian, had a client library I could use, and seemed to be a good fit. It was a good fit, a year on and we're still running around 5000 tests every minute with 10 workers. The monitoring tool is called Custodian Custodian, and I think I've mentioned it before here and on the company blog. It looks like we'll need to re-package the Ruby beanstalk client, and distribute it alongside our project now. That's not ideal, but also not a huge amount of work. In summary? Debian you're awesome. But libraries shouldn't be removed unless it can't be helped, because you have more users than you know.

3 October 2013

Joachim Breitner: Experimenting with the TI eZ430 Chronos

For a while I have been eyeing with the TI eZ430 Chronos watch, a slightly bulky sports watch that not only comes with a built-in thermometer, barometer and accelerometer, but also a freely programmable micro controller (from the TI MSP430 family hence the name). In addition it has a small radio chip included and can communicate wirelessly with the computer, or possibly other devices. All in all, a very geeky toy. A week ago I managed to buy one quite cheaply via eBay (30 including shipping). It took me a while to get an overview of the various resources about the device. Here is what I have learned. After I found that out, I tried to hack a bit on the various software. The first issue I had (and still have) is that I cannot use the buttons of the device while it is attached to the debug board. They do work if forget to take out the battery, but supposedly that is bad for the battery. So my compile-upload-test-cycle becomes a compile-upload-detach-insert-battery-test-remove-battery-attach-cycle. I could not find out how to emulate button presses using mspdebug. Any suggestions are welcome. The other problem was the new altimeter hardware: Neither OpenChronos nor openchronos-ng support that yet. I originally planned to port the driver from the official software to a random OpenChronos fork when I read that Richard Rondu did that to openchronos-ng yet. I found the code in a month-old merge-request. I tried it, it worked, and I started to clean up the code a bit and add a variometer mode to it, e.g. displaying the current rate of climb my ultimate goal is to use this as a surprisingly cheap variometer for paragliding. After a day of hacking and quickly lifting the watch from the room floor to the room ceiling and back I believe that it works okish, although quite a few parameter can benefit from more tweaking: sample rate (currently 20Hz), length of history (currently 2s) and the linear regression method (currently simply the difference between the average of the samples of the current second with the samples of the last second). As always, patches welcome, the code is in my fork on sf.net.
Next to do would be to enable some buzzing noise depending on the rate of change; during flight I don t want to decipher a low-contract LCD screen. But it is questionable whether the buzzer is loud enough we ll see. I was also happy to hear that there are other zE430-hackers in Karlsruhe, one of them for example tries to make the alarm clock sleep-cycle-aware.

2 October 2013

Axel Beckert: How to make wget honour Content-Disposition headers

Download links often point to CGI scripts which actually generate (or just fetch, i.e. proxy) the actual file to be downloaded, e.g. URLs like http://www.example.com/download.cgi?file=foobar.txt. Most of such CGI scripts send the real file name in the Content-Disposition header as specified in the MIME Specification. All browsers I know (well, at least those I use regularily :-) handle that perfectly and propose the file name sent in the Content-Disposition header as file name for saving the downloaded name which is usually exactly what I want. All browsers do that, , just not my favourite commandline download tool GNU Wget Downloading the above URL with wget would look like this with default settings: 
$ wget 'http://www.example.com/download.cgi?file=foobar.txt'
--2013-10-02 16:04:16--  http://www.example.com/download.cgi?file=foobar
Resolving www.example.com (www.example.com)... 93.184.216.119, 2606:2800:220:6d:26bf:1447:1097:aa7
Connecting to www.switch.ch (www.example.com) 2606:2800:220:6d:26bf:1447:1097:aa7 :80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2020 (2.0K) [text/plain]
Saving to:  download.cgi?file=foobar.txt'
100%[============================================>] 2,020       --.-K/s   in 0s
2013-10-02 16:04:24 (12.5 MB/s) -  download.cgi?file=foobar.txt' saved [2020/2020]
Meh! But luckily Wget can do that, it s just not enabled by default because it s an experimental and possibly buggy feature, at least according to the man page. Well, works for me! :-) You can easily enabled it by default for either your user or the whole system by placing the following line in your ~/.wgetrc or /etc/wgetrc: 
content-disposition = on
Given the CGI script sends an appropriate Content-Disposition header, the above output now looks like this: 
$ wget 'http://www.example.com/download.cgi?file=foobar.txt'
--2013-10-02 16:04:16--  http://www.example.com/download.cgi?file=foobar
Resolving www.example.com (www.example.com)... 93.184.216.119, 2606:2800:220:6d:26bf:1447:1097:aa7
Connecting to www.switch.ch (www.example.com) 2606:2800:220:6d:26bf:1447:1097:aa7 :80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2020 (2.0K) [text/plain]
Saving to:  foobar.txt'
100%[============================================>] 2,020       --.-K/s   in 0s
2013-10-02 16:04:24 (12.5 MB/s) -  foobar.txt' saved [2020/2020]
Now Wget does what I mean! You can also set this as flag on the commandline, but typing wget --content-disposition everytime is surely not what I want. ;-)

Raphael Geissert: A bashism a week: dangerous exports

As a user of a shell you have most likely had the need to export a variable to another process; i.e. set/modify an environment variable.

Now, how do you stop exporting an environment variable? can you export anything else?

The bash shell offers the -n option of the export built-in, claiming it "remove[s] the export property". However, this feature is not part of the POSIX:2001 standard and is, therefore, a bashism.

A portable way to stop exporting an environment variable is to unset it. E.g. the effect of "export MY_VAR=foo" can be reverted by calling "unset MY_VAR" - surely enough, this will also destroy the content of the variable.

An equivalent could then be:

# to stop exporting/"unexport" the MY_VAR environment variable:
my_var="$MY_VAR" ; unset MY_VAR ;
MY_VAR="$my_var" ; unset my_var ;

The above code will make a copy of the variable before destroying it and then restoring its content.

How about exporting other things? did you know that you can export shell functions?

With the bash shell, you can export a function with the -f parameter of the export built-in. Needless to say, this is a bashism. Its secret? it's just an environment variable with the name of the function and the rest of the function definition as its value.

Try this:

$ echo="() /bin/echo 'have some bash' ; " bash -c 'echo "Hello World!"'
have some bash

Yes, this means that if you can control the content of an environment variable passed to bash you can probably execute whatever code you want. It comes handy when you want to alter a script's behaviour without modifying the script itself.

Possibilities are endless thanks to bash's support for non-standard characters in function names. Functions with slashes can also be exported, for example:

/sbin/ifconfig()
echo "some people say you should be using ip(1) instead" ;


Are you into bug hunting? export exec='() echo mount this ; '

30 September 2013

Dirk Eddelbuettel: RcppArmadillo 0.3.920.1

Along with the Rcpp 0.10.5 release yesterday, a new minor release 0.3.920.1 of RcppArmadillo came out. It is based on Conrad's Armadillo 3.920.0 plus a minor fix, and uses some of the new Rcpp features. Both package is now on CRAN and also in Debian. This releases contains both a nice set of new Armadillo features as well as some nice additions to RcppArmadillo, due again mostly to Romain. Some of the changes tie into the changes changes in Rcpp 0.10.5 as for example the ability to pass const and const ref more efficiently (and we seem to have forgotten an entry in the NEWS file). The complete list of changes is below.
Changes in RcppArmadillo version 0.3.920.1 (2013-09-27)
  • Upgraded to Armadillo release Version 3.920.1 (Agencia Nacional Stasi)
    • faster .zeros()
    • faster round(), exp2() and log2() when using C++11
    • added signum function: sign()
    • added move constructors when using C++11
    • added 2D fast Fourier transform: fft2()
    • added .tube() for easier extraction of vectors and subcubes from cubes
    • added specification of a fill type during construction of Mat, Col, Row and Cube classes, eg. mat X(4, 5, fill::zeros)
  • Initial implementation of wrap<subview>
  • Improved implementation of as<>() and wrap() for sparse matrices
  • Converted main vignette from LaTeX style minted to lstlisting which permits builds on CRAN; removed set BuildVignettes: FALSE.
Courtesy of CRANberries, there is also a diffstat report for the most recent release As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

Next.

Previous.