Search Results: "ag"

4 February 2014

Enrico Zini: debops

Debops What I like the most about being a Developer is building tools to (hopefully) make someone's life better. I like it when my software gets used, and people thank me for it, because there was a need they had that wasn't met before, and thanks to my software now it is being met. I am maintaining software for meteorological research that is soon going to be 10 years old, and is still evolving and getting Real Work done. I like to develop software as if it is going to become a part of human cultural heritage, developing beyond my capacity, eventually surviving me, allowing society to declare that the need, small as it was, is now met, and move on to worry about some other problem. I feel that if I'm not thinking of my software in that way, then I am not being serious. Then I am not developing something fit for other people to use and rely on. This involves Development as much as it involves Operations: tracking security updates for all the components that make up a system. Testing. Quality assurance. Scalability. Stability. Hardening. Monitoring. Maintenance requirements. Deployment and upgrade workflows. Security. I came to learn that the requirements put forward by sysadmins are to be taken seriously, because they are the ones whose phone will ring in the middle of the night when your software breaks. I am also involved in more than one software project. I am responsible for about a dozen web applications deployed out there in the wild, and possibly another dozen of non-web projects, from terabyte-sized specialised archival tools to little utilities that are essential links in someone's complex toolchain. I build my software targeting Debian Stable + Backports. At FOSDEM I noticed that some people consider it uncool. I was perplexed. It provides me with a vast and reasonably recent set of parts to use to build my systems. It provides me with a single bug tracking system for all of them, and tools to track known issues in the systems I deployed. It provides me with a stable platform, with a well documented upgrade path to the next version. It gives me a release rhythm that allows me to enjoy the sweet hum of spinning fans thinking about my next mischief, instead of spending my waking time chasing configuration file changes and API changes deep down in my dependency chain. It allows me to rely on Debian for security updates, so I don't have to track upstream activity for each one of the building blocks of the systems I deploy. It allows me not to worry about a lot of obscure domain specific integration issues. Coinstallability of libraries with different ABI versions. Flawless support for different versions of Python, or Lua, or for different versions of C++ compilers. It has often happened to me to hear someone rant about a frustrating situation, wonder how come it had never happened to me, and realise that someone in Debian, who happens to be more expert than I can possibly be, had thought hard about how to deal with that issue, years before. I know I cannot be an expert of the entire stack from bare iron all the way up, and I have learnt to stand on the shoulders of giants. 'Devops' makes sense for me in that it hints at this cooperation between developers and operators, having constructive communication, knowing that each side has their own needs, trying their best to meet them all. It hints at a perfect world where developers and operators finally come to understand and trust each other's judgement. I don't know that perfect world, but I, a developer, do like to try to understand and trust the judgement of sysadmins. I sympathise with my sysadmin friends who feel that devops is turning into a trend of developers thinking they can do without sysadmins. Reinventing package managers. Bundling dependencies. Building "apps" instead of components. I wish that people who deploy a system built on such premises, have it become so successful that they end up being paid to maintain them for their whole career. That is certainly what I wish and strive for, for me and my own projects. In my experience, a sustainable and maintainable system won't come out of the startup mindset of building something quick&dirty, then sell it and move on to something else. In my experience, the basis for having sustainable and maintainable systems have been well known and tested in Debian, and several other distributions, for over two decades. At FOSDEM, we thought that we need a name for such a mindset. Between beers, that name came to be "debops". (It's not just Debian, though: many other distributions get it right, too)

Martín Ferrari: Telegram

This weekend I attended FOSDEM, as I did for the past 4 years. As always, it was a great experience, even if most of the talks I attended were not so interesting. One of the great things about these events, is to get together with other Free Software enthusiasts. One of them introduced me to the new kid on the block: Telegram. It is advertised as a free and secure replacement for WhatsApp, a mobile application I have been refusing to use for months, much to the anger of friends and family. People use WhatsApp because it saves them money to send SMS-like messages to other people. Which in my mind does not make much sense, as I can use email or XMPP for that same purpose, without having to enter just another walled garden. Sadly, people nowadays are using email less and less, and think that it is easier to send a Facebook message (it is not), and XMPP has became a de-facto walled garden since Google betrayed its users by dropping federation. WhatsApp has the advantage of having millions of users already inside their walled garden, like it happened with Facebook, MSN, or ICQ back in the day. Their success can possibly be attributed to the foolproof system it uses to discover contacts: it just sends your entire address book to their servers for matching against already registered users, which is in itself enough reason not to use it. So, when I was told of a secure, free and open alternative, I was eager to try it out. I started by opening their website to see what it was about. There, I found the first signs that something was wrong: no mention of licenses, almost no technical detail of how the protocol works, or how security is achieved. Still, my friend wanted to talk to me, so I rushed to install it and accept a laundry list of permissions in Android. That was a big mistake. The first thing the application did was to check my address book for contacts, without my permission or knowledge. I got greeted by being told that some of my contacts already have Telegram installed, and since then I keep getting notification that some more of my geek friends are installing it. So it is obvious that this company got all my records, breaking my privacy and security. This is enough for me to remove this piece of software from my phone, but it is not the end of the story. When checking it in more detail I found several other problems. The supposed "secret chat" cannot be possibly secure, as there is no verification of the remote party. It "just works", who cares if there is a man in the middle or not. Supposedly it employs a peer-to-peer connection, but I haven't verified that. On the other hand, the non-"secret" conversations are all routed to the main server. The client-to-server communication is supposedly encrypted, but it uses a home-made protocol (which everybody knows is a recipe for disaster), and the server has access to the cleartext of all your communications. Then, browsing the website, I've found that this "open" and "free" offer does not even have all the source code released. In particular, the server code is not public, nor you can set up your own server. Of course, it does not have federation, so even with the server code you wouldn't be able to talk with your friends. It all depends on two men funding and maintaining the project, so when the funds run out, one can only expect that all the users will be left in the dark. I think these arguments are enough to realise that Telegram is only marginally better than WhatsApp. It offers encryption at the transport layer, but you still are contributing to another walled garden, you are at the mercy of a company which does not have a funding plan, and the security practises range from weak to disturbing. In closing, if you are going to compromise your privacy, give away all your contacts' information, and rely on a single company to keep in touch with people, you might as well go and use what everybody else is using. Update: Even before submitting this post, I've found that some more qualified people has already dissected Telegram and concluded that it is basically snake oil. Please see this article. and proceed to uninstall Telegram.

Russell Coker: Clothing and Phone Cameras

In 2012 I wrote about my jeans from Rivers that fit the largest available phones (and the smaller tablets) in their pockets [1]. Those jeans are still working well for me, I can add the fact that they don t wear out quickly to the list of positive attributes. Recently my sister asked for advice on getting a new phone, she was considering the Samsung Galaxy Note 2 (the phone I m using now) because it apparently takes better pictures than the Nexus 4 she s using. I ve used both those phones and I hadn t noticed a difference in picture quality, but there is some variation in manufacturing and it could be that I ve got a below average Note 2 and a better than average Nexus 4 so I ll assume for the sake of discussion that my sister would actually get an improvement in picture quality by using a Note 2. If you have a phone that doesn t have the picture quality you desire then one option is to buy a phone with a better camera, but you will be limited by issues of physics. A thin phone has a short focal length which means that the lens has to be small and therefore the amount of light that gets to the sensor is small. The Nokia Lumia 1020 has some of the best camera hardware that you ll find in a phone, but it s still only 14.5mm thick where the camera is and that will limit the quality a lot. Any compact camera should be able to beat all phone cameras in terms of picture quality in most areas. The Samsung Galaxy Camera [2] is also worth considering, it has more features than a typical compact camera and good GUI that allows novice photographers to take advantage of it. Also being able to blog your photos directly from the camera could be a useful feature. But the big down-side of a compact camera is that it s not that compact. Most people won t find it convenient to carry a compact camera with them at all times and therefore they might miss a good opportunity to take a photo. The Galaxy Note series of phones also suffer in this regard because they are larger than most phones. If your phone won t fit in your pocket and you have it in your backpack when on the move or on a bench at home then you will probably miss some good photos. As I was at a Rivers store recently I tested my Note 2 in the pockets of women s jeans. Rivers scored very poorly in this regard, one pair of women s jeans had fake pockets (this is just wrong for working clothes) and of the rest only one pair could fit a Note 2. The pair that fit a Note 2 didn t completely enclose the phone, one corner was sticking out, this would probably give a risk of having the phone fall out of the pocket and cause some discomfort to the wearer. I have a pair of shorts with similar size pockets and find it very annoying with the Note 2 in the pocket (for about 10 months of the year I wear jeans so this isn t a big deal). Rivers jeans only count as geeky jeans for male geeks. It s disappointing that with about a dozen different styles of women s jeans there didn t seem to be a single one with pockets of comparable size to the men s jeans. I had to recommend that my sister not get a phone from the Galaxy Note series if taking pictures is a priority due to the apparent difficulty in getting it to fit in a pocket and the probability that she would miss good photos due to this. In past discussions of phone size there have been mentions of the possibility of getting clothing altered. Does anyone have a good experience in getting clothes altered to have bigger pockets or in the case of women s clothing to have fake pockets replaced with real ones?
Related posts:
  1. Geeky Jeans It s likely that most people make things like comfort, style,...
  2. Phone/Tablet Sizes The above two pictures show me holding a Samsung...
  3. Choosing an Android Phone My phone contract ends in a few months, so I m...

Andrew Pollock: [life] Day 8, another Kindergarten day

Yesterday was pretty crazy busy. I woke up a little bit late because I didn't have Zoe, and decided to head down to Jaycar to grab some components and other bits and pieces for my little spontaneous electronics project. I got home from Jaycar and fiddled around for a bit, and then biked down to Bulimba to get a ferry across to a bus so I could get into the city for lunch with my accountant. I ate my fill of sashimi while discussing the best corporate structure for my endeavours this year. Lunch ran a bit long, and it was going to be tight trying to bus it back to the ferry to retrieve my bike, so I decided to cab it back to Bulimba. I retrieved my bike and raced home and jumped in the car. It was just going to be too tight to bike over there. It probably worked out for the best, because I got to the Kindergarten and Zoe was fast asleep still. She took quite a lot of effort to wake. I think they'd only got her down for a nap not long before. I eventually roused her, but she was very groggy and a little bit sad about the whole thing. I think she would have been a sad panda if she'd had to then get on the bike. I asked her if she wanted to make some play dough or go swimming when we got home, and she chose swimming. I asked her if she wanted to bike or drive, and she wanted to drive. We took the new pool noodle that I picked up the other week and headed down to Colmslie Pool The pool was pretty busy with a lot of after school swim squads, but there was still enough room for Zoe to play around. Her sinkies were very popular and were "borrowed" extensively, and she really enjoyed riding the pool noodle like a horse. We got home and Sarah was waiting to pick up Zoe and I headed off for a massage, and then my girlfriend came over for dinner.

Diego Escalante Urrelo: Booting Fedora 20 from USB on a black MacBook3,1 (2007)

Here s a trick that is so obvious, I m still hitting my head against the desk. Let s say you want to install an EFI-friendly distribution on your MacBook. However, your MacBook does not detect bootable USB drives on startup (known bug on old Apple hardware). I found a blog post that had success rebuilding the ISO without EFI support and installing in BIOS mode, but I had a clear memory of getting this to work a few months ago without doing that. I just couldn t remember how. Well, turns out that OSX can boot from USB drives as long as you tell it to do so in advance. You can do this in the Boot disk/Startup disk panel in System Settings:
Startup disks panel in OSX
You don t see me complaining that everything on the web is in English, do you?
You only need to select the drive and close the panel, or reboot directly using that handy Reboot button. So considerate of Apple. Important: After you install, or try, your EFI friendly live USB you have to go back to the panel and select your OSX drive again, otherwise your MacBook will keep waiting for the (now gone) USB drive for about 30 seconds on startup. For Debian, the last few times I have just booted with a CD in BIOS mode without problems. After the installation you can configure Debian for EFI boot following this instructions. I haven t tried any other distribution on this machine.

NOKUBI Takatsugu: KAKASI 2.3.5 released

KAKASI 2.3.5 had released. The previous release was over 10 yers ago. In 2006, I tried to release 2.3.5, but I hadn t to complete relase jobs. However almost jobs were finished, so it was easy to release for me. But some bugs are discovered so I ll release 2.3.6 in a few days.

3 February 2014

Benjamin Mako Hill: Admiral Ackbar on Persian Governors

Title for a governor in ancient persia? Admiral Ackbar says: It's satrap!Q: The title for a governor in ancient Persia? A: It s satrap!

Evgeni Golov: Monitoring your Puppet nodes using PuppetDB

When you run Puppet, it is very important to monitor whether all nodes have an uptodate catalog and did not miss the last year of changes because of a typo in a manifest or a broken cron-script. The most common solution to this is a script that checks /var/lib/puppet/state/last_run_summary.yaml on each node. While this is nice and easy in a small setup, it can get a bit messy in a bigger environment as you have to do an NRPE call for every node (or integrate the check as a local check into check_mk). Given a slightly bigger Puppet environment, I guess you already have PuppetDB running. Bonuspoints if you already let it save the reports of the nodes via reports = store,puppetdb. Given a central knowledgebase about your Puppet environment one could ask PuppetDB about the last node runs, right? I did not find any such script on the web, so I wrote my own: check_puppetdb_nodes. The script requires a recent (1.5) PuppetDB and a couple of Perl modules (JSON, LWP, Date::Parse, Nagios::Plugin) installed. When run, the script will contact the PuppetDB via HTTP on localhost:8080 (obviously configurable via -H and -p, HTTPS is available via -s) and ask for a list of nodes from the /nodes endpoint of the API. PuppetDB will answer with a list of all nodes, their catalog timestamps and whether the node is deactivated. Based on this result, check_puppetdb_nodes will check the last catalog run of all not deactivated nodes and issue a WARNING notification if there was none in the last 2 hours (-w) or a CRITICAL notification if there was none for 24 hours (-c). As a fresh catalog does not mean that the node was able to apply it, check_puppetdb_nodes will also query the /event-counts endpoint for each node and verify that the node did not report any failures in the last run (for this feature to work, you need reports stored in PuppetDB). You can modify the thresholds for the number of failures that trigger a WARNING/CRITICAL with -W and -C, but I think 1 is quite a reasonable default for a CRITICAL in this case. Using check_puppetdb_nodes you can monitor the health of ALL your Puppet nodes with a singe NRPE call. Or even with zero, if your monitoring host can access PuppetDB directly.

0 comment(s) this blog is flattr enabled

Wouter Verhelst: Waking up

Today is the monday after FOSDEM 2014, and I'm slowly waking up. Being in charge of the video team, and after having spent a whole week preparing followed by a whole weekend stressing about, today is a day of slowly waking up from hibernation and returning gear to various rental companies. At FOSDEM 2013, we had recorded five rooms. Even though we did lose a few talks, I felt pretty good about the whole thing, and I think we did pretty well then. It had been my plan to increase the number of rooms to be recorded from 5 to a reasonably higher number say, 10 or so but someone convinced me to do all of them. I'll readily admit I was afraid. Scratch that, I was terrified. Petrified, that we wouldn't be able to pull it off. For previous editions, we'd kept the number conservative, since video work is a high-risk business: if something goes wrong with volunteers work (say, the heralding volunteer doesn't show up), you find a replacement, or you have a staff member introduce a speaker, and the show will go on. Not so with video work: if something minor goes wrong, you will most likely lose content. If something minor goes wrong in one devroom, I'll deal with it. If something minor goes wrong in 20% of devrooms, that's 4 to 5 devrooms, and I can't deal with it. Moreover, finding enough volunteers to manage the video for five rooms was a challenge last year; I couldn't even begin to imagine how to do so for 22 of them. We did work out a way that didn't seem too far-fetched, and that might work: why not look for volunteers from the devrooms themselves? But then, they couldn't be expected to have the required experience, so we'd have to do a lot of handholding. The week before FOSDEM, four of us sat down in an office in Mechelen, and started preparing. We had a lot to do; and when you do, more jobs keep being added to the pile; TODO lists tend to grow longer, not shorter. Installing servers, testing cameras, setting twinpact dipswitches, configuring laptops. Changing a few tests in the config management system, and having to go over the laptops again. Buying extra gear. Buying yet more extra gear, several times. Buying cardboard boxes when it appeared that the plastic foldable ones we'd ordered wouldn't make it in time, and unloading the pallet of them when it did, after all, so we could load them into the van with all of the other gear. Laminating instruction sheets. Receiving shipments of rented gear. Calling another supplier last minute when one of our other suppliers had contacted us to let us know one of their cameras that they'd promised us had broken down, and couldn't be repaired in time. Calling that last-minute supplier again when we found out that some of the cameras we'd rented could do HDV only (and not DV), and we needed to find more than the ones we'd already asked for. Discovering that his cameras were of the same problematic model and so also couldn't be used. The feeling of relief when he called me a few hours later and told me he'd managed to scrounge up enough cameras to help us out after all. And then it was time to go to the ULB and start setting up everything. On friday, we had a little bit of help, but did have to do most of it ourselves. After setting up everything in every room and testing as much as possible within the time constraints that we had, we picked up the most expensive bits of the gear, brought it down to a safe place, and went home. On saturday and sunday, for me it was mostly a matter of handing out gear to various devrooms, running around to fix various issues, and then in the evening receiving boxes back, using a checklist to ensure everything expected was in the box (and that things not needed would remain in the room, or be sent back). It was exhausting, to the extent that on saturday afternoon, I had to take a bit of time off so I could go and rest. I think it's safe to say that something like this hasn't been done before. FOSDEM is a huge conference; most conferences with multiple rooms don't have more somewhere between 5 and 10 of them. The fact that we have so many more, and the fact that we record all of them, puts FOSDEM video work in a class of its own, to the extent that the professional cameramen I talked to in order to get the right cameras would incredulously ask "what do you need that for?" when I gave them the numbers we needed. As such, I didn't expect, nor was aiming for, perfection. Unfortunately, with video work it is close to impossible to attain perfection; I have told multiple people during the event that I would be happy if we reached 85% of talks recorded at acceptable quality; and though it is much too soon to say for certain, my gut feeling tells m that we've probably achieved that. One thing we hadn't planned for was streaming. During the past three years, FOSDEM enlisted the help and sponsorship of flumotion to get streaming going. Unfortunately, that did not work out this year; and since we already had far too much on our plates with everything else, we decided to forego streaming and focus on recording, only. A few days before the event, however, Steinar Gunderson took it upon himself to fix that. While we couldn't support him a great deal, we could give him access to the secondary laptops (which weren't otherwise doing much anyway) on which he could then do his thing. This was mostly transparent to us; we did communicate to some extent (e.g., when a machine that had been down had fixed and was working again), but he mostly did his thing while we did ours. Full details, for those who want it, in the linked blog post. Today, then, I spent most of my time handling hardware: waiting for other members of FOSDEM to drive the van up to my office; waiting for the laptop rental company to retrieve most of the laptops; waiting for two of the camera rental companies to retrieve their cameras and tripods; driving over to the final camera rental company to return their cameras and tripods; and finally, driving up to the ULB to retrieve the last laptops who had been copying files all night. As I write this, some video files have already been uploaded to the FOSDEM video archive. However, these are extremely low-quality renditions of video snippets that need to be reviewed; they are not ready yet for public consumption. When those files exist, expect an announcement through the main FOSDEM website.

Kees Cook: compiler hardening in Ubuntu and Debian

Back in 2006, the compiler in Ubuntu was patched to enable most build-time security-hardening features (relro, stack protector, fortify source). I wasn t able to convince Debian to do the same, so Debian went the route of other distributions, adding security hardening flags during package builds only. I remain disappointed in this approach, because it means that someone who builds software without using the packaging tools on a non-Ubuntu system won t get those hardening features. Think of a sysadmin trying the latest nginx, or a vendor like Valve building games for distribution. On Ubuntu, when you do that ./configure && make you ll get the features automatically. Debian, at the time, didn t have a good way forward even for package builds since it lacked a concept of global package build flags . Happily, a solution (via dh) was developed about 2 years ago, and Debian package maintainers have been working to adopt it ever since. So, while I don t think any distro can match Ubuntu s method of security hardening compiler defaults, it is valuable to see the results of global package build flags in Debian on the package archive. I ve had an on-going graph of the state of build hardening on both Ubuntu and Debian for a while, but only recently did I put together a comparison of a default install. Very few people have all the packages in the archive installed, so it s a bit silly to only look at the archive statistics. But let s start there, just to describe what s being measured. Here s today s snapshot of Ubuntu s development archive for the past year (you can see development opening after a release every 6 months with an influx of new packages):
Here s today s snapshot of Debian s unstable archive for the past year (at the start of May you can see the archive unfreezing after the Wheezy release; the gaps were my analysis tool failing):
Ubuntu s lines are relatively flat because everything that can be built with hardening already is. Debian s graph is on a slow upward trend as more packages get migrated to dh to gain knowledge of the global flags. Each line in the graphs represents the count of source packages that contain binary packages that have at least 1 hit for a given category. ELF is just that: a source package that ultimately produces at least 1 binary package with at least 1 ELF binary in it (i.e. produces a compiled output). The Read-only Relocations ( relro ) hardening feature is almost always done for an ELF, excepting uncommon situations. As a result, the count of ELF and relro are close on Ubuntu. In fact, examining relro is a good indication of whether or not a source package got built with hardening of any kind. So, in Ubuntu, 91.5% of the archive is built with hardening, with Debian at 55.2%. The stack protector and fortify source features depend on characteristics of the source itself, and may not always be present in package s binaries even when hardening is enabled for the build (e.g. no functions got selected for stack protection, or no fortified glibc functions were used). Really these lines mostly indicate the count of packages that have a sufficiently high level of complexity that would trigger such protections. The PIE and immediate binding ( bind_now ) features are specifically enabled by a package maintainer. PIE can have a noticeable performance impact on CPU-register-starved architectures like i386 (ia32), so it is neither patched on in Ubuntu, nor part of the default flags in Debian. (And bind_now doesn t make much sense without PIE, so they usually go together.) It s worth noting, however, that it probably should be the default on amd64 (x86_64), which has plenty of available registers. Here is a comparison of default installed packages between the most recent stable releases of Ubuntu (13.10) and Debian (Wheezy). It s clear that what the average user gets with a default fresh install is better than what the archive-to-archive comparison shows. Debian s showing is better (74% built with hardening), though it is still clearly lagging behind Ubuntu (99%):

2014, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.
Creative Commons License

Petter Reinholdtsen: Testing sysvinit from experimental in Debian Hurd

A few days ago I decided to try to help the Hurd people to get their changes into sysvinit, to allow them to use the normal sysvinit boot system instead of their old one. This follow up on the great Google Summer of Code work done last summer by Justus Winter to get Debian on Hurd working more like Debian on Linux. To get started, I downloaded a prebuilt hard disk image from http://ftp.debian-ports.org/debian-cd/hurd-i386/current/debian-hurd.img.tar.gz, and started it using virt-manager. The first think I had to do after logging in (root without any password) was to get the network operational. I followed the instructions on the Debian GNU/Hurd ports page and ran these commands as root to get the machine to accept a IP address from the kvm internal DHCP server: 
settrans -fgap /dev/netdde /hurd/netdde 
kill $(ps -ef awk '/[p]finet/   print $2 ')
kill $(ps -ef awk '/[d]evnode/   print $2 ')
dhclient /dev/eth0
After this, the machine had internet connectivity, and I could upgrade it and install the sysvinit packages from experimental and enable it as the default boot system in Hurd. But before I did that, I set a password on the root user, as ssh is running on the machine it for ssh login to work a password need to be set. Also, note that a bug somewhere in openssh on Hurd block compression from working. Remember to turn that off on the client side. Run these commands as root to upgrade and test the new sysvinit stuff: 
cat > /etc/apt/sources.list.d/experimental.list <<EOF
deb http://http.debian.net/debian/ experimental main
EOF
apt-get update
apt-get dist-upgrade
apt-get install -t experimental initscripts sysv-rc sysvinit \
    sysvinit-core sysvinit-utils 
update-alternatives --config runsystem
To reboot after switching boot system, you have to use reboot-hurd instead of just reboot, as there is not yet a sysvinit process able to receive the signals from the normal 'reboot' command. After switching to sysvinit as the boot system, upgrading every package and rebooting, the network come up with DHCP after boot as it should, and the settrans/pkill hack mentioned at the start is no longer needed. But for some strange reason, there are no longer any login prompt in the virtual console, so I logged in using ssh instead. Note that there are some race conditions in Hurd making the boot fail some times. No idea what the cause is, but hope the Hurd porters figure it out. At least Justus said on IRC (#debian-hurd on irc.debian.org) that they are aware of the problem. A way to reduce the impact is to upgrade to the Hurd packages built by Justus by adding this repository to the machine: 
cat > /etc/apt/sources.list.d/hurd-ci.list <<EOF
deb http://darnassus.sceen.net/~teythoon/hurd-ci/ sid main
EOF
At the moment the prebuilt virtual machine get some packages from http://ftp.debian-ports.org/debian, because some of the packages in unstable do not yet include the required patches that are lingering in BTS. This is the completely list of "unofficial" packages installed: 
# aptitude search '?narrow(?version(CURRENT),?origin(Debian Ports))'
i   emacs                   - GNU Emacs editor (metapackage)
i   gdb                     - GNU Debugger
i   hurd-recommended        - Miscellaneous translators
i   isc-dhcp-client         - ISC DHCP client
i   isc-dhcp-common         - common files used by all the isc-dhcp* packages
i   libc-bin                - Embedded GNU C Library: Binaries
i   libc-dev-bin            - Embedded GNU C Library: Development binaries
i   libc0.3                 - Embedded GNU C Library: Shared libraries
i A libc0.3-dbg             - Embedded GNU C Library: detached debugging symbols
i   libc0.3-dev             - Embedded GNU C Library: Development Libraries and Hea
i   multiarch-support       - Transitional package to ensure multiarch compatibilit
i A x11-common              - X Window System (X.Org) infrastructure
i   xorg                    - X.Org X Window System
i A xserver-xorg            - X.Org X server
i A xserver-xorg-input-all  - X.Org X server -- input driver metapackage
#
All in all, testing hurd has been an interesting experience. :) X.org did not work out of the box and I never took the time to follow the porters instructions to fix it. This time I was interested in the command line stuff.

Dirk Eddelbuettel: Rcpp 0.11.0

A new major release 0.11.0 of Rcpp is now on the CRAN network for GNU R and binaries for Debian have been uploaded as well. Before you read on, please note that this release will most likely require a reinstallation of all your packages using Rcpp as it now works without a user-facing shared library. The biggest change in this version is that it is now effectively headers-only. Before you wonder, there is still compiled code provided by Rcpp for use by packages. But this uses the function registration mechanism of GNU R instead via instantiation at package startup. This does make package building easier, will remove the need to query GNU R for the Rcpp library file in src/Makevars when compiling, and should generally avoid building issues such as the dreaded fails on paths with spaces still annoying users of a certain OS. There are a number of other fixes, upgrades and other extensions detailed in NEWS file extract below, in the ChangeLog file in the package and on the Rcpp Changelog page as well as in a release announcement I'll post later.
Changes in Rcpp version 0.11.0 (2014-02-02)
  • Changes in Rcpp API:
    • Functions provided/used by Rcpp are now registered with R and instantiated by client package alleviating the need for explicit linking against libRcpp which is therefore no longer created.
    • Updated the Rcpp.package.skeleton() function accordingly.
    • New class StretchyList for pair lists with fast addition of elements at the front and back. This abstracts the 3 functions NewList, GrowList and Insert used in various packages and in parsers in R.
    • The function dnt, pnt, qnt sugar functions were incorrectly expanding to the no-degree-of-freedoms variant.
    • Unit tests for pnt were added.
    • The sugar table function did not handle NAs and NaNs properly for numeric vectors. Fixed and tests added.
    • The internal coercion mechanism mapping numerics to strings has been updated to better match R (specifically with Inf, -Inf, and NaN.)
    • Applied two bug fixes to Vector sort() and RObject definition spotted and corrected by Kevin Ushey
    • New checkUserInterrupt() function that provides a C++ friendly implementation of R_CheckUserInterrupt.
  • Changes in Rcpp attributes:
    • Embedded R code chunks in sourceCpp are now executed within the working directory of the C++ source file.
    • Embedded R code chunks in sourceCpp can now be disabled.
  • Changes in Rcpp documentation:
    • The Rcpp-FAQ and Rcpp-package vignettes have been updated and expanded.
    • Vignettes are now typeset with grey background for code boxes.
    • The bibtex reference file has been update to reflexct current package versions.
  • Changes in Rcpp unit tests:
    • The file tests/doRUnit.R was rewritten following the pattern deployed in RProtoBuf which is due to Murray Stokely
    • The function test() was rewritten; it provides an easy entry point to running unit tests of the installed package
Thanks to CRANberries, you can also look at a diff to the previous release 0.10.6. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Craig Small: jffnms 0.9.4

JFFNMS version 0.9.4 was released today, this version fixes some bugs that have recently appeared in previous versions.
Alarmed Interfaces and Events

Alarmed Interfaces and Events

The triggers rules editor had a problem where some of the rules clicked off the triggers would not appear or could not be edited correctly. Most of the Admin screens have the ability to sort the rows. This, unfortunately, didn t sort but the functionality has been restored. Most users are probably unaware of this, but the database schema is first created for MySQL and is then converted for PostgreSQL. The conversi0n process is far from ideal and hasn t worked until this release. More testing is required for PostgreSQL support but it should be a lot better.

Andrew Pollock: [geek] Yep, I'm still a geek

I did some soldering tonight, and it felt good. I haven't really done much soldering since I built my cat water bowl monitor, and that's going back to the last decade. My apartment building has an intercom-triggered release to the common front door, and the latch has been sticking a bit lately. It stuck today when Megan's Dad came to pick her up, and it stuck when Sarah came to pick up Zoe. It's annoying to have to trek all the way downstairs to let them in and then walk back up again. I decided I'd try and look into what the problem was today after everyone left. A visual inspection of the latch showed nothing obvious, and no one else was home to push the door unlock button for me while I was at the front door, so I decided to have a look at how it worked from within my apartment. I pulled the intercom phone off the wall and identified the two wires that got shorted when you pressed the unlock button, and shorted them by hand with a wire, and sure enough, I could hear the door solenoid releasing. I was thinking to myself, "if only there was a way to trigger this with my phone". I have a RaspberryPi and an Arduino board at my disposal, but I didn't really want to risk frying either of them. Then I remembered that I had a BoArduino that I'd bought at the Maker Faire years ago, but never assembled. I figured it'd make a good sacrificial lamb. So I rummaged around and found it, and busted out the 240v soldering iron I bought recently, but haven't used yet, and set about slapping it together. Man, it was fun. I had it half built before yoga, and finished it off when I got home. It even worked first go. I felt such a sense of accomplishment. The problem is, my general electronics fu is so weak, I'm not sure where to go from here, and my multimeter needs a new battery. I think the voltage on the wire for the intercom is higher than the BoArduino likes. I'd naively thought I could just connect the two wires of interest, one to ground and the other to one of the digital I/O pins, and have the BoArduino act as a switch, but that's not working. If I wire it one way, it unlocks the door instantly, even with the board powered off, and if I wire it the other way, it seems to half power up the board just using the power of the intercom system. So I'm obviously doing something wrong. My plan is to just have the BoArduino simulate pressing the button for 5 seconds at a time, every 10 seconds or so. Then I can be downstairs wiggling the latch to see what's up with it. Or I could just place a maintenance request with the Body Corporate. At least I had fun assembling the BoArduino, and now I have a spare one. It was nice to be able to go from problem to idea to execution in one night, even if the execution failed.

Andrew Pollock: [life] Day 7, Kindergarten again

Today was a pretty quiet day, because Zoe had Kindergarten again. Nominally, I'm supposed to be doing Debian stuff today, but I had some work presentations I wanted to watch while I still had the chance, so I mostly watched them for four hours while blowing the cobwebs out of my development environment. Zoe slept well last night and slept in a little bit to a whopping 6:30am. We got out the door at about 8:05am and by the time I'd got the bike trailer on the bike and made it to Kindergarten it was about 8:30am. Drop off went pretty smoothly. I decided to go for a run after I got back home, so I banged out 5km. My running fitness has been a bit off of late, partially because of the heat, and partially because I just haven't been doing it enough. I need to get back up to 10km as soon as possible so I can try increasing the distance. Both of Megan's parents work, and so I'm helping them out with picking her up on Mondays and Wednesdays. Today was the day of doing that. It was certainly an interesting experience having to wrangle two kids instead of one. Megan's a good kid, so it was no trouble. To kill time, we went to the car wash, where I bought the girls babyccinos while we waited, and then we headed into the Valley so I could check my post office box. After we got home, I attempted to do something I'd seen on Mister Maker. He basically squirted some PVA glue, some paint and some detergent into a cup and then mixed it all up and then blew into it with a straw until the bubbles rose over the top of the cup and pressed down on it with some paper. I used bubble mix instead of detergent because I figured it would have to be as good, but only Zoe's cup really worked as intended. Then, despite much discussion about it, Zoe decided to suck on the straw instead of blow it and so I had to race her off to the bathroom to rinse her mouth out. After that the girls played a bit more on their own, then Zoe wanted to watch TV but Megan didn't, so we painted a butterfly together and then Megan's Dad arrived to pick her up, and Sarah arrived not long after that. I killed some time doing a spontaneous electronics project before yoga and that was my day.

2 February 2014

Gunnar Wolf: CuBox-i4Pro

CuBox-i4Pro
Somewhere back in August or September, I pre-ordered a CuBox-i A nicely finished, completely hackable, and reasonably powerful ARM system, nicely packaged and meant to be used to hack on. A sweet deal! There are four models (you can see the different models' specs here) I went for the top one, and bought a CuBox-i4Pro. That means, I have a US$130 nice little box, with 4 ARM7 cores, 2GB RAM, WiFi, and... well, all of its basic goodies and features. For some more details, look at the CuBox-i block diagram. I got it delivered by early January, and (with no real ARM experience on my side) I finally got to a point where I can, I believe, contribute something to its adoption/usage: How to get a basic Debian system installed and running in it. The ARM world is quite different to the x86 one: Compatibility is much harder, the computing platform does not self-describe properly, and a kernel must first understand how a specific subarchitecture is before being able to boot on it. Somewhere in the CuBox forums (or was it the IRC channel?) I learnt that the upstream Linux kernel does not yet boot on the i.MX6 chip (although support is rumored to be merged for the 3.14 release), so I am using both a kernel and an uBoot bootloader not built for (or by) Debian people. Besides that, the result I will describe is a kosher Debian install. Yes, I know that my orthodox friends and family will say that 99% kosher is taref... But remember I'm never ever that dogmatic. (yeah, right!) Note that there is a prebuilt image you can run if you are so inclined: In the CuBox-i forums and wiki, you will find links to a pre-installed Debian image you can use... But I cannot advise to do so. First, it is IMO quite bloated (you need a 4GB card for a very basic Debian install? Seriously?) Second, it has a whole desktop environment (LXDE, if I recall correctly) and a whole set of packages I will probably not use in this little box. Third, there is a preinstalled user, and that's a no-no (user: debian, password: debian). But, most importantly, fourth: It is a nightly build of the Testing (Jessie) suite... Built back in December. So no, as a Debian Developer, it's not something we should recommend our users to run! So, in the end and after quite a bit of frustration due to my lack of knowledge, here goes the list of steps I followed:
Using the CuBox
On the i2 and i4 models, you can use it either with a USB keyboard and a HDMI monitor, or by a serial consoles (smaller models do not have a serial console). I don't have a HDMI monitor handy (only a projector), so I prefer to use the serial terminal. Important details to avoid frustration: The USB keyboard has to be connected to the lower USB port, or it will be ignored during the boot process. And make sure your serial terminal is configured not to use hardware flow control. Minicom is configured by default to use hardware flow control, so it was not sending any characters to the CuBox. ^A-O gets you to the Minicom configuration, select Serial port setup, and disable it.
Set up the SD card
I created a 2GB partition, but much less can suffice; I'd leave it at least to 1GB to do the base install, although it can be less once the system is set up (more on this later). Partition and format using your usual tools (fdisk+mke2fs, or gparted, or whatever suits your style).
Install the bootloader
I followed up the instructions on this CuBox-i forums thread to get the SPL and uBoot bootloader running. In short, from this Google Drive folder, download the SPL-U-Boot.img.xz file, uncompress it (xz --decompress SPL-U-Boot.img.xz), and write it to the SD card just after the partition map: As root,
# dd if=SPL-U-Boot.img of=/dev/mmcblk0 bs=1024 seek=1.
Actually, to be honest: As I wanted something basic to be able to debug from, I downloaded (from the same Google Drive) the busybox.img.gz file. That's a bit easier to install from: xz --decompress busybox.img.xz, and just dump it into the SD from the beginning (as it does already include a partition table):
# dd if=busybox.img of=/dev/mmcblk0
This card is already bootable and minimal, and allows to debug some bits from the CuBox-i itself (as we will see shortly).
After this step, I created a second partition, as I said earlier. So, my mmcblk0p1 partition holds Busybox, and the second will hold Debian. We are still working from the x86 system, so we mount the SD card in /media/mmcblk0p2
Installing the base system
Without debian-installer to do the heavy lifting, I went for debootstrap. As I ran it from my PC, debootstrap's role will be for this first stage only to download and do a very initial pre-unpacking of the files: Bootstrapping a foreign architecture implies, right, using the --foreign switch:
debootstrap --foreign --arch=armhf wheezy /media/mmcblk0p2 http://http.debian.net/debian
You can add some packages you often use by specifying --include=foo,bar,baz
So, take note notes: This board is capable of running the armhf architecture (HF for Hardware Float). It can also run armel, but I understand it is way slower.
First boot (with busybox)
So, once debootstrap finishes, you are good to go to the real hardware! Unmount the SD card, put it in the little guy, plug your favorite console in (I'm using the serial port), and plug the power in! You should immediately see something like:
  1. U-Boot SPL 2013.10-rc4-gd05c5c7-dirty (Jan 12 2014 - 02:18:28)
  2. Boot Device: SD1                                              
  3. reading u-boot.img
  4. Load image from RAW...
  5.  
  6.  
  7. U-Boot 2013.10-rc4-gd05c5c7-dirty (Jan 12 2014 - 02:18:28)
  8.  
  9. CPU:   Freescale i.MX6Q rev1.2 at 792 MHz
  10. Reset cause: POR                         
  11. Board: MX6-CuBox-i
  12. DRAM:  2 GiB      
  13. MMC:   FSL_SDHC: 0
  14. In:    serial     
  15. Out:   vga   
  16. Err:   vga
  17. Net:   phydev = 0x0
  18. Phy not found      
  19. PHY reset timed out
  20. FEC                
  21. (Re)start USB...
  22. USB0:   USB EHCI 1.00
  23. scanning bus 0 for devices... 1 USB Device(s) found
  24.        scanning usb for storage devices... 0 Storage Device(s) found
  25.        scanning usb for ethernet devices... 0 Ethernet Device(s) found
  26. Hit any key to stop autoboot:  3

Let it boot (that means, don't stop autoboot), and you will soon see a familiar #, showing you are root in the busybox environment. Great! Now, mount the Debian partition:
# mount /dev/mmcblk0p2 /mnt
Finishing debootstrap's task
With everything in place, it's time for debootstrap to work. Chroot into the Debian partition:
# chroot /mnt
And ask Debootstrap to finish what it started:
# debootstrap --second-stage
Be patient, as this step takes quite a bit to be finished.
Some extra touches...
After this is done, your Debian system is almost ready to be booted into. Why almost? Because it still does not have any users, does not know its own name nor knows I want to use it via a serial terminal. Three very simple tasks to fix. First two:
  1. # passwd
  2. Enter new UNIX password: 
  3. Retype new UNIX password: 
  4. passwd: password updated successfully
  5. # echo cubox-i.gwolf.org > /etc/hostname

For the second one, add a line to /etc/inittab specifying the details of the serial console. You can just do this:
# echo 'T0:23:respawn:/sbin/getty -L ttymxc0 115200 vt100' >> /etc/inittab
Boot into Debian!
So, ready to boot Debian? Ok, first exit the chroot shell, to go back to the Busybox shell, unmount the Debian partition, and set the root partition read-only:
  1. # exit
  2. # umount /mnt
  3. # mount / -o remount,ro

Disconnect and connect power, and now, do interrupt the boot process when you see the Hit any key to stop automount prompt. To see the configuration of uboot, you can type printenv We will only modify the parameters given to the kernel:
  1. CuBox-i U-Boot > setenv root /dev/mmcblk0p2 rootfstype=ext3 ro rootwait
  2. CuBox-i U-Boot > boot

So, the kernel will load, and a minimal Debian system will be initialized. In my case, I get the following output:
  1. ** File not found /boot/busyEnv.txt **
  2. 4703740 bytes read in 390 ms (11.5 MiB/s)
  3. ## Booting kernel from Legacy Image at 10000000 ...
  4.    Image Name:   Linux-3.0.35-8                    
  5.    Image Type:   ARM Linux Kernel Image (uncompressed)
  6.    Data Size:    4703676 Bytes = 4.5 MiB              
  7.    Load Address: 10008000               
  8.    Entry Point:  10008000
  9.    Verifying Checksum ... OK
  10.    Loading Kernel Image ... OK
  11.  
  12. Starting kernel ...
  13.  
  14. Unable to get enet.0 clock
  15. pwm-backlight pwm-backlight.0: unable to request PWM for backlight
  16. pwm-backlight pwm-backlight.1: unable to request PWM for backlight
  17. _regulator_get: get() with no identifier
  18. mxc_sdc_fb mxc_sdc_fb.2: NO mxc display driver found!
  19. INIT: version 2.88 booting
  20. [info] Using makefile-style concurrent boot in runlevel S.
  21. [....] Starting the hotplug events dispatcher: udevd. ok 
  22. [....] Synthesizing the initial hotplug events...done.
  23. [....] Waiting for /dev to be fully populated...done.
  24. [....] Activating swap...done.
  25. [....] Cleaning up temporary files... /tmp. ok 
  26. [....] Activating lvm and md swap...done.
  27. [....] Checking file systems...fsck from util-linux 2.20.1
  28. done.
  29. [....] Mounting local filesystems...done.
  30. [....] Activating swapfile swap...done.
  31. [....] Cleaning up temporary files.... ok 
  32. [....] Setting kernel variables ...done.
  33. [....] Configuring network interfaces...done.
  34. [....] Cleaning up temporary files.... ok 
  35. [....] Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix. ok 
  36. INIT: Entering runlevel: 2
  37. [info] Using makefile-style concurrent boot in runlevel 2.
  38. [....] Starting enhanced syslogd: rsyslogd. ok 
  39. [....] Starting periodic command scheduler: cron. ok 
  40.  
  41. Debian GNU/Linux 7 cubox-i.gwolf.org ttymxc0
  42.  
  43. cubox-i login:

And that's it, the system is live and ready for my commands!
So, how big is this minimal Debian installed system? I cheated a bit on this, as I had already added emacs and screen to the system, so yours will be a small bit smaller. But anyway Lets clear our cache of downloaded packages, and see the disk usage information:
  1. root@cubox-i:~# apt-get clean
  2. root@cubox-i:~# df -h
  3. Filesystem      Size  Used Avail Use% Mounted on
  4. rootfs         1008M  347M  611M  37% /
  5. /dev/root      1008M  347M  611M  37% /
  6. devtmpfs        881M     0  881M   0% /dev
  7. tmpfs           177M  132K  177M   1% /run
  8. tmpfs           5.0M     0  5.0M   0% /run/lock
  9. tmpfs           353M     0  353M   0% /run/shm

So, instead of a 4GB install, we have a 350MB one. Great improvement! Now, lets get it to do something useful, in a most Debianic way!

Steve Kemp: disqus on the cheap?

Last night I was up again, really hard to sleep when you have a bad cold. I decided to do something fun, and allow my tweaking guide to accept comments. Like many of my sites this is 100% static, and generated by templer, so comments are "hard". I've seen a few people try to rewrite disqus as a general-purpose solution, and I like that idea, because I don't trust that particular service. I wasn't so ambitious though, I just hacked up a quick sinatra server: My jquery/javascript is nasty, but the thing seems to work pretty well. The page loads and comments are populated, and new ones are persisted as expected.
I can see the appeal of putting all this magic in one javascript file. You include that and get both the existing comments and the form to add new ones - my approach is to hardwire the submission/display in my generated site. Perhaps something for the future.
In conclusion if people wish they can now leave feedback on most of the pages :)

Chris Lamb: Defaulting to comparing against yourself on Strava

Strava's brand positioning is based around challenging and comparing yourself to others through virtual competitions and leaderboards. I am not against the competitive spirit in principle, but my personal experience is that most psychologically and physiologically rewarding training comes from comparing your own past performances to today or from setting long term goals. I accordingly use the service as a glorified training diary, with the public leaderboards serving only as curiosa or as an occasional check on hubris. (Zen-like introspection aside, using public data as a guide in a metropolis such as London is problematic as rankings are distorted by professional racers, KOM hunters, drafting, fraud and obviously traffic conditions.) To make this easier, I wrote a Chrome extension that changes the default leaderboard to "My Results" instead of "Overall" when viewing a segment effort: https://chris-lamb.co.uk/wp-content/2014/my_results.png The other leaderboards can still be viewed via the usual selector: https://chris-lamb.co.uk/wp-content/2014/my_results_dropdown.png I've found it curious but not altogether surprising how this mere change of default dramatically affects one's mindset. Source code is available. If you found this extension useful, you might like my extension to quickly switch between metric and imperial units.

Vasudev Kamath: Moving weblog from Jekyll to Pelican

Its been a while and I was not actively blogging. I wanted to start blogging again but I was for some reason not happy with Jekyll static site generator which was powering my previous site. So I took this chance to explore other static site generators and redesign my blog. I explored a bit of Pelican, Nikola and Frog. I didn't feel comfortable with Nikola. Frog is a static site generator written using *Racket* and since I'm learning Scheme using racket I was leaning towards using it but at the last moment I decided to settle with Pelican. The main reason for not using Frog was that I just wanted to get my site up rather than waiting for me to complete learning racket and use Frog. My feeling about Pelican is that its a nice tool and is simple and gives all basic things like generating RSS/ATOM feeds and nice theming support. In case of Jekyll I had to write custom page for creating RSS feed and had pain in creating my first theme (yeah I'm not a designer). But now things might have changed in Jekyll land but anyway I don't care anymore. New design uses pelican-bootstrap3 theme and there is no comment facility available. If you want to comment or suggesting things on my post consider writing mail to me :-). So what happened to older entries? Moving the old entries from Jekyll format to Pelican was real PITA so I just renamed my old site to blog-archive. So this is a from scratch site and will contain only new entries. So how do you like my new blog design? :-)

1 February 2014

Paul Tagliamonte: Iron-Blogger starts again

After years under Mako s loving care, iron-blogger is now under new leadership. The triumvirate consists of Me, Molly and John. If any interesting Boston-area people want to join, just email us!

Next.

Previous.