Search Results: "tar"

29 January 2014

Benjamin Mako Hill: Aaron Swartz A Year Later

My friend Aaron Swartz died a little more than a year ago. This time last year, I was spending much of my time speaking with journalists and reading what they were writing about Aaron. Since the anniversary of his death, I have tried to take time to remember Aaron. I ve returned to the things I wrote and the things I said including this short article published last year in Red Pepper that SJ Klein and I wrote together but that I forgot to mention on my blog. I m also excited to see that a documentary film about Aaron premiered at the Sundance Film Festival last week. I was interviewed for the film but am not in it. As I said last year at a memorial for Aaron, I think about Aaron frequently and often think about my own decisions in terms of what Aaron would have done. I continued to be optimistic about the potential for Aaron-inspired action.

Daniel Pocock: Lumicall: giving you choice when you dial

One of the distinguishing features of Lumicall, the free and open SIP/VoIP app for Android, is the unique dialing experience. In fact, even regular softphones for the desktop/laptop don't currently offer the same power when you want to make a call. In my previous blog, I looked at the ENUM technology that underpins part of this compelling new user experience. Here we can step back and look at the bigger picture. Choice is important, sneak preview of Sunday's talk at FOSDEM The fundamental reason for the dialing popup is to give the user choice. Choice is important. With that in mind, one of the key revelations that I will discuss during my talk about Lumicall this Sunday at FOSDEM is that I would like to give users the opportunity to use the Lumicall dialer with other VoIP apps, including Jitsi for Android and CSipSimple. To facilitate this, I hope to separate the dialing logic from the app and make it available as a standalone app or library module that can be used in any free communications software. Users expect convenience, Lumicall tries to deliver Have you noticed how many people will pick up their mobile phone to make a call even when they have a regular phone within reach? Have you found yourself calling your friends' mobile numbers even though you have their home and office numbers in your address book too? These mobile network calls are often at least ten times more expensive than landline calls and by definition they are infinitely more expensive than free calls made over the Internet. So why do people behave this way? The answer is simple: convenience. The caller does not want to make multiple efforts to dial, nor does he want to lose one or two minutes speaking to a secretary, colleague or spouse of the person he is trying to call. Dialing from the address book in his own mobile phone and calling the mobile number of the person he wants to speak with is the shortest path to success. It is efficient, it is compelling and it is what users expect. Free software products wanting to make an impact need to carefully align themselves with these user expectations and that is exactly what Lumicall tries to do. Choose your own adventure: dial the digits, use the address book or surf your call history Any of these everyday dialing practices triggers the Lumicall dialer. In this example, the user is trying to call Bob from the address book: While the user may have simply pressed the "call" button, Lumicall alerts them to several ways that they may proceed:
  • The first choice is a SIP address found in the ENUM directory, sip:+4456190455234@sip5060.net. Lumicall will take all the phone numbers associated with Bob in the address book and report any ENUM results associated with them. Here, just a single ENUM result is display.
  • The second choice is a SIP address constructed from Bob's email address. Lumicall found the email address in the address book entry and discovered that Bob's domain, lumicall.org, has a NAPTR record for SIP. Based on this discovery, Lumicall feels there is a good probability the email address is also a SIP address. If multiple email addresses are in the address book, they will all be checked for NAPTR.
  • (Not shown in the picture) if Lumicall has been configured to use a SIP-to-PSTN gateway account, it will display the option to call Bob's number through the gateway in blue
  • Finally, at the bottom of the list, Lumicall offers the option to call Bob's number in the normal manner, through the GSM network. This option is colour-coded in red to emphasize that it is usually the most expensive.
The tip of the iceberg This experience is a big improvement over the experience of the old rotary dial but it could also be taken even further. People don't always respond to their SIP address, especially if they live in a country where mobile data is not yet the norm. These are areas where Lumicall could be improved. For example, we could use SIP presence to make sure that Bob really is online before displaying the SIP option. This would reduce frustration and boost confidence in SIP. We could even go beyond that: Bob could use the SIP presence mechanism to advertise whether or not he is on a slow mobile data network or a fast network like 4G or wifi. You may choose to defer your call to Bob until he enters wifi coverage where the call quality is likely to be better and maybe video will be possible. This latter opportunity also has privacy implications: Bob may only choose to share that live insight into his network coverage with contacts he trusts. Try it at FOSDEM I will spend some time around the real-time lounge at FOSDEM and would be delighted to get feedback from users who are trying Lumicall, FreePhoneBox.net, JSCommunicator, DruCall or the new Debian SIP service.

Emanuele Rocca: Antifeatures and Antibugs

Software Engineering distinguishes between software features and software bugs. It is usually understood that features are positive, expected characteristics of a computer program. Features make users happy by allowing them to do something useful, interesting, or fun. Something good, anyways. Bugs are instead undesirable and annoying. You're sitting there at your computer writing a long email and the software crashes right before your email is sent. Bad stuff. Features are generally implemented by programmers on purpose, whereas bugs are purely unintentional. They are mistakes. You don't make a mistake on purpose. We might at this point be inclined to think that: i) what is good for users is done on purpose by software manufacturers; ii) what is bad for users was not meant to be. It happened by mistake. Here is a handy table to visualize this idea:
On purpose By mistake
Good Feature
Bad Bug
It seems to make a lot of sense. But you might have noticed that two cells of the table are empty. Right! In a great talk titled When Free Software isn't better, Benjamin Mako Hill mentions the concept of antifeatures, and how they relate to Free Software. Antifeatures are features that make the software do something users will hate. Something they will hate so much they would pay to have those features removed, if that's an option. Microsoft Windows 7 is used in the talk to provide some examples of software antifeatures: the Starter Edition does not allow users to change their background image. Also, it limits the amount of usable memory on the computer to 2GBs, regardless of how much memory the system actually has. Two antifeatures engineered to afflict users to the point that they will purchase a more expensive version of the software, if they have the means to do that. I have another nice example. The Spotify music streaming service plays advertisements between songs every now and then. To make sure users are annoyed as much as possible, Spotify automatically pauses an advertisement if it detects that the volume is being lowered. A poor Spotify user even tried to report the bug on The Spotify Community forum, only to find out that what she naively considered as a software error was "intentional behavior". A spectacular antifeature indeed. Whenever a piece of technology does something you most definitely do not want it to do, such as allowing the NSA to take complete control of your Apple iPhone, including turning on its microphone and camera against your will, that's an antifeature.
On purpose By mistake
Good Feature
Bad Antifeature Bug
Both bugs and antifeatures are bad for users. The difference between them is that antifeatures are engineered. Time and money are spent to make sure the goal is reached. A testing methodology is followed. "Are we really sure customers cannot change their wallpaper even if they try very very hard?" Engineering processes, of course, can fail. If the poor devils at Microsoft who implemented those harassments would have made a mistake that allows users to somehow change their wallpaper on Windows Starter... Well, I would call that a glorious antibug.
On purpose By mistake
Good Feature Antibug
Bad Antifeature Bug
There is no place for antifeatures in Free and Open Source Software. Free Software gives users control over what their software does. Imagine Mozilla adding a feature to Firefox that sets your speakers volume to 11 and starts playing a random song from the black metal artist Burzum every time you add a bookmark, unless you pay for Mozilla Firefox Premium Edition. The source code for Firefox is available under a free license. People who are not into Burzum's music would immediately remove this neat antifeature. I have spent many years of my life advocating Free and Open Source Software, perhaps using the wrong arguments. Mako's talk made me think about all this (thanks mate!). All these years I've been preaching about the technical superiority of Free Software, despite evidence of thousands of bugs and usability issues in the very programs I am using, and contributing to develop. Free Software is not better than Proprietary Software per se. Sometimes it is, sometimes it's not. But it gives you control, and freedom. When it annoys you, when it doesn't do what you expect and want, you can be sure it's not on purpose. And we can fix it together.

Petter Reinholdtsen: A fist full of non-anonymous Bitcoins

Bitcoin is a incredible use of peer to peer communication and encryption, allowing direct and immediate money transfer without any central control. It is sometimes claimed to be ideal for illegal activity, which I believe is quite a long way from the truth. At least I would not conduct illegal money transfers using a system where the details of every transaction are kept forever. This point is investigated in USENIX ;login: from December 2013, in the article "A Fistful of Bitcoins - Characterizing Payments Among Men with No Names" by Sarah Meiklejohn, Marjori Pomarole,Grant Jordan, Kirill Levchenko, Damon McCoy, Geoffrey M. Voelker, and Stefan Savage. They analyse the transaction log in the Bitcoin system, using it to find addresses belong to individuals and organisations and follow the flow of money from both Bitcoin theft and trades on Silk Road to where the money end up. This is how they wrap up their article:
"To demonstrate the usefulness of this type of analysis, we turned our attention to criminal activity. In the Bitcoin economy, criminal activity can appear in a number of forms, such as dealing drugs on Silk Road or simply stealing someone else s bitcoins. We followed the flow of bitcoins out of Silk Road (in particular, from one notorious address) and from a number of highly publicized thefts to see whether we could track the bitcoins to known services. Although some of the thieves attempted to use sophisticated mixing techniques (or possibly mix services) to obscure the flow of bitcoins, for the most part tracking the bitcoins was quite straightforward, and we ultimately saw large quantities of bitcoins flow to a variety of exchanges directly from the point of theft (or the withdrawal from Silk Road). As acknowledged above, following stolen bitcoins to the point at which they are deposited into an exchange does not in itself identify the thief; however, it does enable further de-anonymization in the case in which certain agencies can determine (through, for example, subpoena power) the real-world owner of the account into which the stolen bitcoins were deposited. Because such exchanges seem to serve as chokepoints into and out of the Bitcoin economy (i.e., there are few alternative ways to cash out), we conclude that using Bitcoin for money laundering or other illicit purposes does not (at least at present) seem to be particularly attractive."
These researches are not the first to analyse the Bitcoin transaction log. The 2011 paper "An Analysis of Anonymity in the Bitcoin System" by Fergal Reid and Martin Harrigan is summarized like this:
"Anonymity in Bitcoin, a peer-to-peer electronic currency system, is a complicated issue. Within the system, users are identified by public-keys only. An attacker wishing to de-anonymize its users will attempt to construct the one-to-many mapping between users and public-keys and associate information external to the system with the users. Bitcoin tries to prevent this attack by storing the mapping of a user to his or her public-keys on that user's node only and by allowing each user to generate as many public-keys as required. In this chapter we consider the topological structure of two networks derived from Bitcoin's public transaction history. We show that the two networks have a non-trivial topological structure, provide complementary views of the Bitcoin system and have implications for anonymity. We combine these structures with external information and techniques such as context discovery and flow analysis to investigate an alleged theft of Bitcoins, which, at the time of the theft, had a market value of approximately half a million U.S. dollars."
I hope these references can help kill the urban myth that Bitcoin is anonymous. It isn't really a good fit for illegal activites. Use cash if you need to stay anonymous, at least until regular DNA sampling of notes and coins become the norm. :) As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Alexander Wirt: everything comes to an end

I was a member of the credativ family for almost 10 years. It was a great and and demanding time where I did things I never imagined I would have to do them :). I started as an apprentice and finished as a technical lead. In the last summer one of my best friends - if not my best - and I developed the idea of me joining his company hs42 as their new Head of IT. The whole concept is interesting, most time I ll do home-office and for ~ 1 week in a month I ll join the company in Oldenburg. After a lot of thinking I accepted that offer. That means that I left credativ in December. Being an open source consultant is interesting on the one hand, but somewhat annoying on the other hand. You will always do new things, but often you are not in the position of designing, deciding or even running them. I was always something that is nowadays known as a devop - a long time before this was getting hip . Now I have the opportunity to design, develop and run my own systems. That also means a little bit windows, but opsi exists :). Running systems on your own is different from the usual consultant work. Being a consultant often means that you design and implement something and after it works you have to give your baby into the hands of someone else. Running them on your own also means you can do constant improvement to something, not only when its broken. The job should give me more time for open source and my family, which is a good thing. It is still a little bit odd to work from home, but being together with your family most of the time is a great thing - and I don t want to miss it. The new job also allows me to work as consultant on my own, so if you need a Debian, E-Mail, Linux or whatever guy that helps you in doing things - get in touch with me. The time at credativ was a great one and I look back with a smile to all the good things. If you need Open Source Support they are the people you should ask. I will stay connected with credativ in many ways.

28 January 2014

Andrea Colangelo: The (brand new) Debian 3D-Printing Team is hiring!

The 3D-printing technology has gained quite a big momentum in recent times. Its use is spreading widely even among hobbyists and many sources report daily several (sometimes weird) examples of its usage. The thing I like most, as a Free Software activist, is that several 3D-printing-related free and open source software are already available and quite good, and some printers are even released as free hardware. Unfortunately, Debian is quite lagged behind with regard to the availability of such software in our archive, especially compared to other distributions who already ship much of it. Hopefully this situation will change drastically thanks to the brand new Debian 3D-Printing team, whose purpose is to make Debian a rocking platform to do some serious 3D-printing. The team is bootstrapping right now, and we are looking for volunteers who are willing to help us. Packagers are greatly welcomed, but also triagers, users and passionate people with some experience in the field are extremely valuable. People willing to join the team can take a look at our (in-progress) wiki page and apply on the team page on Alioth. As with any good packaging team, a mailing list is available to share ideas, feel free to subscribe it, and feel free to join #debian-3dprinting on OFTC if you want to hang out with other team members.

Andrew Pollock: [life] Day 1, first day of Kindergarten

Yesterday was the first day of my new routine, except I haven't really established the new routine yet. I think that's going to take a few weeks. It was, however, the first day of Zoe's new routine, she started Kindergarten. "Kindergarten" is a bit of an ambiguous term, even within Australia, it seems. Back when I went to school, there was "pre-school", which was, as best I can determine, what "Kindergarten" is now, except a year later. It was optional, half a week, and the year preceding the start of primary ("elementary") school. I think every other state was already doing an extra year of school (which I think they called "Kindergarten", "Kinder", or something starting with "K") so Queensland blew up "pre-school", created "Prep" and now that's the first year of primary school. Zoe will start "Prep" next year. Then the Queensland education boffins realised that kids were missing out on the stuff that "pre-school" used to offer, so they reintroduced it in the form of "Kindergarten" (by Queensland standards). It's supposed to be play-based, and isn't really "long day care". Zoe does Monday, Tuesday and every other Wednesday, from 8:30am until 2:20pm. I'd only heard about C&K from a friend after moving back to Brisbane, so we were very fortunate to secure a place without having camped out on a waiting list from birth. We got very lucky, and ended up getting multiple offers, but we're pretty thrilled with the one that Zoe got into. It's a relatively new facility, right next to a state school and some bushland. They take the kids on excursions into the bushland, and up to the state school where there's some chickens. The Kindergarten facility itself is really lovely. Anyway, yesterday was Zoe's first day. I biked there yesterday morning (without Zoe) and it took me about 11 minutes. It took me more like 15 minutes to pick her up with the bike trailer. It's uphill on the way there, but not too much of a slog. It's been really great that Zoe's BFF from day care, Megan, also scored a place in the same Kindergarten, as I think this will help the transition for both girls greatly. I was able to stay for an extended period of time to help settle Zoe in, and I was able to leave without her being particularly distressed. I then biked home, running a few errands along the way. I then got stuck into a woodworking project (I'm trying to make a little wooden clothesline for Zoe). It turns out trying to cut a halved joint with a hand saw and chisel is a lot of work. I've managed to aggravate an old rotator cuff injury and the two pieces are a tad too tight, so I'm thinking I'll redo it and cut the joint out with an electric router instead. I probably spent at least a couple of hours faffing around with that. So I banged away with a hammer and chisel for a bit, got a haircut, had a late lunch and next thing I knew it was 2pm and time to head back on the bike to pick up Zoe. She was still asleep on her stretcher when I arrived to pick her up, which wasn't that surprising, since she tends to sleep later and longer at day care. I got a good opportunity to chat with her teacher about her first day and then gently woke her up with a kiss, and we headed home. I introduced Zoe to one of our new neighbours, and after a snack, we drove into Fortitude Valley where I had to pick up some mail from my post office box. When we got back home, I started on dinner while Zoe watched a little bit of TV. I spontaneously invited my cousin over to join us for dinner, and we walked down to the Hawthorne Garage for a babyccino after dinner. It was a good day. I realised that I'm going to have to be super regimented with my time on the days Zoe is in Kindergarten, because 2pm is going to roll around very quickly. "Scarcity brings clarity", as Sergey says...

Andrew Pollock: [life] Day 2, first day of "daddy daycare"

Today was the first weekday of me taking care of Zoe. We got things started more or less on the same routine that we used to do when she had day care, so we were dressed and ready to roll by about 8am. We baked a batch of mini cornbread muffins, and had them ready by the time my 9:30am chiropractic appointment rolled around, so we took a few with us for morning tea for my chiropractor and his receptionist. I've taken Zoe with me on a few occasions, and speak of her often to my chiropractor. He's a dad himself, and the first few times I brought her in, he juggled for her, so she thinks he's pretty cool. Last night, Zoe had expressed a desire to go to Southbank today. It was initially looking like it'd be too rainy, but by the time I was done with the chiropractor, it looked safe enough to venture out, so we went back home, and I prepared a picnic lunch for us, and we caught a bus over to Southbank. We had a bit of a splash around in the water and then ate lunch. Zoe had a great time alternating between feeding and chasing seagulls, crows, and an ibis. My girlfriend, who works in South Brisbane, walked over and joined us briefly and then we all walked back towards the bus and parted ways. Southbank mid-week is pretty nice. Not too busy. It was a slow walk home from the bus, but the reward of a My Little Pony biscuit was enough to get her racing ahead of me from the corner. After a biscuit and some quiet time, she fell asleep and I've had a couple of hours to read and write blog posts, as well as plan future activities. I'm hoping that we'll have enough time to go birthday present shopping for a 4th birthday party (for twins from Zoe's old day care) that we're going to on Saturday morning after swim class, before Sarah arrives to pick her up, but I'm dubious we'll get time. Zoe's still asleep as I write. There's always tomorrow. We might be best off just baking the second batch of cornbread muffins.

26 January 2014

Neil Williams: Home server rack

The uses of this particular rack I ll cover in future entries this is about how I made the rack itself, with help from friends (Steve McIntyre & Andy Simpkins). A common theme is making allowances for using dev kit boards ready-to-rack ARM servers are not here yet. My aim was to have 4, possibly 6 ARM dev kit boards running services from home, so there was no need for a standard 42U rack, a 9U rack is enough. Hence:
Wall Mounted 9U 450mm deep 19 inch rack enclosure glass door To run various other services around the house (firewall, file server etc.), a microserver was also necessary:
HP 704941-421 ProLiant Micro Server (AMD Turion II Neo N54L 2.2GHz, 2GB RAM, 250GB HDD, 2 Core, 7th Generation) I chose to mount that on a bookcase beneath the wall mounted rack as it kept all the cables at the bottom of the rack itself. The microserver needed a second gigabit network card fitted to cope with being a firewall as well, if you do the same, ensure you find a suitable card with a low profile bracket. Some are described as being low profile but do not package the low profile bracket, only a low profile card and a full height bracket.
Intel EXPI9301CTBLK PRO1000 Network Card note the low profile bracket in the pack. The first of the dev kit requirements is the lack of boards which can be racked, so shelves are going to be needed, leading on to something to stop the boards wandering across the shelf when the cables are adjusted velcro pads in my case. Second requirement is that dev kit boards are notorious for not rebooting cleanly. Nothing to do with the image being run, the board just doesn t cut power or doesn t come back after cutting power. Sometimes this is down to early revisions of the board, sometimes the board pulls enough power through the USB serial converter to remain alive, whatever the cause, it won t reboot without user involvement. So a PDU becomes necessary remotely controllable. New units tend to be expensive and/or only suitable for larger racks, I managed to find an older 8 port APC unit, something like: (Don t be surprised if that becomes a dead link search for APC Smart Slot Master Switch Power Distribution Unit). Talking of power, I m looking to use SATA drives with these boards and the boards themselves come with a variety of wall wart plugs or USB cables, so a number of IEC sockets are needed not the usual plugs:
Power cable IEC C14 plug 13A socket 25 cm or, for devices which genuinely need 2A to boot (use the 1A for attached SATA or leave empty):

Black Universal 3.1A 15W 5V Dual USB Rapid Mains Charger Plug Check the power output rating of the USB plugs used to connect to the mains as well many are 1A or less. Keep the right plug for the right board.
Power is going to also be a problem if, like me, you want to run SATA drives off boards which support SATA. The lack of a standard case means that ATX power is awkward, so check out some cheap SATA enclosures to get a SATA connector with USB power. I am using these enclosures (prices seem to have risen since I obtained them):

Startech 2.5 inch eSATA USB External Hard Drive Enclosure for SATA HDD Along with these:
eSATA to SATA Serial External Shielded Cable 1m because the iMx53 boards have SATA connectors but the enclosure exports eSATA. Whilst this might seem awkward, the merit of having both eSATA and simple USB power on one enclosure is not to be under-estimated. (Avoid the stiffer black cables space will get tight inside the rack.) Naturally, a 2.5 inch SATA drive is going to be needed for each enclosure, I m using HDD but SSD is also an option. Also, consider simple 2 or 3 way fused adaptors so that the board and the SATA drive can be powered from a single PDU port, this makes rebooting much simpler if the board needs a power supply with integrated plug instead of over USB. Now to the networking (2 x 8 port was cheaper than 1 x 16):
Netgear GS108 8-port Gigabit Ethernet Unmanaged Switch Don t forget the cat5 cables too you ll want lots of short cables, 1m or shorter inside the rack and a few longer ones going to the microserver and wall socket. I used 8x1m. Naturally, on the floor below your rack you are going to put a UPS, so the PDU power then needs to be converted to draw from the UPS via IEC plugs instead of standard mains. I decided to use a 6 gang extension 1m cable with an IEC plug it was the only bit of wiring I had to do and even those are available ready-made if you want to do it that way. Depending on the board, you may need your own serial to USB converters, you ll certainly need some powered USB hubs. I m using a wall mounted 9U rack, so I also needed a masonry drill and 4 heavy duty masonry M8 bolts. The rack comes with a mounting plate which needs to be bolted to the wall but nothing else is included. This step is much easier with someone else to take the weight of the rack as it is guided into the brackets on the mounting plate the bracket may need a little persuasion to allow for the bolt heads to not get in the way during mounting. Once mounted, the holes in the back of the rack allow for plenty of room, it s just getting to that point. The rack has side panels which latch out easily, providing easy maintenance. The glass door can be easily reversed to open from the opposite side. However, the key in the glass door is largely useless. The expectation is that the units in the rack are attached at the front but the dev boards on shelves are not going to be protected by a key in the front door. The key therefore ends up being little more than a handle for the glass door. OK. If you ve got this far, it s a case of putting things together:
Economy Cage Nut Tool 19 inch racking for cagenut extraction Yes, you really do want one. Fine, do without the premium one, but the economy one will save you a lot of (physical) pain. At this stage, it becomes clear that the normal 19 inch server rack shelves don t leave a lot of room at the back of the rack for the cables and there are a lot of cables. Each board has power, USB serial connection and network. The SATA has power too. The PDU has a power lead and you ll need network switches too. The network switches need power and an uplink cable. I positioned the supports in the rack as far forward as possible and attached the shelves to give enough room for the PDU on the base of the rack, the network switches resting on top and the extension bar (with the heavier, stiffer cables) at the back. (I need to bring my shelves another 1 or 2 positions further forward as there is barely enough room for one cable between the shelf and the back of the rack and that makes moving boards around harder than it needs to be.) The PDU defaults to enabling all ports at startup, so connect to it over telnet and turn off the ports before connecting things and sorting out the network interface to what the rest of the lab needs. (I m using a 10. range and the PDU was originally set to use 192.168.1.) That s about it as far as the hardware setup is concerned. Just time to label up each board, note down which PDU port does which device, which serial to USB converter is on which device on the microserver and check the power my initial problem with one board was traced to the inability of the board to power SATA off the on-board USB port even when using the provided 2A power supply. That meant adding a standard mains adaptor to feed both the SATA power and the board power off the one PDU port there is little point powering off the board but not the SATA, or vice versa. I haven t totalled up the expenditure but the biggest expenses were the microserver and the wall mounted rack but don t underestimate how much it will cost to buy 6 IEC plugs, various USB serial converters and how much you may spend on peripheral items. There is quite a lot of room on the 2 shelves for more boards, what will limit the deployment in this rack is space for the cables, especially power. The shorter the power cables, the easier it will be to maintain the devices in the rack. It might be worth looking at a 12U rack, if only to give plenty of space for cables. Once I ve got the software working, I ll describe what this rack will be doing it s got something to do with Debian, ARM, Linux and tests but you ve probably already guessed that much

Justus Winter: On portability of init systems

There is one thing in the current init system debate that irritates me. It is about the portability of init systems. The new init systems are evaluated by how portable they are. This is one of the arguments that is most often brought against systemd, which is understandable given the polarizing attitude of one of systemd's authors. In this context, the currently used sysvinit is considered portable. But in my book, it is not portable at all. But why can we use sysvinit to boot Debian/ kFreeBSD,Hurd ? Debian/kFreeBSD uses linprocfs for /proc to provide a familiar Linux-like environment for the userspace tools available in Debian (say, pgrep). linprocfs was originally written by the FreeBSD folks to support running Linux binaries using the Linux Binary Compatibility layer. Debian/Hurd uses procfs to provide /poc. This procfs translator is written mainly to provide a Linux-compatible /proc filesystem for the same reason linprocfs is used by Debian/kFreeBSD. So sysvinit works on those system not because it is portable, but because the environment has been made Linux-like enough for sysvinit. We (most likely anyone not using Linux) often do this, because it is the easiest way to run popular software developed (mainly) for Linux. This is often the path of the least resistance, as opposed to getting the upstream project to support the native way of doing things on platform X. During gsoc last year I had to patch our procfs to finally be able to safely shut down Debian/Hurd systems using sysvinit. The problem was, that sysvinit at certain runlevel transitions (like shutting down, or I guess, switching to single user mode), sysvinit assumes that it is okay to stop and kill (almost) all processes on the system (that's what killall5 does). This might be okay on monolithic systems, but on (multiserver) microkernel systems like the Hurd, where your root filesystem and your network driver and stack are running as userspace processes, it is clearly not. I wonder how Linux systems using a FUSE-based root filesystem get away with this. This highlights that not only sysvinit depends on a Linux-specific kernel interface (/proc), but it also hard-codes assumptions about the system architecture. Amusingly, systemd get's this right (ok, I'm not sure if it does, but it could get this right...). systemd organizes processes in cgroups, one for each service it starts and one for each login session or something like that. It can (could?) kill only those processes in it was responsible for, leaving all essential translators (system servers) alone. In fact, even my tiny cgroupfs prototype can keep track of translators that are started by the root filesystem translator.

25 January 2014

Ben Hutchings: Looking back on my time at Solarflare, part 1

Friday was my last day at Solarflare Communications, which I joined in 2006 (originally as Level 5 Networks). When I started, Level 5 Networks (L5N) had a 2-port gigabit Ethernet controller called EF1 and a user-level network stack called EtherFabric which ran on Linux. It had nearly completed the Falcon project to develop a 10-gigabit controller (also supporing a 2-port gigabit mode) and was porting EtherFabric to Windows. Later that year, however, L5N merged with Solarflare Communications (SF), which was developing a 10GBASE-T PHY (for 10-gigabit Ethernet over twisted-pair cables). The investors and management saw the opportunity to create an integrated 10GBASE-T LAN-on-motherboard chip that would be attractive to OEMs and so had the potential for mass sales. EtherFabric was put on the back-burner, but regular net drivers for Linux, Windows and other operating systems became more important. In a change from my previous jobs, my initial role at L5N was to maintain and extend the in-house test automation software, Runbench, written in Python. I worked with Dickon Reed who had written most of Runbench up to then. One of my first major tasks was to extend it to cover DUTs running Windows. Aside from changing Runbench itself, I wrote a simple SSH server to be installed on DUTs, using the Twisted.Python framework. (Cygwin's port of OpenSSH wasn't suitable for running a lot of native Windows commands.) I rewrote the test-installation script for the Windows driver itself, converting it to Python (with a C extension to get around the WOW64 nonsense). As Windows does not allow loading a driver just once (like insmod in Linux), except by using the kernel debugger, I added a time check in the driver so the test-installation script can make it fail early when reloaded after a reboot, so that a crasher bug in the probe path would usually be recoverable. In 2007 I persuaded management in Cambridge to loan several older servers to DebConf. I took them there and back by car, and they were used to convert video for live streams and recordings. (This was repeated in 2009.) I was also asked to work on other small development tasks, including some cleanup on the Linux net driver (sfc). In mid-2007, when SF decided it was time to get sfc upstream (in-tree), I joined Steve Hodgson and Robert Stonehouse in working on that. Out-of-tree (separately distributed) drivers for Linux normally need a large number of preprocessor conditions for compatibility with Linux's kernel module API across a range of supported versions. They may also implement some features with a driver-specific interface that should be replaced with a standard interface (possibly including the work to define that standard). They may not comply with the kernel coding style or unwritten conventions for kernel code. All of these problems were present and had to be dealt with. As part of this work, I extended unifdef so that it could be used to remove all of the backward-compatibility and not-for-upstream code without introducing extra blank lines. I could then export the out-of-tree driver from CVS(!) into a kernel source tree automatically and turn it into a patch or patch series that would stand a chance of being acceptable upstream. Our first few attempts got little response, but eventually we newbies got the message that the driver was simply too big for a single submission and that putting patches on a web site because they're too big for the list was not a solution. In the next week, I removed about half of the code (on a git branch), resulting in a relatively lean driver that could be sent to the netdev list. And finally the 9th (I think) submission was accepted. Following this, I continued to work on sfc but was also brought into the Siena project. Siena (SFL9021) was the LAN-on-motherboard chip that had been planned following the merger. It combined a dual-port controller based on Falcon with a 10GBASE-T PHY and a management controller (MC) to support Lights-Out Management (LOM). (Since 10GBASE-T has not taken off, it is now mostly sold as the SFC9020 variant in which the PHY is disabled.) I worked on some of the test framework and test cases for validating the controller design in software simulation and FPGA. In the course of this I learned to read and (somewhat) understand the chip design written in Verilog, but only made a single trivial fix to it. In August 2009, the first Siena ASICs came back from the fab. There were a few ASIC-specific bugs but all of them were quickly worked around in firmware, so it would soon be ready for production. But there was much work still to be done on the driver and firmware. The firmware had until then been running in a smaller block of RAM with no peripherals to manage, while sfc had earlier been modified just enough to configure a sim or FPGA for running a userland test application. sfc now had many special cases scattered around it, and would tell the MC firmware to peek and poke registers that were no longer directly accessible from the host. The software and firmware developers had agreed that, in order to support LOM, the MC would be responsible for managing the ports and all peripherals and drivers would send higher-level requests to the MC. As a bonus, this removed the need for the driver to know the details of each new board, so that it would not be necessary to backport sfc into distribution kernels very often. Over the next few months, the firmware team did an excellent job of defining and implementing those operations, including writing the driver-side functions to invoke them. Meanwhile, Steve and I concentrated on refactoring sfc so most of the Falcon/Siena differences could be abstracted through a few structures with function pointers. The refactoring and new code for Siena were completed and submitted upstream in several large patch series in October and November. In fact, there were so many patches that Solarflare appeared in LWN's table of who wrote Linux 2.6.33 (as did I, thanks also to another patch series adding firmware metadata to many drivers). Sadly we had missed Linux 2.6.32 which was a longterm stable branch and used in many distributions, but I was able to get this version backported into all the major distributions over the next year. (To be continued.)

Russell Coker: Links January 2014

Fast Coexist has an interesting article about the art that Simon Beck creates by walking in snow [1]. If you are an artist you can create art in any way, even by walking in patterns in the snow. Russ Altman gave an interesting TED talk about using DNA testing before prescribing drugs [2]. I was surprised by the amount of variation in effects of codeine based on genetics, presumably many other drugs have a similar range. Helen Epstein wrote an interesting article about Dr. Sara Josephine Baker who revolutionised child care and saved the lives of a huge number of children [3]. Her tenacity is inspiring. Also it s interesting to note that the US Republican party was awful even before the Southern Strategy . The part about some doctors opposing child care because it s the will of God for children to die and keep them in employment is chilling. Jonathan Weiler wrote an insightful article about the problems with American journalism in defending the government [4]. He criticises the media for paying more attention to policing decorum than to content. Tobias Buckell wrote an interesting post about the so-called socialised health-care in the US [5]. He suggests that Ronald Reagan socialised health-care by preventing hospitals from dumping dying people on the street. I guess if doing nothing for people until they have a medical emergency counts as socialised health-care then the US has it. Kelvin Thomson MP made some insightful comments about climate change, the recent heat-wave in Australia, and renewable energy [6]. Iwan Baan gave an interesting TED talk about ways that people have built cheap homes in unexpected places [7], lots of good pictures. Racialicious has an interesting article by Arturo R. Garc a about research into the effects of concussion and the way the NFL in the US tried to prevent Dr. Bennet Omalu publicising the results of his research [8]. Stani (Jan Schmidt) wrote an interesting post about how they won a competition to design a commemerative Dutch 5 Euro coin [9]. The coin design is really good (a candidate for the geekiest coin ever), I want one! Seriously if anyone knows how to get one at a reasonable price (IE close to face value for circulated or not unreasonably expensive for uncirculated) then please let me know. When writing about Edward Snowden, Nathan says Imagine how great a country would be if if it were governed entirely by people who Dick Cheney would call Traitor [10]. That s so right, that might make the US a country I d be prepared to live in. Andrew Solomon gave an interesting TED talk Love No Matter What about raising different children [11]. Aditi Shankardass gave an interesting TED talk about using an ECG to analyse people diagnosed wit severe Autism and other developmental disorders [12]. Apparently some severe cases of Autism have a root cause that can be treated with anti-seizure medication. George Monbiot wrote an insightful article about the way that Bono and Bob Geldoff promote G8 government intervention in Africa and steal air-time that might be given to allow Africans to represent themselves in public debates [13]. Daniel Pocock wrote an informative article about racism in Australian politics and how it is bad for job-seekers and the economy (in addition to being horribly wrong) [14]. Aeon Magazine has an interesting article by Anne Buchanan about the difference between scientists and farmers [15]. She has some interesting points about the way that the lack of general knowledge impacts research, but misses the point that in most fields of study there is a huge problem of people not knowing about recent developments in their own field. I don t think it s a pipe dream to be well educated in humanities and science, but I guess that depends on the definition of well educated . Brian Cox gave an interesting TED talk titled Why We Need the Explorers about the benefits of scientific research [16]. Yupu Zhang, Abhishek Rajimwale, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau from the University of Wisconsin-Madison wrote an interesting paper about ZFS corruption in the face of disk and memory errors [17]. One thing to note is that turning off atime can reduce the probability of a memory error leading to corrupt data being written to disk, run zfs set atime=off tank to fix this. The comedian Solomon Georgio celebrated Martin Luther King day by tweeting I love you to racists [18]. It s an interesting approach and appears to have worked well.

24 January 2014

Gerfried Fuchs: Julia Engelmann

This is something different. It is something special. She is something special. I don't even remember how I stumbled upon her, actually. I think it was some suggested video somehow. I feel deeply sorry for those who don't understand German. But that's the way it is, the text is in German. And it's touching. Take your time, listen to it in a relaxed environment, when you don't have any distraction. There you are. And I'm sorry for those who don't understand German, but I don't want to hold this back from the others. And I am definitely looking forward to hear more from her. Hopefully she helps me motivate myself to write something new for the upcoming debconf and hold yet another poetry night there. And hopefully it also motivates others to join in.

/haiku permanent link Comments: 0 Flattr this

Julien Danjou: OpenStack Ceilometer Icehouse-2 milestone released

Yesterday, the second milestone of the Icehouse development branch of Ceilometer has been released and is now available for testing and download. This means the first half of the OpenStack Icehouse development has passed! New features For the Icehouse-1 milestone, we barely had enough time to implement 2 blueprints. We almost did a better job this time, but finally only 2 blueprints were implemented again. This is really far from what we planned initially. The infrastructure slowdown issues and the lower number of reviews is probably the root cause here. Anyway, Ceilometer now offers a REST API to accesses the stored event. The initial work to replace the /v2/meters endpoint with something more RESTy has started with the implementation of /v2/samples. Bug fixes Thirty-one bugs were fixed, though most of them might not interest you so I won't elaborate too much on that. Go read the list if you are curious. Toward Icehouse 3 We now have 29 blueprints targeting the Ceilometer's third Icehouse milestone, with some of them are already started and ready to merge. However, it's likely that we won't make all of them. As usual, the priority should indicate how confident we are that we want and need a feature. Still, it's likely the roadmap will be adjusted in the upcoming weeks. I'll try to make sure we'll get there without too much trouble for the 6th March 2013. Stay tuned!

23 January 2014

Gunnar Wolf: Ligatured iceweasel

Ligatured iceweasel
I am not (yet?) reporting this as a bug as this happened with a several days old session open, and just while I was upgrading my Sid system, after a long time without doing so (probably since before the vacations started... In December 2013). But I cannot avoid sharing this interesting screenshot. Of course, this does not happen in other browsers. And AFAICT it only happens while reading the Debian Policy (either online or locally, even recoding it to UTF-8). Funniest thing, the Debian policy specifies no Javascript, no stylesheets at all... (Hey, and FWIW... Why is the online copy of the Debian policy still in iso-8859-1 It's not 1995 anymore...) [update] Of course, it's the default font, not only the Debian policy. Just as an example, the following text:
  1. <html><body><p>Ufffiii flat different!</p></body></html>
Yields the following output: [update 2] And, of course, after finishing the update process... I got a new version of Iceweasel. Restarted it, and everything is back to normal :-

22 January 2014

Bartosz Fe&#324;ski: gift for Debian Developers from Valve

Nice gift from Valve to Debian Developers although when I m reading first reactions on #debian-devel I expect this to be bigger flamewar than systemd/upstart battle ;)

Erich Schubert: The init wars

The init wars have recently caught a lot of media attention (e.g. heise, prolinux, phoronix). However, one detail that is often overlooked: Debian is debating over the default, while all of them are already supported to a large extend, actually. Most likely, at least two of them will be made mandatory to support IMHO.
The discussion seems to be quite heated, with lots of people trying to evangelize for their preferred system. This actually only highlights that we need to support more than one, as Debian has always been about choice. This may mean some extra work for the debian-installer developers, because choosing the init system at install time (instead of switching later) will be much easier. More often than not, when switching from one init system to another you will have to perform a hard reset.
If you want to learn about the options, please go to the formal discussion page, which does a good job at presenting the positions in a neutral way.
Here is my subjective view of the init systems:
  • SysV init is the current default, and thus deserves to be mentioned first. It is slow, because it is based on a huge series of shell scripts. It can often be fragile, but at the same time it is very transparent. For a UNIX system administrator, SysV init is probably the preferred choice. You only reboot your servers every year anyway.
  • upstart seems to be a typical Canonical project. It solves a great deal of problems, but apparently isn't good enough at it for everybody, and they fail at including anyone in their efforts. Other examples of these fails include Unity and Mir, where they also announced the project as-is, instead of trying to get other supporters on board early (AFAICT). The key problem to widespread upstart acceptance seems to be the Canonical Contributor License Agreement that many would-be contributors are unwilling to accept. The only alternative would be to fork upstart completely, to make it independent of Canonical. (Note that upstart nevertheless is GPL, which is why it can be used by Debian just fine. The CLA only makes getting patches and enhancements included in the official version hard.)
  • systemd is the rising star in the init world. It probably has the best set of features, and it has started to incorporate/replace a number of existing projects such as ConsoleKit. I.e. it not only manages services, but also user sessions. It can be loosely tied to the GNOME project which has started to rely on it more and more (much to the unhappyness of Canonical, who used to be a key player for GNOME; note that officially, GNOME chose to not depend on systemd, yet I see this as the only reliable combination to get a complete GNOME system running, and since "systemd can eventually replace gnome-session" I foresee this tie to become closer). As the main drawback, systemd as is will (apparently) only work with the Linux kernel, whereas Debian has to also support kFreeBSD, NetBSD, Hurd and the OpenSolaris kernels (some aren't officially supported by Debian, but by separate projects).
So my take: I believe the only reasonable default is systemd. It has the most active development community and widest set of features. But as it cannot support all architectures, we need mandatory support for an alternative init system, probably SysV. Getting both working reliably will be a pain, in particular since more and more projects (e.g. GNOME) tie themselves closely to systemd, and would then become Linux-only or require major patches.
I have tried only systemd on a number of machines, and unfortunately I cannot report it as "prime time ready" yet. You do have the occasional upgrade problems and incompatibilities, as it is quite invasive. From screensavers activating during movies to double suspends, to being unable to shutdown my system when logged in (systemd would treat the login manager as separate session, and not being the sole user it would not allow me to shut down), I have seen quite a lot of annoyances happen. This is an obvious consequence of the active development on systemd. This means that we should make the decision early, because we will need a lot of time to resolve all these bugs for the release.
There are more disruptions coming on the way. Nobody seems to have talked about kDBUS yet, the integration of an IPC mechanism like DBUS into the Linux kernel. It IMHO has a good chance of making it into the Linux kernel rather soon, and I wouldn't be surprised if it became mandatory for systemd soon after. Which then implies that only a recent kernel (say, mid-2014) version might be fully supported by systemd soon.
I would also like to see less GNOME influence in systemd. I have pretty much given up on the GNOME community, which is moving into a UI direction that I hate: they seem to only care about tablet and mobile phones for dumb users, and slowly turn GNOME into an android UI; selling black background as major UI improvements. I feel that the key GNOME development community does not care about developers and desktop users like me anymore (but dream of being the next Android), and therefore I have abandoned GNOME and switched to XFCE.
I don't give upstart much of a chance. Of course there are some Debian developers already involved in its development (employed by Canonical), so this will cause some frustration. But so far, upstart is largely an Ubuntu-only solution. And just like Mir, I don't see much future in it; instead I foresee Ubuntu going systemd within a few years, because it will want to get all the latest GNOME features. Ubuntu relies on GNOME, and apparently GNOME already has chosen systemd over upstart (even though this is "officially" denied).
Sticking with SysV is obviously the easiest choice, but it does not make a lot of sense to me technically. It's okay for servers, but more and more desktop applications will start to rely on systemd. For legacy reasons, I would however like to retain good SysV support for at least 5-10 more years.

But what is the real problem? After all, this is a long overdue decision.
  • There is too much advocacy and evangelism, from either side. The CTTE isn't really left alone to do a technical decision, but instead the main factors have become of political nature, unfortunately. You have all kinds of companies (such as Spotify) weigh in on the debate, too.
  • The tone has become quite aggressive and emotional, unfortunately. I can already foresee some comments on this blog post "you are a liar, because GNOME is now spelled Gnome!!1!".
  • Media attention. This upcoming decision has been picked up by various Linux media already, increasing the pressure on everybody.
  • Last but not least, the impact will be major. Debian is one of the largest distributions, last but not least used by Ubuntu and Steam, amongst others. Debian preferring one over the other will be a slap in somebodys face, unfortunately.
So how to solve it? Let the CTTE do their discussions, and stop flooding them with mails trying to influence them. There has been so much influencing going on, it may even backfire. I'm confident they will find a reasonable decision, or they'll decide to poll all the DDs. If you want to influence the outcome provide patches to anything that doesn't yet fully support your init system of choice! I'm sure there are hundreds of packages which do neither have upstart nor systemd support yet (as is, I currently have 27 init.d scripts launched by systemd, for example). IMHO, nothing is more convincing than have things just work, and of course, contributing code. We are in open source development, and the one thing that gets you sympathy in the community is to contribute code to someone elses project. For example, contribute full integrated power-management support into XFCE, if you include power management functionality.
As is, I have apparently 7 packages installed with upstart support, and 25 with systemd support. So either, everybody is crazy about systemd, or they have the better record of getting their scripts accepted upstream. (Note that this straw poll is biased - with systemd, the benefits of not using "legacy" init.d script may just be larger).

John Goerzen: Debian-Live Rescue image with ZFS On Linux; Ditched btrfs

I m a geek. I enjoy playing with different filesystems, version control systems, and, well, for that matter, radios. I have lately started to worry about the risks of silent data corruption, and as such, looked to switch my personal systems to either ZFS or btrfs, both of which offer built-in checksumming of all data and metadata. I initially opted for btrfs, because of its tighter integration into the Linux kernel and ability to shrink an existing btrfs filesystem. However, as I wrote last month, that experiment was not a success. I had too many serious performance regressions and one too many kernel panics and decided it wasn t worth it. And that the SuSE people got it wrong, deeply wrong, when they declared btrfs ready for production. I never lost any data, to its credit. But it simply reduces uptime too much. That left ZFS. Before I build a system, I always want to make sure I can repair it. So I started with the Debian Live rescue image, and added the zfsonlinux.org repository to it, along with some key packages to enable the ZFS kernel modules, GRUB support, and initramfs support. The resulting image is described, and can be downloaded from, my ZFS Rescue Disc wiki page, which also has a link to my source tree on github. In future blog posts in the series, I will describe the process of converting existing Debian installations to use ZFS, of getting them to boot from ZFS, some bugs I encountered along the way, and some surprising performance regressions in ZFS compared to ext4 and btrfs.

Craig Small: WordPress 3.8 for Debian

Well if you can read this then you know it s working. After way too many weeks, Debian will have WordPress version 3.8. Thanks to Rapha l for his kind assistance and answering my questions about how it was built. The upload is still gurgling along and will make it there in its own time. He said Handing over packages is hard, I d agree but say taking over them is too. So, what does WordPress 3.8 look like? From the frontend I didn t really notice much. The big changes, at least cosmetically, seem to be for the admin backend. It just look slicker and cleaner. Hopefully Debian users find the update useful and I ve not broken anything. There s always the BTS if there is. I ve deliberately tried to minimise the changes for this version to limit the breakage.

Gunnar Wolf: Finally: A student once again

Formally, today is my first day as a student on a formal, scholarized institution Basically for the first time in almost twenty years! Yes, those that know me know that I aspire to live the life of academia. I have worked at public universities for almost all of my adult life (between 1997 and 1999 I worked at a local ISP and at a private school), and have had a minor academic position ( T cnico Acad mico ) for almost ten years. And not having a proper degree limited me from pursuing anything further. Then, in early 2010 I presented the written exam. By late 2010, the corresponding oral exam. That allowed me to get my formal diploma in December 2010. By the end of 2011, I requested to be a teacher in the Engineering Faculty of UNAM, and started teaching Operating Systems a year ago, in January 2012. So, a good advance in the last few years... But I know that if I just sit here, I won't be able to advance my position towards really entering the Sacred Halls of Academia. And there are some rituals I have to comply with. One of those rituals is... Devoting some long time to studying under the formal structures. Ok, so I'm finally a postgraduate student I have enrolled in Especialidad en Seguridad Inform tica y Tecnolog as de la Informaci n, a short (one year) postgraduate program in ESIME Culhuac n, of Instituto Polit cnico Nacional (a small campus of Mexico's second-largest university). Some friends have asked me, why am I starting with a Specialization and not a Masters degree. Some simple reasons: Just as when I went to Tijuana in 2010 to do my written exam, once I got and started with the paperwork, I didn't want to let it go If I postpone it, I will probably lose the push to do it by May-July, when the Masters admission process starts. Also, this specialization can be linked with the masters degree on the same topic given at the same campus. This program is one year long, and the masters two But having them both takes 2.5 years. So, not such a bad deal after all. And finally, because, after such a long time without being scholarized, I fear not having an easy time getting to grips with the discipline. I can commit to overworking myself for a year If it's too much for me, I'll just stay with that degree and give up. I expect to like it and continue... But it's also a safe bet :-) Now, there has to be a downside to picking up this path: Of course, my free time will be harshly reduced. I have reduced my Debian involvement in the last year, as I devoted a huge chunk of my time to teaching and book-writing... This year... We shall see what happens. I can for now only confirm what I have said publicly but inside our team only: I have requested to my peers and to our DPL to step down as a DebConf chair. I love organizing DebConf, but I don't want to be formally committed to a position I just cannot fulfill as I did when I started with it. As for package maintenance, by far most of my packges are team maintained, and those that are not are relatively easy to keep track of. And of course, I'll keep an eye on my keyring-maint duties as well Will even try to link that work with what I do at school! Anyway, lets see what comes now!

Next.

Previous.