Search Results: "cts"

2 May 2017

Urvika Gola: Debian goes to Hackathon!

Recently I had the opportunity to be a part of Ultrahack as a mentor which was held in Helsinki, Finland.
Ultrahack is a combination of hackathon and startup accelerators.
As a mentor, my role was to ensure that each team have best possible chances of fulfilling the evaluation criteria for the contest. I also helped teams with the development and pitching. It was a very exciting place to brainstorm life changing ideas and convert those ideas into working model. I met so many amazing developers who were building cool stuff. There were a few open source developers and student open source developers like me!
Being a Debian contributor, I spread the what Debian is all about. What makes it the best linux distribution. I talked to students about various programmes like GSoC and Outreachy, Debian participates as a mentoring organisation. I also described my role as a GSoC student under Debian and the free-RTC project I worked under. Many female developers were interested in the Outreachy programme, I described the projects that Debian has currently under the Outreachy programme. During the hackathon period, I talked to people about the upcoming annual DebConf which takes place. I informed them that they can still apply as a speaker or for diversity bursaries and about the logo-making competition. Thank you Debian for always supporting me!

Russ Allbery: Review: Ninefox Gambit

Review: Ninefox Gambit, by Yoon Ha Lee
Series: The Machineries of Empire #1
Publisher: Solaris
Copyright: 2016
ISBN: 1-84997-992-8
Format: Kindle
Pages: 384
Charis is a Kel, which means that she's a soldier of the Hexarchate. A captain, to be precise: Captain Kel Charis of Heron Company, commanding infantry forces to stamp out heresy. The heresy she's stamping out at the start of the book are the Eels, or the Society of the Flourish as they call themselves, and they're strong enough that they can also command heretical technology. Charis manages to win through, but only because she has enough flexibility and quick thinking to adjust to the presence of a heresy and reach beyond the Lexicon Primary, improvising formations on the spot to adjust for the affects of the rebel calendar. Military victory is prized among the Kel, but stepping outside the bounds of Doctrine to achieve it is not. Charis is not particularly surprised when her company is disbanded for re-education after the battle. She is very surprised when she personally is tapped to offer a solution to a far greater attack on the Hexarchate. I first encountered Yoon Ha Lee's fiction in the short story "The Unstrung Zither", was blown away by the creativity and delightfully weird twist on science fiction war, and have been following his writing ever since. Most of it is short fiction, though, and I'm not much of a short fiction reader, so there haven't been many reviews. Ninefox Gambit is his first, and much-anticipated, full-length novel. It's probably not too surprising for someone from the generation that grew up with Star Wars, but I have a soft spot in my heart for magitech. Hard science fiction has its merits, as does the softer sort that takes standard, if impossible, genre tropes for granted. But something about a far-future, space-faring society based on magic that straddles the rules of technology, physics, affinities, and beliefs calls to that part of me that spent hours thinking about the nature of the Force. It has to be good magitech, though: something odd and different but well-thought-out, full of implications and twisty consequences that reshape society and that inspire a whole new type of engineering and science. Magic that's not physics as we know it, but that's knowable, researchable, and something that a society can reshape itself around. This is the good magitech. In the world of the Hexarchate, calendrical systems are more than just a mutual agreement for conveying time. They order and structure the laws of the universe as much as they structure society. What technological devices, and what weapons, are possible is influenced by the calendar in observance, which in turn is based on what calendar people believe in and follow. Close adherence to a calendrical regime enables exotics: weapons with incredibly powerful and often horrific effects, such as the threshold winnower that plays a repeated, nightmarish role in this story. Invariant weapons, ones that will work in any calendar system, are much weaker. The Hexarchate is called that because it is a society formed by six factions, who divide the work of ruling its scattered planets according to the expertise and tendencies of each faction. Together, they impose the high calendar, and maintain it against heresies with an iron fist lest their power be undermined or transformed and their exotics fail to function. The Kel is their military faction, a key component of that fist, and their specialty is formations: specific arrangements of humans or ships that channel the power of the calendar to defend against or attack with exotics. Formations have to be held exactly to hold their power and yet have to be flexible enough to change based on fast-changing battlefield conditions. To assist in this, Kel are programmed with formation instinct: psychological conditioning that helps them obediently take and hold formations. And, not coincidentally, offer nearly absolute obedience to their chain of command. I just finished reading another book that attempted to use math as a key component of its world-building. I think Lee was far more successful. The math here is realistic for its purpose, obviously necessary given the formation structures built into the world's physics, and a lovely nod to the importance of calendars. A single calendar might involve only simple arithmetic, but the formation and technological implications of a calendar, let alone the fuzzy boundaries between two calendars each partially in force, would naturally require tricky advanced mathematics to work out. For someone in Charis's position, mathematical training is a rare but vitally important tactical advantage. As you might have guessed from the amount that I'm talking about combat, Ninefox Gambit is military SF. Charis is a military officer, and a comfortable majority of this book is combat of one kind or another. That's not normally my thing, and I did wish there was a bit more non-military social development. But my normal problem with military SF is that I lack the interest in battlefield tactics and strategy to stay fully engaged by description of battle after battle. Ninefox Gambit is the story of Charis attempting to retake a stronghold of the Hexarchate that's fallen to heretical forces, but Lee adds an important twist that does keep me engaged: Jedao. General Shuos Jedao was the greatest general the Hexarchate had ever seen. He never lost a battle. The only catch is that, in the middle of a highly successful campaign against heretics, he went mad, slaughtering both the heretics and his own troops with a horrific weapon while simultaneously murdering all of his command staff. He's much too dangerous and insane to leave alive, but he was also much too valuable and skillful to lose as a weapon, so for the subsequent centuries he's been kept in a threshold state, an undead ghost. A ghost that the Hexarchate can put into Charis's head, a constant advisor as she's placed in charge of the swarm sent to retake the Fortress of Scattered Needles. A brilliant tactician, sociopath, and mass murderer whose advice can never be trusted. The heart of Ninefox Gambit isn't the combat. It's the interplay of power, analysis, and guesswork under the combat, as Charis attempts to use Jedao's brilliance while not losing her own sense of identity or letting him mess too badly with her head. At the start, she's way out of her depth. But she's thoughtful, careful, has a strong internal sense of identity, and learns fast. And the story of Jedao's past is accurate, but incomplete. For those who are familiar with the often-ornate language and prose style of Yoon Ha Lee's short fiction and who are worried it wouldn't hold up at longer length, note that his style here is much different. There are a few touches of ornate description, but most of the book is written in a straightforward and easily-understandable narrative style. Thankfully, because the layers of tactical thrust and counter-thrust are complicated enough that I would have lost them entirely beneath too-complex prose. There's a lot of brutal death in this book. I got a bit tired of both that and the tactical maneuvering, although that's less the fault of the book and more my own mild antipathy towards military SF. But the unique universe background held my interest long enough to become intrigued by Charis's slow but determined probing at Jedao's secrets and the politics of the Hexarchate. I still would have preferred the story to have a somewhat lower body count, but as long as one can read past some gore, there's plenty here to appeal to someone who normally gives military SF a pass. I think its biggest drawback is that, although it has a narrative arc that comes to a clear conclusion, Ninefox Gambit raises a lot of important questions about its world and mostly doesn't answer them. There are more books coming, and I hope they contain more definitive answers. Followed by Raven Stratagem. Rating: 7 out of 10

1 May 2017

Matthew Garrett: Intel's remote AMT vulnerablity

Intel just announced a vulnerability in their Active Management Technology stack. Here's what we know so far.
Background Intel chipsets for some years have included a Management Engine, a small microprocessor that runs independently of the main CPU and operating system. Various pieces of software run on the ME, ranging from code to handle media DRM to an implementation of a TPM. AMT is another piece of software running on the ME, albeit one that takes advantage of a wide range of ME features.
Active Management TechnologyAMT is intended to provide IT departments with a means to manage client systems. When AMT is enabled, any packets sent to the machine's wired network port on port 16992 or 16993 will be redirected to the ME and passed on to AMT - the OS never sees these packets. AMT provides a web UI that allows you to do things like reboot a machine, provide remote install media or even (if the OS is configured appropriately) get a remote console. Access to AMT requires a password - the implication of this vulnerability is that that password can be bypassed.
Remote managementAMT has two types of remote console: emulated serial and full graphical. The emulated serial console requires only that the operating system run a console on that serial port, while the graphical environment requires drivers on the OS side requires that the OS set a compatible video mode but is also otherwise OS-independent[2]. However, an attacker who enables emulated serial support may be able to use that to configure grub to enable serial console. Remote graphical console seems to be problematic under Linux but some people claim to have it working, so an attacker would be able to interact with your graphical console as if you were physically present. Yes, this is terrifying.
Remote mediaAMT supports providing an ISO remotely. In older versions of AMT (before 11.0) this was in the form of an emulated IDE controller. In 11.0 and later, this takes the form of an emulated USB device. The nice thing about the latter is that any image provided that way will probably be automounted if there's a logged in user, which probably means it's possible to use a malformed filesystem to get arbitrary code execution in the kernel. Fun!

The other part of the remote media is that systems will happily boot off it. An attacker can reboot a system into their own OS and examine drive contents at their leisure. This doesn't let them bypass disk encryption in a straightforward way[1], so you should probably enable that.
How bad is thisThat depends. Unless you've explicitly enabled AMT at any point, you're probably fine. The drivers that allow local users to provision the system would require administrative rights to install, so as long as you don't have them installed then the only local users who can do anything are the ones who are admins anyway. If you do have it enabled, though
How do I know if I have it enabled?Yeah this is way more annoying than it should be. First of all, does your system even support AMT? AMT requires a few things:

1) A supported CPU
2) A supported chipset
3) Supported network hardware
4) The ME firmware to contain the AMT firmware

Merely having a "vPRO" CPU and chipset isn't sufficient - your system vendor also needs to have licensed the AMT code. Under Linux, if lspci doesn't show a communication controller with "MEI" or "HECI" in the description, AMT isn't running and you're safe. If it does show an MEI controller, that still doesn't mean you're vulnerable - AMT may still not be provisioned. If you reboot you should see a brief firmware splash mentioning the ME. Hitting ctrl+p at this point should get you into a menu which should let you disable AMT.
How about over Wifi?Turning on AMT doesn't automatically turn it on for wifi. AMT will also only connect itself to networks it's been explicitly told about. Where things get more confusing is that once the OS is running, responsibility for wifi is switched from the ME to the OS and it forwards packets to AMT. I haven't been able to find good documentation on whether having AMT enabled for wifi results in the OS forwarding packets to AMT on all wifi networks or only ones that are explicitly configured.
What do we not know?We have zero information about the vulnerability, other than that it allows unauthenticated access to AMT. One big thing that's not clear at the moment is whether this affects all AMT setups, setups that are in Small Business Mode, or setups that are in Enterprise Mode. If the latter, the impact on individual end-users will be basically zero - Enterprise Mode involves a bunch of effort to configure and nobody's doing that for their home systems. If it affects all systems, or just systems in Small Business Mode, things are likely to be worse.
We now know that the vulnerability exists in all configurations.
What should I do?Make sure AMT is disabled. If it's your own computer, you should then have nothing else to worry about. If you're a Windows admin with untrusted users, you should also disable or uninstall LMS by following these instructions.
Does this mean every Intel system built since 2008 can be taken over by hackers?No. Most Intel systems don't ship with AMT. Most Intel systems with AMT don't have it turned on.
Does this allow persistent compromise of the system?Not in any novel way. An attacker could disable Secure Boot and install a backdoored bootloader, just as they could with physical access.
But isn't the ME a giant backdoor with arbitrary access to RAM?Yes, but there's no indication that this vulnerability allows execution of arbitrary code on the ME - it looks like it's just (ha ha) an authentication bypass for AMT.
Is this a big deal anyway?Yes. Fixing this requires a system firmware update in order to provide new ME firmware (including an updated copy of the AMT code). Many of the affected machines are no longer receiving firmware updates from their manufacturers, and so will probably never get a fix. Anyone who ever enables AMT on one of these devices will be vulnerable. That's ignoring the fact that firmware updates are rarely flagged as security critical (they don't generally come via Windows update), so even when updates are made available, users probably won't know about them or install them.
Avoiding this kind of thing in futureUsers ought to have full control over what's running on their systems, including the ME. If a vendor is no longer providing updates then it should at least be possible for a sufficiently desperate user to pay someone else to do a firmware build with the appropriate fixes. Leaving firmware updates at the whims of hardware manufacturers who will only support systems for a fraction of their useful lifespan is inevitably going to end badly.
How certain are you about any of this?Not hugely - the quality of public documentation on AMT isn't wonderful, and while I've spent some time playing with it (and related technologies) I'm not an expert. If anything above seems inaccurate, let me know and I'll fix it.

[1] Eh well. They could reboot into their own OS, modify your initramfs (because that's not signed even if you're using UEFI Secure Boot) such that it writes a copy of your disk passphrase to /boot before unlocking it, wait for you to type in your passphrase, reboot again and gain access. Sealing the encryption key to the TPM would avoid this.

[2] Updated after this comment - I thought I'd fixed this before publishing but left that claim in by accident.

(Updated to add the section on wifi)

(Updated to typo replace LSM with LMS)

(Updated to indicate that the vulnerability affects all configurations)

comment count unavailable comments

30 April 2017

Paul Wise: FLOSS Activities April 2017

Changes

Issues

Review

Administration
  • Debian systems: quiet a logrotate warning, investigate issue with DNSSEC and alioth, deploy fix on our first stretch buildd, restore alioth git repo after history rewrite, investigate iptables segfaults on buildd and investigate time issues on a NAS
  • Debian derivatives census: delete patches over 5 MiB, re-enable the service
  • Debian wiki: investigate some 403 errors, fix alioth KGB config, deploy theme changes, close a bogus bug report, ping 1 user with bouncing email, whitelist 9 email addresses and whitelist 2 domains
  • Debian QA: deploy my changes
  • Debian mentors: security upgrades and service restarts
  • Openmoko: debug mailing list issue, security upgrades and reboots

Communication
  • Invite Wazo to the Debian derivatives census
  • Welcome ubilinux, Wazo and Roopa Prabhu (of Cumulus Linux) to the Debian derivatives census
  • Discuss HP/ProLiant wiki page with HPE folks
  • Inform git history rewriter about the git mailmap feature

Sponsors The libconfig-crontab-perl backports and pyvmomi issue were sponsored by my employer. All other work was done on a volunteer basis.

Matthew Garrett: Looking at the Netgear Arlo home IP camera

Another in the series of looking at the security of IoT type objects. This time I've gone for the Arlo network connected cameras produced by Netgear, specifically the stock Arlo base system with a single camera. The base station is based on a Broadcom 5358 SoC with an 802.11n radio, along with a single Broadcom gigabit ethernet interface. Other than it only having a single ethernet port, this looks pretty much like a standard Netgear router. There's a convenient unpopulated header on the board that turns out to be a serial console, so getting a shell is only a few minutes work.

Normal setup is straight forward. You plug the base station into a router, wait for all the lights to come on and then you visit arlo.netgear.com and follow the setup instructions - by this point the base station has connected to Netgear's cloud service and you're just associating it to your account. Security here is straightforward: you need to be coming from the same IP address as the Arlo. For most home users with NAT this works fine. I sat frustrated as it repeatedly failed to find any devices, before finally moving everything behind a backup router (my main network isn't NATted) for initial setup. Once you and the Arlo are on the same IP address, the site shows you the base station's serial number for confirmation and then you attach it to your account. Next step is adding cameras. Each base station is broadcasting an 802.11 network on the 2.4GHz spectrum. You connect a camera by pressing the sync button on the base station and then the sync button on the camera. The camera associates with the base station via WPS and now you're up and running.

This is the point where I get bored and stop following instructions, but if you're using a desktop browser (rather than using the mobile app) you appear to need Flash in order to actually see any of the camera footage. Bleah.

But back to the device itself. The first thing I traced was the initial device association. What I found was that once the device is associated with an account, it can't be attached to another account. This is good - I can't simply request that devices be rebound to my account from someone else's. Further, while the serial number is displayed to the user to disambiguate between devices, it doesn't seem to be what's used internally. Tracing the logon traffic from the base station shows it sending a long random device ID along with an authentication token. If you perform a factory reset, these values are regenerated. The device to account mapping seems to be based on this random device ID, which means that once the device is reset and bound to another account there's no way for the initial account owner to regain access (other than resetting it again and binding it back to their account). This is far better than many devices I've looked at.

Performing a factory reset also changes the WPA PSK for the camera network. Newsky Security discovered that doing so originally reset it to 12345678, which is, uh, suboptimal? That's been fixed in newer firmware, along with their discovery that the original random password choice was not terribly random.

All communication from the base station to the cloud seems to be over SSL, and everything validates certificates properly. This also seems to be true for client communication with the cloud service - camera footage is streamed back over port 443 as well.

Most of the functionality of the base station is provided by two daemons, xagent and vzdaemon. xagent appears to be responsible for registering the device with the cloud service, while vzdaemon handles the camera side of things (including motion detection). All of this is running as root, so in the event of any kind of vulnerability the entire platform is owned. For such a single purpose device this isn't really a big deal (the only sensitive data it has is the camera feed - if someone has access to that then root doesn't really buy them anything else). They're statically linked and stripped so I couldn't be bothered spending any significant amount of time digging into them. In any case, they don't expose any remotely accessible ports and only connect to services with verified SSL certificates. They're probably not a big risk.

Other than the dependence on Flash, there's nothing immediately concerning here. What is a little worrying is a family of daemons running on the device and listening to various high numbered UDP ports. These appear to be provided by Broadcom and a standard part of all their router platforms - they're intended for handling various bits of wireless authentication. It's not clear why they're listening on 0.0.0.0 rather than 127.0.0.1, and it's not obvious whether they're vulnerable (they mostly appear to receive packets from the driver itself, process them and then stick packets back into the kernel so who knows what's actually going on), but since you can't set one of these devices up in the first place without it being behind a NAT gateway it's unlikely to be of real concern to most users. On the other hand, the same daemons seem to be present on several Broadcom-based router platforms where they may end up being visible to the outside world. That's probably investigation for another day, though.

Overall: pretty solid, frustrating to set up if your network doesn't match their expectations, wouldn't have grave concerns over having it on an appropriately firewalled network.

(Edited to replace a mistaken reference to WDS with WPS)

comment count unavailable comments

Russ Allbery: Review: Periodic Tales

Review: Periodic Tales, by Hugh Aldersey-Williams
Publisher: HarperCollins
Copyright: February 2011
ISBN: 0-06-207881-X
Format: Kindle
Pages: 451
Perhaps my favorite chapter in Randall Munroe's What If? is his examination of what would happen if you assembled a periodic table from square blocks of each element. As with most What If? questions, the answer is "everyone in the vicinity dies," but it's all about the journey. The periodic table is full of so many elements that we rarely hear about but which have fascinating properties. It was partly in the memory of that chapter that I bought Periodic Tales on impulse after seeing a mention of it somewhere on the Internet (I now forget where). Hugh Aldersey-Williams is a journalist and author, but with a background in natural sciences. He also has a life-long hobby of collecting samples of the elements and attempting to complete his own private copy of the periodic table, albeit with considerably more precautions and sample containment than Munroe's thought experiment. Periodic Tales is inspired by that collection. It's a tour and cultural history of many of the elements, discussing their discovery, their role in commerce and industry, their appearance, and often some personal anecdotes. This is not exactly a chemistry book, although there's certainly some chemistry here, nor is it a history, although Aldersey-Williams usually includes some historical notes about each element he discusses. The best term might be an anthropology of the elements: a discussion of how they've influenced culture and an examination of the cultural assumptions and connections we've constructed around them. But primarily it's an idiosyncratic and personal tour of the things Aldersey-Williams found interesting about each one. Periodic Tales is not comprehensive. The completionist in me found that a bit disappointing, and there are a few elements that I think would have fit the overall thrust of the book but are missing. (Lithium and its connection to mental health and now computer batteries comes to mind.) It's also not organized in the obvious way, either horizontally or vertically along the periodic table. Instead, Aldersey-Williams has divided the elements he talks about into five major but fairly artificial divisions: power (primarily in the economic sense), fire (focused on burning and light), craft (the materials from which we make things), beauty, and earth. Obviously, these are fuzzy; silver appears in craft, but could easily be in power with gold. I'm not sure how defensible this division was. But it does, for good or for ill, break the reader's mind away from a purely chemical and analytical treatment and towards broader cultural associations. This cultural focus, along with Aldersey-Williams's clear and conversational style, are what pull this book firmly away from being a beautified recitation of facts that could be gleamed from Wikipedia. It also leads to some unexpected choices of focus. For example, the cultural touchstone he chooses for sodium is not salt (which is a broad enough topic for an entire book) but sodium street lights, the ubiquitous and color-distorting light of modern city nights, thus placing salt in the "fire" category of the book. Discussion of cobalt is focused on pigments: the brilliant colors of paint made possible by its many brightly-colored compounds. Arsenic is, of course, a poison, but it's also a source of green, widely used in wallpaper (and Aldersey-Williams discusses the connection with the controversial death of Napoleon). And the discussion of aluminum starts with a sculpture, and includes a fascinating discussion of "banalization" as we become used to use of a new metal, which the author continues when looking a titanium and its currently-occurring cultural transition between the simply new and modern and a well-established metal with its own unique cultural associations. One drawback of the somewhat scattered organization is that, while Periodic Tales provides fascinating glimmers of the history of chemistry and the search to isolate elements, those glimmers are disjointed and presented in no particular order. Recently-discovered metals are discussed alongside ancient ones, and the huge surge in elemental isolation in the 1800s is all jumbled together. Wikipedia has a very useful timeline that helps sort out one's sense of history, but there was a part of me left wanting a more structured presentation. I read books like this primarily for the fascinating trivia. Mercury: known in ancient times, but nearly useless, so used primarily for ritual and decoration (making the modern reader cringe). Relative abundancies of different elements, which often aren't at all what one might think. Rare earths (not actually that rare): isolated through careful, tedious work by Swedish mining chemists whom most people have never heard of, unlike the discoverers of many other elements. And the discovery of the noble gases, which is a fascinating bit of disruptive science made possible by new technology (the spectroscope), forcing a rethinking of the periodic table (which had no column for noble gases). I read a lot of this while on vacation and told interesting tidbits to my parents over breakfast or dinner. It's that sort of book. This is definitely in the popular science and popular writing category, for all the pluses and minuses that brings. It's not a detailed look at either chemistry or history. But it's very fun to read, it provides a lot of conversational material, and it takes a cultural approach that would not have previously occurred to me. Recommended if you like this sort of thing. Rating: 7 out of 10

29 April 2017

Shirish Agarwal: India and the Agricultural Economy

Indian farmer ploughing a field in traditional style - Wikimedia commons I was in two minds when I read Ritesh s blog post about the Indian Economy. I was angry with Ritesh as he seemed to selectively take facts and present it rather than taking it whole. Even if he had searched even a little bit, he would have got much more better material and everybody would have been the gainer. I have to also admit, I feel very much like a hypocrite as I have never slaved in a farm so my understanding and conclusions are a mix of media and limited interaction with farmers some years ago. There is also lots of local customs and politics that come into the picture and it s not as straight-forward as Ritesh thinks. What he has failed to share/account for is the far worse bad and stressed debts for the industry so just saying farm loan waivers are bad without sharing any of the context makes it seem much more worse. This is when our Current Chief Economic Adviser states about loan waivers to corporates You need to be able to forgive those debts because this is how capitalism works. People make mistakes, those have to be forgiven to some extent Let me start though with words from a book I read sometime back On a peasant uprising erupted in . The farmers were angry with high interest rates, high taxes, high inflation and low-government prices for their crops. The system had let them into debt, and debt had meant foreclosure and loss of their fields to the land barons. I intentionally have made a fill in the blanks as both the dates and places was true in India 100 years back and even today, the only difference between the two is the absence of taxes. Many people would think I m talking about Champaran whose tale while well-known in India is sadly a stub-class article in wikipedia with quite a few citation needed tags as well but is also true today as will be seen below. Interestingly, there is/was a remark by some unknown person who said gora sahab gays, bhura sahab aaya meaning the white officer has gone, in his place the brown officer has come. The evidence of this is very much in the Telegraph Act and the story about its usage and its place in Indian politics Surprisingly, sadly and coincidentally, the quote minus the dates and place didn t happen in India but also in Cambodia. The above quote has been taken from for the sake of all living things by John M. Del Vecchio. The quote itself appears in the first 10 odd pages (historical summation) of the somewhat 1200 odd pages book. I actually got an old edition which tops out at 900 pages so probably some more updated input/news isn t available but it still packs a powerful wallop. I want to dedicate a separate blog post for the book itself so will not say more on that book and what it shares. Sadly and coincidentally, there were news reports yesterday itself of farmers agitating for better prices just yesterday. Some of the interesting work if you want to understand the farmer s indebtedness is to study the Income, Expenditure, Productive Assets and Indebtedness of Agricultural Households in India done by NSSO. Again, one does not need to read the whole report, there were some of the analysis shared by the Hindu here and here. This was also echoed by Logical Indian It really boggles the mind to know than an average farming household earns around INR 200 per day. Even if you take a family of four people that comes to INR 50/- per person. Most rural joint families at the very least have 3-4 kids at the very least. Sadly Agricultural incomes do not keep sync even with the inflation index as there is no fair minimum age and wage for the Indian farmer, the concept does not exist for her(im) . Just me and mum going onto a restaurant and having one dish each easily can run anywhere between INR 200~250 easily . Cooking in the house is the same if you add/input the labor (which is usually not calculated) used to make lunch/dinner. There was the idea that contract farming might be a solution but even that was corrupted by Multi-national companies such as Pepsi and others that the Government is showing movement to have a model contracting law . There are loads of stories on downtoearth magazine which deals with the above and all sorts of issues the farmer faces. I should talk about Maharashtra and even here there was/has been an irrigation scam with figures given from 35k to 70k crores or 350 to 700 billion INR . The latest finding by the PAC has been shared here. I will cut the blog post short as I find the whole thing personally very depressing. As far as local customs go this was from one of the farmers interaction some years ago where me and some other friends had gone across a village and came to know that all of them grew the same crop with some minor variances. When asked why is it so, while many said its fate, one of the elderly gentleman shared an experience where a farmer had planted some other thing. The gentleman prospered while the other villagers were suffering from glut of whatever they produced. Knowing he prospered, the other villagers damaged his crops and all sorts of unlucky things started happening for the farmer. In the end he realized his best bet is to follow the ways of the other villagers, at least they would be in peace. What I have shared isn t either unique or even unknown, even Toronto Star of Canada reported on the issue some years ago. At the end of it all, the story is one of no education and limited skill-set and I don t see it changing any time soon. There are some who are earning big figures, but majority of the farmers will always be in the red
Filed under: Miscellenous Tagged: #Agricultal Economy, #Champaran, #Contract farming, #Corruption, #farmer suicide, #irrigation scam, #Loan waivers, #planet-debian, poverty

Antoine Beaupr : My free software activities, April 2017

Debian Long Term Support (LTS) This is my monthly Debian LTS report. My time this month was spent working on various hairy security issues, most notably XBMC (now known as Kodi) and yaml-cpp.

Kodi directory transversal I started by looking in CVE-2017-5982, a "directory traversal" vulnerability in XBMC (now known as Kodi) which is a technical term for "allow attackers to read any world-readable file on your computer from the network". It's a serious vulnerability which has no known fix. When you enable the "remote control" interface in Kodi, it allows anyone with the password (which is disabled by default) to download any files Kodi has read access to on the machine it's running. Considering Kodi is often connected to multiple services, this may mean elevated compromise and more nasty stuff. I furthered the investigation done with my own analysis which showed the problem is difficult to solve: Kodi internally uses the facility to show thumbnails and media to the user, and there are no clear way of restricting which paths Kodi should have access to. Indeed, Kodi is designed to access mounted file systems and paths in arbitrary locations. In Debian bug #855225, I further confirmed confirmed wheezy and jessie-backports as vulnerable and therefore showed with good certainty that stretch and sid are vulnerable as well. I also suggested possible workaround, but at this point, it's in upstream's hands, as the changes will be intrusive. The file transfer mechanism need to be revamped all over Kodi, or authentication (with a proper password policy), need to be enforced.

Squirrelmail Next I looked at that old webmail software, Squirrelmail, which suffers from a remote code execution vulnerability (CVE-2017-7692) when sending mails with sendmail on the commandline. This is arguably an edge case, but considering the patch was simple, I figured I would provide an update to the LTS community. I tried to get a coordinated release for jessie, since the code is the same, but this wasn't completed at the time of writing. A patch is available and will hopefully be picked up by another LTS worker soon.

Fop and Batik Those issues (CVE-2017-5661 and CVE-2017-5662) were more difficult. The patches weren't clearly documented and there were no upstream references other than security advisories for the first release in years (in the case of batik) or months (in the case of fop), which made it hard to track down the issues. Fortunately, I was able to track down the upstream issues (FOP-2668 and BATIK-1139) where I got confirmation on what the proper fixes were. I could then release DLA-927-1 and DLA-926-1 with the backported patches. I do not use fop or batik. In fact, even after reading the homepage of both products, I couldn't quite figure out what use people could possibly have for that thing. Before uploading the packages, I therefore made packages available for testing for fop and batik.

libsndfile Next up was libsndfile which a bunch of overflows when parsing various audio files. I backported a patch for CVE-2017-7585 CVE-2017-7586 and CVE-2017-7741 which all seemed to be fixed by a single patch usptream. CVE-2017-7742 was also fixed, although with a separate patch. In all of those, i could only test CVE-2017-7741 and CVE-2017-7742, as the others were missing test cases. I provided a test package for a few days then I also figured it would be best to incorporate the security fixes done in stable, which brought in fixes for CVE-2015-7805, CVE-2014-9756 and CVE-2014-9496. So in the end, I ported patches from wheezy to jessie and uploaded the jessie version (reverting certain build changes) into wheezy and uploaded DLA-928-1 with the results.

yaml-cpp I then turned to yaml-cpp, a C++ parser for YAML. This one didn't have a known upstream fix, but I figured I would give it a shot anyways. I ended up writing my first C++ code in years which is still pending review and merge upstream. It's not an easy problem to fix: this is basically an excessive recursion problem that can be used to smash the stack. I figured I could introduce a recursion limit, but as the discussion showed, this is a limited approach: stack size varies on different platforms and it's not easy to find the right limit. The real solution is to rewrite the code to avoid recursion but that's a major code refactoring I didn't feel belong in a LTS update. Besides, this could be better handled by upstream, so I will leave things at that for now. It does make you wonder how much code out there is recursing on untrusted data structures...

kedpm Finally, a friend over at Koumbit.org reported Debian bug #860817, as information leak in kedpm, a password manager I previously maintained. I requested and got assigned CVE-2017-8296 and provided a fix for wheezy and jessie. For unstable and the coming stretch release, I have requested kedpm to be completely removed from Debian (Debian bug #860817) which involved a release notes update (Debian bug #861277). It's unfortunate to see software go, but kedpm wasn't maintained. I wasn't the original author: I just gave a few patches and ended up maintaining that software and not using it. It's a bad situation to be in, as you don't really know what's working and not with the tools you are supposed to be responsible for. There are more modern alternatives available now and I encourage everyone to switch.

Triage Looking for more work, I peeked a bit in the secretary tasks to triage some pending issues. I found that trafficserver could be crashed with simple requests (CVE-2017-5659) so I looked into that issue. My analysis showed that the patch is long and complex and could be difficult to backport to the old version available in wheezy. I also couldn't reproduce the issue in wheezy, so it may be a bug introduced only later, although I couldn't confirm that directly. I also triaged wireshark, where I just noted the maintainer expressed concern that we were taking up issues too fast and will probably take care of this one. I also postponed various issues in GnuTLS (marked "no-dsa") as they affect only a (unfortunately) rarely used part of GnuTLS that has been removed in later version: OpenPGP support.

Other free software work

Debiman I finally got around contributing to the debiman project. I worked on ensuring that there is a dman compatibility in debiman, by shipping dman in the debian-goodies package (Debian bug #860920). I also submitted a pull request to fix the fix about page title, document the custom assets repository, fix a stray bracket and link to the link to venerable man7.org project After a discussion on IRC, I also filed a few more issues: I'm happy to be able to contribute to this important service and I hope the new FAQ I created will be online soon!

XMonad and Emacs I started using writeroom-mode again as part of my work on LWN. As it turns out, my setup was not exactly working: I had to port my config to the new version and windows weren't "sticky" as they should be, a known issue with Xmonad. Indeed, Xmonad doesn't obey the "static" or "all desktops" standard directives. Writeroom is a "distraction-free writing" mode for Emacs, so the irony of working on such a deep distraction in establishing a distraction free environment is not lost on me. Needing to scratch that particular itch, and with the help of clever people from the IRC channel, I was able to make Emacs tell Xmonad to show its window (or "frame" as Emacs likes to call it) on all desktops. This involved creating a new function which I think could be useful in the CopyWindow library: 
--   Toggle between "copyToAll" or "killAllOtherCopies". Copies to all
-- workspaces, or remove from all other workspaces, depending on
-- previous state (checked with "wsContainingCopies").
copyToAllToggle :: X ()
copyToAllToggle = do
    -- check which workspaces have copies
    copies <- wsContainingCopies
    if null copies
      then windows copyToAll -- no workspaces, make sticky
      else killAllOtherCopies -- already other workspaces, unstick
There are probably better ways of implementing this directly in the CopyWindow code - wsContainingCopies, in particular, is probably overkill. But it's all I can use directly from my xmonad.hs, so that's what I did. The other bit I needed was something to trigger that function from the outside. I rejected the ServerMode hook because it looked a bit too complicated and there is a built-in facility within X that works without this, which, from Emacs' point of view, is the x-send-client-message function. So I made up a new message identifier and wrote a event hook handler to process it: 
--   handle X client messages that tell Xmonad to make a window appear
-- on all workspaces
--
-- this should really be using _NET_WM_STATE and
-- _NET_WM_STATE_STICKY. but that's more complicated: then we'd need
-- to inspect a window and figure out the current state and act
-- accordingly. I am not good enough with Xmonad to figure out that
-- part yet.
--
-- Instead, just check for the relevant message and check if the
-- focused window is already on all workspaces and toggle based on
-- that.
--
-- this is designed to interoperate with Emacs's writeroom-mode module
-- and called be called from elisp with:
--
-- (x-send-client-message nil 0 nil "XMONAD_COPY_ALL_SELF" 8 '(0))
myClientMessageEventHook :: Event -> X All
myClientMessageEventHook (ClientMessageEvent  ev_message_type = mt, ev_data = dt ) = do
  dpy <- asks display
  -- the client message we're expecting
  copyAllMsg <- io $ internAtom dpy "XMONAD_COPY_ALL_SELF" False
  -- if the event matches the message we expect, toggle sticky state
  when (mt == copyAllMsg && dt /= []) $ do
    copyToAllToggle
  -- we processed the event completely
  return $ All True
All that was left was to hook that into Emacs, and I was done! Whoohoo! Full screen total domination, distraction free work! :) I would love to hear from others what they think of that approach, if they have improvements or if the above copyToAllToggle function could be merged in. Ideally, Xmonad would just parse the STICKY client messages and do the right thing - maybe even directly in CopyWindow - but I have found this enough Haskell for one day. You can see the diff on my home directory to see exactly the changes involved to make this configuration work.

Emacs packaging Speaking of Emacs, after complaining in the noisy #emacs IRC channel about the poor TLS configuration of marmelade.org -- and filing a bug (Debian bug #861106) regarding the use of SHA-1 in certificate pinning -- I was told we shouldn't expect trust from third-party ELPA repositories. Marmelade seems to be dead, as the maintainer is "behind the great firewall of China" and MELPA still hasn't figured out how to sign packages. In the end, it seems like there are tons of elpa packages in Debian and that if your favorite one is missing, that's a bug that can be filed and fixed. I first discovered that 6 of the packages I used were already packaged: And so I went ahead and filed a ton more bugs for the packages I am using but that aren't in Debian just yet: Of those, I can't recommend multiple-cursors (MC) enough: I used it at least 4 times just writing this text. It's just awesome. The other ones are also all great in their own right of course, but I feel they are more specific to my workflow whereas MC is just amazing.

ikiwiki I did some more work on ikiwiki the software driving this blog. I created a new plugin to, at least, fix anchors in the table of contents to be human readable. This is something I had done on the MoinMoin wiki almost a decade ago -- which I called then NicerHeadingIds and that I have always found frustrating with Ikiwiki. It turns out the problem was both easier and hairier than I thought. Right from the start, something weird was happening: something was already adding nice headings, but they were somewhat broken. It turns out that multimarkdown already inserts those headers, but I wasn't satisfied with the way they were generated. But even worse, I had the headinganchors plugin enabled, but that plugin wasn't taking effect, because of multimarkdown. And even if it would take effect, it doesn't behave well with non-ASCII characters, which gets turned in their numeric presentation. So I also wrote the i18nheadinganchors plugin that creates better headings and patched the toc plugin so that it can reuse existing anchors if they exist, while keeping backwards compatibility. I hope this gets merged in a future ikiwiki release so I do not have to carry this patch locally too long... In other news, I have upgraded the ikiwiki-hosting package to the latest version and sent a patch upstream to provide HSTS support.

Other stuff I have migrated all my public repos hosted on my home server to either Gitlab.com or Github. I also have repositories on 0xacab and it seemed ludicrous to have 4 different, canonical, places where my code was hosted. I have now about 40 different projects on Gitlab and about 60 on Github, although most of the latter are forks of existing projects. I also made a manpage for stressant and moved the documentation to RTFD which makes it neatly accessible. I also made small incremental improvements (like --directory support). I installed Rainloop on this server to give a nice, mobile-friendly webmail. Instructions to replicate this setup are in mail. In the constant git-annex documentation effort, I tried to draft a user guide that could be a basis for restructuring the documentation to be more easily accessible. I also helped a friend put his documentation on the wiki in splitting a repository. Finally, I also looked into Android stuff a little more. I wrote a usability review of the F-Droid privileged extension that will bring good changes, I hope. I also opened the discussion regarding reproducible builds to try and clarify exactly how those worked to help the Wallabag people ship consistently signed alphas. So far, it seems that it will remain a standard practice on F-Droid to ship packages that are not signed by the official upstream signature, unfortunately, unless upstream provides a reproducible build that is publicly available... Switching to such build is also a hairy issue, as, obviously, the signature changes, which raises the alarm we are trying to avoid in the first place.

27 April 2017

Joey Hess: Exede Surfbeam 2

My new satellite internet connection is from Exede, connecting to the ViaSat 1 bird in geosync orbit. A few technical details that I've observed follow. antagonistic by design The "Surfbeam 2 wifi modem" is a closed proprietary system. That is important because it's part of Exede's bandwidth management system. The Surfbeam tracks data use and sends it periodically to Exede. When a user has gone over their monthly priority data, Exede then throttles the bandwidth in various ways -- this throttling seems to be implemented, at least partially on the Surfbeam itself. (Perhaps by setting QoS flags?) So, if a user could hack their Surfbeam, they could probably bypass the bandwidth caps, or at least some of them. Perhaps Exede would notice eventually. Of course, doing so would surely violate the Exede TOS. If you're renting the modem, like I am, hacking a device you don't own might also subject you to criminal penalties. Needless to say, I don't plan to hack the SurfBeam. But it's been hacked before. So, this is a device that lives in people's homes and is antagonistic to them by design. weird data throttling The way the Surfbeam reports data use back to Exede periodically and gets throttling configured has some odd effects sometimes. For example, the Surfbeam can be in throttled state left-over from the previous billing month. When a new billing month begins, it can remain throttled for some time (up to multiple hours) until it sends an update to Exede and they un-throttle it. Data downloaded at that time might still be counted as priority data even though it was throttled. I've seen some good indications of that happening, but am not sure yet. But, I've decided that the throttling doesn't matter for me. Why? ViaSat 1 has many spot beams, and the working-class beam I'm in (most of it is in eastern Kentucky) does not seem to get a lot of use between 7 am and 4:30 pm weekdays. Even when throttled, I often get 300 kb/s - 1 mb/s speeds during the day, which is not a lot worse than the ~2.5 mb/s peak when unthrottled. And that's the time when I want to use broadband -- when the sun is shining and I'm at home at work/play. I'm probably going to switch to a cheaper plan with less priority data, because the priority data is not buying me much. This is a big change from the old FAP which rendered the satellite no faster than dialup. a whole network in there Looking at the ports open on the Surfbeam, some very strange things turned up. First, there are not one, not two, but three separate IPs used by the device, and there are at least two and perhaps three distinct computers involved. There are a lot of flickering LEDs inside the box; a whole network in there. 192.168.100.1 is the satellite controller. It's a Linux box, fingerprinted as kernel 3.10 or so (so full of security holes presumably), and it's running thttpd/2.25b (doesn't seem to have any known holes). It seems to have ssh and snmp, but with some port filtering that prevents access. (Note that the above exploit video confirms that snmp is running.) Some machine parsable data is exposed at http://192.168.100.1/index.cgi?page=modemStatusData and http://192.168.100.1/index.cgi?page=triaStatusData. (See (SurfStat program) 192.168.1.1 is the wifi router. It has a dns server, an icslap proxy, and nmap thinks it's Linux 3.x with Synology DiskStation Manager (probably the latter is a false positive?) It has its own separate web server for configuration, which is not thttpd. I'm fairly sure this is a separate processor from the other IP address. 192.168.100.2 responds to ICMP, but has no open ports at all. However, it seems to have filtered ssh, telnet, msrpc, microsoft-ds, and port 8090 (probably http), so perhaps it's running all that stuff. This one is definitely a separate processor, located in the Satellite dish's TRIA (transmit receive integrated assembly). Verified by disconnecting the dish's coax cable and being unable to ping it. lack of source code for GPLed software Exede did not provide anything to me about the GPL licensed source code on the Surfbeam 2. I'm not sure if they're legally required to do so in my case, since they're renting it to me? But, they do let you buy the device too, so it's interesting that nothing mentions the licenses of its software and where to get the source code. I'll try to buy it and ask for the source code eventually. power use The Surfbeam pulls as much power as two beefy laptops, but it is beaming signals thousands of miles into space, so I give it a bit of a pass. I want to find a way to power it from DC power, but have not had any success with that so far, so am wasting more power to run it on an inverter. Its power supply is rated at 30v and 2.5 amps. Interestingly, I've seen a photo of another Surfbeam power supply that only pulls 1.5 amps. This and a different case with fewer (external) LEDs makes me think perhaps the installer stuck me with an old and less efficient version. Maybe they integrated some of those processors into a single board in a new version and perhaps made it waste less power as heat. Mine gets pretty warm. I'm currently only able to run it approximately one hour per hour of sun collected by the house. Still on dialup the rest of the time. With the ability to get on broadband when dialup is being painful, being on dialup the rest of the time is perfectly ok. Of course it helps that with tools like git-annex, I'm very well tuned to popping up onto broadband briefly and making it count.

26 April 2017

Dirk Eddelbuettel: RcppTOML 0.1.3

A bug fix release of RcppTOML arrived on CRAN today. Table arrays were (wrongly) not allowing for nesting; a simply recursion fix addresses this. RcppTOML brings TOML to R. TOML is a file format that is most suitable for configurations, as it is meant to be edited by humans but read by computers. It emphasizes strong readability for humans while at the same time supporting strong typing as well as immediate and clear error reports. On small typos you get parse errors, rather than silently corrupted garbage. Much preferable to any and all of XML, JSON or YAML -- though sadly these may be too ubiquitous now. TOML is making good inroads with newer and more flexible projects such as the Hugo static blog compiler, or the Cargo system of Crates (aka "packages") for the Rust language.

Changes in version 0.1.3 (2017-04-25)
  • Nested TableArray types are now correctly handled (#16)

Courtesy of CRANberries, there is a diffstat report for this release. More information is on the RcppTOML page page. Issues and bugreports should go to the GitHub issue tracker.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

25 April 2017

Reproducible builds folks: Reproducible Builds: week 104 in Stretch cycle

Here's what happened in the Reproducible Builds effort between Sunday April 16 and Saturday April 22 2017: Upcoming events Reproducible work in other projects Packages reviewed and fixed, and bugs filed Chris Lamb: Chris West: Reviews of unreproducible packages 37 package reviews have been added, 64 have been updated and 16 have been removed in this week, adding to our knowledge about identified issues. One issue type has been updated: Two issue types have been added: Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by: diffoscope development Misc. This week's edition was written by Chris Lamb, Vagrant Cascadian & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

22 April 2017

Enrico Zini: Splitting a git-annex repository

I have a git annex repo for all my media that has grown to 57866 files and git operations are getting slow, especially on external spinning hard drives, so I decided to split it into separate repositories. This is how I did it, with some help from #git-annex. Suppose the old big repo is at ~/oldrepo: 
# Create a new repo for photos only
mkdir ~/photos
cd photos
git init
git annex init laptop
# Hardlink all the annexed data from the old repo
cp -rl ~/oldrepo/.git/annex/objects .git/annex/
# Regenerate the git annex metadata
git annex fsck --fast
# Also split the repo on the usb key
cd /media/usbkey
git clone ~/photos
cd photos
git annex init usbkey
cp -rl ../oldrepo/.git/annex/objects .git/annex/
git annex fsck --fast
# Connect the annexes as remotes of each other
git remote add laptop ~/photos
cd ~/photos
git remote add usbkey /media/usbkey
At this point, I went through all repos doing standard cleanup: 
# Remove unneeded hard links
git annex unused
git annex dropunused --force 1-12345
# Sync
git annex sync
To make sure nothing is missing, I used git annex find --not --in=here to see if, for example, the usbkey that should have everything could be missing some thing. Update: Antoine Beaupr pointed me to this tip about Repositories with large number of files which I will try next time one of my repositories grows enough to hit a performance issue.

Manuel A. Fernandez Montecelo: Debian GNU/Linux port for RISC-V 64-bit (riscv64)

This is a post describing my involvement with the Debian GNU/Linux port for RISC-V (unofficial and not endorsed by Debian at the moment) and announcing the availability of the repository (still very much WIP) with packages built for this architecture. If not interested in the story but you want to check the repository, just jump to the bottom. Roots A while ago, mostly during 2014, I was involved in the Debian port for OpenRISC (or1k) about which I posted (by coincidence) exactly 2 years ago. The two of us working on the port stopped in August or September of that year, after knowing that the copyright of the code to add support for this architecture in GCC would not be assigned to the FSF, so it would never be added to GCC upstream unless the original authors changed their mind (which they didn't) or there was a clean-room reimplementation (which didn't happen so far). But a few other key things contributed to the decision to stop working on that port, which bear direct relationship to this story. One thing that particularly influenced me to stop working on it was a sense of lack of purpose, all things considered, for the OpenRISC port that we were working on. For example, these chips are sometimes used as part of bigger devices by Samsung to control or wake up other chips; but it was not clear whether there would ever be devices with OpenRISC as the main chip, specially in devices powerful enough to run Linux or similar kernels, and Debian on top. One can use FPGAs to synthesise OpenRISC or1k, but these are slow, and expensive when using lots of memory. Without prospects of having hardware easily available to users, there's not much point in having a whole Debian port ready to run on hardware that never comes to be. Yeah, sure, it's fun to create such a port, but it's tons of work to maintain and keep it up to-date forever, and with close to zero users it's very unrewarding. Another thing that contributed to decide to stop is that, at least in my opinion, 32-bit was not future-proof enough for general purpose computing, specially for new devices and ports starting to take off on that time and age. There was some incipient work to create another OpenRISC design for 64-bits, but it was still in an early phase. My secret hope and ultimate goal was to be able to run as much a free-ish computer as possible as my main system. Still today many people are buying and using 32-bit devices, like small boards; but very few use it as their main computer or servers for demanding workloads or complex services. So for me, even if feasible if one is very austere and dedicated, OpenRISC or1k failed that test. And lastly, another thing happened at the time... Enter RISC-V In August 2014, at the point when we were fully acknowledging the problem of upstreaming (or rather, lack thereof) the support for OpenRISC in GCC, RISC-V was announced to the world, bringing along papers with suggestive titles such as Instruction Sets Should Be Free: The Case For RISC-V (pdf) and articles like RISC-V: An Open Standard for SoCs - The case for an open ISA in EE Times. RISC-V (as the previous RISC-n marks) had been designed (or rather, was being designed, because it was and is an as yet unfinished standard) by people from UC Berkeley, including David Patterson, the pioneer of RISC computer designs and co-author of the seminal book Computer Architecture: A Quantitative Approach . Other very capable people are also also leading the project, doing the design and legwork to make it happen see the list of contributors. But, apart from throwing names, the project has many other merits. Similarly to OpenRISC, RISC-V is an open instruction set architecture (ISA), but with the advantage of being designed in more recent times (thus avoiding some mistakes and optimising for problems discovered more recently, as technology evolves); with more resources; with support for instruction widths of 32, 64 and even 128 bits; with a clean set of standard but optional extensions (atomics, float, double, quad, vector, ...); and with reserved space to add new extensions in ordered and compatible ways. In the view of some people in the OpenRISC community, this unexpected development in a way made irrelevant the ongoing update of OpenRISC for 64-bits, and from what I know and for better or worse, all efforts on that front stopped. Also interestingly (if nothing else, for my purposes of running as much a free system as possible), was that the people behind RISC-V had strong intentions to make it useful to create modern hardware, and were pushing heavily towards it from the beginning. And together with this announcement or soonly after, it came the promise of free-ish hardware in the form of the lowRISC project. Although still today it seems to be a long way from actually shipping hardware, at least there was some prospect of getting it some day. On top of all that, about the freedom aspect, both the Berkeley and lowRISC teams engaged since very early on with the free hardware community, including attending and presenting at OpenRISC events; and lowRISC intended to have as much free hardware as possible in their planned SoC. Starting to work on the Debian port, this time for RISC-V So in late 2014 I slowly started to prepare the Debian port, switching on and off. The Userland spec was frozen in 2014 just before the announcement, the Privilege one is still not frozen today, so there was no need to rush. There were plans to upstream the support in the toolchain for RISC-V (GNU binutils, glibc and GCC; Linux and other useful software like Qemu) in 2015, but sadly these did not materialise until late 2016 and 2017. One of the main reasons for the delay was due to the slowness to sort out the copyright assignment of the code to the FSF (again). Still today, only binutils and GCC are upstreamed, and Linux and glibc depend on the Privilege spec being finished, so it will take a while. After the experience with OpenRISC and the support in GCC, I didn't want to invest too much time, lest it all became another dead-end due to lack of upstreaming so I was just cross-compiling here and there, testing Qemu (which still today is very limited for this architecture, e.g. no network support and very limited character and block devices) and trying to find and report bugs in the implementations, and send patches (although I did not contribute much in the end). Incompatible changes in the toolchain In terms of compiling packages and building-up a repository, things were complicate, and less mature and stable than the OpenRISC ones were even back in 2014. In theory, with the Userland spec being frozen, regular programs (below the Operating System level) compiled at any time could have run today; but in practice what happened is that there were several rounds of profound or, at least, disrupting changes in the toolchain before and while being upstreamed, which made the binary packages that I had built to not work at all (changes in dynamic loader, registers where arguments are stored when jumping functions, etc.). These major breakages happened several times already, and kind of unexpectedly at least for the people not heavily involved in the implementation. When the different pieces are upstreamed it is expected that these breakages won't happen; but still there's at least the fundamental bit of glibc, which will probably change things once again in incompatible ways before or while being upstreamed. Outside Debian but within the FOSS / Linux world, the main project that I know of is that some people from Fedora also started a port in mid 2016 and did great advances, but from what I know they put the project in the freezer in late 2016 until all such problems are resolved they don't want to spend time rebootstrapping again and again. What happened recently on the Debian front In early 2016 I created the page for RISC-V in the Debian wiki, expecting that things were at last fully stable and the important bits of the toolchain upstreamed during that year I was too optimistic. Some other people (including Debian folks) have been contributing for a while, in the wiki, mailing lists and IRC channels, and in the RISC-V project mailing lists you will see their names everywhere. However, due to the combination of lack of hardware, software not upstreamed and shortcomings of emulators (chiefly Qemu) make contributions hard and very tedious, nothing much happened recently visible to the outside world in terms of software. The private repository-in-the-making In late 2015 and beginning of 2016, having some free time in my hands and expecting that all things would coalesce quickly, I started to build a repository of binary packages in a more systematic way, with most of the basic software that one can expect in a basic Debian system (including things common to all Linux systems, and also specific Debian software like dpkg or apt, and even aptitude!). After that I also built many others outside the basic system (more than 1000 source packages and 2000 or 3000 arch-dependent binary packages in total), specially popular libraries (e.g. boost, gtk+ version 2 and 3), interpreters (several versions of lua, perl and python, also version 2 and 3) and in general packages that are needed to build many other packages (like doxygen). Unfortunately, some of these most interesting packages do not compile cleanly (more because of obscure or silly errors than proper porting), so they are not included at the moment. I intentionally avoided trying to compile thousands of packages in the archive which would be of nobody's use at this point; but many more could be compiled without much effort. About the how, initially I started cross-compiling and using rebootstrap, which was of great help in the beginning. Some of the packages that I cross-compiled had bugs that I did not know how to debug without a live and native (within emulators) system, so I tried to switch to natively built packages very early on. For that I needed many packages built natively (like doxygen or cmake) which would be unnecessary if I remained cross-compiling the host tools would be used in that case. But this also forced me to eat my own dog food, which even if much slower and tedious, it was on the whole a more instructive experience; and above all, it helped to test and make sure that the the tools and the whole stack was working well enough to build hundreds of packages. Why the repository-in-the-making was private Until now I did not attempt to make the repository available on-line, for several reasons. First because it would be kind of useless to publish files that were not working or would soon not work, due to the incompatible changes in the toolchain, rendering many or most of the packages built useless. And because, for many months now, I expected that things would stabilise and to have something stable really soon now but this didn't happen yet. Second because of lack of resources and time since mid 2016, and because I got some packages only compiled thanks to (mostly small and unimportant, but undocumented and unsaved) hacks, often working around temporary bugs and thus not worth sending upstream; but I couldn't share the binaries without sharing the full source and fulfill licenses like the GNU GPL. I did a new round of clean rebuilds in the last few weeks, just finished, the result is close to 1000 arch-dependent packages. And third, because of lack of demand. This changed in the last few weeks, when other people started to ask me to share the results even if incomplete or not working properly (I had one request in the past, but couldn't oblige in time at the time). Finally, the work so far: repository now on-line So finally, with the great help from Kurt Keville from MIT, and Bytemark sponsoring a machine where most of the packages were built, here we have the repository: The lines for /etc/apt/sources.list are: 
 deb [ arch=riscv64 signed-by=/usr/share/keyrings/debian-keyring.gpg ] http://riscv.mit.edu/debian unstable main
 deb-src [ signed-by=/usr/share/keyrings/debian-keyring.gpg ] http://riscv.mit.edu/debian unstable main
The repository is signed with my key as Debian Developer, contained in the file /usr/share/keyrings/debian-keyring.gpg, which is part of the package debian-keyring (available from Debian and derivatives). WARNING!! This repository, though, is very much WIP, incomplete (some package dependencies cannot be fulfilled, and it's only a small percentage of the Debian archive, not trying to be comprehensive at the moment) and probably does not work at all in your system at this point, for the following reasons: The combination of all these shortcomings is specially unfortunate, because without glibc provided it will be difficult that you can get the binaries to run at all; but there are some packages that are arch-dependent but not too tied to libc or the dynamic loader will not be affected. At least you can try one the few static packages present in Debian, like the one in the package bash-static. When one removes moving parts like the dynamic loader and libc, since the basic machine instructions are stable for several years now, it should work, but I wouldn't discard some dark magic that prevents even static binaries from working. Still, I hope that the respository in its current state is useful to some people, at least for those who requested it. If one has the environment set-up, it's easy to unpack the contents of the .deb files and try out the software (which often is not trivial or very slow to compile, or needs lots of dependencies to be built first first). ... and finally, even if not useful at all for most people at the moment, by doing this I also hope that efforts like this spark your interest to contribute to free software, free hardware, or both! :-)

21 April 2017

Joachim Breitner: veggies: Haskell code generation from scratch

How hard it is to write a compiler for Haskell Core? Not too hard, actually! I wish we had a formally verified compiler for Haskell, or at least for GHC s intermediate language Core. Now formalizing that part of GHC itself seems to be far out of reach, with the many phases the code goes through (Core to STG to CMM to Assembly or LLVM) and optimizations happening at all of these phases and the many complicated details to the highly tuned GHC runtime (pointer tagging, support for concurrency and garbage collection).

Introducing Veggies So to make that goal of a formally verified compiler more feasible, I set out and implemented code generation from GHC s intermediate language Core to LLVM IR, with simplicity as the main design driving factor. You can find the result in the GitHub repository of veggies (the name derives from verifiable GHC ). If you clone that and run ./boot.sh some-directory, you will find that you can use the program some-directory/bin/veggies just like like you would use ghc. It comes with the full base library, so your favorite variant of HelloWorld might just compile and run. As of now, the code generation handles all the Core constructs (which is easy when you simply ignore all the types). It supports a good number of primitive operations, including pointers and arrays I implement these as need and has support for FFI calls into C.

Why you don't want to use Veggies Since the code generator was written with simplicity in mind, performance of the resulting code is abysmal: Everything is boxed, i.e. represented as pointer to some heap-allocated data, including unboxed integer values and unboxed tuples. This is very uniform and simplifies the code, but it is also slow, and because there is no garbage collection (and probably never will be for this project), will fill up your memory quickly. Also, the code is currently only supports 64bit architectures, and this is hard-coded in many places. There is no support for concurrency.

Why it might be interesting to you nevertheless So if it is not really usable to run programs with, should you care about it? Probably not, but maybe you do for one of these reasons:
  • You always wondered how a compiler for Haskell actually works, and reading through a little over a thousands lines of code is less daunting than reading through the 34k lines of code that is GHC s backend.
  • You have wacky ideas about Code generation for Haskell that you want to experiment with.
  • You have wacky ideas about Haskell that require special support in the backend, and want to prototype that.
  • You want to see how I use the GHC API to provide a ghc-like experience. (I copied GHC s Main.hs and inserted a few hooks, an approach I copied from GHCJS).
  • You want to learn about running Haskell programs efficiently, and starting from veggies, you can implement all the trick of the trade yourself and enjoy observing the speed-ups you get.
  • You want to compile Haskell code to some weird platform that is supported by LLVM, but where you for some reason cannot run GHC s runtime. (Because there are no threads and no garbage collection, the code generated by veggies does not require a runtime system.)
  • You want to formally verify Haskell code generation. Note that the code generator targets the same AST for LLVM IR that the vellvm2 project uses, so eventually, veggies can become a verified arrow in the top right corner map of the DeepSpec project.
So feel free to play around with veggies, and report any issues you have on the GitHub repository.

Dirk Eddelbuettel: Rblpapi 0.3.6

Time for a new release of Rblpapi -- version 0.3.6 is now on CRAN. Rblpapi provides a direct interface between R and the Bloomberg Terminal via the C++ API provided by Bloomberg Labs (but note that a valid Bloomberg license and installation is required). This is the seventh release since the package first appeared on CRAN last year. This release brings a very nice new function lookupSecurity() contributed by Kevin Jin as well as a number of small fixes and enhancements. Details below:

Changes in Rblpapi version 0.3.6 (2017-04-20)
  • bdh can now store in double preventing overflow (Whit and John in #205 closing #163)
  • bdp documentation has another ovveride example
  • A new function lookupSecurity can search for securities, optionally filtered by yellow key (Kevin Jin and Dirk in #216 and #217 closing #215)
  • Added file init.c with calls to R_registerRoutines() and R_useDynamicSymbols(); also use .registration=TRUE in useDynLib in NAMESPACE (Dirk in #220)
  • getBars and getTicks can now return data.table objects (Dirk in #221)
  • bds has improved internal protect logic via Rcpp::Shield (Dirk in #222)

Courtesy of CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the Rblpapi page. Questions, comments etc should go to the issue tickets system at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

20 April 2017

Norbert Preining: TeX Live 2017 pretest started

Preparations for the release of TeX Live 2017 have started a few days ago with the freeze of updates in TeX Live 2016 and the announcement of the official start of the pretest period. That means that we invite people to test the new release and help fixing bugs. Notable changes are listed on the pretest page, I only want to report about the changes in the core infrastructure: changes in the user/sys mode of fmtutil and updmap, and introduction of the tlmgr shell. User/sys mode of fmtutil and updmap We (both at TeX Live and Debian) regularly got error reports about fonts not being found or formats not updated etc. The reason for all this was unmistakably the same: The user has called updmap or fmtutil without the -sys option, thus creating a copy of set of configuration files under his home directory, shadowing all later updates on the system side. Reason for this behavior is the wide-spread misinformation (outdated information) on the internet suggesting to call updmap. To counteract this, we have changed the behavior so that both updmap and fmtutil now accept a new argument -user (in addition to the already present -sys), and rejects to run when called without either of it given, giving a warning and linking to an explanation page. This page provides more detailed documentation, and best practice examples. tlmgr shell The TeX Live Manager got a new shell mode, invoked by tlmgr shell. Details need to be flashed out, but in principle it is possible to use get and set to query and set some of the options normally passed via command lines, and use all the actions as defined in the documentation. The advantage of this is that it is not necessary to load the tlpdb for each invocation. Here a short example: 
[~] tlmgr shell
protocol 1
tlmgr> load local
OK
tlmgr> load remote
tlmgr: package repository /home/norbert/public_html/tlpretest (verified)
OK
tlmgr> update --list
tlmgr: saving backups to /home/norbert/tl/2017/tlpkg/backups
update:   bchart             [147k]: local:    27496, source:    43928
...
update:   xindy              [535k]: local:    43873, source:    43934
OK
tlmgr> update --all
tlmgr: saving backups to /home/norbert/tl/2017/tlpkg/backups
[ 1/22, ??:??/??:??] update: bchart [147k] (27496 -> 43928) ... done
[ 2/22, 00:00/00:00] update: biber [1041k] (43873 -> 43910) ... done
...
[22/22, 00:50/00:50] update: xindy [535k] (43873 -> 43934) ... done
running mktexlsr ...
done running mktexlsr.
...
OK
tlmgr> quit
tlmgr: package log updated: /home/norbert/tl/2017/texmf-var/web2c/tlmgr.log 
[~]
Please test and report bugs to our mailing list. Thanks

19 April 2017

Steve Kemp: 3d-Printing is cool

I've heard about 3d-printing a lot in the past, although the hype seems to have mostly died down. My view has always been "That seems cool", coupled with "Everybody says making the models is very hard", and "the process itself is fiddly & time-consuming". I've been sporadically working on a project for a few months now which displays tram-departure times, this is part of my drive to "hardware" things with Arduino/ESP8266 devices . Most visitors to our flat have commented on it, at least once, and over time it has become gradually more and more user-friendly. Initially it was just a toy-project for myself, so everything was hard-coded in the source but over time that changed - which I mentioned here, (specifically the Access-point setup): I've now wired up an input-button to the device too, experimenting with the different ways that a single button can carry out multiple actions: Anyway the software is neat, and I can't think of anything obvious to change. So lets move onto the real topic of this post: 3D Printing. I randomly remembered that I'd heard about an online site holding 3D-models, and on a whim I searched for "4x20 LCD". That lead me to this design, which is exactly what I was looking for. Just like open-source software we're now living in a world where you can get open-source hardware! How cool is that? I had to trust the dimensions of the model, and obviously I was going to mount my new button into the box, rather than the knob shown. But having a model was great. I could download it, for free, and I could view it online at viewstl.com. But with a model obtained the next step was getting it printed. I found a bunch of commercial companies, here in Europe, who would print a model, and ship it to me, but when I uploaded the model they priced it at 90+. Too much. I'd almost lost interest when I stumbled across a site which provides a gateway into a series of individual/companies who will print things for you, on-demand: 3dhubs. Once again I uploaded my model, and this time I was able to select a guy in the same city as me. He printed my model for 1/3-1/4 of the price of the companies I'd found, and sent me fun pictures of the object while it was in the process of being printed. To recap I started like this:
Then I boxed it in cardboard which looked better than nothing, but still not terribly great:
Now I've found an online case-design for free, got it printed cheaply by a volunteer (feels like the wrong word, after-all I did pay him), and I have something which look significantly more professional:
Inside it looks as neat as you would expect:
Of course the case still cost 5 times as much as the actual hardware involved (button: 0.05, processor-board 2.00 and LCD I2C display 3.00). But I've gone from being somebody who had zero experience with hardware-based projects 4 months ago, to somebody who has built a project which is functional and "pretty". The internet really is a glorious thing. Using it for learning, and coding is good, using it for building actual physical parts too? That's something I never could have predicted a few years ago and I can see myself doing it more in the future. Sure the case is a little rough around the edges, but I suspect it is now only a matter of time until I learn how to design my own models. An obvious extension is to add a status-LED above the switch, for example. How hard can it be to add a new hole to a model? (Hell I could just drill it!)

Antoine Beaupr : The rise of Linux-based networking hardware

Linux usage in networking hardware has been on the rise for some time. During the latest Netdev conference held in Montreal this April, people talked seriously about Linux running on high end, "top of rack" (TOR) networking equipment. Those devices have long been the realm of proprietary hardware and software companies like Cisco or Juniper, but Linux seems to be making some significant headway into the domain. According to Shrijeet Mukherjee, VP of Engineering at Cumulus Networks: "we are seeing a 28% adoption rate in the Fortune 50" companies. As someone who has worked in system administration and networking for over a decade, I was surprised by this: switches have always been black boxes of proprietary hardware that we barely get a shell into. But as more and more TOR hardware gets Linux support, some of that work trickles down outside of that niche. Are we really seeing the rise of Linux in high-end networking hardware?

Linux as the standard interface During his keynote at Netdev, Jesse Brandeburg explained that traffic is exploding on the Internet: "From 2006 to 2016 the compound annual growth rate was 78% of network traffic. So the network traffic is growing like crazy. In 2010 to 2023, it's going to grow by a thousand times." He also mentioned Intel was working on devices that could do up to 400 Gbps. In his talk, he argued that Linux has a critical place in the modern world by repeating the mantra that "everything is on the network, and the network runs Linux", but does it really? Through Android, Linux has become the most popular end-user device operating system and is also dominant on the server but what about the actual fabric of the network: the routers and switches that interconnect all those machines? Mukherjee, in his own keynote, argued that even though Linux has achieved dominance in the server and virtualization markets, there is a "discontinuity [...] at the boundary where the host and the network meet" and argued "that the host without the network will not survive". In other words, proprietary hardware and software in the network threaten free software in the server. Even though some manufacturers are already providing a "Linux interface" in their hardware, it is often only some sort of compatibility shell which might be compared with the Ubuntu compatibility layer in Windows: it's not a real Linux. Mukherjee pushed this idea further by saying that those companies are limiting themselves by not using the full Linux stack. He presented Linux as the "top vehicle for innovation" that provides a featureful network stack, citing VXLAN, eBPF, and Quagga as prime features used on switches. Mukherjee also praised the diversity of the user-space Linux ecosystem as something that commercial alternatives can't rival; he compared the faster Linux development to the commercial sector where similar top features stay in the beta stage for up to 3 years. Because of its dominance in the server market, consumers are expecting a Linux-like interface on their networking gear now, which means Linux could be the standard interface all providers strive toward. As a Debian developer, I can't help but smile at the thought; if there's one thing we have not been able to do among Linux distributions, it's pretty much standardize user space in a consistent interface. POSIX is old and incomplete, the FHS is showing its age, and most distributions have abandoned LSB. Yet, the idea is certainly enticing: there is a base set of tools and applications, especially for the Linux kernel, that are standardized: iproute2, ethtool, and iptables are generally consistent across distributions, even though each distribution has its own way of using them. Yet Linux is not dominant, why? Mukherjee identified the problem as "packaging issues" and listed a set of features he would like Linux to improve:
  • Standardization of the ethtool interface. The idea is to make ethtool a de-facto standard to manage switches and ports. Mukherjee gave the example that data centers spend more money on cables than any other hardware and explained that making it easier to identify cables is therefore a key functionality. Getting consistent interface naming was also a key problem identified by numerous participants at the conference. While systemd tried to fix this with the predictable network interface names specification, the interface names are not necessarily predictable across virtual machines or on special hardware; in fact, this was the topic of the first talk of the conference. ethtool also needs to support interfaces that run faster than 1 Gbps, something that still has limited support in Linux at the moment.
  • Scaling of the Linux bridge. Through the rise of "software defined networking", we are likely to see multi-switch virtual environments that need to scale to hundreds of interfaces and devices easily. This is something the Linux bridge was never designed to do and it's showing scalability issues. During the conference, there was hope that the new XDP and eBPF developments could help, but also concerns this would create yet another bridge layer inside the kernel.
Cumulus's goal seems to be to establish Linux as the industry standard for this new era of networking and it is not alone. Through its Open Compute project, Facebook is sharing open designs of data center products and, while we have yet to see commercial off-the-shelf (COTS) 24 and 48-port gigabit switches trickle down to the consumer market, the company is definitely deploying new hardware based on those specifications in its own data centers, and those devices are often based on Linux.

The Linux switch implementation So how exactly do switches work in Linux? The Linux kernel manipulates switches with three different operation structures: switchdev_ops, which we previously covered, ethtool_ops, and netdev_ops. Certain switches, however, also need distributed switch architecture (DSA) features to be properly handled. DSA is a more obscure part of the network stack that allows Linux to represent hardware switches or chains of switches using regular Linux tools like bridge, ifconfig, and so on. While switchdev is a new layer, DSA has been in the kernel since 2.6.28 in 2008. Originally developed to support Marvell switches, DSA is now a generic layer deployed in WiFi access points, set-top boxes, on-board flight entertainment systems, trains, and other industrial equipment. Switches that have an Ethernet controller need DSA, whereas the kernel can support switches without Ethernet controllers directly with switchdev drivers. The first years of DSA's development consisted only of basic maintenance but, in the last three years, DSA has seen a resurgence of contributions, as part of Linux networking push to support hardware offloading and network switches. Between 2014 and 2015, DSA added support for Broadcom hardware, wake on LAN, and hardware port bridging, among other features. DSA's development was parallel to swconfig, written by the OpenWrt project to support the small office and home office (SOHO) routers that the project is targeting. The main difference between swconfig and DSA is that DSA-supported switches show one network interface per port, whereas swconfig-configured switches show up as a single port, which limits the amount of information that can be extracted from the switch. For example, you cannot have per-port traffic statistics with swconfig. That limitation is what led to the creation of the switchdev framework, when swconfig was proposed (then refused) for inclusion in mainline. Another goal of switchdev was to support bridge hardware offloading and network interface card (NIC) virtualization. Also, whereas swconfig uses virtual LAN (VLAN) tagging to address ports, DSA enables the use of device-specific tagging headers to address different ports, which enables DSA to have better control over the switches. This allows, for example, DSA to do internet group management protocol (IGMP) snooping or implement the spanning tree protocol, whereas swconfig doesn't have those features. Some switches are actually connected to the host CPU through an Ethernet interface instead of regular PCI-Express interface, and DSA supports this as well. One advantage that remains in the swconfig approach is that it treats the internal switch as a simple external switch, and addresses ports with standard VLAN tags. This is something DSA could do, as well, but no one has bothered implementing this just yet. For now, DSA drivers use device-specific tagging mechanisms that limit the number of supported devices. Other areas of future improvement for DSA are better multi-chip support, IGMP snooping, and bonding, as well as firewall, NAT, and TC offloading.

Where is the freedom? Given all those technical improvements, you might rightfully wonder if your own wireless router or data center switch runs Linux. In recent years, we have seen more and more networking devices shipped with Linux and sometimes even OpenWrt (e.g. in the case of the Turris Omnia, which we previously covered), and especially on SOHO routers, but it sometimes means a crippled operating system that only offers you a proprietary web interface. But at least those efforts make it easier to deploy free operating systems on those devices. Based on my experience running OpenWrt on wireless routers to build the Montreal mesh network, deploying Linux on routers and switches has always been a manual process. The Ubiquiti hardware being used in the mesh network comes with an OpenWrt derivative, but it includes proprietary drivers and a proprietary web interface. To use the mesh networking protocol that was chosen, it was necessary to deploy custom OpenWrt images by hand. For years, it was a continuous struggle for OpenWrt developers to liberate generation after generation of proprietary hardware with companies like Cisco locking down the venerable WRT platform in 2006 and the US Federal Communications Commission (FCC) rules that forced TP-Link to block free software on its routers, a change that was later reverted. Most hardware providers are obviously not dedicated to software freedom: deploying Linux on their hardware is for them an economic, not political choice. As you might expect, a "Linux router" these days often means a closed device and operating system, using Linux as the only free component. I had the privilege of doing some reverse engineering on the SmartRG SR603n VDSL modem, which also doubles as a WiFi router and VoIP phone adapter. I was able to get a shell on that machine, and what I found was a custom operating system built on top of the Linux kernel. I wrote a detailed report about this two years ago and the situation then was pretty grim. Another experience I had was working over a decade in data centers, which tells an even worse story: most switches and routers there are not running free software at all. We have deployed HP ProCurve switches that provide free (as in beer) software updates and have struggled for years to find free (as in speech) software alternatives for those. We built our own routers using COTS server hardware, at a significant performance cost over the dedicated application-specific integrated circuits (ASICs) built into commercial routers, which do not offer us the trust and flexibility we were looking for. But Linux is definitely making some headway, and has been for a while. When we covered switchdev in February 2016, it was just getting started, but now vendors like Mellanox, Broadcom, Cumulus, and Intel are writing and shipping code using the framework. Cumulus, in particular, is developing a Debian-based distribution (Cumulus Linux) that it deploys for clients on targeted hardware. Most of the hardware in that list, however, is not actually open in the more radical sense: they are devices that can run free software but are not generally open-source hardware. There are some exceptions, but they sit at the higher end of the spectrum: most organizations probably don't need a 100 Gbps ports, let alone the 128 ports in the Backpack switch that Cumulus is shipping. How much this translates to actual freedom for the end-user is therefore questionable. While I have seen encouraging progress on the high end of the hardware spectrum at Netdev, I'm not sure this will translate into immediate wins in the data center or even for home users in the short term. In the long term, however, we will hopefully see some progress in Linux's rise in general-purpose networking hardware following its dominance in general-purpose computing.
The author would like to thank the Netdev organizers for travel assistance. Also, thanks to Andrew Lunn for a technical review of this article. Note: this article first appeared in the Linux Weekly News.

18 April 2017

Steinar H. Gunderson: Chinese HDMI-to-SDI converters

I often need to convert signals from HDMI to SDI (and occasionally back). This requires a box of some sort, and eBay obliges; there's a bunch of different sellers of the same devices, selling them around $20 25. They don't seem to have a brand name, but they are invariably sold as 3G-SDI converters (meaning they should go up to 1080p60) and look like this: There are also corresponding SDI-to-HDMI converters that look pretty much the same except they convert the other way. (They're easy to confuse, but that's not a problem unique tothem.) I've used them for a while now, and there are pros and cons. They seem reliable enough, and they're 1/4th the price of e.g. Blackmagic's Micro converters, which is a real bargain. However, there are also some issues: The last issue is by far the worst, but it only affects 3G-SDI resolutions. 720p60, 1080p30 and 1080i60 all work fine. And to be fair, not even Blackmagic's own converters actually send 352M correctly most of the time I wish there were a way I could publish this somewhere people would actually read it before buying these things, but without a name, it's hard for people to find it. They're great value for money, and I wouldn't hesitate to recommend them for almost all use but then, there's that almost. :-)

Bits from Debian: Call for Proposals for DebConf17 Open Day

The DebConf team would like to call for proposals for the DebConf17 Open Day, a whole day dedicated to sessions about Debian and Free Software, and aimed at the general public. Open Day will preceed DebConf17 and will be held in Montreal, Canada, on August 5th 2017. DebConf Open Day will be a great opportunity for users, developers and people simply curious about our work to meet and learn about the Debian Project, Free Software in general and related topics. Submit your proposal We welcome submissions of workshops, presentations or any other activity which involves Debian and Free Software. Activities in both English and French are accepted. Here are some ideas about content we'd love to offer during Open Day. This list is not exhaustive, feel free to propose other ideas! To submit your proposal, please fill the form at https://debconf17.debconf.org/talks/new/ Volunteer We need volunteers to help ensure Open Day is a success! We are specifically looking for people familiar with the Debian installer to attend the Debian installfest, as resources for people seeking help to install Debian on their devices. If you're interested, please add your name to our wiki: https://wiki.debconf.org/wiki/DebConf17/OpenDay#Installfest Attend Participation to Open Day is free and no registration is required. The schedule for Open Day will be announced in June 2017. DebConf17 logo

Next.

Previous.