Search Results: "evo"

16 April 2013

Wouter Verhelst: The devops "installer"

puppet and similar things are very nice tools for managing your systems. Rather than doing the same thing 100 times, it's much more interesting to spend some time describing how to do something, and then have it done hundreds of times automatically. However, I'm starting to see a distressing trend lately. I'd like to say that puppet is not an installer, nor is it a replacement for one. Here's why: Please, pretty please, with sugar on top: just provide packages. If that can't be done, just provide clear installation instructions and/or a puppet module that I can include or something. Don't assume I'm a system administrator not worth his paycheck. Thanks. note: the gitorious example above is just that, an example. I've seen more cases of people doing similar things, not just gitorious. Surprisingly, it's mostly from people who're on the ruby kool-aid. I hope that's not related... Update: (2013-04-18) despite my usage of buzzwords in the title of this post, I did not mean to imply that any of the above has anything to do with devops. Instead, what the post is supposed to say is that you should not fall into the trap of believing that devops methods, which may work wonderfully internal to your organization, will therefore also work wonderfully for processes outside your organization.

12 April 2013

Daniel Pocock: Switzerland Schilthorn with 007 (leg 2)

What has DebConf13 got in common with James Bond? The connection with Switzerland of course. Bond creator Ian Flemming actually spent time living in Switzerland and this beautiful alpine country was featured in more than one of his books/movies, including Goldfinger On Her Majesty's Secret Service took Commander Bond to the summit of the Schilthorn, a distinctive peak of 2,970m altitude in the center of Switzerland. To follow up on my recent Glacier Express and Righ Bahn rack railway videos, here is a video of the second leg of the dramatic four-stage cable car journey to the top of Schilthorn: <video controls="" height="340" poster="http://upload.wikimedia.org/wikipedia/commons/thumb/0/03/Schilthorn_with_Bernese_Alps%2C_2012_August.jpg/800px-Schilthorn_with_Bernese_Alps%2C_2012_August.jpg" width="560">
<source src="http://video.danielpocock.com/VID_20121025_121324.m4v" type="video/mp4"></source>
<source src="http://video.danielpocock.com/VID_20121025_121324.webm" type="video/webm"></source>
</video> Download and watch later:
.m4v
.webm Some observations about Schilthorn visits:
  • There is a revolving restaurant at the top - to get a seat by the window, which is by far the best place to sit, ring and make a reservation the day before. Meal prices are no more extraordinary than any other Swiss restaurant, so it is worthwhile spending some time up there enjoying lunch and the views.
  • Jungfrau is higher - but from Schilthorn, you can see Jungfrau, which is a spectacular view, yet without paying the extraordinary price of a ticket for Jungfrau. The ride to Jungfrau (high altitude railway) is also slightly less spectacular, as much of it is in a tunnel through the rock, while the ride to Schilthorn is all cable car.
  • Almost all rail ticket passes give a 50% discount on the Schilthorn cable car.
  • The SBB sometimes offers an extra 30% discount on Schilthorn tickets, see the monthly railaway' offers, which can be combined with the 50% discount. This type of ticket can't be purchased in the same region, (e.g. within about 20km of Schilthorn)
  • An easy way to save another 50% on the ticket is to walk down from the top. If you do this, plan to start very early, but it is a spectacular walk (requires hiking boots and protective clothing, snowy/slippery at most times of the year)
  • All those discounts are cumulative: so if you use them all and walk down, you only pay 0.5 * 0.7 * 0.5 (which is 17.5%) of the full price.
The next leg of the cable car journey will appear soon...

11 April 2013

Martin Pitt: New fatrace released, Debian package coming

Paul Wise poked me this morning about uploading fatrace ( file access trace , see the original announcement for details) to Debian, thanks for the reminder! So I filed an Intent To Package, and will upload it in a few days, unless some discussion evolves. I also took the opportunity to do some modernization: The power-usage-report script now uses the current PowerTop 2.x instead of the old 1.13, uses Python 3 now, and includes the process device activity in the report. I released this as 0.5. The actual fatrace binary didn t change its behaviour, it just got some code optimizations; thanks to Yann Droneaud for those.

Russ Allbery: Review: Familiar

Review: Familiar, by J. Robert Lennon
Publisher: Greywolf
Copyright: 2012
ISBN: 1-55597-535-6
Format: Hardcover
Pages: 205
This is the first book of an experiment. I'm fairly well-read in science fiction and fantasy and increasingly well-read in non-fiction of interest (although there's always far more of that than I'll get to in a lifetime), but woefully unfamiliar with what's called "mainstream" literature. Under the principal that things people are excited about are probably exciting, I've wanted to read more and understand the appeal. Powell's, which I like to support anyway, has a very nice (albeit somewhat expensive) book club called Indiespensable, which sends its subscribers very nice editions of new works that Powell's thinks are interesting, with a special focus on independent publishers. So I signed up and hope to stick with it for at least a year. (The trick will be fitting these books in amongst my regular reading.) Familiar is the first feature selection I received. Elisa Brown is a mother with a dead son and a living one, a failing marriage, an affair, and a life that is, in short, falling apart. Then, while driving back from her annual pilgrimage from her son's grave, the world seems to twist and change. She finds herself dressed for business, wearing a nametag and apparently coming back from a work-related convention, driving a car that's entirely unfamiliar to her. When she gets home, everything else has changed too: her marriage seems to be on firmer ground, but based on rules she doesn't understand. She has a different job, different clothes, a different body in some subtle ways. And both of her sons are alive. I'm going to have to have a long argument with myself about where to (meaninglessly) categorize this on my review site, since in construction it is an alternate reality story and therefore a standard SF trope. Any SF reader is going to immediately assume Elisa has somehow been transported into an alternate reality with a key divergence point from her own. But that's not Lennon's focus. He stays ambiguous on the question of whether this is really happening or whether Elisa had some sort of nervous breakdown, and while some amount of investigation of the situation does take place, it's the sort of investigation that an average person with no access to special resources or scientific knowledge and a completely unbelievable story would be able to do: Internet conspiracy chatrooms and some rather dodgy characters. The focus is instead on Elisa's reaction to the situation, her choices about how to treat this new life, and on how she processes her complex emotions about her family and herself. I had profoundly mixed feelings about this book when I finished it, and revisiting it to review it, I still do. The writing is excellent: spare, evocative, and enjoyable to read. Lennon has a knack for subjective description of emotion and physical experience. The reader feels Elisa's deep discomfort with her changed body and her changed car, her swings between closed-off emotions and sudden emotional connection with a specific situation, and her struggle with the baffling question of how to come to terms with a whole new life. The part of the book from about the middle to nearly the end is excellent. Video games make an appearance and are handled surprisingly well. And when Elisa starts being blunt with people, I found myself both liking her and caring about what happens to her. On the other hand, Familiar also has some serious problems, and one of the biggest is the reaction I feared I'd have to mainstream literature: until Elisa started opening up and taking action, I found it extremely difficult to care about anyone in this book. They're all so profoundly petty, so closed off and engrossed in what seem like depressing and utterly boring lives. I'm sure that some of this is intentional and is there to lay the groundwork for Elisa's own self-discovery, but even towards the end of that self-discovery, everything here is so relentlessly middle-class suburbia that I felt stifled just reading about it. I think it's telling that no one in this book ever seems to have any substantial problem with money, or even with work. Elisa walks into a job that she's never done before and within a few weeks is doing it so well that she can take large amounts of time to wander around for plot purposes. This is a book about highly privileged people being miserable in a bubble. While those people certainly do exist, and I can believe that they act like this, I'm not sure how much I want to read about them. Thankfully, the plot does lead Elisa to poke some holes in that bubble, if never get out of it entirely. This is also another one of those stories in which every character has massive communication problems. Now, this deserves some caveats: Elisa's communication problems with her husband are part of the problem that starts the book and are clearly intentional, as are her communication difficulties with her children. And she's not really close enough to anyone to confide in them. But even with those caveats, no one in this book really talks to anyone else. It's amazing that anyone forms any connections at all, given how many walls and barriers they have around themselves. As someone with a bit of a thing for communication, this drove me nuts to read about, particularly in the first half of the book. But the worst problem is that Lennon completely blows the ending. And by that I don't just mean that I disliked the ending. I mean the ending is so unbelievable and so contrary to the entire rest of the book, at least the way I was reading and understanding it, that I think Familiar is a much better novel if you just remove the final scene entirely. It was such a bizarre and unnecessary twist that I found it infuriating. I don't want to spoil an ending, even a bad ending, so I'll only say this: it felt to me like Lennon just wasn't comfortable with his setting and plot driver and couldn't leave it alone. I think an experienced SF author wouldn't have made this mistake. There were two obvious possible conclusions to draw from the setting, plus a few interesting combinations, and I think someone comfortable with this sort of alternate reality story would have taken one of those options, any of which would have been a reasonable dismount for the plot. Alternately, they could have left it entirely ambiguous to the end and explored why the explanation may not actually matter. But Lennon seemed to me to have a tin ear for plausibility and for the normal flow of this sort of story and seems to have taken it as license for arbitrary events, thus completely violating the internal consistency and emergent rules that he'd spent the rest of the book building. I've mostly talked about my reactions to the characters and the writing and have not said much about the plot. That's somewhat intentional, since figuring out where the story will go is one of the best parts of this book. It's surprisingly tense and well-crafted for not having that much inherent dramatic tension. The excellent writing kept me reading through the first part, when I hated everyone in the story, and then Elisa started taking responsibility for her own life and actions and I started really enjoying the book while being constantly surprised. I think it's the sort of story that's best to take without too much foreknowledge of where it's going. I'm going to call this first experiment a qualfied success. Familiar was certainly interesting to read, and quite different from what I normally read despite the SF premise. If it weren't for the ending, I'd be recommending it to other people. Rating: 6 out of 10

8 April 2013

Matthias Klumpp: Tanglu status report

Hello everyone! I am very excited to report about the awesome progress we made with Tanglu, the new Debian-based Linux-Distribution. :) First of all, some off-topic info: I don t feel comfortable with posting too much Tanglu stuff to Planet-KDE, as this is often not KDE-related. So, in future Planet-KDE won t get Tanglu information, unless it is KDE-related ;-) You might want to take a look at Planet-Tanglu for (much) more information. So, what happened during the last weeks? Because I haven t had lectures, I nearly worked full-time on Tanglu, setting up most of the infrastructure we need. (this will change with the next week, where I have lectures again, and I also have work to do on other projects, not just Tanglu ^^) Also, we already have an awesome community of translators, designers and developers. Thanks to them, the Tanglu-Website is now translated to 6 languages, more are in the pipeline and will be merged later. Also, a new website based on the Django framework is in progress. The Logo-Contest We ve run a logo-contest, to find a new and official Tanglu logo, as the previous logo draft was too close to the Debian logo (I asked the trademark people at Debian). More than 30 valid votes (you had to be subscribed to a Tanglu Mailinglist) were received for 7 logo proposals, and we now have a final logo: Tanglu Logo (Text) I like it very much :) Fun with dak I decided to use dak, the Debian Archive Kit, to handle the Tanglu archive. Choosing dak over smaller and easy-to-use solutions had multiple reasons, the main reason is that dak is way more flexible than the smaller solution (like reprepro or min-dak) and able to handle the large archive of Tanglu. Also, dak is lightning fast. And I would have been involved with dak sooner or later anyway, because I will implement the DEP-11 extension to the Debian Archive later (making the archive application-friendly). duckWorking with dak is not exactly fun. The documentation is not that awesome, and dak contains many hardcoded stuff for Debian, e.g. it often expects the unstable suite to be present. Also, running dak on Debian Wheezy turned out to be a problem, as the Python module apt_pkg changed the API and dak naturally had problems with that. But with the great help of some Debian ftpmasters (many thanks to that!), dak is now working for Tanglu, managing the whole archive. There are still some quirks which need to be fixed, but the archive is in an usable state, accepting and integrating packages. The work on dak is also great for Debian: I resolved many issues with non-Debian dak installations, and made many parts of dak Wheezy-proof. Also, I added a few functions which might also be useful for Debian itself. All patches have of course been submitted to upstream-dak. ;-) Wanna-build and buildds GearThis is also nearly finished :) Wanna-build, the software which manages all buildds for an archive, is a bit complicated to use. I still have some issues with it, but it does it s job so far. (need to talk to the Debian wanna-build admins for help, e.g. wanna-build seems to be unable to handle arch:all-only packages, also, build logs are only submitted in parts) The status of Tanglu builds can be viewed at the provisoric Buildd-Status pages. Setting up a working buildd is also a tricky thing, it involves patching sbuild to escape bug #696841 and applying various workarounds to make the buildd work and upload packages correctly. I will write instructions how to set-up and maintain a buildd soon. At time, we have only one i386 buildd up and running, but more servers (in numbers: 3) are prepared and need to be turned into buildds. After working on Wanna-build and dak, I fully understand why Canonical developed Launchpad and it s Soyuz module for Ubuntu. But I think we might be able to achieve something similar, using just the tools Debian already uses (maybe a little less confortable than LP, but setting up an own LP instance would have been much more trouble). Debian archive import The import of packages from the Debian archive has finished. Importing the archive resulted in many issues and some odd findings (I didn t know that there are packages in the archive which didn t receive an upload since 2004!), but it has finally finished, and the archive is in a consistent state at time. To have a continuous package import from Debian while a distribution is in development, we need some changes to wanna-build, which will hopefully be possible. Online package search The Online-Package-Search is (after resolving many issues, who expected that? :P ) up and running. You can search for any package there. Some issues are remaining, e.g. the file-contents-listing doesn t work, and changelog support is broken, but the basic functionality is there. Tanglu Bugtracker We now also have a bugtracker which is based on the Trac software. The Tanglu-Bugtracker is automatically synced with the Tanglu archive, meaning that you find all packages in Trac to report bugs against them. The dak will automatically update new packages every day. Trac still needs a few confort-adjustments, e.g. submitting replies via email or tracking package versions. Tanglu base system tanglu-platform-smallThe Tanglu metapackages have been published in a first alpha version. We will support GNOME-3 and KDE4, as long as this is possible (= enough people working on the packaging). The Tanglu packages will also depend on systemd, which we will need in GNOME anyway, and which also allows some great new features in KDE. Side-effect of using systemd is at least for the start that Tanglu boots a bit slow, because we haven t done any systemd adjustments, and because systemd is very old. We will have to wait for the systemd and udev maintainers to merge the packages and release a new version first, before this will improve. (I don t want to do this downstream in Tanglu, because I don t know the plans for that at Debian (I only know the information Tollef Fog Heen & Co. provided at FOSDEM)) The community The community really surprised me! We got an incredible amount of great feedback on Tanglu, and most of the people liked the idea of Tanglu. I think we are one of the less-flamed new distributions ever started ;-) . Also, without the very active community, kickstarting Tanglu would not have been possible. My guess was that we might be able to have something running next year. Now, with the community help, I see a chance for a release in October :) The only thing people complained about was the name of the distribution. And to be really honest, I am not too happy with the name. But finding a name was an incredibe difficult process (finding something all parties liked), and Tanglu was a good compromise. Tanglu has absolutely no meaning, it was taken because it sounded interesting. The name was created by combining the Brazilian Tangerine (Clementine) and the German Iglu (igloo). I also dont think the name matters that much, and I am more interested in the system itself than the name of it. Also, companies produce a lot of incredibly weird names, Tanglu is relatively harmless compared to that ;-) . In general, thanks to everyone participating in Tanglu! You are driving the project forward! The first (planned) release I hereby announce the name of the first Tanglu release, 1.1 Aequorea Victoria . It is Daniel s fault that Tanglu releases will be named after jellyfishes, you can ask him why if you want ;-) I picked Aequorea, because this kind of jellyfish was particularly important for research in molecular biology. The GFP protein, a green fluorescent protein (=GFP), caused a small revolution in science and resulted in a Nobel Prize in 2008 for the researchers involved in GFP research (for the interested people: You can tag proteins with GFP and determine their position using light microscopy. GFP also made many other fancy new lab methods possible). Because Tanglu itself is more or less experimental at time, I found the connection to research just right for the very first release. We don t have a time yet when this version will be officially released, but I expect it to be in October, if the development speed increases a little and more developers get interested and work on it. Project Policy We will need to formalize the Tanglu project policy soon, both the technical and the social policies. In general, regarding free software and technical aspects, we strictly adhere to the Dbian Free Software Guidelines, the Debian Social Contract and the Debian Policy. Some extra stuff will be written later, please be patient! Tanglu OIN membership I was approached by the Open Invention Network, to join it as member. In general, I don t have objections to do that, because it will benefit Tanglu. However, the OIN has a very tolerant public stance on software patents, which I don t like that much. Debian did not join the OIN for this reason. For Tanglu, I think we could still join the OIN without someone thinking that we support the stance on software patents. Joining would simply be pragmatic: We support the OIN as a way to protect the Linux ecosystem from software patents, even if we don t like the stance on software patents and see it differently. Because this affects the whole Tanglu project, I don t want to decide this alone, but get some feedback from the Tanglu community before making a decision. Can I install Tanglu now? Yes and no. We don t provide installation images yet, so trying Tanglu is a difficult task (you need to install Debian and then upgrade it to Tanglu) if you want to experiment with it, I recomment trying Tanglu in a VM. I want to help! Great, then please catch one of us on IRC or subscribe to the mailinglists. The best thing is not to ask for work, but suggest something you want to do, others will then tell you if that is possible and maybe help with the task. Packages can for now only be uploaded by Debian Developers, Ubuntu Developers or Debian Maintainers who contacted me directly and whose keys have been verified. This will be changed later, but at the current state of the Tanglu archive (= less safety checks for packages), I only want people to upload stuff who definitely have the knowledge to create sane packages (you can also proove that otherwise, of course). We will later establish a new-member process. If you want to provide a Tanglu archive mirror, we would be very happy, so that the main server doesn t have to carry all the load. If you have experience in creating Linux Live-CDs or have worked with the Ubiquity installer, helping with these parts would be awesome! Unfortunately, we cannot reuse parts of Linux Mint Debian, because many of their packages don t build from source and are repackaged binaries, which is a no-go for the Tanglu main archive. Sneak peek And here is a screenshot of the very first Tanglu installation (currently more Debian than Tanglu):

tanglu_sneak-peek_screenshot Something else I am involved in Debian for a very long time now, first as Debian Maintainer and then as Debian Developer and I never thought much about the work the Debian system administrators do. I didn t know how dak worked or how Wanna-build handles the buildds and what exactly the ftpmasters have to do. By not knowing, I mean I knew the very basic theory and what these people do. But this is something different than experiencing how much work setting up and maintaining the infrastructure is and what an awesome job the people do for Debian, keeping it all up and running and secure! Kudos for that, to all people maintaining Debian infrastructure! You rock! (And I will never ever complain about slow buildds or packages which stay in NEW for too long ;-) )

5 April 2013

Clint Adams: And the men who hold high places

Antonia came into the room. Where's Alieta? she asked. We don't know, replied Luda. It was true; we hadn't seen Alieta in quite some time, and we had no idea where she was. Antonia left. Varvara eyed me mischievously and said, You could call her. How? I asked. Touch the grandmother clock and you will figure it out. I touched the clock trepidly. Nothing happened. I opened my mouth to speak, but the word I had been about to say resonated in my mind as if spoken through a comb filter. I concentrated. Alieta, dozens of human-like voices sang over and over in different harmonies. I let go. What was that? I demanded. Clock, Varvara said dismissively. Come on, let's do laundry, Luda grunted. We carried our laundry out to the garage, which is where the washer and dryer lived. Alieta slipped in through one of the open garage doors, and surreptitiously pulled Varvara over for a whispered conversation. Antonia wandered in and squinted in Alieta's general direction. Luda quickly pulled Varvara in front of her. It's just Varvara, she announced. Antonia's eyes glazed over and she went back into the house. Alieta pursed her lips and studied the wall as if there might be some reflective object that had alerted her mother to her presence. Then she grabbed Varvara by the hand, pulled her over to push the buttons that closed the garage doors, and dragged her into the house. I watched the garage doors close. Then I watched the one on the left reverse direction. I opened my mouth to notify someone of this, but the doors reversed direction again, and proceeded to partially open and close as if possessed. I tried to hide behind Luda, without success. Now she was watching as well, as reality seemed to to shimmer and bend. The roof became arched, and there were loud mechanical noises as our environs transformed. When it finished, the place where the garage doors had been was now a wall of screens displaying video as you might encounter in a television store. More disconcertingly, we were in front of a railing which overlooked what appeared to be a car dealership showroom. Luda sucked in her breath. Benjamin Snow, you were into some serious shit, she muttered.

29 March 2013

Dirk Eddelbuettel: R / Finance 2013 Open for Registration

The annoucement below just went to the R-SIG-Finance list. More information is as usual at the R / Finance page:
Now open for registrations: R / Finance 2013: Applied Finance with R
May 17 and 18, 2013
Chicago, IL, USA
The registration for R/Finance 2013 -- which will take place May 17 and 18 in Chicago -- is NOW OPEN! Building on the success of the previous conferences in 2009, 2010, 2011 and 2012, we expect more than 250 attendees from around the world. R users from industry, academia, and government will joining 30+ presenters covering all areas of finance with R. We are very excited about the four keynotes by Sanjiv Das, Attilio Meucci, Ryan Sheftel and Ruey Tsay. The main agenda (currently) includes seventeen full presentations and fifteen shorter "lightning talks". We are also excited to offer five optional pre-conference seminars on Friday morning. To celebrate the fifth year of the conference in style, the dinner will be held at The Terrace of the Trump Hotel. Overlooking the Chicago river and skyline, it is a perfect venue to continue conversations while dining and drinking. More details of the agenda are available at:
http://www.RinFinance.com/agenda/
Registration information is available at
http://www.RinFinance.com/register/
and can also be directly accessed by going to
http://www.regonline.com/RFinance2013
We would to thank our 2013 Sponsors for the continued support enabling us to host such an exciting conference:
International Center for Futures and Derivatives at UIC Revolution Analytics
MS-Computational Finance at University of Washington Google
lemnica
OpenGamma
OneMarketData
RStudio
On behalf of the committee and sponsors, we look forward to seeing you in Chicago!
Gib Bassett, Peter Carl, Dirk Eddelbuettel, Brian Peterson,
Dale Rosenthal, Jeffrey Ryan, Joshua Ulrich
See you in Chicago in May!!

Raphael Geissert: Chocolate quote

Kinda appropriate for some recent events:
Il y a autant de g n rosit recevoir qu' donner
- Julien Green

Roughly translated to
There is only as much generosity to receive as there is to give.

28 March 2013

John Goerzen: No Hate

God hates people that are I heard a sentence that began that way on an interview with a protestor outside the Supreme Court yesterday. It is a deeply sad, and deeply wrong, statement. If someone reads the Bible, and can come up with a word, any word, that completes that sentence, they re doing it wrong. If someone thinks that there is anyone God hates, then I have this to say: No. Just no. I saw an article today, taking pages and pages to assess what the Christian response to gay marriage should be. I don t need pages. It s very simple. It s this: God is the God of love. That is all. Where people are doing good, there is God. Where people care about each other, there is God. Where there are flowers blooming and trees shading and birds singing, there is God. Where people marvel in the beauty of the landscape or of another person, there is God. And where people love, there is God. There is too much hate in the world already. Instead of adding more, let s celebrate compassion, devotion, and peace. People that say that God is the God of hate look at the spring landscape and see only last year s thistles. One day soon, I hope to see everyone s hearts set free. What a day of joy that will be! And I hope, too, that those that hate will find the peace of freed hearts, freed from hate and from fear.

22 March 2013

MJ Ray: Wireless Networking on this Clevo

This Clevo laptop is a new machine and like a lot of new machines, not all of its hardware has drivers in the current stable release of debian. Happily, there is a driver for its rtl8723ae wireless networking device in the later 3.8 Linux kernel versions. So it s just a case of installing the package called kernel-package and following the instructions in it, to make a new linux-image package with the latest drivers in it. One small thing which tripped me up is that you usually need to write make-kpkg rootcmd fakeroot initrd kernel-image now. I forgot the initrd option at first.

7 March 2013

Vincent Sanders: The way to get started is to quit talking and begin doing

When Walt Disney said that he almost certainly did not have software developers in mind. However it is still good advice, especially if you have no experience with a piece of software you want to change.

Others have written extensively on the topic of software as more than engineering and the creative aspects, comparing it to a craft, which is a description I am personally most comfortable with. As with any craft though you have to understand the material you have to work with and an existing codebase is often a huge amount of material.

While developing NetSurf we get a lot of people who talk a lot about what we "ought" or "should" do to improve the browser but few who actually get off their collective posteriors and contribute. In fact according to the ohloh analysis of the revision control system there are roughly six of us who contribute significantly on a regular basis and of those only four who have interests beyond a specific frontend.

It has been mentioned by a couple of new people who have recently compiled the browser from source that it is somewhat challenging to get started. To address this criticism I intend to give a whirlwind introduction to getting started with the NetSurf codebase, perhaps we can even get some more contributors!

This first post will cover the mechanics of acquiring and building the source and the next will look at working with the codebase and the Netsurf community.

This is my personal blog, the other developers might disagree with my approach, which is why this is in my blog and not on the NetSurf website. That being said comments are enabled and I am sure they will correct anything I get wrong.
ResourcesNetSurf has a selection of resources which are useful to a new developer:
Build environmentThe first thing a new developer has to consider is their build environment. NetSurf supports nine frontends on several Operating Systems (OS) but is limited on the build environment that can be used.

The developer will require a Unix like system but let's be honest, we have not tried with anything other than Linux distributions in some time or MAC OS X for the cocoa frontend because its a special snowflake.

Traditionally at this point in this kind of introduction it would be traditional to provide the command line for various packaging systems to install the build environment and external libraries. We do have documentation that does this but no one reads it, or at least it feels like that. Instead we have chosen to provide a shell fragment that encodes all the bootstrap knowledge in one place, its kept in the revision control system so it can be updated.

To use: download it, read it (hey running random shell code is always a bad idea), source it into your environment and run ns-apt-get-install on a Debain based system or ns-yum-install on Fedora. The rest of this posting will assume the functionality of this script is available, if you want to do it the hard way please refer to the script for the relevant commands and locations.

For Example:
$ wget http://git.netsurf-browser.org/netsurf.git/plain/Docs/env.sh
$ less env.sh
$ source env.sh
$ ns-apt-get-install

Historically NetSurf built on more platforms natively but the effort to keep these build environments working was extensive and no one was prepared to do the necessary maintenance work. This is strictly a build setup decision and does not impact the supported platforms.

Since the last release NetSurf has moved to the git version control system from SVN. This has greatly improved our development process and allows for proper branching and merging we previously struggled to implement.

In addition to the core requirements external libraries NetSurf depends on will need to be installed. Native frontends where the compiled output is run on the same system it was built on are pretty straightforward in that the native package management system can be used to install the libraries for that system.

For cross building to the less common frontends we provide a toolchain repository which will build the entire cross toolchain and library set (we call this the SDK) direct from source. This is what the CI system uses to generate its output so is well tested.
External LibrariesNetSurf depends upon several external development libraries for image handling, network fetching etc. The libraries for the GTK frontend are installed by default if using the development script previously mentioned.

Generally a minimum of libcurl, libjpeg and libpng are necessary along with whatever libraries are required for the toolkit.
Project Source and Internal Libraries
One important feature of NetSurf is that a lot of functionality is split out into libraries. These are internal libraries and although technically separate projects, releases bundle them all together and for development we assume they will all be built together.

The development script provides the ns-clone function which clones all the project sources directly from their various git repositories. Once cloned the ns-make script can be used to build and install all the internal libraries into a local target ready for building the browser.

For Example:

$ source env.sh
$ ns-clone
$ ns-make-libs install
Frontend selectionAs I have mentioned NetSurf supports several windowing environments (toolkits if you like) however on some OS there is only one toolkit so the two get conflated together.

NetSurf currently has nine frontends to consider:
Building and running NetSurfFor a developer new to the project I recommend that the gtk version be built natively which is what I describe here.

Once the internal libraries have been installed, building NetSurf itself is as simple as running make.

For Example:
$ source env.sh
$ ns-make -C $ TARGET_WORKSPACE /$ NS_BROWSER TARGET=gtk

Though generally most developers would change into the netsurf source directory and run make there. The target (frontend) selection defaults to gtk on Linux systems so that can also be omitted Once the browser is built it can be run from the source tree to test.

For Example:
$ source env.sh
$ cd $ TARGET_WORKSPACE /$ NS_BROWSER
$ ns-make
$ ./nsgtk

The build can be configured by editing a Makefile.config file. An example Makefile.config.example can be copied into place and the configuration settings overridden as required. The default values can be found in Makefile.defaults which should not be edited directly.

Logging is enabled with the command line switch -v and user options can be specified on the command line, options on the command line will override those sourced from a users personal configuration, generally found in ~/.netsurf/Choices although this can be compile time configured.

2 March 2013

Russ Allbery: Review: Asimov's, June 2011

Review: Asimov's Science Fiction, June 2011
Editor: Sheila Williams
Issue: Volume 35, No. 6
ISSN: 1065-2698
Pages: 112
Sheila Williams's editorial is about a new ebook-only anthology of Asimov's stories, with some discussion of electronic sales. This sort of business editorial is what she's best at writing, although in this case I'm far enough in my reading that I'm reading it through a bit of a time machine. Silverberg's essay is about popular bubbles and is mildly interesting, although I've read quite a lot about this at greater length. I think he misses the plot at the end, though, where he brings in the 2008 collapse and works on focusing the blame on speculators rather than those selling the fraudulent investments. James Patrick Kelly has a surprisingly graceful introduction to fanfic that does a better job at conveying the underlying enthusiasm than most mainstream coverage. The review column this month is Peter Heck's workman-like survey of recently released books of interest. "The Cold Step Beyond" by Ian R. MacLeod: Most of MacLeod's writing I've read has focused on lush world-building, usually with a mix of magic and industrial revolution. This felt like a departure: a character study of a heavily-modified warrior and monster hunter against a clearly science-fictional background. The protagonist has no memory of her past beyond her training in a religious institution, and is as much machine as woman. She is on one of her first missions to hunt dangerous monsters. But she's given almost no direction, and rather than encountering a monster, she encounters a feral woman with an ancient ray gun. MacLeod has a very SFnal twist waiting in the wings, but what I liked best were the character dynamics and the interplay of duty and self-determination. (8) "All the News That's Fit" by Carol Emshwiller: This story follows Emshwiller's normal pattern of telling the story of an outsider and representative of outsider culture trying to understand a society, but her protagonist here is both more alone and more proactive than her average protagonist. She lives in a distant mountain village only rarely visited by a traveling reporter who brings news from outside. When he doesn't come, she goes out looking for him, at first to rescue him if he ran into trouble and later to find the news herself. Much of the story has the reader worrying nervously for her as she observes the large city while clearly having no idea how to live there. The ending provides an interesting and (at least for me) unexpected twist on the rural person coming to the big city, but it was marred by a level of anger from the protagonist that I never thought was well-motivated. (6) "Walking Stick Fires" by Alan DeNiro: There is a science fiction subgenre that, for lack of a better term, I'll call post-apocalyptic surrealism. This is an entry in that strange subgenre, featuring two alien bikers having mostly unmotivated adventures with strange objects and even stranger creatures on an Earth that's apparently been invaded and mostly devastated by multiple parties. Like a lot of post-apocalyptic surrealism, it never makes much sense. It reminded me of Rucker but with a bit more characterization and less hand-waving math. (4) "Apocalypse Daily" by Felicity Shoulders: I really liked the premise for this story. The protagonist is a concept designer for a social-oriented MMORPG, but one where only scores and resources carry over rather than the whole world. The goal of the game is survival: the world ends regularly, globally or locally, in a huge variety of ways, and one scores points and earns resources for further iterations by finding ways to survive that catastrophe. The game is structured to encourage cooperation, at least at first. But Katrina gets into a mildly paranoid workplace conflict with someone else on her team, leading to some significant changes in the underlying expectations of the game. Unfortunately, while the background and idea are quite solid, the direction the story goes is predictable pessimism, the conclusion doesn't say much that I found interesting, and I ended up detesting the protagonist. I would have preferred a more unique psychological twist. (6) "The Fighter" by Colin P. Davies: This is a rather short story, essentially an idea study in the context of a police stop of a futuristic entertainment fighter. This is another one of those depressing takes on a common SF trope. I didn't entirely follow the world-building, but it was all dark and pointless enough that I didn't really want to dig further. (4) "Kiss Me Twice" by Mary Robinette Kowal: I've read a few police procedurals featuring AIs before, but this is by far the best one I've read. Huang is a detective, and like all the detectives he makes extensive use of the police department AI. Unlike most of the detectives, though, he treats her as a full partner and close to human, and she reciprocates with a Mae West persona and a closer relationship with him than most of the department. They're in the middle of investigating a murder when there is a raid on the police department and the AI herself is stolen. The rest of the story entangles their investigation with the theft of the AI. The police AI is, of course, restored from backup, but that's the first time that's ever happened to her, and she's shaken by it, and more disturbed by the knowledge that there are two copies of her running. The mystery angle of the story is reasonably satisfying, but the AI characterization and Huang's careful feeling-out of their relationship and appropriate behavior on his part is the best part of the story. This is a great story: long, satisfying, and thoroughly excellent. I'd happily read more fiction in this universe. (8) Rating: 7 out of 10

27 February 2013

Matthew Garrett: Supporting third-party keys in a Secure Boot world

It's fairly straightforward to boot a UEFI Secure Boot system using something like Shim or the Linux Foundation's loader, and for distributions using either the LF loader or the generic version of Shim that's pretty much all you need to care about. The physically-present end user has had to explicitly install new keys or hashes, and that means that you no longer need to care about Microsoft's security policies or (assuming there's no exploitable flaws in the bootloader itself) fear any kind of revocation.

But what about if you're a distribution that cares about booting without the user having to install keys? There's several reasons to want that (convenience for naive users, ability to netboot, that kind of thing), but it has the downside that your system can now be used as an attack vector against other operating systems. Do you care about that? It depends how you weigh the risks. First, someone would have to use your system to attack another. Second, Microsoft would have to care enough to revoke your signature. The first hasn't happened yet, so we have no real idea how likely the second is. However, it doesn't seem awfully unlikely that Microsoft would be willing to revoke a distribution signature if that distribution were being used to attack Windows.

How do you avoid that scenario? There's various bits of security work you need to do, but one of them is to require that all your kernel modules be signed. That's easy for the modules in the distribution, since you just sign them all before shipping them. But how about third party modules? There's three main options here:

  1. Don't support third party modules on Secure Boot systems
  2. Have the distribution sign the modules
  3. Have the vendor sign the modules

The first option is easy, but not likely to please users. Or hardware vendors. Not ideal.

The second option is irritating for a bunch of reasons, and a pretty significant one is license-related. If you sign a module, does that mean you're endorsing it in some way? Does signing the nvidia driver mean that you think there's no license concerns? Even ignoring that, how do you decide whose drivers to sign? We can probably assume that companies like AMD and nvidia are fairly reputable, but how about Honest John's Driver Emporium? Verifying someone's identity is astonishingly expensive to do a good job of yourself, and not hugely cheaper if you farm it out to a third party. It's also irritating for the driver vendor, who needs a separate signature for every distribution they support. So, while possible, this isn't an attractive solution.

The third option pushes the responsibility out to other people, and it's always nice to get other people to do work instead of you. The problem then is deciding whose keys you trust. You can push that off to the user, but it's not the friendliest solution. The alternative is to trust any keys that are signed with a trusted key. But what is a trusted key? Having the distribution sign keys just pushes us back to option (2) - you need to verify everyone's identity, and they need a separate signing key for every distribution they support. In an ideal world, there'd be a key that we already trust and which is owned by someone willing to sign things with it.

The good news is that such a key exists. The bad news is that it's owned by Microsoft.

The recent discussion on LKML was about a patchset that allowed the kernel to install new keys if they were inside a PE/COFF binary signed by a trusted key. It's worth emphasising that this patchset doesn't change the set of keys that the kernel trusts - the kernel trusts keys that are installed in your system firmware, so if your system firmware trusts the Microsoft key then your kernel already trusts the Microsoft key. The reasoning here is pretty straightforward. If your firmware trusts things signed by Microsoft, and if a bad person can get things signed by Microsoft, the bad person can already give you a package containing a backdoored bootloader. Letting them sign kernel modules doesn't alter the power they already have over your system. Microsoft will sign PE/COFF binaries, so a vendor would just have to sign up with Microsoft, pay $99 to Symantec to get their ID verified, wrap their key in a PE/COFF binary and then get it signed by Microsoft. The kernel would see that this object was signed by a trusted key and extract and install the key.

Linus is, to put it mildly, unenthusiastic about this idea. It adds some extra complexity to the kernel in the form of a binary parser that would only be used to load keys from userspace, and the kernel already has an interface for that in the form of X509 certificates. The problem we have is that Microsoft won't sign X509 certificates, and there's no way to turn a PE/COFF signature into an X509 signature. Someone would have to re-sign the keys, which starts getting us back to option (2). One way around this would be to have an automated service that accepts PE/COFF objects, verifies that they're signed by Microsoft, extracts the key, re-signs it with a new private key and spits out an X509 certificate. That avoids having to add any new code to the kernel, but it means that there would have to be someone to run that service and it means that their public key would have to be trusted by the kernel by default.

Who would that third party be? The logical choice might be the Linux Foundation, but since we have members of the Linux Foundation Technical Advisory Board saying that they think module signing is unnecessary and that there's no real risk of revocation, it doesn't seem likely that they'll be enthusiastic. A distribution could do it, but there'd be arguments about putting one distribution in a more privileged position than others. So far, nobody's stood up to do this.

A possible outcome is that the distributions who care about signed modules will all just carry this patchset anyway, and the ones who don't won't. That's probably going to be interpreted by many as giving too much responsibility to Microsoft, but it's worth emphasising that these patches change nothing in that respect - if your firmware trusts Microsoft, you already trust Microsoft. If your firmware doesn't trust Microsoft, these patches will not cause your kernel to trust Microsoft. If you've set up your own chain of trust instead, anything signed by Microsoft will be rejected.

What's next? It wouldn't surprise me too much if nothing happens until someone demonstrates how to use a signed Linux system to attack Windows. Microsoft's response to that will probably determine whether anyone ends up caring.

comment count unavailable comments

24 February 2013

Gunnar Wolf: International Open Data Day - #OpenData / #DatosAbiertos

I just got this message through my University, and the least I can do (given I'm still, although barely, in time) is to repost it here, hoping it helps to spread the activity we have on this regard in Latin America:
Saturday, February 23 is the International Day of Open Data. Following its policy of promoting free, open and unrestricted access to the results of research funded with public money, CLACSO calls research centers and individual researchers to free their public data so they are available for other researchers and, most importantly, for the community as a whole. That's the reason CLACSO invites researchers and institutions to announce in social networks, mentioning "#DatosAbiertos #OpenData", which are their freed public documents, pointing to the web pages where they can be found. The International Open Data Day is an effort to:
  • Spread the concept of Open Data
  • Debate on the why of Open Data
  • Publish analysis done using open public data
  • Find out how more local and national (not reserved) open data can be published (i.e. Brazil)
  • Call towards using open data in research
  • Find which applications have been developed to handle and visualize open data
  • Organize events for the International Open Data Day (i.e. Monterrey
  • Call towards adding data catalogs in Latin America and the Caribbeanin the Data catalogs directory, in Datacatalogs.org, and in Open data census.
For further information on how to spread research and/or archives in digital repositories and in the CLACSO Digital Repository, please mail biblioteca@clacso.edu.ar We can between all contribute for the Latin American and Caribbean open data community to grow, democratizing access to public data about our societies.
So, what do I consider worthy of adding to a list of resources I can point to?

19 February 2013

Roland Mas: A challenge for whoever feels they have too much free time

Open question to enthusiasts, theoretical computing scientists and mathematicians of all sorts: is it possible to construct a valid QR-code that leads to interesting results when used as an initial configuration for the Game of Life? The rules: For Science! Update: The answer seems to be yes. Jurij Smakov assembled a QR-code generator and a Life engine and plugged them together for easy experimenting. And Stefano Zacchiroli noticed that using "free software" (no quotes) as the input leads to a couple of gliders endlessly traveling a field with a few still lifes. This is way beyond awesome.

18 February 2013

Dirk Eddelbuettel: New Rcpp master class scheduled for New York

A new Rcpp master class is scheduled for March 9 in New York. The format will an updated version of the one-day workshops I have given at the University of Rochester in 2010, in San Franciso in 2011 (organised by Revolution Analytics) and at the UseR! conference in 2012. The style will be hands-on, with numerous concrete examples and solid coverage of most aspects of Rcpp and related packages. As before, about six hours of instruction, split into four sessions of around ninety minutes focussing (loosely) on motivation/intro, core parts, extensions and applications. This should leave ample time for informal discussions and Q+A---as well as for lunch and coffee breaks---for a total of eight hours in the classroom. This is being put together in New York with the help of Jared Lander, and we will have some technical assistance from RStudio in order to use their EC2 farm for exercises with Rcpp. Registrations details are available here; information about other Rcpp events is also available. Feel free to contact me or Jared at our usual email addresses with questions.

14 February 2013

Ian Campbell: FOSDEM 2013

So as previously mentioned I gave a talk at FOSDEM this year on the future of paravirtualisation under Xen. I thought the talk was reasonably well attended despite being at six o'clock in the evening. In the end I underran my half hour slot but fortunately people had plenty of interesting questions! I've posted my slides on my xenbits space. I spent a fair chunk of the rest of the time on the Xen.org booth, which is hard work but it is always nice to speak to users and other developers. I also managed to catch up with various friends from other projects and managed to meet a few folks who I've only ever met in email before. I didn't get to see as many talks as I wanted (for example I missed pretty much all of the ARM track :-() due to clashes with my stints on the booth, my own talk, and talks clashing with each other. Still it was all in all a worthwhile, hopefully I'll be able to make it again next year! Next up: LinaroConnect in Hong Kong!

8 February 2013

MJ Ray: Clevo 7872-9040/A built Jan 13 with Debian 6

My trusty Asus seems to have succumbed to graphic fault. I got an OS-free Zoostorm as its replacement, to avoid paying the MS tax. Zoostorm is one brand that Clevo laptops are sold under. It was actually a Clevo 7872-9040/A built Jan 13. I installed Debian 6 on it. The download button was easy to spot on the front page, but I actually used mini.iso so I could use a smaller usb stick. The first larger stick I tried was a dud and I m not sure where other sticks went in the move. The base installation went fine and most things went well, but the wireless networking and sound required an upgrade, but more on that next tech post.

6 February 2013

Thomas Goirand: Openstack Folsom fully uploaded to Experimental

It s been a long time I wanted to blog about my recent work on the Openstack packaging. I finally can find a bit of time to do it. For those who don t know yet Openstack, it s a fairly recent (less than 3 years) cloud computing suite, which is becoming quite huge. If you plan on deploying a private cloud, you should definitively have a look into it. When the Openstack project started, I immediately was interested. I started packaging the Cactus release (that is, the 3rd version of Openstack), created the Alioth group (at the exact moment when Alioth was migrated to new hardware which added some fun so lucky I was!), and began to work on the Debian version (Openstack used to be an Ubuntu only project). After some success in integrating some Debian specific patches into the Ubuntu packages, I left it a bit aside, and skipped completely the Diablo release (which was never uploaded in Debian). Then, I worked a bit on the Essex release before the freeze of Wheezy last June, together with other DDs (big up to Julien, Loic, Ghe, Sileth!). I (re-)started serious packaging work for Openstack just right after Folsom (eg: version 2012.2) was released early last October. I literally worked days and nights on it, in order to provide more automation so that it could become more easy to install. Indeed, it used to be very complicated and painful, with lots of manual tasks to perform on the shell. It s hopefully a lot easier now, with most of the manual boring shell work replaced by debconf and scripts. But I m sure there s more that can be done still. After 4 months of effort, I finally pressed the red button and uploaded everything at once to Debian Experimental since Openstack Essex (version 2012.1) is in Wheezy, I can t of course upload to SID unless Wheezy is out -. This represent 32 sources packages in total (some of them are already uploaded and approved by the FTP masters: sorry to give you so much more work guys ), and 104 binary packages (and counting ). So this isn t exactly small And I already have some fixes for what s currently waiting in the NEW queue: CVE fixes, missing build-depends which we ve found using the Jenkins server of eNovance who sponsors the packaging work, etc. I probably will post again here to announce when Folsom is completely approved by the FTP masters and reaches Debian experimental. In the mean while, it s available on GPLHost non-official repositories (see the howto linked bellow). If you would like to test the latest Openstack release (called Folsom, or 2012.2, if you are following well ), you can read the quite verbose howto I wrote here: https://wiki.debian.org/OpenStackHowto/Folsom If you do test it (don t be afraid, it s not that hard ), I will be very happy to hear from you, and receive any feedback / critics you may have. Note that I would strongly recommend to use the Folsom release, rather than what is currently in Wheezy, for many reasons which would be too long to list in this blog post (let me still drop a few reasons here: less bugs, more automations and it s more easy to install). Now, Openstack is a constantly evolving software, with so many companies and developers involved. I don t think my work on this beast will ever be finished. But it doesn t mater, as it s been quite some fun, and I will enjoy to do more. The next release of Openstack is schedule for next April (they release at the same time as Ubuntu). I hope to be able to join the Portland summit, and see all the persons I have worked with over IRC and mailing list. This time, I hope to be able to release Openstack packages at the same time as the upstream source code is released (Debian packages have historically been lagging a few months behind). The pre-version is by the way already in Alioth. Last, a (private) message for our famous cheese-master: please wait until everything leaves the NEW queue before bugging everyone with translations. We probably should have a serious talk about how to make it less redundant, easier to translate, and probably find a way to avoid duplication of messages across all packages.

5 February 2013

Dirk Eddelbuettel: New Rcpp page on upcoming events -- including Master Class in New York

Lots of exciting things are happening with and around Rcpp. I just added a new page about Upcoming Events to the recently-created Rcpp site. This events page has lots to cover: an upcoming talk at Columbia on March 8 (details still TBD), a day-long workshop in New York on March 9, a possible participation at a CERN / ROOT conference in Switzerland on May 11-14, an upcoming talk in May in Milwaukee, and last but not least the tutorial by Romain and Hadley at UseR! 2013 in Spain. Phew! With that, a few quick words about the upcoming master class in New York. It will be a full day, covering an introduction and motivation, details about the core data types, tools for working with and and extending Rcpp and of course applications galore, including RcppArmadillo and RInside. I have done the same one day class format a few times before, most recently (with Revolution Analytics) in San Francisco in late 2011, and also as a two-part seminar at UseR! 2012. This time, we plan on providing cloud-hosted RStudio instances for participants. Better still, RStudio's own JJ Allaire will be on deck as well for RStudio --- and Rcpp Attributes --- questions. Details and registration information for the New York class are at this page.

Next.

Previous.