Search Results: "cas"

25 June 2012

Raphaël Hertzog: Test Driven Development with CppUTest, now in Debian

I have recently read Test Driven Development with Embedded C by James W. Grenning and published by Pragmatic Programmers. I really enjoyed the book: while I was aware of the huge benefits of having a comprehensive test suite, I never studied seriously the principles behind Test Driven Development (TDD) and this book makes a good introduction to the topic. At the same time it focuses on the C language and contains lots of examples on how you can create tests even for projects which have to interact with hardware or other unpredictable components (the key is to create many abstractions) using all the possibilities that C offers. The author convincingly argues that developing code with TDD forces you to create a modular design that is easier to evolve when the underlying requirements change. He also highlights how the tests serve as reference documentation of the API. James W. Grenning recommends CppUTest as his xUnit test framework of choice. When I wanted to try this test framework, I discovered that it was not available in Debian. I decided to package it because it has some interesting features not offered by the contenders (at least not to my knowledge). It s now available in Debian and in Ubuntu. First, it doesn t require any explicit registration of tests and has a very lightweight syntax. The small downside is that CppUTest requires the usage of C++ for the tests. But C++ is compatible with C so it doesn t matter much if you have a C++ compiler for your target. On the contrary, usage of variables and methods scoped to the test group makes it easy to write clear tests. Here s a short sample of test code:
extern "C"  
#include "timer.h"
#include "timefn.h"
 
 
#include "CppUTest/TestHarness.h"
 
static Time the_time;
static const int start_sec = 123;
static const int start_nsec = 456789000;
static const int delay_sec = 8;
static const int delay_nsec = 111111000; // start_nsec + delay_nsec < 10^9
 
TEST_GROUP(Timer)
 
    /* Class variables available to all tests in the group */
    Timer timer;
    Delay remaining;
 
    /* Standard setup/teardown methods of xUnit tests */
    void setup()  
        timer = timer_new();
        time_set(&the_time, start_sec, start_nsec);
        /* [...] */
     
 
    void teardown()  
        timer_free(timer);
        /* [...] */
     
 
    /* Helper functions specific to the test group */
    void start_timer_with_delay(long sec, long nsec)
     
        timer_set_real_delay(timer, sec, nsec);
        timer_start(timer);
     
 
    void ensure_remaining_is(long sec, long nsec)
     
        CHECK_EQUAL(sec, delay_get_seconds(remaining));
        CHECK_EQUAL(nsec, delay_get_nanoseconds(remaining));
     
 ;
 
TEST(Timer, NewIsNotStarted)
 
    CHECK(!timer->started);
 
/* [...] */
TEST(Timer, GetRemainingTimeWithNanosecondPrecision_ShiftOfSeconds)
 
    start_timer_with_delay(delay_sec, delay_nsec);
    time_set(&the_time, start_sec + delay_sec - 5, start_nsec + delay_nsec + 1000);
 
    remaining = timer_get_remaining_time(timer);
 
    ensure_remaining_is(4, 999999000);
 
To run those tests, you just need this boilerplate code in a main.cpp:
#include "CppUTest/CommandLineTestRunner.h"
 
int main(int argc, char** argv)
 
   return CommandLineTestRunner::RunAllTests(argc, argv);
 
Another interesting feature is its integrated memory leak detection system. Any test that hasn t released allocated memory at the end of the teardown process will be marked as failed. The upstream developers have made some unusual choices (static library only, installation in a private directory) but this will likely change with the switch to an automake and autoconf-based build system. I have reported the oddities that I found and I requested them to provide a pkg-config file to make it easier to compile and link unit tests exploiting CppUTest. I already used CppUTest to develop a small application running on an embedded Linux. At some point, I might try to use CppUTest for dpkg development. I believe that it makes for a good fit. dpkg is already C++ ready since dselect is written in C++ and reuses a good part of dpkg s code. In any case, if you like Test Driven Development and are writing C or C++ based applications, I invite you to try CppUTest.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Robert Collins: Running juju against a private openstack instance.

My laptop has somewhat less than 1/2 the grunt of my desktop at home, but I prefer to work on it as I can go sit in the sun etc, very hard to do that with a mini tower case :) However, running everything through ssh to another machine makes editing and iterating more clumsy; I need to do agent forwarding etc not terribly hard, but not free either, particularly when I travel, I need to remember to sync my source trees back to my laptop. So I prefer to live on my laptop and use my desktop for compute power. I had a couple of Juju charms I wanted to investigate, but I needed enough compute power to make my laptop really quite warm so I thought, its time to update my local cloud provider from Eucalyptus to Openstack. This was easy enough, until I came to run Juju. Turns out that Juju s commands really want to talk to the public DNS name of the instance (in order to SSH tunnel a connection to Zookeeper). But! Openstack returns DNS names like Server-3 , and if you think about a home network, its fairly rare to have a local DNS server *anyway*, so putting a suffix on names like that won t help at all: you either need to use a DNS naming provider (openstack ships with an LDAP provider, which adds even more complexity), and configure your clients to know how to find it, or you need to use the public IP addresses (which default to the FlatNetwork, which is routable within a home LAN by simply adding a route to 10.0.0.0/8 to your wifi interface). Adding to confusion, some wifi routers fail to forward avahi messages, which is a) terrible and b) breaks the only obvious way of doing no-config local DNS :( . So, I did some yak shaving this morning. Turns out other folk have already run into this and filed a Juju bug and a supporting txaws bug. The txaws bug was fixed, but just missed the release of Precise. Clint Byrum is going to SRU it this week though, so we ll have it soon. I ve put a patch up to address the Juju side, which is now pending review. Running the two together works very happily for me. \o/

22 June 2012

Michael Casadevall: Announcement of Calxeda Highbank Images for Quantal

Hello all,

As many of you are aware, Canonical, in coordination with Calxeda and others have been working to bring Ubuntu to this new class of high-performance cluster yet low power-consumption computers built around ARM processors. Many of you who were in attendance at UDS in Oakland may remember seeing Calxeda's talks and demonstration live, and the exciting news that this represents. The full presentation is available here.

In line of this work, I'm am extremely pleased to announce that the initial images for the Calxeda Highbank platform are now available for download, with installation instructions available here. Please remember that Quantal is still in alpha development, and is not currently recommended for use in a production environment. As development of 12.10 continues, we will continue to refine these images, and our tools to fully embrace MAAS on ARM, and make 12.10 to be our best release yet.


As an additional note, Highbank support for Ubuntu 12.04 LTS will be released as part of the 12.04.1 update in mid-August and will join our support for ArmadaXP from Marvell, which was released as part of 12.04.

---
Michael Casadevall
ARM Server Tech Lead
Professional Engineering and Services, Canonical
michael.casadevall@canonical.com

21 June 2012

John Goerzen: I introduced my 5-year-old and 2-year-old to startx and xmonad. They re DELIGHTED!

Two years ago, Jacob (then 3) and I built his first computer together. I installed Debian on it, but never put a GUI on the thing. It s command-line, and has provided lots of enjoyment off and on over the last couple of years. I ve written extensively about what our boys like to do, and the delight they have at learning things on the command line. The looks of shock I get from people when I explain, as if it s perfectly natural, that my child has been able to log in by himself to a Linux shell since age 3, are amusing and astounding. Especially considering that it is really not that hard. Instead of learning how to run an Xbox, he s learned how to run bash. I like that. Lately, Jacob (now 5) hasn t been spending much time with it. He isn t really at a stage where he wants to push his limits too far, I think, but yet also gets bored with the familiar. So I thought it was time to introduce a GUI in a limited fashion, perhaps to let him download photos and video from his Vtech toy camera (that takes real low-res photos and videos which can be downloaded over a USB1 link). He s familiar with the concept, at least somewhat, having seen GUIs on Terah s computer (Gnome 2) and mine (xfce4 + xmonad). So last night, Oliver (age 2) and I went down to the basement on a mouse-finding expedition. Sure enough, I had an old PS/2 mouse down there that would work fine. The boys both helped string it through the desk up on our play room, and were tremendously excited to see the red light underneath it when the computer came on. Barely able to contain the excitement, really. A bit like I remember being when I got my first mouse (at a bit of an older age, I suppose.) I helped him them in as root for the very first time. (Jacob typed root , and I typed the password, and provided the explanation for why we were telling the computer we were root .) Jacob and Oliver alternated typing bits of some apt-get command lines. Then while we waited for software to download, I had to answer repeated questions of how soon will the mouse work? and what does install mean? Finally it was there, and I told Jacob to type startx. I intentionally did not install a display manager; more on that later. He pressed Enter, the screen went blank for about 5 seconds, and then X appeared. Excited can t begin to describe how they acted. They took turns playing with the mouse. They loved how the trash can icon (I started with XFCE) showed trash IN the trash can. But they are just learning the mouse, and there s a lot about a typical GUI that is unfriendly to someone that isn t yet proficient with a mouse. The close buttons are disappointingly small, things can be too easily dragged on and off the panel and menus. When I sat down to think about it, the typical GUI design does not present a very good it always works the same interface that would be good for a child. And then it occurred to me: the perfect GUI for a child would be simply xmonad (a tiling window manager that can be controlled almost entirely by keyboard and has no need for mouse movements in most cases.) No desktop environment, no file manager in the root window. Just a window manager in the classic X way. Of course! So after the boys were in bed, I installed xmonad. I gave Jacob s account a simple .xsession that starts a terminal and xmonad. Today, Jacob informed me that he wanted his computer to look just like yours. Playing right into my hands, that was! But when he excitedly typed startx, he said it wasn t just like mine. Uh oh. Turns out he wanted the same wallpaper as my computer uses. Whew. We found it, I figured out that xli(1) loads it in the root window, and so I added a third line to .xsession. More delight unlocked! Jacob mastered the basics of xmonad really quickly. Alt-Shift-C to close a window. Alt-Shift-Q to quit back to the big black screen . Alt-Shift-Enter to get a terminal window. We launched thunar (the XFCE file manager) and plugged in his camera. He had a good deal of fun looking at photos and videos from it. But then I dropped the true highlight of the day for him: I offered to install Tuxpaint for him. That s probably his favorite program of all time. He watched impatiently as apt-get counted down 1m30s for tuxpaint and its libraries. Then we launched it, and he wanted to skip supper so he could keep playing Tuxpaint on my VERY OWN COMPUTER! I d been debating how to introduce GUIs for a very long time. It has not escaped my attention that children that used Commodores or TRS-80s or DOS knew a lot more about how their computers worked, on average, than those of the same age that use Windows or MacOS. I didn t want our boys to skip an entire phase of learning how their technology works. I am pleased with this solution; they still run commands to launch things, yet get to play with more than text-based programs. At bedtime, Jacob asked me, very seriously: Dad, how do I start tuxpaint again? First you log in and type startx. Then you can use the mouse. Jacob nods, a contemplative look on his face.. Then, I continue, you type tuxpaint in the terminal, and it comes right up. Jacob nodded very seriously a second time, as if committing this very important information to long-term memory. Then gave a single excited clap, yelled Great! , and dashed off.

20 June 2012

Michal &#268;iha&#345;: Rome in May

Finally I've found time to look back at vacation in Rome and post some pictures. It has been great time there and I've really enjoyed that. I would like to thank to Luigi Gangitano for showing us some interesting places in the city. And now back to pictures: Monumento a Vittorio Emanuale Spanish Steps Piramide Via Appia Antica Villa del Quintilli Castel Sant' Angello Arco di Constantino Colloseum

Filed under: English Photography Travelling 0 comments Flattr this!

19 June 2012

Raphael Geissert: Deferring the execution of a command

More than once, I have found myself running a program that calls a given script multiple times, and sometimes, way too many times. In some cases, for instance, it was only necessary to run the script once in one minute, instead of hundreds of times.
Sounds like I want something like dpkg's deferred triggers, right? pretty much so.

With this need, deferred-run was born. Example:

$ deferred-run -l lock -- echo hello world
$ deferred-run -l lock -- echo hello world
$ sleep 3
$ deferred-run -l lock -- echo hello world
[some seconds later...]
hello world

In spite of multiple calls, echo was only called once.
Best part, it only requires sh and flock.

If you are interested in shell trickery, you should take a look at its code.

Steve Kemp: Writing a status panel the modern way

So status displays are cool. Seeing what is happening in real time is cool. As a proof of concept I put together a trivial load-graph: This is broken down into three parts:
Load Client
The load client is a trivial script which reads /proc/loadavg, and sends the 1-minute entry to a remote server, via a single UDP packet.
Load Server
The load-server is a service which listens for UDP traffic, and when it receives a new integer records that in a redis data-store.
Load Display
This is a HTML page which has the values from the store in it, which is then plotted using javscript.
So the UDP-server which receives load will receive two things: This is inserted into a Redis database as an array. This array could then be fetched via an AJAX script to update the HTML display in real-time, but at the moment I just have a shell script which updates it in near-real time. The idea of having a UDP-server receive values from remote clients is interesting. We just need to define a mapping to redis. For me I've just done this:
receive a UDP packet with value "load:1.2" from source 1.2.3.4
append "1.2" to key "1.2.3.4-load".
append the value "1.2.3.4" to the global "known_hosts"
The values received can be truncated (i.e. keep only the most recent 60 entries) with ease, due to the available Redis primitives, and we can easily graph these using the qjplot library. Adding more metrics just means updating the clients to send "memfree:400m", "disk-free:50%", "users:2", "uptime:12345s", or similar. The storage is wonderfully abstract - all you need to do is get the graph-drawing code to a) Know which source to display, and b) which metric. For example, if we did extend the client to send that data I could draw a graph of the memory on host foo.example.com just by selecting "memfree" against the origin "1.2.3.4". ObQuote: "Come here, damn you, I want to touch you. " - Hellraiser

Joey Hess: I work for The Internet now

I have an interesting problem: How do I shoehorn "hired by The Internet for a full year to work on Free Software" into my resume? Yes, the git-annex Kickstarter went well. :) I had asked for enough to get by for three months. Multiple people thought I should instead work on it for a full year and really do the idea justice. Jason Scott was especially enthusiastic about this idea. So I added new goals and eventually it got there. graph: up and to the right, to 800% funded Don Marti thinks the success of my Kickstarter validates crowdfunding for Free Software. Hard to say; this is not the first Free Software to be funded on Kickstarter. Remember Diaspora? Here's what I think worked to make this a success: I've already put two weeks work into developing the git-annex assistant. I'm blogging about my progress every day on its development blog. The first new feature, a git annex watch command that automatically commits changes to git, will be in the next release of git-annex. I can already tell this is going to be a year of hard work, difficult problems, and great results. Thank you for helping make it happen.

Wouter Verhelst: E-voting in Belgium

A post by Gunnar reminded me that I wanted to blog about this: e-voting in Belgium is going to change slightly. Previously, e-voting worked with a computer with a lightpen on which you would enter your vote, a magnetic card that would contain your vote in encoded form, and a magnetic card reader that would count the votes. After more than a decade when I was first eligible to vote back in 1996, it had already been implemented in the city of Antwerp, where my vote would be cast, so I've never voted on paper it had still not been rolled out everywhere, and some of the computers used for the voting were now starting to show their age. A few days ago on the news, they reported that the e-voting system was to change so that rather than magnetic cards, it would use pieces of paper. This way, voters would be able to check that the information on the piece of paper conforms with their vote. Investigating what this means, I found this website by the federal government, where they have some pictures of a test run of the new system which they ran in October last year. Rather than using magnetic cards, the new system uses paper slips with the voted information in two forms: once as a (huge) QR code, and once as a human-readable text. In theory, this is a more transparent system than using a magnetic card, and that's a good thing; additionally, it allows for a traditional recount (by people, not computers) should the need ever arise. However, I do still see some problems: Having said that, I do think the new system is an improvement over the old (in fact, it's exactly the kind of system I had thought about myself earlier when thinking about ways to improve e-voting); and if the two problems are solved in some way (it's not impossible to do that, just requires some clever thinking), I believe it has potential as a way to implement electronic voting in a good way.

18 June 2012

Gunnar Wolf: Turning failure into apparent success, and carrying on: e-voting in Jalisco

I will sound monothematic, but I have been devoting quite a bit of work to this topic lately: Trying to stop the advance of e-voting in Mexico, Latin America and the world. Why trying to stop it? Isn't technology supposed to help us, to get trustable processes? Yes, it's supposed to... but it just cannot achieve it, no matter how hard it is tried I won't get into explanations in this blog post, but there is plenty of information. Feel free to ask me for further details. Anyway Yesterday (Sunday, 2012-06-17) was the fifth simulated voting that will lead to the first wide-scale deployment of electronic voting booths in my country: About 10% of the population of the state of Jalisco (that means, ~500,000 people) will cast their votes on July 1st electronically. This particular case illustrates how simulated votings can be used to forge a lie: Pounce Consulting, the company that won the e-voting project for IEPC (Jalisco's voting authority), delivered their booths over 40 days late, just before the deadline for the project to be canceled. Oh, and by the way, it's the same company that just failed to deliver on time for another planned local authority (10% of the booths in the Federal District, where I live, where fortunately 100% of the votes will be cast on traditional, auditable and cheap paper). After this delay, five voting simulations were programmed, to get the local population acquinted with them. The first ones just failed to get the population's interest and had close to 40% failure rates (mainly regarding transmission). Several other "minor details" were reported, including mechanical details that allowed subsequent voters to see the vote of who had just left. Anyway, making long story short: The fifth and last simulation was held yesterday. Officially, it was finally successful (about time). As these booths include the "facilities" to communicate the results via the cellular network, but the populations where they are to be deployed do not yet have cellular coverage, 10% of the booths will have to be carried back to the Districtal Header (that can be a ~10hr trip) to be counted. Also, in all places, traditional paper stationery and paraphernalia will be printed just in case it is needed (and when will they now? When half of the votes are cast and lost?) Anyway... e-voting is still in its first stage in Mexico. Right now, I'm sure, no attempts to rig the election will be made (centrally). But every effort will be made (as it has been made) to dismiss the obviously big and nontrivial ways it has failed and will fail, and any problems will be labeled as "minor". And probably by 2018 we will be facing many more states (even nationwide) deployments. But propaganda fails to see the obvious: E-voting is more expensive, more complicated, leads to more possible failure states. E-voting should not be deployed in large-scale (i.e. more than a couple of hundred voters) elections. Electronic voting is insecure, violates secrecy, allows for fraud. No matter how many locks are put into it.

Lars Wirzenius: In Kindle store: Getting Things Done for Hackers booklet

cover.jpg I've made some tweaks and fixes to my Gettings Things Done for Hackers booklet on personal productivity, and updated the website and the ebooks (see sidebar on the site). I've also published it on the Amazon Kindle store in case anyone wants to buy it ($1.99). The book remains licensed under the CC-BY-SA 3.0 license, and the Mobipocket version on my site is the same as the one Amazon publishes, modulo any changes Amazon's automation may have done. The Kindle store version is really there for only overly generous people.

17 June 2012

Bastian Blank: DDoS against DNS-Servers

Today I found out that the DNS-Server at home is used for a DDoS against a DNS-Server. The attackers send a small query, "ripe.net ANY" in this case, with a faked sender IP. The DNS-Server answers with a much larger packet to the (faked) sender. Because the domain is not local, the nameserver should have only produced an error. But the bind-config allowed everyone to get answers to cached answers:
allow-query-cache   0.0.0.0/0; ::/0;  ;
With proper restriction, the server only returns errors now. Restricting answer for cached entries does not help if the queried nameserver is authoritative for the domain. In this case it can help to drop queries for the ANY type. In ferm this looks like this:
proto udp dport domain mod string from 40 algo bm hex-string ' 0000ff0001 ' DROP;

Johannes Schauer: port bootstrap build-ordering tool report 2

A copy of this post is sent to soc-coordination@lists.alioth.debian.org as well as to debian-bootstrap@lists.mister-muffin.de. Diary June 4 I added the first version of basenocycles.ml to git. Given an initial set of cross built packages, it tries to compile as much as possible on the resulting system in multiple rounds. June 5 During June 3, I discovered and error in my program that would only come up when using the Debian Sid package lists as the input:
Fatal error: exception Assert_failure("common/edosSolver.ml", 610, 11)
On this day, June 5, I wrote a minimal test case for this problem. The same day, Pietro figured out that this is a bug in dose which will be fixed in the next release. Begin writing code to figure out how important a binary package is for the further build process. Try to use Depsolver.edos_install to find out what packages are needed to make debhelper available. Restructure basenocycles.ml, exclude source packages that already have been built, still trouble with already existing binary packages and Cudf.mem_installed, comment stuff better. June 6 I wrote some crude code (only estimate, not correct, fixed later) that would give a rough overview of how often a given binary package is directly required as a build dependency. Debhelper came out as the most needed package. It is architecture:all, so it does not have to be built but it has unfulfilled runtime dependencies. To make those packages available, 13 (actually 11, fixed later) packages have to be compiled on Ubuntu Natty. But those packages all (except gettext) require debhelper itself to be built. The first dependency cycle. This dependency cycle (actually, the 12 cycles) can be broken by either cross compiling those source packages or by making them build without debhelper. One goal of the program is to help decide what the easier option is, but this is not yet implemented. To play around a bit, I created the possibility to specify a list of packages that are additionally to the minimal set of cross compiled packages also cross compiled. I added the 13 packages found above to the list, thus making the binary packages they build available. This made debhelper available in the system. As a result, 1625 out of 3339 source packages can be built with just a minimal build system (priority:essential packages plus build-essential) and debhelper available. The next package that blocks the most other source packages from being built is cdbs. The next nine packages in that list also require cdbs so it seems to be the next important package to be available. Pietro's suggestions make me:
 - do not open BootstrapCommon but ExtLib, Common, Algo, Debian
 - do proper option parsing and logging
 - use Debcudf.ignore_essential = true
 - do Debcudf.init_tables (binlist@srclist)
 - use @ with shorter list first
 - use more List.rev_append instead of @
 - use CudfAdd.who_provides to find if a package is available
June 7 Pietro and Patrick suggest that for solving the debhelper cycles, one could provide a minimal debhelper version so that the above list of 12 packages can be built without debhelper. I try to figure out how to get a list of packages that are missing to make a package installable/buildable. This functionality should be provided in dose but I fail to find it. June 8 Lacking a solution of the problem of June 7, I write a mail to Pietro. I start my first graphs in ocaml using the ocamlgraph library. The graph I generate, starts off at a binary package. For each binary package it connects new vertices as its runtime dependencies. If a binary package is not arch:all and also not yet otherwise compiled, its source package is also added. The result is a graph in which set of source packages in it will make the initial package available, if those source packages would be cross compiled. The graph is extended further than the source packages. June 9 I refine a couple of functions, make univ_get_pkg_by_name return the package with the highest version number. I wrote a rather lengthy (1027 words) email to the list that explains my status as of this day. I can create graphviz dot files with ocaml, can create node and edge types and create the graph by an imperative pattern that I saw a lot in Pietro's code. Disjunctions are not yet handled correctly (see mail from June 8). The graphs generated look like the following: http://mister-muffin.de/p/8nyc.png June 11 I write a test case which shows how CudfAdd.who_provides doesnt find virtual packages. Automate the process of finding the packages that, if cross compiled, would make another package available. Add more predicates (identifying source packages) and improve input file reading code. Move build_compile_rounds which compiles as many source packages as it can in multiple rounds on a given minimal system a toplevel function and thereby abstract it more. Create a rudimentary text based menu to choose different actions to take for an analysis. Start writing an extended version of simple_dependency_graph for deeper analysis. Use xdot to show graphs from the text menu. Allow saving those graphs to a file. June 12 Move functionality from the extended version of simple_dependency_graph over to the normal version and delete the extended version. Add the new Closure vertex type. Create extended_dependency_graph which is supposed to not contain single binary package vertices but handle a package and its installation set as one vertex. The output of extended_dependency_graph is optionally reduced to the biggest (non degenerate) strongly connected component. User gets the option of choosing the exploration depth. June 13 Pietro replies to my mail from June 8 but apparently I failed to express myself well enough in my last mail, so I rephrase my question. Pietro replies to my email from June 11 and explains how the effect I see is due to "a nuisance of the debian to cudf encoding". As a result I change my code accordingly. June 14 Another lengthy (1130 words) email to the list. I explain what was done in the past days, what parts work and how they work. I list some rationales on why I did things the way I did them. The most important observation is, that after improving my code again and again, I ended up representing the dependency cycle problem in the same (very similar) way that Pietro suggested in the beginning. This is probably a good discovery. Lots of data of that email is now of only little use as of June 16, I make lots of improvements in correctness. As I dont have an answer to my other email to Pietro from June 13, I implement a very crude way to get an answer to the question of what packages are missing for a package to be available/compileable. I called it flatten_vpkgformula_best_effort and it suffers from many faults including disjunctions and package conflicts. Patrick spots a problem. As a result, I make sure that at no point, the source package of an arch:all package can be listed. June 15 As a reply to my mail from June 13, Pietro creates a new branch in the git and adds the code I needed to get a proper installation set. June 16 As a result of Pietro's efforts from June 15, I make great advancements on all fronts. Details of the current status follow in the next section. Results A big leap was made on June 16 due to Pietro's great help on making me understand how Depsolver.listcheck can be used for my purposes. My difficulties in finding the solution myself are rooted in many parts of the dose framework being poorly commented but Pietro did already a couple of documentation commits whenever things were unclear for me. Using Depsolver.listcheck makes it possible to be more distribution agnostic and I dont have to handle vpkgs, virtual packages and constraints myself anymore. The code also doesnt suffer anymore by wrongly analyzed dependencies and conflicts. The only thing that is not yet taken care of, is that Depsolver.listcheck only chooses one out of several possible installation set. A final version should be able to take into account that a different installation set could provide a better solution. Overall, in comparison to two weeks ago, I can now properly build, traverse and analyze graphs, can choose an installation set properly, understand more about dependencies, closures, dose and ocaml in general. Finding the importance of binary packages for building When calculating how many source packages are depending on the availability of a binary package I originally flattened the pkg.Cudf.depends list twice for a rough overview. This is of course wrong due to disjunctions and conflicts and also doesnt provide a deep dependency inspection. The new method is to calculate an installation set that is necessary to compile a source package for every source package. The resulting list of binary packages is then used to find out how often a binary package appears in an installation set. I see three drawbacks though: Removing simple graph The simple graph which contained single binary and source packages was removed. I realized it doesnt really serve any purpose to look at it. As a result, Bin vertices and InstallDep edges are also not part of the graph anymore. Since it was responsible for generating the list of source packages that have to be cross built to make a package available, I created a new function get_cross_source_packages which uses an installation to serve the same purpose. Fix extended_dependency_graph extended_dependency_graph now uses installation sets for generating the list of packages that is needed to compile a source package or install a binary package. The list of build dependencies does not include packages that are already installable. The list of runtime dependencies does not include packages that are otherwise available (cross built, arch:all...). Instead of checking for list membership all the time, I created hash tables for the list of installable as well as for the list of available binary packages. Future There are two big tasks for the next two weeks: Task one is to find a way to give hints on which packages to best consider for having reduced build dependencies. This would then probably finally make use of Pietro's cycle algorithms. Task two is to find a way to break cycles and create a build-DAG from a list of packages that already have staged build dependency information. Patrick is currently working on patching dpkg with Build-Depends-StageN dependencies as making perl cross compilable. If he doesnt need the ability to decide which packages to modify to have staged build dependencies in the near future, then task one is probably less urgent and therefor of less importance right now? On the other hand, I can easily generate fake reduced build dependencies so that doing task two right now would not be a problem. Also, having the solution for task two will make it possible to show the user what effect it would have to add reduced build dependencies to a package. For the reasons above (it's not urgent, task one profits from task two being solved) I will go and implement task two first (if there are no objections from my mentors). Another idea, that I discussed with wookey and Patrick yesterday, was that due to multiarch being used for more and more packages, there should exist a set of packages that is cross compilable without any change to the package. We agreed that I make a list of packages that, if cross compiled, would break dependency cycles and make other packages available. I created such a list of about 160 packages for Ubuntu Natty that, if cross compiled, made it possible to have 87% of Natty available (those numbers have to be treated with caution as I did not yet use the proper way of installation sets when generating that list, but the order of magnitude should be correct). Wookey can then try to cross compile those packages. If some packages of those "crucial" source packages are found to be cross compilable, then they should be cross compiled because it means that no work has to be done to break some cycles. Cross compiling all packages that are cross compilable out of the box is no solution, as only natively compiled packages can go into the archive. This is why the list of potentially additionally cross compiled source packages has to be kept as small as possible.

C.J. Adams-Collier: Service Interruption

They heard my internets were down BEGIN PGP SIGNED MESSAGE
Hash: SHA256 , During the month of June, we experienced a service interruption which
has caused our mutual customer, School, an
inability to initiate outgoing calls. CenturyLink has also caused the
interruption of the school s DSL Internet service. During the interim
period, I have installed a VoIP phone and tied in to one of my backup
providers so that the school can function during the CenturyLink outage. My understanding is that will begin paying CenturyLink
directly for its service again due to the service interruption. Until
such time as I receive acknowledgment from you, we will continue paying
roughly $27/mo for the school s phone service. Once the transfer has
been completed to the satisfaction of the school board, Collier
Technologies LLC will discontinue payment for the DID - - ,
leaving a single DID, - - . I expect that the DID will be
moved back into the school s name within the next week. Please confirm
this or provide information as to why this will not happen. I have CC d
and , who represent the school board on this
email. Please include them in the reply. Since the DSL service has been interrupted at both the School s address
and at ours, and since CenturyLink has failed to provision the remaining
two $14.99/mo 10Mbit DSL circuits which Collier Technologies LLC has
been paying for despite their un-provisioned state, we will be reducing
the monthly payment from $150 to $70. My understanding is that $70 will
be more than sufficient to cover the two voice lines, inside wire
maintenance and regulatory fees. If this is not the case, please keep
us appraised of the situation. Do note that our decision to stop payment for service which is not being
provided does not, in our estimation, relieve CenturyLink from the three
year contract period negotiated verbally by you on its behalf on October
4th 2011. We currently consider CenturyLink to be in breach of
contract. Sincerely, C.J. Adams-Collier BEGIN PGP SIGNATURE
Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBCAAGBQJP3TqdAAoJEEyimPmPQm23qqoH/jRmiIVHNCnH8EqgzsNQiUIm
LVrFg7kvpRIM692mzrGgkZTxG9KlVK+1/dZl1aPsmMP5Q6ni3NigzRDnywJZI53a
yTwTZhid8B4Tf+JOrhUhKPpmm+2YoHTYaobwgxk+rRgUGUHEMCZMPnjKNGGPDCgO
2jNz5fSnI6IinRz4nXwhAtGQRUXVJWXGrUCX1kQXGH4pNvjbgE8n6yAsC6sixlPa
ljDaNjxUEjUY4PmQdKNetkzQLxlq/OGJgRHZXHzfg37cJEF4L28DJBYvf8CHbt2K
87ELXGs6nDn54yI4q4ojODln9k/GzC+NY6MDXYdof7dNbeEPujvjQCiyRgjLc9E=
=uLKs
END PGP SIGNATURE

16 June 2012

Wouter Verhelst: Life: general update

In case you were wondering what had happened to my blog over the past few weeks: the server on which it runs, samba.nixsys.be, has been down for that time, since we've just moved office. There's a story behind it, with which I won't bore you; suffice to say that we had to move out because of circumstances beyond our control. If you really do care, ask me over a beer or something (at debconf would be a good opportunity). I'll happily tell you the story. Meanwhile, the server on which this blog runs and which was located in a separate room at our office was now no longer available. But that's now over, albeit that it's behind a different IP address, and ignoring the fact that I still have some minor things to finish after the move. During this move, I found that we'd accumulated quite an amount of old unused hardware. While I've retained some of it (mostly my m68k machines, and some other machines that were given to me in order to add or improve Debian support for them), most of it has been given or thrown away. And that's even ignoring the other crap that had been in the cellar or the server/storage room. Oh my. I had severely underestimated the amount of work that would be required to clean out the office, which unfortunately also meant that I overshot the official date on which I were to leave the office by about two weeks. Luckily the landlord wasn't in a hurry or something, so no harm done. But I'll say this: if I ever say "yes" to anyone offering me some old decrepit hardware again for which I have no immediate purpose, may I be struck by lightning where I stand. Anyway. There's one significant advantage from this particular move: the view. I've gone from a dead wall to this: IMAG0004 Which is clearly an improvement... Additionally, now just over a month ago, my bother's fianc e gave birth to a boy, which makes me an uncle for the first time ever. Scary.

15 June 2012

John Goerzen: Windows & a dying hard disk: Solving with Linux

Today, my workstation sent me this email:
The following warning/error was logged by the smartd daemon:
Device: /dev/sda [SAT], 1 Currently unreadable (pending) sectors
and then a little later, this one:
The following warning/error was logged by the smartd daemon:
Device: /dev/sda [SAT], 1 Offline uncorrectable sectors
From the hard disk s SMART data, this is a clue that the drive is failing or will soon. Sigh. Incidentally, if smartmontools isn t installed on your machine, whether it s a laptop, desktop, or server, it should be. Although most of you know I run Linux on the metal on my machines almost exclusively, I do maintain a small drive with a Windows installation that I boot into every few months for various reasons. This is that drive. The drive is non-redundant (no RAID), and although it is backed up, the backup is made via backuppc from the NTFS filesystem mounted on Linux, and is a partial backup backing up certain data, not the OS. There are, of course, bare metal Windows backup solutions, but I generally don t want to back up Windows from within Windows on this machine. Restoring Windows isn t quite as simple as an mkfs, an untar, and a grub-install, either. So my first thought is: immediately save whatever of the drive I can. So I ran apt-get install gddrescue to install the GNU ddrescue tool. ddrescue is somewhat similar to dd, but deals much more intelligently with bad blocks on the drive. It will try to read them repeatedly, with decreasing block sizes, in an effort to get every last good byte off the disk that it can. If it ultimately fails to get certain bytes read, it will write placeholder data to the output file in place of the missing data, so that the output file maintains proper size and alignment. It also saves a log file that notes what it found (see info ddrescue for more on that.) So I created an LVM volume for the purpose (not enough free space on /home, and didn t want to have to shrink it somehow later), and ran:
ddrescue /dev/sda /mnt/sdasave.ddrescue /mnt/sdasave.logfile
Then I went to dinner. When I got back, I discovered there were 1 or 2 bad sectors, about halfway through the disk, but everything else was fine. So now, the question became: did I lose any data? If so, what? I needed to know if I had to revert to a backup for anything or not. To answer THAT question, first I had to figure out the offset of the bad spots on the disk. That s not too hard; the logfile gives it to me:
# Rescue Logfile. Created by GNU ddrescue version 1.15
# Command line: ddrescue /dev/sda /mnt/sdasave.ddrescue sdasave.logfile
# current_pos  current_status
0x3BBB8BFC00     +
#      pos        size  status
0x00000000  0x3BBB8BF000  +
0x3BBB8BF000  0x00001000  -
0x3BBB8C0000  0x38B5346000  +
what we see is that the bad sector starts at byte 0x3BBB8BF000 (256549580800 decimal) and extends for 0 1000 bytes (4096 decimal). Both the drive and NTFS use 512-byte sectors. So dividing by 512, we get sector 501073400 501073407 (4096 bytes is 8 sectors). As a check, I ran grep sector /var/log/kern.log and turned up a bunch of lines like this: Jun 14 21:39:11 hephaestus kernel: [35346.929957] end_request: I/O error, dev sda, sector 501073404 Which is within my calculated range. But this is an absolute sector on the disk. We need the sector within the partition, so for that, we have to enlist fdisk to make that calculation. fdisk shows, among other things:
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048   976771071   488384512    7  HPFS/NTFS/exFAT
So the Windows partition starts at disk sector 2048. Let s just confirm that. If I use dd if=/dev/sda1 bs=512 count=1 hd head, I see a line beginning with .R.NTFS . Exactly the same as with dd if=/dev/sda bs=512 count=1 skip=2048 hd head, so I read the partition table information correctly. Subtract offset of 2048 from the earlier values, and I get relative sectors 501071352-501071359. That s enough to get some solid info from the filesystem via ntfscluster, part of Debian s ntfs-3g package. I pass -s to it, and ignoring some irrelevant stuff, get my answer:
ntfscluster -s 501071352-501071359 /dev/sda1
Inode 190604 /System Volume Information/ b4816feb-b609-11e1-a908-50e549b934f7 3808876b-c176-4e48-b7ae-04046e6cc752 /$DATA
I even reran it with a much larger sector range, just to be absolutely sure I had wiggle room in case calculations had an off-by-one error or something somewhere. This is really great news, because the file in question is pretty much useless I believe it s a system restore point, which I won t be needing anyhow. So at this point, all that remains is to reinstall this on a different drive. For that, I could just use my ddrescue image. I thought I would take a second image, just to be very extra careful, and use that; I used:
partclone.ntfs --rescue -c -s /dev/sda1 -o sda1.partclone
although ntfsclone would work just as well. This captures only the partition; I ll need the partition table as well, and perhaps also the space between the partition table and the first partition. I could capture it separately with dd, but it s already in the ddrescue image, so there s no need. (GRUB is installed on this drive, but there is no Linux filesystem on it, so it may well exceed the size of the MBR). Note that for Linux ext[234] filesystems, debugfs can provide the same (and more) info as I got from ntfscluster. I happen to have a drive of the right size sitting here, which I was about to install in a different machine. So a wipe and a swap and a restore later, and I should be good to go. This scenario is commonplace enough that I thought I d post how I dealt with it, in case anyone else ever has hard drive issues.

Matthew Garrett: No, really, secure boot does add security

Carla Schroder wrote a piece for linux.com on secure boot, which gives a reasonable overview of the technology and some of the concerns. It unfortunately then encourages everyone to ignore those legitimate concerns by making the trivially false claim that secure boot is just security theatre.

Security isn't some magic binary state where you're either secure or insecure and you can know which. Right now I'd consider my web server secure. If someone finds an exploitable bug in Apache then that would obviously no longer be the case. I could make it more secure by ensuring that Apache runs in a sufficiently isolated chroot that there's no easy mechanism for anyone to break out, which would be fine up until there's also a kernel exploit that allows someone to escalate their privileges enough to change their process root. So it's entirely possible that someone could right now be breaking into my system through bugs I don't even know exist. Am I secure? I don't know. Does that mean I should just disable all security functionality and have an open root shell bound to a well known port? No. Obviously.

Secure boot depends on the correctness of the implementation and the security of the signing key. If the implementation is flawed or if control of the signing key is lost then it stops providing security, and understanding that is important in order to decide how much trust you place in the technology. But much the same is true of any security technique. Kernel flaws make it possible for an unprivileged user to run with arbitrary privileges. Is user/admin separation security theatre? SSL certificate authorities have leaked keys. Is it security theatre for your bank to insist that you use SSL when logging in?

Secure boot doesn't instantly turn an insecure system into a secure one. It's one more technology that makes it more difficult for attackers to take control of your system. It will be broken and it will be fixed, just like almost any other security. If it's security theatre, so is your doorlock.

Why is this important? Because if you tell anyone that understands the technology that secure boot adds no security, they'll just assume that you're equally uninformed about everything else you're saying. It's a perfect excuse for them to just ignore discussion of market restrictions and user freedoms. We don't get anywhere by arguing against reality. Facts are important.

comment count unavailable comments

14 June 2012

John Goerzen: How to debugging Linux failure to resume from suspend?

I m running a computer with a Gigabyte Z68A-D3H-B3 motherboard, and have never been able to get it to properly resume from suspend to RAM in Linux. It has worked fine on the rare occasion I ve tried it in Windows 7. My somewhat limited usual for debugging aren t particularly helpful. The system appears to suspend perfectly fine. It just doesn t resume. To be more precise, when I push the button to resume, the power comes up (fans whir, HDD spins up, etc.) but nothing happens. The USB keyboard and mouse don t respond, Caps Lock doesn t toggle any LEDs, it doesn t respond on the wired LAN, and the display stays off. Although it s a desktop, I d really like to save power on this thing by suspending it when it s not in use. There s no sense in wasting power I don t need to be consuming. I ve tried what I used to try on laptops. I tried running in single-user mode, without X, or even the kernel modules for video acceleration loaded. I tried unloading whatever hardware modules I thought I could without completely destabilizing the system. I updated the BIOS to the latest release. I tried various combinations of video tweaks. I tried using s2ram from uswsusp instead of pm-suspend. Nothing made any difference. They all behaved exactly the same. Googling showed a lot of resources for people that had trouble getting their machines to go to sleep. And also for people whose machines would wake up but just wouldn t re-activate the display. But precious little for people with my particular symptoms. What s a good place to start looking to fix something like this? Some details CPU is Core i5-2400. Kernel is wheezy s 3.2.0-2-amd64, though this problem has persisted as long as I ve had this machine, which was running squeeze at install time. Video is NVidia GeForce GTX 560 (GF114). Hard drives are SATA, Ethernet is integrated RTL8111/8168B. Userland is up-to-date amd64 wheezy.

Russell Coker: The Financial Value of a University Degree

I ve read quite a few articles about the value of a degree. Most of them come from the US where the combination of increasing tuition fees and uncertain job market makes a degree seem like a risky investment. I think that most analysis of the value of a degree are missing some important points. The Value of Money at Different Times The value of money is different at various stages of your life. The impression that I get is that when a married couple have their house fully paid off and they either don t/won t have children or their children are old enough to leave home the amount of money that they earn seems to matter a lot less. Doing a university degree involves 3 or 4 years not earning money (or more if doing post-graduate studies), which is usually starting at the age of 18. Effectively getting a degree involves giving up some money while young for the opportunity to earn more when older. Any analysis based on directly comparing the money spent on the degree to the amount of financial return without considering when money is needed is not very useful. I think that a reasonable analysis would exclude income earned after the age of about 45. By that age most people have either achieved a solid financial position and learned to live within their means or messed up their finances so badly that they won t live long enough to recover. A Degree as a Signal The Wikipedia page on economic signalling gives education as an example of a signal. A signal in this case means something that doesn t inherently mean anything but which signifies something else. So completing a degree doesn t necessarily mean that you learned anything relevant to work, but if you are able to do it then it means that you can probably also do things which are economically useful for an employer. This raises the question of how else one might signal their ability to work. One obvious answer is by working, someone who has remained steadily employed for 3 or 4 years has demonstrated their ability to work reliably and get along with other people which should be at least as useful as a signal. It s Not Only the Degree Most analysis seem to compare average income of people with degrees with the average of income with people who didn t attend university. That is based on the assumption that the degree was the only difference. When I was young my parents spent a moderate amount of money on a full set of paper encyclopedias (about 2 meters of shelf space). I m sure that this gave me some educational benefit as they intended, and it was something that was apparently quite rare I don t recall seeing a full encyclopedia in anyone else s house before the Wikireader [1]. My parents also bought me quite a lot of computer gear (back when hardware was really expensive), were always available to drive me to computer users group meetings etc, and did everything else that seemed likely to have an educational benefit. The value of such learning opportunities is significant. I think that almost everyone who had similar learning opportunities to me when they were young will probably have experienced similar support and pressure to attend university. I also think that almost everyone who receives such opportunities will be able to earn more than the median income even if they don t attend university. To a large extent people who are going to be successful attend university. A university degree doesn t make anyone successful if they couldn t succeed without a degree. There are some careers that just aren t options if you don t have a relevant degree (such as medicine and law). But I believe that anyone who is capable of completing a difficult course such as medicine or law (or any other career that has legal requirements for a degree) is capable of being successful without a degree in many other fields. So comparing the wages of a doctor or a lawyer to an average person doesn t make sense, it makes more sense to try and compare their wages to someone of similar skill who didn t have such a qualification. Conclusion It seems to me that the question is, of the people who had great learning opportunities when they were young and who wanted to succeed, would they have earned much less if they hadn t attended university? The next question is, of the people who might earn significantly less without getting a degree, would that salary difference really have mattered, or would it just be a matter of earning some luxury money when they are too old to really need it? Related posts:
  1. Ideas for a Home University There seems to be a recent trend towards home-schooling. The...
  2. university degrees Recently someone asked me for advice on what they can...
  3. Microsoft Hires University Drop-Out for Recruiting Campaign news.com.au reports that MS has hired former Miss Australia Erin...

11 June 2012

Petter Reinholdtsen: Debian Edu - some ideas for the future versions

During my work on Debian Edu based on Squeeze, I came across some issues that should be addressed in the Wheezy release. I finally found time to wrap up my notes and provide quick summary of what I found, with a bit explanation. I guess we will discover more as we continue to work on the Wheezy version.

Next.

Previous.