Search Results: "ed"

10 September 2013

Petter Reinholdtsen: Recipe to test the Freedombox project on amd64 or Raspberry Pi

I was introduced to the Freedombox project in 2010, when Eben Moglen presented his vision about serving the need of non-technical people to keep their personal information private and within the legal protection of their own homes. The idea is to give people back the power over their network and machines, and return Internet back to its intended peer-to-peer architecture. Instead of depending on a central service, the Freedombox will give everyone control over their own basic infrastructure. I've intended to join the effort since then, but other tasks have taken priority. But this summers nasty news about the misuse of trust and privilege exercised by the "western" intelligence gathering communities increased my eagerness to contribute to a point where I actually started working on the project a while back. The initial Debian initiative based on the vision from Eben Moglen, is to create a simple and cheap Debian based appliance that anyone can hook up in their home and get access to secure and private services and communication. The initial deployment platform have been the Dreamplug, which is a piece of hardware I do not own. So to be able to test what the current Freedombox setup look like, I had to come up with a way to install it on some hardware I do have access to. I have rewritten the freedom-maker image build framework to use .deb packages instead of only copying setup into the boot images, and thanks to this rewrite I am able to set up any machine supported by Debian Wheezy as a Freedombox, using the previously mentioned deb (and a few support debs for packages missing in Debian). The current Freedombox setup consist of a set of bootstrapping scripts (freedombox-setup), and a administrative web interface (plinth + exmachina + withsqlite), as well as a privacy enhancing proxy based on privoxy (freedombox-privoxy). There is also a web/javascript based XMPP client (jwchat) trying (unsuccessfully so far) to talk to the XMPP server (ejabberd). The web interface is pluggable, and the goal is to use it to enable OpenID services, mesh network connectivity, use of TOR, etc, etc. Not much of this is really working yet, see the project TODO for links to GIT repositories. Most of the code is on github at the moment. The HTTP proxy is operational out of the box, and the admin web interface can be used to add/remove plinth users. I've not been able to do anything else with it so far, but know there are several branches spread around github and other places with lots of half baked features. Anyway, if you want to have a look at the current state, the following recipes should work to give you a test machine to poke at. Debian Wheezy amd64
  1. Fetch normal Debian Wheezy installation ISO.
  2. Boot from it, either as CD or USB stick.
  3. Press [tab] on the boot prompt and add this as a boot argument to the Debian installer:
    url=http://www.reinholdtsen.name/freedombox/preseed-wheezy.dat
  4. Answer the few language/region/password questions and pick disk to install on.
  5. When the installation is finished and the machine have rebooted a few times, your Freedombox is ready for testing.
Raspberry Pi Raspbian
  1. Fetch a Raspbian SD card image, create SD card.
  2. Boot from SD card, extend file system to fill the card completely.
  3. Log in and add this to /etc/sources.list:
    deb http://www.reinholdtsen.name/freedombox wheezy main
    
  4. Run this as root:
    wget -O - http://www.reinholdtsen.name/freedombox/BE1A583D.asc   \
       apt-key add -
    apt-get update
    apt-get install freedombox-setup
    /usr/lib/freedombox/setup
    
  5. Reboot into your freshly created Freedombox.
You can test it on other architectures too, but because the freedombox-privoxy package is binary, it will only work as intended on the architectures where I have had time to build the binary and put it in my APT repository. But do not let this stop you. It is only a short "apt-get source -b freedombox-privoxy" away. :) Note that by default Freedombox is a DHCP server on the 192.168.1.0/24 subnet, so if this is your subnet be careful and turn off the DHCP server by running "update-rc.d isc-dhcp-server disable" as root. Please let me know if this works for you, or if you have any problems. We gather on the IRC channel #freedombox on irc.debian.org and the project mailing list. Once you get your freedombox operational, you can visit http://your-host-name:8001/ to see the state of the plint welcome screen (dead end - do not be surprised if you are unable to get past it), and next visit http://your-host-name:8001/help/ to look at the rest of plinth. The default user is 'admin' and the default password is 'secret'.

Daniel Kahn Gillmor: Support privacy-respecting network services!

Support privacy-respecting network services! Donate to Riseup.net! There's a lot of news recently about some downright orwellian surveillance executed across the globe by my own government with the assistance of major American corporations. The scope is huge, and the implications are depressing. It's scary and frustrating for anyone who cares about civil society, freedom of speech, cultural autonomy, or data sovereignty. As bad as the situation is, though, there are groups like Riseup and May First/People Link who actively resist the data dragnet. The good birds at Riseup have been tireless advocates for information autonomy for people and groups working for liberatory social change for years. They have provided (and continue to provide) impressive, well-administered infrastructure using free software to help further these goals, and they have a strong political committment to making a better world for all of us and to resisting strongarm attempts to turn over sensitive data. And they provide all this expertise and infrastructure and support on a crazy shoestring of a budget. So if the news has got you down, or frustrated, or upset, and you want to do something to help improve the situation, you could do a lot worse than sending some much-needed funds to help Riseup maintain an expanding infrastructure. This fundraising campaign will only last a few more days, so give now if you can! (note: i have worked with some of the riseup birds in the past, and hope to continue to do so in the future. I consider it critically important to have them as active allies in our collective work toward a better world, which is why i'm doing the unusual thing of asking for donations for them on my blog.)

9 September 2013

Kurt Roeckx: State of encryption

With all the articles about the NSA going around it seems to be hard to follow what is still safe. After reading various things it boils down to:
I don't think this should really surprise anybody. There is also speculation that maybe the NSA has better algorithms for reducing the complexity of public key cryptography or that maybe RC4 might be broken. Those things clearly are possible, but it's not clear. Nobody has suggested that AES has any problems, and I think that's still safe to use. One of the question becomes what does properly done encryption mean. There are various factors to this and many applications using various protocols. SSL/TLS is the most used, so I'm going to concentrate on the parts in that. Bit sizes As far as I understand it, 80 bit of security is around what is currently the upper practical limit of a brute force attack. In 2003 NIST said to stop using 80 bit by 2015, in 2005 they said to stop using it by 2010. When using something like SSL/TLS you using various things each having it's own size. So it's important to known which is the weakest part. Public key encryption (RSA, DSA, DH) doesn't offer the same amount of security per bit than symmetric keys. The equivalent sizes I find are:
Symmetric Public
80 1024
112 2048
128 3072
256 15360
There are other sources that have other numbers, but they're similar So this basically means you really should stop using 80 bit symmetric and 1024 bit public keys and that your public key really should be at least 2048 bit, and should maybe even consider going to 3072 bit. There seems to be a push to moving Diffie-Hellman (DH) to provide Perfect Forward Secrecy (PFS). There are many variants to this. We want to use ephemeral (temporary) keys. But it basically boils down to 2 variants:
With DHE you first negotiate a temporary DH key for this session (hopefully) using a random number on both sides, and then using that DH key to exchange a normal symmetric key for the session. This temporary DH key is what provides the PFS, assuming that this temporary key is thrown away. The drawback of this is that creating a sessions takes a lot more CPU time. The standard DH has the same security as other public key encryption. So we really also want to stop using 1024 keys for that. With ECDHE the key size needs to be the double of the symmetric size. This is also faster than DHE, so people want to move to this. There are various curves that can be used for this, and some of them are generated the the NSA and people have no trust in them. Some people even don't trust any of the curves. Most people seem to think that ECDH is the way forward, but then limit the curves they support. There are also hashes used for various things including signatures and MACs. Depending on how it's used the properties of the hash are important. They have 2 important properties collision resistance and preimage resistance. For a collision attack the upper limit is a little more than the output size over 2. But there are attacks that are known to give worst results. I found those numbers:
MD5 2^20
SHA-1 2^60
SHA-2 No known attack (so SHA-256 would have 1.2 * 2^128)
For preimage attacks I found:
MD5 2^123
SHA-1 No known attack (2^160)
SHA-256 No known attack (2^256)
So I think that MD5 is still safe when a collision attack isn't possible, but should really be avoided for anything else and SHA-1 should probably be considered the same. So you want to avoid them in things like certificates. SSL/TLS also use MD5 / SHA-1, but as part of a HMAC. I understand that those are not vulnerable to the collision attack but preimage resistance is important then and so are still considered safe. Randomness A lot of the algorithms depend on good random numbers. That is that the attacker can't guess what a (likely) random number you've selected. There have been many cases of bad RNG that then resulted in things getting broken. It's hard to tell from the output of most random number generators that they are secure or not. One important thing is that the RNGs gets seeded with random information (entropy) to begin with. If it gets no random information, very limited amount of possible inputs or information that is guessable as input it can appear to give random numbers, but they end up being predictable There have been many cases where this was broken. Ciphers There are various encryption algorithms. But since the public key algorithms take a lot of time we usually want to limit the amount we do with them and do most with a symmetric key encryption. There are basically 2 types of symmetric ciphers: stream and block ciphers. For block ciphers there are various ways of combining the blocks. The most popular was CBC. But in combination with TLS 1.0 it was vulnerable to the BEAST attack, but there is a workaround for this. An other mode is GCM but it's not yet widely deployed. This resulted in the stream cipher RC4 suddenly getting popular again. Some ciphers have known weaknesses. For instance triple DES has 3*56=168 bits, but only provides for 2*56=112 bits of security. Other attacks make this even less. The algorithms them self might be secure, but they might be attacked by other mechanism known as sidechannel attacks. The most important of that is timing attacks. If the algorithm has branches they might result in different amount of time taking in each branch. This might result in an attacker finding out some information about the secret key that is being used. It's important that the difference should be reduced as much as possible. This is usually not a problem if it's implemented in hardware. SSL/TLS Session A typical SSL/TLS session uses those things:
Important aspects of using certificates:
The client sends it's list of supported ciphers to the server and the server will then pick one. This is usually the first from that list that is supported by the server, but it might also be the order that is configured on the server that is used. Current status in software If you go looking at all of those things, you find problems in most if not all software. Some of that is to be compatible with old clients or servers, but most of it is not. If you want to test web servers for various of those properties you can do that at ssllabs. Based on that there are stats available at SSL-pulse. Apache only supports 1024 bit DHE keys and they don't want to apply a patch to support other bit size. Apache 2.2 doesn't offer ECDHE but 2.4 does. I don't know of any good site has lists which browser supports which ciphers, but you can test your own browser here. It lacks information about EC curves that your client offers. You can of course as look at this with wireshark. Mozilla currently has a proposal to change it's list of supported ciphers here. They currently don't have TLS 1.1 or 1.2 enabled yet, but as far as I understand the NSS library should support it. They should now have support for GCM, but they still have an open bug for a constant time patch. In general Mozilla seems to be moving very slowly with adopting things. Last I heard Apple still didn't patch Safari for the BEAST attack. For XMPP see client to server, server to server, clients.

For TOR see here.

Bastian Blank: Setting up Ceph the hard way

Components Ceph consists of two main daemons. One is the monitoring daemon, which monitors the health of the cluster and provides location information. The second is the storage daemon, which maintains the actual storage. Both are needed in a minimal setup.
Monitor The monitor daemons are the heart of the cluster. They maintain quorum within the cluster to check if everything can be used. They provide referrals to clients, to allow them to find the data they seek. Without a majority of monitors nothing will work within the cluster.
Storage The storage daemons maintain the actual storage. One daemon maintains one backend storage device.
Configuration The default config is understandable, but several things will just not work with it.
Monitor on localhost By default the monitor daemon will not work on localhost. There is an (undocumented) override to force it to work on localhost:
[mon.noname-admin]
 mon addr = [::1]:6789
The monitor will be renamed to mon.admin internaly.
IPv6 Ceph supports IP (IPv6) or legacy-IP (IPv4), but never both. I don't really use legacy-IP any longer, so I have to configure Ceph accordingly:
[global]
  ms bind ipv6 = true
One-OSD clusters For testing purposes I wanted to create a cluster with exactly one OSD. It never got into a clean state. So I asked and found the answer in #ceph:
[global]
 osd crush chooseleaf type = 0
Disable authentication While deprecated, the following seems to work:
[global]
 auth supported = none
Complete configuration
[global]
 auth supported = none
 log file = $name.log
 run dir =  
 osd pool default size = 1
 osd crush chooseleaf type = 0
 ms bind ipv6 = true
[mon]
 mon data =  /$name
[mon.noname-admin]
 mon addr = [::1]:6789
[osd]
 osd data =  /$name
 osd journal =  /$name/journal
 osd journal size = 100
[osd.0]
 host = devel
Installation This is currently based on my updated packages. And they are still pretty unclean from my point of view.
Setup All the documentation tells only about ceph-deploy and ceph-disk. This tools are abstractions that need root to mount stuff and do all the work. Here I show how to do a minimal setup without needing root.
Keyring setup For some reason even with no authentication the monitor setup wants a keyring. So just set one up:
$ ceph-authtool --create-keyring keyring --gen-key -n mon.
$ ceph-authtool keyring --gen-key -n client.admin
Monitor setup Monitor setup by hand is easy:
$ mkdir $mon_data
$ ceph-mon -c ceph.conf --mkfs --fsid $(uuidgen) --keyring keyring
After that just start it:
$ ceph-mon -c ceph.conf
$
OSD setup First properly add the new OSD to the internal state:
$ ceph -c ceph.conf osd create
$ ceph -c ceph.conf osd crush set osd.0 1.0 root=default
Then setup the OSD itself:
$ mkdir $osd_data
$ ceph-osd -c ceph.conf -i 0 --mkfs --mkkey --keyring keyring
And start it:
$ ceph-osd -c ceph.conf -i 0
starting osd.0 at :/0 osd_data $osd_data $osd_data/journal
$
Health check The health check should return ok after some time:
$ ceph -c ceph.conf health
HEALTH_OK
$

Joey Hess: a rope ladder to the dgit treehouse

Ian has said something along the lines of dgit being a private club; if you're not in the club the strange git-centric views of its members don't affect you. With that said, dgit is now a quite usable and nice tool and we would like more people in the club. To that end, here's one way to get into our little world where the Debian source repository is really a collection of git repositories. I recently made a new package and added it to Debian, and here is every command I typed to do it (other than vim, lintian, and dpkg -i). As you read through, notice that every command has to do with git in one way or another. git clone https://github.com/blake2-ppc/git-remote-gcrypt This software makes git be able to push to encrypted git remotes. Very neat stuff. git archive --format=tar origin/master -o gzip > ../git-remote-gcrypt_0.20130908.orig.tar.gz Like most software on github, this one has no upstream tarball; its git repository is its native source format. Needing to manually generate this tarball appendix is the only unpleasant part of the process. git add debian
git commit -m debianization Had to write 48 lines of control file and stuff to make the package build. dgit build -tc This just runs dpkg-buildpackage. But you need to commit all files before running it. dgit push --new The --new is needed because this is a new package. This created the git repository for the package on alioth and pushed my changes and the upstream history to it, as well as uploadng the package to incoming.

8 September 2013

Gregor Herrmann: RC bugs 2013/36

my tiny last contributions to the perl 5.18 transition around RC bugs, before it hit testing tonight (thanks to the release team for their great work!): there are still quite a few bugs left concerning perl 5.18, unfortunately

Joachim Breitner: You-Say-First has dice

A while ago I wrote a small web application called You-Say-First. It allows multiple users to join a chat room where they can, besides chatting, enter moves that are only shown to everyone once everyone has submitted one. This way you can play games like Rock-Stone-Scissors or Diplomacy online, even if you want to play by your own strange rules that none of the standard, game-specific web sites support. Or even make up many kind of games on the spot!
I was approached by Jason Worley who said he liked the site, but would have a need for dice rolling to play the games he wants to play. So he gets dice rolling: Just say, in the chat, what kind of dice you want and the server will roll them for you and tell everyone about them. I tried to make it understand many variations of the command, so you can say roll a die, please , toss five coins or throw 3D20 . If the server does not react, try modifying your command. As always I m happy about feedback, bugreports and patches, and I hope that it is a useful tool for you.

Paul Tagliamonte: libolla (src:olla) in experimental

It s been a long time since I last talked about libolla, but it s one of those things I use everywhere and never bothered to
package up and upload. Well. I packaged it. And uploaded it. Finally. Now, let s look at the new quickstart guide: (this assumes you use bash, since I decided to abuse bash_completions to insert an LD_PRELOAD. I know this is gross, but I totally need a better idea) Now, look on in dumbfounded obviousness as I show you what doesn t work:
$ ssh master.d.o
ssh: Could not resolve hostname master.d.o: Name or service not known
Sadsies. Now, let s make it work:
$ ssh master.d.o whoami
ssh: Could not resolve hostname master.d.o: Name or service not known
$ sudo apt-get install -t experimental olla
# (restart your terminal)
$ ssh master.d.o whoami
paultag
Thank you, thank you! I ll be here all week! Anyway. Have fun, kids! Remember, hacks are fun!

7 September 2013

Daniel Pocock: Evacuating Australia

With the news that Australia's 100mbit (and potentially gigabit) fibre to the home is to be abolished and replaced with a cheap 25mbps service using Europe's ADSL hand-me-downs (which will be obsolete in 5 years when the deployment is finished), I've had quite a few emails from people asking just how easy it is to move abroad.
The acoustic modem has a nice retro look Word has got out that I've been spending some time in Switzerland enjoying the world-beating rate of IPv6 deployment. That is not the only reason to leave Australia of course, nor is it the only criteria for people choosing which country will give them a better deal than either of the two non-choices we had in Australia's elections. A free $20,000 for leaving Australia The first good news is that any recent graduate with a big bad ugly HECS bill is only obliged to pay this extra tax when they earn a salary from an Australian employer and file an Australian tax return. Australians who live abroad are not automatically obliged to file tax returns and are not slogged with this extra tax for having attended university. If somebody just graduated and has this hanging over their head, or if they've just started working and seen how much is being deducted on the payslip, moving abroad could make a lot of sense financially. If you have any moral qualms about this, just reflect on the fact that while the Government intimidates students with the feeling that they have to "pay" for fulfilling an education that makes them into a productive member of society, thousands of would-be athletes are given money with no obligation to repay it. The easiest route: a second passport One in four Australians has a parent born abroad. Most of those people qualify for a passport from their parent's country of birth. In some cases this also applies to people with grandparents born abroad. With the European Union now spanning 28 countries and guaranteeing the rights of any citizen from any country to live, work and study in any of the other 27 countries, a passport from any of the EU countries gives a lot of choices, including places where I have lived myself such as the UK, Ireland, France and Switzerland. Europe's citizen-friendly immigration In other words, an Australian with an Italian parent can go up to the Italian embassy in Canberra, collect an Italian passport and immediately fly into London and start working there with no bureaucratic delays, invasive and degrading medical checks or other Government interference in their life.
London's nightlife is popular for Australians. The UK started rolling out their high speed 21CN many years before Australia started the NBN. The rights also transfer automatically to all family members (spouse and children) - an Australian with a second passport is not restricted to living in Australia simply because they are married to somebody who does not have a second passport. As long as they arrive together at a European airport, they can simply show a marriage/birth certificate (in any language) and cross the border immediately, enjoy the same rights as a citizen, including work, study, healthcare and better broadband. No paperwork is necessary, no cottage industry for otherwise unemployable immigration agents, these are basic human rights for those living in Europe.
Eiffel tower: potentially part of the wireless network for France's new 4G internet, potentially two to four times the fixed-line speed planned for Australia's crippled broadband Uncle Sam wants more Australians Although it is not widely known, the US has a fast lane for immigration from Australia. The regular programs for skilled visas in the US are usually exhausted every year, whereas 10,000 "E3" class visas for Australians are never fully taken in any year. Surely it's not that easy to go to the US? It is. To qualify, an Australian only needs a university degree and a suitable job offer. Although the visa is for 2 years, it can be renewed and Australians who meet and marry an American have in various cases have gained full residence or citizenship. Anything is possible, you could even end up becoming a NASA Astronaut.
San Francisco, home to leading technology professionals and innovators from many countries. Some US cities such as Austin and Kansas City have let Google install fibre to the home, San Francisco could follow Working holiday schemes are not just for Britain Most Australians know somebody who went to London on a "working holiday" visa. These visas are not just for the UK and Australia has similar reciprocal programs with many other countries too. These rules for these schemes are rooted in the industrial age concept of workers with punch cards. Participants are typically limited to work 20 hours per week. Of particular interest for those in IT, such rules are hard to quantify for freelancers, web designers or creative people who earn their living charging fees per-project or undertaking management work without specified working hours. Although the visas are limited to 4 years, I've heard of Australians benefiting from the open borders of mainland Europe, spending 4 years in one country and then obtaining another 4 year visa for an adjacent country and travelling freely back and forth, building up a business and lifestyle of their own design. Singapore and Hong Kong Singapore and Hong Kong have become enormously popular destinations for Australians, particularly those with special skills in IT, finance, engineering and international business.
Singapore is respected as one of the cleanest and most orderly cities in Asia while embracing significant diversity in dining, shopping, leisure and lifestyle possibilities. They have recently boosted speeds to help people get a full 100MBit over fibre. A SIM card with high speed broadband charges SGD 7 (about $5) for 1 Gigabyte. These are not necessarily destinations that a graduate can easily move to. People are measured by their salary and the Government simply assumes that anybody earning a high salary is going to make a valuable contribution to society and they are given a visa in 1 or 2 weeks. Usually 3-5 years employment experience is required before making a move to one of these countries. The Not Ordinarily Resident (NOR) scheme gives some hints about the minimum salary level that ex-pats aim for, many earn significantly more with bonuses, stock programs and properly engineered tax-free offshore pension arrangements. These countries offer particularly low taxes (around 10%) and great technology. They have some rough edges too: like Europe, spouses are automatically given a visa too, but in contrast, their visa does not permit them to work unless they also work in a profession that is highly paid. Visas are issued quickly but they are also canceled very quickly if the job is terminated or the company goes bust. There have been various stories of people paying the typical 2-month deposit on an appartment in Singapore and then finding that it is forfeited when their job and visa are canceled and they have 4 weeks to leave. The bottom line is that people moving to these locations usually have a wide professional network and confidence to change their job quickly if necessary. Freestyle immigration and the visa run Not too long ago I was at a house party in Prague, Czech Republic, which is like a Paris of eastern Europe. Somebody introduced me to some Americans who I was told had similar interests in technology. They were operating an offshore web site and living very well. They had spent 10 years in Prague living on tourist visas. Authorities seemed quite happy for them to remain as long as they left every 3 months and spent 1 night abroad, a typical visa run. That could be as simple as a train journey to Vienna, Austria and an overnight stay.
Making a visa run in comfort and style with European railways. The Railjet train pictured includes free wifi, a lot cheaper than the $20 billion Tony Abbott plans to spend rolling out yesterday's technology in Australia When European countries abolished border controls, this system became much more convoluted for those who depend on the visa run. However, it remains common in many Asian and Latin American countries and many ex-pats live this way. Officials in many of these medium sized economies recognise the value of people bringing in money from offshore business (such as web sites) and using it to pay rent, food and local services. To make this effective, the ex-pats typically need to have an international private health insurance policy (high quality offshore policies can be found quickly with a search engine) and not aspire to work in a normal 9-5 job. Conclusion Young Australians might not want to just consider travel, it may even become unavoidable. The reality is, by 2020, one in three Australians will be a retired baby boomer and one way or another, whichever Government is in office will be unable to avoid taxing those of working age to pay for the health care and pension promises made to older generations. Serious expenditure on public infrastructure such as broadband or high speed rail appears to remain a pipe dream. Moving abroad, however, is just so much fun. Look at the way multinational companies make countries compete to attract their new factories and other projects: why should any individual citizen be any less demanding in choosing where they work, play and get the best value for whatever taxes they have to pay? Some people raise questions of patriotism or responsibility to the nation. Think again. The 200,000 Australians who live abroad or travel extensively make a significant contribution to Australia's international profile, engage in new trade initiatives, bring skills back to Australia and generally represent what our country is really about much better than some of those who are paid to do so.

Sylvestre Ledru: Include What You Use - Yet a new cool tool in Debian.

iwyu (Include What You Use) has just been accepted in Debian. This tool, based on LLVM/Clang, checks the headers of a C/C++ file. It will detect which include directives are useless, what could be improve and add comments to explain why #include are there. Let's take this C file:

#include <stdio.h>
#include <malloc.h>
#include <limits.h>

int foo(void)
fprintf(stderr, "an error\n");
return INT_MAX;

$ iwyu foo.c will return:

foo.c should remove these lines:
- #include <malloc.h> // lines 2-2

The full include-list for foo.c:
#include <limits.h> // for INT_MAX
#include <stdio.h> // for fprintf, stderr
---
Package on Debian

Uwe Hermann: Using mdadm to recover from a dead disk in a Linux RAID-1 array

2.5
Yes, it's that time of the year again. A disk in my desktop-replacement laptop with 2 disks and a RAID-1 has died. Time for recovery. This laptop has been running 24/7 for the last 3 years or such, so it's not too surprising that a disk dies. Surprisingly though, for the first time in a long series of dead disks, smartctl -a does indeed show errors for this disk. Here's a short snippet of those:
  $ smartctl -a /dev/sda
  [...]
  Error 1341 occurred at disk power-on lifetime: 17614 hours (733 days + 22 hours)
   When the command that caused the error occurred, the device was active or idle.
   After command completion occurred, registers were:
   ER ST SC SN CL CH DH
   -- -- -- -- -- -- --
   40 41 02 1f c0 9c 40  Error: UNC at LBA = 0x009cc01f = 10272799
   Commands leading to the command that caused the error were:
   CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
   -- -- -- -- -- -- -- --  ----------------  --------------------
   60 f8 08 20 c0 9c 40 00  41d+01:51:50.974  READ FPDMA QUEUED
   60 08 00 18 c0 9c 40 00  41d+01:51:50.972  READ FPDMA QUEUED
   ef 10 02 00 00 00 a0 00  41d+01:51:50.972  SET FEATURES [Reserved for Serial ATA]
   ec 00 00 00 00 00 a0 00  41d+01:51:50.971  IDENTIFY DEVICE
   ef 03 45 00 00 00 a0 00  41d+01:51:50.971  SET FEATURES [Set transfer mode]
  SMART Self-test log structure revision number 1
  Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
  # 1  Short offline       Completed: read failure       90%     20511         156170102
  [...]
The status of the degraded RAID array looks like this:
  $ cat /proc/mdstat
  Personalities : [raid1] 
  md1 : active raid1 sdb7[1]
       409845696 blocks [2/1] [_U]
  md0 : active raid1 sda6[0] sdb6[1]
       291776 blocks [2/2] [UU]
The [_U] means that one of two disks has failed, it should normally be [UU]. There are two RAID-1s actually, a small md0 (sda6 + sdb6) for /boot and the main md1 (sda7 + sdb7) which holds the OS and my data. Apparently (at first at least), only sda7 was faulty and got kicked out of the array:
  $ dmesg   grep kick
  md: kicking non-fresh sda7 from array!
Anyway, so I ordered a replacement disk, removed the dead disk (I checked the serial number and brand before, so I don't accidentally remove the wrong one), inserted the new disk and rebooted. Note: In order for this to work you have to have (previously) installed the bootloader (usually GRUB) onto both disks, otherwise you won't be able to boot from either of them (which you'll want to do if one of them dies, of course). In my case, sda was now dead, so I put sdb into its place (physically, by using the other SATA connector/port) and the new replacement disk would become the new sdb. After the reboot, the new disk needs to be partitioned like the other RAID disk. This can be done easily by copying the partition layout of the "good" disk (now sda after the reboot) onto the empty disk (sdb):
  $ sfdisk -d /dev/sda   sfdisk /dev/sdb
Specifically, the RAID disks/partitions need to have the type/ID "fd" ("Linux raid autodetect"), check if that is the case. Then, you can add the new disk to the RAIDs:
  $ mdadm /dev/md0 --add /dev/sdb6
  $ mdadm /dev/md1 --add /dev/sdb7
After a few hours the RAID will be re-synced properly and all is good again. You can check the progress via:
  $ watch -n 1 cat /proc/mdstat
You should probably not reboot during the resync (though I'm not 100% sure if that would be an issue in practice; please leave a comment if you know). Also, don't forget to install GRUB on the new disk so you can still boot when the next disk dies:
  $ grub-mkdevicemap
  $ grub-install /dev/sdb
And it might be a good idea to use S.M.A.R.T. to check the new disk, just in case. I did a quick run for the new disk via:
  $ smartctl -t short /dev/sdb # Wait a few minutes after this.
  $ smartctl -a /dev/sdb
  [...]
  SMART Self-test log structure revision number 1
  Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
  # 1  Short offline       Completed without error       00%        22         -
  [...]
Looks good. So far.

Julian Andres Klode: Review: ThinkPad X230

This week, a few hours after Lenovo announced the X240, I bought an X230. Normally, the X230 model I bought comes with a Core i5-3320M CPU, 4GB RAM, 500GB HDD. My model was a special set including a second 4GB RAM stick and a 128 GB mSATA Plextor SSD. It came without Windows; and the ThinkVantage button is black instead of blue and has no label. I put a fresh installation of unstable on it and tweaked it to save more power when on battery (enabled RC6++, and enabled autosuspend and other powertop suggestions with a script in /etc/pm/power.d); and configured hdparm.conf to put my hard disk into standby after 5 seconds (it s only used for big data anyway, so most of the time it is unused). It now consumes 5W in idle with minimum brightness, and 8-10W with brightness 13 of 15. Consumption when surfing is 10 15 watts. Booting from grub to gdm is fast, I did not measure it, but it probably took about 5 seconds. The IPS screen looks really good. Much much much better than the screen in my 2010 Edge 15 (I reviewed that one in 2010). It seems a bit more glossy than that one, but still matte. Keyboard is good as well. The touch pad is crap however. All hardware seems to work. Comparison to the X240 for others who think about buying one of them: The X240 is lighter and thinner (it s an Ultrabook) and has an optional FullHD 12.5 inch screen. It also offers a much larger touchpad and palm rest. But compared to the X230, the X240 lacks many things: No dedicated buttons for the trackpoint (you need to use the click-pad), it s limited at 8GB RAM, uses a slower low-voltage Haswell CPU, and it uses the new M.2 connector (probably supporting only the shortest cards) instead of mini-PCIe, so it s not really possible to add an additional SSD currently; as M.2 SSDs do not seem to be available yet. I also do not know whether the X240 offers ThinkLight or LEDs for hard drive activity and similar.
Filed under: General

Ben Armstrong: Wifi roaming on the move redux

It has been nearly six years with a netbook and five since I last wrote about wifi roaming from the bus to stay on irc without a costly celluar link during the daily commute. Since then, some readers have asked me to share my refinements to the method in a followup post. So here it is. The software On the server: On the client: Putting it together: on the client Make sure if you have a wireless manager installed (such as NetworkManager) it is configured to skip your wireless interface, disabled entirely, or if possible, removed. Set up /etc/wpa_supplicant/wpa_supplicant.conf and /etc/network/interfaces for roaming, as per the instructions in /usr/share/doc/wpasupplicant/README.modes.gz. Don t forget to add yourself to the netdev group if you are not in it already. In /etc/wpa_supplicant/wpa_supplicant.conf, list common names of open networks. Normally the catch-all network that associates with any essid, i.e. the first stanza below, works well. However, occasionally the strongest signal is neither one of the common networks nor an easily accessible network (e.g. web portals), so having a list of common open networks helps to quickly select from among those instead. The more you travel, the more of these will discover and add. Just use reconfigure from wpa_cli to reload your edited list each time you add a new one.
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
network= 
        key_mgmt=NONE
 
network= 
        ssid="default"
        key_mgmt=NONE
 
network= 
        ssid="linksys"
        key_mgmt=NONE
 
...
Since you ll be using ssh repeatedly to connect and it has to be fast, make sure your server is set up to accept your key and use ssh-add so that you only have to enter your ssh key password once. You can tweak isc-dhcp-client to make connections faster. In /etc/dhcp/dhclient.conf, use:
backoff-cutoff 1;
initial-interval 1;
Here are a few scripts I wrote to facilitate quick roaming from one open AP to another and reconnect to irssi running in screen, to break a connection and try the next one, and to recover from occasional lockups (more about that later). ~/bin/screen_reconnect This is a script to reconnect continuously via ssh to a screen session:
#!/bin/sh
reset
while ! ssh -t 10.9.8.7 'screen -UDr' 2>/dev/null ; do echo -n "." ; sleep .1 ; done
Just substitute the IP of your own server here. Using an IP instead of domain name makes the connection faster because a DNS lookup is not required. ~/bin/wifi_reassociate This script closes any open ssh sessions and informs wpa_supplicant to attempt to connect again.
#/bin/sh
/sbin/wpa_cli rea
killall ssh >/dev/null 2>&1
~/bin/wifi_killall This optional, somewhat ugly script addresses an issue I hope you never have. On my ASUS Eee PC 1001PX, occasionally scanning stops. When this happens, and I have never figured out why, apparently ACPI events are blocked. At this point wifi becomes unusable and ACPI sleep is inhibited. By trial and error I found that if you bring down the interface, kill all network-related processes, and bring it up again, ACPI events are unblocked and wifi is usable once more (and any pending request to sleep will finally happen). The script requires sudo, and to use the openbox key binding, gksudo.
#/bin/sh
sudo ifdown wlan0
# in case any of these are hung
sudo killall dhclient3
sudo killall wpa_cli
sudo killall wpa_action
sudo killall wpa_supplicant
# in case any of these are *really* hung
sleep 1
sudo killall -9 dhclient3
sudo killall -9 wpa_cli
sudo killall -9 wpa_action
sudo killall -9 wpa_supplicant
sudo dhclient -r
sudo ifup wlan0
Openbox Since certain actions need to be performed repeatedly and quickly, it is useful to have hotkeys bound in your window manager to the scripts. In ~/.config/openbox/rc.xml, key bindings for <alt>-r to reassociate and <alt>-d to disconnect a hung connection would look like:
  
<keyboard>
  <!-- My keybindings -->
  <keybind key="A-R">
    <action name="Execute">
        <execute>/home/synrg/bin/wifi_reassociate</execute>
    </action>
  </keybind>
  <keybind key="A-D">
    <action name="Execute">
        <execute>gksudo /home/synrg/bin/wifi_killall</execute>
    </action>
  </keybind>
</keyboard>
Putting it together: on the server There is very little to do here. Just start screen, and start irssi in screen. Running screen on the client as well as the server means you should either bind the screen meta keys to a different key sequence on each system, or else learn to press meta twice to pass through meta to the server screen as needed. I use the latter approach. Alternatively, you could use a tabbed terminal on the client, or separate terminals per client process instead of screen. This is a matter of personal taste. Ready to roam Here is a typical setup for roaming on the bus: In a terminal (I use urxvt), first ssh-add, then start screen with these three processes running in separate virtual terminals: March of the dots Most of the commute, just enjoy watching the dots march by, waiting for a new connection. If you estimate a connection is unusable, press <alt>-r to reassociate immediately, giving the next network a chance. If the connection is already firmly established, this might not work on the first try. If the dots don t resume immediately, wait a bit and press it again. This might take a few tries. Changing selected networks on the fly Use wpa_cli when you need to do some fine-tuning of network selections on the fly. While normally you can just watch the march of the dots until a connection is acquired, sometimes you can improve your chances of connecting to a good network by manually controlling the selected candidate networks here. For example, by watching the speed of the bus relative to known good APs, you can predict which networks are more likely to succeed. Rather than connect to any arbitrary network, you might select a specific one by id, and then later when it goes out of range, revert to the original configuration, e.g.
> select_network 5
...
> reconfigure
You can use tab-completion in wpa_cli to type these commands quickly or else just abbreviate the commands. Another common scenario is when you pass through a business area with many captive portal hotspots. These rarely make good choices because they either require a password not known to you or else you can t click through I agree in time before the bus moves on. In this case, you might just disable the catch-all stanza and let the common open network stanzas you listed ( default , linksys , etc.) do the work:
> disable_network 1
Become a type ahead wizard While running, a continuous stream of periods fills the screen, which provides you with a highly visible cue that no available APs are in range. When the movement stops, you know a connection is being attempted. While waiting to connect, you can type ahead any comments you want to make in the current irssi window (taking care to remember which one you are in!) While having periods interspersed in what you type may be disorienting at first, you get used to it. There is a point when a connection is first established and ssh is accepting input, but anything you type can no longer be seen while you re typing. Depending on whether the connection was completely successful or not, what you type now may or may not finally be sent. For best results, only type ahead before the dots stop moving. Eventually you can become skilled enough at this to type ahead a comment in one channel, switch channels with /win # and continue typing ahead in the new channel, all buffered until the next few seconds (or even fraction of a second) of connection time. Fine-tune antenna direction with wavemon When the bus has come to a standstill, you may find wavemon useful to pull in a weak signal. Because wavemon has continuously updated signal level and link quality bars, you can use it to fine-tune the antenna position. Just turn your laptop until the bars are at their maximum. Captive portals I have not figured out how to do any automation for this, so it really is a crapshoot, as it is likely the bus has moved on by the time you ve managed to manually navigate the login through a captive portal. But in rush hour, you may have the luxury of time to connect to these as you pass them. I have recently learned about the CoovaFX Firefox plugin which automates logins to captive portals. I m going to give it a try to see if it helps. Update: I can t recommend this plugin, as it is not compatible with Iceweasel >= 23.0. Also, the standard it is based on, WISPr, appears to have an uncertain future. That, coupled with the fact that the plugin appears to not be open source means I m still looking for alternatives. Summary If all of this sounds a bit nuts to you, well, it probably is. But after half a decade enjoying free access to irc from the bus, it all seems perfectly natural to me! If you try this method and like it, please let me know in the comments. Likewise, if you have any improvements to the process or scripts, please share them!

Wouter Verhelst: Debconf Mechelen: first meeting (service announcement)

To all who might be interested: The first meeting for organizing Debconf in Mechelen will happen next sunday, at 3:00 PM, in the NixSys offices in the Battelsesteenweg 455e in Mechelen, Belgium. It looks like some of those who were interested might not be able to make it at that that time. If you're one of them, and you would like your availability to be considered for the next meeting, please send me an email with your preferences (that meeting should probably happen in about a month or so). (First point on the agenda: set up a mailinglist, to make this sort of communication somewhat easier ;-)

Russell Coker: The 2013 Federal Election

picture of rubbish left after the federal election Seven hours ago I was handing out how to vote cards for the Greens at the 2013 Australian Federal election. I was hoping that either we would have a Labor/Greens coalition or an outright majority for Labor. Unfortunately we got a Liberal majority in the lower house and it looks like some extreme right wing groups may get into the senate (replacements for Family First the anti-Gay party). For some reason the polling station where I was working only had volunteers from the three major parties (Greens, Labor, and Liberal) while other polling stations in the same electorate had volunteers from smaller parties such as the Sex Party and the Socialist Alliance. The volunteers from the Liberal party ate McDonalds outside the polling station and afterwards McDonalds rubbish was left on the ground, the above picture isn t particularly clear because I took it after 6PM when the polls closed. The Liberals didn t care enough to put their rubbish in a bin, it s an externality for them, if they get enough seats in the senate they will surely take the same approach to governing Australia. The Labor people didn t take the effort to clean up the Liberal mess even though it wasn t particularly difficult to do so, I think that s the type of attitude that led to this election defeat. In the case of the McDonalds rubbish in question I put it in the bin so that when the primary school kids return on Monday their school won t be too messy after the election. But in the case of the mess that is being made in Australian politics it will take many more Greens votes to allow us to clean it up.

6 September 2013

Sylvain Le Gall: OUnit 2.0 progress, August 2013

After a long pause, I have resumed my work on OUnit2. It is going quite well. The basic features: I still need to do the following: There is still a lot of work, but the current results are already quite good. The speed improvement of the processes runner is a good thing to shorten test timing (HINT: tester needed!). Focus on: the new bracket. In OUnit 1, a bracket was very functional:
 bracket 
    (fun () -> "foo")  (* setup *)
    (fun foo -> ())
    (fun foo -> ())  (* tear down *)
So for common bracket, like bracket_tmpfile
   bracket_tmpfile
       (fun (fn, chn) ->
             (* Do something with chn and fn *)
The problem is that if you were using 2 or 3 temporary files, the level of indentation was high. I have decide to switch to a more imperative approach, registering the tear down function inside the test context:
   let (fn1, chn1) = bracket_tmpfile ctxt in
   let (fn2, chn2) = bracket_tmpfile ctxt in
      ....
This is shorter and clearer (albeit less functional). Focus on: non fatal section Sometimes, you want to verify a set of properties but to have a clear vision of what is going wrong, you need to do more than one assert. In OUnit1, you can do:
 assert_equal exp1 v1;
 assert_equal exp2 v2
But if exp1 <> v1, you quit immediately and you'll never know for exp2 and v2. In OUnit2, you can do:
 non_fatal ctxt (fun ctxt -> assert_equal exp1 v1);
 non_fatal ctxt (fun ctxt -> assert_equal exp2 v2)
In this new version, you will test both equality and the result of the test will be the worst failure you get (or success if both of them succeed). Help wanted If you have a long standing issue with OUnit, this is the time to submit a bug. OUnit BTS If you want to try the dev version of OUnit:
 $> darcs get http://forge.ocamlcore.org/anonscm/darcs/ounit/ounit
Patches always welcome. Special thanks to Thomas Wickham who has entirely written OUnitRunnerThreads and kickstarted the processes runner.

Daniel Pocock: Latest NSA revelations no surprise, but...

The latest NSA revelations confirm the systematic manner in which the NSA exploits known and potentially unknown backdoors to work around cryptography. This is not surprising, I speculated briefly about the OpenBSD and Windows NSA key incidents when Snowden originally came into view. Even two days ago, a detailed piece in Wired Magazine predicted many of the things that are in today's headlines., although Schneier indicates in The Guardian he had seen some of the material before publication. What we really need to know If the NSA and CIA have had these capabilities, just how much did they really know before and during the financial crisis that kicked off in 2007? Could the crisis have been averted? It was already known that the Greek parliament was wired, was the NSA involved in similar shenanigans? This could well be part of the Snowden revelations that we are yet to see. In fact, it is this last issue that could be the most damaging for "US interests". If countries (or their citizens) receive such undeniable evidence of a CIA conspiracy exposed by Snowden in broad daylight and they refuse to pay debts then it could lead to major re-alignment of international politics, alliances and treaties. Syria would be just the tip of the iceberg.

Tanguy Ortolo: WebPG, a PGP addon for web browsers

WebPG logo, i.e. GnuPG logo with a web over a spider web One problem with PGP, at least with GnuPG, is that it does not interact with the web. There used to be a Firefox addon for that, called FirePGP, but its development was stopped. So, good news, a new addon has come to fill the gap it left: WebPG, an addons for Firefox and Chrome. I am using it since a while, and it seems to work fine, being able to encrypt, sign, decrypt and check text blocks. Of course, it cannot handle PGP/MIME unless explicitly adapted to the webmail you use, but there seem to be some experimental support for GMail.

Sylvestre Ledru: About optimization flags in software

Following a thread on the Clang mailing list, I did a quick analysis of the optimization flags used to build C or C++ code. I used a rebuild result from last mid July of all Debian native packages (meaning that I don't have the log of full Java, Python or other packages without native code) and wrote a small script to count all their occurrences.
By default, deb packages uses the command "dpkg-buildflags --get CFLAGS" which will set the optimization level to -O2. However, many packages and upstream build systems do not respect such flags. On 10320 binary-only packages (in total, Debian has around 19000), the results are the following:
Optimization levelNumber of occurrencesNumber of packages
-O35597269
-Os30849145
-O013993294
-O125494809
-O29958037685
-O3106048531
-O4140216
-O500
-O6693549
Since a graphic talks more than numbers (note that the Y axis is logarithmic):
Disclaimer:
Some packages are using several optimization flags during the same build.
Some packages are not showing the full command line (see jessie release goal: verbose build logs)
The results might be approximate since I am basically grepping "-OX " on the log files.

Justus Winter: cgroupfs keeps track of processes

Tl;dr!!elfel1 Screenshot (slightly edited and annotated shell trace):
+ settrans -ca /cgroup /hurd/cgroupfs
+ mkdir /cgroup/init /cgroup/rootfs
+ echo $$ >> /cgroup/init/tasks  # $$ is 6
+ echo 3 >> /cgroup/rootfs/tasks # pid 3 is the root filesystem
+ sleep 1m & echo sleep has pid $!
sleep has pid 16
+ cat /proc/cmdline > /dev/null
+ tail /cgroup/init/tasks /cgroup/rootfs/tasks
==> /cgroup/init/tasks <==
6
16
20
==> /cgroup/rootfs/tasks <==
3
19
17
+ pstree -p
init(1)-+-auth(5)
         -cgroupfs(14)
         -ext2fs(3)-+-exec(4)
                     -null(17)
                     -pflocal(8)
                     -procfs(19)
                     -term(7)
         -mach-defpager(10)
         -root=device:hd0s1(2)
         -sh(6)-+-pstree(21)
                 -sleep(16)
Isn't she a beauty? So we bind the cgroupfs translator to /cgroup, create two cgroups, init and rootfs, move the currently executing shell script (that later execs sysvinit) into the former and the root filesystem translator into the latter cgroup. We then spawn a sleep process and cat the content of /proc/cmdline into /dev/null which will make the root filesystem start the /hurd/procfs and the /hurd/null translator. We then inspect /cgroup/ init,rootfs /tasks and find indeed all the newly spawned processes in the cgroup their parent process was in. This is accomplished by: I also filed a bug report containing my patches for the sysvinit package (#721917). This is the second bug report I filed during my gsoc, the first one was for the ifupdown package (#720531) which Andrew Shadura improved and merged on the very next day, thanks Andrew! Next week I'll continue to improve the cgroupfs translator, work on the notification prototype (hopefully fixing non-root subhurds in the process, this requires a similar notification mechanism for newly created tasks and making /hurd/proc just a little subhurd aware) and trying to get my gnumach patch into a working shape (currently the parental relation of processes is a Hurd-only concept and relies upon processes telling the /hurd/proc server that a newly created process is their child. This is automatically done if the process uses fork(2) of course, but not if it uses task_create to start a new Mach task).

Next.

Previous.