Search Results: "beh"

22 May 2012

Matthew Garrett: I've been a terrible person (and so have most of you)

John Scalzi recently wrote a piece on straight white male privilege. If you haven't read it already, go and do so. No rush. I'll wait.

So. Some facts:

(If you disagree with any of these then that's absolutely your right. You're wrong, but that's ok. But please do me a favour and stop reading here. Otherwise you'll just get angry and then you'll write something ill-tempered and still wrong in the comments and then I'll have to delete it and why not just save everybody the time and effort and go and eat ice cream or something instead)

I know I've said this before, but inappropriate and marginalising behaviour is rife in our community, and at all levels of our community. There's the time an open source evangelist just flat out told a woman that her experiences didn't match his so she must be an outlier. There's the time a leading kernel developer said that most rape statistics were basically made up. There's the time that I said the most useful thing Debian could do with its money would be to buy prostitutes for its developers, simultaneously sexualising the discussion, implying that Debian developers were all straight men and casting sex workers as property. These aren't the exceptions. It's endemic. Almost all of us have been part of the problem, and in doing so we've contributed to an environment that has at best driven away capable contributors. You probably don't want to know what it's done at worst.

But what people have done in the past isn't important. What's important is how we behave in the future. If you're not angry about social injustice like this then you're doing it wrong. If you're reading this then there's a pretty high probability that you're a white male. So, it's great that you're angry. You should be! As a straight white male born into a fairly well-off family, a native English speaker in an English speaking country, I have plenty of time to be angry before going back to my nice apartment and living my almost entirely discrimination-free life. So if it makes me angry, I have absolutely no way of comprehending how angry it must make the people who actually have to live with this shit on a daily basis.


(Were tampon mouse able to form and express coherent thoughts, tampon mouse would not put up with this shit)

The point isn't to be smugly self aware of our own shortcomings and the shortcomings of others. The point is to actually do something about it. If you're not already devoting some amount of your resources to improving fairness in the world, then why not? It doesn't have to be about women in technology - if you're already donating to charity or helping out at schools or engaging in local politics or any of the countless other ways an individual can help make the world a better place, large or small, then keep on doing that. But do consider that many of us have done things in the past that contributed to the alienation of an astounding number of potential community members, and if you can then please do do something to make up for it. It might be donating to groups like The Ada Initiative. It might be mentoring students for projects like the GNOME Outreach Program for Women, or working to create similar programs. Even just making our communities less toxic by pointing out unacceptable behaviour when you see it makes a huge difference.

But most importantly, be aware that it was people like me who were responsible for this problem in the first place and people like me who need to take responsibility for solving it. We can't demand the victims do that for us.

comment count unavailable comments

21 May 2012

Johannes Schauer: sisyphus wins ICRA 2012 VMAC

Sisyphus is a piece of software that I wrote as a member of a team from Jacobs University led by Prof. Dr. Andreas N chter. It managed to place our team first in this year's IEEE ICRA 2012 Virtual Manufacturing Automation Competition in all three rounds. The goal was, to stack a given set of boxes of different length, height and width on a pallet in a way that achieved optimal volume utilization, center of mass and interlock of the boxes. Besides the cartesian placement of a box on the pallet, the only other degree of freedom was a 90 rotation of the box around a vertical axis. Since the arrangement of boxes into a three dimensional container is NP hard (three dimensional orthogonal knapsack), I decided for a heuristic for an approximate solution. The premise is, that there are many boxes of equal height which was the case in the test cases that were available from the 2011 VMAC. Given this premise, my heuristic was, to arrange the boxes into layers of equal height and then stack these layers on top of each other. A set of boxes that would be left over or too little from the start to form its own full layer, would then be stacked on the top of the completed layers. There is a video of how this looked like. My code is now online on github and it even documented for everybody who is not me (or a potential future me of course). This blog post is about the "interesting" parts of sisyphus. You can read about the overall workings of it in the project's README. Python dict to XML and XML to Python dict The evaluation program for the challenge is reading XML files and the pallet size and the list of articles with their sizes are also given in XML format. So I had to have a way to easily read article information from XML and to easily dump my data into XML format. Luckily, all the XML involved was not making use of XML attributes at all, so the only children a node had, where other nodes. Thus, the whole XML file could be represented as an XML dictionary with keys being tagnames and the values being other dictionaries or lists or strings or integers. The code doing that uses xml.etree.ElementTree and turns out to be very simple: 
from xml.etree import ElementTree

def xmltodict(element):
    def xmltodict_handler(parent_element):
        result = dict()
        for element in parent_element:
            if len(element):
                obj = xmltodict_handler(element)
            else:
                obj = element.text
            if result.get(element.tag):
                if hasattr(result[element.tag], "append"):
                    result[element.tag].append(obj)
                else:
                    result[element.tag] = [result[element.tag], obj]
            else:
                result[element.tag] = obj
        return result
    return  element.tag: xmltodict_handler(element) 

def dicttoxml(element):
    def dicttoxml_handler(result, key, value):
        if isinstance(value, list):
            for e in value:
                dicttoxml_handler(result, key, e)
        elif isinstance(value, basestring):
            elem = ElementTree.Element(key)
            elem.text = value
            result.append(elem)
        elif isinstance(value, int) or isinstance(value, float):
            elem = ElementTree.Element(key)
            elem.text = str(value)
            result.append(elem)
        elif value is None:
            result.append(ElementTree.Element(key))
        else:
            res = ElementTree.Element(key)
            for k, v in value.items():
                dicttoxml_handler(res, k, v)
            result.append(res)
    result = ElementTree.Element(element.keys()[0])
    for key, value in element[element.keys()[0]].items():
        dicttoxml_handler(result, key, value)
    return result

def xmlfiletodict(filename):
    return xmltodict(ElementTree.parse(filename).getroot())

def dicttoxmlfile(element, filename):
    ElementTree.ElementTree(dicttoxml(element)).write(filename)

def xmlstringtodict(xmlstring):
    return xmltodict(ElementTree.fromstring(xmlstring))

def dicttoxmlstring(element):
    return ElementTree.tostring(dicttoxml(element))
Lets try this out: 
>>> from util import xmlstringtodict, dicttoxmlstring
>>> xmlstring = "<foo><bar>foobar</bar><baz><a>1</a><a>2</a></baz></foo>"
>>> xmldict = xmlstringtodict(xmlstring)
>>> print xmldict
 'foo':  'baz':  'a': ['1', '2'] , 'bar': 'foobar' 
>>> dicttoxmlstring(xmldict)
'<foo><baz><a>1</a><a>2</a></baz><bar>foobar</bar></foo>'
The dict container doesnt preserve order, but as XML doesnt require that, this is also not an issue. Arranging items in layers When it was decided, that I wanted to take the layered approach, it boiled down the 3D knapsack problem to a 2D knapsack problem. The problem statement now was: how to best fit small rectangles into a big rectangle? I decided for a simple and fast approach as it is explained in Jake Gordon's blog article. There is a demo of his code and should the site vanish from the net, the code is on github. This solution seemed to generate results that were "good enough" while simple to implement and fast to execute. If you look very hard, you can still see some design similarities between my arrange_spread.py and his packer.js code. Jake Gordon got his idea from Jim Scott who wrote an article of arranging randomly sized lightmaps into a bigger texture. There is also an ActiveState Code recipe from 2005 which looks very similar to the code by Jake Gordon. The posts of Jake Gordon and Jim Scott explain the solution well, so that I dont have to repeat it. Should the above resources go offline, I made backups of them here and here. There is also a backup of the ActiveState piece here. Spreading items out The algorithm above would cram all rectangles into a top-left position. As a result, there would mostly be space at the bottom and left edge of the available pallet surface. This is bad for two reasons:
  1. the mass is distributed unequally
  2. articles on the layer above at the bottom or left edge, are prone to overhang too much so that they tumble down
Instead all articles should be spread over the available pallet area, creating small gaps between them instead big spaces at the pallet borders. Since articles were of different size, it was not clear to me from the start what "equal distribution" would even mean because it was obvious that it was not as simple as making the space between all rectangles equal. The spacing had to be different between them to accommodate for differently sized boxes. The solution I came up with, made use of the tree structure, that was built by the algorithm that arranged the rectangles in the first place. The idea is, to spread articles vertically first, recursively starting with the deepest nodes and spreading them out in their parent rectangle. And then spreading them horizontally, spreading the upper nodes first, recursively resizing and spreading child nodes. The whole recursion idea created problems of its own. One of the nicest recursive beauty is the following function: 
def get_max_horiz_nodes(node):
    if node is None or not node['article']:
        return [], []
    elif node['down'] and node['down']['article']:
        rightbranch, sr = get_max_horiz_nodes(node['right'])
        rightbranch = [node] + rightbranch
        downbranch, sd = get_max_horiz_nodes(node['down'])
        ar = rightbranch[len(rightbranch)-1]['article']
        ad = downbranch[len(downbranch)-1]['article']
        if ar['x']+ar['width'] > ad['x']+ad['width']:
            return rightbranch, sr+[downbranch[0]]
        else:
            return downbranch, sd+[rightbranch[0]]
    else:
        rightbranch, short = get_max_horiz_nodes(node['right'])
        return [node] + rightbranch, short
get_max_horiz_nodes() traverses all branches of the tree that node has below itself and returns a tuple containing the list of nodes that form the branch that stretches out widest plus the list of nodes that are in the other branches (which are shorter than the widest). Another interesting problem was, how to decide on the gap between articles. This was interesting because the number resulting of the subtraction of the available length (or width) and the sum of the articles lengths (or widths), was mostly not divisible by the amount of gaps without leaving a rest. So there had to be an algorithm that gives me a list of integers, neither of them differing by more than one to any other, that when summed up, would give me the total amount of empty space. Or in other words: how to divide a number m into n integer pieces such that each of those integers doesnt differ more than 1 from any other. Surprisingly, generating this list doesnt contain any complex loop constructs: 
>>> m = 108 # total amount
>>> n = 7 # number of pieces
>>> d,e = divmod(m, n)
>>> pieces = (e)*[(d+1)]+(n-e)*[d]
>>> print pieces
[16, 16, 16, 15, 15, 15, 15]
>>> sum(pieces) == m
True
>>> len(pieces) == n
True
You can test out the algorithms that arrange rectangles and spread them out by cloning the git and then running: 
PYTHONPATH=. python legacy/arrange_spread.py
The results will be svg files test1.svg and test2.svg, the latter showing the spread-out result. Here is an example how the output looks like (without the red border which is drawn to mark the pallet border): arrange_spread2.py contains an adaption of arrange_spread.py for the actual problem. Permutations without known length When creating a layer out of articles of same height, then there are four strategies that I can choose from. It is four because there are two methods that I can either use or not. I can rotate the article by 90 per default or not and I can rotate the pallet or not. So every time that I build a new layer, there are those four options. Depending on which strategy I choose, there is a different amount of possible leftover articles that did not fit into any layer. The amount is different because each strategy is more or less efficient. To try out all combinations of possible layer arrangements, I have to walk through a tree where at each node I branch four times for each individual strategy. Individual neighboring nodes might be the same but this outcome is unlikely due to the path leading to those neighboring nodes being different. To simplify, lets name the four possible strategies for each layers 0, 1, 2 and 3. I now want an algorithm that enumerates through all possible permutations of those four numbers for "some" length. This is similar to counting. And the itertools module comes with the product() method that nearly does what I want. For example, should I know that my tree does not become deeper than 8 (read: no more than eight layers will be built), then I can just run: 
>>> for i in itertools.product([0,1,2,3], repeat=8):
...     print i
...
(0,0,0,0,0,0,0,0)
(0,0,0,0,0,0,0,1)
(0,0,0,0,0,0,0,2)
(0,0,0,0,0,0,0,3)
(0,0,0,0,0,0,1,0)
(0,0,0,0,0,0,1,1)
(0,0,0,0,0,0,1,2)
(0,0,0,0,0,0,1,3)
This would work if the number of layers created with each strategy was the same. But as each strategy behaves differently depending on the input, it cannot be known before actually trying out a sequence of strategies, how many layers it will yield. The strategy (0,0,0,0,0,0,0,0) might create 7 layers, resulting in (0,0,0,0,0,0,0,1), (0,0,0,0,0,0,0,2) and (0,0,0,0,0,0,0,3) yielding the same output as only the first 7 strategies count. This would create duplicates which I should not waste cpu cycles on later. It might also be that (0,0,0,0,0,0,1,0) turns out to be a combination of strategies that creates more than 8 layers in which case the whole thing fails. So what I need is a generator, that gives me a new strategy depending on how often it is asked for one. It should dynamically extend the tree of possible permutations to accommodate for any size. Since the tree will become humongous (4^11 = 4194304), already traversed nodes should automatically be cleaned so that only the nodes that make the current list of strategies stays in memory at any point in time. This sounded all complicated which made me even more surprised by how simple the solution was in the end. Here a version of the algorithm that could easily be ported to C: 
class Tree:
    def __init__(self, branch_factor):
        self.branch_factor = branch_factor
        self.root =  "value": None, "parent": None, "children": [] 
        self.current = self.root

    def next(self):
        if not self.current["children"]:
            self.current["children"] = [ "value":val, "parent":self.current, "children":[]  for val in range(self.branch_factor)]
        self.current = self.current["children"][0]
        return self.current["value"]

    def reset(self):
        if self.current["parent"]:
            self.current["parent"]["children"].pop(0)
        else:
            return False
        if self.current["parent"]["children"]:
            self.current = self.root
            return True
        else:
            self.current = self.current["parent"]
            return self.reset()

    def __str__(self):
        return str(self.root)
It would be used like this: 
>>> tree = Tree(4)
>>> print tree.next(), tree.next(), tree.next()
>>> while tree.reset():
...     print tree.next(), tree.next(), tree.next()
Which would be equivalent to calling itertools.product([1,2,3,4], 3). The special part is, that in each iteration of the loop I can call tree.next() an arbitrary amount of times, just how much it is needed. Whenever I cannot generate an additional layer anymore, I can call tree.reset() to start a new permutation. For my code I used a python specific version which is a generator: 
def product_varlength(branch_factor):
    root =  "value": None, "parent": None, "children": [] 
    current = root
    while True:
        if not current["children"]:
            current["children"] = [ "value":val, "parent":current, "children":[]  for val in range(branch_factor)]
        current = current["children"][0]
        if (yield current["value"]):
            while True:
                if current["parent"]:
                    current["parent"]["children"].pop(0)
                else:
                    return
                if current["parent"]["children"]:
                    current = root
                    break
                else:
                    current = current["parent"]
It is used like this: 
it = product_varlength(4)
print it.next(), it.send(False), it.send(False)
while True:
    print it.send(True), it.send(False), it.send(False)
Again, the expression in the loop can have any number of it.send(False). The first it.send(True) tells the generator to do a reset.

19 May 2012

Andrew Pollock: [life] Maker Faire 2012 trip report

The Maker Faire is one of those awesome Bay Area things that always fills me with excitement and gets my imagination going. Zoe and I went again this year to check it out, as best we could within the time constraints we had to work within (opening time and her nap time, minus travel time). She definitely enjoyed herself. We took the Caltrain, because historically driving and parking has been a bit of a nightmare. The optimal train to get to get there before it opened (at 10am) was the 9:19 train from Mountain View, which was scheduled to get in at Hayward Park a little before 10am. It just so happened that there was a Giants game on in San Francisco today as well, and the train was absolutely packed. We only got a seat because one kind gentleman was getting off and explicitly gave his seat to us. One lesson learned: don't try and take the BOB stroller on the train. Even when collapsed, it's way too bulky. For future Caltrain outings, I'll take our City Mini stroller instead, as it folds much flatter. I also took our macpac Possum child carrier backpack, and Zoe was pretty happy to just sit in it for the bulk of the time. I think it had novelty value for her, as we haven't used it for a while. I probably could have gotten away without taking a stroller at all. I was very glad I took the backpack, as it gave her a much better vantage point for everything that was going on than she would have gotten from sitting in the stroller. There was supposed to be a free shuttle from the Hayward Park station to the Maker Faire, but there was a huge crowd waiting for it, so I decided to just walk. It didn't take too long. For the return trip, I think I exited from the wrong side of the fairgrounds, and couldn't figure out the shuttles, so I just walked to Hillsdale station. At least the return train wasn't crowded. Overall, using Caltrain to get in and out was successful. Zoe was very well behaved for the ~30 minute train ride each way. The Faire was quite a bit bigger this year, and has spilled out into the parking lot on one side. I'd heard stories that O'Reilly had quadrupled booth prices as well. Trying to abide by the program was too difficult, so we mostly just wandered through the main Expo hall and looked at various booths. I just did a full read through the website of all the exhibitors to see what I missed out on. Here's some of the stuff I saw in person, or discovered via the website: Kickstarter is really becoming huge in the maker community. There were heaps of exhibitors there with (mostly robotics) projects that were past the initial prototyping phase and were seeking funding on Kickstarter to go into mass production. Some of the talks I'd have liked to have seen: Zoe was really well behaved for the entire expedition. I don't think she really gave me any grief at all. There was a brief period where she wanted me to carry her, but I managed to negotiate her back into the stroller after not long. I think her favourite was ArcBotics, which had a robot insect that would dance and wave at her. She kept asking for it to do more dancing.

Richard Hartmann: Mongolia

I will start linking to places we stayed at etc, from now on. On the one hand, we already left those places so my paranoid self can rest assured that no one will come hunt us down, on the other hand, I decided to only to name and link to those places which really deserve a mention. If they are linked here, I can whole-heartedly recommend them. Mongolia After leaving through the Russian border fortifications, Mongolia was an instant and welcome change. Trees growing right next to the tracks, branches forming a bit of a natural tunnel; the trains passing by keeping branches at bay, but the trees free to grow wherever else they pleased. A few hundred meters later below a rocky ledge, a cow had apparently fallen down and died. Partially decomposed flesh still clinging halfway to its rib-cage, it proved to be a sign of the (mostly) more laid-back attitude Mongolians tend to take. After finally getting our passports back and being allowed to leave the train, we first headed to the station to get some money. At ~1800:1, the exchange rate for Tugruk against Euro speaks volumes about inflation; imagine my surprise that the MNT 20,000 note is the largest one. It's also the one and only amount any and all ATMs will default to. We were quite surprised by how cheap everything was, even in the one single store within the station directly behind the border. In Russia, travellers expect to pay hefty markups when shopkeepers know they can charge them, not so in Mongolia. The woman in the trash Upon returning from the station, our waggon was just merrily being driving away on its own by a lone engine, something we had not anticipated or appreciated a whole lot. A woman (we hadn't noticed her before) who was busy scavenging a trash container for food for herself and her scrawny-looking son happily stopped what she was doing and, with a big smile, moved her hands around and smacked them back together, signalling us that our waggon would be attached to a different train for the subsequent journey. To put it mildly, I was stunned; we had met people eating from the trash in Russia as well (yes, I always make a point of giving them enough for a few decent meals; anyone eating out of the trash actually needs and definitely deserves something good happening to them), but they were glum and disheartened. Here, there was someone who not only had to look out for herself, but for a three or four year old boy as well, and still was just so... positive... I don't know if I would have it in me to act the same way... As an aside, even though we were just a few kilometers across the border from Russia, she did not actually take the money from me, she cupped her hands to receive it by having me place it into her hands; this is a very Asian thing in my experience and something we would see both in Mongolia and China consistently. Later, when we were back in our waggon, she passed by and happened to see the Spot Messgenger 2 dangling in front of our window, its lights blinking every five seconds. For a solid twenty minutes, until the train started moving again, she squatted down with her son, oggling the Spot. I have no idea what she was thinking it was and I had no chance to explain it to her, but she was really fascinated and just kept staring and staring. When we finally left the station, we waved to and smiled at each other, she ran after us, and then she was gone. All in all, this was one the more memorable things which happened to us. Mongolian Infrastructure When I say that Mongolians as a whole tend to be a lot more laid back than Russians, this extends to infrastructure as well, but not in the good way. Ulan Bator Little Dhingis Khan We arrived in Ulan Bator at six in the morning and started haggling with taxi drivers. Our hostel had (stale, it turned out) information about acceptable fare from station to their place on their website, 5,000 MNT was supposedly OK. Our driver tried to get 20,000, then 15,000, then 12,000, then 10,000, went down to 7,000 and finally settled for 5,000, all this interspersed with frequent "no" by us and walking away. Once on the road, he wanted to convince me that we had agreed upon 5,000 per person, something that did not quite work out for him. When we arrived, he started to tell me that he was good friends with the hostel manager and that said manager would surely want us to pay more; again, that did not work too well. He made the mistake of getting our luggage out of his trunk immediately (we always kept all our stuff in the passenger area afterwards...) before I paid and even though I only had a 10,000 or 20,000 note, I insisted on correct change. The same old dance began anew, with him edging closer to the correct change in steps of 1,000 MNT. He tried to get away several times, but I stood in his door and held it open; if he had just driven off, I couldn't have stopped him, but that thankfully didn't occur to him. After he shoved all small change (MNT is bill only, no coins) he had with him at me and wanted to make off yet again, I was close to throwing all the small crap at him, but after he, a man of maybe 1.65 meter and sitting down, tried to signal his willingness for a fist fight and me, a man of 1.94 meter and standing started laughing, he gave me 2,000 more and drove off cursing. In case you forgot, at that exchange rate, we argued about two to three Euro, but as he tried to cheat, it became a matter of principle. The Hostel, part one All parking lots and other property in Ulan Bator have a small watchtower with private 24/7 guards; said guard ambled down, let us in and woke the poor woman who was forced to let us check in. As she was Mongolian as well, we were a bit surprised when she switched from broken English to perfect German, but the owners of the OASIS are a German-Austrian couple so I guess that makes sense. Fighting through the war Traffic in Ulan Bator is war. Lonely Planet talks about the city being too newly motorized and that it did not have to develop a culture of driving, yet. This is a euphemism for "no one has any qualms maiming or killing you". Everyone is driving like mad, nudging and racing their way in front of each other. If a driver has an oppurtunity to get ahead by a few centimeters with his small car, thereby blocking the way for several buses, making a whole parking lot grind to a halt and thereby deadlocking themselves, they will take said opportunity, and gladly. Red lights are gentle suggestions, there to be mainly ignored; police will not drive over red lights, but they will not stop anybody else from doing so, either. A hearse will have a dozen or more cars following it, all driving in a tight pack, aggressively attacking anyone who dares to wedge in between them and expecting all other traffic to make way even if they are driving over a red light. Crossing a main street in Ulan Bator as a pedestrian is hell. There are a few traffic lights and they help to some extent, but you still need to be extremely careful when crossing a street. Thailand, India, Russia, China, no matter where it's supposedly hard to cross a street by foot, I never had any issues weaseling through. In UB, it takes planning, determination, and a lot of attention, especially as there are stretches where there's not a traffic light in sight. While maintaining eye contact with a driver usually ensures that they will try not to hit you, this does not work in the least in UB. In the evening, I took to shining a high-powered flashlight at the ground to the oncoming traffic's side to mark ourselves and decrease the likelihood of being hit. This worked well in combination with sprinting through gaps in the traffic, except for one guy who actively aimed for us and did not slow down. He even changed lanes, just to keep us in front of his car... Shining the flashlight directly at him in highest mode made him break quite violently though; a good thing, as he may have hit us had he not slowed down. UB itself Ulan Bator is weird. It has clear signs of westernization, but it's still very much an old city. There is one large store which would fit into any random shopping center in Germany (which are tiny when compared to most other countries) and which carries most goods, if with a limited selection. Its supermarket is extremely well stocked with food and drink from all other the world and laughably cheap (for us) prices. While its infrastructure is definitely crumbling, the city is clean, especially when compared to Russia. Two girls from Stuttgart who we met during our Trans-Siberian travels commented on how clean Russian cities were when compared to home, but that may be more of a statement about Stuttgart than about Russia... Anyway, in Mongolia in general and in UB in particular, there's a concise effort to make what they have look nice and clean. There is no trash in the streets, loose gravel and earth is removed from the curb by dedicated workers, all areas of packed earth are sieved and all large stones removed. All in all, it's a lot cleaner (if more dusty and sandy) than Russian cities if somewhat older and more shabby-looking. That being said, Mongolia's countryside is littered There were several street stalls consisting of nothing more than a cardboard box on a stool selling single chewing gums and cigarettes from the original packaging; it's normal for people to buy one cigarette or one piece of chewing gum and walk on. The sights of UB were nice, but nothing too special. Outlook In the next part, I'll cover "The Hostel, part disaster" and generally talk more about excrement than I thought one could write before moving on to China (were I am located at the time of this writing).

Richard Hartmann: Motherland's bosom

I read a translated poem about Russia being "the Motherland" and its vast bosom years ago. Having driven through a significant part of it, I can agree on the "vast" part... Also, as I am on a train and without access to the Internet, I will refrain from linking to a lot of pages; sorry. (Turns out I am posting this a week later, but I will still not link to stuff now; no time). Russia in general Moscow Sights Kreml Our remaining time in Moscow was spent with touring the usual suspects; the Kreml is a lot less impressive in real life, the Red Square is tiny when compared to the stories I heard about it and the Chapel ofi St. Basil is even more colorful and impressive in real life. Lenin's body was inaccessible because workers built seats for the May 9th parade to the left and the right of it and they apparently thought it would be a good idea to block access to one of the main tourist attractions while doing so. A river tour of Moscow was a nice cool-off and we got to see quite a few things. We managed to see the weekly military parade within the Kreml grounds, but it was mostly pomp and little substance. The National Treasure which you can access with an extra ticket within the Kreml grounds is nice, but less impressive than the tourist guides would make you believe. That being said... There's another museum within the museum and.... Whoah... Tourists pay extra, visitors go through the only non-security-theater check I encountered in Russia, guards are armed, people can only enter and leave in batches, and the stuff which is presented is mind-boggling. Disregarding the fist-to-calf-sized chunks of gold and platinum which are still in their original form directly from the mine, there is real, actual treasure galore. Little heaps of uncut and cut diamonds, an outline of Russia filled with cut diamonds and other random "we have this stuff" displays can be found as well. Then, you have various tiaras and other jewellery made from various gems. Not incorporating, but largely made of. All that pales in comparison to the crown, royal apple, scepter, etc. It's hard to put the amount of tiny multi-colored light points that shine at you into words. I was just standing there, swaying back and forth to catch the moving pattern of pinpoints. It's said that this collection is equalled only by the ones in the Tower of London and the one Shaw of Iran had and boy do I believe it. TV Tower Getting up there was funny. The old-style Soviet queuing system was used: "Security" for approaching the tower was multi-level, the guards see you approach along a long walkway way in advance and the main guard shed had several small cabins separated by thick glass. So good so menacing. But in a twist that would make Bizarro and Garry Larson proud, I was required, by means of metal detector gate, metal detector wand and even an x-ray machine to remove every shred of metal and other hard objects from myself and the camera bag and put them onto a table. Once I was without anything except my clothes and the bag was completely empty, I could pass. Everything I had had to remove was just laying there, not inspected in the least, for me to stuff back into pockets and bag and to take with me. This "everything" included a Spot Messenger 2 with lots of green and red blinky lights. The guard did not even glance and it. Security theater? Security theater. The view from 364 meters down on Moscow was nice, but there was a lot of Smog so I couldn't see very far. Jumping on the glass floor while looking down was a lot of fun, though. Subway to Thiefing I bet Christopher Nolan rode the subway in Moscow at least once. That unnerving sound you hear during several key scenes in "The Dark Knight"? Two thirds of all subways make the same sound while moving. Also, I had an encounter with a pickpocket down there; very classical, too. Guy approaches quickly, talks loudly and sounds as if it's really important (in Russian... duh... that's sure to keep me interested). His approach made me turn and protect my left leg pocket automatically, most likely marking the target for the tiny woman standing behind me. Now, I have to tell you something about my usual travel layout. As my normal pockets are very deep, it looks as if their content was in the leg pocket. Plus, there's an extra, hidden leg pocket where I keep the passports and train tickets. The outermost leg pocket is protected by a velcro flap, but it contains nothing of value; usually the appropriate phrasebook, local map, maybe a tissue or chewing gum. Due to this layering, the outermost pocket looks as if it's full to the brim with stuff. Also, I took pains to make it a habit to protect said leg pocket with my hand, nothing else. This looks as if that's the target, but what I am actually doing is protect my normal pocket with my forearm. The right side is different, but the most easily accessibly pocket always holds some small change. I pay from that stash but my actual wallet is well out of reach. Anyway, once the guy ran off, talking to several others, most likely marking all them for the actual pickpockets, I wanted to enter the subway. While the Russian-style queuing took place, I felt an unusual tug at the velcro flap. I looked down and saw a tiny woman to the left of me with a jacket held over her right side with the left arm; I look up to check no one is trying to steal from my permanently assigned female, feel another tug, look the woman into the eyes, look up again and around me, look down again and she is gone. All that took maybe three seconds and I had boarded the subway after an additional two. In hindsight, it makes sense to choose the time of entry for attack. It's crowded, you are being pushed around, and once you are in the subway, it will start moving more or less immediately while the thief remains in the station. In this case, she would only have gotten a grubby map of Moscow's subway and an English-Russian phrasebook, but she got nothing at all. Moscow-Novosibirsk Where to begin... If you think a few hours on a train are a long time, try over fifty hours. Things get so bad, you start getting land-sick while not in a moving train. You even start missing the familiar tunk-cachunk, tunk-cachunk, tunk-cachunk... of driving over rails with gaps in them when you are not moving. The defining element of the Trans-Siberian Railway are birch trees. And birch trees. And then more birch trees. You would not believe how many birch trees there are. This is made "worse" by the way the Russian Railway protects their rails. Left and right of the track, there's a cleared area of maybe ten to twenty meters, sometimes as little as three. Outside of that, they plant ten to twenty meters of birch trees, presumably to catch snow during winter. Beyond that protective perimeter, there's the normal landscape.As a result, on top of the near endless stretches of birch woods, you see most if not all scenery through a layer of birch trees. You get sick sick of birch trees after a few hours and you see them for days on end. Bullet points to save myself some typing and you some reading... Novosibirsk The non-existent hostel We arrived at ~0200 local and made our way to the hostel we had booked a room with. Walking to the correct address, we saw several signs but they all turned out to be for a police station and some other state agency. We walked back, forth, double-checked, triple-checked: no hostel. We then walked around the building through some not-quite-nice back alleys, but other than a few entries to private flats, there was nothing. Thankfully, the booking slip included a number which we called and after at least twenty rings (no kidding), when I had given up and wanted to hang up, it stopped ringing. Dead silence. After maybe ten seconds, someone started talking in Russian. I asked him if he spoke English and told him that we could not find the hostel. He mumbled something about being sorry and that we should wait, he would come down. Fast forward a minute or two and someone walked towards us. Again, he mumbled about being sorry, that the hostel "did not work" at the moment and that we would need to sleep in his private apartment. He ushered us into some back alley entrance, into his flat, and proceeded to remove the sheets from the couch on which he had slept; after putting on new sheets, we had our "hostel" bed, ready to sleep on. We briefly considered if he would murder us in our sleep, but him and me even got to talking a bit. Over cheese, sausage and rum (at 0300), he admitted that the hostel did not exist and he merely planned to turn his flat into a hostel for the summer while he and his family moved into their summer house (the Russian term of which escapes me, at the moment) in the countryside. He had accepted our reservation as he thought he would be finished by that time. He did not even get started, though. While he sent us an overbooking notice through booking.com two days before, we were on the train at that time, so... booking.com even called him to check what happenend to us as we did not book another place through them. Good customer service/protection, that. Next morning, he didn't even want to take our money (we paid anyway) and, as a means of compensation, drove us into the city in the morning and to a train museum well outside the city limits, one of the fabled scientist cities, and a large lake which everyone in Novosibirsk claims is an ocean, in the afternoon. Foreigners, foreigners! All in all, Novosibirsk was relatively uneventful, safe for one bizarre episode. We took our lunch in a local fast food joint (why do all the good stories happen there, and not at the various truly local places?) and threw the cashier our well-rehearsed "Niet Russkie; anglisky?" with phrasebook in hand and he actually understood a few words of English (beef, chicken, fries). We told him, in our worst Russian, that we are from Germany wished him a nice day and went to sit down. A few minutes later, a girl approached us, literally hopping from one foot to the other and wringing her hands. She told us that the cashier had told her that we spoke English and if it would be OK if she talked to us. We suspected some sort of elaborate ruse, but went with it. Turns out, she had English at school and really wanted someone to practice English on. Two young men passed our table and exchanged a few words with her, sitting down out of sight. When she told us that she had to leave now but if it would be OK if the two boys joined us we suspected a ruse yet again. But those two were law students, one with a minor in English and one with a minor in German; both of them also extremely nervous, asking us if we would talk to them. When they had to leave, they told us that the three of them worked at the burger joint and that their shift was just about to start when the news that foreigners were here spread amongst staff like wildfire. The girl stopped by several times in between cleaning tables, getting in a sentence or two before being cussed at by her supervisor. All in all, this took about twenty minutes and seeing three people so nervous and grateful to talk with us felt beyond absurd. On the other hand, not a single traveller we met even considered stopping in Novosibirsk during their transit so there really does seem to be a shortage of non-Russians there. Weird, and memorable. Novosibirsk-Irkutsk Irkutsk / Listvianka / Lake Baikal Listvianka Aah, lake Baikal... the oldest and deepest lake on Earth which holds a fifth of the global non-salt water reserves; a must-see in my book. Quad tours at break-neck speeds, dry-suit diving with Russian regulators, walking barefoot in between and across drift ice that made its way onto the shorei, and extended hiking around the lake's coast... All of which I could not do because I was ill and had to spend two solid days in bed. The draft from the open window in between Novosibirsk and Irkutsk was enough to give me a rather bad cold which peaked at Lake Baikal. Still, the area was lovely and we were glad to be out of a train and able to unpack our stuff without having to repack immediately for once. I am not sure where my current losing streak with regards to diving is coming from (Grimsey, diving north of the Arctic circle with birds that plummet into the water and hunt fish: Only guy who does this is on the Icelandic mainland that day; Svalbard, diving north of the Arctic circle in permanent darkness: The few people who do this privately did not reply while I was there; Baikal, oldest, deepest, largest lake on Earth: ill), but I will most likely return to Russia for a week of ice diving in Lake Baikal next winter or the one after that. As an aside, I saw several people walking to Lake Baikal with buckets to get their water. Other people got it from a well which was still half frozen. If you have running water consider yourself lucky... Irkutsk Nice city, largely uneventful. The farther east you get within Russia, the more normal women look. In Moscow, just as in Paris, they are way over-dressed and even service personnel will walk with high heels. Thankfully, I don't have to wear heels, but for the other males out there: Walking and standing in these things hurts and thus most if not all people who stand and walk for a living have flat shoes. We happened upon preparations for a military parade, complete with cordon, viewing podests, at least half a dozen TV cameras etc, but were not sure if it would start soon enough for us to catch our train.We asked someone who told us it would start at 2100 local, at 1945 local it seemed about to start, and sure enough at 1955 sharp, the whole thing went under way. About a dozen groups of 50-100 people each, all in their own, respective uniforms stood against one side of a cordoned-off street and several higher-ups on the other side. Two highest-ups shouted into microphones and the throng of people on the other side shouted back answers. Then, the two highest-ups stood in the back of a jeep each and drove past said throng, stopping in front of each group, shouting into microphones mounted in the back of the jeeps and the groups shouted back once again. After that, all groups marched around the make-shift plaza once, saluting the higher ups. Once they were done, and they took ages, two trucks drove by with soldiers jumping out of the moving trucks and moving into crouching positions. They ran around in a circle a few times and engaged in pretend hand-to-hand combat. I am sure they are skilled at whatever style they wanted to show, but they were overdoing things so badly, they were funny, not imposing. When they jumped over some barriers, the barriers fell to pieces and everyone scrambled to make it look as if that was part of the show. While carrying off the gear, it fell into further pieces which was even more funny. An armoured personnel carrier ended the show; several tougher looking guys jumped off of that one and their mock combat involved fully automatic fire (of blanks), several flashbangs, smoke grenades and, to top things off, the machine gun mounted on the APC moving down the opposing team with blanks. I never witnessed a "real" military parade in person but this one was somewhat disappointing. On the one hand, there was a distinct lack of ballistic missile carriers and tanks like you see in movies, documentaries and games, on the other hand, the whole thing had a make-do feeling to it. The cordoning police had designated spots to stand on, yet walked around. They were standing to attention, yet checking their cell phones. Several people in one uniformed group were wearing track suits and jeans. Another uniformed guy had a grocery bag with him; yet another one was carrying a huge water bottle. Bikers zig-zagged through the cordon and when the whole show was just about to wrap up the police finally started putting up barriers around the unmoving pedestrians, not blocking the bikers. One little girl was standing well within the cordoned area, watching with big eyes and after she did not react to the police talking to her, they just built the barriers in a curve around her. And to top it all off, some guy with a cane walked all through the parade with his personal camcorder, trying to direct the whole show while being ignored by everyone. Still, I am sure he managed to mess up some otherwise perfectly good TV scenes. Irkutsk-Russian border TL;DR 3000 kilometers of birch trees

18 May 2012

Johannes Schauer: a periodic counter

tldr: counting without cumulative timing errors Sometimes I want just a small counter, incrementing an integer each second running somewhere in a terminal. Maybe it is because my wristwatch is in the bathroom or because I want to do more rewarding things than counting seconds manually. Maybe I want not only to know how long something takes but also for how long it already ran in the middle of its execution? There are many reason why I would want some script that does nothing else than simply counting upward or downward with some specific frequency. Some bonuses: Now this can not be hard, right? One would probably write this line and be done with it: 
while sleep 1; do echo $i; i=$((i+1)); done
or to count for a certain number of steps: 
for i in  seq 1 100 ; do echo $i; sleep 1; done
This would roughly do the job but in each iteration some small offset would be added and though small, this offset would quickly accumulate. Sure that cumulative error is tiny but given that this task seems to be so damn trivial I couldn't bear anymore with running any of the above but started looking into a solution. Sure I could just quickly hack a small C script that would check gettimeofday(2) at each iteration and would adjust the time to usleep(3) accordinly but there HAD to be people before me with the same problem who already came up with a solution. And there was! The solution is the sleepenh(1) program which, when given the timestamp of its last invocation and the sleep time in floating point seconds, will sleep for just the right amount to keep the overall frequency stable. The author suggests, that sleepenh is to be used in shell scripts that need to repeat an action in a regular time interval and that is just what I did. The result is trivial and simple but does just what I want: You can check it out and read how to use and what to do with it on github: https://github.com/josch/periodic Now lets compare the periodic script with the second example from above: 
$ time sh -c 'for i in  seq 1 1000 ; do echo $i; sleep 1; done'
0.08s user 0.12s system 0% cpu 16:41.55 total
So after only 1000 iterations, the counter is already off by 1.55 seconds. This means that instead of having run with a frequency of 1.0 Hz, the actual frequency was 1.00155 Hz. Is it too much to not want this 0.155% of error? 
$ time ./periodic -c 1000
0.32s user 0.00s system 0% cpu 16:40.00 total
1000 iterations took exactly 1000 seconds. Cool.

17 May 2012

Neil McGovern: Government Open Standards Consultation

The government is currently consulting on open standards, and I have responded on behalf of Collabora. We believe that efforts to avoid vendor lock in, and to open up government from the extra expense this occurs is to be applauded. I have previously blogged about my decision to purchase Microsoft Office for Cambridge City Council, and the reasons why there was only one vendor. The consultation runs until the 4th June due to a potential conflict of interest which was revealed last month, and so everyone has time to influence government and ensure that government is more open to everyone who wants to access it. It's really easy to respond, so doing so is important. Make sure your voice is heard before it's too late.

16 May 2012

Christian Perrier: Bug #1000000 in Launchpad

Way before Debian, Launchpad bug tracking system just reached 1 million bugs reported with one "bug" reported against Edubuntu basically mentioning it should invade schools. What to say about this? Hard without being harsh towards my friends working in the Ubuntu "world", indeed. Still, I really think that here, too much noise kills signal and the LP BTS is often hardly usable. I counted up to 217 bugs reported against samba4 (which is, after all, not so widely used yet) just because it apparently has upgrading issues between pre 12.04 versions of Ubuntu and Oneiric. It indeed seems that some automated bug reporting is now active and whenever a user encouters an upgrade issue with a package, a bug is being reported. I guess this is somehow an opt-in system (I hope so..:-)) but the default is very clearly using it. This feature is apparently what caused the recent bump in number of bugs reported in LP, making them even less useful, particularly to Debian package maintainers. I'm sure there are tools to help dealing with that and I was already answered that work is in progress to change this (and use a dedicated website for such reports or something like this). But, still, that seems to be the scary side of popularity...the very same popularity that is slowly but constantly hiding the work we're doing in Debian to indirectly make Ubuntu popular. (moving to more general concerns) I know that things are not all black or all white, but it always saddens me to feel that slowly....but, again, constantly, more and more people tend to forget that Debian is behind Ubuntu, is the ground on which it is built and Ubuntu wouldn't exist without it. When doing work, a human need is to get reward for it...and we are getting less of it...slowly, but constantly. Don't take me wrong. I have many friends working directly for Ubuntu. Some paid by Canonical for this. Some really involved up to "top level" (yes, including the very very top level even if I killed him once). I don't want to throw offense on them. I don't even know if they can do something about what I'm expressing below. I would just have them (and others) know. Let's take an example. I recently activated a few languages in D-I (Burmese, Tibetan, Uyghur). I'm happy with that, this is something I'm doing for 8 years now. But all these new translators were indeed only interested in one thing : "have Ubuntu translated in their language". No offense intended, but they didn't really care about *Debian* being translated in their language. I think that some didn't even know what Debian is. In the same field, I am more and more "fighting" to keep the level of translation completeness in Debian (see my regular spa^W reports). In some way, I still succeed, but the price to pay is more and more and more personal investment and work. That's still working for the strong set of languages we support. That works much less for most others. When someone "disappears" (or just switches to some other priorities), it's more and more difficult to find someone else popping up. And, for the "strong set", something else is happening : work duplication. There are "strong" French, German, Italian, whatever, l10n teams in Debian.....and there are similar teams for Ubuntu. And, mostly, those do not really work together. And sometimes, this is kinda discouraging. So, seeing the explosion happening on what is, whatever we think or write, the "other side", is not somethnig that can make one entirely happy. And this is why I won't celebrate Launchpad's millionth bug report. Particularly when I see that millionth bug report not even ack'ing that this Edubuntu marvel is based on the grounds set by some pionneers many years ago in a few schools in Norway (hello, Petter and others). Yeah, sometimes sad. To balance this, let's release wheezy and have millions of people benefit from it without even knowing.

13 May 2012

Petter Reinholdtsen: Debian Edu interview: J rgen Leibner

It has been a few busy weeks for me, but I am finally back to publish another interview with the people behind Debian Edu and Skolelinux. This time it is one of our German developers, who have helped out over the years to make sure both a lot of major but also a lot of the minor details get right before release. Who are you, and how do you spend your days? My name is J rgen Leibner, I'm 49 years old and living in Bielefeld, a town in northern Germany. I worked nearly 20 years as certified engineer in the department for plant design and layout of an international company for machinery and equipment. Since 2011 I'm a certified technical writer (tekom e.V.) and doing technical documentations for a steam turbine manufacturer. From April this year I will manage the department of technical documentation at a manufacturer of automation and assembly line engineering. My first contact with linux was around 1993. Since that time I used it at work and at home repeatedly but not exclusively as I do now at home since 2006. How did you get in contact with the Skolelinux/Debian Edu project? Once a day in the early year of 2001 when I wanted to fetch my daughter from primary school, there was a teacher sitting in the middle of 20 old computers trying to boot them and he failed. I helped him to get them booting. That was seen by the school director and she asked me if I would like to manage that the school gets all that old computers in use. I answered: "Yes". Some weeks later every of the 10 classrooms had one computer running Windows98. I began to collect old computers and equipment as gifts and installed the first computer room with a peer-to-peer network. I did my work at school without being payed in my spare time and with a lot of fun. About one year later the school was connected to Internet and a local area network was installed in the school building. That was the time to have a server and I knew it must be a Linux server to be able to fulfil all the wishes of the teachers and being able to do this in a transparent and economic way, without extra costs for things like licence and software. So I searched for a school server system running under Linux and I found a couple of people nearby who founded 'skolelinux.de'. It was the Skolelinux prerelease 32 I first tried out for being used at the school. I managed the IT of that school until the municipal authority took over the IT management and centralised the services for all schools in Bielefeld in December of 2006. What do you see as the advantages of Skolelinux/Debian Edu? When I'm looking back to the beginning, there were other advantages for me as today. In the past there were advantages like: Today some of the advantages has been lost, changed or new ones came up in this way: What do you see as the disadvantages of Skolelinux/Debian Edu? Which free software do you use daily? I use Debian stable on my home server and on my little desktop computer. On my laptop I use Debian testing/sid. The applications I use on my laptop and my desktop are Open/Libre-office, Iceweasel, KMail, DigiKam, Amarok, Dolphin, okular and all the other programs I need from the KDE environment. On console I use newsbeuter, mutt, screen, irssi and all the other famous and useful tools. My home server provides mail services with exim, dovecot, roundcube and mutt over ssh on the console, file services with samba, NFS, rsync, web services with apache, moinmoin-wiki, multimedia services with gallery2 and mediatomb and database services with MySQL for me and the whole family. I probably forgot something. Which strategy do you believe is the right one to use to get schools to use free software? I believe, we should provide concepts for IT companies to integrate Debian Edu into their product portfolio with use cases for different countries and areas all over the world.

12 May 2012

Ben Armstrong: Collaborative editing, the missing Vim feature (pentadactyl + etherpad?)

Do you wish, like I do, you could edit collaboratively in Vim? This feature is number 10 on the Vim voting page, so it seems I m not alone. How about Pentadactyl coupled with any of the existing web-based collaborative editors, such as Etherpad? OK, so it s not quite Vim, and there are some rough edges to this particular pairing, but I m finding it s good enough for my needs. It even gives me a Vim-like editing experience while other participants use the default Etherpad editor. Yes, I know about whiteboard.debian.net, but for the past three years I have been using a single instance of Etherpad with my family to maintain a shopping list to which we all make contributions. First of all, that s not a Debian activity, so to make the switch, I d need to make a personal clone of the service for our personal use. But more importantly, we find Etherpad features such as colours for different participants and the timeline are just too useful to give up on. On the other hand, the less the web editor interferes with your web browser s default textarea behaviour, the easier time Pentadactyl is going to have. Indeed, I asked on #pentadactyl @ irc.oftc.net about some problems I was having and I was told flat out that Pentadactyl does not work with graphical web editors. So, you may wish to use another web-based collaborative editor for this reason. That being said, I did learn a few things about helping Pentadactyl get along better with Etherpad, so if you would like to try it yourself, read on. The key to getting started was to enter text edit mode within the textarea with <C-t>. For the most part, this behaves like Vim normal mode . I am still learning, but many basic motion and editing keys behave just as they would in Vim. Fantastic! However, the moment I tried to :undo I hit my first problem. Using the latest release version of Pentadactyl (1.0rc1 at time of writing), pressing "u" to :undo produced no visible result. I tried the latest daily build as well, and only saw a marginally more helpful "Node not found" error message displayed in the status area. But it turns out you can use passthrough mode to use the textarea s own undo. Just :tmap u <C-v><C-z> and we re back in business again. I m still experimenting with this setup, so the jury s still out on whether I ll stick with it, or whether the remaining incompatibilities between Pentadactyl and Etherpad will drive me nuts. But it looks promising. Clearly, judicious use of :autocmd to always start in text edit mode and bind that undo key whenever I enter the site will help make the experience even better. If you try it out yourself, I d love to hear how things went for you. Or if you have found an even better solution that works for you, do share.

11 May 2012

Andrew Pollock: [life] Zoe at 24 months

Once again, another 3 months have flown by. Honestly, it's been a total blur. What on earth has happened since February? Well, heaps more travel of course. There was the trip to New Orleans, which I already wrote about. The other big trip we did was to Washington D.C. for a week, which I'm yet to write up. We stayed with friends in Alexandria for the week. Unfortunately, Zoe came down with conjunctivitis before Sarah's very eyes on the flight over (I'd been in North Carolina for work the week before, so they flew without me and I flew up to D.C. to meet them). She also developed an ear infection and a general cough/cold. Fortunately, we caught it all pretty much as soon as it happened, and spent Easter Sunday at an Urgent Care in Virginia getting antibiotics for her, but her sleep was not so great the whole time we were in D.C. Other than that, we've just been doing the normal stuff. Swimming classes have continued (she's very confident in the water now). Her speech continues developing well. Toilet training is continuing to go very well. There haven't been very many accidents at all. She's really loving the "new" day care. It was definitely the right thing to do to move her. Physically, she's in the 80th percentile for height and the 50th percentile for weight, so I guess she's currently tracking to be on the tall and thin side of things. We had another birthday party in the park behind our home, and this year the weather was much nicer, so we had people hanging around until about 7pm, and a good time was had by all. Zoe even successfully blew out her candle on the first attempt. Her two-year molars still haven't come through. They seem to go through various stages of giving her a lot of grief, and then they stop messing with her sleep. I was beginning to think we'd never get back to an uninterrupted night's sleep ever again. It's still a bit hit and miss, but she usually settles down again fairly quickly if she does wake up, and doesn't always require intervention from us. I've arrived at the theory that the "terrible twos" are largely the fault of trying to cut these two year molars. Her tantrums are so much worse when her teeth are actively giving her trouble. If she's had a good night's sleep and her teeth aren't driving her crazy, she's still pretty well behaved. If she's been awake half the night and her teeth are hurting during the day, she can be quite difficult. We seem to be currently at a point where they're not causing too much trouble at night, but she'll chew on her fingers like crazy if given the chance during the day. I think seasonal allergies are also messing with her a bit. After she got over the cold she came down with in Washington D.C., she's still had a runny nose and a bit of a cough, but I don't think she's actually "sick" I think it's more related to allergies. She passed on whatever she had to me, and since I've recovered from it I'm also not feeling quite right. I read somewhere that pollen counts are off the charts this year due to the weird weather, and Sarah's having a bit of a hard time as well, so I think we're all going to just have to ride it out with the judicious use of antihistamines. Speaking of allergies, it's about time to re-test her for her egg allergy. We'll have to try that at a point when she's not regularly taking antihistamines for seasonal allergies, so it might be a while yet before we can do that. I guess the next big development will be graduating from the crib to a toddler bed. I'm not in any hurry to do that though, so as long as she's not escaping the crib, her sleeping arrangements can stay the way they are. It'd be nice if she can cut her teeth first and then once she's sleeping properly, we can try converting the crib into a toddler bed. Zoe walking down Castro Street in Mountain View

10 May 2012

Jo Shields: Sleeping with the enemy: my life with Windows Phone

In my last blog post about smartphones, I urged the universe at large to help maintain a variety of ecosystems, to help foster competition and originality amongst vendors and the same day I hit publish, WebOS was killed. Apparently the universe hates me. Since then, a few things have changed. My main phone since the day of its release was the HP Pre 3, running WebOS and whilst I still have a soft spot for the OS, the Pre 3 was simply too buggy for me to use full time. The main issue is that I use my phone as an MP3 player in the car but the Pre 3 would pause playback at the end of a track every half dozen tracks or so making it impossible to drive the 85 miles to work without needing to root around in the armrest and poke a touchscreen. Not something I really want to do whilst moving and ultimately too big a papercut to deal with. So, come the new year, I moved on to my next device, a Nokia N9 running MeeGo Harmattan. Ultimately, this was an even bigger failure for me than the Pre 3 was, and I lasted maybe two weeks with it before giving up and going back to the HP. Beyond massive usability errors in the software (especially the braindead unkillable pop-up demanding Internet access, even when none is available), the worst for me was how it handled the MP3 player task. My usual way of working is to have the phone hooked up to the stereo with a 3.5mm jack, and the car switches to headset Bluetooth profile to handle calls this is pretty common on cars too old to support A2DP profile (Stereo music-capable headphones). WebOS and Android are fine with this but not the N9. The N9, instead, will output all audio through the last connected audio device, regardless of how much that might not be helpful. Get in car, start music playing, plug in cable, start engine and it plays audio for about three seconds before the Bluetooth connects, and it switches to outputting music via the Headset bluetooth profile (not something that my car can do). Unplug and replug the cable, and music works but incoming calls are silent until I disconnect the 3.5mm jack, as it outputs the headset audio through the headphone socket. I just couldn t deal with this big a step back from WebOS as far as my workflow goes, and gave up. So, where next? Well, a funny thing happened a co-worker with generally very good instincts regarding consumer electronics usability told me that his housemate had just bought a Nokia Lumia 800 Windows phone (the WP7-based cousin to the N9) and loved it. Enough that said co-worker was considering getting one himself. This was a very strange thing to hear, especially from an iPhone owner, about a Microsoft product. I d been generally interested in WP7 on an academic level for a while, but to hear that degree of praise of the actual product was interesting. Also interesting, and roughly simultaneous, was seeing Sajid Anwar s reverse engineering of the proprietary Zune file transfer protocol go from theory into an actual set of libmtp patches. So if the capability to use Banshee to transfer music on is here or near and it can t be as braindead as Harmattan when it comes to headphone/bluetooth behaviour, then why not jump ship and squeeze a handset out of Orange? About a week after my co-worker replaced his iPhone with a Lumia 800, I bought one too. So where to begin? Well, I ll begin at the start: WP7 is a joy to use. It really just is. It s the first mobile OS to try something radically different in the UI department for years. Everyone else these days (especially Android) builds iPhone rip-offs to varying degrees, and even the iPhone interface has a lot in common with the old old OLD interfaces found on the dumb Nokia phones of the 1990s. WP7 has an interface which provides just the right level of passively visible information and interactivity, and manages to do it with an elegance that no Android home screen filled with widgets will ever manage. The uncluttered screens are easy to read, and the Metro usability paradigms are trivial to pick up and learn. Without a doubt I d recommend WP7 to friends and family from a usability perspective, and the Microsoft engineers and designers responsible for cooking up the WP7 interface are worthy of praise. And I m not the only one saying this Apple co-founder Steve Woz Wozniak recently came out with a similar line. That s the good. There s also some bad, make no mistake. I m going to cover all the reasons WP7 sucks over several paragraphs. But overall, a smartphone is a device which I expect to suck the question is how bad the suck is, and whether it gets in the way of me using the device for what I need at the time. Moreso than MeeGo, moreso than Android, and even WebOS (and I m still a big WebOS fan), WP7 has more good points than bad points. But there s still some room for improvement, and some room for caution and since I know there are a few Microsoft folks following me on Twitter, I m going to go over my prescription for continued platform success. Oh, one more thing before I start: I know WP7 isn t Free Software. As an end user, I really don t care about that. I just want something that works something I didn t get from WebOS and Harmattan, both of which are primarily Free Software stacks. I m not saying there s a causal relationship there, or that a mobile OS can t be both Free Software and good just that as an end user, my favourite platform right now is non-Free. Take from that whatever you like. It s also vitally important, as Free Software folks, never to lose sight of what the other players in the market are up to. If you can t objectively assess why people are using a proprietary option by using it & recognising its good points (i.e. what to steal & what to improve) then you can t hope to win over users. So. WP7 s downsides in detail. In-place updates. Seriously guys, even Apple can manage this now. Why can t Windows Phone? I understand that making backups is smart and all updates come with a mandatory backup but I really shouldn t be tied to a PC to update a post-PC device. Also, those backups are useless, since they cannot be restored onto replacement devices in the case of failure or theft, so fix that too. Update all the things. An iPhone sold in June 2009 still has access to the latest iOS releases. Android phones are notorious for shipping with an outdated version of the OS, then getting at most one major update over the phone s lifetime (usually the device is abandoned by its manufacturer within months of release). Which camp does Microsoft want to align with, there? Every Windows phone 7 device released should receive Windows Phone 8, even with some features disabled. Anything less is punishing every existing customer, in the hope that you ll attract new ones not a winning strategy for a fringe platform whose biggest evangelists are its users. Fix IMAP. IMAP isn t hard. Yet WP7 never seems to work properly with a subset of my mail, never showing the message body & just saying Downloading forever. Fix it. Bing sucks. Bing s search results are terrible. Either do something to make them bearable, or allow me to pick which search engine I get when I hit the search button. A Google live tile isn t the same thing. Make killing apps easier. I know you stole the WebOS card view for multitasking (hold the back button) please also steal the WebOS ability to close apps. I don t want to have to go into an app and bash back repeatedly until it quits. This is particularly annoying for Internet Explorer. Make reinstalling apps easier. If I want to install every app I previously had installed on a new device, without restoring a backup, this should be easy. There are third party apps which try to plug this gap. Find a way to support copyleft. I d like to port a few C# apps to WP7, but because they re LGPL, I can t. The code s copyright holders would have no issue with their code being on WP7, as long as end users have a mechanism to replace the libraries, so why not find a way to allow this? e.g. when compiling an app, let me mark a library as user-replaceable , then allow for some mechanism where an end user can replace those assemblies with their own version. Let me use multiple Google calendars. WP7 only lets me add/see appointments on my default Google calendar. I want to add/see things on my wife s calendar, which is shared with me. WebOS can do this. MTP-Z is the devil. I do not need or want encrypted end to end communication between my PC and my camera device, to transfer a photo off. I do not need or want encrypted end to end communication between my PC and my MP3 player to transfer a photo on. Let s be honest, the only reason for MTP-Z is to enforce DRM on Zune-rented music tracks and honestly, there s no good reason to require MTP-Z for *all* communications if all you want to do is protect one folder or file extension. Now, since MTP-Z theoretically forces me to use Zune for many tasks better handled by other apps, now I get to write multiple criticisms of Zune s desktop app and as long as MTP-Z is enforced, every Zune failing is a Windows Phone failing too. Zune: Support Windows codec infrastructure, and transcode where needed. Windows Media Player can play Ogg Vorbis files. No, not out of the box, but if one installs the required codecs. Zune should support the same files as WMP if you want to ensure people don t try to copy files to a portable device which are not supported on that device, then you should have an API in place to allow for pluggable seamless transcoding of files as required Banshee allows me to do this (e.g. to copy files I have as .flac to devices which do not support it). Zune: Search my tracks, not the web. Zune s searching is terrible it doesn t do as-you-type searches, and when I hit enter, matches from my collection are given a tiny little space compared to matches from the Zune music store. Let me easily pick the track I feel like listening to, don t make it a chore Zune: Let s solve metadata together. I absolutely love how nicely the Zune app on desktop and on phone shifts as appropriate to the currently playing artist (e.g. changing the lock screen to an image of the artist in question). However, Zune doesn t make it obvious how to set an album s metadata to support this, and it s particularly frustrating when it s a minor difference of spelling causing a track not to get the nice treatment e.g. UNKLE versus U.N.K.L.E. . Either start making heavy use of audio fingerprinting services like MusicBrainz to fill in metadata, or allow me to search for fully supported artists when filling in track metadata Zune: Random playlists are useless on devices. I like smart playlists. In Banshee, I have one to pick 12GB of random tracks, which I can sync to my phone. I can t do this with Zune. If I try to just sync all my random music to my phone, it errors out due to lack of space. If I have a random playlist, the random selection changes multiple times during a sync resetting the sync, wiping out half the tracks that were transferred on, and starting again. As a result, the sync goes on for literally hours, never ending up with more than a gig or so of tracks on the phone. Random playlists should be freezable, so I can transfer them to my device in peace, then get a new random selection when I want. So, that s my list of miserable failure and it s still a less painful list than any other mobile OS I ve used. Perhaps one day Android will approach being usable, perhaps Blackberry s BBX will actually appeal to human beings rather than corporate IT managers, and perhaps Mozilla s delightfully named Boot to Gecko will get some traction. Who knows. All I know is, My Lumia 800 is the best phone I think I ve ever owned, and it s important for anyone working in the mobile space to understand why.

7 May 2012

Lars Wirzenius: Quality of discussion in free software development

The Online Photographer has a meta-article on some discussion in the photography world. Summary: someone wrote an opinion piece on one site, and people on the discussion forum of another site got his name wrong, possibly repeatedly. And the quality of the discussion went down from there. The quality of the discourse of free software development is frequently of some concern. Debian has a reputation as being a host to, er, particularly vigorous discussions. That reputation is not unwarranted, but, I think, we've improved a lot since 2005. The problem is hardly restricted to Debian, however. How can we improve this? I don't know. As a community, I'm not even sure we agree what the problems are. Here's my list. Insults, personal attacks, and other such outrageously bad behavior is uncommon. It crosses the line so clearly it becomes easy to deal with; I don't think handling this needs much attention. What can we do about this? I'm not sure. I have, for the time being, abandonded Debian mailing lists as a way to influence what goes on in the project, but that's just a way for me to clear some space in my head and time in my day to actually do things. My pet hypothetical solution of the day is that mailing lists might raise the quality of the debates by limiting the number of messages written by each person per day in each thread. This might, I think, induce people to write with more thought and put more effort into making each message count.

6 May 2012

Stefano Zacchiroli: overlapping bits from the DPL for April 2012

Just posted to d-d-a, here is the monthly report about my DPL activities.
Dear project members,
last bits of the past DPL term and first bits of the current term, all in one. Here is a report of what has happened in DPL land last April. Highlight: call for DPL helpers Before the report, though, let me point out that your friendly neighborhood DPL could use some help. As discussed during campaign, there are some intrinsic transparency and scalability limits in the DPL institution, when run by a single person. Before trying something new to fix that, I'd like to give a last try to an old "tactic": calling for help. If you're considering running for DPL or if you're simply interested in the job the DPL does and willing to help with that, please let me know. Ideally, if I find a group of people I'm happy to work with, I'd like to set up periodic IRC meetings with all "DPL helpers" to publicly discuss items in the DPL agenda and share the work-load. Ongoing discussions A big topic of last month has been the proposal by Francesca Ciceri to publish a diversity statement for the Debian Project. After a lively discussion on -project, we reached consensus on a text, and I've been happy to help with that. To finalize statement publication we now need to vote on it with a GR. I've helped drafting a corresponding GR proposal that has already been posted to -vote by Francesca. A final one, looking for seconds, will be posted there soon. Wrapping up March discussions on a revenue sharing agreement with DuckDuckGo, I've announced my intention to finalize the agreement and have done so shortly thereafter. The Iceweasel maintainer has deployed the corresponding search engine query string and other web browser maintainers could do the same, if they want to. In April I've also spent some time to move forward the long running conflict on Python maintenance, reported to the tech-ctte more than 2 years ago. With the help of people on the -python mailing list, I've now submitted to the tech-ctte an up to date list of potential maintenance teams. I hope the tech-ctte now have all the information needed to come to a decision. Speaking of which, I'm also discussing with tech-ctte members the possibility of having periodic ctte meetings; the idea is to ensure that outstanding issues are periodically reassessed, improving the reliability of tech-ctte decision times. I've also discussed at length with members of the pkg-multimedia-maintainers team the relationships with the unofficial debian-multimedia.org (d-m.o) repository, that have been a cause of tension for Debian multimedia users and maintainers for quite some time. On behalf of the team and of the Project I've now reached out to the d-m.o maintainer, hoping to come to some sort of amicable agreement on which packages belong where. Hardware replacement As anticipated in last report, I've started approving hardware purchases to implement the yearly hardware replacement plan prepared by DSA. During April I've approved requests to buy servers to replace the machines running the bugs-master, bugs-mirror, and UDD services. The total expected expenditure is about 15'000 USD. Communication I've delivered my classic Debian "18^W 19 years" talk at UNIVPM, a polytechnic university in center Italy; slides are available. I've then been contacted by people from the European Synchrotoron in Grenoble who, beside having recently migrated their infrastructure to Debian, are looking into organizing a workshop on Debian usage for large science facilities. I've been happy to help out providing a list of potential topics and speakers for the event. Also as anticipated last month, the Debian Project has been present at the OpenStack summit. Loic Dachary has represented Debian at the event and provided a nice report about his experience there. Speaking of which, I've also coordinated a news release about the availability of cloud technologies in Wheezy, taking the chance to point out the relationships between what Debian stands for and the ability to deploy your own private cloud. Sprints April has been a rather calm month on the sprint front, with the notable exception of the I18n team who is organizing a sprint for June in Paris. Miscellanea Thanks for reading thus far,
HDH! (Happy Debian Hacking)
PS the boring day-to-day activity log for April is available at master:/srv/leader/news/bits-from-the-DPL.txt.201204

3 May 2012

Rudy Godoy: FLISOL Puno

Past Saturday April 28 I attended to the FLISOL 2012 Puno hosted at the Casa de la Cultura in downtown Puno. It was a great experience to deliver a talk after a while, the second speech since I enrolled into the UCSP s Computer and Engineering Faculty to get my Computer Science BS degree. Puno is really a nice city with a great landscape for those like me who enjoy the sun, lakes, wonderful mountains and, as it s name says, the Andes plane 3860m above sea level. Pretty high and very cold at night indeed but a lovely place. FLISOL is short for Latin American Free Software Installation Festival that just aged 12 this year. In my talk I wanted to describe the motivations behind the Free Software movement origins. I specially remarked the fact that the movement doesn t look for technical excellence, even though this is something we thrive for, rather than preserving a common set of guidelines for protecting assets we do value such software and freedom. Later I discussed about how Debian have become a key actor in the free software ecosystem. Primarily as a huge software library, second as a building block for specialized distributions (such as Ubuntu) and third, but not the last, as a facilitator for new developments and contributions to the ecosystem, and I mentioned here the Debian GNU/Hurd project. Through such topics I explained what we do, why we do it and why they may want to contribute. There was a good reception from the audience and there were many questions on the subjects I elaborated on. I was impressed to see many Debian users, some of them power users , and even tried to help one to configure his obscure Xorg settings on his laptop (didn t succeed because lack of connectivity, will send him testing CDs eventually). I d like to thank to the organization who made a great job. The team, nearly 30 students from Universidad Nacional del Altiplano plus local activists, started working in February collecting funds and managed to gather more than 330 attendees during the almost 10 hours of the event. Also special thanks for the coffee machine that was available!

2 May 2012

Matthew Garrett: Anatomy of a Fedora 17 ISO image

My spare time has been limited lately, so progress on producing Mac-bootable Fedora install images has been a bit slow. Thankfully it looks like everything's gong to land in time for Fedora 17[1] so hurrah for that - all that's missing right now are the last couple of patches that make sure the boot picker displays useful labels on the drives.

So how is this accomplished? Here's a hex dump of the first few K of the image.
00000000  45 52 08 00 00 00 90 90  00 00 00 00 00 00 00 00   ER.............. 
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00000020  33 ed fa 8e d5 bc 00 7c  fb fc 66 31 db 66 31 c9   3...... ..f1.f1. 
00000030  66 53 66 51 06 57 8e dd  8e c5 52 be 00 7c bf 00   fSfQ.W....R.. .. 
00000040  06 b9 00 01 f3 a5 ea 4b  06 00 00 52 b4 41 bb aa   .......K...R.A.. 
00000050  55 31 c9 30 f6 f9 cd 13  72 16 81 fb 55 aa 75 10   U1.0....r...U.u. 
00000060  83 e1 01 74 0b 66 c7 06  f1 06 b4 42 eb 15 eb 00   ...t.f.....B.... 
00000070  5a 51 b4 08 cd 13 83 e1  3f 5b 51 0f b6 c6 40 50   ZQ......?[Q...@P 
00000080  f7 e1 53 52 50 bb 00 7c  b9 04 00 66 a1 b0 07 e8   ..SRP.. ...f.... 
00000090  44 00 0f 82 80 00 66 40  80 c7 02 e2 f2 66 81 3e   D.....f@.....f.> 
000000a0  40 7c fb c0 78 70 75 09  fa bc ec 7b ea 44 7c 00   @ ..xpu.... .D . 
000000b0  00 e8 83 00 69 73 6f 6c  69 6e 75 78 2e 62 69 6e   ....isolinux.bin 
000000c0  20 6d 69 73 73 69 6e 67  20 6f 72 20 63 6f 72 72    missing or corr 
000000d0  75 70 74 2e 0d 0a 66 60  66 31 d2 66 03 06 f8 7b   upt...f f1.f... 
000000e0  66 13 16 fc 7b 66 52 66  50 06 53 6a 01 6a 10 89   f... fRfP.Sj.j.. 
000000f0  e6 66 f7 36 e8 7b c0 e4  06 88 e1 88 c5 92 f6 36   .f.6. .........6 
00000100  ee 7b 88 c6 08 e1 41 b8  01 02 8a 16 f2 7b cd 13   . ....A...... .. 
00000110  8d 64 10 66 61 c3 e8 1e  00 4f 70 65 72 61 74 69   .d.fa....Operati 
00000120  6e 67 20 73 79 73 74 65  6d 20 6c 6f 61 64 20 65   ng system load e 
00000130  72 72 6f 72 2e 0d 0a 5e  ac b4 0e 8a 3e 62 04 b3   rror...^....>b.. 
00000140  07 cd 10 3c 0a 75 f1 cd  18 f4 eb fd 00 00 00 00   ...<.u.......... 
00000150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................
This is the boot sector. One of the fun things about ISO9660 is that it doesn't have a boot sector in the usual x86 sense - firmwares are expected to read enough of the filesystem that they can find the bootloader in it and run it directly. x86 isn't really smart enough to manage that, so it uses something called El Torito[2]. In any case, this is usually empty space on an x86 CD. But with an increasing number of systems shipping without optical drives, real CD installs are decreasing in popularity. We use a piece of software called isohybrid that allows the building of ISO9660 images that can be written directly onto a USB stick. Asking a machine to boot off them will execute the boot sector located here, which then loads a real bootloader that's capable of reading ISO and booting the OS. It'll just be ignored when it's on a real CD.

There's one difference here, though. The isohybrid bootsector has been modified so that the first 32 bytes are just noops, and this has been inserted:
00000000  45 52 08 00 00 00 90 90  00 00 00 00 00 00 00 00   ER..............
45 52 ("ER") indicates the presence of an Apple partition map. This will become important later. 0800 indicates that it's using 2048-byte sectors. The 9090 is a lie to convince the firmware that it's not a zero-sized disk without representing dangerous code. We need an Apple partition map because some older Macs won't boot off CD unless the CD has one. But it's sitting right at the start of the boot sector, and this code is going to be executed. Thankfully, the entire Apple header decodes to
00000000  45                inc bp
00000001  52                push dx
00000002  0800              or [bx+si],al
00000004  0000              add [bx+si],al
00000006  90                nop
00000007  90                nop
00000008  0000              add [bx+si],al
0000000A  0000              add [bx+si],al
0000000C  0000              add [bx+si],al
0000000E  0000              add [bx+si],al
00000010  0000              add [bx+si],al
00000012  0000              add [bx+si],al
00000014  0000              add [bx+si],al
00000016  0000              add [bx+si],al
00000018  0000              add [bx+si],al
0000001A  0000              add [bx+si],al
0000001C  0000              add [bx+si],al
0000001E  0000              add [bx+si],al
which is completely harmless - we can execute all these instructions without any interesting changes in state. They'll all be undone by the rest of the boot sector.
000001b0  14 05 00 00 00 00 00 00  4c 1e ed 76 00 00 80 00   ........L..v.... 
000001c0  01 00 00 3f a0 89 00 00  00 00 00 50 14 00 00 fe   ...?.......P.... 
000001d0  ff ff ef fe ff ff a4 00  00 00 70 04 00 00 00 fe   ..........p..... 
000001e0  ff ff 00 fe ff ff 44 05  00 00 c0 08 00 00 00 00   ......D......... 
000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa   ..............U.
This is the MBR partition table. The aim here was to produce media that would boot via BIOS just as well as it does via EFI, and it turns out that there are some systems that refuse to BIOS-boot off GPT media. So we need an MBR partition map. The first entry covers the entire disk and is of type 0. This is important, because some EFI implementations are strict about MBR parsing - if there's an MBR with overlapping partitions, the entire MBR will be ignored. Inconvenient. Fortunately, we've got the source code to this code, and it turns out that partitions of type 0 are ignored when performing this check. So, there's a partition of type 0.

Why does it cover the entire disk? Once we've booted the OS, we need to be able to read the ISO image contained on the USB stick. That means the kernel needs to be able to mount it, which means we need a partition entry that covers the entire disk. Linux is perfectly happy mounting a filesystem from a partition of type 0.

So, the next partition. This is an EFI boot partition that points at the embedded VFAT El-Torito image. Some firmware will decide that we've got an MBR and so will only boot off a partition that exists in it. So, here's an MBR partition to keep them happy.

The final partition covers the embedded HFS+ image. It's there purely for convenience - it means we can access it under Linux in order to update its contents for testing purposes.
00000200  45 46 49 20 50 41 52 54  00 00 01 00 5c 00 00 00   EFI PART....\... 
00000210  09 27 93 7e 00 00 00 00  01 00 00 00 00 00 00 00   .'.~............ 
00000220  fe 4f 14 00 00 00 00 00  30 00 00 00 00 00 00 00   .O......0....... 
00000230  de 4f 14 00 00 00 00 00  2a 26 0f 37 23 c9 7f 4f   .O......*&.7#..O 
00000240  90 df 5e fe 0a 60 1c b0  10 00 00 00 00 00 00 00   ..^.. .......... 
00000250  80 00 00 00 80 00 00 00  ec 70 63 c1 00 00 00 00   .........pc..... 
00000260  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................
And here we have the GPT header. Nothing complicated here, other than it indicates that the first partition descriptor is at LBA 0x10, ie byte 0x2000 - that's further into the disk than normal. That leaves us enough space to fit the Apple partition entries.

Why have a GPT at all? The EFI spec says that machines should boot fine from an MBR partition. Sadly, not all seem to. It also lets us represent the HFS+ partition in a way that makes the Mac firmware happy.
00000800  50 4d 00 00 00 00 00 03  00 00 00 01 00 00 00 10   PM.............. 
00000810  41 70 70 6c 65 00 00 00  00 00 00 00 00 00 00 00   Apple........... 
00000820  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00000830  41 70 70 6c 65 5f 70 61  72 74 69 74 69 6f 6e 5f   Apple_partition_ 
00000840  6d 61 70 00 00 00 00 00  00 00 00 00 00 00 00 00   map............. 
00000850  00 00 00 00 00 00 00 0a  00 00 00 03 00 00 00 00   ................ 
00000860  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................
2K into the disk, here's an Apple partition map. It's 2K into the disk because we set the sector size to 2K in the partition map header. Why did we do that? Because the default is 512 bytes, and if we'd put the partition map at 512 bytes it would have been on top of the GPT header. Why 2K rather than 1K? A couple of reasons. First, the Apple partition map is only used when we boot off an actual CD, and the CD sector size is genuinely 2K. Secondly, because CDs have a sector size of 2K, 2K is a value that's actually been tested in the real world. It's nice to avoid tempting fate.
00001000  50 4d 00 00 00 00 00 03  00 00 00 29 00 00 04 70   PM.........)...p 
00001010  45 46 49 00 00 00 00 00  00 00 00 00 00 00 00 00   EFI............. 
00001020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00001030  41 70 70 6c 65 5f 48 46  53 00 00 00 00 00 00 00   Apple_HFS....... 
00001040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00001050  00 00 00 00 00 00 04 70  00 00 00 33 00 00 00 00   .......p...3.... 
00001060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
*
00001800  50 4d 00 00 00 00 00 03  00 00 01 51 00 00 08 c0   PM.........Q.... 
00001810  45 46 49 00 00 00 00 00  00 00 00 00 00 00 00 00   EFI............. 
00001820  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00001830  41 70 70 6c 65 5f 48 46  53 00 00 00 00 00 00 00   Apple_HFS....... 
00001840  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00001850  00 00 00 00 00 00 08 c0  00 00 00 33 00 00 00 00   ...........3.... 
00001860  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................
The Apple partition map entries. One partition covers the entire disk, the other the embedded HFS+ filesystem. Why do this? Because not all Macs understand EFI El Torito, so won't EFI boot a CD unless there's an Apple partition map. Why have an HFS+ partition at all? Because the same Macs won't boot off FAT.
00002000  a2 a0 d0 eb e5 b9 33 44  87 c0 68 b6 b7 26 99 c7   ......3D..h..&.. 
00002010  d0 84 a5 d4 aa 89 e0 40  81 f9 fc e4 04 e9 c2 99   .......@........ 
00002020  00 00 00 00 00 00 00 00  10 4b 14 00 00 00 00 00   .........K...... 
00002030  00 00 00 00 00 00 00 00  49 53 4f 48 79 62 72 69   ........ISOHybri 
00002040  64 20 49 53 4f 00 49 53  4f 48 79 62 72 69 64 00   d ISO.ISOHybrid. 
00002050  41 70 70 6c 00 00 00 00  00 00 00 00 00 00 00 00   Appl............ 
00002060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
*
00002080  a2 a0 d0 eb e5 b9 33 44  87 c0 68 b6 b7 26 99 c7   ......3D..h..&.. 
00002090  68 d5 e0 27 72 83 c6 4c  a7 ae 92 dd ed 6c 89 71   h..'r..L.....l.q 
000020a0  a4 00 00 00 00 00 00 00  13 05 00 00 00 00 00 00   ................ 
000020b0  00 00 00 00 00 00 00 00  49 53 4f 48 79 62 72 69   ........ISOHybri 
000020c0  64 00 41 70 70 6c 65 00  41 70 70 6c 00 00 00 00   d.Apple.Appl.... 
000020d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
*
00002100  00 53 46 48 00 00 aa 11  aa 11 00 30 65 43 ec ac   .SFH.......0eC.. 
00002110  68 d5 e0 27 72 83 c6 4c  a7 ae 92 dd ed 6c 89 71   h..'r..L.....l.q 
00002120  44 05 00 00 00 00 00 00  03 0e 00 00 00 00 00 00   D............... 
00002130  00 00 00 00 00 00 00 00  49 53 4f 48 79 62 72 69   ........ISOHybri 
00002140  64 00 41 70 70 6c 65 00  41 70 70 6c 00 00 00 00   d.Apple.Appl.... 
00002150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................
Three GPT entries - one covers the entire CD, one covers the embedded FAT filesystem, one covers the embedded EFI filesystem. That ensures that they're all accessible to the firmware.
00008000  01 43 44 30 30 31 01 00  4c 49 4e 55 58 20 20 20   .CD001..LINUX    
00008010  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20                    
00008020  20 20 20 20 20 20 20 20  46 65 64 6f 72 61 2d 4c           Fedora-L 
00008030  69 76 65 43 44 20 20 20  20 20 20 20 20 20 20 20   iveCD            
00008040  20 20 20 20 20 20 20 20  00 00 00 00 00 00 00 00           ........ 
00008050  c4 12 05 00 00 05 12 c4  00 00 00 00 00 00 00 00   ................ 
00008060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00008070  00 00 00 00 00 00 00 00  01 00 00 01 01 00 00 01   ................ 
00008080  00 08 08 00 40 00 00 00  00 00 00 40 15 00 00 00   ....@......@.... 
00008090  00 00 00 00 00 00 00 17  00 00 00 00 22 00 1d 00   ............"... 
000080a0  00 00 00 00 00 1d 00 08  00 00 00 00 08 00 70 05   ..............p. 
000080b0  01 09 30 36 f0 02 00 00  01 00 00 01 01 00 20 20   ..06..........   
000080c0  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20
The ISO9660 superblock. This is completely standard. From here on there's nothing terribly surprising - the CD filesystem itself is entirely normal. The only slight oddity is that we have three embedded El Torito images. The first of these is a BIOS-bootable disk image. That'll be executed whenever the CD is put in a non-EFI machine. The second is a VFAT EFI boot image. That's used for the majority of EFI systems. The third is an HFS+ EFI boot image. There's no strict requirement for the HFS+ one to be an El Torito image - CD booting on Macs is already handled by the Apple partition map. Storing it as an El Torito is purely for convenience, since it guarantees correct alignment and avoids us having to parse the entire ISO9660 image to find its location when generating the partition maps.

In summary: Three partition maps, three bootable images, support for BIOS, UEFI and Mac platforms, works whether it's burned to a CD or written directly to a USB stick. I'm pretty pleased with that.

[1] Other than an awkward bug where something goes horribly wrong during Radeon graphics init, so Radeon-based Macs aren't working too well at the moment. We're working on it.

[2] The world decided that writing a real ISO9660 driver for BIOS was hard, so came up with El Torito. El Torito is a spec for embedding disk images inside ISO9660. There's a pointer to the disk image in the Cd header, so the BIOS simply reads a few blocks off CD, finds that pointer and then sets up a bunch of disk i/o interrupts to point at the embedded filesystem rather than a real disk. From that point on, the El Torito image simply behaves like a floppy or hard drive. This was a reasonable compromise given how resource limited older BIOS implementations were.

For reasons that aren't entirely clear, UEFI doesn't mandate that implementations support ISO9660. So even though our firmware is now sufficiently capable that the only thing standing between it and emacs is someone being sufficiently bored, we still use El Torito. Fortunately the spec allows multiple El Torito images to be embedded, so we can include one that's bootable by BIOS systems and another that's bootable by UEFI systems.

comment count unavailable comments

30 April 2012

John Goerzen: Review: Amateur Radio Handhelds (HTs), APRS, and Battery Tuning Tips

There are a lot of opinions about handhelds for amateur radio users (these handhelds are often known as HTs). I ve used three of them, two with APRS capability, and thought it s about time I write about them. First, I ll talk about the units in particular, then I ll get to some broader Yaesu vs. Kenwood generalizations and conclusions. Yaesu VX-7R a rugged first HT without APRS Cost: $350 as of 4/30/2012 This was the first HT I got as a ham, at the suggestion of Mike AE0MW. It truly does make a great first handheld for a ham. Radio-wise, one unique feature is a receive that s wideband enough to cover both broadcast FM and broadcast AM, the various ham bands, and plenty of others besides. The battery life quoted in the manual is 6 hours on 2m and 5.5 hours on 440 (using standard 6s TX / 6s RX / 48s squelched metric), and in my experience it seems it must be even better in some cases. The real piece of news about this unit, though, is just how rugged it is. It says submersible across the front and I absolutely believe it. I have personal experience with its ruggedness. I once was at an event and put the radio on my car roof while helping one of my children into the car. I then started driving home and forgot it there. When we got home, I realized what had happened, and also realized what that mysterious CLUNK along the busy county road was. So I drove 45 minutes back to where I heard the clunk, and found the VX-7R still lying in the roadway, where it had been for 1.5 hours on the 55 MPH road. The thing was really beat up. It had obviously been hit by numerous vehicles. The antenna was ripped off, taking the SMA mount with it. But wouldn t you know it, the darn thing still powered up. I took it to a local amateur radio shop, and about $80 later it was repaired. Incredible. The only real problem was the antenna connector. I took this radio with me on my bicycle many days, mounted on my handlebars. It got bounced around, but still worked great. Some days I listened to FM radio from it while riding my bike, monitoring a local repeater while I rode. Other days, I spent most of my ride (about an hour each way) chatting with people on the repeaters I have a mobile antenna mounted on the bike, fed to the VX-7R, and a headset. It all worked fine never a problem at all. The ruggedness has its downside. All of the connectors on the radio, except for DC in, are screw-down even the speaker/mic connector. This obviously helps keep water out, but means that it can take a lot longer than normal to do a simple thing like plug in a speaker/mic. It doesn t need any tools, but can get a little annoying. The radio physically is small (unless you install the belt clip, which has an odd peg-based mechanism that protrudes unnecessarily far and is surprisingly non-rugged). It can easily fit into a pocket. The other downsides of the VX-7R mostly surround its interface. I m not one of the Yaesu haters that seem to be so common. I find the interface usable if you read the manual; it seems a lot of people that complain about it don t. But it is quirky. The SET mode is one long menu in sort of a ring. You scroll with the knob, and when you get to the end, it repeats to the beginning. There are one or two shortcut keys to a specific setting, but overall either you use the thing so much you know exactly where you re going, or you re twisting the knob for awhile until the option you want rolls around. (I know of nobody that uses it so much they know exactly where they re going.) They do have options grouped into categories, but it doesn t help much because there s no quick way to skip between categories. The dual transceivers work less elegantly than on the TH-D72A or VX-8GR. Instead of being two equal transceivers A and B, they are main and sub . The sub transceiver is not broadband receive like the main receiver is. But there are also other limitations that have no apparent logic. For instance, you can program memory groups into the device putting, say, all the local repeaters into a certain group, or all the public safety frequencies (to use as a scanner). You can go to the special set menu, select group, then activate SCAN, and it will scan only the memories in that group. But you can t select the special group mode on the sub transceiver for no apparent reason. One other complaint is that the AC charger introduces so much noise to a transmission that you cannot really use the VX-7R to transmit while the charger is connected. Yaesu sells a programming cable that plugs into the speaker/mic port on one end, and a DB-9 serial port on the other, and includes programming software. For an incredibly high price. For far less, I found a similar cable on eBay that has a USB port on the other end (with a built-in USB/serial converter), and used VX-7R Commander though now that Chirp is available for Linux, I ll probably switch to it. I would probably continue to recommend this transceiver for a new ham. The ruggedness, plus the broadband RX, are features that should make it appealing. It is an excellent emergency preparedness/response radio due to those properties; having TX and RX on ham bands, plus RX on broadcast and public safety could be quite the asset. However, for someone that has any interest in APRS, one of the other two radios I mention is probably a better choice. At only $10 more, the VX-8GR sacrifices ruggedness and broadband receive for APRS functionality and could make a compelling alternative to the VX-7R at the same price point. The very low-power transmit at 50MHz and 220MHz on the VX-7R strikes me as a gimmick feature at best that has very little actual use. Shipped battery: FNB-80LI, 7.4V, 1400 mAh, lithium-ion Kenwood TH-D72A / D72E The Cadillac HT, with APRS and GPS Cost: $464 as of April 30, 2012 It s hard to use any word but incredible to describe this HT, though it does have its own rough edges. Starting with the basics, it has dual transceivers with full duplex (meaning you can be transmitting on one band while receiving on another). Its stored memory system and memory bank scanning system are both a lot easier and faster to use than Yaesu s, and work the same on both transceivers A and B, though perhaps offer slightly less flexibility than Yaesu s (on the VX-7R, a single memory can be a member of more than one group). The quoted operating time is 6 hours, which is probably accurate with APRS and GPS off, matching (or nearly matching) the VX-7R. The TH-D72A has the usual mic and speaker ports. Unlike Yaesu, Kenwood has used the same jack for years, so it is not necessary to purchase new accessories with each upgrade. The unit also has both serial (Kenwood connector) and standard mini USB ports, and even ships with a USB cable. The USB port is designed to interface to a PC, and the radio presents itself as a serial-over-USB device. Kenwood also includes MCP-4A, the programming software for the unit, free. This is one example of several things that Kenwood includes for free but are costly add-ons with Yaesu. Kenwood s configuration menu system is hierarchical. You can scroll through the four top-level menus (RADIO, GPS, APRS, and SKY), then navigate through sub menus. The result is that it s both faster and easier to find what you want than on the Yaesu menus. Plus, as you navigate, the radio shows you the shortcut key to get to a particular option (each level of menu is numbered starting) so, for instance, Menu-1-1-0 takes me directly to the configuration for RX battery saver interval far faster than I could get there on the VX-7R or VX-8GR. Only the APRS menu gets a bit crowded with this setup. The D72A supports Kenwood s Sky Command II an intriguing way of making it essentially a remote for a Kenwood HF rig such as the TS-2000. It works by engaging a crossband repeater for the audio, and setting up a AX.25 packet link for the control. It works, but as the response time to frequency adjustments is measured in seconds, is klunky for tuning around a HF band. A nifty, though perhaps not supremely useful, feature with some hackable potential that I haven t explored yet. The TH-D72A is one of only two shipping general-purpose amateur radios that have a built-in GPS the other is the Yaesu VX-8GR, which I ll review below. The GPS works well, and can get a fix indoors if desired. It has a system of saving waypoints, which you can then navigate back to later (using distance/bearing indicators). It also has a track log. While the track log can t be viewed on the unit, it can be downloaded with the included MCP-4A software and saved in a variety of formats including Google s KML for later importing into Google Maps or Google Earth. I enjoy this feature on trips to see where I went later on. There is also a GPS-only mode, which disables all of the other radio circuitry on the radio for very low power consumption. The GPS navigation screens all support heading-up or north-up indicators. But where this radio really shines is APRS. The D72 has all the basics you d expect sending and receiving beacons, sending and receiving messages and bulletins, etc. It has built-in support for being a digipeater in many different ways, including new-N UI-Trace. Coupled with an MFJ VHF amplifier, this could make a nice temporary (or even permanent) digi. The message interface permits sending, receiving, and shortcuts for replying to messages. Beacon transmission can be completely manual, at set intervals, or using SmartBeaconing. Outbound beacons also support proportional pathing (meaning adjusting via-path to not have wide coverage with every transmission). Also, if not using SmartBeaconing, a decay algorithm can be used to send beacons less frequently when the station s location isn t changing. An alternative tells the system to transmit a beacon every time PTT is released, subject to rate limiting by interval. So lots of choices there. For incoming beacons, there are various filters by callsign pattern, packet type, etc. that control whether they will be processed at all, and what kind of alert (visual or audible) to present. Beacons are also placed in a received beacon list, sorted with the most recent at the top (duplicates from the same station are always removed, so only the most recent is presented.) A long press of the LIST button shows time received and type of device next to the callsign. Selecting a specific beacon shows status text, comment, device type, bearing/distance (with graphical north-up or heading-up display), telemetry (weather information and the like), and a little bit of info about the via-path. Pretty nice to see weather reports from local weather stations on there. You can sort the list by received time, station call, or distance from your position but it doesn t stay sorted by anything other than received time, and takes several seconds to re-sort. Seems like a bit of an oversight there. The D72A supports QSY information in beacons. This can automatically insert the frequency of the other band on the transceiver into your transmitted beacons for you, or give you a quick way to tune to the frequency mentioned on others beacons. Either way, a nice touch. The D72 also supports query packets, such as packets asking your station to transmit its position right now. The TH-D72A can directly interface with several types of weather stations via serial link to read weather sensor information and transmit it into APRS packets directly, with no need for a PC to be in the picture at all. But if you do hook up a PC to it, there are quite a few more possibilities. You can hook it up to PC APRS software (or a hardware device such as the AVMAP) to see locations of other APRS stations on a map. It can emit APRS packets, plus even GPS sentences, down the USB port to the PC. Moreover, in PACKET mode, it acts as a full TNC, with a fairly robust TNC command set. In short, it can do more than APRS; it can also do regular AX.25 such as DX clusters, BBS, nodes, etc. As far as I know, this is the only HT that can interface with a PC in this manner, and only Kenwood s mobile D710 has the same kind of feature set at that. Kenwood includes a lot of documentation with the TH-D72A. The 51-page printed manual is a summary or introductory guide. The included CD-ROM supplies another 75 pages of detailed reference material, and the 92-page in-depth APRS manual has a level of detail that true geeks like me appreciate. I ve referred to Kenwood s documentation more than once while figuring out things about the VX-8GR (which woefully under-documents things like SmartBeaconing). That said, it appears that at least some of the documentation has been copied from the earlier TH-D7A. Page 4 of the printed book is a good example. It talks about the memory effect of the battery, and warns against unplugging and replugging the charger because the charge cycle will be reset and the battery will be over-charged (I find it hard to believe that a Li-Ion charge controller would be that stupid, and furthermore direct observation suggests that it isn t.) Parts of the in-depth APRS guide appear to have been written for the D710, but that s a really minor nit. Unlike Yaesu, Kenwood issues periodic firmware updates to the D72A, which you can apply over USB. They have fixed bugs and added features to the unit over time. Now onto the things that aren t so good about the D72A. I ll start with the battery system, since we were already discussing it. Despite shipping with a much larger battery than the VX-8GR (1800 mAh vs. 1100 mAh), it only manages roughly equal battery life with APRS and GPS engaged. (It will probably do better as a simple voice unit, however.) This has been confirmed by numerous reports on the Internet. I will discuss this topic more below. There is very little else to fault the D72A on, feature-wise. I have a minor nit in that it is impossible to cause the keylock feature to also lock PTT. It would be nice to be able to see raw APRS packets on beacons and messages. Other than that, I can t think of a feature it really lacks. Compared to the VX-7R and VX-8GR, the TH-D72A is significantly larger physically, in every dimension. It is not uncomfortably large, and still fits in my hand fine. But its size is enough different that it feels like the design is dated and could have been more compact if Kenwood would have bothered. It s a nit, sure, but a nontrivial one. With the single exception of the sturdy metal belt clip, the D72 doesn t feel nearly as rugged as the VX-7R, and not even as rugged as the VX-8GR. The keys have a squishy feel to them, the PTT button works fine but has a cheap plastic design (as opposed to the rubberized versions on Yaesu s HTs). The manual doesn t mention it, but one Kenwood brochure mentions and IP54 weatherproofing. That means it is protected against limited ingress of dust and against water sprayed from all directions limited ingress permitted. Contrast that to the VX-7R, which is rated for 30 minutes of submersion at 3ft and has a magnesium case. It seems to be specced similarly to the VX-8GR, though the VX-8GR certainly feels a lot more solid. I have no proof, as I m not about to sacrifice my HTs for science, but I doubt that the TH-D72A would have survived over an hour on a busy road as well as the VX-7R did, and probably not even as well as the VX-8GR would. The case is plastic a stout plastic, but still plastic. Like the VX-8GR, and unlike the VX-7R, the D72A does not have broadband receive. It can receive some bands adjacent to the ham bands it supports, which includes a lot of bands of interest to people with scanners, but still can t receive broadcast FM signals like the VX-7R. Shipped battery: PB-45L, 1800 mAh, lithium-ion Yaesu VX-8GR: Cheaper APRS & GPS Cost: $360 as of 4/30/2012 One could probably call this Yaesu s closest competitor to the D72A. At about $100 cheaper, it doesn t have a featureset that matches Kenwood s, but it does have a few innovations that the D72A lacks. Starting with the basics, it lacks the VX-7R s broadband receive, with the receive roughly matching the TH-D72A s. The 8GR is a dual-transceiver HT, and it works this effectively (moreso than the VX-7R). Yaesu has a bad habit of changing connectors on radios with regularity, and charging an arm and a leg for accessories to boot. Even within the VX-8 line, the 8GR uses different accessories than the 8DR, and of course has different plugs than the 7R (though it appears that the DC power plug is the same, and with some luck you might be able to use a VX-7R speaker/mic on the 8GR, though not the reverse). The menu system on the 8GR is still similar to the 7R, though Yaesu has done several things to improve it. For one, scrolling through the menu now shows several lines of context on the screen at once, making it faster to scan. Also, there are more ways to jump into the menu at a certain location than before and even a small bit of clumsily-implemented hierarchy (there are several settings on the SmartBeaconing page, for instance). The menu key now toggles between various displays: the default frequency/status display, a GPS display, an APRS staion list, and an APRS message list. Just about everything operationally takes more keypresses than the D72A. On the D72A, it s one or two keys (a physical button or F+button) to do almost everything: toggle GPS state, change TNC/APRS state (turn on or off), etc. All of that is in the scrolling menus on the 8GR. The quickest way there, which is documented in the manual, is to press MENU until a certain screen comes up, then press and hold MENU to bring up the settings (defaulting to a certain APRS-related category), then scroll and change settings. It s a pain to turn APRS on and off, because you have to do this process for both the GPS and the APRS modem. Inexplicably, there is a hotkey to change the APRS beacon method (manual, auto, or SmartBeaconing) but not to turn APRS itself on or off. (The D72A has a hotkey to toggle the beaconing, though not to change its method; though in truth, if you do it frequently, the menu shortcuts will get you there rapidly.) Physically, the VX-8GR is closer to the size of the 7R than the size of the D72A, and notably is significantly smaller than the D72A (at least with the standard battery). As noted in the D72A review, the VX-8GR feels more rugged than the D72A, though whether that is actually true or not is hard to pin down. It is definitely less rugged than the VX-7R. The keys require more pressure than the D72A, and in this they are similar to the 7R, but feel satisfyingly solid. Unlike the other radios here, the 8GR does not have a dedicated volume knob, having only one dial on the top. To adjust volume, you hold the volume button on the left while twisting the knob. In practice, this is probably not usually very annoying, but if the radio is on your belt or otherwise not in your hand, this would be a frustration. (I think it would be particularly annoying if using it with a speaker/mic). Setting 99 will let you change the key so you don t have to hold it in, which could probably go a long way to making the situation more tolerable. One unique feature of the VX-8GR, as far as I know completely unique to it, is a vibrating alert. This would most frequently be used with APRS to provide a nearly-silent alert of incoming messages and the like. The screen of the 8G, while similar in size to both the D72A and the 7R, appears to be higher-resolution (or if not, they make better use of it). More information is packed onto the screen at once, and although the D72A does have a small font, it uses it so rarely that lots of scrolling is needed. The 8GR uses a smaller font than the D72A to show content of messages and the like, which makes a much faster experience since there is a lot less scrolling. Neither radio has a setting to change this behavior, so if your eyesight isn t good, you may prefer the D72A anyhow. The VX-8GR supports a heading-up or a north-up view of the received GPS data, but strangely it does not support this when navigating to a received beacon (north-up is the only option). Unlike the TH-D72A, you can engage keylock on the 8GR while someone s beacon is on the display. Also unlike the TH-D72A, you can lock out the PTT key on the 8GR. However, the keylock feature is just a quick press of the power button, which makes me a bit nervous. And worse, if you tell it to lock the PTT key, then it also won t transmit any automatic APRS beacons! There are a lot of things the D72A supports in the APRS area that the 8GR doesn t. Here s a summary of features missing in the 8GR compared to the D72A: voice alert (can t even do it manually since tone/CTCSS are locked out in APRS mode), quick replying to messages, TNC mode (meaning you can t use it with PC-based APRS programs), full AX.25 packet mode, full PC linking, weather station inputs, built-in digipeater mode, heading-up display, GPS track logs, QSY in beacons, quick tuning to others QSY info, responding to position queries, proportional pathing, automatic decay, beaconing on PTT release, sorting of beacon list, hotkey access to many features, and I m sure the list goes on. On the flipside, the only thing feature-wise that you get on the 8GR s APRS implementation that the D72A lacks is ability to inspect raw APRS packets. The VX-8GR has the lowest-capacity battery of the three radios covered in this post, and this is an oft-cited problem for users of the unit. Yeasu does seem to do a much better job of power management than Kenwood, though, so in full APRS and GPS mode, the battery life is roughly the same, amazingly enough. It is unlikely that the talk time on the 8GR would match either the 7R or the D72A, however. Kenwood and Yaesu both have a battery saver feature which cuts power to the RX circuit except for brief checks for signals. Yaesu completely disables this while the APRS modem is on (and STILL matches Kenwood s battery life, despite the smaller battery!) Kenwood doesn t, though recommends changing the signal-check interval from 0.2s to 0.03s in order to avoid truncating APRS packets. Yaesu offers an 1800 mAh extended-life battery which should significantly exceed Kenwood s battery life. Aftermarket vendors have 2000 mAh batteries as well. Either way, expect APRS to be a drain on batteries. More on that below. Although Yaesu includes 168 pages of documentation with the 8GR, don t expect it to match the quality and detail of Kenwood s 218 pages. There are a lot of gimmicky/useless features on the 8GR (WIRES, for one) that get extensive treatment, and there is a fair bit of repetition. I particularly single out the SmartBeaconing documentation on the 8GR as being, at best, woefully incomplete and probably also inaccurate to boot. Read the D72A manuals to figure out how that feature works on your 8GR. I bought the VX-8GR intending it to be something akin to a more versatile TinyTrak. On that score, it mostly succeeds. It may be my go-to rig for use at bicycle rides, marathons, emergencies, etc., while the D72A will probably be the one that goes with me on trips. Shipped battery: FNB-101LI, 7.4V, 1100 mAh, lithium-ion Not covered: VX-8DR poor value for the dollar Cost: $434 ($550 with GPS and GPS connector) Yaesu s top-of-the-line HT is sometimes considered to be the VX-8DR. It maintains the broadband capability of the 7R, is probably almost as rugged as the 7R, and has APRS built in, like the 8GR. It even has an option to add a Bluetooth board. What is doesn t have built-in is a GPS. You can add one on for $77 for the FGPS-2 plus $39 for the CT-136 connector which connects the GPS to the unit. But then you have a unit a lot less rugged than even the D72A (that s a small connector for the GPS), a lot less small, with the same APRS featureset as the 8GR (meaning a lot less than the D72A), which costs about $100 more than the D72A. Frankly I think this makes for a poor APRS package, and would probably only be reasonable for someone that wants other 8DR features and has a desire for some very occasionaly APRS work. Battery Life and APRS One of my first surprises with APRS HTs was the poor battery life I saw. Doing my research seemed to suggest that, contrary to popular opinion, the transmitted beacons were not the culiprit, but the receive circuits were to blame. Because it is not acceptable to cut off the first 200ms of an AX.25 packet like it is with a repeater, the RX circuit is either engaged much more frequently (Kenwood) or continuously (Yaesu), putting a lot more drain on the battery. (This has also been confirmed by multiple others that have done testing) The GPS receiver also adds drain, and on the D72A, so does the TNC, which is always active when APRS is. If you don t care about received beacons, you can dramatically raise the battery life on the D72A by setting the power saver to 1 second or greater as high as 14 hours by one report. On either unit, raising the squelch should have some effect (though not as much; but in any case, weak packets won t decode anyhow, so there s no sense in letting them in.) On the 8GR, getting the extended life 1800 mAh (or aftermarket version) could produce battery life in nearly that same region, without compromising on the RX side of things. Conclusions If you want a HT capable of APRS, seriously think of the TH-D72A before anything else. It is $100 more expensive than the 8GR, but has more than $100 worth of extra features (and included things, like PC link cable and software, that are extra on the 8GR), so it has a better value. If you re particularly budget-conscious or want the most ruggedness you can find, the 7R and 8GR are solid contenders if you can sacrifice the extra APRS features. Update May 1: Clarified what the lack of TNC mode on the VX-8GR means, and also how the VX-8GR PTT keylock also locks out APRS transmissions.

26 April 2012

Wouter Verhelst: Switching to duckduckgo

In the late 90s, google became popular for one reason: because they had a no-nonsense frontpage that loaded quickly and didn't try to play with your mind. Well, at least that was my motivation for switching. The fact that they were using a revolutionary new search algorithm which changed the way you search the web had nothing to do with it, but was a nice extra. Over the years, that small hand-written frontpage has morphed into something else. A behind-the-scenes look at the page shows that it's no longer the hand-written simple form of old, but something horrible that went through a minifier (read: obfuscator). Even so, a quick check against the Internet Wayback machine shows that the size of that page has increased twenty-fold, which is a lot. But I could live with that, since at least it looked superficially similar. Recently, however, they've changed their frontpage so that search-as-you-type is enabled by default. Switching that off requires you to log in. So, you have a choice between giving up your privacy by logging in before you enter a search term, or by having everything you type, including any typos and stuff you may not have confirmed yet, be sent over to a data center god knows where. Additionally, at the first character you type, the front page switches away to the results page, causing me to go "uh?!?" as I try to find where they moved my cursor to. This is annoying. Duckduckgo doesn't do these things; and since they also don't do things like combining my typing skills, phone contact list, calendar, and chat history to figure out that I might be interested in a date, I'm a lot more comfortable using them. So a few days ago, I decided to switch my default search engine in chromium to duckduckgo. It still feels a bit weird, to be using a browser written by one search engine to search something on another; but all in all, it's been a positive experience. And the fact that wikipedia results are shown first, followed by (maybe) one ad, followed by other search results, is refreshing. We'll see how far this gets us.

25 April 2012

Vincent Bernat: XBMC Eden on Debian Wheezy

I bought some HTPC a few years ago to run XBMC, a neat media center solution. At the time, to avoid any problems, I installed it on top of a minimal Ubuntu Lucid installation with the official packages from the team XBMC. Recently, XBMC Eden has been released and XBMC has landed into Debian unstable. It was a good occasion to make the switch. Unofficial XBMC logo for Eden TL;DR: Installing XBMC on Debian Wheezy is quite easy: it almost works out of the box. The big difficulty is the configuration of the remote control: either it works as you expect or you will have to scratch your head over the pile of layers needed to work with a remote control. The configuration of my HTPC is as follows:

Installation

Installing Debian Wheezy Installing Debian Wheezy2 is pretty easy. Nowadays, getting a bootable USB key from a netinst image of Debian Installer for Wheezy is simplified: 
$ sudo dd if=debian-testing-i386-netinst.iso \
>         of=/dev/disk/by-id/usb...
The installation was smooth with the exception of GRUB which was unable to install itself on the disk. This is a known bug when dealing with LVM and it comes with a simple workaround. I hope it will be corrected in time for Wheezy release. While this has little to do with the installation of XBMC, I wanted to test systemd which may become the default init in Debian (at least in Debian GNU/Linux). From README.Debian:
systemd can be installed alongside sysvinit and will not change the behaviour of the system out of the box. This is intentional. To test systemd, add init=/bin/systemd to the kernel command line and then rebooting, or install the systemd-sysv package.
The final system boots in about 15 seconds.

Configuring X Because video decoding in nouveau driver is still a work in progress, the use of the proprietary NVIDIA drivers is mandatory to be able to watch high resolution videos. Therefore, /etc/apt/sources.list should be completed with contrib and non-free repository. Then, you can install the appropriate packages: xserver-xorg-video-nvidia, nvidia-vdpau-driver and xserver-xorg. Here is my /etc/X11/xorg.conf.d/nvidia.conf: 
Section "Device"
    Identifier     "NVidia ION"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    Option         "HWCursor" "False"
    Option         "NoFlip" "False"
    Option         "FlatPanelProperties" "Scaling = Native"
    Option         "DynamicTwinView" "False"
    Option         "ConnectedMonitor" "DFP-1"
    Option         "CustomEDID" "DFP-1:/etc/X11/edid.bin"
    Option         "NoLogo" "True"
EndSection
Section "Extensions"
    Option "Composite" "false"
EndSection
The CustomEDID option allows the driver to get an appropriate EDID even when the AV receiver is off. You can get yours, free of charge, with get-edid from read-edid package.

Installing XBMC Thanks to the work of Andr s Mej a, XBMC is now available in Debian Wheezy. To install it, just type aptitude install xbmc. I have dropped the following xbmc.service in /etc/systemd/system directory: 
[Unit]
Description = XBMC media center
After = syslog.target
[Service]
User = xbmc
Group = xbmc
Type = simple
ExecStart = /usr/bin/xinit /usr/bin/xbmc-standalone -- :0
Restart = on-failure
[Install]
WantedBy = multi-user.target
Enable this service on boot with systemctl enable xbmc.service. You need to allow xbmc user to run X. The simplest way is to run dpkg-reconfigure -plow x11-common and to allow anybody to run X. sudo may be an alternative.

Configuration

Sound While I wanted to use PulseAudio, I want the AV receiver to be able to upmix stereo streams itself. With PulseAudio, it would always receive a 6-channel signal. Therefore, I directly use ALSA. First, unmute the appropriate outputs: 
$ amixer scontrols   grep IEC958
Simple mixer control 'IEC958',0
Simple mixer control 'IEC958 Default PCM',0
Simple mixer control 'IEC958',1
$ amixer sset 'IEC958',0 unmute
$ amixer sset 'IEC958 Default PCM',0 unmute
$ amixer sset 'IEC958',1 unmute
$ sudo systemctl stop alsa-utils.service
The order of channels is incorrect. With the following /etc/asound.conf, we declare a new output, hdmi2, with a different mapping: 
pcm.hdmi2  
  type asym
  playback.pcm  
    type plug
    slave.pcm "remap-surround51"
   
 
pcm.!remap-surround51  
  type route
  slave.pcm "hdmi"
  ttable  
    0.0= 1
    1.1= 1
    2.4= 1
    3.5= 1
    4.2= 1
    5.3= 1
In XBMC, this output should be used instead of the default one. hdmi should still be used for passthrough. To check if each speaker is mapped correctly, one can use speaker-test -D hdmi2 -c 6.

LCD display The LCD display integrated into the SoundGraph iMON is supported by the imon kernel module and the lcdproc package. I have only modified a few lines of /etc/LCDd.conf to make it work: 
[server]
Driver=imonlcd
ServerScreen=off
[imonlcd]
Protocol=1
OnExit=2
Contrast=400

Remote control This is the most difficult part. I have a Logitech Harmony remote which is a great universal remote. Its support in Linux is acceptable: you can configure through Logitech website and use congruity to push the new configuration.

Remote controls and Linux Before Linux 2.6.36, most remote controls would need LIRC to work:
  • The driver receives the signal from the IR receiver and make it available through /dev/lirc.
  • lircd, with the help of a configuration file describing the protocol used by the remote control, will read the signal and turn it into the appropriate LIRC code.
  • XBMC connects to lircd and receives incoming LIRC codes. It will translate them to an XBMC command. This translation is specified in Lircmap.xml.
  • XBMC maps each command to an action (like Play, Fullscreen, ) using a keymap. This keymap can handle commands received by a remote control, but also by a keyboard, a mouse or a joystick.
Since Linux 2.6.36, remote controls will be mapped as a generic input device (just like a keyboard):
  • The driver receives the signal from the IR receiver.
  • The signal will be handled by a decoder. The configuration of this decoder is done in userland by ir-keytable. The decoder will turn the signal into the appropriate event (usually, some keypress).
  • X will listen to those events and turn them into X key events.
  • XBMC will receive them and use the appropriate keymap to turn them into actions.
And to add more complexity to the mix, in this last case, you can still use LIRC: lircd will listen to events generated by the kernel and turn them into LIRC codes. This can be very confusing. Moreover, the SoundGraph iMON IR receiver accepts two IR protocols: the iMON protocol and the RC-6 one. The Linux driver accepts both of them but uses the first one by default. The RC-6 protocol is the protocol used by many MCE remote controls. I hope you are still with me here.

The easy way To get a reasonable configuration out of the box, here is how to configure each layer:
Logitech Harmony remote
Configure it as a Microsoft branded Media Center PC: Windows Media Center SE.
iMON IR receiver
It must use RC-6 protocol. See below for more details.
LIRC
In /etc/lirc/hardware.conf, put DEVICE=/dev/input/by-id/usb-15c2_0038-event-if00 and DRIVER=devinput. In /etc/lirc/lircd.conf, just put include "/usr/share/lirc/remotes/devinput/lircd.conf.devinput".
XBMC
With the previous bits done, it should just work out of the box.
To switch to RC-6 protocol, install the ir-keytable package and use the following commands: 
$ sudo modprobe rc-imon-mce
$ sudo ir-keytable -s rc0 -p rc-6 -c -w /lib/udev/rc_keymaps/imon_mce
Read imon_mce table
Old keytable cleared
Wrote 77 keycode(s) to driver
Protocols changed to RC-6
To make the change permanent, add the rc-imon-mce module to /etc/modules and create /etc/udev/rules.d/90-imon.rules with the following content: 
# Override the keytable for iMON
ACTION=="add change", SUBSYSTEM=="rc", DRV_NAME="imon", \
   RUN+="/usr/bin/ir-keytable -s $name -p rc-6 -c -w /lib/udev/rc_keymaps/imon_mce"

The hard way Now, you may want to bind custom actions to some (physical or virtual) buttons. Basically, you are left with two solutions:
  1. Start from the basic configuration with LIRC and add more buttons at each levels (there are five of them!).
  2. Remove LIRC and start with the Logitech Harmony acting as a Microsoft MCE keyboard.
The first option can be quite difficult. You need to find an unused code for the Logitech Harmony. You can try to make it learn a new code if you have some RC-6 remote control. Then, you need to ensure that this code will be present in the keytable used by ir-keytable. If not, you need to add it. That s not easy since you need a to enable some debug stuff in the kernel to find the appropriate scancode. After that, the code needs to be translated in lircd.conf. You will then have to translate it again in Lircmap.xml. At least, you need to add it to a keymap in XBMC. The other way is not ideal but seems less cumbersome. The first step is to configure the Logitech Harmony as a Microsoft MCE keyboard: it has a lot of available keys. Because of the lack of multimedia keys, let s match the keyboard configuration of XBMC:
Button Command Button Command
Channel Down PageDown Stop X
Channel Up PageUp Skip back Comma
Prev Backspace Skip forward .
Up DirectionUp Play P
Down DirectionDown Rewind R
Left DirectionLeft Fast forward F
Right DirectionRight Star Delete
OK Enter Pound W
Menu C Red F1
Exit Esc Green F2
Guide Tab Yellow F3
Info I Blue F4
Unfortunately, the keytables provided with ir-keytable are not complete enough. I have built a more complete table3. With this table and the bindings described above, most functions will work out of the box without LIRC. Additional keys can be configured in a dedicated keymap4. Here is an excerpt of mine: 
<keymap>
 <global>
  <keyboard>
    <end>XBMC.ShutDown()</end>
    <f1>XBMC.ActivateWindow(MusicLibrary)</f1>
    <f2>XBMC.ActivateWindow(Videos,TvShowTitles)</f2>
    <f3>XBMC.ActivateWindow(Videos,MovieTitles)</f3>
    <f4>XBMC.ActivateWindow(Weather)</f4>
  </keyboard>
 </global>
 <FullscreenVideo>
  <keyboard>
   <opensquarebracket>SubtitleDelayMinus</opensquarebracket>
   <closesquarebracket>SubtitleDelayPlus</closesquarebracket>
   <f6>xbmc.runscript(script.xbmc.subtitles)</f6>
  </keyboard>
 </FullscreenVideo>
</keymap>

FTP Instead of using SSH, I prefer to drop new files with anonymous FTP. vsftpd fits this purpose. Here is my configuration file: 
listen=YES
xferlog_enable=YES
use_localtime=YES
setproctitle_enable=YES
secure_chroot_dir=/var/run/vsftpd/empty
nopriv_user=ftp
ftpd_banner=XBMC
hide_ids=YES
ftp_username=xbmc
anon_umask=022
anon_root=/home/xbmc/media
anonymous_enable=YES
write_enable=YES
anon_upload_enable=YES
anon_world_readable_only=YES
It is currently not compatible with systemd (see bug #670308). I have removed the symlink in /etc/rc2.d and I have used the following unit file: 
[Unit]
Description=Vsftpd ftp daemon
After=syslog.target network.target
[Service]
Type=simple
ExecStart=/usr/sbin/vsftpd /etc/vsftpd.conf
ExecReload=/bin/kill -HUP $MAINPID
ExecStartPre=-/bin/mkdir -p /var/run/vsftpd/empty
[Install]
WantedBy=multi-user.target

Miscellaneous
  1. In /etc/default/grub, reduce TIMEOUT to 0 to shorten the boot time.
  2. Enabling dirty regions can help speed up XBMC.
  3. aptitude install upower pm-utils to be able to shutdown/suspend from XBMC. Since XBMC was configured to be started outside any session, you need to explicitely give the appropriate rights by creating the following /var/lib/polkit-1/localauthority/50-local.d/xbmc.pkla: 
    [Actions for xbmc user]
Identity=unix-user:xbmc
Action=org.freedesktop.upower.*;org.freedesktop.consolekit.system.*
ResultAny=yes
ResultInactive=yes
ResultActive=yes
  1. The readibility of the LCD screen is very bad. You should look at the VFD version. The IR receiver reception is poor. The provided remote control is a joke.
  2. Debian Wheezy is not yet released. If you are unfamiliar with Debian, it may be cumbersome to maintain it until the freeze happens in a few months.
  3. Some keys are missing from the provided table. For example, there is no exclamation mark. While there is a scan code for such a key in RC-6 protocol, there is no appropriate key code to translate to: on a QWERTY keyboard, the exclamation mark is on the same key as the number 1. It is possible to map it to some other key code, but the mapping would have been difficult to use.
  4. For example, in ~/.xbmc/userdata/keymaps/harmony.xml.

23 April 2012

Benjamin Drung: Unreliable SSDs

Today my system froze and failed to reboot. I plugged in an Ubuntu live USB stick and booted from it. Then I discovered the problem that my Intel SSD 320 broke. The output of hdparm is attached to the end of the blog post. You can see that the device size is reduced to 8 MB from 120 GB and the serial number is called BAD_CTX 00000159. The firmware of the SSD was up-to-date and the last firmware update should have fixed the 8 MB bug. The Intel SSD 320 is my second SSD. My first SSD was a Super Talent Ultradrive GX 64GB, which died after around fifteen month of heavy use. It left a big bunch of my data corruption behind. SSDs seems to be very unreliable. Both SSDs died, but I cant remember that one of my HDDs died. $ sudo hdparm -I /dev/sda /dev/sda: ATA device, with non-removable media
Model Number: INTEL SSDSA2CW120G3
Serial Number: BAD_CTX 00000159
Firmware Revision: 4PC10362
Transport: Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6
Standards:
Used: unknown (minor revision code 0x0029)
Supported: 8 7 6 5
Likely used: 8
Configuration:
Logical max current
cylinders 16383 16
heads 16 16
sectors/track 63 63
--
CHS current addressable sectors: 16128
LBA user addressable sectors: 16384
LBA48 user addressable sectors: 16384
Logical Sector size: 512 bytes
Physical Sector size: 512 bytes
device size with M = 1024*1024: 8 MBytes
device size with M = 1000*1000: 8 MBytes
cache/buffer size = unknown
Nominal Media Rotation Rate: Solid State Device
Capabilities:
LBA, IORDY(can be disabled)
Standby timer values: spec'd by Standard, no device specific minimum
R/W multiple sector transfer: Max = 16 Current = 16
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
Enabled Supported:
Security Mode feature set
* Power Management feature set
* Write cache
* Look-ahead
* Host Protected Area feature set
* WRITE_BUFFER command
* READ_BUFFER command
* NOP cmd
* DOWNLOAD_MICROCODE
SET_MAX security extension
* 48-bit Address feature set
* Device Configuration Overlay feature set
* Mandatory FLUSH_CACHE
* FLUSH_CACHE_EXT
* General Purpose Logging feature set
* WRITE_ DMA MULTIPLE _FUA_EXT
* 64-bit World wide name
* IDLE_IMMEDIATE with UNLOAD
* WRITE_UNCORRECTABLE_EXT command
* READ,WRITE _DMA_EXT_GPL commands
* Segmented DOWNLOAD_MICROCODE
* Gen1 signaling speed (1.5Gb/s)
* Gen2 signaling speed (3.0Gb/s)
* Phy event counters
* Software settings preservation
* SMART Command Transport (SCT) feature set
* SCT LBA Segment Access (AC2)
* SCT Error Recovery Control (AC3)
* SCT Features Control (AC4)
* SCT Data Tables (AC5)
* Data Set Management TRIM supported (limit 8 blocks)
* Deterministic read ZEROs after TRIM
Security:
Master password revision code = 65534
supported
not enabled
not locked
frozen
not expired: security count
supported: enhanced erase
2min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 500151795951d4b9
NAA : 5
IEEE OUI : 001517
Unique ID : 95951d4b9
Checksum: correct

Next.

Previous.