I just released version 1.3.0 of Cubemap, my high-performance video reflector. For a change, both new features are from (indirect) user requests; someone wanted support for raw TS inputs and it was easy enough to add. And then I heard a rumor that people had found Cubemap useless because it was logging so much . Namely, if you have a stream that's down, Cubemap will connect to it every 200 ms, and log two lines for every failed connection attempt. Now, why people discard software on ~50 MB/day of logs (more like 50 kB/day after compression) on a broken setup (if you have a stream that's not working, why not just remove it from the config file and reload?) instead of just asking the author is beyond me, but hey, eventually it reached my ears, and after a grand half hour of programming, there's rate-limiting of logging failed connection attempts. :-) The new version hasn't hit Debian unstable yet, but I'm sure it will very soon.

Hello Markus Raab, the author of Elektra project, has created a survey to get FLOSS developer s point of view on the configuration of application. If you are a developer, please fill this survey to help Markus work on improving application configuration management. Feeling this survey should take about 15 mns. Note that the survey will close on July 18th. The fact that this blog comes 1 month after the beginning of the survey is entirely my fault. Sorry about that All the best
Tagged: configuration, Perl

Author: ceridwen After testing and looking at the code for Plumbum, I decided that it wouldn't work for my purposes. When a command is created by something like remote['ls'], it actually looks up which ls executable will be run and uses in the command object an internal representation of a path like /bin/ls. To make it work with autopkgtest's code would have required writing some kind of middle layer that would take all of the Plumbum code that makes subprocess calls, does path lookups, or uses the Python standard library to access OS functions and convert them into shell scripts. Another similar library, sh, has the same problems. I think there's a strong argument that something like Plumbum's or sh's API would be much easier to work with than adt_testbed, and I may take steps to improve it at some point, but for the moment I've focused on getting reprotest working with the existing autopkgtest/adt_testbed API. To do this, I created a minimalistic shell AST library in _shell_ast.py using the POSIX shell grammar. I omitted a lot of functionality that wasn't immediately relevant for reprotest and simplified the AST in some places to make it easier to work. Using this, it generates a shell script and runs it with adt_testbed.Testbed.execute(). With these pieces in place, the tests succeed for both null and schroot! I haven't tested the rest of the containers, but barring further issues with autopkgtest, I expect they should work. At this point, my goal is to push the next release out by next week, though as usual it will depend on how many snags I hit in the process. I see the following as the remaining blockers:

• Test chroot and qemu in addition to null and schroot.
• PyPi still doesn't install the scripts in virt/ properly.
• While I fixed part of adt_testbed's error handling, some build failures still cause it to hang, and I have to kill the process by hand.
• Better user documentation. I don't have time to be thorough, but I can provide a few more pointers.

Just finished my first book of Jonas Jonasson, a Swedish journalist and author. Most famous for his book The Hundred-Year-Old Man Who Climbed Out the Window and Disappeared, but author of two others. The one I read was The Girl Who Saved the King of Sweden, which strange enough became in German Die Analphabetin die rechnen konnte (The analphabet who could compute). The story recounts the countless turns the life of Nombeko Mayeki, a black girl born in Soweto as latrine cleaner, who manages to save the Swedish king as well as most of the world from an atomic desaster by first getting driven over by a drunkard of South African nuclear bomb engineer, then meeting a clique of three Chinese sisters excelling in faking antiquities, and two Mossad agents. With the (unwilling) help of those agents she escapes to Sweden (including the atomic bomb) where she meets twins of a psychotic father who brought them up as one child so that the spare one can eradicate the Swedish monarchy. After many twists and setbacks, including several meetings with the Chinese premier Hu Jintao, she finally manages to get rid of the atomic bomb, get her undercover twin a real identity, and set up a proper life ah, and not to forget, save the King of Sweden! A fast paced, surprisingly funny and lovely story about how little things can change our lives completely.

At the begin of this week, I thought my architecture needed to be closer to the call . The goal was to create one class per call to save the different information. With this method, I only need to call the instance who I am interested and I can easly pull the information.
So, I began to rewrite my architecture on the daemon to create an instance of my class link directly with the callID.
After implementing it, I was really disappointed. This class was hardly to call from the upper software layers. Indeed, I didn t know what is the current call display on my client.
I change my mind and I rewrite the architecture again. I observed the information I want to pull (frame rate, bandwidth, resolution ) and they are all generating in my daemon. Therefore, it will update every time something change in the client. So, I just need to catch it and send it to the upper software layers. My new architecture is simply a singleton because I just need one instance and I need to pull it from everywhere in my program.

Beyond that, I wanted to pull some information about the video (frame rate, resolution, codec for the local and remote computer). So, I looked to understand how the frame generate work. Now I can pull:
-Local and remote video codec
-Local and remote frame rate
-Remote audio codec
-Remote resolution
-CallID

Next week, I will begin with working on creating an API on the client library. After that, I will continue to retrieve other information.

Today, we have completed our first Britney run with mips64el enabled in testing. At the current time, the set of packages in mips64el are not very connected (and you probably cannot even install build-essential yet[1]). Hopefully this will change over the next few days. For now, Britney does not enforce installability of packages on mips64el in general, so do not expect the architecture to be stable at the moment. Cheat sheet for package maintainers:

• Issues with builds (only) on mips64el are not blockers for testing migration (nor RC yet).
  • Such bugs should be filed as important for now (unless they also affect a release architecture, in which case you should still make it an RC bug)
• Your package be an older version on mips64el compared to other architectures.
• Britney may decide to break your package on mips64el if it means something else can migrate on a release architecture.

We will slowly remove these special cases for mips64el as it matures in testing. [1] Update on this: mips64el currently does not have a libc library yet, so build-essential is definitely not installable at the moment. It will hopefully migrate very soon.
Filed under: Debian, Release-Team

Propellor is my second big Haskell program. I recently described the motivation for it like this, in a proposal for a Linux.Conf.Au talk:

The configuration of Linux hosts has become increasingly declarative, managed by tools like puppet and ansible, or by the composition of containers. But if a server is a collection of declarative properties, how do you make sure that changes to that configuration make sense? You can test them, but eventually it's 3 AM and you have an emergency fix that needs to go live immediately. Data types to the rescue! While data types are usually used to prevent eg, combining an Int and a Bool, they can be used at a much more abstract level, for example to prevent combining a property that needs a Debian system with a property that needs a Red Hat system. Propellor leverages Haskell's type system to prove the consistency of the properties it will apply to a host.

The real origin story though, is that I wanted to finally start using configuration management, but the tools for it all seemed very complicated and built on shaky foundations (like piles of yaml), and it seemed it would be easier to write my own than deal with that. Meanwhile, I had Haskell burning a hole in my pocket, ready to be used in a second large project after git-annex. Propellor has averaged around 2.5 contributions per month from users since it got started, but increasing numbers recently. That's despite having many fewer users than git-annex, which remember gets perhaps 1 patch per month. Of course, I've "cheated" by making sure that propellor's users know Haskell, or are willing to learn some. And, propellor is very compositional; adding a new property to it is not likely to be complicated by any of the existing code. So it's easy to extend, if you're able to use it. At this point propellor has a small community of regular contributors, and I spend some pleasant weekend afternoons reviewing and merging their work. Much of my best work on propellor has involved keeping the behavior of the program the same while making its types better, to prevent mistakes. Propellor's core data types have evolved much more than in any program I worked on before. That's exciting! Next: twenty years of free software -- part 13 past and future

I have finally made a version of the queries code that can viably be integrated into the master branch of OpenDHT. I am awaiting my mentor s approval and/or comments.

What s done Queries. The library will provide the additional following functions in its API:

void get(InfoHash id, GetCallbackSimple cb, DoneCallback donecb= , Value::Filter f = Value::AllFilter(), Where w =  )  
void query(const InfoHash& hash, QueryCallback cb, DoneCallback done_cb =  , Query q =  );

The structure Where in the first signature will allow the user to narrow the set of values received through the network those that verify the where statement. The Where actually encapsulates a statement of the following SQL-ish form: SELECT * WHERE <some_field>=<some_field_value>. Also, the DhtRunner::query function will let the user do something similar to what s explained in the last paragraph, but where the returned data is encapsulated in FieldValueIndex structure instead of Value. This structure has a std::map<Value::Field, FieldValue>. You can think of the FieldValueIndex as a subset of fields of a given Value. The Query structure then allows you to create both Select and Where statements .

What s next

• Value pagination. I have begun working on this and I now have a more clear idea of the first step to accomplish. I have to redesign the get operation callbacks calling process by making a callback execution per SearchNode instead of per Search (list of SearchNode). This will let us properly write the value pagination code with node concurrency in mind. This will therefor make sure we don t request a value from a node if it doesn t stores it;
• Optimizing announce operations;

The problem So. I was stuck with a container of tuples that I wanted to see in a Qt view (QTableView, QtQuick ListView or similar). So how to do that? Another problem: I haven t been doing fun things with templates recently. A solution? After a bit of hacking, it seems like it can just be done like

typedef std::tuple<std::string, QString> Element;
typedef std::vector List;
List list =   std::make_tuple("first", "second"), 
              std::make_tuple("third", "fourth")  ;
std::unique_ptr<TableModel> magic = createTableModel(list);
QTableView view;
view.setModel(magic->model());

and tada:
Of course, we are also QtQuick friendly

std::unique_ptr<ListModel>List>> magic = createListModel(list);
// expose magic->model() to your quickview

and a delegate containing the following

Text  
    text: role0
 
Text  
    text: role1
 

can give:
But enough about creation. Whattabout manipulation? Luckily we got you covered. Insert two extra rows at position 1?

auto lines =   std::make_tuple("extra", "extra"),
               std::make_tuple("extra2","extra2")  ;
magic->insertRows(1,lines.begin(), lines.end());

Append a row?

magic->appendRow(std::make_tuple("",""));

Remove 2 rows at position 3?

magic->removeRow(3,2);

Replace the underlying list?

List newList;
// fill list
magic->reset(newList);

Read-only looping over the elements?

for(const Element& e : magic->list())
 
    ...
 

The Qt model of course also accepts setData calls. Future? If anyone is interested I will polish the code a bit and publish it. If that s the case, how should I name this thing? And I did get around doing fun things with templates again.

The procps developers are happy to announce that version 3.3.12 of procps was released today. This version has a mixture of bug fixes and enhancements. This unfortunately means another API bump but we are hoping this will be fixed with the new library API coming soon. procps is developed on gitlab and the new version of procps can be found at https://gitlab.com/procps-ng/procps/tree/newlib procps 3.3.12 can be found at https://gitlab.com/procps-ng/procps/tags/v3.3.12 From the NEWS file, procps 3.1.12 has the following:

• build: formerly optional enable-oomem unconditional
• free: man document rewritten for shared Debian #755233
• free: interpret intervals in non-locale way Debian #692113
• kill: report error if cannot kill process Debian #733172
• library: refine calculation of cached memory
• library: find tty quicker Debian #770215
• library: eliminate threads display inconsistencies Redhat #1284091
• pidof: check cmd if space found in argv0
• pmap: fixed detail parsing on long mapping lines
• pmap: fix occasional incorrect memory usage values Redhat #1262864
• ps: sort by cgroup Debian #692279
• ps: display control group name with -o cgname
• ps: fallback to attr/current for context Debian #786956
• ps: enabled broken thcount option Redhat #1174313
• tests: conditionally add prctl Debian #816237
• top: displays the 3 new linux-4.5 RES memory fields
• top: man page memory fields corrected + new narrative
• top: added display of CGNAME (control group name)
• top: is now more responsive to cpus brought online
• top: namespace cols use suppressible zero

We are hoping this will be the last one to use the old API and the new format API ( imaginatively called newlib ) will be used in subsequent releases. Feedback for this and any other version of procps can be sent to either the issue tracker or the development email list.

My experiences with Amazon reviewing have been somewhat unusual. A review of a smart switch I wrote received enough attention that the vendor pulled the product from Amazon. At the time of writing, I'm ranked as around the 2750th best reviewer on Amazon despite having a total of 18 reviews. But the world of Amazon reviews is even stranger than that, and the past couple of weeks have given me some insight into it.

Amazon's success is fairly phenomenal. It's estimated that there's over 50 million people in the US paying $100 a year to get free shipping on Amazon purchases, and combined with Amazon's surprisingly customer friendly service there's a lot of people with a very strong preference for choosing Amazon rather than any other retailer. If you're not on Amazon, you're hurting your sales.

And if you're an established brand, this works pretty well. Some people will search for your product directly and buy it, leaving reviews. Well reviewed products appear higher up in search results, so people searching for an item type rather than a brand will still see your product appear early in the search results, in turn driving sales. Some proportion of those customers will leave reviews, which helps keep your product high up in the results. As long as your products aren't utterly dreadful, you'll probably maintain that position.

But if you're a brand nobody's ever heard of, things are more difficult. People are unlikely to search for your product directly, so you're relying on turning up in the results for more generic terms. But if you're selling a more generic kind of item (say, a Bluetooth smart bulb) then there's probably a number of other brands nobody's ever heard of selling almost identical objects. If there's no reason for anybody to choose your product then you're probably not going to get any reviews and you're not going to move up the search rankings. Even if your product is better than the competition, a small number of sales means a tiny number of reviews. By the time that number's large enough to matter, you're probably onto a new product cycle.

In summary: if nobody's ever heard of you, you need reviews but you're probably not getting any.

The old way of doing this was to send review samples to journalists, but nobody's going to run a comprehensive review of 3000 different USB cables and even if they did almost nobody would read it before making a decision on Amazon. You need Amazon reviews, but you're not getting any. The obvious solution is to send review samples to people who will leave Amazon reviews. This is where things start getting more dubious.

Amazon run a program called Vine which is intended to solve this problem. Send samples to Amazon and they'll distribute them to a subset of trusted reviewers. These reviewers write a review as normal, and Amazon tag the review with a "Vine Voice" badge which indicates to readers that the reviewer received the product for free. But participation in Vine is apparently expensive, and so there's a proliferation of sites like Snagshout or AMZ Review Trader that use a different model. There's no requirement that you be an existing trusted reviewer and the product probably isn't free. You sign up, choose a product, receive a discount code and buy it from Amazon. You then have a couple of weeks to leave a review, and if you fail to do so you'll lose access to the service. This is completely acceptable under Amazon's rules, which state "If you receive a free or discounted product in exchange for your review, you must clearly and conspicuously disclose that fact". So far, so reasonable.

In reality it's worse than that, with several opportunities to game the system. AMZ Review Trader makes it clear to sellers that they can choose reviewers based on past reviews, giving customers an incentive to leave good reviews in order to keep receiving discounted products. Some customers take full advantage of this, leaving a giant number of 5 star reviews for products they clearly haven't tested and then (presumably) reselling them. What's surprising is that this kind of cynicism works both ways. Some sellers provide two listings for the same product, the second being significantly more expensive than the first. They then offer an attractive discount for the more expensive listing in return for a review, taking it down to approximately the same price as the original item. Once the reviews are in, they can remove the first listing and drop the price of the second to the original price point.

The end result is a bunch of reviews that are nominally honest but are tied to perverse incentives. In effect, the overall star rating tells you almost nothing - you still need to actually read the reviews to gain any insight into whether the customer actually used the product. And when you do write an honest review that the seller doesn't like, they may engage in heavy handed tactics in an attempt to make the review go away.

It's hard to avoid the conclusion that Amazon's review model is broken, but it's not obvious how to fix it. When search ranking is tied to reviews, companies have a strong incentive to do whatever it takes to obtain positive reviews. What we're left with for now is having to laboriously click through a number of products to see whether their rankings come from thoughtful and detailed reviews or are just a mass of 5 star one liners.

comments

Author: ceridwen With some more help from Martin Pitt, it became clear to me that my previous mental model of how autopkgtest worked is very different from how it does work. I'll illustrate by borrowing my previous example. I know that schroot has the following behavior:

The default behaviour is as follows (all directory paths are inside the chroot). A login shell is run in the current working directory. If this is not available, it will try $HOME (when --preserve-environment is used), then the user's home directory, and / inside the chroot in turn. A command is always run in the current working directory inside the chroot. If none of the directories are available, schroot will exit with an error status.

I was naively thinking that the way autopkgtest would work is that it would set the current working directory of the schroot call and the ensuing subprocess call would thus take place in that directory inside the schroot. That is not how it works. If you want to change directories inside the virtual server, you have to use cd. The same is true of, at least, environment variables, which have their own specific handling in the adt_testbed.Testbed methods but have to be passed as strings, and umask. I'm assuming this is because the direct methods with qemu images or LXC containers don't work. What this means is that I was thinking about the problem the wrong way: what reprotest needs to do is generate shell scripts. This is how autopkgtest works. If this goes beyond laying out commands linearly one after another, for instance if it demands conditionals or other nested constructs, the right way to do it is to build an abstract syntax tree representation of the shell script and then convert it to code. Whether I need more complicated shell scripts depends on my approach to handling state in the containers. I need to know what state persists across separate command executions: if I call adt_testbed.Testbed.execute() twice, what if any changes I make to the container will carry forward from the first to the second? There are three categories here. First, some properties of a system aren't preserved even from one command execution to the next, like working directory and environment variables. (I thought working directory would be preserved, but it's not). The second is state that persists while the testbed is open and is then automatically reverted when it's closed, like files copied into temporary directories on the tesbed. The third is state that persists across different sessions on the same container and must be cleaned up by reprotest. It's worth noting that which state falls into which category may vary by the container in question, though for the most part I can either do unnecessary cleanup or issue unnecessary commands to handle the differences. autopkgtest itself has a very different approach to cleanup, as it relies almost entirely on the builtin reversion capabilities from some of its containers. I would prefer to avoid doing the same, partly because I know that some of the modifications I need to make, for instance creating new users or mounting disorderfs, can't be reverted by the faster, simpler containers like schroot. From discussions with Lunar, I think that the variations that correspond to environment variables (captures_environment, home, locales, path, and timezone) fall into the first category, but because of the special handling for them they don't require sending a separate command. The shell (bash foo) accepts a script or command as an argument so it also doesn't need a separate command. Setting the working directory and umask require separate commands that have to be joined. On Linux, setarch also accepts a command/script as an argument so can be handled like the shell, but there's no unified POSIX protocol for mocking uname so other OSes will require different approaches. Users, groups, file ordering, host, and domain will require cleanup in all containers except for (maybe) qemu. If I want to handle the cleanup in the shell scripts themselves, I need conditionals so that for instance the shell script only tries to unmount disorderfs if disorderfs was successfully mounted. This approach would simplify the error handling problems I've had before, where when a build crashes cleanup code run from Python doesn't get run until after the testbed stop accepting commands. Lunar suggested the Plumbum library to me, and I think I can use it to avoid writing my own shell AST library. It has a method that converts the Python representation of a shell script into a string that can be passed to Testbed.command(). Integrating Plumbum to generate the necessary scripts is where I'm going in the next week. Any feedback on any of this is welcome. I'm also curious what other projects are using autopkgtest code. Holger brought to my attention piuparts, which brings the list up to four that I'm aware of, autopkgtest itself, sbuild, piuparts, and now reprotest.

wlc 0.4, a command line utility for Weblate, has been just released. This release doesn't bring much changes, but still worth announcing. The most important change is that development repository has been moved under WeblateOrg organization at GitHub, you can now find it at https://github.com/WeblateOrg/wlc. Another important news is that Debian package is currently waiting in NEW queue and will hopefully soon hit unstable. wlc is built on API introduced in Weblate 2.6 and still being in development. Several commands from wlc will not work properly if executed against Weblate 2.6, first fully supported version will be 2.7 (current git is okay as well, it is now running on both demo and hosting servers). You can usage examples in the wlc documentation.

Filed under: Debian English SUSE Weblate 0 comments

I've spent the past days updating several of my websites to be "responsive". Mostly that means I open the site in firefox then press Ctrl-alt-m to switch to mobile-view. Once I have the mobile-view I then fix the site to look good in small small space. Because my general design skills are poor I've been fixing most sites by moving to bootstrap, and ensuring that I don't use headers/footers that are fixed-position. Beyond the fixes to appearances I've also started rationalizing the domains, migrating content across to new homes. I've got a provisional theme setup at steve.fi, and I've moved my blog over there too. The plan for blog-migration went well:

• Setup a redirect to from https://blog.steve.org.uk to https://blog.steve.fi/
• Replace the old feed with a CGI script which outputs one post a day, telling visitors to update their feed.
  • This just generates one post, but the UUID of the post has the current date in it. That means it will always be fresh, and always be visible.
• Updated the template/layout on the new site to use bootstrap.

The plan was originally to setup a HTTP-redirect, but I realized that this would mean I'd need to keep the redirect in-place forever, as visitors would have no incentive to fix their links, or update their feeds. By adding the fake-RSS-feed, pointing to the new location, I am able to assume that eventually people will update, and I can drop the dns record for blog.steve.org.uk entirely - Already google seems to have updated its spidering and searching shows the new domain already.

Just over a month ago I ordered a new Nexus 6P [1]. I ve had it for over a month now and it s time to review it and the Samsung Galaxy S5 Mini I also bought. Security The first noteworthy thing about this phone is the fingerprint scanner on the back. The recommended configuration is to use your fingerprint for unlocking the phone which allows a single touch on the scanner to unlock the screen without the need to press any other buttons. To unlock with a pattern or password you need to first press the power button to get the phone s attention. I have been considering registering a fingerprint from my non-dominant hand to reduce the incidence of accidentally unlocking it when carrying it or fiddling with it. The phone won t complete the boot process before being unlocked. This is a good security feature. Android version 6 doesn t assign permissions to apps at install time, they have to be enabled at run time (at least for apps that support Android 6). So you get lots of questions while running apps about what they are permitted to do. Unfortunately there s no allow for the duration of this session option. A new Android feature prevents changing security settings when there is an overlay running . The phone instructs you to disable overlay access for the app in question but that s not necessary. All that is necessary is for the app to stop using the overlay feature. I use the Twilight app [2] to dim the screen and use redder colors at night. When I want to change settings at night I just have to pause that app and there s no need to remove the access from it note that all the web pages and online documentation saying otherwise is wrong. Another new feature is to not require unlocking while at home. This can be a convenience feature but fingerprint unlocking is so easy that it doesn t provide much benefit. The downside of enabling this is that if someone stole your phone they could visit your home to get it unlocked. Also police who didn t have a warrant permitting search of a phone could do so anyway without needing to compel the owner to give up the password. Design This is one of the 2 most attractive phones I ve owned (the other being the sparkly Nexus 4). I think that the general impression of the appearance is positive as there are transparent cases on sale. My phone is white and reminds me of EVE from the movie Wall-E. Cables This phone uses the USB Type-C connector, which isn t news to anyone. What I didn t realise is that full USB-C requires that connector at both ends as it s not permitted to have a data cable with USB-C at the device and and USB-A at the host end. The Nexus 6P ships with a 1M long charging cable that has USB-C at both ends and a ~10cm charging cable with USB-C at one end and type A at the other (for the old batteries and the PCs that don t have USB-C). I bought some 2M long USB-C to USB-A cables for charging my new phone with my old chargers, but I haven t yet got a 1M long cable. Sometimes I need a cable that s longer than 10cm but shorter than 2M. The USB-C cables are all significantly thicker than older USB cables. Part of that would be due to having many more wires but presumably part of it would be due to having thicker power wires for delivering 3A. I haven t measured power draw but it does seem to charge faster than older phones. Overall the process of converting to USB-C is going to be a lot more inconvenient than USB SuperSpeed (which I could basically ignore as non-SuperSpeed connectors worked). It will be good when laptops with USB-C support become common, it should allow thinner laptops with more ports. One problem I initially had with my Samsung Galaxy Note 3 was the Micro-USB SuperSpeed socket on the phone being more fiddly for the Micro-USB charging plug I used. After a while I got used to that but it was still an annoyance. Having a symmetrical plug that can go into the phone either way is a significant convenience. Calendars and Contacts I share most phone contacts with my wife and also have another list that is separate. In the past I had used the Samsung contacts system for the contacts that were specific to my phone and a Google account for contacts that are shared between our phones. Now that I m using a non-Samsung phone I got another Gmail account for the purpose of storing contacts. Fortunately you can get as many Gmail accounts as you want. But it would be nice if Google supported multiple contact lists and multiple calendars on a single account. Samsung Galaxy S5 Mini Shortly after buying the Nexus 6P I decided that I spend enough time in pools and hot tubs that having a waterproof phone would be a good idea. Probably most people wouldn t consider reading email in a hot tub on a cruise ship to be an ideal holiday, but it works for me. The Galaxy S5 Mini seems to be the cheapest new phone that s waterproof. It is small and has a relatively low resolution screen, but it s more than adequate for a device that I ll use for an average of a few hours a week. I don t plan to get a SIM for it, I ll just use Wifi from my main phone. One noteworthy thing is the amount of bloatware on the Samsung. Usually when configuring a new phone I m so excited about fancy new hardware that I don t notice it much. But this time buying the new phone wasn t particularly exciting as I had just bought a phone that s much better. So I had more time to notice all the annoyances of having to download updates to Samsung apps that I ll never use. The Samsung device manager facility has been useful for me in the past and the Samsung contact list was useful for keeping a second address book until I got a Nexus phone. But most of the Samsung apps and 3d party apps aren t useful at all. It s bad enough having to install all the Google core apps. I ve never read mail from my Gmail account on my phone. I use Fetchmail to transfer it to an IMAP folder on my personal mail server and I d rather not have the Gmail app on my Android devices. Having any apps other than the bare minimum seems like a bad idea, more apps in the Android image means larger downloads for an over-the-air update and also more space used in the main partition for updates to apps that you don t use. Not So Exciting In recent times there hasn t been much potential for new features in phones. All phones have enough RAM and screen space for all common apps. While the S5 Mini has a small screen it s not that small, I spent many years with desktop PCs that had a similar resolution. So while the S5 Mini was released a couple of years ago that doesn t matter much for most common use. I wouldn t want it for my main phone but for a secondary phone it s quite good. The Nexus 6P is a very nice phone, but apart from USB-C, the fingerprint reader, and the lack of a stylus there s not much noticeable difference between that and the Samsung Galaxy Note 3 I was using before. I m generally happy with my Nexus 6P, but I think that anyone who chooses to buy a cheaper phone probably isn t going to be missing a lot.

• [1] https://etbe.coker.com.au/2016/06/01/i-just-ordered-a-nexus-6p/
• [2] https://play.google.com/store/apps/details?id=com.urbandroid.lux

The best known smart bulb setups (such as the Philips Hue and the Belkin Wemo) are based on Zigbee, a low-energy, low-bandwidth protocol that operates on various unlicensed radio bands. The problem with Zigbee is that basically no home routers or mobile devices have a Zigbee radio, so to communicate with them you need an additional device (usually called a hub or bridge) that can speak Zigbee and also hook up to your existing home network. Requests are sent to the hub (either directly if you're on the same network, or via some external control server if you're on a different network) and it sends appropriate Zigbee commands to the bulbs.

But requiring an additional device adds some expense. People have attempted to solve this in a couple of ways. The first is building direct network connectivity into the bulbs, in the form of adding an 802.11 controller. Go through some sort of setup process[1], the bulb joins your network and you can communicate with it happily. Unfortunately adding wifi costs more than adding Zigbee, both in terms of money and power - wifi bulbs consume noticeably more power when "off" than Zigbee ones.

There's a middle ground. There's a large number of bulbs available from Amazon advertising themselves as Bluetooth, which is true but slightly misleading. They're actually implementing Bluetooth Low Energy, which is part of the Bluetooth 4.0 spec. Implementing this requires both OS and hardware support, so older systems are unable to communicate. Android 4.3 devices tend to have all the necessary features, and modern desktop Linux is also fine as long as you have a Bluetooth 4.0 controller.

Bluetooth is intended as a low power communications protocol. Bluetooth Low Energy (or BLE) is even lower than that, running in a similar power range to Zigbee. Most semi-modern phones can speak it, so it seems like a pretty good choice. Obviously you lose the ability to access the device remotely, but given the track record on this sort of thing that's arguably a benefit. There's a couple of other downsides - the range is worse than Zigbee (but probably still acceptable for any reasonably sized house or apartment), and only one device can be connected to a given BLE server at any one time. That means that if you have the control app open while you're near a bulb, nobody else can control that bulb until you disconnect.

The quality of the bulbs varies a great deal. Some of them are pure RGB bulbs and incapable of producing a convincing white at a reasonable intensity[2]. Some have additional white LEDs but don't support running them at the same time as the colour LEDs, so you have the choice between colour or a fixed (and usually more intense) white. Some allow running the white LEDs at the same time as the RGB ones, which means you can vary the colour temperature of the "white" output.

But while the quality of the bulbs varies, the quality of the apps doesn't really. They're typically all dreadful, competing on features like changing bulb colour in time to music rather than on providing a pleasant user experience. And the whole "Only one person can control the lights at a time" thing doesn't really work so well if you actually live with anyone else. I was dissatisfied.

I'd met Mike Ryan at Kiwicon a couple of years back after watching him demonstrate hacking a BLE skateboard. He offered a couple of good hints for reverse engineering these devices, the first being that Android already does almost everything you need. Hidden in the developer settings is an option marked "Enable Bluetooth HCI snoop log". Turn that on and all Bluetooth traffic (including BLE) is dumped into /sdcard/btsnoop_hci.log. Turn that on, start the app, make some changes, retrieve the file and check it out using Wireshark. Easy.

Conveniently, BLE is very straightforward when it comes to network protocol. The only thing you have is GATT, the Generic Attribute Protocol. Using this you can read and write multiple characteristics. Each packet is limited to a maximum of 20 bytes. Most implementations use a single characteristic for light control, so it's then just a matter of staring at the dumped packets until something jumps out at you. A pretty typical implementation is something like:

0x56,r,g,b,0x00,0xf0,0x00,0xaa

where r, g and b are each just a single byte representing the corresponding red, green or blue intensity. 0x56 presumably indicates a "Set the light to these values" command, 0xaa indicates end of command and 0xf0 indicates that it's a request to set the colour LEDs. Sending 0x0f instead results in the previous byte (0x00 in this example) being interpreted as the intensity of the white LEDs. Unfortunately the bulb I tested that speaks this protocol didn't allow you to drive the white LEDs at the same time as anything else - setting the selection byte to 0xff didn't result in both sets of intensities being interpreted at once. Boo.

You can test this out fairly easily using the gatttool app. Run hcitool lescan to look for the device (remember that it won't show up if anything else is connected to it at the time), then do gatttool -b deviceid -I to get an interactive shell. Type connect to initiate a connection, and once connected send commands by doing char-write-cmd handle value using the handle obtained from your hci dump.

I did this successfully for various bulbs, but annoyingly hit a problem with one from Tikteck. The leading byte of each packet was clearly a counter, but the rest of the packet appeared to be garbage. For reasons best known to themselves, they've implemented application-level encryption on top of BLE. This was a shame, because they were easily the best of the bulbs I'd used - the white LEDs work in conjunction with the colour ones once you're sufficiently close to white, giving you good intensity and letting you modify the colour temperature. That gave me incentive, but figuring out the protocol took quite some time. Earlier this week, I finally cracked it. I've put a Python implementation on Github. The idea is to tie it into Ulfire running on a central machine with a Bluetooth controller, making it possible for me to control the lights from multiple different apps simultaneously and also integrating with my Echo.

I'd write something about the encryption, but I honestly don't know. Large parts of this make no sense to me whatsoever. I haven't even had any gin in the past two weeks. If anybody can explain how anything that's being done there makes any sense at all[3] that would be appreciated.

[1] typically via the bulb pretending to be an access point, but also these days through a terrifying hack involving spewing UDP multicast packets of varying lengths in order to broadcast the password to associated but unauthenticated devices and good god the future is terrifying

[2] For a given power input, blue LEDs produce more light than other colours. To get white with RGB LEDs you either need to have more red and green LEDs than blue ones (which costs more), or you need to reduce the intensity of the blue ones (which means your headline intensity is lower). Neither is appealing, so most of these bulbs will just give you a blue "white" if you ask for full red, green and blue

[3] Especially the bit where we calculate something from the username and password and then encrypt that using some random numbers as the key, then send 50% of the random numbers and 50% of the encrypted output to the device, because I can't even

comments

A year ago I ve blogged about people requesting CVE without letting upstream know. On the other hand, per requests from Debian, I m working on improving PHP upstream CVE request process. For the last few release this means I ask the security list members which issues they think should have CVE and ask for them in parallel to the release being made (usually in the space between the release being tagged publicly and is actually announced). In the last week, I ve encountered a case where a few CVE were assigned to old PHP issues without any public notice. The fixes for these issues have been published a year ago (August 2015). And I find out about these assignment through warning published by the distributions (mostly Debian, which I m close to). Sometimes things fall between the chairs, and it s perfectly OK to ask for CVE to make sure security issues do get attention even if time has passed. But after the issues (and fixes) are public, I don t see a reason to do so without making the request itself public as well. And even if the request wasn t public, at least notify upstream so this info can be added to the right places. Most of these bug were found out when I started to add sequential number into the CVE search after getting an a notice from Debian for two of the issues.

• CVE-2015-8873 for PHP #69793
• CVE-2015-8874 for PHP #66387
• CVE-2015-8876 for PHP #70121
• CVE-2015-8877 for PHP #70064
• CVE-2015-8878 for PHP #70002
• CVE-2015-8879 for PHP #69975
• CVE-2015-8880 for PHP aa8cac57 (Dec 2015)

And while working on processing these issues for PHP, I also notice they weren t updated for libGD where appropriate (including recent issues). Beyond keeping the eco-system up to date, another aspect of publicity is getting opinions from other parties. For example, in the case of CVE-2015-8879, RedHat doesn t agree with the classification of the bug as security. To give an example of a way things should happen is the request of CVE for PHP 5.5.34 in April, in which the Gentoo security team asked for assignment without upstream knowledge, Debian representative CCs upstream (Thanks!) and also asks CVE for issues covered by Ubuntu, to which the Ubuntu guy then adds some details. I hope this blog post will reach the anonymous people behind these CVE requests, and also the people assigning them. Without transparency and keeping things in synchronization, the idea of having a centralized location for security warning is not going to accomplish its goals.
Filed under: Debian GNU/Linux, PHP

Over the past year I ve been refining my understanding of the core claims of Kantian ethics. I ve realised that I have deeply Kantian intuitions about a lot of issues, and I understand these intuitions better now that I can put them in Kantian terms. Consider two exercises of state power: riot police suppressing protesters by non-lethal means, and soldiers shooting protesters to death. I feel more uncomfortable thinking about the first: there s something altogether more sinister about it than the second, even though the second is much more sad. I think that the reason is that non-lethal weaponry is designed to take away people s agency, and it often achieves this aim by means of emotional manipulation. Riot police use so-called baton charges to incite fearful retreat. Protesters have reasoned that in their political situation, they have a duty to resist the incumbent government. Riot police seek to unseat this conviction and cause fear to determine what the protesters will do. In Kantian terms, the riot police fail to respect the moral agency of the protesters by seeking to unseat the moral personality s determination of what the protester will do. A controversial example that Kant uses to make this point is the axe murderer case. Kant asks us to imagine that someone bangs on our front door and begs us to hide him in our house, because someone who wishes to kill him is coming up behind them. We do so. When the axe murderer arrives, he goes door-to-door and asks us whether the intended victim is in each house. Kant says that it is morally wrong to lie to the axe murderer and say that the victim is not in your house. How could this be? Surely there is a moral duty to protect the victim from being killed? Indeed there is, but when it comes into conflict with the duty not to lie, that second duty wins out. That s because respecting the moral agency of individuals is paramount. In this case, we would fail to respect the murderer s agency if we didn t allow him to take the decision to murder or not murder the victim; by lying to him we (disrespectfully) bypass his choice as to whether to do it. It s obviously crazy to say that we are morally required to give up the victim. Kant gives the wrong answer in this case. However, the case definitely reveals a requirement to respect other people s view of what they should do, and give them a chance to do it. Similarly it seems like we shouldn t give semi-automatics to our riot police, but there s something wrong with a lot of what they do. During the recent campaigns about Britain s EU referendum, some criticised Jeremy Corbyn s campaigning on the grounds that he failed to make an emotional appeal, instead asking people to make their own rational decision when they come to cast their vote. So he lost out the emotional appeals other people were making. It seems that he was successfully respecting individual agency. You ve got to give people a chance to live up to their own idea of what they should do. I m not sure how to reconcile these various ideas, and I m not sure what it says about me that I find non-lethal weaponry as uncomfortable as I do.