Previously, I wrote about my project to create an audio depiction of network traffic. In this second post, I explore how I model aspects of the network that will be captured in the audio representation. Before getting started, I'll pass along a link. This is not the first time someone has tried to put sound to packets flying through the ether: I was pointed at Peep. I haven't looked at Peep, but will do so after I finish my own write up. Not being an academic, I feel no obligation to compare and contrast my work to others:-)
I started with an idea of what I'd like to hear. One of my motivations was to explore some automated updates we run at work. So, I was hoping to capture the initial DNS and ARP traffic as the update discovered the systems it would contact. Then I was hoping to capture the ssh and other traffic of the actual update.
To Packet or Stream
One of the simplest things to do would simply be to model network packets. For DNS I chose that approach. I was dubious that a packet-based model would capture the aspects of TCP streams I typically care about. I care about the source and destination (both address and port) of course. However I also care about how much traffic is being carried over the stream and the condition of the stream. Are there retransmits? Are there a bunch of unanswered SYNs? But I don't care about the actual distribution of packets. Also, a busy TCP stream can generate thousands of packets a second. I doubted my ability to distinguish thousands of sounds a second at all, especially while trying to convey enough information to carry stream characteristics like overall traffic volume.
So, for TCP, I decided to model some characteristics of streams rather than individual packets.
For DNS, I decided to represent individual requests/replies.
I came up with something clever for ARPP. There, I model the request/reply as an outstanding request. A lot of unanswered ARPs can be a sign of a scan or a significant problem. The mornful sound of a TCP stream trailing off into an unanswered ARP as the cache times out on a broken network is certainly something I'd like to capture. So, I track when an ARP request is sent and when/if it is answered.
Sound or Music
I saw two approaches. First, I could use some sound to represent streams. As an example, a running diesel engine could make a great representation of a stream. The engine speed could represent overall traffic flow. There are many opportunities for detuning the engine to represent various problems that can happen with a stream. Perhaps using stereo separation and slightly different fundamental frequencies I could even represent a couple of streams and still be able to track them.
However, at least with me as a listener, that's not going to scale to a busy network. The other option I saw was to try and create melodic music with various musical phrases modified as conditions within the stream or network changed. That seemed a lot harder to do, but humans are good at listening to complicated music.
I ended up deciding that at least for the TCP streams, I was going to try and produce something more musical than sound. I was nervous: I kept having visions of a performance of "Peter and the Wolf" with different instruments representing all the characters that somehow went dreadfully wrong.
As an aside, the decision to approach music rather than sound depended heavily on what I was trying to capture. If I'm modeling more holistic properties of a system--for example, total network traffic without splitting into streams--I think parameterized sounds would be a better approach.
The decision to approach things musically affected the rest of the modeling. Somehow I was going to need to figure out notes to play. I'd already rejected the idea of modeling packets, so I wouldn't simply be able to play notes when a packet arrived.
Energy Decay
As I played with various options, I realized that the critical challenge would be figuring out how to focus the listener's attention on the important aspects of what was going on. Clutter was the great enemy. My job would be figuring out how to spend sound wisely. When something interesting happened, that part of the model should get more focus--more of the listener's energy.
Soon I found myself thinking a lot about managing the energy of network streams. I imagined streams getting energy when something happened, and spending that energy to convey that interesting event to the listener. Energy needed to accumulate fast enough that even low-traffic streams could be noticed. Energy needed to be spent fast enough that old events were not taking listener focus from new, interesting things going on. However, if the energy were spent slow enough, then network events could be smoothed out to give a better picture of the stream rather than individual packets.
This concept of managing some decaying quantity and managing the rate of decay proved useful at multiple levels of the model.
Two Layer Model
I started with a python script that parses tcpdump output. It associates a packet with a stream and batches packets together to avoid overloading other parts of the system.
The output of this script are stream events. Events include a source and destination address, a stream ID, traffic in each direction, and any special events on the stream.
For DNS, the script just outputs packet events. For ARP, the script outputs request start, reply, and timeout events. There's some initial support for UDP, but so far that doesn't make sound.
Right now, FINs are modeled, but SYNs and the interesting TCP conditions aren't directly modeled. If you get retransmissions you'll notice because packet flow will decrease. However, I'd love to explicitly sound retransmissions. I also think a window filling as an application fails to read is important. I imagine either narrowing a band-pass filter to clamp the audio bandwidth available to a stream with a full window. Or perhaps taking it the other direction and adding an echo.
The next layer down tracks the energy of each stream. But that, and how I map energy into music, is the topic of the next post.

At the end of December I decided I was going to do hardware "things", and so far that has worked out pretty well. One of the reasons I decided to play with Arduinos is that I assumed I could avoid all forms of soldering. I've done soldering often enough to know I can manage it, but not quite often enough that I feel comfortable doing so. Unfortunately soldering has become a part of my life once again, as too many of the things I've been playing with have required pins soldering to them before I can connect them. Soldering aside I've been having fun, and I have deployed several "real" projects in and around my flat. Perhaps the most interesting project shows the arrival time of the next tram to arrive at the end of my street:

• Helsinki Tram Time Display

That's simple, reliable, and useful. I have another project which needs to be documented which combineds a WeMos D1 and a vibration sensor - no sniggers - to generate an alert when the washing machine is done. Having a newborn baby around the place means that we have a lot of laundry to manage, and we keep forgetting that we've turned the washing machine on. Oops. Anyway. Hardware. More fun than I expected. I've even started ordering more components for bigger projects. I'll continue to document the various projects online, mostly to make sure I remember the basics:

• https://steve.fi/Hardware/

A new release, now at version 0.6.12, of the digest package is now on CRAN and in Debian. The digest creates hash digests of arbitrary R objects (using the 'md5', 'sha-1', 'sha-256', 'crc32', 'xxhash' and 'murmurhash' algorithms) permitting easy comparison of R language objects. This release extends sha1 digest methods to even more types, thanks to another contribution by Thierry Onkelinx. CRANberries provides the usual summary of changes to the previous version. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Human languages, part 1 I do enjoy writing blog posts, but sometimes time is lacking, other times inspiration. As I was eating dinner today, I was lost in thoughts and my eyes stopped on the documentation from a certain medication for coughing (the thing almost noone reads, like EULAs). I was surprised quite surprised with te text that I said well, some blog posts on languages might be interesting . Background: My mother tongue is Romanian. While growing up and learning foreign languages, I considered only the utilitarian aspect of languages, but as I get older (not old, older! ), I find human languages more and more interesting. Back to the subject: this being Switzerland, most everything is written in German, French and Italian (in this order of frequency). German is still a foreign language to me (let's say I get by, but not nicely), French is the fancy high class cousin, and Italian Well Italian is a special case. When I first saw Italian on TV (a newscast while travelling in Italy) I was shocked at how much one can understand without learning Italian in any way (much, much more than German after years of trying). So Italian language is quite close, and usually one understands a third to half; this is both spoken (less in common language, more in official language) and in writing. In this particular case, the instructions of use say (sorry for typos, manually copying):

In casi rarissimi possono manifestarsi reazioni di ipersensibilit grave con tumefazione del viso, difficolt respiratoria (dispnea) e diminuzione della pressione arteriosa.

What was surprising here was not the list of side effects (hah), but that this short phrase is 98% identical to the translation in Romanian; not only words, but also phrase structure. I don't think I've ever seen this before:

n cazuri rare se pot manifesta reac ii de hipersensibilitate grav cu tumefac ie a fe ei, dificultate respiratorie (dispnee) i diminuare a presiunii arteriale.

Not all the words are identical, but even the one that is obviously different (it. 'del viso', ro. 'a fe ei') is easily translatable as 'visiune' in Romanian means 'to see', so the link is clear. This phrase structure is also quite a natural way to say the things in Romanian. I was then curious to see the French version, which is:

Dans de tr s rares cas, [the medicine] peut d clencher de violentes r actions d'hypersensibilit s'accompagnant d'un gonflement du visage, de d tresse respiratorie e d'une chute de tension.

French is usually quite different from Romanian that one has to learn it (for quite a while, especially for grammar) in order to be proficient in it, but here you can make also word-by-word translation (transposition?) that doesn't lose the meaning:

n cazuri tare rari, [the medicime] poate declan a violente reac ii de hypersensibilitate acompaniate de o umflare a fe ei, de ???? respiratorie i de ???? a tensiunii.

Basically here we have two non-equivalent words, and a bit more wierd phrase structure it sounds more like coloquial speech than written language but for French is also surprisingly close. You'd invert some of the adjective-noun pairs (fr. 'violentes r actions' is understandable in Romanian as 'violente reac ii', but it sounds very poetical and you'd usually write it as 'reac ii violente'). The next phrase is no longer that similar, but the one after is again obviously identical:

Se osserva effetti collaterali qui non descritti, dovrebbe informare il suo medico, il suo farmacista o il suo drogerie.

which is:

[Dac /If] se observ effecte colaterale care nu sunt descrise, trebuie informat medicul vostru, farmacistul vostru sau [xxxx - no real equivalent].

And the French is also identical, modulo again 'droguiste':

Si vouz remarquez de effets secondaires qui ne sont pas mentionn s dans cette notice, veuillez en informer votre m decin, votre pharmacien ou votre droguiste.

which is:

Dac remarca i efecte secundare care nu sunt men ionate n aceast not , informa i medicul vostru, farmacistul vostru sau al vostru [xxxx].

This is even closer; 'votre' is more similar to '[al] vostru' than 'suo', and the phrase structure is much more natural - this is exactly how you'd write it in 'native' Romanian, whereas the Italian is not (I had to add the 'if' to make it parseable). The 'vous' in 'vouz remarquez' is 'voi' in Romanian, but doesn't need to be added as it would be redundant; but it doesn't confuse the phrase. The 'veuillez en informer' doesn't have a 1:1 translation (it would be written as 'v rug m s informa i'), but is still understandable; a false friend translation would be 'vede i s informa i'/see to inform/voir informer. Why is this all surprising? Because Romanian has a significant amount of words of Slavic origin (~11% in overall vocabulary, 15% in most commonly used 2500 words) and some from other nearby countries (Turkey, Greek, some Hungarian and German). At a stretch, it's even possible to write simple but complete sentences entirely with words from Slavic origin, as I learned from this interesting youtube video. Also, our accent is almost always confused with Russian, not with Italian or French. So to get to a summary: normally you see sentence elements that are similar or identical, but not entire sentences, and definitely not phrases. What made the three languages here keep, in this particular case, not only similar but almost identical words and also almost identical phrase structure? Is it the subject (medicine)? Maybe. Is it a random fluke? If so, I don't remember seeing it before. Do I just see similarities where there are none? Possibly , but at least I thought it worth mentioning; it was quite surprising to me. Did my brain get confused by too many languages and I misinterpreted words that don't really exist (e.g. I was sure that 'vizajul' is a Romanian word, but upon checking, it isn't )? Also possible. In any case, for me it was a good subject for a blog post. Now let's not go near Spanish and definitely not near Portuguese

As previously mentioned, at the end of last year I got involved with a project involving the use of 1-Wire. In particular a DS28E15 device, intended to be used as a royalty tracker for a licensed piece of hardware IP. I d no previous experience with 1-Wire (other than knowing it s commonly used for driving temperature sensors), so I took it as an opportunity to learn a bit more about it. The primary goal was to program a suitable shared key into the DS28E15 device that would also be present in the corresponding hardware device. A Maxim programmer had been ordered, but wasn t available in stock so had to be back ordered. Of course I turned to my trusty Bus Pirate, which claimed 1-Wire support. However it failed to recognise the presence of the device at all. After much head scratching I finally listened to a co-worker who had suggested it was a clock speed issue - the absence of any option to select the 1-Wire speed in the Bus Pirate or any mention of different speeds in the documentation I had read had made me doubt it was an issue. Turns out that the Bus Pirate was talking standard 1-Wire and the DS28E15 only talks overdrive 1-Wire, to the extent that it won t even announce its presence if the reset pulse conforms to the standard, rather than overdrive, reset time period. Lesson learned: listen to your co-workers sooner. A brief period of yak shaving led to adding support to the Bus Pirate for the overdrive mode (since landed in upstream), and resulted in a search request via the BP interface correctly finding the device and displaying its ROM ID. This allowed exploration of the various commands the authenticator supports, to verify that the programming sequence operated as expected. These allow for setting the shared secret, performing a SHA256 MAC against this secret and a suitable nonce, and retrieving the result. Next problem: the retrieved SHA256 MAC did not match the locally computed value. Initially endianness issues were suspected, but trying the relevant permutations did not help. Some searching found an implementation of SHA256 for the DS28E15 that showed differences between a standard SHA256 computation and what the authenticator performs. In particular SHA256 normally adds the current working state (a-g) to the current hash value (h0-h7) at the end of every block. The authenticator does this for all but the final block, where instead the hash value is set to the working state. I haven t been able to find any documentation from Maxim that this is how things are calculated, nor have I seen any generic implementation of SHA256 which supports this mode. However rolling my own C implementation based on the code I found and using it to compare the results retrieved from the device confirms that this is what s happening. So at this point we re done, right? Wait for the proper programming hardware to turn up, write the key to the devices, profit? Well, no. There was a bit of a saga involving the programmer (actually programmers, one with at least some documentation that allowed the creation of a Python tool to allow setting the key and reading + recording the ROM ID for tracking, and one with no programming documentation that came with a fancy GUI for manually doing the programming), but more importantly it was necessary to confirm that the programmed device interacted with the hardware correctly. Initial testing with the hardware was unsuccessful. Again endianness issues were considered and permutations tried, but without success. A simple key constructed to avoid such issues was tried, and things worked fine. There was a hardware simulation of both components available, so it was decided to run that and obtain a capture of the traffic between them. As the secret key was known this would then allow the random nonce to be captured, and the corresponding (correct) hash value. Tests could then be performed in software to determine what the issue was & how to generate the same hash for verification. Two sets of analyzer software were tried, OpenBench LogicSniffer (OLS) and sigrok. As it happened both failed to correctly decode the bitstream detected as 1-Wire, but were able to show the captured data graphically, allowing for decoding by eye. A slight patch to OLS to relax the timing constraints allowed it to successfully decode the full capture and provided the appropriate data for software reproduction. The end issue? A 256 bit number (as defined in VHDL) is not the same as 32 element byte array Obvious when you know what the issue is! So? What did I learn, other than a lot about 1-Wire? Firstly, don t offhandedly discount suggestions that you don t think make sense. Secondly, having a tool (in this case the Bus Pirate) that lets you easily play with a protocol via a simple interface is invaluable in understanding it. Thirdly, don t trust manufacturers to be doing something in a normal fashion when they claim to be using a well defined technology. Fourthly, be conscious about all of the different ways bitstreams can be actually processed in memory. It s not just endianness. Finally, spending the time to actually understand what s going on up front can really help when things don t work as you d expect later on - without the yak shaving to support Overdrive on the BP I wouldn t have been able to so quickly use the simulation capture to help diagnose the issue.

The Budget The story which I wanted to share is there are few friends (from Debian as well as elsewhere) who shared that they didn t get the whole demonetisation play or what the Government is/was trying to do. As budget is just round the corner (India will be presenting its yearly budget on 1st of February), thought it is prudent to share at least some basics, ideas and theories of what goals the Finance Minister would be looking at when presenting his budget. I would NOT talk of Inflation targeting or some such exotica as those topics would require their own blog-posts altogether. I would mainly be talking a bit about Taxation and in that Personal Income Tax. I would also not use words like Receivables and like which thought bit more accurate are not used in everyday language. Just like Private Companies and increasingly public utilities, The Government of the day has two-three different aims when it is presenting a budget a. The first is to give an update about how things went last year. Did all the incomes that were projected, did it happen or was there a short-fall ? If there was a short-fall what were the reasons for the shortfall. Similarly, did all the budgeted expenditure earmarked for the year was spent and were it spent under the heads they were supposed to ? If not what went wrong there ? There is usually a tussle between Planned and Unplanned expenditure and one of the hallmarks of good governance is that unplanned expenditure is kept at minimum, while planned expenditure and projects completion or/and assets coming on-line were within the estimated time-frames. So these updates are given to the Parliament and hence public at large. The second and the more interesting part are the plans for the immediate future, 1 year down-the-line. Based on the performance last year, a bit of crystal-ball gazing of external and internal conditions of the country, the Finance Minister along with her/his colleagues of Finance Ministry. Trivia There hasn t been a female finance minister till date in India. The Finance Ministry as a whole also holds consultations with most sections of the society before sharing/putting his Fiscal Policy (Planned Expenditure) for debate and passage in form of the Budget. While the budget itself is a technical exercise, it is also a Political exercise as both the budget and the finance bill (which contains the taxation proposals) need to be passed in Lok Sabha (Lower house). After passing scrutiny of Lok Sabha (Lower House having people s representatives directly elected) and Rajya Sabha (Upper House, indirectly elected), the taxation proposals becomes the law. It isn t that simple but for our understanding, keeping it simple. This Political model of governance with two houses is modeled under the British (Westminister) model since 1947. The Government, just like any other Organization gives a similar Profit and Loss Account and a Balance Sheet. I am a newbie to graphviz. The graph was made like this
graph Budget
subgraph tier1
node [color="limegreen",style="filled",group="tier1"]
Country_Budget

......
Country_Budget -- Profit_and_Loss_Account [type=s];
It might be possible to make the graph much better than it is currently . The Profit and Loss Account of the Government tells what Incomes it is projected to earn in the upcoming year and whatever Expenditures it hopes to do this year. The Income and Expenditure independently can be bifurcated into two, Revenue Income and Capital Income and Revenue Expenditure and Capital Expenditure. The simplest example of such planned expenditure which comes to my mind is the Indian Railways Budget which is all planned expenditure. As can be seen even with ample funds Railways were able to spent only 50% of the total amount disbursed last year. Similarly income generation for Railways was far below the target. Examples of Revenue Income include taxes of all sorts, while Capital Income are rare, like divestment/stake sale of a company owned by the Government. These are usually one-off events. Examples of Capital Expenditure is when the Government makes a road, makes a bridge etc. Usually large expenditures come under Capital Expenditure while salaries to Government employees and routine expenditures are known as Revenue Expenditure. There was a statement by the present Government that the last 6-7 years the budgets has been more or less static as far as numbers are concerned. This hampers Government s ability to take up any new work. The Revenue income earned by the Government can again be bifurcated primarily into two Direct Taxes and Indirect Taxes. Direct Taxes are those which the Government earns through Personal Income Tax and Corporate Tax. As only 1 percent of Indians pay Personal income tax, the rest Government tries to raise by

• Corporate Tax (which is told to be higher than neighbouring countries around us)
• Indirect Taxes like VAT (Value Added Tax introduced in 2005 over select items at 1% and now it s almost 15%,)
• Duties on all imported goods depending on what the Government thinks of a certain class of goods.For instance, should life-saving drugs be taxed or not and going either way has immediate as well as long-term consequences.

Hence the Government of the day is in fix. It needs to have more money if it wants to invest into infrastructure, defence spending, social spending such as health and education and so on and so forth. It cannot

• raise VAT rates more as it may stall public spending
• raise Corporate Taxes beyond a point otherwise people may either start doing work in black or shadow economy or may leave the country, a term used to describe this is also Capital Flight.
• raise personal Income Taxes beyond a point otherwise the same thing will happen as above.

Another point is that unlike China which is a Large State-backed Enterprises Export-led Economy which has its own problems, India s economy is much more consumption-based, hence any large tinkering upwards may possibly stall whatever little spending the middle-class does, similar to the stall in consumer durables which has been happening over the last few years. There are a couple of short-term solutions that the Government may do

• It can raise finances from the market by using a mix of debt and equity instruments such as Bonds and targeted finances for specific schemes under EEE (Exempt, Exempt, Exempt or EET i.e. Exempt, Exempt Tax)
• The Government prints more money

While both seem to be attractive ways, but both have their disadvantages also, both have costs associated for them. In the first one, like any other scheme, when any scheme is launched, it needs to be underwritten by GOI which means even if it s not a success they would have to service all and any obligations towards investors. Also they have to be careful how much they are borrowing as excessive borrowing for today could lead to a Greece-like meltdown situation, whether internal or external borrowers. With external borrowers they also usually like to have a guarantee that the Rupee will not slide beyond a point otherwise the Government will have to pay all and any losses but this is going beyond what I wanted to share. Printing excess money in the system could lead to loss in the value of the money itself as well as leading to inflationary pressures which leads to more problems for the poor and greater inequality between the classes among other things. So while the Government may use all of the above ways in varying degrees, the present Government had the idea that if we were to reduce black money or hidden economy (AFAIK no country can claim to completely eliminate it) we would be able to raise the finance we need without a major cost associated to it. For instance, I was reading that even in Canada, it is expected that 20% of black money/shadow economy works and that assessment is by their own taxation authorities. So While doing demonetization, it came out with an equivalent Black Money Declaration Scheme (IDS). The idea is simple, even if 1 percent of the population comes in the traditional tax net the Government of the day would be able to enhance budgets to various expenditure. Now while the idea is good in theory, implementation has been the Achilees heel. While the Government s expected something like 15% of the whole economy was black money or shadow money, almost 95% of the money in circulation came back in Banks during demonetization ( These are unofficial figures, Finance Ministry/RBI would be disclosing the real figures on 1st of February 2017 so we will know). It is suspected that 10% of money in Banks is black money. There are considerable costs to search analyze, prove in the court of law that it is so. There are and would be considerable costs to train new officers as existing Income Tax Officers are already burdened with Advance Tax being paid by Corporates and small business-man paying round the year (every 3 months), The existing Income Tax Officers already have their hands full. Also till Governments don t fix up realty sector/real-estate sector and other places where the black money/shadow economy may prevail. Hence all the training, salary, buildings where new Income Tax Officers could work, infrastructure, new buildings where suspect cases have to be tried and lawyers for those. As have shared a few times on this blog, India has almost 29 million court cases pending in the lower judiciary alone. Unless any such cases are not successfully tried within time by the Government, it would be a waste. Now whether the Government knew of these issues or not would probably be never known. Lastly, there is a voluntary part that the Government hopes, that they will by themselves join the mainstream tax-paying public. This might happen but any such happening will happen over years. People make their own choices. And unless there are not any stick and carrot approach to the Government s Policies people will tend to go back to their old ways. I would share an example from the demonetisation process which would help prove my point During demonetization, there was a great push towards doing digital transaction either via smartphones or greater usage of debit and credit cards etc. For the first 60 days till 31st December 2016, you could do digital transactions without paying any transaction fee. During that period, I used my Debit card to shop, to eat at restaurants or/and even small shops. But come 1st January 2017, the charges for digital transactions are anything between 1.5% to 3% of transactions. Naturally, I stopped using them and use them very sparingly where cash won t work. So at the end, while the Government made the whole demonetization drive to drive out shadow economy, terror financing etc. While terror financing has been hurt quite a bit, the same cannot be said of the shadow/black economy. It seems that the Government would need to close many more doors and windows before people join the mainstream. While Politically it was risky, socially it was also a bit risky move as it was uncertain how and where things will move. Venezuela tried the same thing and fell flat on its face. All said and done, if and when people become part of the tax-paying class/people, The most optimistic idea that the Government has that everybody will go cashless and it would be far easier to find out who s not paying taxes. As shared before, I don t think this will happen unless the charges for cashless is at 0.05% or something similar. Even IF people do join the mainstream, it is very much possible that the present Govt. will not enjoy fruits of this labour as fruits might come in 2018/19 or even later even if they do come. So whether the decision had the right affect or not, we may never come to know. Governments tend to tinker around with the figures as well. But I hope some idea of how things happen is known now.
Filed under: Miscellenous Tagged: #demonetization, #Government Budget, #graphviz, #Limitations, #Profit and Loss Account, #Taxation

I wrote a piece a few days ago about how the Meitu app asked for a bunch of permissions in ways that might concern people, but which were not actually any worse than many other apps. The fact that Android makes it so easy for apps to obtain data that's personally identifiable is of concern, but in the absence of another stable device identifier this is the sort of thing that capitalism is inherently going to end up making use of. Fundamentally, this is Google's problem to fix.

Around the same time, Kaspersky, the Russian anti-virus company, wrote a blog post that warned people about this specific app. It was framed somewhat misleadingly - "reading, deleting and modifying the data in your phone's memory" would probably be interpreted by most people as something other than "the ability to modify data on your phone's external storage", although it ends with some reasonable advice that users should ask why an app requires some permissions.

So, to that end, here are the permissions that Kaspersky request on Android:

• android.permission.READ_CONTACTS
  
• android.permission.WRITE_CONTACTS
  
• android.permission.READ_SMS
  
• android.permission.WRITE_SMS
  
• android.permission.READ_PHONE_STATE
  
• android.permission.CALL_PHONE
  
• android.permission.SEND_SMS
  
• android.permission.RECEIVE_SMS
  
• android.permission.RECEIVE_BOOT_COMPLETED
  
• android.permission.WAKE_LOCK
  
• android.permission.WRITE_EXTERNAL_STORAGE
  
• android.permission.SUBSCRIBED_FEEDS_READ
  
• android.permission.READ_SYNC_SETTINGS
  
• android.permission.WRITE_SYNC_SETTINGS
  
• android.permission.WRITE_SETTINGS
  
• android.permission.INTERNET
  
• android.permission.ACCESS_COARSE_LOCATION
  
• android.permission.ACCESS_FINE_LOCATION
  
• android.permission.READ_CALL_LOG
  
• android.permission.WRITE_CALL_LOG
  
• android.permission.RECORD_AUDIO
  
• android.permission.SET_PREFERRED_APPLICATIONS
  
• android.permission.WRITE_APN_SETTINGS
  
• android.permission.READ_CALENDAR
  
• android.permission.WRITE_CALENDAR
  
• android.permission.KILL_BACKGROUND_PROCESSES
  
• android.permission.RESTART_PACKAGES
  
• android.permission.MANAGE_ACCOUNTS
  
• android.permission.GET_ACCOUNTS
  
• android.permission.MODIFY_PHONE_STATE
  
• android.permission.CHANGE_NETWORK_STATE
  
• android.permission.ACCESS_NETWORK_STATE
  
• android.permission.ACCESS_LOCATION_EXTRA_COMMANDS
  
• android.permission.ACCESS_WIFI_STATE
  
• android.permission.CHANGE_WIFI_STATE
  
• android.permission.VIBRATE
  
• android.permission.READ_LOGS
  
• android.permission.GET_TASKS
  
• android.permission.EXPAND_STATUS_BAR
  
• com.android.browser.permission.READ_HISTORY_BOOKMARKS
  
• com.android.browser.permission.WRITE_HISTORY_BOOKMARKS
  
• android.permission.CAMERA
  
• com.android.vending.BILLING
  
• android.permission.SYSTEM_ALERT_WINDOW
  
• android.permission.BATTERY_STATS
  
• android.permission.MODIFY_AUDIO_SETTINGS
  
• com.kms.free.permission.C2D_MESSAGE
  
• com.google.android.c2dm.permission.RECEIVE

Every single permission that Kaspersky mention Meitu having? They require it as well. And a lot more. Why does Kaspersky want the ability to record audio? Why does it want to be able to send SMSes? Why does it want to read my contacts? Why does it need my fine-grained location? Why is it able to modify my settings?

There's no reason to assume that they're being malicious here. The reasons that these permissions exist at all is that there are legitimate reasons to use them, and Kaspersky may well have good reason to request them. But they don't explain that, and they do literally everything that their blog post criticises (including explicitly requesting the phone's IMEI). Why should we trust a Russian company more than a Chinese one?

The moral here isn't that Kaspersky are evil or that Meitu are virtuous. It's that talking about application permissions is difficult and we don't have the language to explain to users what our apps are doing and why they're doing it, and Google are still falling far short of where they should be in terms of making this transparent to users. But the other moral is that you shouldn't complain about the permissions an app requires when you're asking for even more of them because it just makes you look stupid and bad at your job.

comments

Beware, this would be slightly longish. Debian Contributions In the last couple of weeks, was lucky to put up a patch against debian-policy which had been bothering me for a long-long time. The problem statement is simple, man-pages historically were made by software engineers for software-engineers. The idea, probably then was you give the user some idea of what the software does and the rest the software engineer would garner from reading the source-code. But over period of time, the audience has changed. While there are still software engineers who use GNU/Linux for the technical excellence, the man-pages have not kept up with this new audience who perhaps are either not technically so sound that or they do not want to take the trouble to reading the source-code to understand how things flow. An example or examples in a man-page gives us (the lesser mortals) some insight as how the command works, how the logic flows. A good example of a man-page is the ufw man-page

EXAMPLES
Deny all access to port 53: ufw deny 53 Allow all access to tcp port 80: ufw allow 80/tcp Allow all access from RFC1918 networks to this host: ufw allow from 10.0.0.0/8
ufw allow from 172.16.0.0/12
ufw allow from 192.168.0.0/16 Deny access to udp port 514 from host 1.2.3.4: ufw deny proto udp from 1.2.3.4 to any port 514 Allow access to udp 1.2.3.4 port 5469 from 1.2.3.5 port 5469: ufw allow proto udp from 1.2.3.5 port 5469 to 1.2.3.4 port 5469

Now if we had man-pages like the above which give examples, then the user at least can try to accomplish whatever s/he is trying to do. I truly believe not having examples in a man-page kills 50% of your audience and people who could potentially use your tool. Personal wishlist The only thing (and this might be my failure) is we need a good way to search through a man-page. The only way I know is using / and try to give a pattern. Lots of times it fails because I, the user doesn t know the exact keyword which the documenter was using. What would be nice, great if we do have some sort of parser where I tell it, $this is what I m looking for and the parser tries the pattern + all its synonyms and whatever seems to be most relevant passage from the content, in this case a manpage it tells me. It would make my life a lot easier while at the same time force people to document more and more. I dunno if there has been any research or study of the relationship between good documentation and popularity of a program. I know there are lots of different tiny bits which make or break a program, one of which would definitely be documentation and in that a man-page IF it s a command-line tool. A query on Quora gives some indication https://www.quora.com/How-comprehensible-do-you-find-Unix-Linux-Man-pages although the low response rate tells its own story. there have been projects like man2html and man2pdf and others which try to make the content more accessible to people who are not used to the man-page interface but till you don t have Examples the other things can work only so far. Also if anybody talks about X project which claims to solve this problem they will have to fight manpages who have been around like forever. As can be seen in the patch, did some rookie mistakes as can be seen. I also filed a lintian bug at the same time. Hope the patch does get merged at some point in debian-policy and then a check introduced in lintian in some future release. I do agree with anarcat s assertion that it should be at the level of the manpage missing level. I am no coder but finding 14,000 binary packages without a manpage left me both shocked and surprised. I came to know about manpage-alert from the devscripts package to know which all binary packages that have been installed but not have man-pages. I hope to contribute a manpage or two if I across a package I m somewhat comfortable with. I have made a beginning of sorts by running manpage alert and putting the output in a .txt file which I would grep through manually and see if something interesting jumps at me. The learning garnered from putting the patch to the debian package resulted in another patch but this time for an upstream project altogether. As can be seen all are just baby-steps that even a non-coder can take. Another couple of bugs I filed which were fixed were of a sim called unknown-horizons . A 2D realtime strategy simulation. I had filed three bugs, two of which were fixed in 2 days, the 3rd I hope is also fixed soonish. Lastly, I spent most of the week-end poring over packages who have left files in /etc/bash_completion.d/ . I spent almost 4-5 odd hours as each package in question as well as entries found in /etc/bash-completion.d/$filename I had to find which package it belonged to first

[$] dpkg -S /etc/bash_completion.d/git-prompt git: /etc/bash_completion.d/git-prompt
I know that dpkg-query also does the same
[$] dpkg-query -S /etc/bash_completion.d/git-prompt git: /etc/bash_completion.d/git-prompt
But I am used to plain dpkg although do know that dpkg-query can do lot more intimate searching in various ways than dpkg can. Once the package name was established, first simulate the purge [$] sudo aptitude -s purge git

[sudo] password for shirish:
The following packages will be REMOVED:
git p
0 packages upgraded, 0 newly installed, 1 to remove and 14 not upgraded.
Need to get 0 B of archives. After unpacking 29.5 MB will be freed.
The following packages have unmet dependencies:
libgit-wrapper-perl : Depends: git but it is not going to be installed
git-extras : Depends: git (>= 1.7.0) but it is not going to be installed
bup : Depends: git but it is not going to be installed
git-remote-gcrypt : Depends: git but it is not going to be installed
git-svn : Depends: git (> 1:2.11.0) but it is not going to be installed
Depends: git ( 1:2.11.0) but it is not going to be installed
Depends: git (= 1:1.8.1) but it is not going to be installed
git-core : Depends: git (> 1:1.7.0.2) but it is not going to be installed
The following actions will resolve these dependencies: Remove the following packages:
1) bup [0.29-2 (now, testing, unstable)]
2) fdroidserver [0.7.0-1 (now, testing, unstable)]
3) git-annex [6.20161012-1 (now, testing)]
4) git-core [1:2.11.0-2 (now, testing, unstable)]
5) git-extras [4.2.0-1 (now, testing, unstable)]
6) git-remote-gcrypt [1.0.1-1 (now, testing, unstable)]
7) git-repair [1.20151215-1 (now, unstable)]
8) git-svn [1:2.11.0-2 (now, testing, unstable)]
9) gitk [1:2.11.0-2 (now, testing, unstable)]
10) libgit-wrapper-perl [0.047-1 (now, testing, unstable)]
11) python3-git [2.1.0-1 (now, testing, unstable)]
12) svn2git [2.4.0-1 (now, testing, unstable)] Leave the following dependencies unresolved:
13) devscripts recommends libgit-wrapper-perl
14) dh-make-perl recommends git
15) fdroidserver recommends git
16) git-annex recommends git-remote-gcrypt (>= 0.20130908-6)
17) gplaycli recommends fdroidserver
18) python-rope recommends git-core Accept this solution? [Y/n/q/?] q
Abandoning all efforts to resolve these dependencies.
Abort.
Then I made a note of all the packages being affected, saw purging all of them wouldn t call others (the Package dependency Hell), made the purge and then reinstalled anew. The reason I did this is that many a times during upgrade, either during update/upgrade sometimes the correct action doesn t happen. To take the git s example itself, there were two files git-extras and git-prompt which were in /etc/bash_completion.d/ both of which were showing their source as git. Purging git and installing git afresh removed git-extras file and git-prompt is the only one remaining. While blogging about the package, did try to grep through changelog.Debian.gz and changelog.gz in git

[shirish@debian] - [/usr/share/doc/git] - [10046]
[$] zless changelog.gz
and similarly

[shirish@debian] - [/usr/share/doc/git] - [10046]
[$] zless changelog.Debian.gz
But failed to find any mention of the now gone git-extras. Doing this with all the packages took considerable time as didn t want to deal with any potential fallout later on. For instance, ufw (uncomplicated firewall) also had an entry in /etc/bash_completion.d/, hence before purging ufw, took backup of all the rules I have made, did a successful simulation

[$] sudo aptitude -s purge ufw gufw

The following packages will be REMOVED:
gufw p ufw p
0 packages upgraded, 0 newly installed, 2 to remove and 14 not upgraded.
Need to get 0 B of archives. After unpacking 4,224 kB will be freed. Note: Using 'Simulate' mode.
Do you want to continue? [Y/n/?] y
Would download/install/remove packages.
purged the packages, reinstalled it and then re-added all the rules. Doing it all for various sundry packages, had to do it manually as there is no one size fits all solution. A sensitive one was grub which still has an entry in /etc/bash_completion.d/grub. Doing it wrong could have resulted in a non-bootable situation. There are workarounds for that, but it would have taken quite a bit of time, energy, notes and bit of recall factor what I did the last time something like that happened. Doing it manually, being present meant I could do it rightly the first time. So, was it worth it It would be if the package maintainers do the needful and the remaining entries are moved out of /etc/bash_completion.d/ to /usr/share/bash-completions and some to my favourite /usr/share/zsh/vendor-completions/ for instance

[shirish@debian] - [/usr/share/zsh/vendor-completions] - [10064]
[$] ll -h _youtube-dl -rw-r--r-- 1 root root 3.2K 2016-12-01 08:48 _youtube-dl But trying to get all or even major packages to use zsh-completions would be hard work and would take oodles of time and this concerns upstream stuff, also very much outside what I was sharing. World History Before, during and even after South-African experience, I was left wondering why India and South Africa, two countries who had similar histories at least the last couple of hundred years or more, the final result of Independence was so different for both the countries. It took me quite sometime to articulate that in a form of question , while the answers were interesting, from what little I know of India itself, if I were an Englishman I would never leave Hindustan . What the people answering failed to take into account was that in that era it was Hindustan or un-divided India. This map can be found at https://commons.wikimedia.org/wiki/File:British_Indian_Empire_1909_Imperial_Gazetteer_of_India.jpg and is part of quite a few Indian articles. I would urge people to look at the map in-depth. Except for the Central India Agency and Central India Provinces, most of the other regions were quite comfortable weather-wise. Hence I can t help but feel the assertion that Britishers didn t like India (as to live here) slightly revolting. See the excerpt/take on Dale Kennedy s The Magic Mountains: Hill Stations and the British Raj. Berkeley: University of California Press, 1996. xv + 264 pp. . A look at the list of hill stations of divided India is enough to tell that there were lot of places which either were founded by the Britishers or they chose to live there. And this is not all, there are supposed to lot of beautiful places even in Pakistan, especially in North East Frontier, Swat for instance. While today it s infamous for Taliban and Islamic Terrorism, there was a time it was known for its beauty. Trivia After Everest, K2 is the smaller one although whatever I have read of people s accounts, most people who ascended all 14 8,000 metre peaks say K2 is technically more tougher than Everest and after Everest has the highest casualty rate. Also places like the disputed North half of Pakistan Occupied Kashmir, Gilgit Baltistan, Extreme northern Punjab of Pakistan , Northern half of Khyber-Pakhtunkhawa province and Northern Balochistan all of these places would have been more than conducive to the Britishers as it is near to the British climate (snow and pleasant weather all year round). It really is a pity that Pakistan chose to become a terrorist state where it could have become one of the more toured places of Asia. I really feel nauseous and sad at the multiple chances that Pakistan frittered away, it could have been something else.
Filed under: Miscellenous Tagged: #bash-completion, #British Raj, #contributions, #debian, #debian-policy, #Debian-QA, #Hill Stations, #India Independance Movement, #lintian, #obsolete-conffiles, #unknown-horizons, #weather, #World History, adequate, Pakistan, tourism

During useR! 2016, Nick Tierney had asked on Twitter about rmarkdown and metropolis and whether folks had used RMarkdown-driven LaTeX Beamer presentations. My firm hell yeah answer, based on having used mtheme outright or in local mods for quite some time (see my talks page), lead to this blog post of mine describing this GitHub repo I had quickly set up during breaks at useR! 2016. The corresponding blog post and the repo have some more details on how I do this, in particular about local packages (also with sources on GitHub) for the non-standard fonts I use. This week I got around to updating the repo / example a little by making the default colours (in my example) a little less awful, and a page on blocks and, most importantly, turning the example into the animated gif below: And thanks to the beautiful tint package -- see its repo and CRAN package --- I now know how to create a template package. So if there is interest (and spare time), we could build a template package for RStudio too. With that, may I ask a personal favour of anybody still reading the post? Please do not hit my twitter handle with questions for support. All my code is an GitHub, and issue tickets there are much preferred. Larger projects like Rcpp also have their own mailing lists, and it is much better to use those. And if you like neither, maybe ask on StackOverflow. But please don't spam my Twitter handle. Thank you.

This is first security bugfix release for Weblate. This has to come at some point, fortunately the issue is not really severe. But Weblate got it's first CVE ID today, so it's time to address it in a bugfix release. Full list of changes:

• Do not leak account existence on password reset form (CVE-2017-5537).

If you are upgrading from older version, please follow our upgrading instructions. You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used on https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Aptoide, FreedomBox, Weblate itself and many other projects. Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure. Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far!

Filed under: Debian English SUSE Weblate 0 comments

What happened in the Reproducible Builds effort between Sunday January 8 and Saturday January 14 2017: Upcoming Events

• The Reproducible Build Zoo will be presented by Vagrant Cascadian at the Embedded Linux Conference in Portland, Oregon, February 22nd
• Dennis Gilmore and Holger Levsen will present about "Reproducible Builds and Fedora" at Devconf.cz on February, 27th.
• Introduction to Reproducible Builds will be presented by Vagrant Cascadian at Scale15x in Pasadena, California, March 5th

Reproducible work in other projects Reproducible Builds have been mentioned in the FSF high-priority project list. The F-Droid Verification Server has been launched. It rebuilds apps from source that were built by f-droid.org and checks that the results match. Bernhard M. Wiedemann did some more work on reproducibility for openSUSE. Bootstrappable.org (unfortunately no HTTPS yet) was launched after the initial work was started at our recent summit in Berlin. This is another topic related to reproducible builds and both will be needed in order to perform "Diverse Double Compilation" in practice in the future. Toolchain development and fixes Ximin Luo researched data formats for SOURCE_PREFIX_MAP and explored different options for encoding a map data structure in a single environment variable. He also continued to talk with the rustc team on the topic. Daniel Shahaf filed #851225 ('udd: patches: index by DEP-3 "Forwarded" status') to make it easier to track our patches. Chris Lamb forwarded #849972 upstream to yard, a Ruby documentation generator. Upstream has fixed the issue as of release 0.9.6. Alexander Couzens (lynxis) has made mksquashfs reproducible and is looking for testers. It compiles on BSD systems such as FreeBSD, OpenBSD and NetBSD. Bugs filed Chris Lamb:

• #850683 filed against gerbv.

Lucas Nussbaum:

• #850973 filed against shogun.

Nicola Corna:

• #851190 filed against gerbv.

Reviews of unreproducible packages 13 package reviews have been added and 13 have been removed in this week, adding to our knowledge about identified issues. 1 issue type has been added:

• help2man_puts_traceback_in_generated_man_page

Weekly QA work During our reproducibility testing, the following FTBFS bugs have been detected and reported by:

• Chris Lamb (3)
• Lucas Nussbaum (11)
• Nicola Corna (1)

diffoscope development Many bugs were opened in diffoscope during the past few weeks, which probably is a good sign as it shows that diffoscope is much more widely used than a year ago. We have been working hard to squash many of them in time for Debian stable, though we will see how that goes in the end

• Mattia Rizzolo:
  • Code quality and style improvements.
• Maria Glukhova:
  • Add image metadata comparison.
  • Zipinfo included in APK files comparison. (Closes: #850502)
  • Remove archive name from apktool.yml and rename it. (Closes: #850501)
• Chris Lamb:
  • Correctly escape value of href="" elements (re. #849411)
  • Support comparing .ico files using img2txt (Closes: #850730) and fixes and extra tests in subsequent commits.
  • comparators.utils.file: Include magic file type when we know the file format but can't find file-specific details. (Closes: #850850)
  • And other code quality and style improvements.

reproducible-website development

• Daniel Shahaf:
  • Add how-to-chair-a-meeting.md.
• Holger Levsen:
  • Add links to reports by several attendees of berlin2016

tests.reproducible-builds.org

• Ximin Luo and Holger Levsen worked on stricter tests to check that /dev/shm and /run/shm are both mounted with the correct permissions. Some of our build machines currently still fail this test, and the problem is probably the root cause of the FTBFS of some packages (which fails with issues regarding sem_open). The proper fix is still being discussed in #851427.
• Valerie Young worked on creating and linking autogenerated schema documentation for our database used to store the results.
• Holger Levsen added a graph with diffoscope crashes and timeouts.
• Holger also further improved the daily mail notifications about problems.

Misc. This week's edition was written by Ximin Luo, Chris Lamb and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

https://manpages.debian.org has been modernized! We have just launched a major update to our manpage repository. What used to be served via a CGI script is now a statically generated website, and therefore blazingly fast. While we were at it, we have restructured the paths so that we can serve all manpages, even those whose name conflicts with other binary packages (e.g. crontab(5) from cron, bcron or systemd-cron). Don t worry: the old URLs are redirected correctly. Furthermore, the design of the site has been updated and now includes navigation panels that allow quick access to the manpage in other Debian versions, other binary packages, other sections and other languages. Speaking of languages, the site serves manpages in all their available languages and respects your browser s language when redirecting or following a cross-reference. Much like the Debian package tracker, manpages.debian.org includes packages from Debian oldstable, oldstable-backports, stable, stable-backports, testing and unstable. New manpages should make their way onto manpages.debian.org within a few hours. The generator program ( debiman ) is open source and can be found at https://github.com/Debian/debiman. In case you would like to use it to run a similar manpage repository (or convert your existing manpage repository to it), we d love to help you out; just send an email to stapelberg AT debian DOT org. This effort is standing on the shoulders of giants: check out https://manpages.debian.org/about.html for a list of people we thank. We d love to hear your feedback and thoughts. Either contact us via an issue on https://github.com/Debian/debiman/issues/, or send an email to the debian-doc mailing list (see https://lists.debian.org/debian-doc/).

Issue ticket #20 demonstrated that we had not yet set up Windows for version 3 of Google Protocol Buffers ("Protobuf") -- while the other platforms support it. So I made the change, and there is release 0.4.8. RProtoBuf provides R bindings for the Google Protocol Buffers ("Protobuf") data encoding and serialization library used and released by Google, and deployed as a language and operating-system agnostic protocol by numerous projects. The NEWS file summarises the release as follows:

Changes in RProtoBuf version 0.4.8 (2017-01-17)

• Windows builds now use the proto3 library as well (PR #21 fixing #20)

CRANberries also provides a diff to the previous release. The RProtoBuf page has an older package vignette, a 'quick' overview vignette, a unit test summary vignette, and the pre-print for the JSS paper. Questions, comments etc should go to the GitHub issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Debian is very difficult, a puzzle. This surprising statement was what I got last week when talking with a group of new IT students (and their teachers). I would like to write down here what I was able to obtain from that conversation. From time to time, as part of my job at CICA, we open the doors of our datacenter to IT students from all around Andalusia (our region) who want to learn what we do here and how we do it. All our infraestructure and servers are primarily built using FLOSS software (we have some exceptions, like backbone routers and switches), and the most important servers run Debian. As part of the talk, when I am in such a meeting with a visiting group, I usually ask about which technologies they use and learn in their studies. The other day, they told me they use mostly Ubuntu and a bit of Fedora. When I asked why not Debian, the answer was that it was very difficult to install and manage. I tried to obtain some facts about this but I failed in what seems to be a case of bad fame, a reputation problem which was extended among the teachers and therefore among the students. I didn t detect any branding biasing or the like. I just seems lack of knowledge, and bad Debian reputation. Using my DD powers and responsabilities, I kindly asked for feedback to improve our installer or whatever they may find difficult, but a week later I have received no email so far. Then, what I obtain is nothing new:

• we probably need more new-users feedback
• we have work to do in the marketing/branding area
• we have very strong competitors out there
• we should keep doing our best

I myself recently had to use the Ubuntu installer in a laptop, and it didn t seem that different to the Debian one: same steps and choices, like in every other OS installation. Please, spread the word: Debian is not difficult. Certainly not perfect, but I don t think that installing and using Debian is such a puzzle.

I've been working on a fun holiday project in my spare time lately. It all started innocently enough. The office construction was nearing its end, and it was time for my workspace to be set up. Our deployment wizard and I were discussing. Normally we stick two high-end monitors on a desk. I'm blind; that seemed silly. He wanted to do something similarly nice for me, so he replaced one of the monitors with excellent speakers. They are a joy to listen to, but I felt like I should actually do something with them. So, I wanted to play around with some sort of audio project.
I decided to take a crack at an audio representation of network traffic. The solaris version of ping used to have an audio option, which would produce sound for successful pings. In the past I've used audio queues to monitor events like service health and build status.
It seemed like you could produce audio to give an overall feel for what was happening on the network. I was imagining a quick listen would be able to answer questions like:

1. How busy is the network


2. How many sources are active


3. Is the traffic a lot of streams or just a few?


4. Are there any interesting events such as packet loss or congestion collapse going on?


5. What's the mix of services involved

I divided the project into three segments, which I will write about in future entries:

• What parts of the network to model


• How to present the audio information


• Tools and implementation

I'm fairly happy with what I have. It doesn't represent all the items above. As an example, it doesn't directly track packet loss or retransmissions, nor does it directly distinguish one service from another. Still, just because of the traffic flow, rsync sounds different from http. It models enough of what I'm looking for that I find it to be a useful tool. And I learned a lot about music and Linux audio. I also got to practice designing discrete-time control functions in ways that brought back the halls of MIT.

Yesterday afternoon, the nineth update in the 0.12.* series of Rcpp made it to the CRAN network for GNU R. Windows binaries have by now been generated; and the package was updated in Debian too. This 0.12.9 release follows the 0.12.0 release from late July, the 0.12.1 release in September, the 0.12.2 release in November, the 0.12.3 release in January, the 0.12.4 release in March, the 0.12.5 release in May, the 0.12.6 release in July, the 0.12.7 release in September, and the 0.12.8 release in November --- making it the thirteenth release at the steady bi-montly release frequency. Rcpp has become the most popular way of enhancing GNU R with C or C++ code. As of today, 906 packages on CRAN depend on Rcpp for making analytical code go faster and further. That is up by sixthythree packages over the two months since the last release -- or about a package a day! Some of the changes in this release are smaller and detail-oriented. We did squash one annoying bug (stemming from the improved exception handling) in Rcpp::stop() that hit a few people. Nathan Russell added a sample() function (similar to the optional one in RcppArmadillo; this required a minor cleanup by for small number of other packages which used both namespaces 'opened'. Date and Datetime objects now have format() methods and << output support. We now have coverage reports via covr as well. Last but not least James "coatless" Balamuta was once more tireless on documentation and API consistency --- see below for more details.

Changes in Rcpp version 0.12.9 (2017-01-14)

• Changes in Rcpp API:
  • The exception stack message is now correctly demangled on all compiler versions (Jim Hester in #598)
  • Date and Datetime object and vector now have format methods and operator<< support (#599).
  • The size operator in Matrix is explicitly referenced avoiding a g++-6 issues (#607 fixing #605).
  • The underlying date calculation code was updated (#621, #623).
  • Addressed improper diagonal fill for non-symmetric matrices (James Balamuta in #622 addressing #619)
• Changes in Rcpp Sugar:
  • Added new Sugar function sample() (Nathan Russell in #610 and #616).
  • Added new Sugar function Arg() (James Balamuta in #626 addressing #625).
• Changes in Rcpp unit tests
  • Added Environment::find unit tests and an Environment::get(Symbol) test (James Balamuta in #595 addressing issue #594).
  • Added diagonal matrix fill tests (James Balamuta in #622 addressing #619)
• Changes in Rcpp Documentation:
  • Exposed pointers macros were included in the Rcpp Extending vignette (MathurinD; James Balamuta in #592 addressing #418).
  • The file Rcpp.bib move to directory bib which is guaranteed to be present (#631).
• Changes in Rcpp build system
  • Travis CI now also calls covr for coverage analysis (Jim Hester in PR #591)

Thanks to CRANberries, you can also look at a diff to the previous release. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Many of you are big fans of S.W.O.T analysis, I am sure of that! :-) Technical competence is our strongest suit, but we have reached a size and sphere of influence which requires an increase in organisation.

We all love our project and want to make sure Debian still shines in the next decades (and centuries!). One way to secure that goal is to identify elements/events/things which could put that goal at risk. To this end, we've organized a short S.W.O.T analysis session at DebConf16. Minutes of the meeting can be found here. I believe it is an interesting read and is useful for Debian old-timers as well as newcomers. It helps to convey a better understanding of the project's status. For each item, we've tried to identify an action.

Here are a few things we've worked on:

• Identify new potential contributors by attending and speaking at conferences where Free and Open Sources software are still not very well-known, or where we have too few contributors.
  
  Each Debian developer is encouraged to identify events where we can promote FOSS and Debian. As DPL, I'd be happy to cover expenses to attend such events.
• Our average age is also growing over the years. It is true that we could attract more new contributors than we already do.
  
  We can organize short internships. We should not wait for students to come to us. We can get in touch with universities and engineering schools and work together on a list of topics. It is easy and will give us the opportunity to reach out to more students.
  
  It is true that we have tried in the past to do that. We may organize a sprint with interested people and share our experience on trying to do internships on Debian-related subjects. If you have successfully done that in the past and managed to attract new contributors that way, please share your experience with us!
  
  If you see other ways to attract new contributors, please get in touch so that we can discuss!
• Not easy to get started in the project.
  
  It could be argued that all the information is available, but rather than being easily findable from on starting point, it is scattered over several places (documentation on our website, wiki, metadata on bug reports, etc ).
  
  Fedora and Mozilla both worked on this subject and did build a nice web application to make this easier and nicer. The result of this is asknot-ng.
  
  A whatcanidofor.debian.org would be wonderful! Any takers? We can help by providing a virtual machine to build this. Being a DD is not mandatory. Everyone is welcome!
• Cloud images for Debian.
  
  This is a very important point since cloud providers are now major distributions consumers. We have to ensure that Debian is correctly integrated in the cloud, without making compromises on our values and philosophy.
  
  I believe this item has been worked on during the last Debian Cloud sprint. I am looking forward to seeing the positive effects of this sprint in the long term. I believe it does help us to build a stronger relationship with cloud providers and gives us a nice opportunity to work with them on a shared set of goals!

During next DebConf, we can review the progress that has been made on each item and discuss new ones. In addition to this session acting as a health check, I see it as a way for the DPL to discuss, openly and publicly, about the important changes that should be implemented in the project and imagine together a better future.

In the meantime, everyone should feel free to pick one item from the list and work on it. :-)

A month or so ago I got involved in a discussion on IRC about notification methods for a headless NAS. One of the options considered was some sort of USB attached LED. DealExtreme had a cheap Webmail notifier , which was already supported by mainline kernels as a Riso Kagaku device but it had been sold out for some time. This seemed like a fun problem to solve with a tinyAVR and V-USB. I had my USB relay board so I figured I could use that to at least get some code to the point that the kernel detected it as the right device, and the relay output could be configured as one of the colours to ensure it was being driven in roughly the right manner. The lack of a full lsusb dump (at least when I started out) made things a bit harder, plus the fact that the Riso uses an output report unlike the relay code, which uses a control message. However I had the kernel source for the driver and with a little bit of experimentation had something which would cause the driver to be loaded and the appropriate files in /sys/class/leds/ to be created. The relay was then successfully activated when the red LED was supposed to be on. I subsequently ordered some Digispark clones and modified the code to reflect the pins there (my relay board used pins 1+2 for USB, the Digispark uses pins 3+4). I then soldered a tricolour LED to the board, plugged it in and had a clone of the Riso Kaguku device for about 1.50 in parts (no doubt much cheaper in bulk). Very chuffed. In case it s useful to someone, the code is released under GPLv3+ and is available at https://the.earth.li/gitweb/?p=riso-kagaku-clone.git;a=summary or on GitHub at https://github.com/u1f35c/riso-kagaku-clone. I m seeing occasional issues on an older Dell machine that only does USB2 with enumeration, but it generally is fine once it gets over that. (FWIW, Jon, who started the original discussion, ended up with a BlinkStick Nano which is a neater device with 2 LEDs but still based on an Tiny85.)

What happened in the Reproducible Builds effort between Sunday January 1 and Saturday January 7 2017: GSoC and Outreachy updates

• Maria Glukhova blogged about Getting to know diffoscope better.

Toolchain development

• #849999 was filed: "dpkg-dev should not set SOURCE_DATE_EPOCH to the empty string"

Packages reviewed and fixed, and bugs filed Chris Lamb:

• #849968 filed against pciutils.
• #849972 filed against yard.
• #850556 filed against sed.

Dhole:

• #849926 filed against nxt-firmware.

Reviews of unreproducible packages 13 package reviews have been added, 4 have been updated and 6 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been added/updated:

• Add new randomness_in_documentation_generated_by_yardoc toolchain issue.
• Add fix for randomness_in_documentation_generated_by_yardoc toolchain issue.

Upstreaming of reproducibility fixes Merged:

• calibre
• jna
• rocksndiamonds
• avfs
• jackaudio1
• jackaudio2
• gri

Opened:

• rockdodger

Weekly QA work During our reproducibility testing, the following FTBFS bugs have been detected and reported by:

• Chris Lamb (4)

diffoscope development diffoscope 67 was uploaded to unstable by Chris Lamb. It included contributions from :

[ Chris Lamb ]
* Optimisations:
  - Avoid multiple iterations over archive by unpacking once for an ~8X
    runtime optimisation.
  - Avoid unnecessary splitting and interpolating for a ~20X optimisation
    when writing --text output.
  - Avoid expensive diff regex parsing until we need it, speeding up diff
    parsing by 2X.
  - Alias expensive Config() in diff parsing lookup for a 10% optimisation.
* Progress bar:
  - Show filenames, ELF sections, etc. in progress bar.
  - Emit JSON on the the status file descriptor output instead of a custom
    format.
* Logging:
  - Use more-Pythonic logging functions and output based on __name__, etc.
  - Use Debian-style "I:", "D:" log level format modifier.
  - Only print milliseconds in output, not microseconds.
  - Print version in debug output so that saved debug outputs can standalone
    as bug reports.
* Profiling:
  - Also report the total number of method calls, not just the total time.
  - Report on the total wall clock taken to execute diffoscope, including
    cleanup.
* Tidying:
  - Rename "NonExisting" -> "Missing".
  - Entirely rework diffoscope.comparators module, splitting as many separate
    concerns into a different utility package, tidying imports, etc.
  - Split diffoscope.difference into diffoscope.diff, etc.
  - Update file references in debian/copyright post module reorganisation.
  - Many other cleanups, etc.
* Misc:
  - Clarify comment regarding why we call python3(1) directly. Thanks to J r my
    Bobbio <lunar@debian.org>.
  - Raise a clearer error if trying to use --html-dir on a file.
  - Fix --output-empty when files are identical and no outputs specified.
[ Reiner Herrmann ]
* Extend .apk recognition regex to also match zip archives (Closes: #849638)
[ Mattia Rizzolo ]
* Follow the rename of the Debian package "python-jsbeautifier" to
  "jsbeautifier".
[ siamezzze ]
* Fixed no newline being classified as order-like difference.

reprotest development reprotest 0.5 was uploaded to unstable by Chris Lamb. It included contributions from:

[ Ximin Luo ]
* Stop advertising variations that we're not actually varying.
  That is: domain_host, shell, user_group.
* Fix auto-presets in the case of a file in the current directory.
* Allow disabling build-path variations. (Closes: #833284)
* Add a faketime variation, with NO_FAKE_STAT=1 to avoid messing with
  various buildsystems. This is on by default; if it causes your builds
  to mess up please do file a bug report.
* Add a --store-dir option to save artifacts.

Other contributions (not yet uploaded):

• Chris Lamb:
  • setup.py: Specify a Python3 env shebang
  • Drop reprotest/ runner,tools from DEP-5 copyright wildcard matching.

reproducible-builds.org website development

• website.git notifications now go to#reproducible-builds. Thanks todanielsh and the KGB bot maintainers.
• Daniel Shahaf:
  • events: add IRC meeting
  • Move mailing list from 'who' to 'resources'.
• Holger Levsen:
  • link to meetbot.debian.net IRC meetings archives
• Chris Lamb:
  • tools: Underscore that disorderfs is deliberately adding such randomness.

tests.reproducible-builds.org

• Debian arm64 architecture was fully tested in all three suites in just 15 days. Thanks again to Codethink.co.uk for their support!
• Log diffoscope profiling info. (lamby)
• Run pg_dump with -O --column-inserts to make easier to import our main database dump into a non-PostgreSQL database. (mapreri)
• Debian armhf network: CPU frequency scaling was enabled for three Firefly boards, enabling the CPUs to run at full speed. (vagrant)
• Arch Linux and Fedora tests have been disabled (h01ger)
• Improve mail notifications about daily problems. (h01ger)

Misc. This week's edition was written by Chris Lamb, Holger Levsen and Vagrant Cascadian, reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Last week, Josh sent the call for papers to the R-SIG-Finance list making everyone aware that we will have our nineth annual R/Finance conference in Chicago in May. Please see the call for paper (at the link, below, or at the website) and consider submitting a paper. We are once again very excited about our conference, thrilled about upcoming keynotes and hope that many R / Finance users will not only join us in Chicago in May 2017 -- but also submit an exciting proposal. We also overhauled the website, so please see R/Finance. It should render well and fast on devices of all sizes: phones, tablets, desktops with browsers in different resolutions. The program and registration details still correspond to last year's conference and will be updated in due course. So read on below, and see you in Chicago in May!