Search Results: "reg"

15 October 2016

Dirk Eddelbuettel: anytime 0.0.3: Extension and fixes

anytime arrived on CRAN with releases 0.0.1 and 0.0.2 about a month ago. anytime aims to convert anything in integer, numeric, character, factor, ordered, ... format to POSIXct (or Date) objects. Release 0.0.3 brings a bugfix for Windows (where for dates before the epoch of 1970-01-01, accessing the tm_isdst field for daylight savings would crash the session) and a small (unexported) extension to test format strings. This last feature plays well the ability to add format strings which we added in 0.0.2. The NEWS file summarises the release:

Changes in Rcpp version 0.0.3 (2016-10-13)
  • Added (non-exported) helper function testFormat()
  • Do not access tm_isdst on Windows for dates before the epoch (pull request #13 fixing issue #12); added test as well

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the anytime page. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

14 October 2016

Antoine Beaupr : Managing good bug reports

Bug reporting is an art form that is too often neglected in software projects. Bug reports allow contributors to participate without deep technical knowledge and at the same time provide a crucial space for developers to be made aware of issues with their software that they could not have foreseen or found themselves, for lack of resources, variety or imagination.

Prior art Unfortunately, there are rarely good guidelines for submitting bug reports. Historically, people have pointed towards How to report bugs effectively or How to ask questions the smart way. While those guides can be useful for motivated people and may seem attractive references for project managers, they suffer from serious issues:
  • they are written by technical people, for non-technical people
  • as a result, they have a deeply condescending attitude such as calling people "stupid" or various animal names like "mongoose"
  • they are also very technical themselves: one starts with a copyright notice and a changelog, the other uses magic words like "Core dumps" and $Id$
  • they are too long: sgtatham's is about 3600 words long, esr's is even longer at about 11800 words. those texts will take about 20 to 60 minutes to read by an average reader, according to research
Individual projects have their own guides as well. Linux has the REPORTING_BUGS file that is a much shorter 1200 that can be read under 5 minutes, provided that you can understand the topic at hand. Interestingly, that guide refers to both esr's and sgtatham's guidelines, which means, in the degenerate case where the user hasn't had the "privilege" of reading esr's prose already, they will have an extra hour and a half of reading to do to have honestly followed the guidelines before reporting the bug. I often find good documentation in the Tails project. Their bug reporting guidelines are easily accessible and quick to read, although they still might be too technical. It could be argued that you need to get technical at some point to get that information out, of course. In the Monkeysign project, I have started a bug reporting guide that doesn't yet address all those issues. I am considering writing a new guide, but I figured I would look at other people's work and get feedback before writing my own standard.

What's the point? Why have those documents been written? Are people really expected to read them before seeking help? It seems to me unlikely that someone would:
  1. be motivated enough to do something about a broken part of their computer
  2. figure out they can do something about it
  3. read a fifteen thousand words novel about how to report a bug...
  4. just to finally write a 20-line bug report that has no warranty of support attached to it
And if I would be a paying customer, I wouldn't want to be forced to waste my time reading that prose either: it's your job to help me fix your broken things, not the reverse. As someone doing consulting these days: I totally understand: it's not you, the user, it's us, the developers, that have a problem. We have been socialized through computers, and it makes us weird and obtuse, but that's no excuse, and we need to clean up our act. Furthermore, it's surprising how often we get (and make!) bug reports that can be difficult to use. The Monkeysign project is very "technical" and I have expected that the bug reports I would get would be well written, with ways to reproduce and so on, but it happened that I received bug reports that were all over the place, didn't have any ways of reproducing or were simply incomplete. Those three bug reports were filed by people that I know to be very technically capable: one is a fellow Debian developer, the second had filed a good bug report 5 days before, and the third one is a contributor that sent good patches before. In all three cases, they knew what they were doing. Those three people probably read the guidelines mentioned in the past. They may have even read the Monkeysign bug reporting guidelines as well. I can only explain those bug reports by the lack of time: people thought the issue was obvious, that it would get fixed rapidly because, obviously, something is broken. We need a better way.

The takeaway What are those guides trying to tell us?
  1. ask questions in the right place
  2. search for similar questions and issues before reporting the bug
  3. try to make the developers reproduce the issues
  4. failing that, try to describe the issue as well as you can
  5. write clearly, be specific and verbose yet concise
There are obviously contradictions in there, like sgtatham telling us to be verbose and esr tells us to, basically, not be verbose. There is definitely a tension in there, and there are many, many more details about how great bug reports can be if done properly. I tend towards the side of terseness in our descriptions: people that will know how to be concise will be, people that don't will most likely not learn by reading a 12 000 words novel that, in itself, didn't manage to be parsimonious. But I am willing to allow for verbosity in bug reports: I prefer too many details instead of missing a key bit of information.

Issue trackers Step 1 is our job: we should send people in the right place, and give them the right tools. Monkeysign used to manage bugs with bugs-everywhere and this turned out to be a terrible idea: you had to understand git and bugs-everywhere to file any bug reports. As a result, there were exactly zero bug reports filed by non-developers during the whole time BE was used, although some bugs were filed in the Debian Bugtracker. So have a good bug tracker. A mailing list or email address is not a good bug tracker: you lose track of old issues, and it's hard for newcomers to search the archives. It does have the advantage of having a unified interface for the support forum and bug tracking, however. Redmine, Gitlab, Github and others are all decent-enough bug trackers. The key point is that the issue tracker should be publicly available, and users should be able to register easily to file new issues. You should also be able to mass-edit tickets and users should be able to discover the tracker's features easily. I am sorry to say that the Debian BTS somewhat falls short on those two features. Step 2 is a shared responsibility: there should be an easy way to search for issues, and we should help the user looking for similar issues. Stackexchange sites do an excellent job at this, by automatically searching for similar questions while you write your question, suggesting similar ones in an attempt to weed out duplicates. Duplicates still happen, but they can then clearly be marked and linked with a distinct mechanism. Most bug trackers do not offer such high level functionality, but should, so I feel the fault lies more on "our" end than at the user's end.

Reproducing the environment Step 3 and 4 are more or less the user's responsibility. We can detail in our documentation how to clearly share the environment where we reproduced the bug, for example, but in the end, the user decides if they want to share that information or not. In Monkeysign, I have finally implemented joeyh's suggestion of shipping the test suite with the program. I can now tell people to run the test suite in their environment to see if this is a regression that is specific to their environment - so a known bug, in a way - or a novel bug for which I can look at writing a new unit test. I also include way more information about the environment in the --version output, an idea I brought forward in the borg project to ease debugging. That way, people can just send the output of monkeysign --test and monkeysign --version, and I have a very good overview of what is happening on their end. Of course, Monkeysign also supports the usual --verbose and --debug flag that users should enable when reproducing issues. Another idea is to report bugs directly from the application. We have all seen Firefox or other software have automatic bug reporting tools, but somehow those seem unsatisfactory for a user: we have no feedback of where the report goes, if it's followed up on. It is useful for larger project to get statistical data, but not so useful for users in the short term. Monkeysign tries to handle exceptions in the code in a graceful way, but could do better. We use a small library to handle exceptions, but that library has since then been improved to directly file bugs against the Github project. This assumes the user is logged into Github, but it is nice to pre-populate bug reports with the relevant information up front.

Issue templates In the meantime, to make sure people provide enough information, I have now moved a lot of the bug reporting guidelines to a separate issue template. That issue template is available through the issue creation form now, although it is not enabled by default, a weird limitation of Gitlab. Issue templates are available in Gitlab and Github. Issue templates somewhat force users in a straight jacket: there is already something to structure their bug report. Those could be distinct form elements that had to be filled in, but I like the flexibility of the template, and the possibility for users to just escape the formalism and just plead for help in their own way.

Issue guidelines In the end, I opted for a short few paragraphs in the style of the Tails documentation, including a reference to sgtatham, as an optional future reference:
  • Before you report a new bug, review the existing issues in the online issue tracker and the Debian BTS for Monkeysign to make sure the bug has not already been reported elsewhere.
  • The first aim of a bug report is to tell the developers exactly how to reproduce the failure, so try to reproduce the issue yourself and describe how you did that.
  • If that is not possible, try to describe what went wrong in detail. Write down the error messages, especially if they have numbers.
  • Take the necessary time to write clearly and precisely. Say what you mean, and make sure it cannot be misinterpreted.
  • Include the output of monkeysign --test, monkeysign --version and monkeysign --debug in your bug reports. See the issue template for more details about what to include in bug reports.
If you wish to read more about issues regarding communication in bug reports, you can read How to Report Bugs Effectively, which takes around 20 to 30 minutes.
Unfortunately, short of rewriting sgtatham's guide, I do not feel there is much more we can do as a general guide. I find esr's guide to be too verbose and commanding, so sgtatham it will be for now.

The prose and literacy In the end, there is a fundamental issue with reporting bugs: it assumes our users are literate and capable of writing amazing prose that we will enjoy reading as the last J.K. Rowling novel (if you're into that kind of thing). It's just an unreasonable expectation: some of your users don't even speak the same language as you, let alone read or write it. This makes for challenging collaboration, to say the least. This is where automated reporting makes sense: it doesn't require user's intervention, and the communication is mediated by machines without human intervention and their pesky culture. But we should, as maintainers, "be liberal in what we accept and conservative in what we send". Be tolerant, and help your users in fixing their issues. It's what you are there for, after all. And in the end, we all fail the same way. In an attempt to improve the situation on bug reporting guides, I seem to have myself written a 2000 short story that will have taken up a hopefully pleasant 10 minutes of your time at minimum. Hopefully I will have succeeded at being clear, specific, verbose and concise all at once and look forward to your feedback on how to improve our bug reporting culture.

Daniel Silverstone: Gitano - Approaching Release - Changes

Continuing on from the previous article, here is a (probably incomplete) list of the critical changes to Gitano which have been, or will be, worked on during the run toward a 1.0 release. Each of these will have a blog posting to discuss what the changes mean for current and future users. Sometimes I'll aggregate postings, sometimes I won't. The following are some highlights from the past little while of development which has been undertaken by Richard and myself. Each item is, I feel, important enough to warrant commentary, even for those who already use Gitano. Any number of smaller things have been done which fall below some arbitrary barrier for telling you about. If you're aware of any of them and feel they are worthwhile telling the world about, then please prod me and I'll add an article to the series. Finally it's worth noting that the effort to get all this into Debian Stretch proceeds apace. Of the eight packages needed, at the time of posting: one was already in and has been updated (luxio), three have been accepted into Debian already (supple, clod, lua-scrypt), two are in NEW (gall and lace), and that leaves the newest library (tongue) and then Gitano itself still to go. The Debian FTP team have been awesome in helping me with all this, so thanks go to them.

13 October 2016

Dirk Eddelbuettel: Seinfeld streak at GitHub: Round Three

Two years ago in this post I reference the Seinfeld Streak used in an even earlier post of regular updates to to the Rcpp Gallery:
This is sometimes called Jerry Seinfeld's secret to productivity: Just keep at it. Don't break the streak.
and showed the this first chart of GitHub streaking
github activity october 2013 to october 2014
Last year a follow-up appeared in this post:
github activity october 2014 to october 2015
And as it is that time again, here is this year's version:
github activity october 2015 to october 2016
Special thanks go to Alessandro Pezz for the Chrome add-on GithubOriginalStreak.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

10 October 2016

Petter Reinholdtsen: Experience and updated recipe for using the Signal app without a mobile phone

In July I wrote how to get the Signal Chrome/Chromium app working without the ability to receive SMS messages (aka without a cell phone). It is time to share some experiences and provide an updated setup. The Signal app have worked fine for several months now, and I use it regularly to chat with my loved ones. I had a major snag at the end of my summer vacation, when the the app completely forgot my setup, identity and keys. The reason behind this major mess was running out of disk space. To avoid that ever happening again I have started storing everything in userdata/ in git, to be able to roll back to an earlier version if the files are wiped by mistake. I had to use it once after introducing the git backup. When rolling back to an earlier version, one need to use the 'reset session' option in Signal to get going, and notify the people you talk with about the problem. I assume there is some sequence number tracking in the protocol to detect rollback attacks. The git repository is rather big (674 MiB so far), but I have not tried to figure out if some of the content can be added to a .gitignore file due to lack of spare time. I've also hit the 90 days timeout blocking, and noticed that this make it impossible to send messages using Signal. I could still receive them, but had to patch the code with a new timestamp to send. I believe the timeout is added by the developers to force people to upgrade to the latest version of the app, even when there is no protocol changes, to reduce the version skew among the user base and thus try to keep the number of support requests down. Since my original recipe, the Signal source code changed slightly, making the old patch fail to apply cleanly. Below is an updated patch, including the shell wrapper I use to start Signal. The original version required a new user to locate the JavaScript console and call a function from there. I got help from a friend with more JavaScript knowledge than me to modify the code to provide a GUI button instead. This mean that to get started you just need to run the wrapper and click the 'Register without mobile phone' to get going now. I've also modified the timeout code to always set it to 90 days in the future, to avoid having to patch the code regularly. So, the updated recipe for Debian Jessie:
  1. First, install required packages to get the source code and the browser you need. Signal only work with Chrome/Chromium, as far as I know, so you need to install it.
    apt install git tor chromium
    git clone https://github.com/WhisperSystems/Signal-Desktop.git
    
  2. Modify the source code using command listed in the the patch block below.
  3. Start Signal using the run-signal-app wrapper (for example using pwd /run-signal-app).
  4. Click on the 'Register without mobile phone', will in a phone number you can receive calls to the next minute, receive the verification code and enter it into the form field and press 'Register'. Note, the phone number you use will be user Signal username, ie the way others can find you on Signal.
  5. You can now use Signal to contact others. Note, new contacts do not show up in the contact list until you restart Signal, and there is no way to assign names to Contacts. There is also no way to create or update chat groups. I suspect this is because the web app do not have a associated contact database.
I am still a bit uneasy about using Signal, because of the way its main author moxie0 reject federation and accept dependencies to major corporations like Google (part of the code is fetched from Google) and Amazon (the central coordination point is owned by Amazon). See for example the LibreSignal issue tracker for a thread documenting the authors view on these issues. But the network effect is strong in this case, and several of the people I want to communicate with already use Signal. Perhaps we can all move to Ring once it work on my laptop? It already work on Windows and Android, and is included in Debian and Ubuntu, but not working on Debian Stable. Anyway, this is the patch I apply to the Signal code to get it working. It switch to the production servers, disable to timeout, make registration easier and add the shell wrapper:
cd Signal-Desktop; cat <<EOF   patch -p1
diff --git a/js/background.js b/js/background.js
index 24b4c1d..579345f 100644
--- a/js/background.js
+++ b/js/background.js
@@ -33,9 +33,9 @@
          );
      );
 
-    var SERVER_URL = 'https://textsecure-service-staging.whispersystems.org';
+    var SERVER_URL = 'https://textsecure-service-ca.whispersystems.org';
     var SERVER_PORTS = [80, 4433, 8443];
-    var ATTACHMENT_SERVER_URL = 'https://whispersystems-textsecure-attachments-staging.s3.amazonaws.com';
+    var ATTACHMENT_SERVER_URL = 'https://whispersystems-textsecure-attachments.s3.amazonaws.com';
     var messageReceiver;
     window.getSocketStatus = function()  
         if (messageReceiver)  
diff --git a/js/expire.js b/js/expire.js
index 639aeae..beb91c3 100644
--- a/js/expire.js
+++ b/js/expire.js
@@ -1,6 +1,6 @@
 ;(function()  
     'use strict';
-    var BUILD_EXPIRATION = 0;
+    var BUILD_EXPIRATION = Date.now() + (90 * 24 * 60 * 60 * 1000);
 
     window.extension = window.extension    ;
 
diff --git a/js/views/install_view.js b/js/views/install_view.js
index 7816f4f..1d6233b 100644
--- a/js/views/install_view.js
+++ b/js/views/install_view.js
@@ -38,7 +38,8 @@
             return  
                 'click .step1': this.selectStep.bind(this, 1),
                 'click .step2': this.selectStep.bind(this, 2),
-                'click .step3': this.selectStep.bind(this, 3)
+                'click .step3': this.selectStep.bind(this, 3),
+                'click .callreg': function()   extension.install('standalone')  ,
              ;
          ,
         clearQR: function()  
diff --git a/options.html b/options.html
index dc0f28e..8d709f6 100644
--- a/options.html
+++ b/options.html
@@ -14,7 +14,10 @@
         <div class='nav'>
           <h1>  installWelcome  </h1>
           <p>  installTagline  </p>
-          <div> <a class='button step2'>  installGetStartedButton  </a> </div>
+          <div> <a class='button step2'>  installGetStartedButton  </a>
+	    <br> <a class="button callreg">Register without mobile phone</a>
+
+	  </div>
           <span class='dot step1 selected'></span>
           <span class='dot step2'></span>
           <span class='dot step3'></span>
--- /dev/null   2016-10-07 09:55:13.730181472 +0200
+++ b/run-signal-app   2016-10-10 08:54:09.434172391 +0200
@@ -0,0 +1,12 @@
+#!/bin/sh
+set -e
+cd $(dirname $0)
+mkdir -p userdata
+userdata=" pwd /userdata"
+if [ -d "$userdata" ] && [ ! -d "$userdata/.git" ] ; then
+    (cd $userdata && git init)
+fi
+(cd $userdata && git add . && git commit -m "Current status."   true)
+exec chromium \
+  --proxy-server="socks://localhost:9050" \
+  --user-data-dir=$userdata --load-and-launch-app= pwd 
EOF
chmod a+rx run-signal-app
As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

9 October 2016

Bits from Debian: Debian is participating in the next round of Outreachy!

Following the success of the last round of Outreachy, we are glad to announce that Debian will take part in the program for the next round, with internships lasting from the 6th of December 2016 to the 6th of March 2017. From the official website: Outreachy helps people from groups underrepresented in free and open source software get involved. We provide a supportive community for beginning to contribute any time throughout the year and offer focused internship opportunities twice a year with a number of free software organizations. Currently, internships are open internationally to women (cis and trans), trans men, and genderqueer people. Additionally, they are open to residents and nationals of the United States of any gender who are Black/African American, Hispanic/Latin@, American Indian, Alaska Native, Native Hawaiian, or Pacific Islander. If you want to apply to an internship in Debian, you should take a look at the wiki page, and contact the mentors for the projects listed, or seek more information on the (public) debian-outreach mailing-list. You can also contact the Outreach Team directly. If you have a project idea and are willing to mentor an intern, you can submit a project idea on the Outreachy wiki page. Here's a few words on what the interns for the last round achieved within Outreachy: Debian would not be able to participate in Outreachy without the help of the Software Freedom Conservancy, who provides administrative support for Outreachy, as well as the continued support of Debian's donors, who provide funding for the internships. If you want to donate, please get in touch with one of our trusted organizations. Debian is looking forward to welcoming new interns for the next few months, come join us!

Ben Armstrong: Annual Hike with Ryan: Salt Marsh Trail, 2016

Once again, Ryan Neily and I met last month for our annual hike. This year, to give our aging knees a break, we visited the Salt Marsh Trail for the first time. For an added level of challenge and to access the trail by public transit, we started with the Shearwater Flyer Trail and finished with the Heritage Trail. It was a perfect day both for hiking and photography: cool with cloud cover and a refreshing coastal breeze. The entire hike was over 25 km and took the better part of the day to complete. Good times, great conversations, and I look forward to visiting these beautiful trails again!
Salt Marsh trail hike, 2016. Click to start the slideshow.Salt Marsh trail hike, 2016. Click to start the slideshow.
We start here, on the Shearwater flyer trail.We start here, on the Shearwater flyer trail.
Couldn t ID this bush. The berries are spectacular! A pond to the side of the trail. Different angle for dramatic lighting effect. Rail bridge converted to foot bridge. Cranberries! Reviewing our progress. From the start Map of the Salt Marsh trail ahead. Off we go again! First glimpse through the trees. Appreciating the cloud cover today. Salt-marshy grasses. Never far from rocks in NS. Rocks all laid out in stripes. Lunch & selfie time. Ryan attacking his salad. Vantage point. A bit of causeway coast. Plenty of eel grass. Costal flora. We head for the bridge next. Impressed by the power of the flow beneath. Snapping more marsh shots. Ripples. Gulls, and if you squint, a copter. More ripples. Swift current along this channel. Until it broadens out and slows down. Nearly across. Heron! Sorry it s so tiny. Heron again, before I lost it. Ducks at the head of the Atlantic View trail where we rested and then turned back. Attempt at artsy. Nodding ladies tresses on the way back. Several of them. Sky darkening, but we still have time. A lonely wild rose. The last gasp of late summer. Back across the marshes. A short breather on the Heritage Trail.
Here s the Strava record of our hike:
Facebooktwittergoogle_plusredditpinterestlinkedinmail

Laura Arjona Reina: New phone: Samsung Galaxy S III phone with Replicant

Thanks to the Bazaar effort of The Guardian Project, I ve been offered a phone to test F-Droid and other free software apps for Android. I accepted the offer, and chose a Samsung Galaxy S III phone with Replicant 4.2.2, installed and shipped by Tehnoetic.
I m using it now as my main phone, and since it uses Android 4.x I m able to install more modern apps than in my old Galaxy Ace (which remains usable with CyanongenMod 7.2 (Android 2.3.7)).
My plans with this new phone are:
  • Test Replicant and free software for Android on it
  • Get more involved in translations of Android apps
  • Get more involved in the F-Droid community
  • Keep an eye on Android tools in Debian
  • Post here in my blog articles about what I ve been doing (and of course report issues and contributions upstream)
Migration to the new phone
I ve migrated my stuff from the old phone to this one. Some notes:
  • Wrote down my list of apps
  • Used Slight Backup for contacts, call logs and messages
  • Periodical has its own backup tool
  • Whatsapp has its own backup tool
  • Exported settings in K-9 Mail
  • Exported Kontalk GPG key
  • Simply Do has its own backup tool
  • I don t use calendars in the phone so I didn t migrate any events (I have Offline Calendar to ad temporary notes/reminders, but that s all)
I moved the SIM card and the SD Card to the new phone and tried the restore tool for each app.
I found out that several apps could not find the backups because they were not looking at the SD Card for the files (seems that they were using internal memory locations). So for recovering my backups, I made new backups in the new phone with the empty apps, then found out where those backups were created (in the internal memory, /storage/emulated/0), and then copied the authentic backup files there (overwriting teh dummy ones), and then used the app to restore the backup.
For some apps (K-9) I had to set again the folder for attachments, since the SD was not anymore in /media/sdcard, now it was in /storage/sdcard1.
Apart from that, everything went well.
I was a bit upset that I could not migrate Kontalk conversations (there is no backup/export tool, and I am not sure where are the files/database stored).
I noticed that although Kontalk is registered using the phone number, and it uses the phone numbers for contacts, it kept working in the old phone (Whatsapp detects when you change to a new phone and kind of deactivates itself in the old one, but that s not the case for Kontalk: it works as any XMPP client (if it s open, it can send/receive messages)).
Replicant 4.2 in a Galaxy S III (i9300)
Here I write some particularities that I found in the phone, mostly bugs or problems. But don t get me wrong: overall I m very happy with it!
I experienced a problem when using the phone to make/receive calls, it seemed that the proximity sensor was not working well. I thought it was a Replicant issue, but later I realized that there was a Tehnoetic sticker that was partially covering the sensor. I removed the sticker and everything worked well.
The phone came with F-Droid installed which is nice. I upgraded to the latest alpha and I m testing the alpha releases since then
I found that I cannot choose where to install apps nor move apps from internal memory to the SD Card: there is no such option in Settings > Apps > Manage Apps (there is such setting in my CyanogenMod 7.2 phone, though). Since my phone is rooted and I have full access to both internal memory and SDCard, and I have plenty of room in the internal memory, I didn t bother too much. I m not sure if this is a bug, a feature, something related to Android 4 or specific to Replicant, o specific to this phone model. Pending to investigate, but low priority.
Replicant is almost fully translated to Spanish, yay!. I only found one untranslated string: You go to Settings > Wireless > Cell Broadcasts, and in the settings page, Cell Broadcasts is untranslated (but the settings themselves are). I still need to find where/how to send a patch for this (not sure if it comes from Android, CyanogenMod, or it s something specific for Replicant. Also, being Android 4.x, I m not sure about the usefulness of reporting such a minimal and unimportant patch upstream ).
When I turn on the phone, I get the Samsung S III splash screen, later the Replicant Splash screen, later the numeric pad to unlock the SIM card. After that, I see the screen lock but when I press the lock to enter the pattern, the screen turns off and on, screen lock appearing again (and I have to press the lock again to enter the pattern). If after unlocking the SIM card I wait a bit, I see the screen lock and again black screen and screen lock, so it s not my tap causing it. Doing like this (waiting a bit for the phone to show the screen lock for 2nd time) is less annoying, but I wonder why this happen and I cannot unlock the screen directly in the first attempt. This is also pending for research, but low priority.
When the phone boots, I find the splash screens too bright (the Samsung Galaxy S III splash, and later the red Replicant one). I don t know if I can change that. I know that other people have created different Replicant splash screens, so maybe I can create one almost black and only the Replicant text in very dark grey. But this is obviously a workaround, not a fix. OTOH, it s an annoying thing just some seconds: when the unlock screen is shown, the phone shows the brightness level that I ve set (usually, the lowest one).
From time to time, I suffer soft reboots:
  1. the phone hangs for 2-3 seconds
  2. then the red Replicant splash screen is shown (the phone is not totally rebooted, because I don t see the Samsung Galaxy S III splash screen and and the SIM card unlock PIN is not requested)
  3. after unlocking the screen, I see a normal desktop (similar to what I see after rebooting the phone: no apps running, and no last used apps history. Time and date are ok, wireless or 3G starts correctly etc).
I ve tried to track the causes of these soft reboots, but I couldn t find anything specific. They are not frequent at all, and when I decide to launch CatLog to try to catch any hint, the phone works perfectly for hours or days :s
Replicant is currently using the fallback Android EGL implementation, which is incomplete. The missing features of this implementation cause multiple issues, which are described in #705. These are the ones that I experience (or I miss):
  • The phone comes with a video editor preinstalled: Movie Studio. I got excited about it, because I was jealous of the small built-in video editor that comes with Whatsapp, but I became sad because Movie Studio does not work
  • The camera does not record video.
  • When I long-press the central button of my phone to see the list of recent apps, I don t see their thumbnails (only the name, and their icons). This is quite unimportant for me, names and icons are enough.
  • The stock Gallery app does not work well: I cannot see thumbnails of the albums. This is not very important, because I installed Gallery.
  • I cannot use Firefox, Orfox and other derivative web browsers (I usually use the stock browser, and I installed Lightning too).
  • I cannot use barcode or QR scanners.
  • My son cannot play Shattered Pixel Dungeon (nor Pixel Dungeon). Fortunately he uses now my old Android 2.x devide for that.
I installed the non-free firmware to be able to use Wifi and tethering, GPS and some other things. This does not fix the graphics problems listed above.
New apps, and translations
Note: when I write about Android apps, I usually link to their pages in the F-Droid website. Here I talk about translations (contributions), so I ll link to their original website or souce code repos. But you can find all those apps in F-Droid too.
As I told before, I installed another gallery app called Gallery and submitted an update to it Spanish translation.
I installed Red Moon to reduce (even more) the screen brightness. At night it s a relieve. Maybe the brightness of the splash screen is not so much, and I perceive them annoying because I got accostumed to Red Moon! I contributed some strings to the Spanish translation.
I liked RadioDroid very much, and I translated the app to Spanish.
I translated Wifi Privacy Police, and I used for some time, but I became tired that it keeps asking all the time that I walk across my workplace (multiple buildings within the same Wifi network, but quite a lot access points ).
I keep on contributing to K-9 Mail to make it 100% translated to Spanish. Now with a modern Android I can move to the development branch (5.1xx releases), and just did it.
I submitted a Spanish translation to DAVDroid, although I m not using it yet (I have to see if my University s Owncloud instance allows to sync contacts and calendar).
I updated the Spanish translation of PassAndroid, although I don t use it yet (I tend to print my train/airplane tickets ). I keep it installed in my phone, just in case.
Other apps that I use
I m testing OwnCloud, NextCloud and NexCloud Beta clients with my University s Owncloud and with Davros in my Sandstorm box (with Davros, I could only make it work installing an old version of Owncloud/Nexcloud client, and then upgrading. See #65).
I didn t get accostumed to Conversations. Not sure why, though. Maybe it s just that I got accostumed to Xabber-Classic, so I upgraded to Xabber. It works like a charm, dark theme, and I can close it easily when I don t want to chat.
I got in love with KDE Connect. Later I realized that I could have been using it in my Android 2.x phone since long
Sometimes I have fun activating Voice Notification and entering the redeslibres XMPP multi user chat at salas.mijabber.es, for example while I m cooking in the kitchen (in that room people talk in Spanish and make many wordplays, mixing Spanish and English, and use tech slang, etc so it s really fun to hear the Spanish-TTS deal with the conversation there!).
More to come
As I told at the beginning of this long post, my plan is to keep on tinkering with the phone, testing and translating apps, and becoming more involved. So expect some more posts about Android in this blog, in the future.
For now, some big things in my TODO:
  • Watching again some videos: DebConf16 videos about Android tools in Debian, FOSDEM talks about Replicant, and some other talks about free software in Android.
  • I track the #fdroid and #fdroid-dev channels in IRC, but I m not very talkative there. I guess I could do more user support.
  • Participate more in the F-Droid (client, server, data) issue trackers (I send reports when the alpha version crashes, and comment on few issues, but I don t triage the issue tracker to find issues that I could reproduce or help to diagnose or contribute to fix).
  • Long time ago I learned to setup an Android development environment and build apps. I would like to re-learn and maybe do some small fixes in unmaintained or near unmaintained apps, and maybe adopt them or join their development teams (I m thinking, for example, in Puma, an Android client for pump.io network, the MediaGoblin app, or the DebianDroid app). And ship new versions of unmaintained apss, including Spanish translations.
We ll see how far I can go!
Comments?
You can comment about this post in this pump.io thread.

Filed under: My experiences and opinion, Tools Tagged: Android, Debian, English, F-Droid

Nathan Handler: Ohio Linux Fest

This weekend, I traveled to Columbus, Ohio to attend Ohio Linux Fest. I departed San Francisco early on Thursday. It was interesting getting to experience the luxurious side of flying as I enjoyed a mimosa in the American Express Centurion lounge for the first time. I even happend to cross paths with Corey Quinn, who was on his way to [DevOpsDays Boise]. While connecting in Houston, I met up with the always awesome Jos Antonio Rey, who was to be my travel companion for this trip. The long day of travel took its toll on us, so we had a lazy Friday morning before checking in for the conference around lunch time. I was not that interested in the afternoon sessions, so I spent the majority of the first day helping out at the Ubuntu booth and catching up with friends and colleagues. The day ended with a nice Happy Hour sponsored by Oracle. Saturday was the main day for the conference. Ethan Galstad, Founder and CEO of Nagios, started the day with a Keynote about Becoming the Next Tech Entrepreneur. Next up was Elizabeth K. Joseph with A Tour of OpenStack Deployment Scenarios. While I ve read plenty about OpenStack, I ve never actually used it before. As a result, this demo and introduction was great to watch. It was entertaining to watch her login to CirrOS with the default password of cubswin:), as the Chicago Cubs are currently playing the San Francisco Giants in the National League Divisional Series (and winning). Unfortunately, I was not able to win a copy of her new Common OpenStack Deployments book, but it was great getting to watch her signing copies for other attendees after all of the hard work that went into writing the book. For lunch, Jos , Elizabeth, and Svetlana Belkin all gathered together for an informal Ubuntu lunch. Finally, it was time for me to give my talk. This was the same talk I gave at FOSSCON, but this time, I had a significantly larger audience. Practice definitely makes perfect, as my delivery was a lot better the second time giving this talk. Afterwards, I had a number of people come up to me to let me know that they really enjoyed the presentation. Pro Tip: If you ever attend a talk, the speaker will really appreciate any feedback you send their way. Even if it is a simple, Thank You , it really means a lot. One of the people who came up to me after the talk was Unit193. We have known each other through Ubuntu for years, but there has never been an opportunity to meet in person. I am proud to be able to say with 99% confidence that he is not a robot, and is in fact a real person. Next up was a lesson about the /proc filesystem. While I ve explored it a bit on my own before, I still learned a few tips and tricks about information that can be gained from the files in this magical directory. Following this was a talk about Leading When You re Not the Boss. It was even partially taught by a dummy (the speaker was a ventriloquist). The last regular talk of the day was one of the more interesting ones I attended. It was a talk by Patrick Shuff from Facebook about how they have built a load balancer than can handle a billion users. The slide deck was well-made with very clear diagrams. The speaker was also very knowledgeable and dealt with the plethora of questions he received. Prior to the closing keynote was a series of lightning talks. These served as a great means to get people laughing after a long day of talks. The closing keynote was given by father and daughter Joe and Lilly Born about The Democratization of Invention. Both of them had very interesting stories, and Lily was quite impressive given her age. We skipped the Nagios After Party in favor of a more casual pizza dinner. Overall, it was a great conference, and I am very glad to have had the opportunity to attend. A big thanks to Canonical and the Ubuntu Community for fudning my travel through the Ubuntu Community Fund and to the Ohio Linux Fest staff for allowing me the opportunity to speak at such a great conference.

8 October 2016

Norbert Preining: Debian/TeX update October 2016: all of TeX Live and Biber 2.6

Finally a new update of many TeX related packages: all the texlive-* including the binary packages, and biber have been updated to the latest release. This upload was delayed by my travels around the world, as well as the necessity to package a new Perl module (libdatetime-calendar-julian-perl) as required by new Biber. Also, my new job leaves me only the weekends for packaging. Anyway, the packages are now uploaded and should appear soon on your friendly local server. texlive2016-debian There are several highlights: The binaries have been patched with several upstream fixes (tex4ht and XeTeX compatibility, as well as various Japanese TeX engine fixes), updated biber and biblatex, and as usual loads of new and updated packages. Last but not least I want to thank one particular author: His package was removed from TeX Live due to the addition of a rather unusual clause in the license. Instead of simply uploading new packages to Debian with the rather important removed, I contacted the author and asked for clarification. And to my great pleasure he immediately answered with an update of the package with fixed license. All of us user of these many packages should be grateful to the authors of the packages who invest loads of their free time into supporting our community. Thanks! Enough now, here as usual the list of new and updated packages with links to their respective CTAN pages. Enjoy. New packages addfont, apalike-german, autoaligne, baekmuk, beamerswitch, beamertheme-cuerna, beuron, biblatex-claves, biolett-bst, cooking-units, cstypo, emf, eulerpx, filecontentsdef, frederika2016, grant, latexgit, listofitems, overlays, phonenumbers, pst-arrow, quicktype, revquantum, richtext, semantic-markup, spalign, texproposal, tikz-page, unfonts-core, unfonts-extra, uspace. Updated packages achemso, acmart, acro, adobemapping, alegreya, allrunes, animate, arabluatex, archaeologie, asymptote, attachfile, babel-greek, bangorcsthesis, beebe, biblatex, biblatex-anonymous, biblatex-apa, biblatex-bookinother, biblatex-chem, biblatex-fiwi, biblatex-gost, biblatex-ieee, biblatex-manuscripts-philology, biblatex-morenames, biblatex-nature, biblatex-opcit-booktitle, biblatex-phys, biblatex-realauthor, biblatex-science, biblatex-true-citepages-omit, bibleref, bidi, chemformula, circuitikz, cochineal, colorspace, comment, covington, cquthesis, ctex, drawmatrix, ejpecp, erewhon, etoc, exsheets, fancyhdr, fei, fithesis, footnotehyper, fvextra, geschichtsfrkl, gnuplottex, gost, gregoriotex, hausarbeit-jura, ijsra, ipaex, jfontmaps, jsclasses, jslectureplanner, latexdiff, leadsheets, libertinust1math, luatexja, markdown, mcf2graph, minutes, multirow, mynsfc, nameauth, newpx, newtxsf, notespages, optidef, pas-cours, platex, prftree, pst-bezier, pst-circ, pst-eucl, pst-optic, pstricks, pstricks-add, refenums, reledmac, rsc, shdoc, siunitx, stackengine, tabstackengine, tagpair, tetex, texlive-es, texlive-scripts, ticket, translation-biblatex-de, tudscr, turabian-formatting, updmap-map, uplatex, xebaposter, xecjk, xepersian, xpinyin. Enjoy.

7 October 2016

Dirk Eddelbuettel: tint 0.0.2: Tint Is Not Tufte

The tint package is now on CRAN. Its name stands for Tint Is Not Tufte and it offers a fresh take on the excellent Tufte-style html (and now also pdf) presentations. As a little teaser, here is what the html variant looks like:
and the full underlying document is available too. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

6 October 2016

Nathan Handler: FOSSCON

This post is long past due, but I figured it is better late than never. At the start of the year, I set a goal to get more involved with attending and speaking at conferences. Through work, I was able to attend the Southern California Linux Expo (SCALE) in Pasadena, CA in January. I also got to give a talk at O'Relly's Open Source Convention (OSCON) in Austin, TX in May. However, I really wanted to give a talk about my experience contributing in the Ubuntu community. Jos Antonio Rey encouraged me to submit the talk to FOSSCON. While I've been aware of FOSSCON for years thanks to my involvement with the freenode IRC network (which has had a reference to FOSSCON in the /motd for years), I had never actually attended it before. I also wasn't quite sure how I would handle traveling from San Francisco, CA to Philadelphia, PA. Regardless, I decided to go ahead and apply. Fast forward a few weeks, and imagine my surprise when I woke up to an email saying that my talk proposal was accepted. People were actually interested in me and what I had to say. I immediately began researching flights. While they weren't crazy expensive, they were still more money than I was comfortable spending. Luckily, Jos had a solution to this problem as well; he suggested applying for funding through the Ubuntu Community Donations fund. While I've been an Ubuntu Member for over 8 years, I've never used this resource before. However, I was happy when I received a very quick approval. The conference itself was smaller than I was expecting. However, it was packed with lots of friendly and familiar faces of people I've interacted with online and in person over the years at various Open Source events. I started off the day by learning from Jos how to use Juju to quickly setup applications in the cloud. While Juju has definitely come a long way over the last couple of years, and it appears t be quite easy to learn and use, it still appears to be lacking some of the features needed to take full control over how the underlying applications interact with each other. However, I look forward to continuing to watch it grow and mature. Net up, we had a lunch break. There was no catered lunch at this conference, so we decided to get some cheesesteak at Abner's (is any trip to Philadelphia complete without cheesesteak?). Following lunch, I took some time to make a few last minute changes to my presentation and rehearse a bit. Finally, it was time. I got up in front of the audience and gave my presentation. Overall, I was quite pleased. It was not perfect, but for the first time giving the talk, I thought it went pretty well. I will work hard to make it even better for next tme. Following my talk was a series of brief lightning talks prior to the closing keynote. Another long time friend of mine, Elizabeth Krumbach Joseph, was giving the keynote about listening to the needs of your global open source community. While I have seen her speak on several other occassions, I really enjoyed this particular talk. It was full of great examples and anecdotes that were easy for the audience to relate to and start applying to their own communities. After the conference, a few of us went off and played tourist, paying the Liberty Bell a visit before concluding our trip in Philadelpha. Overall, I had a great time as FOSSCON. It was great being re-united with so many friends. A big thank you to Jos for his constant support and encouragement and to Canonical and the Ubuntu Community for helping to make it possible for me to attend this conference. Finally, thanks to the terrific FOSSCON staff for volunteering so much time to put on this great event.

Reproducible builds folks: Reproducible Builds: week 75 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday September 25 and Saturday October 1 2016: Statistics For the first time, we reached 91% reproducible packages in our testing framework on testing/amd64 using a determistic build path. (This is what we recommend to make packages in Stretch reproducible.) For unstable/amd64, where we additionally test for reproducibility across different build paths we are at almost 76% again. IRC meetings We have a poll to set a time for a new regular IRC meeting. If you would like to attend, please input your available times and we will try to accommodate for you. There was a trial IRC meeting on Friday, 2016-09-31 1800 UTC. Unfortunately, we did not activate meetbot. Despite this participants consider the meeting a success as several topics where discussed (eg changes to IRC notifications of tests.r-b.o) and the meeting stayed within one our length. Upcoming events Reproduce and Verify Filesystems - Vincent Batts, Red Hat - Berlin (Germany), 5th October, 14:30 - 15:20 @ LinuxCon + ContainerCon Europe 2016. From Reproducible Debian builds to Reproducible OpenWrt, LEDE & coreboot - Holger "h01ger" Levsen and Alexander "lynxis" Couzens - Berlin (Germany), 13th October, 11:00 - 11:25 @ OpenWrt Summit 2016. Introduction to Reproducible Builds - Vagrant Cascadian will be presenting at the SeaGL.org Conference In Seattle (USA), November 11th-12th, 2016. Previous events GHC Determinism - Bartosz Nitka, Facebook - Nara (Japan), 24th September, ICPF 2016. Toolchain development and fixes Michael Meskes uploaded bsdmainutils/9.0.11 to unstable with a fix for #830259 based on Reiner Herrmann's patch. This fixed locale_dependent_symbol_order_by_lorder issue in the affected packages (freebsd-libs, mmh). devscripts/2.16.8 was uploaded to unstable. It includes a debrepro script by Antonio Terceiro which is similar in purpose to reprotest but more lightweight; specific to Debian packages and without support for virtual servers or configurable variations. Packages reviewed and fixed, and bugs filed The following updated packages have become reproducible in our testing framework after being fixed: The following updated packages appear to be reproducible now for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.) Some uploads have addressed some reproducibility issues, but not all of them: Patches submitted that have not made their way to the archive yet: Reviews of unreproducible packages 77 package reviews have been added, 178 have been updated and 80 have been removed in this week, adding to our knowledge about identified issues. 6 issue types have been updated: Weekly QA work As part of reproducibility testing, FTBFS bugs have been detected and reported by: diffoscope development A new version of diffoscope 61 was uploaded to unstable by Chris Lamb. It included contributions from: Post-release there were further contributions from: reprotest development A new version of reprotest 0.3.2 was uploaded to unstable by Ximin Luo. It included contributions from: Post-release there were further contributions from: tests.reproducible-builds.org Misc. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC.

5 October 2016

Gustavo Noronha Silva: Web Engines Hackfest 2016!

I had a great time last week and the web engines hackfest! It was the 7th web hackfest hosted by Igalia and the 7th hackfest I attended. I m almost a local Galician already. Brazilian Portuguese being so close to Galician certainly helps! Collabora co-sponsored the event and it was great that two colleagues of mine managed to join me in attendance. It had great talks that will eventually end up in videos uploaded to the web site. We were amazed at the progress being made to Servo, including some performance results that blew our minds. We also discussed the next steps for WebKitGTK+, WebKit for Wayland (or WPE), our own Clutter wrapper to WebKitGTK+ which is used for the Apertis project, and much more.
Zan giving his talk on WPE (former WebKitForWayland)Zan giving his talk on WPE (former WebKitForWayland)
One thing that drew my attention was how many Dell laptops there were. Many collaborans (myself included) and igalians are now using Dells, it seems. Sure, there were thinkpads and macbooks, but there was plenty of inspirons and xpses as well. It s interesting how the brand make up shifted over the years since 2009, when the hackfest could easily be mistaken with a thinkpad shop. Back to the actual hackfest: with the recent release of Gnome 3.22 (and Fedora 25 nearing release), my main focus was on dealing with some regressions suffered by users experienced after a change that made putting the final rendering composited by the nested Wayland compositor we have inside WebKitGTK+ to the GTK+ widget so it is shown on the screen. One of the main problems people reported was applications that use WebKitGTK+ not showing anything where the content was supposed to appear. It turns out the problem was caused by GTK+ not being able to create a GL context. If the system was simply not able to use GL there would be no problem: WebKit would then just disable accelerated compositing and things would work, albeit slower. The problem was WebKit being able to use an older GL version than the minimum required by GTK+. We fixed it by testing that GTK+ is able to create GL contexts before using the fast path, falling back to the slow glReadPixels codepath if not. This way we keep accelerated compositing working inside WebKit, which gives us nice 3D transforms and less repainting, but take the performance hit in the final blit .
Introducing "WebKitClutterGTK+"Introducing WebKitClutterGTK+
Another issue we hit was GTK+ not properly updating its knowledge of the window s opaque region when painting a frame with GL, which led to some really interesting issues like a shadow appearing when you tried to shrink the window. There was also an issue where the window would not use all of the screen when fullscreen which was likely related. Both were fixed. Andr Magalh es also worked on a couple of patches we wrote for customer projects and are now pushing upstream. One enables the use of more than one frontend to connect to a remote web inspector server at once. This can be used to, for instance, show the regular web inspector on a browser window and also use IDE integration for setting breakpoints and so on. The other patch was cooked by Philip Withnall and helped us deal with some performance bottlenecks we were hitting. It improves the performance of painting scroll bars. WebKitGTK+ does its own painting of scrollbars (we do not use the GTK+ widgets for various reasons). It turns out painting scrollbars can be quite a hit when the page is being scrolled fast, if not done efficiently. Emanuele Aina had a great time learning more about meson to figure out a build issue we had when a more recent GStreamer was added to our jhbuild environment. He came out of the experience rather sane, which makes me think meson might indeed be much better than autotools.
Igalia 15 years cakeIgalia 15 years cake
It was a great hackfest, great seeing everyone face to face. We were happy to celebrate Igalia s 15 years with them. Hope to see everyone again next year =)

Dirk Eddelbuettel: RcppGSL 0.3.1

A new version of RcppGSL in now on CRAN and in Debian. The RcppGSL package provides an interface from R to the GNU GSL using our Rcpp package. This release is one that entirely focused on maintenance. CRAN asked us to change on aspect relative to vignettes, and we added a few more small updates related entirely to packaging. In other words, no user-facing changes. The NEWS file entries follows below:

Changes in version 0.3.1 (2016-10-02)
  • The unit test driver was updated and simplified, (by request of CRAN) no longer leaves files in '/tmp', and removes two unexported (and unused) test helper functions (PR #10)
  • Switched to run.sh for Travis (PR #11)
  • Use canonical CRAN URLs in README.md
  • Restored 'boxed' display of code in vignette (PR #12)

Courtesy of CRANberries, a summary of changes to the most recent release is available. More information is on the RcppGSL page. Questions, comments etc should go to the issue tickets at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Kees Cook: security things in Linux v4.8

Previously: v4.7. Here are a bunch of security things I m excited about in Linux v4.8: SLUB freelist ASLR Thomas Garnier continued his freelist randomization work by adding SLUB support. x86_64 KASLR text base offset physical/virtual decoupling On x86_64, to implement the KASLR text base offset, the physical memory location of the kernel was randomized, which resulted in the virtual address being offset as well. Due to how the kernel s -2GB addressing works (gcc s -mcmodel=kernel ), it wasn t possible to randomize the physical location beyond the 2GB limit, leaving any additional physical memory unused as a randomization target. In order to decouple the physical and virtual location of the kernel (to make physical address exposures less valuable to attackers), the physical location of the kernel needed to be randomized separately from the virtual location. This required a lot of work for handling very large addresses spanning terabytes of address space. Yinghai Lu, Baoquan He, and I landed a series of patches that ultimately did this (and in the process fixed some other bugs too). This expands the physical offset entropy to roughly $physical_memory_size_of_system / 2MB bits. x86_64 KASLR memory base offset Thomas Garnier rolled out KASLR to the kernel s various statically located memory ranges, randomizing their locations with CONFIG_RANDOMIZE_MEMORY. One of the more notable things randomized is the physical memory mapping, which is a known target for attacks. Also randomized is the vmalloc area, which makes attacks against targets vmalloced during boot (which tend to always end up in the same location on a given system) are now harder to locate. (The vmemmap region randomization accidentally missed the v4.8 window and will appear in v4.9.) x86_64 KASLR with hibernation Rafael Wysocki (with Thomas Garnier, Borislav Petkov, Yinghai Lu, Logan Gunthorpe, and myself) worked on a number of fixes to hibernation code that, even without KASLR, were coincidentally exposed by the earlier W^X fix. With that original problem fixed, then memory KASLR exposed more problems. I m very grateful everyone was able to help out fixing these, especially Rafael and Thomas. It s a hard place to debug. The bottom line, now, is that hibernation and KASLR are no longer mutually exclusive. gcc plugin infrastructure Emese Revfy ported the PaX/Grsecurity gcc plugin infrastructure to upstream. If you want to perform compiler-based magic on kernel builds, now it s much easier with CONFIG_GCC_PLUGINS! The plugins live in scripts/gcc-plugins/. Current plugins are a short example called Cyclic Complexity which just emits the complexity of functions as they re compiled, and Sanitizer Coverage which provides the same functionality as gcc s recent -fsanitize-coverage=trace-pc but back through gcc 4.5. Another notable detail about this work is that it was the first Linux kernel security work funded by Linux Foundation s Core Infrastructure Initiative. I m looking forward to more plugins! If you re on Debian or Ubuntu, the required gcc plugin headers are available via the gcc-$N-plugin-dev package (and similarly for all cross-compiler packages). hardened usercopy Along with work from Rik van Riel, Laura Abbott, Casey Schaufler, and many other folks doing testing on the KSPP mailing list, I ported part of PAX_USERCOPY (the basic runtime bounds checking) to upstream as CONFIG_HARDENED_USERCOPY. One of the interface boundaries between the kernel and user-space are the copy_to_user()/copy_from_user() family of functions. Frequently, the size of a copy is known at compile-time ( built-in constant ), so there s not much benefit in checking those sizes (hardened usercopy avoids these cases). In the case of dynamic sizes, hardened usercopy checks for 3 areas of memory: slab allocations, stack allocations, and kernel text. Direct kernel text copying is simply disallowed. Stack copying is allowed as long as it is entirely contained by the current stack memory range (and on x86, only if it does not include the saved stack frame and instruction pointers). For slab allocations (e.g. those allocated through kmem_cache_alloc() and the kmalloc()-family of functions), the copy size is compared against the size of the object being copied. For example, if copy_from_user() is writing to a structure that was allocated as size 64, but the copy gets tricked into trying to write 65 bytes, hardened usercopy will catch it and kill the process. For testing hardened usercopy, lkdtm gained several new tests: USERCOPY_HEAP_SIZE_TO, USERCOPY_HEAP_SIZE_FROM, USERCOPY_STACK_FRAME_TO,
USERCOPY_STACK_FRAME_FROM, USERCOPY_STACK_BEYOND, and USERCOPY_KERNEL. Additionally, USERCOPY_HEAP_FLAG_TO and USERCOPY_HEAP_FLAG_FROM were added to test what will be coming next for hardened usercopy: flagging slab memory as safe for copy to/from user-space , effectively whitelisting certainly slab caches, as done by PAX_USERCOPY. This further reduces the scope of what s allowed to be copied to/from, since most kernel memory is not intended to ever be exposed to user-space. Adding this logic will require some reorganization of usercopy code to add some new APIs, as PAX_USERCOPY s approach to handling special-cases is to add bounce-copies (copy from slab to stack, then copy to userspace) as needed, which is unlikely to be acceptable upstream. seccomp reordered after ptrace By its original design, seccomp filtering happened before ptrace so that seccomp-based ptracers (i.e. SECCOMP_RET_TRACE) could explicitly bypass seccomp filtering and force a desired syscall. Nothing actually used this feature, and as it turns out, it s not compatible with process launchers that install seccomp filters (e.g. systemd, lxc) since as long as the ptrace and fork syscalls are allowed (and fork is needed for any sensible container environment), a process could spawn a tracer to help bypass a filter by injecting syscalls. After Andy Lutomirski convinced me that ordering ptrace first does not change the attack surface of a running process (unless all syscalls are blacklisted, the entire ptrace attack surface will always be exposed), I rearranged things. Now there is no (expected) way to bypass seccomp filters, and containers with seccomp filters can allow ptrace again. That s it for v4.8! The merge window is open for v4.9

2016, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.
Creative Commons License

4 October 2016

Rapha&#235;l Hertzog: My Free Software Activities in September 2016

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS With the increasing number of paid contributors, easy fixes (CVE with patches available) tend to be processed rather quickly. All the package I worked on had issues that were open for a long time because they were hard to handle. I prepared DLA-613-1 fixing 3 CVE on roundcube. The fix required to manually backport the CRSF handling code which was not available in the wheezy version. I spent almost 8 hours on roundcube. Then I started to work on tiff3. I reviewed many CVE: CVE-2016-3658, CVE-2015-7313, CVE-2015-7554, CVE-2015-8668, CVE-2016-5318, CVE-2016-3625, CVE-2016-5319. I updated their status for tiff3 in wheezy, requested reproducer files to people who reported the CVE when the files were not publicly available and made sure that everything was recorded in the upstream bug tracker. The 4.25 hours I spent on the package were not enough to work on patches, so I put the package back in the work queue. GNOME 3.22 transition I uploaded a new gnome-shell-timer that would work with GNOME 3.21 that had been uploaded to sid. Unfortunately, that new GNOME (and GTK+) version caused many regressions that affected Debian Testing (and thus Kali) users in particular in gnome-control-center. I uploaded a new version fixing some of those issues and I reported a bunch of them to upstream too (#771515, #771517, #771696). Kali I worked on #836211 creating a dpkg patch to work-around the overlayfs limitation (we use it in Kali because persistence of live system relies on overlayfs) and I contacted the upstream overlayfs maintainer to hopefully get a proper fix on the overlayfs side instead. I uploaded radcli 1.2.6-2.1 to fix RC bug #825121 as the package was removed from testing and openvas depends on it in Kali. As part of the pkg-security team, I sponsored/uploaded acccheck and arp-scan for Marcos Fouces, and p0f 3.09b as well. Misc Debian work Distro Tracker. I tested, fixed and merged Paul Wise s patch integrating multiarch hints into tracker.debian.org (#833623). Debian Handbook. I enabled the new Vietnamese translation on debian-handbook.info and updated all translations with Weblate updates. systemd units for apache2. I prepared systemd units for apache2 which I submitted in #798430. With approval of Stefan Fritsch, I committed my work to the git repository and then uploaded the result in version 2.4.23-5. Hindsight packaging. I first packaged lua-sandbox (#838969) which is a dependency of Hindsight and then Hindsight itself (#838968). In this process, I opened a couple of upstream tickets. PIE by default. I uploaded a new version of cpputest compiled with -fPIC so shat executable linking to its static library can be compiled with -fPIE (#837363, forwarded upstream here). Bugs filed. Bad homepage link in haskell-dice-entropy-conduit. Inconsistent options --onlyscripts and --noscripts in debhelper. pidgin entry in security-support-limited is out of date in debian-security-support. New upstream version (2.0.2) in puppet-lint. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

3 October 2016

Markus Koschany: My Free Software Activities in September 2016

Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you re interested in Android, Java, Games and LTS topics, this might be interesting for you. Debian Android Debian Games Debian Java Debian LTS This was my eight month as a paid contributor and I have been paid to work 12,25 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following: Non-maintainer uploads Misc

Matthew Garrett: The importance of paying attention in building community trust

Trust is important in any kind of interpersonal relationship. It's inevitable that there will be cases where something you do will irritate or upset others, even if only to a small degree. Handling small cases well helps build trust that you will do the right thing in more significant cases, whereas ignoring things that seem fairly insignificant (or saying that you'll do something about them and then failing to do so) suggests that you'll also fail when there's a major problem. Getting the small details right is a major part of creating the impression that you'll deal with significant challenges in a responsible and considerate way.

This isn't limited to individual relationships. Something that distinguishes good customer service from bad customer service is getting the details right. There are many industries where significant failures happen infrequently, but minor ones happen a lot. Would you prefer to give your business to a company that handles those small details well (even if they're not overly annoying) or one that just tells you to deal with them?

And the same is true of software communities. A strong and considerate response to minor bug reports makes it more likely that users will be patient with you when dealing with significant ones. Handling small patch contributions quickly makes it more likely that a submitter will be willing to do the work of making more significant contributions. These things are well understood, and most successful projects have actively worked to reduce barriers to entry and to be responsive to user requests in order to encourage participation and foster a feeling that they care.

But what's often ignored is that this applies to other aspects of communities as well. Failing to use inclusive language may not seem like a big thing in itself, but it leaves people with the feeling that you're less likely to do anything about more egregious exclusionary behaviour. Allowing a baseline level of sexist humour gives the impression that you won't act if there are blatant displays of misogyny. The more examples of these "insignificant" issues people see, the more likely they are to choose to spend their time somewhere else, somewhere they can have faith that major issues will be handled appropriately.

There's a more insidious aspect to this. Sometimes we can believe that we are handling minor issues appropriately, that we're acting in a way that handles people's concerns, while actually failing to do so. If someone raises a concern about an aspect of the community, it's important to discuss solutions with them. Putting effort into "solving" a problem without ensuring that the solution has the desired outcome is not only a waste of time, it alienates those affected even more - they're now not only left with the feeling that they can't trust you to respond appropriately, but that you will actively ignore their feelings in the process.

It's not always possible to satisfy everybody's concerns. Sometimes you'll be left in situations where you have conflicting requests. In that case the best thing you can do is to explain the conflict and why you've made the choice you have, and demonstrate that you took this issue seriously rather than ignoring it. Depending on the issue, you may still alienate some number of participants, but it'll be fewer than if you just pretend that it's not actually a problem.

One warning, though: while building trust in this way enhances people's willingness to join your community, it also builds expectations. If a significant issue does arise, and if you fail to handle it well, you'll burn a lot of that trust in the process. The fact that you've built that trust in the first place may be what saves your community from disintegrating completely, but people will feel even more betrayed if you don't actively work to rebuild it. And if there's a pattern of mishandling major problems, no amount of getting the details right will matter.

Communities that ignore these issues are, long term, likely to end up weaker than communities that pay attention to them. Making sure you get this right in the first place, and setting expectations that you will pay attention to your contributors, is a vital part of building a meaningful relationship between your community and its members.

comment count unavailable comments

Kees Cook: security things in Linux v4.7

Previously: v4.6. Onward to security things I found interesting in Linux v4.7: KASLR text base offset for MIPS Matt Redfearn added text base address KASLR to MIPS, similar to what s available on x86 and arm64. As done with x86, MIPS attempts to gather entropy from various build-time, run-time, and CPU locations in an effort to find reasonable sources during early-boot. MIPS doesn t yet have anything as strong as x86 s RDRAND (though most have an instruction counter like x86 s RDTSC), but it does have the benefit of being able to use Device Tree (i.e. the /chosen/kaslr-seed property) like arm64 does. By my understanding, even without Device Tree, MIPS KASLR entropy should be as strong as pre-RDRAND x86 entropy, which is more than sufficient for what is, similar to x86, not a huge KASLR range anyway: default 8 bits (a span of 16MB with 64KB alignment), though CONFIG_RANDOMIZE_BASE_MAX_OFFSET can be tuned to the device s memory, giving a maximum of 11 bits on 32-bit, and 15 bits on EVA or 64-bit. SLAB freelist ASLR Thomas Garnier added CONFIG_SLAB_FREELIST_RANDOM to make slab allocation layouts less deterministic with a per-boot randomized freelist order. This raises the bar for successful kernel slab attacks. Attackers will need to either find additional bugs to help leak slab layout information or will need to perform more complex grooming during an attack. Thomas wrote a post describing the feature in more detail here: Randomizing the Linux kernel heap freelists. (SLAB is done in v4.7, and SLUB in v4.8.) eBPF JIT constant blinding Daniel Borkmann implemented constant blinding in the eBPF JIT subsystem. With strong kernel memory protections (CONFIG_DEBUG_RODATA) in place, and with the segregation of user-space memory execution from kernel (i.e SMEP, PXN, CONFIG_CPU_SW_DOMAIN_PAN), having a place where user-space can inject content into an executable area of kernel memory becomes very high-value to an attacker. The eBPF JIT was exactly such a thing: the use of BPF constants could result in the JIT producing instruction flows that could include attacker-controlled instructions (e.g. by directing execution into the middle of an instruction with a constant that would be interpreted as a native instruction). The eBPF JIT already uses a number of other defensive tricks (e.g. random starting position), but this added randomized blinding to any BPF constants, which makes building a malicious execution path in the eBPF JIT memory much more difficult (and helps block attempts at JIT spraying to bypass other protections). Elena Reshetova updated a 2012 proof-of-concept attack to succeed against modern kernels to help provide a working example of what needed fixing in the JIT. This serves as a thorough regression test for the protection. The cBPF JITs that exist in ARM, MIPS, PowerPC, and Sparc still need to be updated to eBPF, but when they do, they ll gain all these protections immediatley. Bottom line is that if you enable the (disabled-by-default) bpf_jit_enable sysctl, be sure to set the bpf_jit_harden sysctl to 2 (to perform blinding even for root). fix brk ASLR weakness on arm64 compat There have been a few ASLR fixes recently (e.g. ET_DYN, x86 32-bit unlimited stack), and while reviewing some suggested fixes to arm64 brk ASLR code from Jon Medhurst, I noticed that arm64 s brk ASLR entropy was slightly too low (less than 1 bit) for 64-bit and noticeably lower (by 2 bits) for 32-bit compat processes when compared to native 32-bit arm. I simplified the code by using literals for the entropy. Maybe we can add a sysctl some day to control brk ASLR entropy like was done for mmap ASLR entropy. LoadPin LSM LSM stacking is well-defined since v4.2, so I finally upstreamed a small LSM that implements a protection I wrote for Chrome OS several years back. On systems with a static root of trust that extends to the filesystem level (e.g. Chrome OS s coreboot+depthcharge boot firmware chaining to dm-verity, or a system booting from read-only media), it s redundant to sign kernel modules (you ve already got the modules on read-only media: they can t change). The kernel just needs to know they re all coming from the correct location. (And this solves loading known-good firmware too, since there is no convention for signed firmware in the kernel yet.) LoadPin requires that all modules, firmware, etc come from the same mount (and assumes that the first loaded file defines which mount is correct , hence load pinning ). That s it for v4.7. Prepare yourself for v4.8 next!

2016, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.
Creative Commons License

Next.

Previous.