Search Results: "spa"

5 September 2016

Dirk Eddelbuettel: Rcpp 0.12.7: More updates

The seventh update in the 0.12.* series of Rcpp just arrived on the CRAN network for GNU R as well as in Debian. This 0.12.7 release follows the 0.12.0 release from late July, the 0.12.1 release in September, the 0.12.2 release in November, the 0.12.3 release in January, the 0.12.4 release in March, the 0.12.5 release in May, and the 0.12.6 release in July --- making it the eleventh release at the steady bi-montly release frequency. Keeping with the established pattern, this is again more of a maintenance release which addresses small bugs, nuisances or documentation issues without adding any major new features. One issue that got to a few people was our casual use of NORET in the definition of Rcpp::stop(). We had (ahem) overlooked that NORET is only defined by R 3.2.0 or later, and several folks trying to build on older releases of R (why?) got bitten. Well, at least we have a new record for most frequently reported bug ... Kidding aside, this is now fixed. Rcpp has become the most popular way of enhancing GNU R with C or C++ code. As of today, 759 packages on CRAN depend on Rcpp for making analytical code go faster and further. That is up by well over fifty packages since the last release in mid-July! We are once again fortunate to have a number of pull request, from first-timers to regulars. James "coatless" Balamuta in particular relentlessly pushed for better documentation and cleanup of numerous dangling issue tickets. Artem Klevtsov also contributed again. Qiang, Kevin and I also got some changes in for the Rcpp Core team. More details are again below.
Changes in Rcpp version 0.12.7 (2016-09-04)
  • Changes in Rcpp API:
    • The NORET macro is now defined if it was not already defined by R itself (Kevin fixing issue #512).
    • Environment functions get() & find() now accept a Symbol (James Balamuta in #513 addressing issue #326).
    • Several uses of Rf_eval were replaced by the preferred Rcpp::Rcpp_eval (Qiang in PR #523 closing #498).
    • Improved Autogeneration Warning for RcppExports (James Balamuta in #528 addressing issue #526).
    • Fixed invalid C++ prefix identifiers in auto-generated code (James Balamuta in #528 and #531 addressing issue #387; Simon Dirmeier in #548).
    • String constructors now set default UTF-8 encoding (Qiang Kou in #529 fixing #263).
    • Add variadic variants of the RCPP_RETURN_VECTOR and RCPP_RETURN_MATRIX macro when C++11 compiler used (Artem Klevtsov in #537 fixing #38).
  • Changes in Rcpp build system
    • Travis CI is now driven via run.sh from our fork, and deploys all packages as .deb binaries using our PPA where needed (Dirk in #540 addressing issue #517).
  • Changes in Rcpp unit tests
    • New unit tests for random number generators the R namespace which call the standalone Rmath library. (James Balamuta in #514 addressing issue #28).
  • Changes in Rcpp Examples:
    • Examples that used cxxfunction() from the inline package have been rewritten to use either sourceCpp() or cppFunction() (James Balamuta in #541, #535, #534, and #532 addressing issue #56).
Thanks to CRANberries, you can also look at a diff to the previous release. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

4 September 2016

Carl Chenet: Retweet 0.9: Automatically retweet & like

Retweet 0.9, a self-hosted Python app to automatically retweet and like tweets from another user-defined Twitter account, was released this September, 2nd. Retweet 0.9 is already in production for Le Journal du hacker, a French-speaking Hacker News-like website, LinuxJobs.fr, a French-speaking job board and this very blog. fiesta What s the purpose of Retweet? Let s face it, it s more and more difficult to communicate about our projects. Even writing an awesome app is not enough any more. If you don t appear on a regular basis on social networks, everybody thinks you quit or that the project is stalled. But what if you already have built an audience on Twitter for, let s say, your personal account. Now you want to automatically retweet and like all tweets from the account of your new project, to push it forward. Sure, you can do it manually, like in the old good 90 s or you can use Retweet! Twitter Out Of The Browser Have a look at my Github account for my other Twitter automation tools: What about you? Do you use tools to automate the management of your Twitter account? Feel free to give me feedback in the comments below.

Iustin Pop: Usefulness of real-time data during cycling

Wow, a mouthful of a title, for a much simpler post. Up until earlier this year, I had only one sport GPS device, and that was a watch. Made sense, as my "main" sport was running (not that I did it very consistently). Upgraded over a few generations of Garmins, I mainly used it to record my track and statistics (pace, heart rate, etc.) The newest generation of watch and heart rate monitor even give more statistics (left-right leg balance, ground contact time, so on). Most of this data can be looked at while running, but only as an exception; after all, it's very hard to run with one hand up in front of your face. The other useful features guided workouts and alerts during normal runs I've used, but not consistently. So when I started biking a bit more seriously, I wondered whether it would make sense to buy a bike computer. The feature intersection between watch and bike GPS is quite large, so clearly this is a "want" not a "need" by far. Reading forum posts showed the same question repeated multiple times What convinced me to nevertheless buy such a bike GPS were the mapping/routing features. A bike GPS with good routing features, or even just maps on which tracks can be overlayed, can certainly ease the discovery of new routes. After a few months of use, my most useful feature is one that I didn't expect. While the mapping is useful and I do use it, the one that actually is significantly better than my watch is the large display with data fields that I can trivially check almost continuously on road biking, and during technically easy climbing sections for mountain biking. My setup looks like this: It's a 9-field setup; the Edge 1000 can go to 10, but I like "headline" field. The watch can only go to four, and is basically not usable during rides, unless one would use a quick-release kit for mounting on the handle bar. This setup allowed me to learn much better my physical capabilities, why I sometimes run out of energy, and how the terrain affects me. Random things that I learned: Seeing all this data only after the ride is less useful since you don't remember exactly how you felt during specific points in the ride. But as real-time data, one can easily see/feel the correlation between how the body feels and what data says. One thing that would be also useful would be real-time power data (3 sec average, not instantaneous) to correlate even better with the body state. I now use heart rate and cadence as a proxy for that, but being able to see actual power numbers would increase the usefulness of the data even more. Unfortunately, none of these makes the climbs easier. But at least it allows me to understand why one climb feels hard and another easy (relatively speaking). I wonder if, and how this could be applied to running; maybe with smart glasses? Conclusion: yes, I do recommend a bike computer with a large display (to be able to see many fields at once). Just in case one has disposable income at hand and doesn't know which hobby to spend it on ;-)

3 September 2016

Antonio Terceiro: testing build reprodubility with debrepro

Earlier today I was handling a reproducibility bug and decided I had to try a reproducibility test by myself. I tried reprotest, but I was being hit by a disorderfs issue and I was not sure whether the problem was with reprotest or not (at this point I cannot reproduce that anymore). So I decided to hack a simple script to that, and it works. I even included it in devscripts after writing a manpage. Of course reprotest is more complete, extensible, and supports arbitrary virtualization backends for doing the more dangerous/destructive variations (such as changing the hostname and other things that require root) but for quick tests debrepro does the job. Usage examples:

$ debrepro                                 # builds current directory
$ debrepro /path/to/sourcepackage          # builds package there
$ gbp-buildpackage --git-builder=debrepro  # can be used with vcs wrappers as well

debrepro will do two builds with a few variations between them, including $USER, $PATH, timezone, locale, umask, current time, and will even build under disorderfs if available. Build path variation is also performed because by definition the builds are done in different directories. If diffoscope is installed, it will be used for deep comparison of non-matching binaries. If you are interested and don t want to build devscripts from source or wait for the next release, you can just grab the script, save it as debrepro somewhere on your $PATH and make it executable.

Iustin Pop: Nationalpark Bike Marathon 2016

Nationalpark Bike Marathon 2016 report: gravity rules! "Marathon" is a bit of a misnomer, since I did the short route, not the medium nor the long one. But hey, it's the official name! Setup This race has four possible lengths: "Valader" which is full round-trip Scuol-Scuol (137km/4'026m), "Jauer" (Fuldera-Scuol, 103km/2'934m), "Livignasco" (Livigno-Scuol, 66km/1'871m) and "Put r" (S-chanf to Scuol, 47Km/1'051m). After many debates, I settled on the Livignasco route, as that was what I was reasonably confident to be able to do. The only problem was that this route (and all the longer ones, of course) were going over Pass Chaschauna, which is a quite hard climb. My bike shop person, who did the full length a number of times, just said when hearing the route I was planning: "Chaschauna is a bitch " Bad language, proper characterisation. The route choice also impacts the logistics: except for the full route, the question is where to sleep the night before? One can sleep either at the start place, or sleep in Scuol itself and take the official shuttle to the start place, but this means waking up much earlier. I decided to sleep in Livigno, as with a 7:45 start this would allow me to sleep until ~6:30, have a quick breakfast and be in time for the race. Problem #1 So on Friday (26th) I drove to Scuol, picked up my start number, and then drove over to Livigno and checked in to the hotel. Speaking of picking up the start number, I saw this at the start area: While picking up my start number: the race is so fast that an STI is needed as pacer?? An STI needed as pacer? How fast are these people, I wondered? Day of the race My plan to have a relaxing sleep until relatively late failed. I went to sleep with difficulties: late dinner due to my late arrival in Livigno, and then I was stressed enough about not missing the race that at 05:05 I already woke up, and was checking the clock every five minutes. Finally at 05:30 I gave up and got out of bed, with only about 6 hours of sleep. Problem #2. The other problem was that I na vely thought the hotel will have breakfast from 07:00 or even 07:30. Hah, as if this was Switzerland. The hotel had breakfast from 08:00, no reason to hurry, right? So I had a poor man's breakfast in the room, some energy bars and an ice tea. Problem #3. At least I had time to grab this picture from the hotel room: Good morning Livigno! Not bad. On to the race! But before that, remember to fill my hydration pack with 3L of water, and put it in my not-so-light backpack. Of course I had tools and some spare parts with me, what if something happens? Problem #4, over-planning. The race The setup at the start was pretty easy. Wait near the start until 07:45, then go. For me, not too in front, of course: Ready for the start! Pass Chaschauna About 3 kilometres of easy/flat road, and then it starts. Pass Chaschauna, killing your legs softly Already climbing  on foot! But not everybody. But at least the pictures were nice! The climb is hard, but the view is worth it! Still climbing So high already! Good legs :) Looking back: it's been a loong way up I can see the top!  and the remaining steep meters to climb :( The climb is difficult. It was around 22% for most of the time, and very few people were able to bike up. Not only the absolute meters were the problem, but also the fact that the ground was quite loose, and pushing the bike, or rather climbing up while pushing the bike, was difficult to do in bike shoes with cleats. Proper hiking shoes would have been much more adequate. Maybe this is why despite the effort, some people biked as much as possible? I was careful not to over-exert myself and walked almost all the way up. Hence the pictures But at last, reached the top of the pass, and was very happy to be done with it. The GPX file from my Garmin says it took only 4 minutes (1.2km) after the race start to actually start climbing (it was not obvious we were climbing so early), and then 1h:20m/8Km to get from 1'817m altitude to the highest point at 2'658m. 8Km in 1h:20m, faster than walking pace but quite slower than running pace (on flat terrain, of course). At the top of the pass I was quite happy: Finally at the top! Clipping in and Go! Smilling and happy! And then the descent started: I wonder how steep it will be going down This was an easy section :/ All the way leading to the race I kept thinking only about climbing Chaschauna, and not about going down on it. It was quite an experience, which I won't forget soon. The trail was already well travelled, which means that besides the loose ground in form of large clumps of dried earth, there was a layer of fine, somewhat moist earth, about 3-5 centimetres deep, which made the downhill "interesting". Coupled with what was a much steeper trail than going up I saw most of the time ~33% it's no surprise that many, or even most, of the people were walking their bikes down. The GPX track says that the trail goes 319m down in 1Km (so a 31.9% over one entire kilometre), then another 130m over the next 1.4Km (9.28%), and then (not sure if this is the pass per se or after) another longer segment of ~260m over 3Km (8.56%). The first steep segment needs lots of skill and concentration, the others are normal gradients. But at least things go fast. What took uphill took ~40 minutes for the last 450m up, downhill it took only 15 minutes! Falling I was able, thanks to recent training, to bike down the trail, or at least I thought so. So on the steep segment I was struggling to bike down and keep control of my bike, putting one leg down on some sections but in general being "on the bike", being hit by the dirt I was throwing in the air despite the mud-guards, being careful but at the same time enjoying this difficult section. That is, right up to the point where I got, I think, too comfortable. After the second-to-last curve, the rear wheel slides, I lose balance and fall in the (thankfully soft) dirt a bit more forcefully than I should have, the bike also sliding along in the dirt. No damage to myself, just some trivial scratches; the bike seems a bit shaken (fork was turned 180 , full of dirt, etc.). I get again on the bike and I continue down; however, something seems off: the rear brake level doesn't have a return anymore, I have to push it to go back. Still usable, but strange. So I continue the other two segment of this descent, and over these ~4-5Km I feel the brake issue going worse and worse: it's actually hard to push the brake lever back, and even pressing it doesn't seem to have much effect. And then the realisation dawns on my: I have hydraulic brakes, and the worsening thing leads me to think my brake hose has been punctured (by what? I fell in loose dirt) and I'm losing the brake fluid with every press. Now, on flat road, biking with only the front brake is tricky but doable. On an MTB course, having only one brake is not a smart thing . Not easy, not smart, and definitely dangerous. It also felt very strange to not have both brakes, and then I realised how much I work with the brakes in unison. I knew that there are repair posts in the race, but I didn't know if they can fix such things (and doing a brake fluid refill how complicated is it, how much time does it take?). Anyway, I resolved to try and continue if the terrain allows until I reach the first repair post, and stop if it gets too steep. It would be pretty sad to have to abort the race after only 10Km, right? The terrain did cooperate: up and down, so I was able to bleed speed easily, and even though through forest, the visibility was good enough that I could plan ahead. It was strangely lonely at this time: I was going slowly and thus not reaching from behind anyone, but also not being reached by riders from behind. I did enjoy it a bit, the quietness of the forest, just interrupted by the squeaking of my front brake And then I finally see the sign for "Repair: 1Km". Yaaay! maybe I don't have to quit the race. And then, about 100m further on, another sign: "Danger! Steep descent". Uh-oh tantalisingly close Let's go slow. And yes, here I was overtaken by 3-4 other cyclists. This was the single moment in the race where I did have a moment of real fear: at one point, I was going down with the remaining brake as pressed as I could without compromising stability, and I was still gaining speed. Not smart, and as I was debating how to fall to stop my descent, the terrain started to get less steep, and I finally reached the repair station, at kilometre 21. Difficult repairs There was another person being serviced for a flat rear tire, so I wait in line and discuss with him my problem. Upon hearing I only had the front brake, he said: "Wow, how did you manage to descend this last segment?" "Well, slowly ". My turn came up, I explain my problem and my theory, and the repair guy gives me one look, the bike one look, but doesn't look at the brake hose. Instead, he looks once at the brake lever, sees that another cable guide on my handle bar was bent during the fall and actually was mechanically interfering with the brake lever, unbends that thing, and says "here you go". I was stunned. 15 seconds of looking at the problem without stress, and the problem was solved. I could have done this myself if I took the time to look and think, and not hurry. And I could have enjoyed all the downhill on the pass, with two functioning brakes. Some lessons have to be learned the hard way, and this was one. I got off easy though, as I didn't have a real accident while biking with only one brake. And they say you get wiser as you get older Anyway, brake repaired, I stopped at the Sanit r tent to have my scratches disinfected, and onwards. The following few kilometres until S-chanf I enjoyed very much. S-chanf And then, 2 hours 10 minutes into the race, I exit the forest trail, reach S-chanf, and start climbing and descending and climbing and descending. In the sun, which now (after 10 AM) is not that light anymore. The initial climb at S-chanf, while not much, took away my remaining short-term reserves. The route is 50m ascent, then down and then another 50m ascent, and after a bit of flat another 75m ascent I was getting tired, despite eating energy bars and drinking water (and juice at stops). This goes one for an entire hour, during which I cover about 23Km, so not a bad speed everything considered. But getting more and more tired. There were also some very nice segments, along the river, in the shade of the forest. Some fast downhill, some fast flat. I got overtaken by quite a few real racers, that were going so fast I couldn't keep up with them as in I didn't dare ride that fast even when I had the energy. Clearly they were from the longer segments, and boy were they fast! A number of couples (or pairs?) one woman one man, a number of bigger groups, all going as if they had E-bikes or super-powers, not just biking along tired like me Last climb And then, the last big climb: half an hour of going about 300m up (absolute values, so cumulative number would probably be higher), in the sun, over about 6Km, so only 5% average gradient. But I was tired and even resorted to walking on some portions At least I could take some pictures again: Much later, climbing again. Or have pictures taken: A bit of a smile left. Yes, that's a smile, not a grimace! Nice views: Look, nice castle! Stopping for a picture, and not because I was *dead tired* :) At one point, in a bit of a shade, I stop to catch my breath. Somebody else stopped as well and, seeing as I was checking my GPS, asks: I got overtaken by a lot of cyclists here, although many of them were from S-chanf route (based on the number colour). Sad The thing is, besides the climbing and the sun, this and the previous part was a very nice route: through small towns, high above the valley, beautiful landscapes, etc. Just enjoying it was hard, since I wasn't a) trained well enough and b) prepared well enough. Last segment, last downhill In any case, after that hard half hour, it's flat (or rather said, average gradient zero): Flat route, good pavement. Can catch my breath, and take a another picture. Good road, good views, but I'm on the last virtual bar of my battery, so the remaining 15Km I do in 40 minutes, also "eating" the last ascent meters still about 300 left, I think, but up/down/up/down, so you could reuse kinetic energy to gain potential one. The only question I had now was, the route was at this moment about 400m higher (in absolute altitude) than the finish point, so when do we lose that altitude? The answer was a very very nice answer: in the last 5Km, for an average gradient of 8%, just perfect. I was tired however, so I couldn't really enjoy or go fast here, but it was a good feeling. Finish! And then, entering Scuol, not really believing I'm near the finish, that I will be able to actually stop and relax. My finish picture doesn't do justice to how happy I was at that moment: At the finish, happy happy happy! However, after passing the finish line, it was a bit weird. I just stop now? What next? So weird to just walk, not push pedals, and not go up. Or down. The funny thing here was that I thought I didn't use much of my water (hydration pack) during the race. But soon after finishing, I drank the equivalent of about two glasses (let's say 0.5L) and it was over. So I did manage to drink 2.5L during the race, plus the sport drinks at the stops. Time for stats: my Garmin says 67.07Km, 4h:29m, average speed 15.0kph (hah), 1'615m elevation gain. Official numbers say 66Km, 1'871m elevation, and thus 14.22kph. Ranking-wise: Sigh, could have done better. I was aiming for somewhere around last 33%, not last 25% :-P First classed was ~66% faster than me Going to the train station was hard as in difficult to move my legs. Took one nice picture of Scuol though, so was able to think a bit: After the race: Scuol is not so bad And then the long way back to Livigno. Took me 4 h to bike to Scuol, and about 3 hours to go back via public transport. At least I ate a sandwich in the meantime. Then reached Livigno, which was also looking nice: Took me almost as long by public transport as by bike, but finally back in Livigno. Also nice! Then got on my bike, which at this point was magically light and easy to ride (it seems I was at least partially recovered), rode to the hotel, packed my stuff, and drove home. On the highway I had my first real meal for that day, but I was quite tired so didn't feel like the achievement it should have been Lessons The biggest takeaway from this race was the huge learning experience I got. The race was itself awesome, but the learning even more so. First, for this particular race, it's much better to sleep in Scuol and (if not doing the whole route) take the shuttle to the actual start. Yes, one has to wake up earlier, but you have assured breakfast via the organisers (++++!), and after the race you don't have to get even more tired by taking the train and the bus and Second, I need better logistics: I don't need to carry that heavy load (spare tube, pump, tools, jacket in case it's cold) when the race is well organised. The water in the hydration pack was useful, but carrying it up Chaschauna was a pain. Not sure what I'll do next; it's possible to survive only on water/drinks from the food posts (basically 2.5-2.8L is equivalent to only 4 750ml bottles, and there were I think about 6 stops?), but the hydration pack is so easy to use Keeping one's head cool and not stress about things also would help: had I looked at my brake lever in peace, I could have solved it myself, and not ride ~10Km with only one brake. Also, losing some weight (as in losing fat, not having a lighter bike) would definitely help. I could stand to lose 5Kg easily, probably 10Kg, and that would bring me more in line with other people in the race (at least looking at the pictures). And hey, my Watts/Kg would improve magically! And last but most important, I need better training. Training on how to use my energy, training longer rides so that my body gets used to it and doesn't bonk after 3 hours, training training training. Which I like, but the problem is time to do it Also, in case anybody wonders: yes, I do recommend this race! It was definitely fun, and the route itself is very nice, nicer than the pictures here show. Closing words: gravity rules. Up or down, it's hard and punishing. Still: Ride on!

31 August 2016

Chris Lamb: Free software activities in August 2016

Here is my monthly update covering what I have been doing in the free software world (previously):

Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most Linux distributions provide binary (or "compiled") packages to end users. The motivation behind the Reproducible Builds effort is to allow verification that no flaws have been introduced either maliciously and accidentally during this compilation process by promising identical binary packages are always generated from a given source.

Diffoscope diffoscope is our "diff on steroids" that will not only recursively unpack archives but will transform binary formats into human-readable forms in order to compare them:
  • Added a command-line interface to the try.diffoscope.org web service.
  • Added a JSON comparator.
  • In the HTML output, highlight lines when hovering to make it easier to visually track.
  • Ensure that we pass str types to our Difference class, otherwise we can't be sure we can render them later.
  • Testsuite improvements:
    • Generate test coverage reports.
    • Add tests for Haskell and GitIndex comparators.
    • Completely refactored all of the comparator tests, extracting out commonly-used routines.
    • Confirm rendering of text and HTML presenters when checking non-existing files.
    • Dropped a squashfs test as it was simply too unreliable and/or has too many requirements to satisfy.
  • A large number of miscellaneous cleanups, including:
    • Reworking the comparator setup/preference internals by dynamically importing classes via a single list.
    • Split exceptions out into dedicated diffoscope.exc module.
    • Tidying the PROVIDERS dict in diffoscope/__init__.py.
    • Use html.escape over xml.sax.saxutils.escape, cgi.escape, etc.
    • Removing hard-coding of manual page targets names in debian/rules.
    • Specify all string format arguments as logging function parameters, not using interpolation.
    • Tidying imports, correcting indentation levels and drop unnecessary whitespace.

disorderfs disorderfs is our FUSE filesystem that deliberately introduces nondeterminism in system calls such as readdir(3).
  • Added a testsuite to prevent regressions. (f124965)
  • Added a --sort-dirents=yes no option for forcing deterministic ordering. (2aae325)

Other
  • Improved strip-nondeterminism, our tool to remove specific nondeterministic information after a build:
    • Match more styles of Java .properties files.
    • Remove hyphen from "non-determinism" and "non-deterministic" throughout package for consistency.
  • Improvements to our testing infrastucture:
    • Improve the top-level navigation so that we can always get back to "home" of a package.
    • Give expandable elements cursor: pointer CSS styling to highlight they are clickable.
    • Drop various trailing underlined whitespaces after links.
    • Explicitly log that build was successful or not.
    • Various code-quality improvements, including prefering str.format over concatentation.
  • Miscellaneous updates to our filter-packages internal tool:
    • Add --random=N and --url options.
    • Add support for --show=comments.
    • Correct ordering so that --show-version runs after --filter-ftbfs.
    • Rename --show-ftbfs to --filter-ftbfs and --show-version to --show=version.
  • Created a proof-of-concept reproducible-utils package to contain commonly-used snippets aimed at developers wishing to make their packages reproducible.


I also submitted 92 patches to fix specific reproducibility issues in advi, amora-server, apt-cacher-ng, ara, argyll, audiotools, bam, bedtools, binutils-m68hc1x, botan1.10, broccoli, congress, cookiecutter, dacs, dapl, dateutils, ddd, dicom3tools, dispcalgui, dnssec-trigger, echoping, eekboek, emacspeak, eyed3, fdroidserver, flashrom, fntsample, forkstat, gkrellm, gkrellm, gnunet-gtk, handbrake, hardinfo, ircd-irc2, ircd-ircu, jack-audio-connection-kit, jpy, kxmlgui, libbson, libdc0, libdevel-cover-perl, libfm, libpam-ldap, libquvi, librep, lilyterm, mozvoikko, mp4h, mp4v2, myghty, n2n, nagios-nrpe, nikwi, nmh, nsnake, openhackware, pd-pdstring, phpab, phpdox, phpldapadmin, pixelmed-codec, pleiades, pybit, pygtksourceview, pyicu, python-attrs, python-gflags, quvi, radare2, rc, rest2web, roaraudio, rt-extension-customfieldsonupdate, ruby-compass, ruby-pg, sheepdog, tf5, ttf-tiresias, ttf-tiresias, tuxpaint, tuxpaint-config, twitter-bootstrap3, udpcast, uhub, valknut, varnish, vips, vit, wims, winswitch, wmweather+ & xshisen.

Debian GNU/Linux
Debian LTS

This month I have been paid to work 15 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Authored the patch & issued DLA 596-1 for extplorer, a web-based file manager, fixing an archive traversal exploit.
  • Issued DLA 598-1 for suckless-tools, fixing a segmentation fault in the slock screen locking tool.
  • Issued DLA 599-1 for cracklib2, a pro-active password checker library, fixing a stack-based buffer overflow when parsing large GECOS fields.
  • Improved the find-work internal tool adding optional colour highlighting and migrating it to Python 3.
  • Wrote an lts-missing-uploads tool to find mistakes where there was no correponding package in the archive after an announcement.
  • Added optional colour highlighting to the lts-cve-triage tool.

Uploads
  • redis 2:3.2.3-1 New upstream release, move to the DEP-5 debian/copyright format, ensure that we are running as root in LSB initscripts and add a README.Source regarding our local copies of redis.conf and sentinel.conf.
  • python-django:
    • 1:1.10-1 New upstream release.
    • 1:1.10-2 Fix test failures due to mishandled upstream translation updates.

  • gunicorn:
    • 19.6.0-2 Reload logrotate in the postrotate action to avoid processes writing to the old files and move to DEP-5 debian/copyright format.
    • 19.6.0-3 Drop our /usr/sbin/gunicorn ,3 -debian and related Debian-specific machinery to be more like upstream.
    • 19.6.0-4 Drop "template" systemd .service files and point towards examples and documentation instead.

  • adminer:
    • 4.2.5-1 Take over package maintenance, completely overhauling the packaging with a new upstream version, move to virtual-mysql-server to support MariaDB, updating package names of dependencies and fix the outdated Apache configuration.
    • 4.2.5-2 Correct the php5 package names.




FTP Team As a Debian FTP assistant I ACCEPTed 90 packages: android-platform-external-jsilver, android-platform-frameworks-data-binding, camlpdf, consolation, dfwinreg, diffoscope, django-restricted-resource, django-testproject, django-testscenarios, gitlab-ci-multi-runner, gnome-shell-extension-taskbar, golang-github-flynn-archive-go-shlex, golang-github-jamesclonk-vultr, golang-github-weppos-dnsimple-go, golang-golang-x-time, google-android-ndk-installer, haskell-expiring-cache-map, haskell-hclip, haskell-hdbc-session, haskell-microlens-ghc, haskell-names-th, haskell-persistable-record, haskell-should-not-typecheck, haskell-soap, haskell-soap-tls, haskell-th-reify-compat, haskell-with-location, haskell-wreq, kbtin, libclipboard-perl, libgtk3-simplelist-perl, libjs-jquery-selectize.js, liblemon, libplack-middleware-header-perl, libreoffice, libreswan, libtest-deep-json-perl, libtest-timer-perl, linux, linux-signed, live-tasks, llvm-toolchain-3.8, llvm-toolchain-snapshot, lua-luv, lua-torch-image, lua-torch-nn, magic-wormhole, mini-buildd, ncbi-vdb, node-ast-util, node-es6-module-transpiler, node-es6-promise, node-inline-source-map, node-number-is-nan, node-object-assign, nvidia-graphics-drivers, openhft-chronicle-bytes, openhft-chronicle-core, openhft-chronicle-network, openhft-chronicle-threads, openhft-chronicle-wire, pycodestyle, python-aptly, python-atomicwrites, python-click-log, python-django-casclient, python-git-os-job, python-hypothesis, python-nosehtmloutput, python-overpy, python-parsel, python-prov, python-py, python-schema, python-tackerclient, python-tornado, pyvo, r-cran-cairo, r-cran-mi, r-cran-rcppgsl, r-cran-sem, ruby-curses, ruby-fog-rackspace, ruby-mixlib-archive, ruby-tzinfo-data, salt-formula-swift, scapy3k, self-destructing-cookies, trollius-redis & websploit.

30 August 2016

Daniel Stender: My work for Debian in August

Here's again a little list of my humble off-time contributions I'm happy to add to the large amount of work we're completing all together each month. Then there is one more "new in Debian" (meaning: "new in unstable") announcement. First, the uploads (a few of them are from July): New packages: Sponsored uploads: Requested resp. suggested for packaging: New in Debian: Lasagne (deep learning framework) Now that the mathematical expression compiler Theano is available in Debian, deep learning frameworks resp. toolkits which have been build on top of it can become available within Debian, too (like Blocks, mentioned before). Theano is an own general computing engine which has been developed with a focus on machine learning resp. neural networks, which features an own declarative tensor language. The toolkits which have build upon it vary in the way how much they abstract the bare features of Theano, if they are "thick" or "thin" so to say. When the abstraction gets higher you gain more end user convenience up to the level that you have the architectural components of neural networks available for combination like in a lego box, while the more complicated things which are going on "under the hood" (like how the networks are actually implemented) are hidden. The downside is, thick abstraction layers usually makes it difficult to implement novel features like custom layers or loss functions. So more experienced users and specialists might to seek out for the lower abstraction toolkits, where you have to think more in terms of Theano. I've got an initial package of Keras in experimental (1.0.7-1), it runs (only a Python 3 package is available so far) but needs some more work (e.g. building the documentation with mkdocs). Keras is a minimalistic, high modular DNN library inspired by Torch1. It has a clean, rather easy API for experimenting and fast prototyping. It can also run on top of Google's TensorFlow, and we're going to have it ready for that, too. Lasagne follows a different approach. It's, like Keras and Blocks, a Python library to create and train multi-layered artificial neural networks in/on Theano for applications like image recognition resp. classification, speech recognition, image caption generation or other purposes like style transfers from paintings to pictures2. It abstracts Theano as little as possible, and could be seen rather like an extension or an add-on than an abstraction3. Therefore, knowledge on how things are working in Theano would be needed to make full use out of this piece of software. With the new Debian package (0.1+git20160728.8b66737-1)4, the whole required software stack (the corresponding Theano package, NumPy, SciPy, a BLAS implementation, and the nividia-cuda-toolkit and NVIDIA kernel driver to carry out computations on the GPU5) could be installed most conveniently just by a single apt-get install python ,3 -lasagne command6. If wanted with the documentation package lasagne-doc for offline use (no running around on remote airports seeking for a WIFI spot), either in the Python 2 or the Python 3 branch, or both flavours altogether7. While others have to spend a whole weekend gathering, compiling and installing the needed libraries you can grab yourself a fresh cup of coffee. These are the advantages of a fully integrated system (sublime message, as always: desktop users switch to Linux!). When the installation of packages has completed, the MNIST example of Lasagne could be used for a quick check if the whole library stack works properly8:
$ THEANO_FLAGS=device=gpu,floatX=float32 python /usr/share/doc/python-lasagne/examples/mnist.py mlp 5
Using gpu device 0: GeForce 940M (CNMeM is disabled, cuDNN 5005)
Loading data...
Downloading train-images-idx3-ubyte.gz
Downloading train-labels-idx1-ubyte.gz
Downloading t10k-images-idx3-ubyte.gz
Downloading t10k-labels-idx1-ubyte.gz
Building model and compiling functions...
Starting training...
Epoch 1 of 5 took 2.488s
  training loss:        1.217167
  validation loss:      0.407390
  validation accuracy:      88.79 %
Epoch 2 of 5 took 2.460s
  training loss:        0.568058
  validation loss:      0.306875
  validation accuracy:      91.31 %
The example on how to train a neural network on the MNIST database of handwritten digits is refined (it also provides --help) and explained in detail in the Tutorial section of the documentation in /usr/share/doc/lasagne-doc/html/. Very good starting points are also the IPython notebooks that are available from the tutorials by Eben Olson9 and Geoffrey French on the PyData London 201610. There you have Theano basics, examples for employing convolutional neural networks (CNN) and recurrent neural networks (RNN) for a range of different purposes, how to use pre-trained networks for image recognition, etc.

  1. For a quick comparison of Keras and Lasagne with other toolkits, see Alex Rubinsteyn's PyData NYC 2015 presentation on using LSTM (long short term memory) networks on varying length sequence data like Grimm's fairy tales (https://www.youtube.com/watch?v=E92jDCmJNek 27:30 sq.)
  2. https://github.com/Lasagne/Recipes/tree/master/examples/styletransfer
  3. Great introduction to Theano and Lasagne by Eben Olson on the PyData NYC 2015: https://www.youtube.com/watch?v=dtGhSE1PFh0
  4. The package is "freelancing" currently being in collab-maint, to set up a deep learning packaging team within Debian is in the stage of discussion.
  5. Only available for amd64 and ppc64el.
  6. You would need "testing" as package source in /etc/apt/sources.list to install it from the archive at the present time (I have that for years, but if Debian Testing could be advised as productive system is going to be discussed elsewhere), but it's coming up for Debian 9. The cuda-toolkit and pycuda are in the non-free section of the archive, thus non-free (mostly used in combination with contrib) must be added to main. Plus, it's a mere suggestion of the Theano packages to keep Theano in main, so --install-suggests is needed to pull it automatically with the same command, or this must be given explicitly.
  7. For dealing with Theano in Debian, see this previous blog posting
  8. Like suggested in the guideline From Zero to Lasagne on Ubuntu 14.04. cuDNN isn't available as official Debian package yet, but could be downloaded as a .deb package after registration at https://developer.nvidia.com/cudnn. It integrates well out of the box.
  9. https://github.com/ebenolson/pydata2015
  10. https://github.com/Britefury/deep-learning-tutorial-pydata2016, video: https://www.youtube.com/watch?v=DlNR1MrK4qE

Christoph Egger: DANE and DNSSEC Monitoring

At this year's FrOSCon I repeted my presentation on DNSSEC. In the audience, there was the suggestion of a lack of proper monitoring plugins for a DANE and DNSSEC infrastructure that was easily available. As I already had some personal tools around and some spare time to burn I've just started a repository with some useful tools. It's available on my website and has mirrors on Gitlab and Github. I intent to keep this repository up-to-date with my personal requirements (which also means adding a xmpp check soon) and am happy to take any contributions (either by mail or as "pull requests" on one of the two mirrors). It currently has smtp (both ssmtp and starttls) and https support as well as support for checking valid DNSSEC configuration of a zone. While working on it it turned out some things can be complicated. My language of choice was python3 (if only because the ssl library has improved since 2.7 a lot), however ldns and unbound in Debian lack python3 support in their bindings. This seems fixable as the source in Debian is buildable and useable with python3 so it just needs packaging adjustments. Funnily the ldns module, which is only needed for check_dnssec, in debian is currently buggy for python2 and python3 and ldns' python3 support is somewhat lacking so I spent several hours hunting SWIG problems.

Joachim Breitner: Explicit vertical alignment in Haskell

Chris Done s automatic Haskell formatter hindent is released in a new version, and getting quite a bit of deserved attention. He is polling the Haskell programmers on whether two or four spaces are the right indentation. But that is just cosmetics I am in principle very much in favor of automatic formatting, and I hope that a tool like hindent will eventually be better at formatting code than a human. But it currently is not there yet. Code is literature meant to be read, and good code goes at length to be easily readable, and formatting can carry semantic information. The Haskell syntax was (at least I get that impression) designed to allow the authors to write nicely looking, easy to understand code. One important tool here is vertical alignment of corresponding concepts on different lines. Compare
maze :: Integer -> Integer -> Integer
maze x y
  abs x > 4    abs y > 4  = 0
  abs x == 4   abs y == 4 = 1
  x ==  2    && y <= 0     = 1
  x ==  3    && y <= 0     = 3
  x >= -2    && y == 0     = 4
  otherwise                = 2
with
maze :: Integer -> Integer -> Integer
maze x y
  abs x > 4   abs y > 4 = 0
  abs x == 4   abs y == 4 = 1
  x == 2 && y <= 0 = 1
  x == 3 && y <= 0 = 3
  x >= -2 && y == 0 = 4
  otherwise = 2
The former is a quick to grasp specification, the latter (the output of hindent at the moment) is a desert of numbers and operators. I see two ways forward: What could such ways be? (This post is cross-posted on reddit.) Update (2016-09-05) Shortly after this post, the Haskell formatter brittany gets released, which supports vertial alignment. Yay!

29 August 2016

David Moreno: Webhook Setup with Facebook::Messenger::Bot

The documentation for the Facebook Messenger API points out how to setup your initial bot webhook. I just committed a quick patch that would make it very easy to setup a quick script to get it done using the unreleased and still in progress Perl s Facebook::Messenger::Bot:
use Facebook::Messenger::Bot;
use constant VERIFY_TOKEN => 'imsosecret';
my $bot = Facebook::Messenger::Bot->new(); # no config specified!
$bot->expect_verify_token( VERIFY_TOKEN );
$bot->spin();
This should get you sorted. What endpoint would that be, though? Well that depends on how you re giving Facebook access to your Plack s .psgi application.

Michal &#268;iha&#345;: motranslator 1.1

Four months after 1.0 release, motranslator 1.1 is out. If you happen to use it for untrusted data, this might be as well called security release, though this is still not good idea until we remove usage of eval() used to evaluate plural formula. Full list of changes: The motranslator is a translation library used in current phpMyAdmin master (upcoming 4.7.0) with focus on speed and memory usage. It uses Gettext MO files to load the translations. It also comes with testsuite (100% coverage) and basic documentation. Recommended way to install it is using composer from Packagist repository:
composer require phpmyadmin/motranslator
The Debian package will be available probably at point phpMyAdmin 4.7.0 will be out, but if you see need to have it earlier, just let me know.

Filed under: Debian English phpMyAdmin 0 comments

Gergely Nagy: Recruitment mistakes: part 3

It has been a while that I have been contacted by a recruiter, and the last few ones were fairly decent conversations, where they made an effort to research me first, and even if they did not get everything right, they still listened, and we had a productive talk. But four days ago, I had another recruiter reach out to me, from a company I know oh so well: one I ranted about before: Google. Apparently, their recruiters still do carpet-bombing style outreach. My first thought was "what took them so long?" - it has been five years since my last contact with a Google recruiter. I almost started missing them. Almost. To think that Google is now powerful enough to read my mind, is scary. Yet, I believe, this is not the case; rather, it's just another embarrassing mistake.To make my case, let me quote the full e-mail I was sent, with the name of the sender redacted, and my comments - which I'm also sending to the recruiter:
Hi Gergely,
Hi!
How are you? Hope you're well.
Thank you, I'm fine, just back from vacation, and I was thrilled to read your e-mail. Although, I did find it surprising too, and considering past events, I thought it to be spam first.
My name is XXX and I am recruiting for the Google Engineering team. I have just found your details on Github...
I'm happy that you found me through GitHub, but I'm curious why you mailed me at my debian.org address then? That address is not on my GitHub profile, and even though I have some code signed with that address, that's not what I use normally. My GitHub profile also links to a page I created especially for recruiters, which I don't think you have read - but more on that below.
...and your experience with software development combined with your open source contributions is particularly relevant to Google
Which part of my recent contributions are relevant to Google? For the past few months, the vast majority (over 90%) of my open source work has been on keyboard firmware. If Google is developing a keyboard, then this may be relevant, otherwise, I find it doubtful.Some of my past OSS contributions may be more relevant, but that's in the past, and it would take some digging to see those from GitHub. And if you did that kind of digging, you would have found the page on my site for recruiters, and would not have e-mailed me at my Debian address, either.
We are always interested in talking to top engineers with your mix of skills and I was wondering if you are at all open to exploring roles with Google in EMEA.
To make things short, my stance is the same as it was five years ago, when I wrote - and I quote - "So, here's a public request: do not email me ever again about job opportunities within Google. I do not wish to work for Google. Not now, not tomorrow, not ever."This still stands. Please do not ever e-mail me about job opportunities within Google. I do not wish to work for Google, not now, not tomorrow, not five years from now, not ever. This will not change. Many things may change, this however, will not.But even if I ignore this, let me ask you mix of skills are you exactly interested in? Keyboard firmware hacking, mixed with Emacs Lisp, some Clojure, and hacking on Hy (which, if you have explored my GitHub profile, will surely know) from time to time? Or is it my Debian Developer hat that got your interest? If so, why didn't you say so? Not that it would change anything, but I'm curious.Is it my participation in 24pullrequests? Or my participation in GSoC in years past?Nevertheless, my list of conditions I mentioned above, apply. Is Google able to fulfill all my requirements, and the preferences too? Last time I heard, working for Google required one to relocate, which I clearly said on that page, I'm not willing to do.
The teams I recruit for are responsible for Google's planet-scale systems. Just to give you more of an idea, some of the projects we work on that might be of interest to you would include:
  • MapReduce
  • App Engine
  • Mesa
  • Maglev
And how would these be interesting to me, considering my recent OSS work? (Hint: none of them interest me, not a bit. Ok perhaps MapReduce, a little.)
I think you could be a great fit here, where you can develop a great career and at the same time you will be part of the most mission critical team, designing and developing systems which run Google Search, Gmail, YouTube, Google+ and many more as you can imagine.
My career is already developing fine, thank you, I do not need Google to do that. I am already working on things I consider important and interesting, if I wanted to change, I would. But I certainly would not consider a company which I had asked numerous times NOT to contact me about opportunities. If you can't respect this wish, and forget about this in mere five years, why would I trust you to keep any other promises you make now?
Thank you so much for your time and I look forward to hearing from you.
I wish you had spent as much time researching me - or even half that - as I have spent replying to you. I suppose this is not the reply you were expecting, or looking for, but this is the only one I'll ever give to anyone from Google.And here, at the end, if you read this far, I'm asking you, and everyone at Google to never contact me about job opportunities. I will not work for Google. Not now, not tomorrow, not five years from now, not ever. Please do not e-mail me again, and do not reply to this e-mail either. I'm not interested in neither apologies, nor promises that you won't contact me - just simply don't do it.Thank you.

Russell Coker: Monitoring of Monitoring

I was recently asked to get data from a computer that controlled security cameras after a crime had been committed. Due to the potential issues I refused to collect the computer and insisted on performing the work at the office of the company in question. Hard drives are vulnerable to damage from vibration and there is always a risk involved in moving hard drives or systems containing them. A hard drive with evidence of a crime provides additional potential complications. So I wanted to stay within view of the man who commissioned the work just so there could be no misunderstanding. The system had a single IDE disk. The fact that it had an IDE disk is an indication of the age of the system. One of the benefits of SATA over IDE is that swapping disks is much easier, SATA is designed for hot-swap and even systems that don t support hot-swap will have less risk of mechanical damage when changing disks if SATA is used instead of IDE. For an appliance type system where a disk might be expected to be changed by someone who s not a sysadmin SATA provides more benefits over IDE than for some other use cases. I connected the IDE disk to a USB-IDE device so I could read it from my laptop. But the disk just made repeated buzzing sounds while failing to spin up. This is an indication that the drive was probably experiencing stiction which is where the heads stick to the platters and the drive motor isn t strong enough to pull them off. In some cases hitting a drive will get it working again, but I m certainly not going to hit a drive that might be subject to legal action! I recommended referring the drive to a data recovery company. The probability of getting useful data from the disk in question seems very low. It could be that the drive had stiction for months or years. If the drive is recovered it might turn out to have data from years ago and not the recent data that is desired. It is possible that the drive only got stiction after being turned off, but I ll probably never know. Doing it Properly Ever since RAID was introduced there was never an excuse for having a single disk on it s own with important data. Linux Software RAID didn t support online rebuild when 10G was a large disk. But since the late 90 s it has worked well and there s no reason not to use it. The probability of a single IDE disk surviving long enough on it s own to capture useful security data is not particularly good. Even with 2 disks in a RAID-1 configuration there is a chance of data loss. Many years ago I ran a server at my parents house with 2 disks in a RAID-1 and both disks had errors on one hot summer. I wrote a program that s like ddrescue but which would read from the second disk if the first gave a read error and ended up not losing any important data AFAIK. BTRFS has some potential benefits for recovering from such situations but I don t recommend deploying BTRFS in embedded systems any time soon. Monitoring is a requirement for reliable operation. For desktop systems you can get by without specific monitoring, but that is because you are effectively relying on the user monitoring it themself. Since I started using mon (which is very easy to setup) I ve had it notify me of some problems with my laptop that I wouldn t have otherwise noticed. I think that ideally for desktop systems you should have monitoring of disk space, temperature, and certain critical daemons that need to be running but which the user wouldn t immediately notice if they crashed (such as cron and syslogd). There are some companies that provide 3G SIMs for embedded/IoT applications with rates that are significantly cheaper than any of the usual phone/tablet plans if you use small amounts of data or SMS. For a reliable CCTV system the best thing to do would be to have a monitoring contract and have the monitoring system trigger an event if there s a problem with the hard drive etc and also if the system fails to send a I m OK message for a certain period of time. I don t know if people are selling CCTV systems without monitoring to compete on price or if companies are cancelling monitoring contracts to save money. But whichever is happening it s significantly reducing the value derived from monitoring.

28 August 2016

Reproducible builds folks: Reproducible builds: week 70 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday August 21 and Saturday August 27 2016: GSoC and Outreachy updates Packages reviewed and fixed, and bugs filed Reviews of unreproducible packages 10 package reviews have been added and 6 have been updated this week, adding to our knowledge about identified issues. A large number of issue types have been updated: Weekly QA work 29 FTBFS bugs have been reported by: diffoscope development Holger also created another test job for diffoscope on jenkins.debian.net, so that now also all commits to branches other than master are being tested. strip-nondeterminism development strip-nondeterminism 0.023-1 was uploaded by Chris Lamb:
 * Support Android .apk files with the JAR normalizer.
 * handlers/png.pm: Drop unused Archive::Zip import
 * Remove hyphen from non-determinism and non-deterministic.
 * javaproperties.pm: Match more styles of .properties and loosen filename matching.
 * Improve tests:
   - Make fixture runner generic to all normalizer types.
   - Replace (single) pearregistry test with a fixture.
   - Set a canonical time for fixture tests.
   - Add gzip testcase fixture.
   - Replace t/javadoc.t with fixture
   - Replace t/ar.t with a fixture.
   - t/javaproperties: move pom.properties and version.properties tests to fixtures
   - t/fixtures.t: move to using subtests
   - t/fixtures.t: Explicitly test that we can find a normalizer
   - t/fixtures.t: Don't run normalizer if we didn't find one.
strip-nondeterminism 0.023-2 uploaded by Mattia Rizzolo to allow stderr in autopkgtest. disorderfs development tests.reproducible-builds.org Debian: Somewhat related to reproducible builds there has been a first Debian jenkins team maintainance meeting on the #debian-qa IRC channel, to discuss current issues with the setup and to start the work of migrating jenkins.debian.net to jenkins.debian.org. The next meeting will take place on September 28th 2016 at 19 UTC. Misc. This week's edition was written by Chris Lamb and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

26 August 2016

Norbert Preining: ESSLLI 2016 Course: Algebraic Specification and Verification with CafeOBJ

During this year s ESSLLI (European Summer School in Logic, Language and Information) I was teaching a course on Algebraic Specification and Verification with CafeOBJ. Now that the course is over I can relax a bit and enjoy the beauty of the surrounding North Tyrol.
ESSLLI-2016-logo-darkblue-small For those who couldn t attend the ESSLLI or the course, here are the course materials: Thanks to the participants for the interesting questions and participation. What follows is the original description of the course. Abstract Ensuring correctness of complex systems like computer programs or communication protocols is gaining ever increasing importance. For correctness it does not suffice to consider the finished system, but correctness already on the level of specification is necessary, that is, before the actual implementation starts. To this aim, algebraic specification and verification languages have been conceived. They are aiming at a mathematically correct description of the properties and behavior of the systems under discussion, and in addition often allow to prove (verify) correctness within the same system. This course will give an introduction to algebraic specification, its history, logic roots, and actuality with respect to current developments. Paired with the theoretical background we give an introduction to CafeOBJ, a programming language that allows both specification and verification to be carried out together. The course should enable students to understand the theoretical background of algebraic specification, as well as enable them to read and write basic specifications in CafeOBJ. Description CafeOBJ is a specification language based on three-way extensions to many-sorted equational logic: the underlying logic is order-sorted, not just many-sorted; it admits unidirectional transitions, as well as equations; it also accommodates hidden sorts, on top of ordinary, visible sorts. A subset of CafeOBJ is executable, where the operational semantics is given by a conditional order-sorted term rewriting system. The language system CafeOBJ has been under constant development at the institute of the lecturers since the late 80ies. It is closely related to other algebraic specification languages in the OBJ family, including Maude. The CafeOBJ language and the range of verification methods and tools it supports including its support for inductive theorem proving, verification of behavioral specifications, deductive invariant proof, and reachability analysis of concurrent systems has played a key role in both extending and bringing algebraic specification techniques into contact with many software engineering applications. The following topics will be discussed: To make the lectures not too heavy , we will structure each lecture into two parts: A first part providing an introduction of some theoretical concept, or framework, and a second part dealing with actual programming and implementation. Especially for the second part of each lecture students are encouraged to use their laptops to try out code and experiment.

25 August 2016

Francois Marier: Debugging gnome-session problems on Ubuntu 14.04

After upgrading an Ubuntu 14.04 ("trusty") machine to the latest 16.04 Hardware Enablement packages, I ran into login problems. I could log into my user account and see the GNOME desktop for a split second before getting thrown back into the LightDM login manager. The solution I found was to install this missing package:
apt install libwayland-egl1-mesa-lts-xenial

Looking for clues in the logs The first place I looked was the log file for the login manager (/var/log/lightdm/lightdm.log) where I found the following:
DEBUG: Session pid=12743: Running command /usr/sbin/lightdm-session gnome-session --session=gnome
DEBUG: Creating shared data directory /var/lib/lightdm-data/username
DEBUG: Session pid=12743: Logging to .xsession-errors
This told me that the login manager runs the gnome-session command and gets it to create a session of type gnome. That command line is defined in /usr/share/xsessions/gnome.desktop (look for Exec=):
[Desktop Entry]
Name=GNOME
Comment=This session logs you into GNOME
Exec=gnome-session --session=gnome
TryExec=gnome-shell
X-LightDM-DesktopName=GNOME
I couldn't see anything unexpected there, but it did point to another log file (~/.xsession-errors) which contained the following:
Script for ibus started at run_im.
Script for auto started at run_im.
Script for default started at run_im.
init: Le processus gnome-session (GNOME) main (11946) s'est achev  avec l' tat 1
init: D connect  du bus D-Bus notifi 
init: Le processus logrotate main (11831) a  t  tu  par le signal TERM
init: Le processus update-notifier-crash (/var/crash/_usr_bin_unattended-upgrade.0.crash) main (11908) a  t  tu  par le signal TERM
Seaching for French error messages isn't as useful as searching for English ones, so I took a look at /var/log/syslog and found this:
gnome-session[4134]: WARNING: App 'gnome-shell.desktop' exited with code 127
gnome-session[4134]: WARNING: App 'gnome-shell.desktop' exited with code 127
gnome-session[4134]: WARNING: App 'gnome-shell.desktop' respawning too quickly
gnome-session[4134]: CRITICAL: We failed, but the fail whale is dead. Sorry....
It looks like gnome-session is executing gnome-shell and that this last command is terminating prematurely. This would explain why gnome-session exits immediately after login.

Increasing the amount of logging In order to get more verbose debugging information out of gnome-session, I created a new type of session (GNOME debug) by copying the regular GNOME session:
cp /usr/share/xsessions/gnome.desktop /usr/share/xsessions/gnome-debug.desktop
and then adding --debug to the command line inside gnome-debug.desktop:
[Desktop Entry]
Name=GNOME debug
Comment=This session logs you into GNOME debug
Exec=gnome-session --debug --session=gnome
TryExec=gnome-shell
X-LightDM-DesktopName=GNOME debug
After restarting LightDM (service lightdm restart), I clicked the GNOME logo next to the password field and chose GNOME debug before trying to login again. This time, I had a lot more information in ~/.xsession-errors:
gnome-session[12878]: DEBUG(+): GsmAutostartApp: starting gnome-shell.desktop: command=/usr/bin/gnome-shell startup-id=10d41f1f5c81914ec61471971137183000000128780000
gnome-session[12878]: DEBUG(+): GsmAutostartApp: started pid:13121
...
/usr/bin/gnome-shell: error while loading shared libraries: libwayland-egl.so.1: cannot open shared object file: No such file or directory
gnome-session[12878]: DEBUG(+): GsmAutostartApp: (pid:13121) done (status:127)
gnome-session[12878]: WARNING: App 'gnome-shell.desktop' exited with code 127
which suggests that gnome-shell won't start because of a missing library.

Finding the missing library To find the missing library, I used the apt-file command:
apt-file update
apt-file search libwayland-egl.so.1
and found that this file is provided by the following packages:
  • libhybris
  • libwayland-egl1-mesa
  • libwayland-egl1-mesa-dbg
  • libwayland-egl1-mesa-lts-utopic
  • libwayland-egl1-mesa-lts-vivid
  • libwayland-egl1-mesa-lts-wily
  • libwayland-egl1-mesa-lts-xenial
Since I installed the LTS Enablement stack, the package I needed to install to fix this was libwayland-egl1-mesa-lts-xenial. I filed a bug for this on Launchpad.

22 August 2016

Luciano Prestes Cavalcanti: AppRecommender - Last GSoC Report

My work on Google Summer of Code is to create a new strategy on AppRecommender, where this strategy should be able to get a referenced package, or a list of referenced packages, then analyze the packages that the user has already installed and make a recommendation using the referenced packages as a base, for example: if the user runs "$ sudo apt install vim", the AppRecommender uses "vim" as the referenced package, and should recommend packages with relation between "vim" and the other packages that the user has installed. This work is done and added to the official AppRecommender repository.
During the GSoC program, more contributions were done with the AppRecommender project helping the system to improve the recommendations, installation and configurations to help Debian package.
The following link contains my commits on AppRecommender:
https://github.com/tassia/AppRecommender/commits/master?author=LucianoPC
During the period destined to students get to know the community of the project, I talked with the Debian community about my project to get feedback and ideas. When talking to the Debian community on the IRC channels, we came up with the idea of using the popularity-contest data to improve the recommendations. I talked with my mentors, who approved the idea, then we increased the project scope to use the popularity-contest data to improve the AppRecommender recommendations.
The popularity-contest has several privacy political terms, then we did a research and published, on the Debian Planet, a post that explains why we need the popularity-contest data to improve the recommendations and how we use this data. This post also contains an explanation about the risks and the measures taken to minimize them.
Then two activities were added to be made. One of them is to create a script to be added on popularity-contest. This script is destined to get the popularity-contest data, which is the users' packages, and generate clusters that group these packages analyzing similar users. The other activity is to add collaborative data into the AppRecommender, where this will download the clusters data and use it to improve the recommendations.
The popularity-contest cluster script was done and reviewed by my mentor, but was not integrated into popularity-contest yet. The usage of clusters data into AppRecommender has been already implemented, but still not added on official repository because it is waiting the cluster cript's acceptance into the popularity-contest. This work is not complete, but I will continue working with AppRecommender and Debian community, and with my mentors' help, I will finish this work.
The following link contains my commits on repository with the popularity-contest cluster script's feature, as well as other scripts that I used to improve my work, but the only script that will be sent to popularity-contest is the create_popcon_clusters.py:
https://github.com/TCC-AppRecommender/scripts/commits/master?author=LucianoPC
The following link contains my commits on repository with the AppRecommender collaborative data feature:
Google Drive folder with the patch:
https://drive.google.com/drive/folders/0BzmGBlxBo2G3Q3F5YjBpRl9yWUk?usp=sharing

Zlatan Todori : When you wake up with a feeling

I woke up at 5am. Somehow made myself to soon go back to sleep again. Woke up at 6am. Such is the life of jet-lag. Or I am just getting old for it. But the truth wouldn't be complete with only those assertion. I woke inspired and tired and the same time. Tired because I am doing very time consumable things. Also in the same time very emotional things. AND at the exact same time things that inspire me. On paper, I am technical leader of Purism. In reality, I have insanely good relations with my CEO for such a short time. So good that I for months were not leading the technical shift only, but also I overtook operations (getting orders and delivering them while working with our assembly line to automate most of the tasks in this field). I was playing also as first line of technical support (forums, IRC and email). Actually I was pretty much the only line of support for few months. I was doing some website changes: change some wording, updating bunch of plugins and making it sure all works, resolved (hopefully) Tor and Cloudflare issues for it, annoying caching system for forums, stopped forum spam and so on. I worked on better messaging for Purism public relations. I thought my team to use keys for signing and encryption. I interviewed (and read all mails) for people that were interested in working or helping Purism. In process of doing all that, I maybe wasn't the most speedy person for all our users needs but I hope they understand and forgive me. I was doing all that while I was researching and developing tablets (which ended up not being the most successful campaign but we now do have them as product). I was doing all that while seeing (and resolving) that our kernel builds were failing. Worked on pushing touchpad (not so good but we are still working on) patches upstream (and they ended being upstreamed). While seeing repos being down because of our host. Repos being down because of broken sync with Debian. Repos being down because of our key mis-management. Metadata not working well. PureBrowser getting broken all the time. Tor browser out of date. No real ISO updates. Wrong sources.list entries and so on. And the hardest part on work was, I was doing all this with very limited scope and even more limited resources. So what kept me on, what is pushing me forward and what am I doing? One philosophy - Free software. Let me not explain it as a technical debt. Let me explain it as social movement. In age, where people are "bombed" by media, by all-time lying politicians (which use fear of non-existent threats/terror as model to control population), in age where proprietary corporations are selling your freedom so you can gain temporary convenience the term Free software is like Giordano Bruno in age of Inquisitions. Free software does not only preserve your Freedom to software source usage but it preserves your Freedom to think and think out of the box and not being punished for that. It preserves the Freedom to live - to choose what and when to do, without having the negative impact on your or others people lives. The Freedom to be transparent and to share. Because not only ideas grow with sharing, but we, as human beings, grow as we share. The Freedom to say "NO". NO. I somehow learnt, and personally think, that the Freedom to say NO is the most important Freedom in our lives. No I will not obey some artificially created master that think they can plan and choose my life decision. No I will not negotiate my Freedom for your convenience (also, such Freedom is anyway not real and it is matter of time where you will be blown away by such illusion). No I will not accept your credit because it has STRINGS attached to it which you either don't present or you blur it in mountain of superficial wording. No I will not implant a chip inside me for sake of your research or my convenience. No I will not have social account on media where majority of people are. No, I will not have pacemaker which is a blackbox with proprietary (buggy) software and it harvesting my data without me being able to look at it. Yin-Yang. Yes, I want to collaborate on making world better place for us all. I don't agree with most of people, but that doesn't make them my enemies (although media would like us to feel and think like that). I will try to preserve everyones Freedom as much as I can. Yes I will share with my community and friends. Yes I want to learn from better than I am. Yes I want to have awesome mentors. Yes, I will try to be awesome mentor. Yes, I choose to care and not ignore facts and actions done by me and other people. Yes, I have the right to be imperfect and do mistakes as long as I will aknowledge and work on them. Bugfixing ourselves as humans is the most important task in our lives. As in software, it is very time consumable but also as in software, it is improvement and incredible satisfaction to see better version of yourself, getting more and more features (even if that sometimes means actually getting read of other/bad features). This all is blending with my work at Purism. I spend a lot of time thinking about projects, development and future. I must do that in order not to make grave mistakes. Failing hardware and software is not grave mistake. Serious, but not grave. Grave is if we betray ourselves and our community in pursue for Freedom. We are trying to unify many things - we want to give you security, privacy and FREEDOM with convenience. So I am pushing myself out of comfort zones and also out of conventional and sometimes even my standard way of thinking. I have seen that non-existing infrastructure for PureOS is hurting is a lot but I needed to cope with it to the time where I will be able to say: not anymore, we are starting to build our own infrastructure. I was coping with Cloudflare being assholes to Tor users but now we also shifting away from them. I came to team where people didn't properly understand what and why are we building this. Came to very small and not that efficient team. Now, we employed a dedicated and hard working person on operations (Goran) which I trust. We have dedicated support person (Mladen) which tries hard to work with people. A very creative visual mastermind (Francois). We have a capable Debian Developer (Matthias Klumpp) working on PureOS new infra. We have a capable and dedicated sysadmins (Theo and Stelio) which we didn't even have in past. We are trying to LEVEL UP Free software and unify them in convenient solution which is lead by Joey Hess. We have a hard-working PureOS developer (Hema) who is coping with current non-existent PureOS infra. We have GNOME Boards of Directors person (Jeff) who is trying to light up our image in world (working with James, to try bring some lights into our shadows caused by infinite supply chain delays). We have created Advisory Board for Freedom, Privacy and Security which I don't want to name now as we are preparing to announce soon that (and trust me, we have good people in here). But, the most important thing here is not that they are all capable or cool people. It is the core value in all of them - they care about Freedom and I trust them on their paths. The trust is always important but in Purism it is essential for our work. I built the workflow without time management (everyone spends their time every single day as they see it fit as long as the work gets done). And we don't create insane short deadlines because everyone else thinks it is important (and rarely something is more important than our time freedom). So the trust is built out of knowledge and the knowledge I have about them and their works is because we freely share with no strings attached. Because of them, and other good people from our community I have the energy to sacrifice my entire time for Purism. It is not white and black: CEO and me don't always agree, some members of my team don't always agree with me or I with them, some people in community are very rude, impolite and don't respect our work but even with disagreement everyone in Purism finds agreement at the end (we use facts in our judgments) and all the people who just try to disturb my and mine teams work aren't as efficient as all the lovely words of people who believe in us, who send us words of support and who share ideas and their thoughts with us. There is no more satisfaction for me than reading a personal mail giving us kudos for the work and their understanding of underlaying amount of work and issues. While we are limited with resources we had an occasional outcry from community to help us. Now I want to help them to help me (you see the Freedom of sharing here?). PureOS has now a wiki. It will be a community wiki which is endorsed by Purism as company. Yes you read it right, Purism considers its community part of company (you don't need to get paycheck to be Purism member). That is why a call upon contributors (technical but mostly non-technical too) to help us make PureOS wiki the best resource on net for our needs. Write tutorials for others, gather and put info on wiki, create an ideas page and vote on them so we can see what community wants to see, chat with us so we all understand what, why and how are we working on things. Make it as transparent as possible. Everyone interested please get in touch with our teams by either poking us online (IRC, social accounts) or via emails (our personal or [hr, pr, feedback]@puri.sm. To finish this writing (as it is 8am here and I still want to rest a bit because I will have meetings for 6 hours straight today) - I wanted to share some personal insight into few things from my point of view. I wanted to say despite all the troubles and people who tried to make our time even harder (and it is already hard by all the limitation which come naturally today with our kind of work), we still create products, we still ship them, we still improved step by step, we still hired and we are still building. Keeping all that together and making progress is for me a milestone greater than just creating a technical product. I just hope we will continue and improve our pace so we can start progressing towards my personal great goal - integrate and cooperate with most of FLOSS ecosystem. P.S. yes, I also (finally!) became an official Debian Developer - still didn't have time to sit and properly think and cry (as every good men) about it.

21 August 2016

Gregor Herrmann: RC bugs 2016/30-33

not much to report but I got at least some RC bugs fixed in the last weeks. again mostly perl stuff:

David Moreno: WIP: Perl bindings for Facebook Messenger

A couple of weeks ago I started looking into wrapping the Facebook Messenger API into Perl. Since all the calls are extremely simple using a REST API, I thought it could be easier and simpler even, to provide a small framework to hook bots using PSGI/Plack. So I started putting some things together and with a very simple interface you could do a lot:
use strict;
use warnings;
use Facebook::Messenger::Bot;
my $bot = Facebook::Messenger::Bot->new( 
    access_token   => '...',
    app_secret     => '...',
    verify_token   => '...'
 );
$bot->register_hook_for('message', sub  
    my $bot = shift;
    my $message = shift;
    my $res = $bot->deliver( 
        recipient => $message->sender,
        message =>   text => "You said: " . $message->text()  
     );
    ...
 );
$bot->spin();
You can hook a script like that as a .psgi file and plug it in to whatever you want. Once you have some more decent user flow and whatnot, you can build something like:



using a simple script like this one. The work is not finished and not yet CPAN-ready but I m posting this in case someone wants to join me in this mini-project or have suggestions, the work in progress is here. Thanks!

Next.

Previous.