Search Results: "phil"

16 October 2015

Norbert Preining: Debian/TeX Live multiarch update

A big update of all related packages (tex-common 6.04, texlive-bin 2015.20150524.37493-7, texlive-base/lang/extra package 2015.20151016-1) due to the move to support multi-arch. Of course, the regular updates of the TeX Live are included, too. With this change it should be possible to run a multi-arch system with only one TeX Live installed. Debian - TeX Live 2015 Thanks to the excellent support and testing of the Multi-arch guys, in particular Thorsten Glaser, Helmut Grohne, Johannes Schauer, and Wookey, I learned a lot about multi-arch, and I hope that the current setup is safe. All the packages but the various lib* packages are tagged as Multi-Arch: foreign, while the lib packages are tagged Multi-Arch: same. Anyway, if you find a bug concerning multi-arch, that is that some of the programs exhibit architecture information, please let us know via a bug report. Updated packages acro, alegreya, amiri, assoccnt, attachfile, babel-french, babel-hungarian, barr, beebe, biblatex-philosophy, bidi, bnumexpr, caption, chemfig, chemformula, chemmacros, cjk-gs-integrate, csplain, dantelogo, dataref, dtxgen, dvipdfmx-def, dvips, eledmac, elements, fcolumn, fithesis, fontspec, genealogytree, gradstudentresume, gtl, jfontmaps, knuth-local, koma-script, kotex-oblivoir, kotex-plain, kotex-utf, kpathsea, l3build, l3experimental, l3kernel, l3packages, latex, latexconfig, ledmac, ltxfileinfo, lualatex-math, luamplib, luatex, luatexbase, luatexja, luatexko, make4ht, mcf2graph, mflogo, modiagram, multiexpand, newtx, odsfile, old-arrows, paracol, pdfpages, pdftex, plain, pst-stru, pxchfon, randomwalk, reledmac, resumecls, rubik, selnolig, showhyphens, siunitx, suftesi, tetex, teubner, tex4ebook, tex4ht, texlive-scripts, tikzsymbols, tipfr, tools, tudscr, uassign, unicode-math, unravel, visualfaq, xepersian, xetex-def, xint. New packages archaeologie, ctablestack, dynamicnumber, exercises, fibeamer, h2020proposal, imfellenglish, lstbayes, tempora, xellipsis. Enjoy.

14 October 2015

Lunar: Reproducible builds: week 24 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes Scott Kitterman fixed an issue with non-deterministic Depends generated by dh-python identified by Santiago Vila and Chris Lamb. Lunar updated the patch against dpkg which makes the order of files in control.tar.gz deterministic using the new --sort=name option available in GNU Tar 1.28. josch released sbuild version 0.66.0-1 with several fixes and improvements. The most notable one for reproducible builds is the new --build-path option and $build_path configuration variable added by akira which allows to explicitly chose a given build path. Reiner Herrmann wrote a new patch for dh-systemd to sort the list of unit files in the generated maintainer scripts. Packages fixed The following packages became reproducible due to changes in their build dependencies: aoeui, apron, camlmix, cudf, findlib, glpk-java, hawtjni, haxe, java-atk-wrapper, llvm-py, misery, mtasc, ocamldsort, optcomp, spamoracle. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Untested Patches submitted which have not made their way to the archive yet: reproducible.debian.net ProfitBricks once again increased their support for reproducible builds in Debian and in other free software projects by adding 58 new cores and 138 GiB of RAM to the already existing setup. Two new amd64 build nodes and 16 new amd64 build jobs have been added which doubles the build capacity per day and allows us to spot many kind of problems earlier. The size of the tmpfs where builds are performed has also been increased from 70 to 200 GiB on all amd64 build nodes. Huge thanks! When examining a package, a link now points to a table listing all previous recorded tests for the same package. (Mattia) The menu on the package pages has also been improved. (h01ger) Packages in the depwait state are now rescheduled automatically after five days. (h01ger) Links to documentation and other projects being tested have been made more visible on the landing page. (h01ger) To reduce noise on the team IRC channel five different types of notifications have been turned into mail notifications. The remaining ones have been shortened and the status changes have been limited to unstable and experimental. (h01ger) Maintainer notifications about status changes in a package will only be sent out once per day, and not on each status change. (h01ger) diffoscope development Some more experiments of concurrent processing have been made. None were good and reliable enough to be shared, though. Package reviews 48 reviews have been removed, 189 added and 23 updated this week. 9 FTBFS bugs were reported by Chris Lamb. Misc. h01ger met with Levente Polyak to discuss testing Arch Linux on Debian continuous test system with an easily extensible framework. The idea is to also allow testing of other distributions, and provide a nice package based view like the one for Debian.

13 October 2015

Norbert Preining: (Kim Doo Soo) ( , Free Spirit)

On the recently reviewed album International Sad Hits Vol 1 I wrote that Kim Doo Soo was the great surprise of the CD for me. So I went ahead and bought (not that easy, import from US!) one of his most recommended album, the Free Spirit. Kim_Doo_Soo-Free_Spirit Sad songs, full of melancholy, speaking of desperation, isolation, destroyed love. The music and singing is simple, mainly based on guitar and voice only, at times harmonica, strings and others are added. In contrast to the Crying Philosopher Tomokawa Kazuki, the songs are all very melodic and easy to listen, but still carrying a strong impact. After listening to these songs, the melodies wander around my brain resonating again and again, like (mis)guiding lights in a dark forest. kim-doo-soo-japanese-textAll the songs are in Korean, but fortunately the accompanying booklet carries a Japanese translation for the Korean incapables like me. In the future I will provide some translations into English, since I believe that this artist should be more known outside this small area. While a friend of mine said I don t like (bimbokusai) music, I have to say I enjoy reading through the sad stories while listening to this special music.
This is the 2002 fourth album by Korea s true cult folk/singer-songwriter hero, Kim Doo Soo. Kim Doo Soo is perhaps the deepest and most introspective of Korea s acid folk singers. Legends surround his songs political oppression, alcoholism, suicide, a 10-year period of mountain seclusion. [ ] On Free Spirit he mines productive veins of profound melancholy, animistic nature, and unfathomable, hermetic affection. These songs are couched in a veil of gorgeous, still melodicism, Kim s quavering vocals and guitar shaded with subtle accordion, cello, organ and harmonica. A reflective and unearthly masterpiece.
Forced Exposure
To this I cannot add anything more eloquent! The songs on this CD are as follows, the English translation is taken from Kim Doo Soo s official web page.
  1. (Wild flower)
  2. (Boat for the shore)
  3. (Butterfly)
  4. (Sweetbrier)
  5. (Bohemian)
  6. (Dawn rain)
  7. 19 Blues (Blues of 19th street)
  8. (Mountain)
  9. (As time goes by)
  10. Romantic Horizon
  11. ( ) (Reminiscence)
  12. (Evening river)
  13. ( ) (Bohemian poetry)
  14. , (Dandelion, dandelion)
My favorites under these songs are Boat for the shore ( You aren t coming any more onto a wild sea I am leaving ), Bohemian ( I m going to the shores of emptiness, my free spirit flows down the river probably his most famous song, assumed to be responsible for his retreat to the mountains), Dawn rain (A lovely song, and the only one without Japanese translation! Very strange.), Mountain ( Mountains Mountains my dream, mountains ). But I can recommend each and every song on this CD. To close this short review, if you are interested in something else but the usual happy-pappy KPop or JPop or USPop, and are searching for inspiring and songs with Tiefgang (deep meaning, as we say it in German), give this special artist a try! Some links concerning Kim Doo Soo that might be useful:

4 October 2015

Philipp Kern: Root on LVM on Debian s390x, new Hercules


Two s390x changes landed in Debian unstable today:
With this it should be possible to install Debian on s390x with root on LVM. I'd be happy to hear feedback about installations with any configuration, be it root on a single DASD or root on LVM. Unless you set both mirror/udeb/suite and mirror/suite to unstable you'll need to wait until the changes are in testing, though. (The debian-installer build does not matter as zipl-installer is not part of the initrd and sysconfig-hardware is part of the installation.)

Furthermore I uploaded a new version of Hercules - a z/Architecture emulator - to get a few more years of maintenance into Debian. See its upstream changelog for details on the changes (old 3.07 new 3.11).

At this point qemu at master is also usable for s390x emulation. It is much faster than Hercules, but it uses newfangled I/O subsystems like virtio. Hence we will need to do some more patching to make debian-installer just work. One patch for netcfg is in to support virtio networking correctly, but then it forces the user to configure a DASD. (Which would be as wrong if Fibre Channel were to be used.) In the end qemu and KVM on s390x look so much like a normal x86 VM that we could drop most of the special-casing of s390x (netcfg-static instead of netcfg; network-console instead of using the VM console; DASD configuration instead of simply using virtio-blk devices; I guess we get to keep zIPL for booting).

2 October 2015

Daniel Pocock: Want to be selected for Google Summer of Code 2016?

I've mentored a number of students in 2013, 2014 and 2015 for Debian and Ganglia and most of the companies I've worked with have run internships and graduate programs from time to time. GSoC 2015 has just finished and with all the excitement, many students are already asking what they can do to prepare and be selected for Outreachy or GSoC in 2016. My own observation is that the more time the organization has to get to know the student, the more confident they can be selecting that student. Furthermore, the more time that the student has spent getting to know the free software community, the more easily they can complete GSoC. Here I present a list of things that students can do to maximize their chance of selection and career opportunities at the same time. These tips are useful for people applying for GSoC itself and related programs such as GNOME's Outreachy or graduate placements in companies. Disclaimers There is no guarantee that Google will run the program again in 2016 or any future year until the Google announcement. There is no guarantee that any organization or mentor (including myself) will be involved until the official list of organizations is published by Google. Do not follow the advice of web sites that invite you to send pizza or anything else of value to prospective mentors. Following the steps in this page doesn't guarantee selection. That said, people who do follow these steps are much more likely to be considered and interviewed than somebody who hasn't done any of the things in this list. Understand what free software really is You may hear terms like free software and open source software used interchangeably. They don't mean exactly the same thing and many people use the term free software for the wrong things. Not all projects declaring themselves to be "free" or "open source" meet the definition of free software. Those that don't, usually as a result of deficiencies in their licenses, are fundamentally incompatible with the majority of software that does use genuinely free licenses. Google Summer of Code is about both writing and publishing your code and it is also about community. It is fundamental that you know the basics of licensing and how to choose a free license that empowers the community to collaborate on your code well after GSoC has finished. Please review the definition of free software early on and come back and review it from time to time. The The GNU Project / Free Software Foundation have excellent resources to help you understand what a free software license is and how it works to maximize community collaboration. Don't look for shortcuts There is no shortcut to GSoC selection and there is no shortcut to GSoC completion. The student stipend (USD $5,500 in 2014) is not paid to students unless they complete a minimum amount of valid code. This means that even if a student did find some shortcut to selection, it is unlikely they would be paid without completing meaningful work. If you are the right candidate for GSoC, you will not need a shortcut anyway. Are you the sort of person who can't leave a coding problem until you really feel it is fixed, even if you keep going all night? Have you ever woken up in the night with a dream about writing code still in your head? Do you become irritated by tedious or repetitive tasks and often think of ways to write code to eliminate such tasks? Does your family get cross with you because you take your laptop to Christmas dinner or some other significant occasion and start coding? If some of these statements summarize the way you think or feel you are probably a natural fit for GSoC. An opportunity money can't buy The GSoC stipend will not make you rich. It is intended to make sure you have enough money to survive through the summer and focus on your project. Professional developers make this much money in a week in leading business centers like New York, London and Singapore. When you get to that stage in 3-5 years, you will not even be thinking about exactly how much you made during internships. GSoC gives you an edge over other internships because it involves publicly promoting your work. Many companies still try to hide the potential of their best recruits for fear they will be poached or that they will be able to demand higher salaries. Everything you complete in GSoC is intended to be published and you get full credit for it. Imagine a young musician getting the opportunity to perform on the main stage at a rock festival. This is how the free software community works. It is a meritocracy and there is nobody to hold you back. Having a portfolio of free software that you have created or collaborated on and a wide network of professional contacts that you develop before, during and after GSoC will continue to pay you back for years to come. While other graduates are being screened through group interviews and testing days run by employers, people with a track record in a free software project often find they go straight to the final interview round. Register your domain name and make a permanent email address Free software is all about community and collaboration. Register your own domain name as this will become a focal point for your work and for people to get to know you as you become part of the community. This is sound advice for anybody working in IT, not just programmers. It gives the impression that you are confident and have a long term interest in a technology career. Choosing the provider: as a minimum, you want a provider that offers DNS management, static web site hosting, email forwarding and XMPP services all linked to your domain. You do not need to choose the provider that is linked to your internet connection at home and that is often not the best choice anyway. The XMPP foundation maintains a list of providers known to support XMPP. Create an email address within your domain name. The most basic domain hosting providers will let you forward the email address to a webmail or university email account of your choice. Configure your webmail to send replies using your personalized email address in the From header. Update your ~/.gitconfig file to use your personalized email address in your Git commits. Create a web site and blog Start writing a blog. Host it using your domain name. Some people blog every day, other people just blog once every two or three months. Create links from your web site to your other profiles, such as a Github profile page. This helps reinforce the pages/profiles that are genuinely related to you and avoid confusion with the pages of other developers. Many mentors are keen to see their students writing a weekly report on a blog during GSoC so starting a blog now gives you a head start. Mentors look at blogs during the selection process to try and gain insight into which topics a student is most suitable for. Create a profile on Github Github is one of the most widely used software development web sites. Github makes it quick and easy for you to publish your work and collaborate on the work of other people. Create an account today and get in the habbit of forking other projects, improving them, committing your changes and pushing the work back into your Github account. Github will quickly build a profile of your commits and this allows mentors to see and understand your interests and your strengths. In your Github profile, add a link to your web site/blog and make sure the email address you are using for Git commits (in the ~/.gitconfig file) is based on your personal domain. Start using PGP Pretty Good Privacy (PGP) is the industry standard in protecting your identity online. All serious free software projects use PGP to sign tags in Git, to sign official emails and to sign official release files. The most common way to start using PGP is with the GnuPG (GNU Privacy Guard) utility. It is installed by the package manager on most Linux systems. When you create your own PGP key, use the email address involving your domain name. This is the most permanent and stable solution. Print your key fingerprint using the gpg-key2ps command, it is in the signing-party package on most Linux systems. Keep copies of the fingerprint slips with you. This is what my own PGP fingerprint slip looks like. You can also print the key fingerprint on a business card for a more professional look. Using PGP, it is recommend that you sign any important messages you send but you do not have to encrypt the messages you send, especially if some of the people you send messages to (like family and friends) do not yet have the PGP software to decrypt them. If using the Thunderbird (Icedove) email client from Mozilla, you can easily send signed messages and validate the messages you receive using the Enigmail plugin. Get your PGP key signed Once you have a PGP key, you will need to find other developers to sign it. For people I mentor personally in GSoC, I'm keen to see that you try and find another Debian Developer in your area to sign your key as early as possible. Free software events Try and find all the free software events in your area in the months between now and the end of the next Google Summer of Code season. Aim to attend at least two of them before GSoC. Look closely at the schedules and find out about the individual speakers, the companies and the free software projects that are participating. For events that span more than one day, find out about the dinners, pub nights and other social parts of the event. Try and identify people who will attend the event who have been GSoC mentors or who intend to be. Contact them before the event, if you are keen to work on something in their domain they may be able to make time to discuss it with you in person. Take your PGP fingerprint slips. Even if you don't participate in a formal key-signing party at the event, you will still find some developers to sign your PGP key individually. You must take a photo ID document (such as your passport) for the other developer to check the name on your fingerprint but you do not give them a copy of the ID document. Events come in all shapes and sizes. FOSDEM is an example of one of the bigger events in Europe, linux.conf.au is a similarly large event in Australia. There are many, many more local events such as the Debian UK mini-DebConf in Cambridge, November 2015. Many events are either free or free for students but please check carefully if there is a requirement to register before attending. On your blog, discuss which events you are attending and which sessions interest you. Write a blog during or after the event too, including photos. Quantcast generously hosted the Ganglia community meeting in San Francisco, October 2013. We had a wild time in their offices with mini-scooters, burgers, beers and the Ganglia book. That's me on the pink mini-scooter and Bernard Li, one of the other Ganglia GSoC 2014 admins is on the right. Install Linux GSoC is fundamentally about free software. Linux is to free software what a tree is to the forest. Using Linux every day on your personal computer dramatically increases your ability to interact with the free software community and increases the number of potential GSoC projects that you can participate in. This is not to say that people using Mac OS or Windows are unwelcome. I have worked with some great developers who were not Linux users. Linux gives you an edge though and the best time to gain that edge is now, while you are a student and well before you apply for GSoC. If you must run Windows for some applications used in your course, it will run just fine in a virtual machine using Virtual Box, a free software solution for desktop virtualization. Use Linux as the primary operating system. Here are links to download ISO DVD (and CD) images for some of the main Linux distributions: If you are nervous about getting started with Linux, install it on a spare PC or in a virtual machine before you install it on your main PC or laptop. Linux is much less demanding on the hardware than Windows so you can easily run it on a machine that is 5-10 years old. Having just 4GB of RAM and 20GB of hard disk is usually more than enough for a basic graphical desktop environment although having better hardware makes it faster. Your experiences installing and running Linux, especially if it requires some special effort to make it work with some of your hardware, make interesting topics for your blog. Decide which technologies you know best Personally, I have mentored students working with C, C++, Java, Python and JavaScript/HTML5. In a GSoC program, you will typically do most of your work in just one of these languages. From the outset, decide which language you will focus on and do everything you can to improve your competence with that language. For example, if you have already used Java in most of your course, plan on using Java in GSoC and make sure you read Effective Java (2nd Edition) by Joshua Bloch. Decide which themes appeal to you Find a topic that has long-term appeal for you. Maybe the topic relates to your course or maybe you already know what type of company you would like to work in. Here is a list of some topics and some of the relevant software projects:
  • System administration, servers and networking: consider projects involving monitoring, automation, packaging. Ganglia is a great community to get involved with and you will encounter the Ganglia software in many large companies and academic/research networks. Contributing to a Linux distribution like Debian or Fedora packaging is another great way to get into system administration.
  • Desktop and user interface: consider projects involving window managers and desktop tools or adding to the user interface of just about any other software.
  • Big data and data science: this can apply to just about any other theme. For example, data science techniques are frequently used now to improve system administration.
  • Business and accounting: consider accounting, CRM and ERP software.
  • Finance and trading: consider projects like R, market data software like OpenMAMA and connectivity software (Apache Camel)
  • Real-time communication (RTC), VoIP, webcam and chat: look at the JSCommunicator or the Jitsi project
  • Web (JavaScript, HTML5): look at the JSCommunicator
Before the GSoC application process begins, you should aim to learn as much as possible about the theme you prefer and also gain practical experience using the software relating to that theme. For example, if you are attracted to the business and accounting theme, install the PostBooks suite and get to know it. Maybe you know somebody who runs a small business: help them to upgrade to PostBooks and use it to prepare some reports. Make something Make some small project, less than two week's work, to demonstrate your skills. It is important to make something that somebody will use for a practical purpose, this will help you gain experience communicating with other users through Github. For an example, see the servlet Juliana Louback created for fixing phone numbers in December 2013. It has since been used as part of the Lumicall web site and Juliana was selected for a GSoC 2014 project with Debian. There is no better way to demonstrate to a prospective mentor that you are ready for GSoC than by completing and publishing some small project like this yourself. If you don't have any immediate project ideas, many developers will also be able to give you tips on small projects like this that you can attempt, just come and ask us on one of the mailing lists. Ideally, the project will be something that you would use anyway even if you do not end up participating in GSoC. Such projects are the most motivating and rewarding and usually end up becoming an example of your best work. To continue the example of somebody with a preference for business and accounting software, a small project you might create is a plugin or extension for PostBooks. Getting to know prospective mentors Many web sites provide useful information about the developers who contribute to free software projects. Some of these developers may be willing to be a GSoC mentor. For example, look through some of the following: Getting on the mentor's shortlist Once you have identified projects that are interesting to you and developers who work on those projects, it is important to get yourself on the developer's shortlist. Basically, the shortlist is a list of all students who the developer believes can complete the project. If I feel that a student is unlikely to complete a project or if I don't have enough information to judge a student's probability of success, that student will not be on my shortlist. If I don't have any student on my shortlist, then a project will not go ahead at all. If there are multiple students on the shortlist, then I will be looking more closely at each of them to try and work out who is the best match. One way to get a developer's attention is to look at bug reports they have created. Github makes it easy to see complaints or bug reports they have made about their own projects or other projects they depend on. Another way to do this is to search through their code for strings like FIXME and TODO. Projects with standalone bug trackers like the Debian bug tracker also provide an easy way to search for bug reports that a specific person has created or commented on. Once you find some relevant bug reports, email the developer. Ask if anybody else is working on those issues. Try and start with an issue that is particularly easy and where the solution is interesting for you. This will help you learn to compile and test the program before you try to fix any more complicated bugs. It may even be something you can work on as part of your academic program. Find successful projects from the previous year Contact organizations and ask them which GSoC projects were most successful. In many organizations, you can find the past students' project plans and their final reports published on the web. Read through the plans submitted by the students who were chosen. Then read through the final reports by the same students and see how they compare to the original plans. Start building your project proposal now Don't wait for the application period to begin. Start writing a project proposal now. When writing a proposal, it is important to include several things:
  • Think big: what is the goal at the end of the project? Does your work help the greater good in some way, such as increasing the market share of Linux on the desktop?
  • Details: what are specific challenges? What tools will you use?
  • Time management: what will you do each week? Are there weeks where you will not work on GSoC due to vacation or other events? These things are permitted but they must be in your plan if you know them in advance. If an accident or death in the family cut a week out of your GSoC project, which work would you skip and would your project still be useful without that? Having two weeks of flexible time in your plan makes it more resilient against interruptions.
  • Communication: are you on mailing lists, IRC and XMPP chat? Will you make a weekly report on your blog?
  • Users: who will benefit from your work?
  • Testing: who will test and validate your work throughout the project? Ideally, this should involve more than just the mentor.
If your project plan is good enough, could you put it on Kickstarter or another crowdfunding site? This is a good test of whether or not a project is going to be supported by a GSoC mentor. Learn about packaging and distributing software Packaging is a vital part of the free software lifecycle. It is very easy to upload a project to Github but it takes more effort to have it become an official package in systems like Debian, Fedora and Ubuntu. Packaging and the communities around Linux distributions help you reach out to users of your software and get valuable feedback and new contributors. This boosts the impact of your work. To start with, you may want to help the maintainer of an existing package. Debian packaging teams are existing communities that work in a team and welcome new contributors. The Debian Mentors initiative is another great starting place. In the Fedora world, the place to start may be in one of the Special Interest Groups (SIGs). Think from the mentor's perspective After the application deadline, mentors have just 2 or 3 weeks to choose the students. This is actually not a lot of time to be certain if a particular student is capable of completing a project. If the student has a published history of free software activity, the mentor feels a lot more confident about choosing the student. Some mentors have more than one good student while other mentors receive no applications from capable students. In this situation, it is very common for mentors to send each other details of students who may be suitable. Once again, if a student has a good Github profile and a blog, it is much easier for mentors to try and match that student with another project. GSoC logo generic Conclusion Getting into the world of software engineering is much like joining any other profession or even joining a new hobby or sporting activity. If you run, you probably have various types of shoe and a running watch and you may even spend a couple of nights at the track each week. If you enjoy playing a musical instrument, you probably have a collection of sheet music, accessories for your instrument and you may even aspire to build a recording studio in your garage (or you probably know somebody else who already did that). The things listed on this page will not just help you walk the walk and talk the talk of a software developer, they will put you on a track to being one of the leaders. If you look over the profiles of other software developers on the Internet, you will find they are doing most of the things on this page already. Even if you are not selected for GSoC at all or decide not to apply, working through the steps on this page will help you clarify your own ideas about your career and help you make new friends in the software engineering community.

27 September 2015

Lunar: Reproducible builds: week 22 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes Packages fixed The following 22 packages became reproducible due to changes in their build dependencies: breathe, cdi-api, geronimo-jpa-2.0-spec, geronimo-validation-1.0-spec, gradle-propdeps-plugin, jansi, javaparser, libjsr311-api-java, mac-widgets, mockito, mojarra, pastescript, plexus-utils2, powerline, python-psutil, python-sfml, python-tldap, pythondialog, tox, trident, truffle, zookeeper. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet: diffoscope development The changes to make diffoscope run under Python 3, along with many small fixes, entered the archive with version 35 on September 21th. Another release was made the very next day fixed two encoding-related issues discovered when running diffoscope on more Debian packages. strip-nondeterminism development Version 0.12.0 now preserves file permissions on modified zip files and dh_strip_nondeterminism has been made compatible with older debhelper. disorderfs development Version 0.3.0 implemented a multi-user mode that was required to build Debian packages using disorderfs. It also added command line options to control the ordering of files in directory (either shuffled or reversed) and another to do arbitrary changes to the reported space used by files on disk. A couple days later, version 0.4.0 was released to support locks, flush, fsync, fsyncdir, read_buf, and write_buf. Almost all known issues have now been fixed. reproducible.debian.net disorderfs is now used during the second build. This makes file ordering issue very easy to identify as such. (h01ger) Work has been done on making the distributed build setup more reliable. (h01ger) Documentation update Matt Kraii fixed the example on how to fix issues related to dates in Sphinx. Recent Sphinx versions should also be compatible with SOURCE_DATE_EPOCH. Package reviews 53 reviews have been removed, 85 added and 13 updated this week. 46 packages failing to build from source has been identified by Chris Lamb, Chris West, and Niko Tyni. Chris Lamb was the lucky reporter of bug #800000 on vdr-plugin-prefermenu. Issues related to disorderfs are being tracked with a new issue.

23 September 2015

Daniel Pocock: The only way to ensure the VW scandal never happens again

The automotive industry has been in the spotlight after a massive scandal at Volkswagen, using code hidden in the engine management software to cheat emissions tests. What else is hidden in your new car's computer? Every technology we use in our lives is becoming more computerized, even light bulbs and toilet seats. In a large piece of equipment like a car, there are many opportunities for computerization. In most cases, consumers aren't even given a choice whether or not they want software in their car. It has long been known that such software is spying on the habits of the driver and this data is extracted from the car when it is serviced and uploaded to the car company. Car companies are building vast databases about the lifestyles, habits and employment activities of their customers. Computers aren't going away, so what can be done? Most people realize that computers aren't going to go away any time soon. That doesn't mean that people have to put up with these deceptions and intrusions on our lives. For years, many leading experts in the software engineering world have been promoting the benefits and principles of free software. What we mean by free is that users, regulators and other independent experts should have the freedom to see and modify the source code in the equipment that we depend on as part of modern life. In fact, experts generally agree that there is no means other than software freedom to counter the might of corporations like Volkswagen and their potential to misuse that power, as demonstrated in the emissions testing scandal. If Governments and regulators want to be taken seriously and protect society, isn't it time that they insisted that the car industry replaces all hidden code with free and open source software?

20 September 2015

Matthew Garrett: The Internet of Incompatible Things

I have an Amazon Echo. I also have a LIFX Smart Bulb. The Echo can integrate with Philips Hue devices, letting you control your lights by voice. It has no integration with LIFX. Worse, the Echo developer program is fairly limited - while the device's built in code supports communicating with devices on your local network, the third party developer interface only allows you to make calls to remote sites[1]. It seemed like I was going to have to put up with either controlling my bedroom light by phone or actually getting out of bed to hit the switch.

Then I found this article describing the implementation of a bridge between the Echo and Belkin Wemo switches, cunningly called Fauxmo. The Echo already supports controlling Wemo switches, and the code in question simply implements enough of the Wemo API to convince the Echo that there's a bunch of Wemo switches on your network. When the Echo sends a command to them asking them to turn on or off, the code executes an arbitrary callback that integrates with whatever API you want.

This seemed like a good starting point. There's a free implementation of the LIFX bulb API called Lazylights, and with a quick bit of hacking I could use the Echo to turn my bulb on or off. But the Echo's Hue support also allows dimming of lights, and that seemed like a nice feature to have. Tcpdump showed that asking the Echo to look for Hue devices resulted in similar UPnP discovery requests to it looking for Wemo devices, so extending the Fauxmo code seemed plausible. I signed up for the Philips developer program and then discovered that the terms and conditions explicitly forbade using any information on their site to implement any kind of Hue-compatible endpoint. So that was out. Thankfully enough people have written their own Hue code at various points that I could figure out enough of the protocol by searching Github instead, and now I have a branch of Fauxmo that supports searching for LIFX bulbs and presenting them as Hues[2].

Running this on a machine on my local network is enough to keep the Echo happy, and I can now dim my bedroom light in addition to turning it on or off. But it demonstrates a somewhat awkward situation. Right now vendors have no real incentive to offer any kind of compatibility with each other. Instead they're all trying to define their own ecosystems with their own incompatible protocols with the aim of forcing users to continue buying from them. Worse, they attempt to restrict developers from implementing any kind of compatibility layers. The inevitable outcome is going to be either stacks of discarded devices speaking abandoned protocols or a cottage industry of developers writing bridge code and trying to avoid DMCA takedowns.

The dystopian future we're heading towards isn't Gibsonian giant megacorporations engaging in physical warfare, it's one where buying a new toaster means replacing all your lightbulbs or discovering that the code making your home alarm system work is now considered a copyright infringement. Is there a market where I can invest in IP lawyers?

[1] It also requires an additional phrase at the beginning of a request to indicate which third party app you want your query to go to, so it's much more clumsy to make those requests compared to using a built-in app.
[2] I only have one bulb, so as yet I haven't added any support for groups.

comment count unavailable comments

19 September 2015

Philipp Kern: Working with z/VM DIRMAINT as a mere user

Two helpful CMS commands to know if the users on the z/VM host you connect to are managed using DIRMAINT:

REVIEW is something I always expire from my mind after a few weeks of not using a mainframe. And then I do not usually find it quickly, even knowing where to look. (For some reason GET won't work unless you are a privileged user of the system.)

I guess by now most of the systems use DIRMAINT, but it's an IBM product that requires a separate license in addition to z/VM. Hence on some systems the user directory is still maintained by hand. In this case the password is written verbatim into the file and the administrator needs to change it manually for you.

18 September 2015

Russ Allbery: Review: The Three-Body Problem

Review: The Three-Body Problem, by Cixin Liu & Ken Liu
Series: Three-Body Problem #1
Author: Liu Cixin ( )
Translator: Ken Liu
Publisher: Tor
Copyright: 2006, 2014
Printing: November 2014
ISBN: 1-4668-5344-1
Format: Kindle
Pages: 399
Liu Cixin ( ) is one of the best-known and most popular SF authors in China, but, due to the paucity of translated SF in the English-speaking world, was largely unknown to English-speaking SF readers. That made this translation by Ken Liu (no relation) highly anticipated: a window into a large world of SF by authors most of us have never read. And Ken Liu is an excellent writer in his own right (see, for instance, "The Literomancer" or "Altogether Elsewhere, Vast Herds of Reindeer"), which was a hopeful sign for a good translation. Indeed, the translation is very good. One can tell that the book comes from a different literary tradition, but primarily from unusual (to US readers, at least) emotional focal points for the characters and lots of Chinese history that the author feels little need to explain. (Ken Liu adds translator notes for the bits most likely to lose non-Chinese audiences.) I found the characters a bit odd and occasionally hard to identify with, but I didn't mind: it added to the feeling of exploring a slightly different literary tradition with different character conventions. Unfortunately, the end of the book went entirely off the rails, at least for me. The Three-Body Problem opens during the Cultural Revolution, with the murder of Ye Wenjie's father, a physics professor, and the suicide of her advisor and friend. Two years later, she's at a work camp in inner Mongolia, cutting trees, numb, keeping her head down. But a traveling journalist smuggles in a secret copy of Silent Spring and gives it to her. The book both hardens her own belief in the evil of humanity and gets her into deeper trouble when she agrees to recopy a draft of a letter to the central leadership about the environmental devastation caused by the logging. Ye Wenjie is forced to choose permanent exile to an experimental radar facility on a remote mountain in Mongolia. Ye Wenjie is vital to the slowly-developing plot of the book, but this is the last the reader sees of her for some time. Part II jumps to present time and to a different protagonist: Wang Miao, a distinguished nanotechnology researcher who is recruited by the PLA and the police into a secret war room. Distinguished scientists are killing themselves in frighteningly large numbers, including one that Wang Miao had a crush on. She left a suicide note before overdosing on sleeping pills, one that says only, cryptically, that physics doesn't exist. The government is desperate to figure out what's going on. The "radar" facility and the suicide of scientists are linked, of course, but it takes much of the book for Wang Miao and his colleagues to piece together how. Key clues come in the form of an odd puzzle virtual reality MMORPG that becomes a popular sensation. In it, players find themselves on an alien world whose suns rise and set at strange and unpredictable intervals: sometimes far too far away and horribly cold, sometimes far too close and destructively hot, and sometimes not seen for long periods of time. The players can somehow dehydrate themselves and the rest of the population to weather the worst seasons, but life is a constant struggle against apparently unpredictable elements. Despite that, players slowly find ways to build civilizations and attempt to predict the strange cycles of heat and cold. It takes the characters a long time to understand what's happening. Readers will probably be much faster, since they have out-of-band information the characters don't. At first, this game seems entirely unrelated to the suicides of the scientists or, for that matter, to Ye Wenjie, who is easy to almost forget. But they all weave together in a way that's almost compelling. I say almost because the motivations of the characters are quite tangled and interesting, but the science is... awful. Those who have read my other book reviews know that I don't insist on hard science in my SF. Usually I'm willing to roll with it, and I can read past even obvious scientific nonsense. But science is critical to the plot of this book; some very specific actions, with a detailed in-book scientific justifications, are the underpinning of the entire plot. And they make utter nonsense of physics. I've rarely had science in a novel destroy my suspension of disbelief this thoroughly. Physics isn't even alone: biology, chemistry, computing, and astronomy are all in for it as Liu reveals the causes of the book's events. I wanted to be engaged by the conflict of ethics and philosophy that drive the plot, and some of that conflict is quite interesting. But every other page would have another colossal pile of scientific nonsense that would throw me out of the book again, usually in the service of a set piece that missed grand and landed on absurdly silly. Be warned that this book also ends on a bit of a cliff-hanger, and there are two more books in the series. The second has been released in a translated edition, but with a different translator, which is disappointing (Ken Liu did an excellent job); the third isn't out yet. If you can get past the awful science, The Three-Body Problem is doing some interesting things. The characters are a bit wooden and forgettable, but the philosophical problems they encounter are ones I don't see often in western SF. This a refreshing change, and I'm interested enough that I'm still considering picking up the sequels. But the science hurts the book so badly. Followed by The Dark Forest. Rating: 6 out of 10

17 September 2015

Norbert Preining: Debian/TeX Live 2015.20150917-1

Usual regular update of the TeX Live packages. Now that Debian/unstable is back to full development speed thanks to the end of the bit gcc transition, probably this TeX Live update will be lost between the hundreds of others. Anyway a few interesting changes are still in there. Debian - TeX Live 2015 Most importantly the fix for unicode-math that was broken for a short time, and for the Japanese users it seems that the LaTeX3 packages have gained support for upTeX. Good to hear. Of course, these are not the only updates and newcomers, see the list below. Updated packages abc, acro, alegreya, animate, arabxetex, babel-bosnian, babel-french, babel-greek, babel-latin, babel-welsh, beamer-FUBerlin, bidi, bigfoot, blochsphere, bxjscls, chemformula, chemmacros, chet, classicthesis, crossrefware, csplain, ctable, ctanify, datetime2, datetime2-basque, detlev-cm, disser, doclicense, drm, dtxgen, dtxtut, dvips, ecclesiastic, eledmac, embrac, etex-pkg, etoc, factura, fbb, fithesis, font-change, fontname, glossaries, gothic, greek-fontenc, IEEEtran, l3build, l3experimental, l3kernel, l3packages, latexconfig, lettrine, luatexja, mathastext, mcf2graph, media9, metrix, mhequ, minted, moderntimeline, newtx, ocgx2, pagecolor, pdftex, pgf, pstricks, reledmac, resphilosophica, roboto, shdoc, siunitx, suftesi, tetex, tex4ht, texlive-scripts, tikz-bayesnet, translations, ucharcat, udesoftec, ulthese, unicode-math, ut-thesis, withargs, xint. New packages checklistings, cleanthesis, easyreview, fei, medstarbeamer, mfirstuc, nevelok, old-arrows, proofread, uassign, xebaposter. Enjoy.

Russ Allbery: Review: How to Make Your Dreams Come True

Review: How to Make Your Dreams Come True, by Mark Forster
Publisher: Hodder
Copyright: 2002
Printing: 2006
ISBN: 0-340-78629-9
Format: Kindle
Pages: 208
Mark Forster is my favorite time management author. His books are less comprehensive systems and more collections of tips, mental shortcuts, and ways of thinking about time management and prioritization that tend to resonate for me. I've already read and reviewed several other books by Forster, but hadn't read this one because it was thoroughly out of print (at least in the US). When it turned up for the Kindle, I decided to give it a try, if for no other reason than completionism. After reading How to Make Your Dreams Come True, I can see why it fell out of print. It's a rather odd book, and quite different from Forster's other work. The topic is still time management, if you squint at it properly, but in the form of a philosophical discussion of life goals and becoming an artist who creates one's own life. Forster structures the book as a series of dialogues between Forster in the present and his own future self and takes the mental exercise surprisingly seriously, even comparing his own coaching technique with that of his future self. The levels of meta get even deeper towards the end of the book, where Forster writes a dialogue between his future self and his even more future self. Those drift into philosophy of life, high-level (and personal) life goals, and some speculations about the reason for existence. The primary concept Forster is exploring is the difference between push, pull, and drift modes of life. He argues that a lot of time management falls into the category of push systems: you're exerting effort to push yourself along towards goals. In this book, he argues that this is a starting point but not a technique to stop on. Instead, one should try to move to pull mode, where one has a compelling and detailed vision of where one wants to be, repeatedly developing and refining it, and then lets that vision pull you through life. The advantages, in his conception, over the push form of time management is that this requires less expenditure of effort, feels more natural, and lets one be more flexible and nimble in how one achieves one's goals. It can be tricky, however, to avoid settling into the middle ground of drift mode, where one does not have a goal in mind and doesn't act towards anything in particular. I found the idea interesting and mildly useful, but Forster's presentation of it seemed too convinced of its own correctness. The exercises he proposes don't seem interesting to me; the dialogues with one's future self in particular felt very gimmicky. I admire his ability to invent some of the interactions he came up with, but I don't think it would be a useful technique for me, or as universal as he seems to think it could be. Forster talks quite a lot about reducing effort in his blog (somewhat less in his other books), and seems constantly convinced that he's found a new way to reduce the sense of effort around time management. After having tried a few of these, I suspect that new systems feel less effortful than older systems, and that it's the constant changing of systems that reduces the feeling of effort more than the details of any specific system. I can develop a lot of enthusiasm about a new system for tracking work, and enthusiasm makes everything feel like it requires less effort. Other than that central idea, and quite a lot of philosophy that didn't particularly speak to me, this book reiterates a few of Forster's ideas in a less specific framework: adopt a process and trust it, use it for long enough to give it a chance to work, keep refreshing your vision, and try to reach a flow where you're not exerting lots of effort to force yourself to do things. Good ideas, but better, and more practically, presented in Forster's other books. I'd characterize this as an odd and minor work. I like Forster's writing in general and still only read this for the sake of completeness. His other books offer more practical and actionable ideas. Unless this idea and method of presentation really speaks to you, I think you'll get more out of other books. Rating: 5 out of 10

15 September 2015

Norbert Preining: In Memoriam Adrian Frutiger

Just a few days ago another great font designer has passed away, Adrian Frutiger, 24 May 1928 10 September 2015. Famous in particular for his fonts Frutiger, Avenir, and the Univers family. His fonts are used throughout the worlds for visual communication like road signs. Much has been written about him: Linotype s excellent obituary, Adam Twardoch, David Airey, New York Times article. For an interview conducted with Frutiger in 1999, see this Eye Magazine article. An interesting article on Frutiger s typefaces by Charles Bigelow.
frutiger1 Here I want to recall a different quality of Adrian Frutiger his interest in the visual language of symbols, their development and interaction. His book Der Mensch und seine Zeichen (Signs and Symbols) is a profound study on the development, history, and use of all kind of symbols. In this book, translated into many languages, Frutiger explores the depth and breath of symbols, but the most important part for him, easily to be seen from the German title, is the human part ( Der Mensch ). Symbols are created, changed, and used by and for humans. His studies exhibit connections between various cultures when it comes to sign usage and design.
frutiger5 He also explores signets as modern sign language, and their importance for visual communcation and identity building.
frutiger4 The book also discusses the development of the Roman alphabet, and of course passes the Univers font family.
frutiger2 My favorite piece is this overlay of a s from different fonts, to show that there is a common pattern in these fonts.
frutiger3 He concludes this book with a very wise statement:
Zum Festhalten des Gedankens, zur Vermittlung der Aussage gen gen seit langem schon die Alphabet-Zeichen nicht mehr allein. Orientierung und Kommunikation sind heute unm glich ohne Schemata, Zeichen und Signale. Der geschriebene Ausdruck wird durch die Bildvermittlung notwendig erg nzt.
SInce long the alphabet letters alone do not suffice to take hold of thoughts and to transmit statements. Nowadays, orientation and communication are impossible without schemata, signs and signals. The written expression is necessarily enhanced through image transmission.
(my translation) After Hermann Zapf, this is now the second loss of the typographic world in this year. We will miss both of them.

1 September 2015

Lunar: Reproducible builds: week 18 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes Aur lien Jarno uploaded glibc/2.21-0experimental1 which will fix the issue were locales-all did not behave exactly like locales despite having it in the Provides field. Lunar rebased the pu/reproducible_builds branch for dpkg on top of the released 1.18.2. This made visible an issue with udebs and automatically generated debug packages. The summary from the meeting at DebConf15 between ftpmasters, dpkg mainatainers and reproducible builds folks has been posted to the revelant mailing lists. Packages fixed The following 70 packages became reproducible due to changes in their build dependencies: activemq-activeio, async-http-client, classworlds, clirr, compress-lzf, dbus-c++, felix-bundlerepository, felix-framework, felix-gogo-command, felix-gogo-runtime, felix-gogo-shell, felix-main, felix-shell-tui, felix-shell, findbugs-bcel, gco, gdebi, gecode, geronimo-ejb-3.2-spec, git-repair, gmetric4j, gs-collections, hawtbuf, hawtdispatch, jack-tools, jackson-dataformat-cbor, jackson-dataformat-yaml, jackson-module-jaxb-annotations, jmxetric, json-simple, kryo-serializers, lhapdf, libccrtp, libclaw, libcommoncpp2, libftdi1, libjboss-marshalling-java, libmimic, libphysfs, libxstream-java, limereg, maven-debian-helper, maven-filtering, maven-invoker, mochiweb, mongo-java-driver, mqtt-client, netty-3.9, openhft-chronicle-queue, openhft-compiler, openhft-lang, pavucontrol, plexus-ant-factory, plexus-archiver, plexus-bsh-factory, plexus-cdc, plexus-classworlds2, plexus-component-metadata, plexus-container-default, plexus-io, pytone, scolasync, sisu-ioc, snappy-java, spatial4j-0.4, tika, treeline, wss4j, xtalk, zshdb. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet: Chris Lamb also noticed that binaries shipped with libsilo-bin did not work. Documentation update Chris Lamb and Ximin Luo assembled a proper specification for SOURCE_DATE_EPOCH in the hope to convince more upstreams to adopt it. Thanks to Holger it is published under a non-Debian domain name. Lunar documented easiest way to solve issues with file ordering and timestamps in tarballs that came with tar/1.28-1. Some examples on how to use SOURCE_DATE_EPOCH have been improved to support systems without GNU date. reproducible.debian.net armhf is finally being tested, which also means the remote building of Debian packages finally works! This paves the way to perform the tests on even more architectures and doing variations on CPU and date. Some packages even produce the same binary Arch:all packages on different architectures (1, 2). (h01ger) Tests for FreeBSD are finally running. (h01ger) As it seems the gcc5 transition has cooled off, we schedule sid more often than testing again on amd64. (h01ger) disorderfs has been built and installed on all build nodes (amd64 and armhf). One issue related to permissions for root and unpriviliged users needs to be solved before disorderfs can be used on reproducible.debian.net. (h01ger) strip-nondeterminism Version 0.011-1 has been released on August 29th. The new version updates dh_strip_nondeterminism to match recent changes in debhelper. (Andrew Ayer) disorderfs disorderfs, the new FUSE filesystem to ease testing of filesystem-related variations, is now almost ready to be used. Version 0.2.0 adds support for extended attributes. Since then Andrew Ayer also added support to reverse directory entries instead of shuffling them, and arbitrary padding to the number of blocks used by files. Package reviews 142 reviews have been removed, 48 added and 259 updated this week. Santiago Vila renamed the not_using_dh_builddeb issue into varying_mtimes_in_data_tar_gz_or_control_tar_gz to align better with other tag names. New issue identified this week: random_order_in_python_doit_completion. 37 FTBFS issues have been reported by Chris West (Faux) and Chris Lamb. Misc. h01ger gave a talk at FrOSCon on August 23rd. Recordings are already online. These reports are being reviewed and enhanced every week by many people hanging out on #debian-reproducible. Huge thanks!

30 August 2015

Philipp Kern: Automating the 3270 part of a Debian System z install

If you try to install Debian on System z within z/VM you might be annoyed at the various prompts it shows before it lets you access the network console via SSH. We can do better. From within CMS copy the default EXEC and default PARMFILE:

COPYFILE DEBIAN EXEC A DEBAUTO EXEC A
COPYFILE PARMFILE DEBIAN A PARMFILE DEBAUTO A

Now edit DEBAUTO EXEC A and replace the DEBIAN in 'PUNCH PARMFILE DEBIAN * (NOHEADER' with DEBAUTO. This will load the alternate kernel parameters file into the card reader, while still loading the original kernel and initrd files.

Replace PARMFILE DEBAUTO A's content with this (note the 80 character column limit):

ro locale=C
s390-netdevice/choose_networktype=qeth s390-netdevice/qeth/layer2=true
s390-netdevice/qeth/choose=0.0.fffc-0.0.fffd-0.0.fffe
netcfg/get_ipaddress=<IPADDR> netcfg/get_netmask=255.255.255.0
netcfg/get_gateway=<GW> netcfg/get_nameservers=<FIRST-DNS>
netcfg/confirm_static=true netcfg/get_hostname=debian
netcfg/get_domain=
network-console/authorized_keys_url=http://www.kern.lc/id_rsa.pub
preseed/url=http://www.kern.lc/preseed-s390x.cfg

Replace <IPADDR>, <GW>, and <FIRST-DNS> to suit your local network config. You might also need to change the netmask, which I left in for clarity about the format. Adjust the device address of your OSA network card. If it's in layer 3 mode (very likely) you should set layer2=false. Note that mixed case matters, hence you will want to SET CASE MIXED in xedit.

Then there are the two URLs that need to be changed. The authorized_keys_url file contains your SSH public key and is fetched unencrypted and unauthenticated, so be careful what networks you traverse with your request (HTTPS is not supported by debian-installer in Debian).

preseed/url is needed for installation parameters that do not fit the parameters file - there is an upper character limit that's about two lines longer than my example. This is why this example only contains the bare minimum for the network part, everything else goes into this preseeding file. It file can optionally be protected with a MD5 checksum in preseed/url/checksum.

Both URLs need to be very short. I thought that there was a way to specify a line continuation, but in my tests I was unable to produce one. Hence it needs to fit on one line, including the key. You might want to use an IPv4 as the hostname.

To skip the initial boilerplate prompts and to skip straight to the user and disk setup you can use this as preseed.cfg:

d-i debian-installer/locale string en_US
d-i debian-installer/country string US
d-i debian-installer/language string en
d-i time/zone US/Eastern
d-i mirror/country manual
d-i mirror/http/mirror string httpredir.debian.org
d-i mirror/http/directory string /debian
d-i mirror/http/proxy string

I'm relatively certain that the DASD disk setup part cannot be automated yet. But the other bits of the installation should be preseedable just like on non-mainframe hardware.

18 August 2015

Matthew Garrett: Canonical's deliberately obfuscated IP policy

I bumped into Mark Shuttleworth today at Linuxcon and we had a brief conversation about Canonical's IP policy. The short summary:
  • Canonical assert that the act of compilation creates copyright over the binaries, and you may not redistribute those binaries unless (a) the license prevents Canonical from restricting redistribution (eg, the GPL), or (b) you follow the terms of their IP policy. This means that, no matter what Dustin's blogpost says, Canonical's position is that you must ask for permission before distributing any custom container images that contain Ubuntu binaries, even if you use no Ubuntu trademarks in the process. Doing so without their permission is an infringement of their copyright.
  • Canonical have no intention of clarifying their policy, because Canonical benefit from companies being legally uncertain as to whether they have permission to do something or not.
  • Mark justifies maintaining this uncertainty by drawing an analogy between it and the perceived uncertainties that exist around certain aspects of the GPL. I disagree with this analogy pretty strongly. One of the main reasons for the creation of GPLv3 was to deal with some more ambiguous aspects of GPLv2 (such as what actually happened after license termination and how patents interacted with the GPL). The FSF publish a large FAQ intended to provide further clarity. The major ambiguity is in what a derivative work actually is, which is something the FSF can't answer absolutely (that's going to be up to courts) but will give its opinion on when asked. The uncertainties in Canonical's IP policy aren't a result of a lack of legal clarity - they're a result of Canonical's refusal to answer questions.

The even shorter summary: Canonical won't clarify their IP policy because they believe they can make more money if they don't.

Why do I keep talking about this? Because Canonical are deliberately making it difficult to create derivative works, and that's one of the core tenets of the definition of free software. Their IP policy is fundamentally incompatible with our community norms, and that's something we should care about rather than ignoring.

comment count unavailable comments

16 August 2015

Lunar: Reproducible builds: week 16 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes
  • St phane Glondu uploaded dh-ocaml/1.0.10 which now generates *.info with a deterministic order. Original patch by Chris Lamb. St phane also uploaded ocaml/4.02.3-1 to experimental which enables ocamldoc to build reproducible manpages using a patch by Valentin Lorentz.
  • Paul Gevers uploaded pasdoc/0.14.0-1 with upstream changes which should make the generated documentation reproducible by default.
  • Osamu Aoki uploaded debiandoc-sgml/1.2.31-1 adding spport for a DEBIANDOC_DATE environment variable to override the content of the <date> tag.
  • Dmitry Shachnev uploaded sphinx/1.3.1-4 which fixes many reproducibility issues and add support for SOURCE_DATE_EPOCH.
Valentin Lorentz sent a patch for ispell to initialize memory structures before dumping their content. In our experimental repository, qt4-x11 has been rebased on the latest version (Dhole), as was doxygen (akira). Packages fixed The following packages became reproducible due to changes in their build dependencies: backup-manager, cheese, coinor-csdp, coinor-dylp, ebook-speaker, freefem, indent, libjbcrypt-java, qtquick1-opensource-src, ruby-coffee-script, ruby-distribution, schroot, twittering-mode. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet:
  • #795203 on xlsx2csv by Dhole: set PODDATE to the date of the latest debian/changelog entry.
  • #795392 on blkreplay by Dhole: tell pod2man to use the date of the latest debian/changelog entry.
  • #795394 on libitpp by Dhole: use SOURCE_DATE_EPOCH as source for the manpage date instead of the currentdate.
  • #795395 on adblock-plus by Dhole: set TZ to UTC when using zip.
  • #795438 on wims-extra by Chris Lamb: ask grep to cope with non-UTF8 files.
  • #795441 on aprx by Chris Lamb: use SOURCE_DATE_EPOCH as source for the manpage date instead of the currentdate.
  • #795462 on ogre-1.9 by Chris Lamb: removes timestamps from the documentation system.
  • #795484 on ruby-rmagick by Chris Lamb: patch the bindings to allow the PRNG seed to be set, and set it to a fixed value when generating examples.
  • #795562 on htp by Chris Lamb: export TZ=UTC in debian/rules.
akira found another embedded code copy of texi2html in maxima. reproducible.debian.net Work on testing several architectures has continued. (Mattia/h01ger) Package reviews 29 reviews have been removed, 187 added and 34 updated this week. 172 new FTBFS reports were filled, 137 solely by Chris West (Faux). josch spent time investigating the issue with fonts in PDF files. Chris Lamb documented the issue affecting documentation generated by ocamldoc. Misc. Lunar presented a general Reproducible builds HOWTO talk at the Chaos Communication Camp 2015 in Germany on August 13th. Recordings are already available, as well as slides and script. h01ger and Lunar also used CCCamp15 as an opportunity to have discussions with members of several different projects about reproducible builds. Good news should be coming soon.

13 August 2015

Matthew Palmer: Your "Infrastructure as Code" is still code!

Whether you re a TDD zealot, or you just occasionally write a quick script to reproduce some bug, it s a rare coder who doesn t see value in some sort of automated testing. Yet, somehow, in all of the new-age Infrastructure as Code mania, we appear to have forgotten this, and the tools that are commonly used for implementing Infrastructure as Code have absolutely woeful support for developing your Infrastructure Code. I believe this has to change. At present, the state of the art in testing system automation code appears to be, spin up a test system, run the manifest/state/whatever, and then use something like serverspec or testinfra to SSH in and make sure everything looks OK . It s automated, at least, but it isn t exactly a quick process. Many people don t even apply that degree of rigour to their system config systems, and rely on manual testing, or even just doing it live! , to shake out the bugs they ve introduced. Speed in testing is essential. As the edit-build-test-debug cycle gets longer, frustration grows exponentially. If it takes two minutes to get a something went wrong report out of my tests, I m not going to run them very often. If I m not running my tests very often, then I m not likely to write tests much, and suddenly oops. Everything s on fire. In traditional software development, the unit tests are the backbone of the fast feedback cycle. You re running thousands of tests per second, ideally, and an entire test run might take 5-10 seconds. That s the sweet spot, where you can code and test in a rapid cycle of ever-increasing awesomeness. Interestingly, when I ask the users of most infrastructure management systems about unit testing, they either get a blank look on their face, or, at best, point me in the direction of the likes of Test Kitchen, which is described quite clearly as an integration platform, not a unit testing platform. Puppet has rspec-puppet, which is a pretty solid unit testing framework for Puppet manifests although it isn t widely used. Others, though nobody seems to have any ideas. The blank look is near-universal. If infrastructure developers want to be taken seriously, we need to learn a little about what s involved in the development part of the title we ve bestowed upon ourselves. This means knowing what the various types of testing are, and having tools which enable and encourage that testing. It also means things like release management, documentation, modularity and reusability, and considering backwards compatibility. All of this needs to apply to everything that is within the remit of the infrastructure developer. You don t get to hand-wave away any of this just because it s just configuration! . This goes double when your just configuration! is a hundred lines of YAML interspersed with templating directives (SaltStack, I m looking at you).

6 August 2015

Eduard Sanou: Reproducible builds on Debian for GSoC 2015, 1st update

This is the second blog post I m writing about my experiences contributing to Debian for Google Summer of Code 2015 (check my first post)

Status update

First month It s been two months and a few days since the GSoC started. During the first month I worked on fixing specific packages, mainly concerning issues with timestamps, which is a very common source of unreproducibility. In many cases, during the build process files are compressed into gzip or zip archives, which store the creation time of files in the metadata. This can lead to unreproducible results when there is timezone variation between builds (easily fixed setting the timezone to UTC before the compression happens). In some cases the compressed files are generated during the build, and thus add build times in the metadata of compressed files (in this case the creation date of the files needs to be normalized somehow). As explained in my application, I finished exams on the end of June, that s why I chose to work on small fixes first, so that I could make the most out of my time between studying and finishing university projects and reports. I m happy with my first month, as I have worked as originally planned. Actually, my estimation of the number of bugs I could submit every week was surpassed in reality!

Second month Once the university was over, I started dedicating myself fully to the project. This allowed me to start working on toolchain fixes, following my original plan on working with timestamp related issues. In particular I have been working a lot in implementing a proposal for deterministic timestamps that appeared in the reproducible builds project. The idea is to define an environment variable called SOURCE_DATE_EPOCH that contains a known timestamp in Unix epoch format. With this variable exported, tools that would embed the localtime in their generated or processed files, can use this externally supplied date. This would happen only if SOURCE_DATE_EPOCH is exported, so the behaviour of the tool wouldn t change if the variable is not set. The first package I patched to implement this behaviour was gcc. The reason behind this is that there are about 420 unreproducible packages due to using the __DATE__, __TIME__ and __TIMESTAMP__ C macros. My patch changes the behavior of the macros __DATE__ and __TIME__ if SOURCE_DATE_EPOCH is exported. I submitted this patch to the gcc-patches list. Even though there was some interesting discussions in the list, the patch has not been accepted yet. Seeing how the reproducible builds idea is gaining momentum and becoming widespread, I m positive that at some point the gcc people will be more receptive for such patch. The second work with SOURCE_DATE_EPOCH was in debhelper; I patched this building tool to export the variable with the latest debian/changelog entry timestamp. With this patch, all the tools that run under dh will be able to use it to embed deterministic timestamps. Unfortunately some parts of the build process of some packages don t happen under debhelper, so the variable needs to be exported in a different way. Having submitted the debhelper patch allowed many packages to become reproducible after the tools that embedded timestamps were patched to honour SOURCE_DATE_EPOCH. As of today, the toolchain packages I have patched to do that are: gcc, libxslt, gettext, ghostscript and qt4-x11 (qhelpgenerator). I have also continued working on fixing individual packages affected by timestamps, random orderings (such as the ones from listing hash keys) and locale depending orderings; I have tagged packages in our infrastructure to note what kind of issue makes them unreproducible; I have updated some parts of the Reproducible Builds Wiki.

Impressions about reproducible builds The work I did during the first month felt a bit tedious sometimes: it didn t require much creativity or thinking as most of the fixes where quite mechanical, following a recipe. After I became free from university duties, I started looking into less trivial issues, which require more deep investigation and feel more rewarding once they are fixed. I also worked on toolchain fixes, which need more work and need more care. Fixing toolchain packages feels particularly rewarding because they can cause many packages to become reproducible at once. There is a very active community in the reproducible builds project! It s great to see so many people contributing to this project spending so many hours and putting so much effort. I ve felt very welcome from the beginning and I have gotten kind replies and helpful answers to all the questions and doubts I ve had, both from my mentor and from the other people in the project. I want to add that I m still amazed by the awesome infrastructure set up for the reproducible builds project. The team is using a Jenkins machine to continuously build packages to check for reproducibility, with irc notifications for changes, and also with a really useful web interface to list all the packages organized by issues that allows exploring them individually with all the available information. Also not only the infrastructure is used to build Debian amd64 packages, but also FreeBSD, NetBSD, OpenWRT, coreboot and lately Debian armhf with the help of a few new arm nodes.

Impressions about working on a free software project This was my first time working on a community driven free software project and I ve learned so many things. Something I learned during the first days, which is even more present in such wide project like the reproducible builds, is that contributing is not just writing patches for the features you want; you also need to convince the maintainer of the package to accept the patch! I was a bit surprised at the beginning because even if this is a Debian project, that aims to make changes to the whole distribution, decisions are not absolute for the whole Debian project. After taking decisions within the reproducible builds teams on how to approach things, we need to convince the rest of the Debian developers (mainly maintainers) to follow them, as they are allowed to disagree. So it is required to work together for solutions that makes everyone comfortable, usually with discussions on mailing lists or irc. There is another fact that I wasn t expecting before getting involved in this project. The kind of teamwork I have done previously involves having a leader who decides how stuff is done, who takes decisions when needed and oversees the whole project. There seems to be a different philosophy in Debian. Instead of having a leader, everyone tries their best, trying to convince the others that their solution is good, often by showing an implementation of the solution and providing proof that it works, rather than trying to get the solution accepted before coding it. Also, solutions and ideas are valued for their quality rather than by the position of the person submitting them, and there is no hierarchy within the group: all comments and advices are taken equally, valuing their usefulness regardless of who gives them. Usually there are no votes when deciding things. Members try their best on their approaches, trying to convince others as best as they can. And even if someone disagrees they may end up accepting the solution if they don t manage to convince the original proposer of doing things differently. The idea is to spend more time working and coding than arguing and deciding on the way to do things. So far I ve seen this approach to be very efficient :) I ve been told by my mentor that for difficult cases there exist a Debian committee that helps mediate on disagreements, but that is only used as a last option, probably when the discussion gets heated up.

Personal experience Overall I m very happy to finally having set my foot in the free software community, where I m able to contribute to the kind of software I have been using for years. The sense of community in Debian is really big, and everyone is invited to participate. I think that Google is doing an awesome job with the Google Summer of Code, not only because it gives a lot to free software but because it helps students to join the free software world as contributors (which is something that may be difficult to get into when you don t know how to begin, as it happened to myself for some time). I plan to continue contributing to the free software world, and I d encourage anyone to find projects to get involved and contribute as well! Happy hacking to everyone!

30 July 2015

DebConf team: DebConf15 Schedule Published and Additional Featured Speakers Announced (Posted by DebConf Content Team)

DebConf15 Schedule The DebConf content team is pleased to announce the schedule of DebConf15, the forthcoming Debian Developers Conference. From a total of nearly 100 talk submissions, the team selected 75 talks. Due to the high number of submissions, several talks had to be shortened to 20 minute slots, of which a total of 30 talks have made it to the schedule. In addition, around 50 meetings and discussions (BoFs) have been submitted so far, as well as several other events like lightning talk sessions, live demos, a movie screening, a poetry night or stand-up comedy. The Schedule is available online at the DebConf15 conference site. Further changes to the schedule can and will be made, but today s announcement represents the first stable version. Featured Speakers In addition to the previously announced invited speakers, the content team also announces the following list of additional featured speakers: The full list of invited and featured speakers, including the invited speakers profiles and the titles of their talks is available here.

Next.

Previous.