Search Results: "ag"

27 April 2015

Russ Allbery: rra-c-util 5.7

This release has a somewhat random collection of accumulated fixes. A couple of them are for the PAM libraries: new support for the Mac OS X PAM implementation, which doesn't use the same options and error codes as the rest of the world. It also uses a different pattern of const declaration, which required some additional Autoconf probing for the fake PAM library for testing. There are also a few fixes for the systemd probe framework: libsystemd-daemon has been rolled into libsystemd in current versions, and the probe was using $(), which doesn't work on Solaris 10. The Kerberos Autoconf macros should now hopefully work with the version of Kerberos bundled with Solaris 10. Finally, this release supports checking for Clang as a compiler and choosing compiler warning flags accordingly, although rra-c-util isn't warning-free with Clang -Weverything yet. You can get the latest release from the rra-c-util distribution page.

Russ Allbery: C TAP Harness 3.3

The primary purpose of this release is to merge work by D. Brashear on making it easier to show verbose test success and failure output. Now, if runtests is run with the -v option, or the C_TAP_VERBOSE environment variable is set, the full output of each test case will be printed. The output is currently pretty messy, and I hope to improve it in the future, but it's a start. This release also supports compilation with Clang with all warnings enabled, and resolves all warnings shown by -Weverything (except the ones about padding, because that's not actually a useful warning). It includes new machinery to detect the compiler and switch warning flags. I will hopefully get a chance to go through my other projects and make them build cleanly with Clang, but it requires adding casts for any conversion from unsigned to signed or vice versa, so that may take a while. You can get the latest version from the C TAP Harness distribution page.

Steve Kemp: Validating puppet manifests via git hooks.

It looks like I'll be spending a lot of time working with puppet over the coming weeks. I've setup some toy deployments on virtual machines, and have converted several of my own hosts to using it, rather than my own slaughter system. When it comes to puppet some things are good, and some things are bad, as exected, and as any similar tool (even my own). At the moment I'm just aiming for consistency and making sure I can control all the systems - BSD, Debian GNU/Linux, Ubuntu, Microsoft Windows, etc. Little changes are making me happy though - rather than using a local git pre-commit hook to validate puppet manifests I'm now doing that checking on the server-side via a git pre-receive hook. Doing it on the server-side means that I can never forget to add the local hook and future-colleagues can similarly never make this mistake, and commit malformed puppetry. It is almost a shame there isn't a decent collection of example git-hooks, for doing things like this puppet-validation. Maybe there is and I've missed it.
It only crossed my mind because I've had to write several of these recently - a hook to rebuild a static website when the repository has a new markdown file pushed to it, a hook to validate syntax when pushes are attempted, and another hook to deny updates if the C-code fails to compile.

26 April 2015

Erich Schubert: Your big data toolchain is a big security risk!

This post is a follow-up to my earlier post on the "sad state of sysadmin in the age of containers". While I was drafting this post, that story got picked up by HackerNews, Reddit and Twitter, sending a lot of comments and emails my way. Surprisingly many of the comments are supportive of my impression - I would have expected to see much more insults along the lines "you just don't like my-favorite-tool, so you rant against using it". But a lot of people seem to share my concerns. Thanks, you surprised me!
Here is the new rant post, in the slightly different context of big data:
Everybody is doing "big data" these days. Or at least, pretending to do so to upper management. A lot of the time, there is no big data. People do more data anylsis than before, and therefore stick the "big data" label on them to promote themselves and get green light from management, isn't it?
"Big data" is not a technical term. It is a business term, referring to any attempt to get more value out of your business by analyzing data you did not use before. From this point of view, most of such projects are indeed "big data" as in "data-driven revenue generation" projects. It may be unsatisfactory to those interested in the challenges of volume and the other "V's", but this is the reality how the term is used.
But even in those cases where the volume and complexity of the data would warrant the use of all the new toys tools, people overlook a major problem: security of their systems and of their data.
The currently offered "big data technology stack" is all but secure. Sure, companies try to earn money with security add-ons such as Kerberos authentication to sell multi-tenancy, and with offering their version of Hadoop (their "Hadoop distribution").
The security problem is deep inside the "stack". It comes from the way this world ticks: the world of people that constantly follow the latest tool-of-the-day. In many of the projects, you no longer have mostly Linux developers that co-function as system administrators, but you see a lot of Apple iFanboys now. They live in a world where technology is outdated after half a year, so you will not need to support product longer than that. They love reinstalling their development environment frequently - because each time, they get to change something. They also live in a world where you would simply get a new model if your machine breaks down at some point. (Note that this will not work well for your big data project, restarting it from scratch every half year...)
And while Mac users have recently been surprisingly unaffected by various attacks (and unconcerned about e.g. GoToFail, or the fail to fix the rootpipe exploit) the operating system is not considered to be very secure. Combining this with users who do not care is an explosive mixture...
This type of developer, who is good at getting a prototype website for a startup kicking in a short amount of time, rolling out new features every day to beta test on the live users is what currently makes the Dotcom 2.0 bubble grow. It's also this type of user that mainstream products aim at - he has already forgotten what was half a year ago, but is looking for the next tech product to announced soon, and willing to buy it as soon as it is available...
This attitude causes a problem at the very heart of the stack: in the way packages are built, upgrades (and safety updates) are handled etc. - nobody is interested in consistency or reproducability anymore.
Someone commented on my blog that all these tools "seem to be written by 20 year old" kids. He probably is right. It wouldn't be so bad if we had some experienced sysadmins with a cluebat around. People that have experience on how to build systems that can be maintained for 10 years, and securely deployed automatically, instead of relying on puppet hacks, wget and unzipping of unsigned binary code.
I know that a lot of people don't want to hear this, but:
Your Hadoop system contains unsigned binary code in a number of places, that people downloaded, uploaded and redownloaded a countless number of times. There is no guarantee that .jar ever was what people think it is.
Hadoop has a huge set of dependencies, and little of this has been seriously audited for security - and in particular not in a way that would allow you to check that your binaries are built from this audited code anyway.
There might be functionality hidden in the code that just sits there and waits for a system with a hostname somewhat like "yourcompany.com" to start looking for its command and control server to steal some key data from your company. The way your systems are built they probably do not have much of a firewall guarding against such. Much of the software may be constantly calling home, and your DevOps would not notice (nor would they care, anyway).
The mentality of "big data stacks" these days is that of Windows Shareware in the 90s. People downloading random binaries from the Internet, not adequately checked for security (ever heard of anybody running an AntiVirus on his Hadoop cluster?) and installing them everywhere.
And worse: not even keeping track of what they installed over time, or how. Because the tools change every year. But what if that developer leaves? You may never be able to get his stuff running properly again!
Fire-and-forget.
I predict that within the next 5 years, we will have a number of security incidents in various major companies. This is industrial espionage heaven. A lot of companies will cover it up, but some leaks will reach mass media, and there will be a major backlash against this hipster way of stringing together random components.
There is a big "Hadoop bubble" growing, that will eventually burst.
In order to get into a trustworthy state, the big data toolchain needs to:
  • Consolidate. There are too many tools for every job. There are even too many tools to manage your too many tools, and frontends for your frontends.
  • Lose weight. Every project depends on way too many other projects, each of which only contributes a tiny fragment for a very specific use case. Get rid of most dependencies!
  • Modularize. If you can't get rid of a dependency, but it is still only of interest to a small group of users, make it an optional extension module that the user only has to install if he needs this particular functionality.
  • Buildable. Make sure that everybody can build everything from scratch, without having to rely on Maven or Ivy or SBT downloading something automagically in the background. Test your builds offline, with a clean build directory, and document them! Everything must be rebuildable by any sysadmin in a reproducible way, so he can ensure a bug fix is really applied.
  • Distribute. Do not rely on binary downloads from your CDN as sole distribution channel. Instead, encourage and support alternate means of distribution, such as the proper integration in existing and trusted Linux distributions.
  • Maintain compatibility. successful big data projects will not be fire-and-forget. Eventually, they will need to go into production and then it will be necessary to run them over years. It will be necessary to migrate them to newer, larger clusters. And you must not lose all the data while doing so.
  • Sign. Code needs to be signed, end-of-story.
  • Authenticate. All downloads need to come with a way of checking the downloaded files agree with what you uploaded.
  • Integrate. The key feature that makes Linux systems so very good at servers is the all-round integrated software management. When you tell the system to update - and you have different update channels available, such as a more conservative "stable/LTS" channel, a channel that gets you the latest version after basic QA, and a channel that gives you the latest versions shortly after their upload to help with QA. It covers almost all software on your system, so it does not matter whether the security fix is in your kernel, web server, library, auxillary service, extension module, scripting language etc. - it will pull this fix and update you in no time.
Now you may argue that Hortonworks, Cloudera, Bigtop etc. already provide packages. Well ... they provide crap. They have something they call a "package", but it fails by any quality standards. Technically, a Wartburg is a car; but not one that would pass todays safety regulations...
For example, they only support Ubuntu 12.04 - a three year old Ubuntu is the latest version they support... Furthermore, these packages are roughly the same. Cloudera eventually handed over their efforts to "the community" (in other words, they gave up on doing it themselves, and hoped that someone else would clean up their mess); and Hortonworks HDP (any maybe Pivotal HD, too) is derived from these efforts, too. Much of what they do is offering some extra documentation and training for the packages they built using Bigtop with minimal effort.
The "spark" .deb packages of Bigtop, for example, are empty. They forgot to include the .jars in the package. Do I really need to give more examples of bad packaging decisions? All bigtop packages now depend on their own version of groovy - for a single script. Instead of rewriting this script in an already required language - or in a way that it would run on the distribution-provided groovy version - they decided to make yet another package, bigtop-groovy.
When I read about Hortonworks and IBM announcing their "Open Data Platform", I could not care less. As far as I can tell, they are only sticking their label on the existing tools anyway. Thus, I'm also not surprised that Cloudera and MapR do not join this rebranding effort - given the low divergence of Hadoop, who would need such a label anyway?
So why does this matter? Essentially, if anything does not work, you are currently toast. Say there is a bug in Hadoop that makes it fail to process your data. Your business is belly-up because of that, no data is processed anymore, your are vegetable. Who is going to fix it? All these "distributions" are built from the same, messy, branch. There is probably only a dozen of people around the world who have figured this out well enough to be able to fully build this toolchain. Apparently, none of the "Hadoop" companies are able to support a newer Ubuntu than 2012.04 - are you sure they have really understood what they are selling? I have doubts. All the freelancers out there, they know how to download and use Hadoop. But can they get that business-critical bug fix into the toolchain to get you up and running again? This is much worse than with Linux distributions. They have build daemons - servers that continuously check they can compile all the software that is there. You need to type two well-documented lines to rebuild a typical Linux package from scratch on your workstation - any experienced developer can follow the manual, and get a fix into the package. There are even people who try to recompile complete distributions with a different compiler to discover compatibility issues early that may arise in the future.
In other words, the "Hadoop distribution" they are selling you is not code they compiled themselves. It is mostly .jar files they downloaded from unsigned, unencrypted, unverified sources on the internet. They have no idea how to rebuild these parts, who compiled that, and how it was built. At most, they know for the very last layer. You can figure out how to recompile the Hadoop .jar. But when doing so, your computer will download a lot of binaries. It will not warn you of that, and they are included in the Hadoop distributions, too.
As is, I can not recommend to trust your business data into Hadoop.
It is probably okay to copy the data into HDFS and play with it - in particular if you keep your cluster and development machines isolated with strong firewalls - but be prepared to toss everything and restart from scratch. It's not ready yet for prime time, and as they keep on adding more and more unneeded cruft, it does not look like it will be ready anytime soon.
One more examples of the immaturity of the toolchain:
The scala package from scala-lang.org cannot be cleanly installed as an upgrade to the old scala package that already exists in Ubuntu and Debian (and the distributions seem to have given up on compiling a newer Scala due to a stupid Catch-22 build process, making it very hacky to bootstrap scala and sbt compilation).
And the "upstream" package also cannot be easily fixed, because it is not built with standard packaging tools, but with an automagic sbt helper that lacks important functionality (in particular, access to the Replaces: field, or even cleaner: a way of splitting the package properly into components) instead - obviously written by someone with 0 experience in packaging for Ubuntu or Debian; and instead of using the proven tools, he decided to hack some wrapper that tries to automatically do things the wrong way...
I'm convinced that most "big data" projects will turn out to be a miserable failure. Either due to overmanagement or undermanagement, and due to lack of experience with the data, tools, and project management... Except that - of course - nobody will be willing to admit these failures. Since all these projects are political projects, they by definition must be successful, even if they never go into production, and never earn a single dollar.

Joachim Breitner: Fifth place in Godingame World Cup

Last evening, Codingame held a Programming World Cup titled There is no Spoon . The format is that within four hours, you get to write a program that solves a given task. Submissions are first rated by completeness (there are 13 test inputs that you can check your code again, and further hidden tests that will only be checked after submission) and then by time of submission. You can only submit your code once. What I like about Codingame is that they support a great number of programming languages in their Web- IDE , including Haskell. I had nothing better to do yesterday, so I joined. I was aiming for a good position in the Haskell-specific ranking. After nearly two hours my code completed all the visible test cases and I submitted. I figured that this was a reasonable time to do so, as it was half-time and there are supposed to be two challenges. I turned out that the first, quite small task, which felt like a warm-up or qualification puzzle, was the first of those two, and that therefore I was done, and indeed the 5th fastest to complete a 100% solution! With only less than 5 minutes difference to the 3rd, money-winning place if I had known I had such a chance, I had started on time... Having submitted the highest ranked Haskell code, I will get a T-Shirt. I also defended Haskell s reputation as an efficient programming language, ranked third in the contest, after C++ (rank 1) and Java (rank 2), but before PHP (9), C# (10) and Python (11), listing only those that had a 100% solution. The task, solving a Bridges puzzle, did not feel like a great fit for Haskell at first. I was juggling Data.Maps around where otherwise I d simple attach attributes to object, and a recursive function simulated nothing but a plain loop. But it played off the moment I had to implement guessing parts of the solution, trying what happens and backtracking when it did not work: With all state in parameters and pure code it was very simple to get a complete solution. My code is of course not very polished, and having the main loop live in the IO monad just to be able to print diagnostic commands is a bit ugly. The next, Lord of the Ring-themed world cup will be on June 27th. Maybe we will see more than 18 Haskell entries then?

Petter Reinholdtsen: First Jessie based Debian Edu beta release

I am happy to report that the Debian Edu team sent out this announcement today: 
the Debian Edu / Skolelinux project is pleased to announce the first
*beta* release of Debian Edu "Jessie" 8.0+edu0~b1, which for the first
time is composed entirely of packages from the current Debian stable
release, Debian 8 "Jessie".
(As most reading this will know, Debian "Jessie" hasn't actually been
released by now. The release is still in progress but should finish
later today ;)
We expect to make a final release of Debian Edu "Jessie" in the coming
weeks, timed with the first point release of Debian Jessie. Upgrades
from this beta release of Debian Edu Jessie to the final release will
be possible and encouraged!
Please report feedback to debian-edu@lists.debian.org and/or submit
bugs: http://wiki.debian.org/DebianEdu/HowTo/ReportBugs
Debian Edu - sometimes also known as "Skolelinux" - is a complete
operating system for schools, universities and other
organisations. Through its pre- prepared installation profiles
administrators can install servers, workstations and laptops which
will work in harmony on the school network.  With Debian Edu, the
teachers themselves or their technical support staff can roll out a
complete multi-user, multi-machine study environment within hours or
days.
Debian Edu is already in use at several hundred schools all over the
world, particularly in Germany, Spain and Norway. Installations come
with hundreds of applications pre-installed, plus the whole Debian
archive of thousands of compatible packages within easy reach.
For those who want to give Debian Edu Jessie a try, download and
installation instructions are available, including detailed
instructions in the manual explaining the first steps, such as setting
up a network or adding users.  Please note that the password for the
user your prompted for during installation must have a length of at
least 5 characters!
== Where to download ==
A multi-architecture CD / usbstick image (649 MiB) for network booting
can be downloaded at the following locations:
    http://ftp.skolelinux.org/skolelinux-cd/debian-edu-8.0+edu0~b1-CD.iso
    rsync -avzP ftp.skolelinux.org::skolelinux-cd/debian-edu-8.0+edu0~b1-CD.iso . 
The SHA1SUM of this image is: 54a524d16246cddd8d2cfd6ea52f2dd78c47ee0a
Alternatively an extended DVD / usbstick image (4.9 GiB) is also
available, with more software included (saving additional download
time):
    http://ftp.skolelinux.org/skolelinux-cd/debian-edu-8.0+edu0~b1-USB.iso
    rsync -avzP ftp.skolelinux.org::skolelinux-cd/debian-edu-8.0+edu0~b1-USB.iso 
The SHA1SUM of this image is: fb1f1504a490c077a48653898f9d6a461cb3c636
Sources are available from the Debian archive, see
http://ftp.debian.org/debian-cd/8.0.0/source/ for some download
options.
== Debian Edu Jessie manual in seven languages ==
Please see https://wiki.debian.org/DebianEdu/Documentation/Jessie/ for
the English version of the Debian Edu jessie manual.
This manual has been fully translated to German, French, Italian,
Danish, Dutch and Norwegian Bokm l. A partly translated version exists
for Spanish.  See http://maintainer.skolelinux.org/debian-edu-doc/ for
online version of the translated manual.
More information about Debian 8 "Jessie" itself is provided in the
release notes and the installation manual:
- http://www.debian.org/releases/jessie/releasenotes
- http://www.debian.org/releases/jessie/installmanual
== Errata / known problems ==
    It takes up to 15 minutes for a changed hostname to be updated via
    DHCP (#780461).
    The hostname script fails to update LTSP server hostname (#783087). 
Workaround: run update-hostname-from-ip on the client to update the
hostname immediately.
Check https://wiki.debian.org/DebianEdu/Status/Jessie for a possibly
more current and complete list.
== Some more details about Debian Edu 8.0+edu0~b1 Codename Jessie released 2015-04-25 ==
=== Software updates ===
Everything which is new in Debian 8 Jessie, e.g.:
 * Linux kernel 3.16.7-ctk9; for the i386 architecture, support for
   i486 processors has been dropped; oldest supported ones: i586 (like
   Intel Pentium and AMD K5).
 * Desktop environments KDE Plasma Workspaces 4.11.13, GNOME 3.14,
   Xfce 4.12, LXDE 0.5.6
   * new optional desktop environment: MATE 1.8
   * KDE Plasma Workspaces is installed by default; to choose one of
     the others see the manual.
 * the browsers Iceweasel 31 ESR and Chromium 41
 * LibreOffice 4.3.3
 * GOsa 2.7.4
 * LTSP 5.5.4
 * CUPS print system 1.7.5
 * new boot framework: systemd
 * Educational toolbox GCompris 14.12
 * Music creator Rosegarden 14.02
 * Image editor Gimp 2.8.14
 * Virtual stargazer Stellarium 0.13.1
 * golearn 0.9
 * tuxpaint 0.9.22
 * New version of debian-installer from Debian Jessie.
 * Debian Jessie includes about 43000 packages available for installation.
 * More information about Debian 8 Jessie is provided in its release
   notes and the installation manual, see the link above.
=== Installation changes ===
    Installations done via PXE now also install firmware automatically
    for the hardware present.
=== Fixed bugs ===
A number of bugs have been fixed in this release; the most noticeable
from a user perspective:
 * Inserting incorrect DNS information in Gosa will no longer break
   DNS completely, but instead stop DNS updates until the incorrect
   information is corrected (710362)
 * shutdown-at-night now shuts the system down if gdm3 is used (775608). 
=== Sugar desktop removed ===
As the Sugar desktop was removed from Debian Jessie, it is also not
available in Debian Edu jessie.
== About Debian Edu / Skolelinux ==
Debian Edu, also known as Skolelinux, is a Linux distribution based on
Debian providing an out-of-the box environment of a completely
configured school network. Directly after installation a school server
running all services needed for a school network is set up just
waiting for users and machines being added via GOsa , a comfortable
Web-UI. A netbooting environment is prepared using PXE, so after
initial installation of the main server from CD or USB stick all other
machines can be installed via the network. The provided school server
provides LDAP database and Kerberos authentication service,
centralized home directories, DHCP server, web proxy and many other
services.  The desktop contains more than 60 educational software
packages and more are available from the Debian archive, and schools
can choose between KDE, GNOME, LXDE, Xfce and MATE desktop
environment.
== About Debian ==
The Debian Project was founded in 1993 by Ian Murdock to be a truly
free community project. Since then the project has grown to be one of
the largest and most influential open source projects. Thousands of
volunteers from all over the world work together to create and
maintain Debian software. Available in 70 languages, and supporting a
huge range of computer types, Debian calls itself the universal
operating system.
== Thanks ==
Thanks to everyone making Debian and Debian Edu / Skolelinux happen!
You rock.

Russell Coker: Anti-Systemd People

For the Technical People This post isn t really about technology, I ll cover the technology briefly skip to the next section if you aren t interested in Linux programming or system administration. I ve been using the Systemd init system for a long time, I first tested it in 2010 [1]. I use Systemd on most of my systems that run Debian/Wheezy (which means most of the Linux systems I run which aren t embedded systems). Currently the only systems where I m not running Systemd are some systems on which I don t have console access, while Systemd works reasonably well it wasn t a standard init system for Debian/Wheezy so I don t run it everywhere. That said I haven t had any problems with Systemd in Wheezy, so I might have been too paranoid. I recently wrote a blog post about systemd, just some basic information on how to use it and why it s not a big deal [2]. I ve been playing with Systemd for almost 5 years and using it in production for almost 2 years and it s performed well. The most serious bug I ve found in systemd is Bug #774153 which causes a Wheezy->Jessie upgrade to hang until you run systemctl daemon-reexec [3]. I know that some people have had problems with systemd, but any piece of significant software will cause problems for some people, there are bugs in all software that is complex enough to be useful. However the fact that it has worked so well for me on so many systems suggests that it s not going to cause huge problems, it should be covered in the routine testing that is needed for a significant deployment of any new version of a distribution. I ve been using Debian for a long time. The transitions from libc4 to libc5 and then libc6 were complex but didn t break much. The use of devfs in Debian caused some issues and then the removal of devfs caused other issues. The introduction of udev probably caused problems for some people too. Doing major updates to Debian systems isn t something that is new or which will necessarily cause significant problems, I don t think that the change to systemd by default compares to changing from a.out binaries to ELF binaries (which required replacing all shared objects and executables). The Social Issue of the Default Init Recently the Debian technical committee determined that Systemd was the best choice for the default init system in Debian/Jessie (the next release of Debian which will come out soon). Decisions about which programs should be in the default install are made periodically and it s usually not a big deal. Even when the choice is between options that directly involve the user (such as the KDE and GNOME desktop environments) it s not really a big deal because you can just install a non-default option. One of the strengths of Debian has always been the fact that any Debian Developer (DD) can just add any new package to the archive if they maintain it to a suitable technical standard and if copyright and all other relevant laws are respected. Any DD who doesn t like any of the current init systems can just package a new one and upload it. Obviously the default option will get more testing, so the non-default options will need more testing by the maintainer. This is particularly difficult for programs that have significant interaction with other parts of the system, I ve had difficulties with this over the course of 14 years of SE Linux development but I ve also found that it s not an impossible problem to solve. It s generally accepted that making demands of other people s volunteer work is a bad thing, which to some extent is a reasonable position. There is a problem when this is taken to extremes, Debian has over 1000 developers who have to work together so sometimes it s a question of who gets to do the extra work to make the parts of the distribution fit together. The issue of who gets to do the work is often based on what parts are the defaults or most commonly used options. For my work on SE Linux I often have to do a lot of extra work because it s not part of the default install and I have to make my requests for changes to other packages be as small and simple as possible. So part of the decision to make Systemd be the default init is essentially a decision to impose slightly more development effort on the people who maintain SysVInit if they are to provide the same level of support of course given the lack of overall development on SysVInit the level of support provided may decrease. It also means slightly less development effort for the people who maintain Systemd as developers of daemon packages MUST make them work with it. Another part of this issue is the fact that DDs who maintain daemon packages need to maintain init.d scripts (for SysVInit) and systemd scripts, presumably most DDs will have a preference for one init system and do less testing for the other one. Therefore the choice of systemd as the default means that slightly less developer effort will go into init.d scripts. On average this will slightly increase the amount of sysadmin effort that will be required to run systems with SysVInit as the scripts will on average be less well tested. This isn t going to be a problem in the short term as the current scripts are working reasonably well, but over the course of years bugs may creep in and a proposed solution to this is to have SysVInit scripts generated from systemd config files. We did have a long debate within Debian about the issue of default init systems and many Debian Developers disagree about this. But there is a big difference between volunteers debating about their work and external people who don t contribute but believe that they are entitled to tell us what to do. Especially when the non-contributors abuse the people who do the work. The Crowd Reaction In a world filled with reasonable people who aren t assholes there wouldn t be any more reaction to this than there has been to decisions such as which desktop environment should be the default (which has caused some debate but nothing serious). The issue of which desktop environment (or which version of a desktop environment) to support has a significant affect on users that can t be avoided, I could understand people being a little upset about that. But the init system isn t something that most users will notice apart from the boot time. For some reason the men in the Linux community who hate women the most seem to have taken a dislike to systemd. I understand that being conservative might mean not wanting changes to software as well as not wanting changes to inequality in society but even so this surprised me. My last blog post about systemd has probably set a personal record for the amount of misogynistic and homophobic abuse I received in the comments. More gender and sexuality related abuse than I usually receive when posting about the issues of gender and sexuality in the context of the FOSS community! For the record this doesn t bother me, when I get such abuse I m just going to write more about the topic in question. While the issue of which init system to use by default in Debian was being discussed we had a lot of hostility from unimportant people who for some reason thought that they might get their way by being abusive and threatening people. As expected that didn t give the result they desired, but it did result in a small trend towards people who are less concerned about the reactions of users taking on development work related to init systems. The next thing that they did was to announce a fork of Debian. Forking software means maintaining a separate version due to a serious disagreement about how it should be maintained. Doing that requires a significant amount of work in compiling all the source code and testing the results. The sensible option would be to just maintain a separate repository of modified packages as has been done many times before. One of the most well known repositories was the Debian Multimedia repository, it was controversial due to flouting legal issues (the developer produced code that was legal where they lived) and due to confusion among users. But it demonstrated that you can make a repository containing many modified packages. In my work on SE Linux I ve always had a repository of packages containing changes that haven t been accepted into Debian, which included changes to SysVInit in about 2001. The latest news on the fork-Debian front seems to be the call for donations [4]. Apparently most of the money that was spent went to accounting fees and buying a laptop for a developer. The amount of money involved is fairly small, Forbes has an article about how awful people can use controversy to get crowd-funding windfalls [5]. MikeeUSA is an evil person who hates systemd [6]. This isn t any sort of evidence that systemd is great (I m sure that evil people make reasonable choices about software on occasion). But it is a significant factor in support for non-systemd variants of Debian (and other Linux distributions). Decent people don t want to be associated with people like MikeeUSA, the fact that the anti-systemd people seem happy to associate with him isn t going to help their cause. Conclusion Forking Debian is not the correct technical solution to any problem you might have with a few packages. Filing bug reports and possibly forking those packages in an external repository is the right thing to do. Sending homophobic and sexist abuse is going to make you as popular as the GamerGate and GodHatesAmerica.com people. It s not going to convince anyone to change their mind about technical decisions. Abusing volunteers who might consider donating some of their time to projects that you like is generally a bad idea. If you abuse them enough you might get them to volunteer less of their time, but the most likely result is that they just don t volunteer on anything associated with you. Abusing people who write technical blog posts isn t going to convince them that they made an error. Abuse is evidence of the absence of technical errors.
Related posts:
  1. systemd a Replacement for init etc The systemd projecct is an interesting concept for replacing init...
  2. Systemd Notes A few months ago I gave a lecture about systemd...
  3. more on anti-spam In response to my last entry about anti-spam measures and...

Patrick Schoenfeld: Sharing code between puppet providers

So you ve written that custom puppet type for something and start working on another puppet type in the same module. What if you needed to share some code between this types? Is there a way of code-reuse that works with the plugin sync mechanism? Yes, there is. Puppet even has two possible ways of sharing code between types. Option #1: a shared base provider A provider in puppet is basically a class associated with a certain (puppet) type and there can be a lot providers for a single type (just look at the package provider!). It seems quiet natural, that it s possible to define a parent class for those providers. So natural, that even the official puppet documentation writes about it. Option #2: Shared libraries The second option is a shared library, shipped in a certain namespace in the lib directory of the module, whereas the idea is mostly sketched in the feature ticket #14149. Basically one defines a class in the special Puppetx namespace, using the author- and module-name in the class name, in order to avoid conflicts with other modules.
require 'puppetx'
module Puppetx::<Author>
  module Puppetx::<Author>::<Modulename>
   ... your helper functions go here ...
  end
end
This example would be saved to
lib/<author>/<modulename>
in your module s folder and be included in your provider with something along the following:
require 
  File.expand_path(File.join(File.dirname(__FILE__), "..", "..", "..",
    "puppet_x", "<author>", "<modulename>.rb"))
Compatibility with Puppet 4: In puppet 4 the name of the namespace has changed slightly. It s now called PuppetX instead of Puppetx and is stored in a file puppet_x.rb , which means that the require and the module name itself need to be changed:
require 'puppet_x'
module PuppetX::<authorname>
  module PuppetX::<authorname>::<modulename>
For backward compatibility with puppet 3 you could instead add something like this, according to my co-worker mxey, who knows way more about ruby then I do:
module PuppetX
 module <Author>
 <Modulename> = Puppetx::<Authorname>::<Modulename>
 end
end
Apart from this you d need to change the require to be conditional on the puppet-version and refer to the module by the aliased version (which is left as an exercise for the reader ;))

Zlatan Todori : Geekdom Happiness Arrived

Debian Jessie Banner Debian 8 aka Jessie has just been released and the celebrations are being held all around the world. So grab a cake or a beer because today your precious server is so much happy and you love your server. Debian will take care of the rest. Cheers!

Bits from Debian: Debian 8.0 Jessie has been released!

Alt Jessie has been released There's a new sheriff in town. And her name is Jessie. We're happy to announce the release of Debian 8.0, codenamed Jessie. Want to install it? Choose your favourite installation media among Blu-ray Discs, DVDs, CDs and USB sticks. Then read the installation manual. For cloud users Debian also offers pre-built OpenStack images ready to use. Already a happy Debian user and you only want to upgrade? You are just an apt-get dist-upgrade away from Jessie! Find how, reading the installation guide and the release notes. Do you want to celebrate the release? Share the banner from this blog in your blog or your website!

25 April 2015

Neil Williams: ARMMP Jessie

Whilst the rest of Debian was very busy actually making the Jessie release, I was trying to prepare the ground to test the armhf installer images. Cubietruck is fine, it s a nice install and the documentation on the wiki works nicely. I tried to automate it (as it would be something I d like to test in LAVA which is new in Jessie) with preseeding but although I have got a jessie-x86 preseed file which works in libvirt, the cubietruck installer doesn t seem to want to listen to all of the preseeding. Some of the options (notably the wifi firmware screen, hostname and user passwords) don t go away. x86 wasn t completely as expected either though it works nicely in libvirt and Virtual Machine Manager but trying to (automate it again) do it manually on the command line with qemu raised an awkward grub2 bug where it couldn t work out where to install and failed to then reboot if manually installed. iMX.53 was the next target got a few steps along the road with that one, except that the kernel in the installer isn t able to see the SATA even though the bootloader can, nor can it see the USB stick. This means that the only media the installer can see is the SD card from which the installer was booted trying to use that ends badly. Beaglebone-black required a little bit of tweaking. Don t forget that the firmware & partition.img together don t give a lot of space, so creating a second partition on the SD card upon which to put the CD image itself is useful. The Debian ARMMP kernel in Jessie doesn t see USB, so the option here is actually what I wanted. Install from SD card onto eMMC. It s a 2Gb space, but it s ideal for a default location, leaving the SD card for other testing. I ran out of time to see about preseeding BBB installs to eMMC but one word of warning, installing to the eMMC doesn t mean that you have a bootloader on the eMMC the SD card is still needed to reboot, unless that is fixed manually post-install. (Haven t got that working yet either.) Speaking of time, this post is more brief on the detail than I would like for a couple of reasons:
  1. I haven t actually got things working well enough (clearly from some of the end states above)
  2. the new code to test these in LAVA isn t ready yet either
I did mean to test on an iMX6.quad Wandboard will see about that, hopefully, later. In general, the armhf SD card support is the best start for installing Jessie on a range of ARMv7 hardware. (I haven t got as far as ARMv8 yet either.) However, when I describe that as the best start , it is lacking in many areas typically due to restrictions in the u-boot support, mainline kernel support and general board variability. It s a lot easier than it used to be but ARMv7 boards are some distance from a single installer setup or even a single set of installer documentation, for any distribution.

Dominique Dumont: The #newinjessie game: automatic configuration upgrade and other stuff in Debian/Jessie

Here are my contribution for the #newinjessie game, i.e. what new stuff I ve contributed to the Jessie release of Debian. See you in 2 years for Stretch release All the best

Patrick Schoenfeld: WordPress(ed) again.

I just migrated my blog(s) to WordPress. Just recently I decided to put more time into blogging again. I wasn t entirely happy with Movable Type anymore, especially since the last update broke my customized theme and I struggled with the installation of another theme, which basically just were never found, no matter which path I put it into. What I wanted is just more blogging, not all that technical stuff. And since the Movable Type makers also seem to have gone crazy (their Getting started site tells users to head over to movabletype.com to get a 999$ license) I decided to get back to WordPress. There were reasons, why I haven t chosen WordPress back when I migrated from blogger to MT, but one has to say anyway, that things have moved a lot since then. And WordPress is as easy as it can be and has a prosper community, something I cannot say about MovableType. The migration went okay, although there were some oddities in the blog entries exported by MT (e.g. datetime strings with EM and FM at the EOL) and I needed to figure out how the multisite feature in WordPress works. But now I have exactly what I want.

Jonathan McCrohan: New packages in Debian 8.0 Jessie

[I liked the #newinwheezy effort for the last Debian release, so I tipped Mika off about it again this time] A short post about what is #newinjessie, Debian's new 8.0 Jessie release. See Mika's debian-devel post for more information. For this release cycle, I have uploaded two new packages: I've also have taken over as (co-)maintainer of some existing packages during this release cycle: I've updated nearly all of my existing packages during this release cycle: The following package received no updates during this release cycle due to a combination of no upstream releases and the existing package already being in good shape: I hope you find them useful. Enjoy!

24 April 2015

Jonathan Wiltshire: What to expect on Jessie release day

Release day is a nerve-wracking time for several teams. Happily we ve done it a few times now*, so we have a rough idea of how the process should go. There have been some preparations going on in advance:
  1. Last week we imposed a quiet period on migrations. That s about as frozen as we can get; it means that even RC bugs need to be exceptional if they aren t to be deferred to the first point release. Only late-breaking documentation (like the install guide) was accepted.
  2. The security team opened jessie-updates for business, carried out a test upload, and have since released several packages so that those updates are available immediately after release.
  3. The debian-installer team made a final release.
  4. Final debtags data was updated.
  5. Yesterday the testing migration script britney and other automatic maintenance scripts that the release team run were disabled for the duration.
  6. We made final preparations of things that can be done in advance, such as drafting the publicity announcements. These have to be done in advance so translators get chance to do their work overnight (the announcement was signed off at 15:30UTC today, translations are starting to arrive right now!).
The following checklist makes the release actually happen:
  1. Once dinstall is completed at 07:52, archive maintenance is suspended the FTP masters will do manual work for now.
  2. Very large quantities of coffee will be prepared all across Europe.
  3. Release managers carry out consistency checks of the Jessie index files, and confirm to FTP masters that there are no last-minute changes to be made. RMs get a break to make more coffee.
  4. While they re away FTP masters begin the process of relabelling Wheezy as oldstable and Jessie as stable. If an installer needs to be, er, installed as well, that happens at this point. Old builds of the installer are removed.
  5. A new suite for Stretch (Debian 9) is initialised and populated, and labelled testing.
  6. Release managers check that the newly-generated suite index files look correct and consistent with the checks made earlier in the day. Everything is signed off both in logistical and cryptographic terms.
  7. FTP masters trigger a push of all the changes to the CD-building mirror so that production of images can begin. As each image is completed, several volunteers download and test it in as many ways as they can dream up (booting and choosing different paths through the installer to check integrity).
  8. Finally a full mirror push is triggered by FTP masters, and the finished CD images are published.
  9. Announcements are sent by the publicity team to various places, and by the release team to the developers at large.
  10. Archive maintenance scripts are re-enabled.
  11. The release team take a break for a couple of weeks before getting back into the next cycle.
During the day much of the coordination happens in the #debian-release, #debian-ftp and #debian-cd IRC channels. You re welcome to follow along if you re interested in the process, although we ask that you are read-only while people are still concentrating (during the Squeeze release, a steady stream of people turned up to say congratulations! at the most critical junctures; it s not particularly helpful while the process is still going on). The publicity team will be tweeting and denting progress as it happens, so that makes a good overview too. If everything goes to plan, enjoy the parties! (Disclaimer: inaccuracies are possible since so many people are involved and there s a lot to happen in each step; all errors and omissions are entirely mine.) * yes yes, I am being particularly English today.
What to expect on Jessie release day is a post from: jwiltshire.org.uk Flattr

Michael Prokop: The #newinjessie game: new forensic packages in Debian/jessie

Repeating what I did for the last Debian release with the #newinwheezy game it s time for the #newinjessie game: Debian/jessie AKA Debian 8.0 includes a bunch of packages for people interested in digital forensics. The packages maintained within the Debian Forensics team which are new in the Debian/jessie stable release as compared to Debian/wheezy (and ignoring wheezy-backports): Join the #newinjessie game and present packages which are new in Debian/jessie.

Richard Hartmann: Release Critical Bug report for Week 17

At the current rate, Jessie should release... tomorrow! :) The UDD bugs interface currently knows about the following release critical bugs: How do we compare to the Squeeze and Wheezy release cycles?
Week Squeeze Wheezy Jessie
43 284 (213+71) 468 (332+136) 319 (240+79)
44 261 (201+60) 408 (265+143) 274 (224+50)
45 261 (205+56) 425 (291+134) 295 (229+66)
46 271 (200+71) 401 (258+143) 427 (313+114)
47 283 (209+74) 366 (221+145) 342 (260+82)
48 256 (177+79) 378 (230+148) 274 (189+85)
49 256 (180+76) 360 (216+155) 226 (147+79)
50 204 (148+56) 339 (195+144) ???
51 178 (124+54) 323 (190+133) 189 (134+55)
52 115 (78+37) 289 (190+99) 147 (112+35)
1 93 (60+33) 287 (171+116) 140 (104+36)
2 82 (46+36) 271 (162+109) 157 (124+33)
3 25 (15+10) 249 (165+84) 172 (128+44)
4 14 (8+6) 244 (176+68) 187 (132+55)
5 2 (0+2) 224 (132+92) 175 (124+51)
6 release! 212 (129+83) 161 (109+52)
7 release+1 194 (128+66) 147 (106+41)
8 release+2 206 (144+62) 147 (96+51)
9 release+3 174 (105+69) 152 (101+51)
10 release+4 120 (72+48) 112 (82+30)
11 release+5 115 (74+41) 97 (68+29)
12 release+6 93 (47+46) 87 (71+16)
13 release+7 50 (24+26) 97 (77+20)
14 release+8 51 (32+19) ???
15 release+9 39 (32+7) 82 (49+17)
16 release+10 20 (12+8) 53 (49+4)
17 release+11 24 (19+5) 66 (57+9)
18 release+12 2 (2+0)
Graphical overview of bug stats thanks to azhag:

Jonathan Wiltshire: Jessie Countdown: 1

One further contributor became a non-uploading Debian Developer in 2015, joining 11 others who achieved that status its introduction in 2010. Non-uploading developers are really important to our project. They re a recognition of the invaluable work that contributors do which doesn t involve packaging, for which they may not have the skill nor inclination. Nevertheless we would be much weaker without those tireless contributions in the community, translations, bug handling the list is much longer and covers every aspect of Debian life. If you re reading this and thinking I have a sustained involvement in some aspect of Debian, and make regular contributions , please consider chatting to those you work with and see if you re ready to be recognised too. (source: https://nm.debian.org/public/people#dd)
Jessie Countdown: 1 is a post from: jwiltshire.org.uk Flattr

23 April 2015

Zlatan Todori : Internet freedom

Why did internet succeed and is growing by incredible pace? Because there were no politicians, no lawyers no economist or managers. Period.

Jonathan Dowland: Deterministic Doom

What happens if you remove randomness from Doom? For some reason, recently I have been thinking about Doom. This evening I was wanting to explore some behaviour in an old version of Doom and to do so, I hex-edited the binary and replaced the random number lookup table with static values. Rather than consume system randomness, Doom has a fixed 256-value random number table from which numbers are pulled by aspects of the game logic. By replacing the whole table with a constant value, you essentially make the game entirely deterministic. What does it play like? I tried two values, 0x00 and 0xFF. With either value, the screen "melt" effect that is used at the end of levels is replaced with a level vertical wipe: the randomness was used to offset each column. Monsters do not make different death noises at different times; only one is played for each category of monster. The bullet-based (hitscan) weapons have no spread at all: the shotgun becomes like a sniper rifle, and the chain-gun is likewise always true. You'd think this would make the super-shotgun a pretty lethal weapon, but it seems to have been nerfed: the spread pattern is integral to its function. With 0x00, monsters never make their idle noises (breathing etc.) On the other hand, with 0xFF, they always do: so often, that each sample collides with the previous one, and you just get a sort-of monster drone. This is quite overwhelming with even a small pack of monsters. With 0xFF, any strobing sectors (flickering lights etc.), are static. However, with 0x00, they strobe like crazy. With 0x00, monsters seem to choose to attack much more frequently than usual. Damage seems to be worst-case. The most damaging floor type ("super hellslime"/20%) can hurt you even if you are wearing a radiation suit: There was a very low chance of it hurting whilst wearing the suit (~2.6%) each time the game checked; this is rounded up to 100%. Various other aspects of the game become weird. A monster may always choose to use a ranged attack, regardless of how close you are. They might give up pursuing you. I've seen them walk aimlessly in circles if they are obstructed by another thing. The chance of monster in-fighting is never, or a certainty. The player is either mute, or cries in pain whenever he's hurt. If you want to try this yourself, the easiest way is to hack the m_random.c file in the source, but you can hex-edit a binary. Look for a 256-byte sequence beginning beginning ['0x0', '0x8', '0x6d', '0xdc', '0xde', '0xf1'] and ending ['0x78', '0xa3', '0xec', '0xf9'].

Next.

Previous.