Search Results: "reg"

31 March 2015

Zlatan Todori : Interviews with FLOSS developers: Francesca Ciceri

Debian and FLOSS community don't only occupy coding developers. They occupy people who write news, who talk about FLOSS, who help on booths and conferences, who create artistic forms of the community and so many others that contribute in countless ways. A lady, that is doing many of that is Francesca Ciceri, known in Debian as MadameZou. She is non-packaging Debian Developer, a fearless warrior for diversity and a zombie fan. Although it sounds intimidating, she is deep caring and great human being. So, what has MadaZou to tell us? Picture of MadameZou Who are you? My name is Francesca and I'm totally flattered by your intro. The fearless warrior part may be a bit exaggerated, though. What have you done and what are you currently working on in FLOSS world? I've been a Debian contributor since late 2009. My journey in Debian has touched several non-coding areas: from translation to publicity, from videoteam to www. I've been one of the www.debian.org webmasters for a while, a press officer for the Project as well as an editor for DPN. I've dabbled a bit in font packaging, and nowadays I'm mostly working as a Front Desk member. Setup of your main machine? Wow, that's an intimate question! Lenovo Thinkpad, Debian testing. Describe your current most memorable situation as FLOSS member? Oh, there are a few. One awesome, tiring and very satisfying moment was during the release of Squeeze: I was member of the publicity and the www teams at the time, and we had to pull a 10 hours of team work to put everything in place. It was terrible and exciting at the same time. I shudder to think at the amount of work required from ftpmaster and release team during the release. Another awesome moment was my first Debconf: I was so overwhelmed by the sense of belonging in finally meeting all these people I've been worked remotely for so long, and embarassed by my poor English skills, and overall happy for just being there... If you are a Debian contributor I really encourage you to participate to Debian events, be they small and local or as big as DebConf: it really is like finally meeting family. Some memorable moments from Debian conferences? During DC11, the late nights with the "corridor cabal" in the hotel, chatting about everything. A group expedition to watch shooting stars in the middle of nowhere, during DC13. And a very memorable videoteam session: it was my first time directing and everything that could go wrong, went wrong (including the speaker deciding to take a walk outside the room, to demonstrate something, out of the cameras range). It was a disaster, but also fun: at the end of it, all the video crew was literally in stitches. But there are many awesome moments, almost too many to recall. Each conference is precious on that regard: for me the socializing part is extremely important, it's what cements relationships and help remote work go smoothly, and gives you motivation to volunteer in tasks that sometimes are not exactly fun. You are known as Front Desk member for DebConf's - what work does it occupy and why do you enjoy doing it? I'm not really a member of the team: just one of Nattie's minions! You had been also part of DebConf Video team - care to share insights into video team work and benefits it provides to Debian Project? The video team work is extremely important: it makes possible for people not attending to follow the conference, providing both live streaming and recording of all talks. I may be biased, but I think that DebConf video coverage and the high quality of the final recordings are unrivaled among FLOSS conferences - especially since it's all volunteer work and most of us aren't professional in the field. During the conference we take shifts in filming the various talks - for each talk we need approximately 4 volunteers: two camera operators, a sound mixer and the director. After the recording, comes the boring part: reviewing, cutting and sometimes editing the videos. It's a long process and during the conference, you can sometimes spot the videoteam members doing it at night in the hacklab, exhausted after a full day of filming. And then, the videos are finally ready to be uploaded, for your viewing pleasure. During the last years this process has become faster thanks to the commitment of many volunteers, so that now you have to wait only few days, sometimes a week, after the end of the conference to be able to watch the videos. I personally love to contribute to the videoteam: you get to play with all that awesome gear and you actually make a difference for all the people who cannot attend in person. You are also non-packaging Debian Developer - how does that feel like? Feels awesome! The mere fact that the Debian Project decided - in 2009 via a GR - to recognize the many volunteers who contribute without doing packaging work is a great show of inclusiveness, in my opinion. In a big project like Debian just packaging software is not enough: the final result relies heavily on translators, sysadmins, webmasters, publicity people, event organizers and volunteers, graphic artists, etc. It's only fair that these contributions are deemed as valuable as the packaging, and to give an official status to those people. I was one of the firsts non-uploading DD, four years ago, and for a long time it was just really an handful of us. In the last year I've seen many others applying for the role and that makes me really happy: it means that finally the contributors have realized that they deserve to be an official part of Debian and to have "citizenship rights" in the project. You were the leading energy on Debian's diversity statement - what gave you the energy to drive into it? It seemed the logical conclusion of the extremely important work that Debian Women had done in the past. When I first joined Debian, in 2009, as a contributor, I was really surprised to find a friendly community and to not be discriminated on account of my gender or my lack of coding skills. I may have been just lucky, landing in particularly friendly teams, but my impression is that the project has been slowly but unequivocally changed by the work of Debian Women, who raised first the need for inclusiveness and the awareness about the gender problem in Debian. I don't remember exactly how I stumbled upon the fact that Debian didn't have a Diversity Statement, but at first I was very surprised by it. I asked zack (Stefano Zacchiroli), who was DPL at the time, and he encouraged me to start a public discussion about it, sending out a draft - and helped me all the way along the process. It took some back and forth in the debian-project mailing list, but the only thing needed was actually just someone to start the process and try to poke the discussion when it stalled - the main blocker was actually about the wording of the statement. I learned a great deal from that experience, and I think it changed completely my approach in things like online discussions and general communication within the project. At the end of the day, what I took from that is a deep respect for who participated and the realization that constructive criticism does require certainly a lot of work for all parts involved, but can happen. As for the statement in itself: these things are as good as you keep them alive with best practices, but I think that are better stated explicitly rather than being left unsaid. You are involved also with another Front Desk, the Debian's one which is involved with Debian's New Members process - what are tasks of that FD and how rewarding is the work on it? The Debian Front Desk is the team that runs the New Members process: we receive the applications, we assign the applicant a manager, and we verify the final report. In the last years the workflow has been simplified a lot by the re-design of the nm.debian.org website, but it's important to keep things running smoothly so that applicants don't have too lenghty processes or to wait too much before being assigned a manager. I've been doing it for a less more than a month, but it's really satisfying to usher people toward DDship! So this is how I feel everytime I send a report over to DAM for an applicant to be accepted as new Debian Developer: Crazy pic How do you see future of Debian development? Difficult to say. What I can say is that I'm pretty sure that, whatever the technical direction we'll take, Debian will remain focused on excellence and freedom. What are your future plans in Debian, what would you like to work on? Definetely bug wrangling: it's one of the thing I do best and I've not had a chance to do that extensively for Debian yet. Why should developers and users join Debian community? What makes Debian a great and happy place? We are awesome, that's why. We are strongly committed to our Social Contract and to users freedom, we are steadily improving our communication style and trying to be as inclusive as possible. Most of the people I know in Debian are perfectionists and outright brilliant in what they do. Joining Debian means working hard on something you believe, identifying with a whole project, meeting lots of wonderful people and learning new things. It ca be at times frustrating and exhausting, but it's totally worth it. You have been involved in Mozilla as part of OPW - care to share insights into Mozilla, what have you done and compare it to Debian? That has been a very good experience: it meant have the chance to peek into another community, learn about their tools and workflow and contribute in different ways. I was an intern for the Firefox QA team and their work span from setting up specific test and automated checks on the three version of Firefox (Stable, Aurora, Nightly) to general bug triaging. My main job was bug wrangling and I loved the fact that I was a sort of intermediary between developers and users, someone who spoke both languages and could help them work together. As for the comparison, Mozilla is surely more diverse than Debian: both in contributors and users. I'm not only talking demographic, here, but also what tools and systems are used, what kind of skills people have, etc. That meant reach some compromises with myself over little things: like having to install a proprietary tool used for the team meetings (and getting crazy in order to make it work with Debian) or communicating more on IRC than on mailing lists. But those are pretty much the challenges you have to face whenever you go out of your comfort zone . You are also volunteer of the Organization for Transformative Works - what is it, what work do you do and care to share some interesting stuff? OTW is a non profit organization to preserve fan history and cultures, created by fans. Its work range from legal advocacy and lobbying for fair use and copyright related issues, developing and maintaining AO3 -- a huge fanwork archive based on open-source software --, to the production of a peer-reviewed academic journal about fanworks. I'm an avid fanfiction reader and writer, and joining the OTW volunteers seemed a good way to give back to the community - in true Debian fashion . As a volunteer, I work for the Translation Committee: we are more than a hundred people - divided in several language teams - translating the OTW website, the interface of AO3 archive, newsletter, announcements and news posts. We have a orga-wide diversity statement, training for recruits, an ever growing set of procedures to smooth our workflow, monthly meetings and movie nights. It's an awesome group to work with. I'm deeply invested in this kind of work: both for the awesomeness of OTW people and for the big role that fandom and fanworks have in my life. What I find amazing is that the same concept we - as in the FLOSS ecosystem - apply to software can be applied to cultural production: taking a piece of art you love and expand, remix, explore it. Just for the fun of it. Protect and encourage the right to play in this cultural sandbox is IMO essential for our society. Most of the participants in the fandom come from marginalised group or minorities whose point of view is usually not part of the mainstream narratives. This makes the act of writing, remixing and re-interpreting a story not only a creative exercise but a revolutionary one. As Elizabeth Minkel says: "My preferred explanation is the idea that the vast majority of what we watch is from the male perspective authored, directed, and filmed by men, and mostly straight white men at that. Fan fiction gives women and other marginalised groups the chance to subvert that perspective, to fracture a story and recast it in her own way." In other words, "fandom is about putting debate and conversation back into an artistic process". On a personal side - you do a lot of DIY, handmade works. What have you done, what joy does it bring to you and share with us a picture of it? I like to think that the hacker in me morphs in a maker whenever I can actually manipulate stuff. The urge to explore ways of doing things, of create and change is probably the same. I've been blessed with curiousity and craftiness and I love to learn new DIY techniques: I cannot describe it, really, but if I don't make something for a while I actually feel antsy. I need to create stuff. Nowadays, I'm mostly designing and sewing clothes - preferably reproductions of dresses from the 40s and the 50s - and I'm trying to make a living of that. It's a nice challenge: there's a lot of research involved, as I always try to be historically accurate in design, sewing tecniques and material, and many hours of careful attention to details. I'm right in the process of make photoshoots for most of my period stuff, so I'll share with you something different: a t-shirt refashion done with the DebConf11 t-shirt! (here's the tutorial) T-shirt pic

Jonathan McDowell: Shipping my belongings across the globe

I previously wrote about tracking a ship around the world, but never followed up with the practical details involved with shipping my life from the San Francisco Bay Area back to Belfast. So here they are, in the hope they provide a useful data point for anyone considering a similar move. Firstly, move out. I was in a one bedroom apartment in Fremont, CA. At the time I was leaving the US I didn t have anywhere for my belongs to go - the hope was I d be back in the Bay Area, but there was a reasonable chance I was going to end up in Belfast or somewhere in England. So on January 24th 2014 I had my all of my belongings moved out and put into storage, pending some information about where I might be longer term. When I say all of my belongings I mean that; I took 2 suitcases and everything else went into storage. That means all the furniture for probably a 2 bed apartment (I d moved out of somewhere a bit larger) - the US doesn t really seem to go in for the concept of a furnished lease the same way as the UK does. I had deliberately picked a moving company that could handle the move out, the storage and the (potential) shipping. They handed off to a 3rd party for the far end bit, but that was to be expected. Having only one contact to deal with throughout the process really helped. Fast forward 8 months and on September 21st I contacted my storage company to ask about getting some sort of rough shipping quote and timescales to Belfast. The estimate came back as around a 4-6 week shipping time, which was a lot faster than I was expecting. However it turned out this was the slow option. On October 27th (delay largely due to waiting for confirmation of when I d definitely have keys on the new place) I gave the go ahead. Container pickup (I ended up with exclusive use of a 20ft container - not quite full, but not worth part shipment) from the storage location was originally due on November 7th. Various delays at the Port of Oakland meant this didn t happen until November 17th. It then sat in Oakland until December 2nd. At that point the ETA into Southampton was January 8th. Various other delays, including a week off the coast of LA (yay West Coast Port Backups) meant that the ship finally arrived in Southampton on January 13th. It then had to get to Belfast and clear customs. On January 22nd 2015, 2 days shy of a year since I d seen them, my belongings and I were reunited. So, on the face of it, the actual time on the ship was only slightly over 6 weeks, but all of the extra bits meant that the total time from Ship it to I have it was nearly 3 months. Which to be honest is more like what I was expecting. The lesson: don t forget to factor in delays at every stage. The relocation cost in the region of US$8000. It was more than I d expected, but far cheaper than the cost of buying all my furniture again (plus the fact there were various things I couldn t easily replace that were in storage). That cost didn t cover the initial move into storage or the storage fees - it covered taking things out, packing them up for shipment and everything after that. Including delivery to a (UK) 3rd floor apartment at the far end and insurance. It s important to note that I d included this detail before shipment - the quote specifically mentioned it, which was useful when the local end tried to levy an additional charge for the 3rd floor aspect. They were fine once I showed them the quote as including that detail. Getting an entire apartment worth of things I hadn t seen in so long really did feel a bit like a second Christmas. I d forgotten a lot of the things I had, and it was lovely to basically get a home in a container delivered.

Dirk Eddelbuettel: R / Finance 2015 Open for Registration

The annoucement below just went to the R-SIG-Finance list. More information is as usual at the R / Finance page.
Registration for R/Finance 2015 is now open! The conference will take place on May 29 and 30, at UIC in Chicago. Building on the success of the previous conferences in 2009-2014, we expect more than 250 attendees from around the world. R users from industry, academia, and government will joining 30+ presenters covering all areas of finance with R. We are very excited about the four keynote presentations given by Emanuel Derman, Louis Marascio, Alexander McNeil, and Rishi Narang.
The conference agenda (currently) includes 18 full presentations and 19 shorter "lightning talks". As in previous years, several (optional) pre-conference seminars are offered on Friday morning. There is also an (optional) conference dinner at The Terrace at Trump Hotel. Overlooking the Chicago river and skyline, it is a perfect venue to continue conversations while dining and drinking. Registration information and agenda details can be found on the conference website as they are being finalized.
Registration is also available directly at the registration page. We would to thank our 2015 sponsors for the continued support enabling us to host such an exciting conference: International Center for Futures and Derivatives at UIC Revolution Analytics
MS-Computational Finance and Risk Management at University of Washington Ketchum Trading
OneMarketData
RStudio
SYMMS On behalf of the committee and sponsors, we look forward to seeing you in Chicago! For the program committee:
Gib Bassett, Peter Carl, Dirk Eddelbuettel, Brian Peterson,
Dale Rosenthal, Jeffrey Ryan, Joshua Ulrich
See you in Chicago in May!

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

30 March 2015

Carl Chenet: Verify the backups of backup-manager

Follow me on Identi.ca or Twitter or Diaspora*diaspora-banner Backup-manager is a tool creating backups and storing them locally. It s really usefult to keep a regular backup of a quickly-changing trees of files (like a development environment) or for traditional backups if you have a NFS mount on your server. Backup-managers is also able to send backup itself to another server by FTP. In order to verify the backups created by backup-manager, we will use also Backup Checker (stars appreciated :) ), the automated tool to verify backups. For each newly-created backup we want to control that: Installing what we need We install backup-manager and backup checker. If you use Debian Wheezy, just use the following command:
apt-key adv --keyserver pgp.mit.edu --recv-keys 2B24481A \
&& echo "deb http://debian.mytux.fr wheezy main" > /etc/apt/sources.list.d/mytux.list \
&& apt-get update \
&& apt-get install backupchecker backup-manager
Backup Checker is also available for Debian Squeeze, Debian Sid, FreeBSD. Check out the documentation to install it from PyPi or from sources. Configuring Backup-Manager Backup-manager will ask what directory you want to store backups, in our case we choose /home/joe/dev/wip In the configuration file /etc/backup-manager.conf, you need to have the following lines:
export BM_BURNING_METHOD="none"
export BM_UPLOAD_METHOD="none"
export BM_POST_BACKUP_COMMAND="backupchecker -c /etc/backupchecker -l /var/log/backupchecker.log"
Configuring Backup Checker In order to configure Backup Checker, use the following commands:
# mkdir /etc/backupchecker && touch /var/log/backupchecker.log
Then write the following in /etc/backupchecker/backupmanager.conf:
[main]
name=backupmanager
type=archive
path=/var/archives/laptop-home-joe-dev-wip.%Y%m%d.master.tar.gz
files_list=/etc/backupchecker/backupmanager.list
You can see we re using placeholders for the path value, in order to match each time the latest archive. More information about Backup Checker placeholders in the official documentation. Last step, the description of your controls on the backup:
[files]
wip/data  type d
wip/config/accounts  md5 27c9d75ba5a755288dbbf32f35712338
wip/dump/dump.sql  >100mb
Launch Backup Manager Just launch the following command:
# backup-manager
After Backup Manager is launched, Backup Checker is automatically launched and verify the new backup of the day where Backup Manager stores the backups. Possible control failures Lets say the dump does not have the expected size. It means someone may have messed up with the database! Backup Checker will warn you with the following message in /var/log/backupchecker.log:
$ cat /var/log/backupchecker.log
WARNING:root:1 file smaller than expected while checking /var/archives/laptop-home-joe-dev-wip-20150328.tar.gz: 
WARNING:root:wip/dump/dump.sql size is 18. Should have been bigger than 104857600.
Other possible failures : someone created an account without asking anyone. The hash sum of the file will change. Here is the alert generated by Backup Checker:
$ cat /var/log/backupchecker.log
WARNING:root:1 file with unexpected hash while checking /var/archives/laptop-home-joe-dev-wip-20150328.tar.gz:
WARNING:root:wip/config/accounts hash is 27c9d75ba5a755288dbbf32f35712338. Should have been 27c9d75ba3a755288dbbf32f35712338.
Another possible failure: someone accidentally (or not) removed the data directory! Backup Checker will detect the missing directory and warn you:
$ cat /var/log/backupchecker.log
WARNING:root:1 file missing in /var/archives/laptop-home-joe-dev-wip-20150328.tar.gz: 
WARNING:root:wip/data
Awesome isn t it? The power of a backup tool combined with an automated backup checker. No more surprise when you need your backups. Moreover you spare the waste of time and efforts to control the backup by yourself. weneedyou What about you? Let us know what you think of it. We would be happy to get your feedbacks. The project cares about our users and the outdated feature was a awesome idea in a feature request by one of the Backup Checker user, thanks Laurent!

Matthias Klumpp: Limba Project: Another progress report

And once again, it s time for another Limba blogpost :-)limba-small Limba is a solution to install 3rd-party software on Linux, without interfering with the distribution s native package manager. It can be useful to try out different software versions, use newer software on a stable OS release or simply to obtain software which does not yet exist for your distribution. Limba works distribution-independent, so software authors only need to publish their software once for all Linux distributions. I recently released version 0.4, with which all most important features you would expect from a software manager are complete. This includes installing & removing packages, GPG-signing of packages, package repositories, package updates etc. Using Limba is still a bit rough, but most things work pretty well already. So, it s time for another progress report. Since a FAQ-like list is easier to digest. compared to a long blogpost, I go with this again. So, let s address one important general question first: How does Limba relate to the GNOME Sandboxing approach? (If you don t know about GNOMEs sandboxes, take a look at the GNOME Wiki Alexander Larsson also blogged about it recently) First of all: There is no rivalry here and no NIH syndrome involved. Limba and GNOMEs Sandboxes (XdgApp) are different concepts, which both have their place. The main difference between both projects is the handling of runtimes. A runtime is the shared libraries and other shared ressources applications use. This includes libraries like GTK+/Qt5/SDL/libpulse etc. XdgApp applications have one big runtime they can use, built with OSTree. This runtime is static and will not change, it will only receive critical security updates. A runtime in XdgApp is provided by a vendor like GNOME as a compilation of multiple single libraries. Limba, on the other hand, generates runtimes on the target system on-the-fly out of several subcomponents with dependency-relations between them. Each component can be updated independently, as long as the dependencies are satisfied. The individual components are intended to be provided by the respective upstream projects. Both projects have their individual up and downsides: While the static runtime of XdgApp projects makes testing simple, it is also harder to extend and more difficult to update. If something you need is not provided by the mega-runtime, you will have to provide it by yourself (e.g. we will have some applications ship smaller shared libraries with their binaries, as they are not part of the big runtime). Limba does not have this issue, but instead, with its dynamic runtimes, relies on upstreams behaving nice and not breaking ABIs in security updates, so existing applications continue to be working even with newer software components. Obviously, I like the Limba approach more, since it is incredibly flexible, and even allows to mimic the behaviour of GNOMEs XdgApp by using absolute dependencies on components. Do you have an example of a Limba-distributed application? Yes! I recently created a set of package for Neverball Alexander Larsson also created a XdgApp bundle for it, and due to the low amount of stuff Neverball depends on, it was a perfect test subject. One of the main things I want to achieve with Limba is to integrate it well with continuous integration systems, so you can automatically get a Limba package built for your application and have it tested with the current set of dependencies. Also, building packages should be very easy, and as failsafe as possible. You can find the current Neverball test in the Limba-Neverball repository on Github. All you need (after installing Limba and the build dependencies of all components) is to run the make_all.sh script. Later, I also want to provide helper tools to automatically build the software in a chroot environment, and to allow building against the exact version depended on in the Limba package. Creating a Limba package is trivial, it boils down to creating a simple control file describing the dependencies of the package, and to write an AppStream metadata file. If you feel adventurous, you can also add automatic build instructions as a YAML file (which uses a subset of the Travis build config schema) This is the Neverball Limba package, built on Tanglu 3, run on Fedora 21: Limba-installed Neverball Which kernel do I need to run Limba? The Limba build tools run on any Linux version, but to run applications installed with Limba, you need at least Linux 3.18 (for Limba 0.4.2). I plan to bump the minimum version requirement to Linux 4.0+ very soon, since this release contains some improvements in OverlayFS and a few other kernel features I am thinking about making use of. Linux 3.18 is included in most Linux distributions released in 2015 (and of course any rolling release distribution and Fedora have it). Building all these little Limba packages and keeping them up-to-date is annoying Yes indeed. I expect that we will see some bigger Limba packages bundling a few dependencies, but in general this is a pretty annoying property of Limba currently, since there are so few packages available you can reuse. But I plan to address this. Behind the scenes, I am working on a webservice, which will allow developers to upload Limba packages. This central ressource can then be used by other developers to obtain dependencies. We can also perform some QA on the received packages, map the available software with CVE databases to see if a component is vulnerable and publish that information, etc. All of this is currently planned, and I can t say a lot more yet. Stay tuned! (As always: If you want to help, please contact me) Are the Limba interfaces stable? Can I use it already? The Limba package format should be stable by now since Limba is still Alpha software, I will however, make breaking changes in case there is a huge flaw which makes it reasonable to break the IPK package format. I don t think that this will happen though, as the Limba packages are designed to be easily backward- and forward compatible. For the Limba repository format, I might make some more changes though (less invasive, but you might need to rebuilt the repository). tl;dr: Yes! Plase use Limba and report bugs, but keep in mind that Limba is still in an early stage of development, and we need bug reports! Will there be integration into GNOME-Software and Muon? From the GNOME-Software side, there were positive signals about that, but some technical obstancles need to be resolved first. I did not yet get in contact with the Muon crew they are just implementing AppStream, which is a prerequisite for having any support for Limba[1]. Since PackageKit dropped the support for plugins, every software manager needs to implement support for Limba.
So, thanks for reading this (again too long) blogpost :) There are some more exciting things coming soon, especially regarding AppStream on Debian/Ubuntu! [1]: And I should actually help with the AppStream support, but currently I can not allocate enough time to take that additional project as well this might change in a few weeks. Also, Muon does pretty well already!

Dimitri John Ledkov: Boiling frog, or when did we loose it with /etc ?

$ sudo find /etc -type f wc -l
2794

StatelessWhen was the last time you looked at /etc and thought - "I honestly know what every single file in here is". Or for example had a thought "Each file in here is configuration changes that I made". Or for example do you have confidence that your system will continue to function correctly if any of those files and directories are removed?

Traditionally most *NIX utilities are simple enough utilities, that do not require any configuration files what's so ever. However most have command line arguments, and environment variables to manipulate their behavior. Some of the more complex utilities have configuration files under /etc, sometimes with "layer" configuration from user's home directory (~/). Most of them are generally widely accepted. However, these do not segregate upstream / distribution / site administrator / local administrator / user configuration changes. Most update mechanisms created various ways to deal with merging and maintaining the correct state of those. For example both dpkg & RPM (%config) have elaborate strategies and policies and ways to deal with them. However, even today, still, they cause problems: prompting user for whitespace changes in config files, not preserving user changes, or failing to migrate them.

I can't find exact date, but it has now been something like 12 years since XDG Base directory specification was drafted. It came from Desktop Environment requirements, but one thing it achieves is segregation between upstream / distro / admin / user induced changes. When applications started to implement Base directory specification, I started to feel empowered. Upstream ships sensible configs in /usr, distribution integrators ship their overlay tweaks packaged in /usr, my site admin applies further requirements in /etc, and as I user I am free to improve or brake everything with configs in ~/. One of the best things from this setup - no upgrade prompts, and ease of reverting each layer of those configs (or at least auditing where the settings are coming from).

However, the uptake of XDG Base directory spec is slow / non-existing among the core components of any OS today. And at the same time /etc has grown to be a dumping ground for pretty much everything under the sun:
  • Symlink farms - E.g. /etc/rc*.d/*, /etc/systemd/system/*.wants/*, /etc/ssl/certs/*
  • Cache files - E.g. /etc/ld.so.cache
  • Empty (and mandatory) directories
  • Empty (and mandatory) "configuration" files. - E.g. whitespace & comments only
Let's be brutally honest and say that none of the above belongs in /etc. /etc must be for end-user configuration only, made by the end user alone and nobody else (or e.g. an automation tool driven by the end-user, like puppet).

Documentation of available configuration options and syntax to specify those in the config files should be shipped... in the documentation. E.g. man pages, /usr/share/doc, and so on. And not as the system-wide "example" config files. Absence of the files in /etc must not be treated as fatal, but a norm, since most users use default settings (especially for the most obscure options). Lastly compiled-in defaults should be used where possible, or e.g. layer configuration from multiple locations (e.g. /usr, /etc, ~/ where appropriate).

Above observations are not novel, and shared by most developers and users in the wider open source ecosystem. There are many projects and concepts to deal with this problem by using automation (e.g. puppet, chef), by migrating to new layouts (e.g. implementing / supporting XDG base dir spec), using "app bundles" (e.g. mobile apps, docker), or fully enumerating/abstracting everything in a generic manner (e.g. NixOS). Whilst fixing the issue at hand, these solutions do increase the dependency on files in /etc to be available. In other words we grew a de-facto user-space API we must not break, because modifications to the well known files in /etc are expected to take effect by both users and many administrator tools.

Since August last year, I have joined Open Source Technology Center at Intel, and have been working on Clear Linux* Project for Intel Architecture. One of the goals we have set out is to achieve stateless operation - that is to have empty /etc by default, reserved for user modification alone, yet continuing to support all legacy / well-known configuration paths. The premise is that all software can be patched with auto-detection, built-in defaults or support for layered configuration to achieve this. I hope that this work would interest everyone and will be widely adopted.

Whilst the effort to convert everything is still on going, I want to discuss a few examples of any core system.

ShadowThe login(1) command, whilst having built-in default for every single option exits with status 1, if it cannot stat(2) login.defs(5) file.

The passwd(1) command will write out the salted/hashed password in the passwd(5) file, rather than in shadow(5), if it cannot stat the shadow(5) file. There is similar behavior with gshadow. I found it very ironic, that upstream project "shadow" does not use shadow(5) by default.

Similarly, stock files manipulated by passwd/useradd/groupadd utilities are not created, if missing.

Some settings in login.defs(5) are not applicable, when compiled with PAM support, yet present in the default shipped login.defs(5) file.

Patches to resolve above issues are undergoing review on the upstream mailing list.
DBusIn xml based configuration, includedir' elements are mandatory to exist on disk, that is empty directory must be present, if referenced. If these directories are non-existant, the configuration fails to load and the system or session bus are not started.

Similarly, upstream have general agreement with the stateless concept and patches to move all of dbus default configurations from /etc to /usr are being reviewed for inclusion at the bug tracker. I hope this change will make into the 1.10 stable release.

GNU Lib CToday, we live in a dual-stack IPv4 and IPv6 world, where even the localhost has multiple IP addresses. As a slightly ageist time reference, the first VCS I ever used was git. Thus when I read below, I get very confused:
$ cat /etc/host.conf
# The "order" line is only used by old versions of the C library.
order hosts,bind
multi on
Why not simply do this:
--- a/resolv/res_hconf.c
+++ b/resolv/res_hconf.c
@@ -309,6 +309,8 @@ do_init (void)
if (hconf_name == NULL)
hconf_name = _PATH_HOSTCONF;

+ arg_bool (ENV_MULTI, 1, "on", HCONF_FLAG_MULTI);
+
fp = fopen (hconf_name, "rce");
if (fp)

There are still many other packages that needed fixes similar to above. Stay tuned for further stateless observations about Glibc, OpenSSH, systemd and other well known packages.

In the mean time, you can try out https://clearlinux.org/ images that implement above and more already. If you want to chat about it more, comment on G+, find myself on irc - xnox @ irc.freenode.net #clearlinux and join our mailing list to kick the conversation off, if you are interested in making the world more stateless.

ps.
I am a professional Linux Distribution developer, currently employed by Intel, however the postings on this site are my own and don't necessarily represent Intel's or any other past/present/future employer positions, strategies, or opinions.

* Other names and brands may be claimed as the property of others


29 March 2015

Zlatan Todori : Its all about fun

The percentage that women in Debian occupy as DDs is ~2%. Yes, just ~2% ladies that are DDs! So that means ~98% of DDs are gentelmen. some picture with rage meme I know there are more of ladies in Debian, so I firstly urge you, for love of Debian, to apply if you are contributing to this project, love its community and want to see Debian taking over the universe (okay, it seems that we conquered outer space so we need a help on Earth). So why is the number this low? Well maybe it's too precious to us currently inside that we want to prevent it being spoiled from outside. Also there seems to be not that much of younger DDs. Why is that important - well, young people like to do it and not to think about it. Many time they just break it, but many time they also do a breakthrough. Why is difference important and why should we embrace it? It's very important because it breaks a monopoly on view and behavior. It brings views not just from a larger number of people, but also from people from different backgrounds, and in constructive conversation it can put even more pluses on current workflow or it can counter it with good arguments. In a project of its size and worldwide geolocation of its developers, this is true for Debian more then any other projects I know. We need more women so we can balance our inner workings and have a better understanding of humanity and how is it moving, what and why does it need and where is it steering. That way we can produce a community which will improve quality of OS that we produce - because of sheer number of different people working on the same thing bringing to it its own personal touch. So, ladies and youth all over the world, unite and join in Debian because without diversity Debian can't grow beyond its current size. Also, no, Debian is not about code only, it needs painters, musicians, people that want to talk about Debian, people that share love and happiness, people that want to build better communities, UI/UX designers, makers, people who know how to repair a bike, athletes, homebrew beer producers, lawyers (just while world gets rid of laws, then we don't need you), actors, writters... Why, well because world and communities are made up from all that diversity and that's what makes it a better and not a monotone place. But I just use Debian. Well, do you feel love towards Debian and its work? Would you like to feel more as integral part of community? If the answer is big fat YES, then you should be a DD too. Every person that feels it's part of Debians philosophy about freedom and behaving in good manner should join Debian. Every person that feels touched and enhanced by Debian's work should become part of community and share its experience how Debian touched their soul, impacted their life. If you love Debian, you should be free to contribute to it in whatever manner and you should be free to express your love towards it. If you think lintian is sexy, or shebang is a good friends of yours, or you enjoy talking to MadameZou about Debian and zombies (yeah, we do have all kinds of here), or you like Krita, or you hate the look of default XFCE theme, or you can prove that you a more crazy developer then paultag - just hop into community and try to integrate in it. You will meet great folks, have a lot of conversation about wine and cheese, play some dangerous card games and even learn about things like bokononism (yeah I am looking at you dkg!). Now for the current Debian community - what the hell is packaging and non-packaging Debian Developer? Are one better then others? Do others stink? They don't know to hug? WHAT? Yes I know that inexperienced person shouldn't have a permission to access Debian packaging infrastructure, but I have the feeling that even that person knows that. Every person should have a place in Debian and acknowledge other fields. So yes, software developers need access to Debian packaging infrastructure, painters don't. I think we can agree on this. So lets abolish the stupid term and remove the difference in our community. Lets embrace the difference, because if someone writes a good poem about Debian heroism I could like it more then flashplugin-nonfree! Yep, I made that comparison on purpose so you can give a thought about it. Debian has excellent community regarding operating system that it's producing. And it's not going away, not at least anytime soon. But it will not go forward if we don't give additional push as human beings, as people who care about their fellow Debianites. And we do care, I know that, we just need to push it more public. We don't hide bugs, we for sure shouldn't hide features. It will probably bring bad seeds too, but we have mechanisms and will to counter that. If we, on average 10 bad seeds, get some crazy good hacker or crazy lovely positive person like this lady, we will be on right path. Debian is a better place, it should lead in effort to bring more people into FLOSS world and it should allow people to bring more of diversity into Debian. draw a picture where it says next year 3 dpl candidates should be only women and at least one of them not involved in packaging

25 March 2015

Francois Marier: Keeping up with noisy blog aggregators using PlanetFilter

I follow a few blog aggregators (or "planets") and it's always a struggle to keep up with the amount of posts that some of these get. The best strategy I have found so far to is to filter them so that I remove the blogs I am not interested in, which is why I wrote PlanetFilter.

Other options In my opinion, the first step in starting a new free software project should be to look for a reason not to do it :) So I started by looking for another approach and by asking people around me how they dealt with the firehoses that are Planet Debian and Planet Mozilla. It seems like a lot of people choose to "randomly sample" planet feeds and only read a fraction of the posts that are sent through there. Personally however, I find there are a lot of authors whose posts I never want to miss so this option doesn't work for me. A better option that other people have suggested is to avoid subscribing to the planet feeds, but rather to subscribe to each of the author feeds separately and prune them as you go. Unfortunately, this whitelist approach is a high maintenance one since planets constantly add and remove feeds. I decided that I wanted to follow a blacklist approach instead.

PlanetFilter PlanetFilter is a local application that you can configure to fetch your favorite planets and filter the posts you see. If you get it via Debian or Ubuntu, it comes with a cronjob that looks at all configuration files in /etc/planetfilter.d/ and outputs filtered feeds in /var/cache/planetfilter/. You can either:
  • add file:///var/cache/planetfilter/planetname.xml to your local feed reader
  • serve it locally (e.g. http://localhost/planetname.xml) using a webserver, or
  • host it on a server somewhere on the Internet.
The software will fetch new posts every hour and overwrite the local copy of each feed. A basic configuration file looks like this:
[feed]
url = http://planet.debian.org/atom.xml
[blacklist]

Filters There are currently two ways of filtering posts out. The main one is by author name:
[blacklist]
authors =
  Alice Jones
  John Doe
and the other one is by title:
[blacklist]
titles =
  This week in review
  Wednesday meeting for
In both cases, if a blog entry contains one of the blacklisted authors or titles, it will be discarded from the generated feed.

Tor support Since blog updates happen asynchronously in the background, they can work very well over Tor. In order to set that up in the Debian version of planetfilter:
  1. Install the tor and polipo packages.
  2. Set the following in /etc/polipo/config:
     proxyAddress = "127.0.0.1"
     proxyPort = 8008
     allowedClients = 127.0.0.1
     allowedPorts = 1-65535
     proxyName = "localhost"
     cacheIsShared = false
     socksParentProxy = "localhost:9050"
     socksProxyType = socks5
     chunkHighMark = 67108864
     diskCacheRoot = ""
     localDocumentRoot = ""
     disableLocalInterface = true
     disableConfiguration = true
     dnsQueryIPv6 = no
     dnsUseGethostbyname = yes
     disableVia = true
     censoredHeaders = from,accept-language,x-pad,link
     censorReferer = maybe
    
  3. Tell planetfilter to use the polipo proxy by adding the following to /etc/default/planetfilter:
     export http_proxy="localhost:8008"
     export https_proxy="localhost:8008"
    

Bugs and suggestions The source code is available on repo.or.cz. I've been using this for over a month and it's been working quite well for me. If you give it a go and run into any problems, please file a bug! I'm also interested in any suggestions you may have.

24 March 2015

DebConf team: Working towards a child-friendly DebConf (Posted by Martin Krafft)

The Debian Project will celebrate its 22nd birthday during DebConf15 in Heidelberg in August 2015. At this age, it s unsurprising that children of Debian contributors have attended our developer conference for several years. Going with the times, we would like to work further towards making DebConf15 a child-friendly (parents-friendly) conference. The conference venue is far away from traffic, self-contained, and there is a dedicated children s play room. There are green areas around, and the Heidelberg Zoo is literally within sight of the venue. We haven t yet discussed deals with them, but we could. In short: if you d like to attend DebConf, but you are yet unsure what to do with your children bring your kids along! The hostel has a number of 3 and 4 bed-rooms with en-suite bathrooms, plus a good supply of cots available for the very little ones. We will allocate such rooms to families exclusively for your privacy (subject to availability, so please register yourself ASAP, and include a note about your kids). We are maintaining answers to commonly-asked questions on the wiki. Please let us know if anything is missing, and feel free to update the page yourself. We would also like to explore additional possibilities to make it easier for parents to participate in the conference. At the moment, we re still scouting for ideas and there are already a number of promising leads. To help us figure out what we d best offer, we need to know about the demand. If you are planning to bring your children, or if you re thinking about it, please drop a short note with number and ages and any other relevant information to kids@debconf.org. Your mail will be read by a few parents involved in the organisation of DebConf15 and we will obviously keep your data private. We also created a (publicly archived) mailing list to discuss options and keep people updated on our plans. Please subscribe yourself to the list, if interested, and feel free to write to debconf-kids@lists.debian.org with any questions or ideas you might have.

Russ Allbery: Review: Fukushima

Review: Fukushima, by David Lochbaum, et al.
Author: David Lochbaum
Author: Edwin Lyman
Author: Susan Q. Stranahan
Author: Union of Concerned Scientists
Publisher: The New Press
Copyright: 2014
ISBN: 1-59558-927-9
Format: Kindle
Pages: 320
This is a very interesting book, and I can recommend it, but there are two things you should be aware of up-front. The packaging does not necessarily make clear what expectations you should have of it going in. First, the subtitle (The Story of a Nuclear Disaster) should have appended to it And Its Implications for US Nuclear Power Policy. This book is very concerned with the impact of the Fukushima disaster on US policy and nuclear regulation, to the point where I think more than half of the book is about US agencies, nuclear regulatory history, and US reaction. There's nothing wrong with that, of course: the US should take a hard look at its own nuclear energy policy given the events at Fukushima, and it's a worthy topic for a book. But if you go into this book expecting a broader perspective, you will be disappointed. For example, I think the fact that France has a lot of nuclear power was mentioned maybe twice in the whole book, and French reaction was never discussed at all. There is a very detailed examination of exactly what happened at Fukushima (more on that in a moment), but most of the policy implications are examined purely from a US perspective. Even Japanese nuclear policy gets somewhat short shrift. Second, note that the fourth listed co-author is the Union of Concerned Scientists. For those not familiar with US environmental groups, the UCS has a reputation as an anti-nuclear advocacy organization. I don't think that's entirely fair; I think the UCS's position on nuclear power is better summarized as holding that it is theoretically possible to run a nuclear power plant safely, but the actual US nuclear power industry is not very close to that standard, and it would require much tighter regulation and more investment in safety systems to reach that standard. But be aware that the authors of this book have a clear position on the adequacy of current nuclear power safety standards, namely that they aren't. And they don't try to conceal that position in this book. Personally, I prefer authors to be open about their perspective in books like this, but your mileage may vary. There, disclaimers out of the way. I bought this book for a specific reason: I had followed some of the news coverage at the time of the earthquake and tsunami, and then (like many people, I suspect) lost track of the final outcome as the story fell out of the news and I started ignoring people who didn't understand how large the Pacific Ocean is. Now that we've had the benefit of several years of analysis and thoughtful reconstruction of events, I wanted to know what had actually happened. I'm happy to say that this book delivers quite well on that front. Roughly the first half of the book is a detailed blow-by-blow description of exactly what happened at Fukushima, at least as well as we've been able to reconstruct, told as an engrossing and dramatic narrative. There may be a little too much interleaving of reactions within the US government, which I suspect will particularly annoy non-US readers, but the level of factual detail is excellent, clear, and well-explained. What I wasn't expecting, but was pleasantly surprised by, is that it's also a great story. There's tension, conflict, heroism, hard choices, and moral quandries, and the authors do a great job conveying factual information while still giving the reader the sense of being in the middle of the unfolding drama. They resist the urge to disclose all the results of later analysis in the middle of the story, which may provide a slightly less clear view of the disaster, but which makes the telling far more compelling. I usually read non-fiction more slowly than fiction, but Fukushima dragged me in. I found myself grabbing moments to read just another few pages. Unfortunately, this is only about half the book. The other half is a mix of other things that won't have as broad of appeal: an analysis of the challenges of US nuclear regulation, a history of the US nuclear power industry, and a presentation of the authors' opinions about the best path forward for regulation of nuclear power in the US. Since I'm a US citizen and resident with an interest in both nuclear power and regulation of nuclear power in my country, I found this interesting, if not as engrossing as the rest of the book. But it felt a bit oddly tacked on, and I think it's a stretch to say that it's part of the story of Fukushima. The authors try to draw that link by presenting the Japanese nuclear power industry as heavily influenced by their US counterparts, and their regulatory problems as similar to the problems in the US, but there is nowhere near enough detail about Japanese regulatory practices here to support that conclusion. I think the largest weakness, and the most obvious gap, in this book is the lack of detailed analysis of the history and players in the Japanese nuclear regulatory environment. This is an odd miss. If one is concerned about regulatory inadequacy, Japanese government policy is far more obviously part of the story of Fukushima than US policy. I can only speculate that the authors had inside sources for the US policy discussions but not for the Japanese policy discussions (and, sadly, fall back on painting with a rather broad brush and making unsupported generalizations about Japanese regulatory approaches in a few spots). The result feels like two partly-unrelated books stacked and partly shuffled together. So, there are parts of Fukushima that are rather disappointing, particularly for non-US readers. But I still recommend it as a great detailed history of the actual incident and a summary of what we now think happened. That summary is unfortunately sketchy and still very unclear, but I don't think that's the fault of the authors. The inside of a nuclear power plant during a meltdown is a very difficult environment to measure or analyze, and there's a lot of data that we will probably never have. Some details may never be known. But what we do know, and how that knowledge unfolded, is told very well. This is the only book-length treatment on Fukushima I've read, so I can't compare it against other books on the same topic. But it satisfied my curiousity nicely. If you have a similar curiosity, I recommend this book to your attention, although be aware of its approach and its US-centric analysis going in so that you're not surprised by a mismatch of expectations. Rating: 8 out of 10

20 March 2015

Zlatan Todori : Interviews with FLOSS developers: Laura Arjona

One of fresh additions to Debian, that is showing Debian's commitment to diversity in all fields is Laura Arjona Reina. A helpful hand on channels and a great flux of FLOSS energy she brings with herself. Although applied for non-packaging Debian Developer status, Laura does recognize that there are still some technical aspects what must grasp on. Her dedication to FLOSS and trying to solve some of its issues is astonishing, as this woman is doing a lot of self-hosting and system administration. Yes, you read it right - she does all of that and still applied for non-packaging Debian Developer. She is perfect example how FLOSS enhances humans in many ways. Hello Laura. Picture of Laura Who are you? I am Laura Arjona, I work as IT assistant at Technic University of Madrid, I am married and I have a son, and I use and promote free (libre) software both at job and at home and with friends. I have a nice time contributing in Debian and other FLOSS projects but I always want to do more than what I manage to actually do (I hope I can improve that, as time goes by... and maybe when I get retired I am the SuperLArjona that you wrote about!). What parts of FLOSS community are you engaged? I use Debian, and CyanogenMod + F-Droid in my phone. I coordinate the translation of the Debian website into Spanish, help with FSFE website translation too, and translate some other free software (GNU MediaGoblin, F-Droid, Android apps that I use, web services that we use at work...). I use and promote some free social networks: pump.io, GNU Social, and XMPP. My work/friends environment is mostly Windows/Android so I try to find/promote libre software replacements or interesting applications for them. I give free FLOSS stickers to everybody showing interest for libre software, and a nice Debian sticker if they finally install it in their computers. Setup of your main machine? My machine is a humble Compaq Mini 110C laptop (32bits, Atom N270@1.60GHz, 1GB RAM) and I have Debian Jessie (future-stable ATM) with xfce on it. I'm not tied to particular tools, for example I use Mousepad for editing here in my xfce, Kate in my desktop at work, nano in the server. The only "tuning" that I always do is to set a dark background for terminals and text editors, but I don't even switch to a desktop dark theme... (BTW I love Jessie's theme, "Lines"!). I know there are awesome pieces of software out there (hey emacs-org-mode!), but I just don't have fun having to learn them by myself (no LUG near, I'm afraid...). Some memorable moments from Debian conferences? I've only been at Barcelona MiniDebConf Women 2014 and it was great. It was memorable that I promoted the keysigning for that MiniDebConf, and came home with lots of signatures and papers to verify and sign... and then I was not remembering my GPG main key passphrase! so I hid under my desk for two months, and then, decided to start again (created a new key and tried to meet some Debian people in Madrid...). So I guess I should go to some (Mini)DebConf again. You are currently involved in process of becoming non-packaging Debian Developer - what made you take that step? I began translating in 2011, and since then, I enjoyed contributing in Debian (women, l10n-es, website, publicity). I'm quite regular with the translation work, and applying for DD is a plan to 'force' myself to find chunks of time to contribute more in the other areas too. I also believe that applying I may help other people to also apply or get more involved or become more visible. So here I am. Although you applied for non-packaging Debian Developer you recognize that there is still a technical learning curve in Debian even for that - what are the technical aspects a non-Developer should grasp? Well, I suppose it depends on the area you are contributing. In Publicity you find repositories in git (bits.debian.org), and subversion (Debian Project News). The website uses CVS (www.debian.org)... So you need the basics of 3 different version control systems to commit your changes (or send them to the mailing list and wait somebody to commit them). We use a mail robot to coordinate the translation work, so you need to write the subject with a certain format, and some people complains when somebody send mail in HTML (plain text is preferred). There are some other tools such as IRC and GPG that I began using just for contributing to Debian. Once that you learn them a bit and you learn how Debian works, you understand they are the great tools and you get in love (hey meetbot and KGB! hey i18n.debian.org!), but I wonder how people with no technical background, or even Computer Engineering students nowadays, accostumed to instant messaging in the mobile, fancy web interfaces and so on, look at these tools and just don't even try. How do you see future of Debian development? I don't know, Debian is huge... Some areas in which I hope we, as a community, find the way to work more: packaging (or help configuring) web applications or network services, provide LTS support, and keep on improving outreach/diversity. What are your future plans in Debian, what would you like to work on? In the Spanish team area, my plan is to go on translating the website, jump more often into translating package descriptions too, and help first-time-contributors to keep themselves involved. In the website and publicity teams, I hope I manage to put some weekly time to help with pending tasks/bugs, and serve as liaison with the other areas in which I'm involved (women, l10n, contributors...). If I become DD, I would like to create/adopt some data sources for contributors.debian.org, or convince people to do it I'm not sure if I will be able to attend DebConf some year; meanwhile, at least, I'll continue trying to help with the blog and promotion (as -publicity-team member). Why should developers and users join Debian community? What makes Debian a great and happy place? For using it: the desktop experience has improved very much in the last years, there's a clear separation between free and non-free software so the Debian users always know where are we, and there is wide documentation (in English, at least. Probably in other languages too). For getting involved: I like very much that you can lurk what almost everybody does: just join some mailing lists or IRC channels, the Debian people work in the open. So you know a bit where are you jumping in. Later you learn that everything is easier than what was looking from the outside, because you make friends and with friends everything is better.
Contributions made to Debian have many chances to reach a very wide community: Debian users, upstream projects, and the hundreds of derivatives. It's a quite horizontal, decentralized organization (that has its downsides too, but I can live with them). Is there something you would change in FLOSS ecosystem? We need much more internationalization and localization efforts. People don't need English for using libre software nowadays in their desktop, it's one of our big strenghts, but they definitely need English for using libre software for Android, or solving problems with the libre software they use in your computers/devices, or to contribute to any community. I think we need more local groups for user support/outreach, more libre-software-based translation tools and online services (replacements for Google Translator, for Transifex...),and more internationalization and localization efforts (manuals, websites... not only the software itself). If we work hard in this area, we'll gain much more users and much more contributors. Why does privacy matter to you? I have a son, at our family we interchange photos, and sometimes I have private conversations using the smartphone, mail or other internet services. I want to have the chance that the day that I (or my family) need privacy, we can have it easily. And I want that the people that really need privacy today, have proper tools at their hand. So I try to use PGP, selfhost my multimedia website, use decentralized, free software based networks and XMPP mobile apps... to help those projects to thrive. I try do my part of the network effect! You are being upset with rise of Github - why is that and what would change would you like to see? I totally agree with Mako's essay "Free software needs free tools". Yes, many nonfree web services are easier and have better features right now, but the key is that only dogfooding can change that, the same like changed it with the GNU/Linux desktop and Open/LibreOffice for example...
I would like to see more people trying to selfhost, use and promote libre software based forges, so in addition to avoid vendor lock in and win consistency in our discourse, we polish the available tools and eventually win the battle also in the technical side. You are hosting yours own instances of Mediagoblin - as it is not officially packaged in Debian yet, how do you manage it and how would you encourage others to do it? I followed the MediaGoblin documentation for its last stable release, and hanged on the IRC channel when in doubts/problems. It was not so hard, because it's well documented for Debian systems, and most of the dependencies are already packaged (in stable and testing). MediaGoblin is in its way to stretch too (thanks simonft and the rest of people working on this!). I'm documenting my adventures with selfhosting in [my blog (http://larjona.wordpress.com), but I need to write more often, and put more time in my small server (now I try to selfhost my git projects with cgit, and I want to setup an XMPP server and Etherpad Lite too). You are trying to resolve with self-hosting personal issues with services such as WhatsUp and other non-free parts of our everyday lives- what issues are you hitting on during your way and how do you resolve them? "#iloveemail", but people don't love it anymore, it seems... I've researched a bit about instant messaging to try to propose alternatives to WhatsApp to my family and friends. It seems Conversations with a community XMPP server where to create multi user chat rooms can be a replacement, so my plan is to try during this year. Meanwhile, I've setup the MediaGoblin site so I upload the photos there instead of sending them with the phone, and for the 1-to-1 chat I try to move people to Kontalk (instant messaging, GPG, photos, voice notes...). For the videocalls, I promote Jitsi or just point people to meet.jit.si/FancyNameofChatRoom. I have account and owndrive.com and will try to host Owncloud too. We'll see what happens. You are interested in radio shows - what drives you into that field and will we see soon any podcasts from Laura? I like to talk and I'm not a shy person, so the few times that in any of my social groups there was a chance to "talk in the radio", I volunteered and enjoyed. This has been, in my life, 6 or 7 times (in Spanish, talking about social activism or politics. No records, though!). Some months ago the people of "El Binario" invited me to talk at "findenegro" about pump.io and free networks (in Spanish), I accepted and had a very nice time (audios in my mediagoblin). I wish I have more free time to listen to podcasts and maybe to join some other people to participate in a program in a regular basis. OTOH, my son asks for a tale almost each day... I follow one of Gianni Rodari "The grammar of fantasy"'s approach: take some day-to-day facts and add something unexpected and crazy, and tailor a short story. Maybe I could record them and publish in my MediaGoblin... Of course their literary quality is not even near to Gianni Rodari's but people that listened to them when I was storytelling (in the metro, my mother at home, some friends...) say they are fun and interesting. Who knows! Work of Laura's son

18 March 2015

DebConf team: DebConf15 Call for Proposals (Posted by Michael Banck)

We re now calling for proposals for DebConf15. Proposals are accepted from now until 15 June 2015. To submit an event, go to the Propose an Event page once you are registered for the conference. The DebConf Content Team will decide on a first round of submissions in May, so be sure to submit your proposal soon if you need it to be accepted by then, e.g. for sponsorship requests. The current, non-exhaustive list of proposed topics is: For all further information, please see the Proposals page of the DebConf15 website.

16 March 2015

Bits from Debian: Debian is now welcoming applicants for Outreachy and GSoC Summer 2015

We'd like to reshare a post from Nicolas Dandrimont. Hi all, I am delighted to announce that Debian will be participating in the next round of Outreachy and GSoC, and that we are currently welcoming applications! Outreachy logo Outreachy helps people from groups underrepresented in free and open source software get involved. The current round of internships is open to women (cis and trans), trans men, genderqueer people, and all participants of the Ascend Project regardless of gender. GSoC 2015 logo Google Summer of Code is a global program, sponsored by Google, that offers post-secondary student developers ages 18 and older stipends to write code for various open source software projects. Interns for both programs are granted a $5500 stipend (in three installments) allowing them to dedicate their summer to working full-time on Debian. Our amazing team of mentors has listed their project ideas on the Debian wiki, and we are now welcoming applicants for both programs. If you want to apply for an internship with Debian this summer, please fill out the template for either Outreachy or GSoC. If you re eligible to both programs, we ll encourage you to apply to both (using the same application), as Debian only has funds for a single Outreachy intern this round. Don t wait up! The application period for Outreachy ends March 24th, and the GSoC application period ends March 27th. We really want applicants to start contributing to their project before making our selection, so that mentors can get a feel of how working with their intern will be like for three months. The small task is a requirement for Outreachy, and we re strongly encouraging GSoC applicants to abide by that rule too. To contribute in the best conditions, you shouldn t wait for the last minute to apply :-) I hope we ll work with a lot of great interns this summer. If you think you re up for the challenge, it s time to apply! If you have any doubts, or any question, drop us a line on the soc-coordination mailing list or come by on our IRC channel (#debian-soc on irc.debian.org) and we ll do our best to guide you.

14 March 2015

Niels Thykier: Imminent steep decline in RC bugs affecting Jessie need more RC bug fixes

Earlier today, I posted a mail to debian-devel about how approximately 25 RC bugs affecting Jessie have been unblocked. As mentioned, I planned to age some of them. The expected result is that about 18 of them will migrate tonight and the remaining 7 of them will migrate tomorrow night. After that, there are no more RC bugs waiting for the RT to unblock them! The only remaining item on the list is cgmanager, for which we are requesting a t-p-u (maintainer already contacted about it). If you want a release sooner, please have a look at the list of remaining RC bugs or/and start testing upgrades. In other news, the glibc regression got fixed. The new version of glibc has already been approved by us. It is now waiting for the debian-installer team to testing it and approve it.
Filed under: Debian, Release-Team

Dirk Eddelbuettel: littler 0.2.3

max-heap image A new minor release of littler is available now. It adds or extends a number of things: See the littler examples page for more details. Full details for the littler release are provided as usual at the ChangeLog page. The code is available via the GitHub repo, from tarballs off my littler page and the local directory here. A fresh package has gone to the incoming queue at Debian; Michael Rutter will probably have new Ubuntu binaries at CRAN in a few days too. Comments and suggestions are welcome via the mailing list or issue tracker at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

13 March 2015

Dirk Eddelbuettel: Why Drat? A Guest Post by Steven Pav

Editorial Note: The following post was kindly contributed by Steven Pav.

Why Drat? After playing around with drat for a few days now, my impressions of it are best captured by Dirk's quote:
It just works.

Demo To get some idea of what I mean by this, suppose you are a happy consumer of R packages, but want access to, say, the latest, greatest releases of my distribution package, sadist. You can simply add the following to your .Rprofile file:
drat::add("shabbychef")
After this, you instantly have access to new releases in the github/shabbychef drat store via the package tools you already know and tolerate. You can use
install.package('sadists')
to install the sadists package from the drat store, for example. Similarly, if you issue
update.packages(ask=FALSE)
all the drat stores you have added will be checked for package updates, along with their dependencies which may well come from other repositories including CRAN.

Use cases The most obvious use cases are:
  1. Micro releases. For package authors, this provides a means to get feedback from the early adopters, but also allows one to push small changes and bug fixes without burning through your CRAN karma (if you have any left). My personal drat store tends to be a few minor releases ahead of my CRAN releases.
  2. Local repositories. In my professional life, I write and maintain proprietary packages. Pushing package updates used to involve saving the package .tar.gz to a NAS, then calling something like R CMD INSTALL package_name_0.3.1.9001.tar.gz. This is not something I wanted to ask of my colleagues. With drat, they can instead add the following stanza to .Rprofile: drat:::addRepo('localRepo','file:///mnt/NAS/r/local/drat'), and then rely on update.packages to do the rest.
I suspect that in the future, drat might be (ab)used in the following ways:
  1. Rolling your own vanilla CRAN mirror, though I suspect there are better existing ways to accomplish this.
  2. Patching CRAN. Suppose you found a bug in a package on CRAN (inconceivable!). As it stands now, you email the maintainer, and wait for a fix. Maybe the patch is trivial, but suppose it is never delivered. Now, you can simply make the patch yourself, pick a higher revision number, and stash it in your drat store. The only downside is that eventually the package maintainer might bump their revision number without pushing a fix, and you are stuck in an arms race of version numbers.
  3. Forgoing CRAN altogether. While some package maintainers might find this attractive, I think I would prefer a single huge repository, warts and all, to a landscape of a million microrepos. Perhaps some enterprising group will set up a CRAN-like drat store on github, and accept packages by pull request (whether github CDN can or will support the traffic that CRAN does is another matter), but this seems a bit too futuristic for me now.

My wish list In exchange for writing this blog post, I get to lobby Dirk for some features in drat:
  1. I shudder at the thought of hundreds of tiny drat stores. Perhaps there should be a way to aggregate addRepo commands in some way. This would allow curators to publish their suggested lists of repos.
  2. Drat stores are served in the gh-pages branch of a github repo. I wish there were some way to keep the index.html file in that directory reflect the packages present in the sources. Maybe this could be achieved with some canonical RMarkdown code that most people use.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Wouter Verhelst: New toy: Fujitsu Lifebook e734

My Lenovo x220, which I've owned for almost four years now (I remember fetching it from the supplier shortly before driving off to Banja Luka), was getting somewhat worn out. The keyboard and the screen had both been replaced at some point already, and the wwan interface had given up as well. The case was all cracked, and the NIC connector wasn't doing very well anymore either; there have been a few cases of me trying to configure the wireless network at a customer, but this being harder than it needs to be because the NIC will only work if I put in the network cable just so, and someone dropped a piece of paper onto the cable. In other words, it was time for a new one. At first I wanted to buy a Lenovo x250, but then I noticed that the Fujitsu came with an i7 4712MQ, which I liked (as today it is still quite exceptional for an ultrabook to have a quadcore processor). Fujitsu also claims up to 9 hours of battery life, but it's not clear to me whether this is supposed to be the case with the default battery only. They also have a battery for the modular bay, which I bought as well (to replace the optical drive whic I sometimes use, but only rarely), and on top of that it came with a free port replicator. Not all is well, however. In the x220, getting the WWAN interface to work involved some creative use of chat against /dev/ttyACM0 wherein I issue a few AT commands to put the WWAN interface into a particular mode, and from then on the WWAN interface is just a regular Ethernet interface on which I can do DHCP. The new laptop has a "Sierra Wireless, Inc." WWAN interface (USB id 1199:9041) which annoyingly doesn't seem to expose the ttyACM (or similar) devices, and I'm not sure what to use instead. Just trying to do DHCP doesn't work -- yes, I tried. Unfortunately, the keyboard isn't very good; it's of the bubble gum type, and I keep getting annoyed at it not picking up my keystrokes all the time. When I'm at home or at my main customer, I have a Das Keyboard Ultimate S (3rd (customer) and 4th (home) generation), so it's only a problem when I'm not at home, but it's still extremely annoying. There is a "backlight" function in that keyboard, but that's not something I think I'll ever use (hint: "das keyboard ultimate s"). The display can't do more than 1366x768, which is completely and utterly wrong for a computer -- but it's the same thing as my x220, so it's not really a regression. The "brightness" ACPI keys don't seem to work. I may have to fiddle with some ACPI settings at some point, I suppose, but it's not a major problem. When I plugged it in, I noticed that fdpowermon ignored the second battery. I had originally written fdpowermon with support for such a second battery, but as my x220 had only one, I never tested it. Apparently there was a bug, but that's been fixed now -- at least in unstable. On the good side of the equation, it has three USB3 ports in the laptop, and four in the port replicator, with no USB2; this is a major leap forwards from the one USB3 and six USB2 in the x220. A positive surprise was the CCID smartcard reader that I somehow missed while reading the specs, but which -- given my current major customer, is very welcome, indeed. Update: After having used it a few days, there were a few minor annoyances:

12 March 2015

Enrico Zini: crypttab-reuse-passwords

Reuse passwords in /etc/crypttab Today's scenario was a laptop with an SSD and a spinning disk, and the goal was to deploy a Debian system on it so that as many things as possible are encrypted. My preferred option for it is to setup one big LUKS partition in each disk, and put a LVM2 Physical Volume inside each partition. At boot, the two LUKS partition are opened, their contents are assembled into a Volume Group, and I can have everything I want inside. This has advantages: However, by default this causes cryptsetup to ask for the password once for each LUKS partition, even if the passwords are the same. Searching for ways to mitigate this gave me unsatisfactory results, like: The solution that I found was something that did not show up in any of my search results, so I'm documenting it here:
    # <target name> <source device>   <key file>   <options>
    ssd             /dev/sda2         main         luks,initramfs,discard,keyscript=decrypt_keyctl
    spin            /dev/sdb1         main         luks,initramfs,keyscript=decrypt_keyctl
This caches each password for 60 seconds, so that it can be reused to unlock other devices that use it. The documentation can be found at the beginning of /lib/cryptsetup/scripts/decrypt_keyctl, beware of the leopard . main is an arbitrary tag used to specify which devices use the same password. This is also useful to work easily with multiple LUKS-on-LV setups:
    # <target name> <source device>          <key file>  <options>
    home            /dev/mapper/myvg-chome   main        luks,discard,keyscript=decrypt_keyctl
    backup          /dev/mapper/myvg-cbackup main        luks,discard,keyscript=decrypt_keyctl
    swap            /dev/mapper/myvg-cswap   main        swap,discard,keyscript=decrypt_keyctl

Bits from Debian: DebConf15: Call for Proposals

The DebConf Content team is pleased to announce the Call for Proposals for the DebConf15 conference, to be held in Heidelberg, Germany from the 15th through the 22nd of August, 2015. Submitting an Event In order to submit an event, you must be registered as an attendee of DebConf15. If you have any questions about the registration process, please check the related information on the conference website. Once registered, go to "Propose an event" and describe your proposal. Please note, events are not limited to traditional presentations or informal sessions (BoFs). We welcome submissions of tutorials, performances, art installations, debates, or any other format of event that you think would be beneficial to the Debian community. Please include a short title, suitable for a compact schedule, and an engaging description of the event. You should use the field "Notes for Content Team" to provide us information such as additional speakers, scheduling restrictions, or any special requirements we should consider for your event. Regular sessions may either be 20 or 45 minutes long (including time for questions) and will be followed by a 10 or 15 minutes break, respectively. Other kinds of sessions (like workshops) could have different durations. Please make sure to choose the most suitable duration for your event and justify any special requests. Timeline The first batch of accepted proposals will be announced in May. If you depend on having your proposal accepted in order to attend the conference, please submit it as soon as possible so that it can be considered during this first evaluation period. All proposals must be submitted before June 15th, 2015 to be evaluated for the official schedule. Topics and Tracks Though we invite proposals on any Debian or FLOSS related subject, we will have some broad topics arranged as tracks for which we encourage people to submit proposals. The currently proposed list is: If you have ideas for further tracks, or would like to volunteer as a track coordinator, please contact content@debconf.org. In order for a track to take place during DebConf15, we must have received a sufficient amount of proposals on that specific theme. Track coordinators will play an important role in inviting people to submit events. Video Coverage Providing video of sessions amplifies DebConf achievements and is one of the conference goals. Unless speakers opt-out, official events will be streamed live over the Internet to promote remote participation. Recordings will be published later under the DebConf license, as well as presentation slides and papers whenever available. Contact and Thanks to Sponsors DebConf would not be possible without the generous support of all our sponsors, especially our platinum sponsor HP. DebConf15 is still accepting sponsors; if you are interested, please get in touch! You are welcome to contact the Content Team with any concerns about your event, or with any ideas or questions ambout DebConf events in general. You can reach us at content@debconf.org. We hope to see you all in Heidelberg!

Marko Lalic: solicit - an HTTP/2 Library for Rust

For quite some time now, I've been following Rust, a new(ish) programming language by Mozilla. It's been around for a while, but recently it's been attracting a lot of buzz. Around the new year, I finally decided to take a deeper look into the language. I find that the best way to learn a new programming language is to simply dive in and try to build something nontrivial. At that point I'd also been interested in learning more about the specifics behind the new HTTP/2 spec, so I decided to combine the two... The result is a (work in progress) client library for HTTP/2 written in Rust, which can be found at this GitHub repository: https://github.com/mlalic/solicit What follows is a description of what I aim to achieve with the library, as well as an overview of its current design. If that doesn't interest you, you can just check out the GitHub repo for some examples or the source code itself. Goals Given that there are already some fairly popular HTTP libraries for Rust (hyper leading the way), I decided that I wouldn't aim to replace them. Rather, the idea is to focus on the lower levels of the HTTP/2 protocol what is essentially another transport protocol and expose an API that allows those libraries to use an HTTP/2 connection to send requests and receive responses. HTTP/2 doesn't change the semantics of the protocol, so once any HTTP/1.1 library gets the raw headers and body, it should be able to represent the response regardless of the fact that it came on an HTTP/2 connection. The same holds for sending requests: only the representation of the request changes, not its content. There are some minor differences, such as HTTP/2 doing away with the reason phrases being part of the response, but those were only meant to be human readable and "MUST NOT" have had any influence on the semantics of the response. Therefore, the primary goal for the library is to provide an API for using an HTTP/2 connection on the lowest level, but making it easy for clients to customize or augment any particular step of handling a response, without having to know the details of functionality below the layer they want to customize. As an example, if one wanted to build a client that sends out progress notifications for downloads, it wouldn't be good if they needed to muck about with raw HTTP/2 frames ideally, they'd only need to implement a new way to handle incoming data chunks for streams. The same applies for implementing a client that streams the download into a file (instead of saving it in memory) or one that sends the chunks to a different component for processing as soon as they're received (e.g. parsing HTML). Since the Rust standard library only exposes blocking IO primitives, I decided to base this library on blocking IO, as well. However, the goal was to make it flexible enough so that clients that would like to allow multiple requests to be sent concurrently on the same HTTP/2 connection can be built, but without the library itself spawning any threads. Finally, on a less conceptual level (and more of an implementation level), the code should have extensive test coverage. Design The library is split into several layers, each of which provides services to the one above it. A brief description of each layer follows. Transport Layer This layer provides the API for raw byte stream communication with the peer. Essentially, it's a wrapper around the particular transport protocol that is used for the HTTP/2 connection. As the HTTP/2 spec allows the transport protocol to be either a cleartext TCP connection or a TLS-protected one, those are the ones that need to be implemented at this level. The API of the layer is represented by a single trait TransportStream that simply requires io::Read and io::Write to be implemented and (for now) provides a single convenience method based on the underlying io::Read implementation. For a clear-text TCP connection, the only thing that was required to implement it was one line of code:
impl TransportStream for TcpStream  
HTTP/2 Connection Layer The connection layer provides an API for reading and writing to a raw HTTP/2 connection. This means reading and writing HTTP/2 frames, not arbitrary bytes. Each HTTP/2 frame has to follow the same basic structure(a 9-byte header followed by a sequence of byte representing the payload of the frame). This is represented by the RawFramestruct. Each HTTP/2 frame type can have its own specific format for the content of the payload and assign meaning to the flags found in the header. Therefore, each frame needs to have its own implementation that can parse a RawFrame or return an error if the raw frame does not represent a valid frame of that type. Some common methods that each frame type implementation needs to support are given by the Frametrait that must be implemented by the implementations. Finally, the HttpConnectionstruct handles reading the byte stream from the underlying TransportStream instance, correctly delineating frames and parsing them into the appropriate frame type implementation (by delegating to the correct Frame implementation based on the frame type found in the header). Obviously, varying the implementation of the transport layer is a matter of choosing the appropriate TransportStream implementation. This includes using mock implementations for testing the HttpConnection itself. HTTP/2 Client Connection The ClientConnectionbuilds upon the HttpConnection by using it to expose functionality that is specific to the client side of an HTTP/2 connection. This includes sending requests and reading responses. The struct takes care of creating frames that need to be sent for a given request (including HPACK header encoding) and shipping them off to the server (using the underlying HttpConnection instance). As for handling a response, the ClientConnection expects to get an instance of a struct implementing the Sessiontrait when constructed. This trait essentially defines a number of callbacks: methods that will be called on this instance when the corresponding event occurs in the ClientConnection. Therefore, the ClientConnection understands the semantics of the particular frames that it reads (e.g. performs the decoding of the HEADERS frames' payloads, passing the raw data received in a DATA frame to the session, etc.). However, it does not know anything about streams; it only passes the stream ID up to the session layer, without thinking about whether the stream is open, valid, etc. HTTP/2 Session As mentioned in the previous section, each ClientConnection requires an instance of a struct implementing the Sessiontrait to be passed to it at construction time. The Session trait defines a number of callbacks that are invoked in the appropriate moments during the processing of frames read on a client connection. For example, new_headers (takes a list of heaers and a stream on which they were received) or new_data_chunk (a buffer and a stream on which the data arrived). The session is in charge of handling these events as they arise. It is up to it to decide what (if anything) should be done with the data that it receives. There is a default implementation of the Session trait (unimaginatively named DefaultSession), which provides an implementation that tracks which streams are currently open and delegates the processing of new events on particular streams to instances of a Streamtrait. This allows for client implementations to simply vary the implementation of a Stream that they provide to the DefaultSession in order to change how a particular part of a response is handled, but let the DefaultSession take care of tracking which streams are open and valid. Different implementations of a Session still can exist, if the particular client needs more fine-grained customization over how these events are handled, but for most purposes the default session, along with Stream customization should probably be enough. HTTP/2 Stream Again, as mentioned previously, the DefaultSessionimplementation of the Session trait relies on a Streamtrait providing the handling for events arising on individual streams. This level should be the one that gets customized most often in alternate implementations. A default stream implementation DefaultStreamhandles the messages by building up a response in-memory. Different implementations could instead send progress notifications on an associated channel, for instance. HTTP/2 Client The final layeris that of a client that exposes methods that would usually be expected for a client HTTP library: sending requests, obtaining responses. This layer is not the primary focus, though, and it only exposes two simple implementations that showcase how the lower level APIs can be used to build a client. The second purpose it fulfills is as a "sanity check" that the APIs actually do allow the goals to be met. Therefore, one of the clients allows for requests to be sent from concurrently running threads and the responses to those requests to be obtained asynchronously (in the form of a Future<Response>). However, the responses are basic and very raw: they are stored fully in memory (not available until they've been read until the end) and both the body, as well as the headers, exposed as raw byte sequences. As an example how this clientcan be used (as well as the simpler single-threaded version), check out the Examplessection in the GitHub repository's README file. Future Work Refactoring APIs As the project was started as a Rust learning exercise, the initial APIs are slightly awkward and fairly inefficient to work with. For example, the solicit::hpackmodule requires headers to be represented as a tuple of Vec<u8>, which means that whenever we need to encode a header list, even though their values might be (statically known) byte sequence literals, such as (b":method", b"GET"), we need to make a bunch of to_vec calls on the &[u8]s before passing them to the encoder, leading not only to a really ugly and tedious way of obtaining the encoded representation of the headers, but also to a lot of unnecessary allocations. Another API that will see some refactoring and improvement is the TransportStream, as well as the HttpConnection in order to make it possible to have separate threads perform reads and writes on the connection. Currently, the socket is owned by the HttpConnection instance and all writes and reads require a &mut reference, making this impossible. This change would tackle this by embracing Rust's new try_clone method on the TcpStream into the TransportStream and refactoring the HttpConnection to allow splitting it into a write and read handle. HTTP/2 Features As already mentioned, only a small subset of the full HTTP/2 spec is currently implemented, so, obviously, implementing the rest of it falls under the most important future work. Some near- to mid-term priorities that will be tackled are: Obviously, those aren't all the features of HTTP/2, but it's a rough list of things I'm planning on working on myself as the next steps. Integration With an (existing?) HTTP Library I would like it if in the end the library or at least some of its pieces ended up being used in the scope of one of the fully featured HTTP libraries. If there's any interest in this, I wouldn't mind putting in some extra legwork to help make it happen. Conclusion While I'm certain that there can be better, more efficient, more elegant implementations, I've had a lot of fun working on this so far (learning new things is always fun!) and if nothing else I'll continue in order to gain the deepest possible understanding of HTTP/2. For anyone that might have read all the way through, thanks for reading and I'm open for any sort of feedback!

Next.

Previous.