Search Results: "spa"

27 March 2015

Olivier Berger: New short paper : Designing a virtual laboratory for a relational database MOOC with Vagrant, Debian, etc.

Here s a short preview of our latest accepted paper (to appear at CSEDU 2015), about the construction of VMs for the Relational Database MOOC using Vagrant, Debian, PostgreSQL (previous post), etc. :

Designing a virtual laboratory for a relational database MOOC Olivier Berger, J Paul Gibson, Claire Lecocq and Christian Bac Keywords: Remote Learning, Virtualization, Open Education Resources, MOOC, Vagrant Abstract: Technical advances in machine and system virtualization are creating opportunities for remote learning to provide significantly better support for active education approaches. Students now, in general, have personal computers that are powerful enough to support virtualization of operating systems and networks. As a conse- quence, it is now possible to provide remote learners with a common, standard, virtual laboratory and learning environment, independent of the different types of physical machines on which they work. This greatly enhances the opportunity for producing re-usable teaching materials that are actually re-used. However, configuring and installing such virtual laboratories is technically challenging for teachers and students. We report on our experience of building a virtual machine (VM) laboratory for a MOOC on relational databases. The architecture of our virtual machine is described in detail, and we evaluate the benefits of using the Vagrant tool for building and delivering the VM. TOC :

Introduction

A brief history of distance learning

Virtualization : the challenges

The design problem

The virtualization requirements

Scenario-based requirements

Related work on requirements

Scalability of existing approaches

The MOOC laboratory

Exercises and lab tools

From requirements to design

Making the VM as a Vagrant box

Portability issues

Delivery through Internet

Security

Availability of the box sources

Validation

Reliability Issues with VirtualBox

Student feedback and evaluation

Future work

Laboratory monitoring

More modular VMs

Conclusions

Bibliography

Alario-Hoyos et al., 2014
Alario-Hoyos, C., P rez-Sanagust n, M., Kloos, C. D., and Mu oz Merino, P. J. (2014).
Recommendations for the design and deployment of MOOCs: Insights about the MOOC digital education of the future deployed in Mir adaX.
In Proceedings of the Second International Conference on Technological Ecosystems for Enhancing Multiculturality, TEEM 14, pages 403-408, New York, NY, USA. ACM.

Armbrust et al., 2010
Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., and Zaharia, M. (2010).
A view of cloud computing.
Commun. ACM, 53:50-58.

Billingsley and Steel, 2014
Billingsley, W. and Steel, J. R. (2014).
Towards a supercollaborative software engineering MOOC.
In Companion Proceedings of the 36th International Conference on Software Engineering, pages 283-286. ACM.

Brown and Duguid, 1996
Brown, J. S. and Duguid, P. (1996).
Universities in the digital age.
Change: The Magazine of Higher Learning, 28(4):11-19.

Bullers et al., 2006
Bullers, Jr., W. I., Burd, S., and Seazzu, A. F. (2006).
Virtual machines an idea whose time has returned: Application to network, security, and database courses.
SIGCSE Bull., 38(1):102-106.

Chen and Noble, 2001
Chen, P. M. and Noble, B. D. (2001).
When virtual is better than real [operating system relocation to virtual machines].
In Hot Topics in Operating Systems, 2001. Proceedings of the Eighth Workshop on, pages 133-138. IEEE.

Cooper, 2005
Cooper, M. (2005).
Remote laboratories in teaching and learning-issues impinging on widespread adoption in science and engineering education.
International Journal of Online Engineering (iJOE), 1(1).

Cormier, 2014
Cormier, D. (2014).
Rhizo14-the MOOC that community built.
INNOQUAL-International Journal for Innovation and Quality in Learning, 2(3).

Dougiamas and Taylor, 2003
Dougiamas, M. and Taylor, P. (2003).
Moodle: Using learning communities to create an open source course management system.
In World conference on educational multimedia, hypermedia and telecommunications, pages 171-178.

Gomes and Bogosyan, 2009
Gomes, L. and Bogosyan, S. (2009).
Current trends in remote laboratories.
Industrial Electronics, IEEE Transactions on, 56(12):4744-4756.

Hashimoto, 2013
Hashimoto, M. (2013).
Vagrant: Up and Running.
O Reilly Media, Inc.

Jones and Winne, 2012
Jones, M. and Winne, P. H. (2012).
Adaptive Learning Environments: Foundations and Frontiers.
Springer Publishing Company, Incorporated, 1st edition.

Lowe, 2014
Lowe, D. (2014).
MOOLs: Massive open online laboratories: An analysis of scale and feasibility.
In Remote Engineering and Virtual Instrumentation (REV), 2014 11th International Conference on, pages 1-6. IEEE.

Ma and Nickerson, 2006
Ma, J. and Nickerson, J. V. (2006).
Hands-on, simulated, and remote laboratories: A comparative literature review.
ACM Computing Surveys (CSUR), 38(3):7.

Pearson, 2013
Pearson, S. (2013).
Privacy, security and trust in cloud computing.
In Privacy and Security for Cloud Computing, pages 3-42. Springer.

Prince, 2004
Prince, M. (2004).
Does active learning work? A review of the research.
Journal of engineering education, 93(3):223-231.

Romero-Zaldivar et al., 2012
Romero-Zaldivar, V.-A., Pardo, A., Burgos, D., and Delgado Kloos, C. (2012).
Monitoring student progress using virtual appliances: A case study.
Computers & Education, 58(4):1058-1067.

Sumner, 2000
Sumner, J. (2000).
Serving the system: A critical history of distance education.
Open learning, 15(3):267-285.

Watson, 2008
Watson, J. (2008).
Virtualbox: Bits and bytes masquerading as machines.
Linux J., 2008(166).

Winckles et al., 2011
Winckles, A., Spasova, K., and Rowsell, T. (2011).
Remote laboratories and reusable learning objects in a distance learning context.
Networks, 14:43-55.

Yeung et al., 2010
Yeung, H., Lowe, D. B., and Murray, S. (2010).
Interoperability of remote laboratories systems.
iJOE, 6(S1):71-80.

26 March 2015

25 March 2015

Work around Google evil .ics feeds I've happily been using 2015/akonadi-install for my calendars, and yesterday I added an .ics feed export from Google, as a URL file source. It is a link in the form: https://www.google.com/calendar/ical/person%40gmail.com/private-12341234123412341234123412341234/basic.ics After doing that, I noticed that the fan in my laptop was on more often than usual, and I noticed that akonadi-server and postgres were running very often, and doing quite a lot of processing. The evil I investigated and realised that Google seems to be doing everything they can to make their ical feeds hard to sync against efficiently. This is the list of what I have observed Gmail doing to an unchanged ical feed:

Date: headers in HTTP replies are always now
If-Modified-Since: is not supported
DTSTAMP of each element is always now
VTIMEZONE entries appear in random order
ORGANIZER CN entries randomly change between full name and plus.google.com user ID
ATTENDEE entries randomly change between having a CN or not having it
TRIGGER entries change spontaneously
CREATED entries change spontaneously

This causes akonadi to download and reprocess the entire ical feed at every single poll, and I can't blame akonadi for doing it. In fact, Google is saying that there is a feed with several years worth of daily appointments that all keep being changed all the time. The work-around As a work-around, I have configured the akonadi source to point at a local file on disk, and I have written a script to update the file only if the .ics feed has actually changed. Have a look at the script: I consider it far from trivial, since it needs to do a partial parsing of the .ics feed to throw away all the nondeterminism that Google pollutes it with. The setup The script needs to be run periodically, and I used it as an opportunity to try systemd user timers:

    $ cat ~/.config/systemd/user/update-ical-feeds.timer
    [Unit]
    Description=Updates ical feeds every hour
    # Only run when on AC power
    ConditionACPower=yes
    [Timer]
    # Run every hour
    OnActiveSec=1h
    # Run a minute after boot
    OnBootSec=1m
    Unit=update-ical-feeds.service
    $ cat ~/.config/systemd/user/update-ical-feeds.service
    [Unit]
    Description=Update ICal feeds
    [Service]
    # Use oneshot to prevent two updates being run in case the previous one
    # runs for more time than the timer interval
    Type=oneshot
    ExecStart=/home/enrico/tmp/calendars/update
    $ systemctl --user start update-ical-feeds.timer
    $ systemctl --user list-timers
    NEXT                         LEFT       LAST                         PASSED UNIT                    ACTIVATES
    Wed 2015-03-25 22:19:54 CET  59min left Wed 2015-03-25 21:19:54 CET  2s ago update-ical-feeds.timer update-ical-feeds.service
    1 timers listed.
    Pass --all to see loaded but inactive timers, too.

To reload the configuration after editing: systemctl --user daemon-reload. Further investigation I wonder if ConditionACPower needs to be in the .timer or in the .service, since there is a [Unit] section is in both. Update: I have been told it can be in the .timer. I also wonder if there is a way to have the timer trigger only when online. There is a network-online.target and I do not know if it is applicable. I also do not know how to ask systemd if all the preconditions are currently met for a .service/.timer to run. Finally, I especially wonder if it is worth hoping that Google will ever make their .ics feeds play nicely with calendar clients.

Francois Marier: Keeping up with noisy blog aggregators using PlanetFilter

I follow a few blog aggregators (or "planets") and it's always a struggle to keep up with the amount of posts that some of these get. The best strategy I have found so far to is to filter them so that I remove the blogs I am not interested in, which is why I wrote PlanetFilter.

Other options In my opinion, the first step in starting a new free software project should be to look for a reason not to do it So I started by looking for another approach and by asking people around me how they dealt with the firehoses that are Planet Debian and Planet Mozilla. It seems like a lot of people choose to "randomly sample" planet feeds and only read a fraction of the posts that are sent through there. Personally however, I find there are a lot of authors whose posts I never want to miss so this option doesn't work for me. A better option that other people have suggested is to avoid subscribing to the planet feeds, but rather to subscribe to each of the author feeds separately and prune them as you go. Unfortunately, this whitelist approach is a high maintenance one since planets constantly add and remove feeds. I decided that I wanted to follow a blacklist approach instead.

PlanetFilter PlanetFilter is a local application that you can configure to fetch your favorite planets and filter the posts you see. If you get it via Debian or Ubuntu, it comes with a cronjob that looks at all configuration files in /etc/planetfilter.d/ and outputs filtered feeds in /var/cache/planetfilter/. You can either:

add file:///var/cache/planetfilter/planetname.xml to your local feed reader
serve it locally (e.g. http://localhost/planetname.xml) using a webserver, or
host it on a server somewhere on the Internet.

The software will fetch new posts every hour and overwrite the local copy of each feed. A basic configuration file looks like this:

[feed]
url = http://planet.debian.org/atom.xml
[blacklist]

Filters There are currently two ways of filtering posts out. The main one is by author name:

[blacklist]
authors =
  Alice Jones
  John Doe

and the other one is by title:

[blacklist]
titles =
  This week in review
  Wednesday meeting for

In both cases, if a blog entry contains one of the blacklisted authors or titles, it will be discarded from the generated feed.

Tor support Since blog updates happen asynchronously in the background, they can work very well over Tor. In order to set that up in the Debian version of planetfilter:

Install the tor and polipo packages.

Set the following in /etc/polipo/config:

 proxyAddress = "127.0.0.1"
 proxyPort = 8008
 allowedClients = 127.0.0.1
 allowedPorts = 1-65535
 proxyName = "localhost"
 cacheIsShared = false
 socksParentProxy = "localhost:9050"
 socksProxyType = socks5
 chunkHighMark = 67108864
 diskCacheRoot = ""
 localDocumentRoot = ""
 disableLocalInterface = true
 disableConfiguration = true
 dnsQueryIPv6 = no
 dnsUseGethostbyname = yes
 disableVia = true
 censoredHeaders = from,accept-language,x-pad,link
 censorReferer = maybe

Tell planetfilter to use the polipo proxy by adding the following to /etc/default/planetfilter:
```
 export http_proxy="localhost:8008"
 export https_proxy="localhost:8008"
```

Bugs and suggestions The source code is available on repo.or.cz. I've been using this for over a month and it's been working quite well for me. If you give it a go and run into any problems, please file a bug! I'm also interested in any suggestions you may have.

23 March 2015

Carl Chenet: Unverified backups are useless. Automatize the controls!

Follow me on Identi.ca

or Twitter

or Diaspora*

Unverified backups are useless, every sysadmins know that. But manually verifying a backup means wasting time and resources. Moreover it s boring. You should automatize it!

Charlie Chaplin Modern Times

Backup Checker is a command line software developed in Python 3.4 on GitHub (stars appreciated :) ) allowing users to verify the integrity of archives (tar, gz, bz2, lzma, zip, tree of files) and the state of the files inside an archive in order to find corruptions or intentional of accidental changes of states or removal of files inside an archive.

Backup Checker on github

The new feature of the latest version 1.4 is the control of outdated archives with the new outdated parameter. Lots of data are outdated quite fast, because they are dependent of other data, or because they are only useful in a specific context.

Hey, this database dump is 6 months old, it s useless today!

Backup Checker now controls the expiration duration and triggers a warning if the given duration starting from the last modification of the archive (mtime) is expired. Short examples of the warning: WARNING:root:/backups/backups-12022015.tar.gz is outdated. Was good until 01/03/15 00:00:00 now 22/03/15 21:38:20 You won t be surprized any more by outdated useless data in your backups. Backup Checker also offers lots of other controls. Check the features list! Installing Backup Checker Backup Checker is available from PyPI using the following command:

# pip3.4 install backupchecker

It s also available for your Debian Squeeze or Debian Wheezy. Check how to get it for your specific distributions.

What about you? How and what for do you use Backup Checker? We would be happy to get your feedbacks. The project cares about our users and the outdated feature was a awesome idea in a feature request by one of the Backup Checker user, thanks Laurent!

Martin-Éric Racine: This and That

I haven't blogged anything in months and figured that now might be a good time to get around that. Here it goes:
Free Software
While I occasionally upgrade the packaging of the software I maintain at Debian to keep up with best practices, my activity downsizing goes on. Simply put: I never had any ambition to become a Debian Developer. My involvement has always remained pragmatic and mostly from the perspective of packaging software that I found useful. Even then, my motivation for doing that keeps on dwindling into nothingness, because key pieces of software keep on breaking, whenever someone upstream decides to reinvent the wheel.
For instance, GNOME no longer works at all on Geode chipsets and it barely works on Nouveau chipsets. This happened as soon as GNOME 3.14 was uploaded into unstable, right before the freeze started. Then again, I wouldn't jump to a conclusion that GNOME itself might be at fault, since Plymouth also stopped working on the same two video platforms at the same time. For all we know, this could be caused by some changes in the X.Org server code. Bugs were filed, additional information was provided, but no fix has taken place.
Given how Geode and Nouveau represent 80% of my hardware investment (my Intel laptop being the sole exception), it essentially means that the upcoming Debian "stable" is useless for me. Now try and remain motivated, even just as a mere Free Software end-user. At this point, I'm done.
Politics
Finland is holding national elections this April. I still have no idea who I'll vote for this time. The guy I voted for last time has become a career politician with an inflated ego and zero connection to the average Finn's aspirations and worries. Meanwhile, two friends are standing as candidates: one who is a razor-sharp fact finder and who is a proven pragmatic decision-maker, but whose values are slightly off with mine, and one whose actions come straight from the heart but whose concept of today's Finnish reality leaves a lot to be desired.
National Defence
There's been a lot of recent articles about how former hardware and locations of the Finnish defence forces and border guards have been sold, often for peanuts, to Russian interests. In some cases, we're only talking about buildings formerly used for on-site staff accommodations. In other cases, former patrol boats and navy harbours changed hands. Now, to top it all, it appears that our north-western neighbour, Norway, has sold a former submarine base to German investors who, in turn, leased it to you guessed it Russian interests.
Looking at Russian actions in Ukraine, I cannot help but feel great concern that strategic locations are falling into potentially dangerous hands. Just seeing the picture of a former navy harbour with a handful of patrol boats on standby, right on the Finnish coastline, half-way between Helsinki and Turku, was a sobering experience. While the whole idea of shooting at people even invading armies gives me the creeps, at this point, I cannot help but start pondering whether defending this country might in fact be an occupation worth training for.
Employment
It has now been 6 years since I held my last dayjob. Since then, the only thing I've found is an unpaid training in the national bureaucracy. I've also freelanced as an actor and model, but that barely brought me pocket change, if even that. Seeing my face on posters advertising a movie I participated in last year was indeed nice, getting some media attention in connection to that too, but it hasn't lead to additional gigs. As far as I can tell, this was just my Warholian 15 minutes of fame.
However, there's a larger issue at stake. Newspapers recently published an employment statistics map for Nordic countries and the truth couldn't be more bleak: while Norway and Sweden's employment figures are nearly spotless for almost every province, those of Finland are save for a couple of mildly successful provinces outright catastrophic. Given this and despite feeling relatively happy living in Finland and having developed a will to defend this country from an eventual Russian assault, I've come to the conclusion that I would be better off going West, with a strong preference for Norway.
Now, the main question is, doing what? 6 years later, I have strong doubts that I would be remotely considered for any high-tech job. Besides, come to think of it, I wouldn't want any new office job. Off the top of my head, my idea of a cool job that would allow me to stay physically fit would be working as a tourist guide in Lapland. However, if Norway is anything like Finland, someone probably needs a dozen of permits of all sorts (first aid certification, C or even D class driving license, college degree in tourism, etc.) that I cannot afford. What then?

Mario Lang: Why is Qt5 not displaying Braille?

While evaluating the cross-platform accessibility of Qt5, I stumbled across this deficiency:

#include <QApplication>
#include <QTextEdit>
int main(int argv, char **args)
 
  QApplication app(argv, args);
  QTextEdit textEdit;
  textEdit.setText(u8"\u28FF");
  textEdit.show();
  return app.exec();

(compile with -std=c++11). On my system, this "application" does not show the correct glyph always. Sometimes, it renders a a white square with black border, i.e., the symbol for unknown glyph. However, if I invoke the same executable several times, sometimes, it renders the glyph correctly. In other words: The glyph choosing mechansim is apparently non-deterministic!!! UPDATE: Sune Vuorela figured out that I need to set QT_HARFBUZZ=old in the environment for this bug to go away. Apparently, harfbuzz-ng from Qt 5.3 is buggy.

22 March 2015

Lars Wirzenius: Obnam 1.9 released (backup software)

I have just released version 1.9 of Obnam, my backup program. See the website at http://obnam.org for details. The new version is available from git (see http://git.liw.fi) and as Debian packages from http://code.liw.fi/debian. Due to the freeze of Debian for the jessie release, I've not uploaded this version to Debian yet (not experimental and not backports). This is the first Obnam release since May 13, 2014, 313 days ago. That's a long time. I make no excuses: Obnam is a hobby project, which I work on when I have the time and energy. The past year has been very /interesting/ year for me, in all sorts of stressful ways: I've changed jobs, moved to another country, and dealt with the loss of a close relative. Because of this, I've not been able to spend as much time on Obnam as I'd like. The NEWS file extract below gives the highlights of what has happened to Obnam during this time. There's been a lot of things, actually. My plans for Obnam next are mainly centered around performance. This will require developing a new repository format, to allow things that are not possible with the current format. For example, the current format stores each data chunk in its own file in the repository, and that is quite wasteful when live data files (and therefore their chunks) are quite small. As preparation for this work, the silly-looking "simple" format has been added, mostly to make sure the internal code infrastructure is ready to support multiple repository formats in the same Obnam version. Those interested in discussing ways to make Obnam fast should join the obnam-dev mailing list. Version 1.9, released 2015-03-22 New features:

James Vasile changed Obnam so it can backup an individual file, instead of an entire directory.
James Vasile added the --include option to Obnam, allowing one to include files that would otherwise be excluded (see --exclude).
Carlo Teubner changed obnam fsck to remove unused chunks, if the --fsck-fix or --fsck-rm-unused settings are used. He also made it not check for unused chunks when it's useless to do so, because of various --fsck-skip settings are used.
A start of a French translation of the manual by pedrito2.
Ian Cambell provided a new Obnam command, obnam kdirstat, which makes the KDE k4dirstat utility be able to show graphically which parts of a backup generation use most space.
Lars Wirzenius added the simple repository format, which is for demonstration only. It is much too simplistic to be used for real.

Minor changes:

The manual page and obnam --help are now clearer that the --root setting and command line arguments to obnam backup can be SFTP URLs. Thanks to Simone Piccardi for reporting the issue.
David Fries filled in the displayed file permission mode bits.
Grammar and typo fixes for the obnam.1 manual page, from Jean Jordaan.
Tom Chiverton suggested a clarification to the manual page for "obnam mount" to say that each generation is a subdirectory.
David Fries changed restore to set the group ownership if possible even when not root. No warnings are issued if the attempt fails.
Jan Niggemann added a little to the German translation of the Obnam manual.
Lars Wirzenius added the path to the error message about a missing chunk (R43272X).
Lars Wirzenius made the message at the end of a backup report more statistics about transfers during the backup.

Bug fixes:

The Obnam SFTP plugin would loop infinitely if it lost the connection to the SSH server while creating a temporary file. Itamar Turner-Trauring provided a fix for this.
Will Dyson fixed a bug about locking while removing checkpoint generations.
Michel Alexandre Salim fixed a Python 2.6 compatibility problem in the unit tests (use of assertRaises as a context manager).
Lars Kruse fixed a bug with backing up of overlapping backup roots (e.g., / and /boot), given a test case by Adrien Clerc.
Thomas Eschenbacher fixed a bug in the format 6 repository code that would crash when there is an obscure problem and a B-tree code can't be found in the tree.
Tom Chiverton pointed out that the manual page was using "obnam restore" instead of "obnam mount" in an example for "obnam mount".
The yarn test suite now runs FUSE tests (obnam mount) when fusermount is available, rather than checking for membership in the group fuse. The latter is a Debianism (fixed in Debian jessie).
Thomas Waldmann noticed that obnam verify didn't notice that a file had new data, when the modification time was the same. Obnam now notices this.
Thomas Waldmann fixed many typos and minor bugs in the source code.
Laurence Perkins reported that the Tahoe-LAFS SFTP server returned some stat fields as None. Fixed to change those to be 0 instead.
Lars Wirzenius fixed double-downloading of chunks during restores.

20 March 2015

Zlatan Todori : Interviews with FLOSS developers: Laura Arjona

One of fresh additions to Debian, that is showing Debian's commitment to diversity in all fields is Laura Arjona Reina. A helpful hand on channels and a great flux of FLOSS energy she brings with herself. Although applied for non-packaging Debian Developer status, Laura does recognize that there are still some technical aspects what must grasp on. Her dedication to FLOSS and trying to solve some of its issues is astonishing, as this woman is doing a lot of self-hosting and system administration. Yes, you read it right - she does all of that and still applied for non-packaging Debian Developer. She is perfect example how FLOSS enhances humans in many ways. Hello Laura. Picture of Laura

Who are you? I am Laura Arjona, I work as IT assistant at Technic University of Madrid, I am married and I have a son, and I use and promote free (libre) software both at job and at home and with friends. I have a nice time contributing in Debian and other FLOSS projects but I always want to do more than what I manage to actually do (I hope I can improve that, as time goes by... and maybe when I get retired I am the SuperLArjona that you wrote about!). What parts of FLOSS community are you engaged? I use Debian, and CyanogenMod + F-Droid in my phone. I coordinate the translation of the Debian website into Spanish, help with FSFE website translation too, and translate some other free software (GNU MediaGoblin, F-Droid, Android apps that I use, web services that we use at work...). I use and promote some free social networks: pump.io, GNU Social, and XMPP. My work/friends environment is mostly Windows/Android so I try to find/promote libre software replacements or interesting applications for them. I give free FLOSS stickers to everybody showing interest for libre software, and a nice Debian sticker if they finally install it in their computers. Setup of your main machine? My machine is a humble Compaq Mini 110C laptop (32bits, Atom N270@1.60GHz, 1GB RAM) and I have Debian Jessie (future-stable ATM) with xfce on it. I'm not tied to particular tools, for example I use Mousepad for editing here in my xfce, Kate in my desktop at work, nano in the server. The only "tuning" that I always do is to set a dark background for terminals and text editors, but I don't even switch to a desktop dark theme... (BTW I love Jessie's theme, "Lines"!). I know there are awesome pieces of software out there (hey emacs-org-mode!), but I just don't have fun having to learn them by myself (no LUG near, I'm afraid...). Some memorable moments from Debian conferences? I've only been at Barcelona MiniDebConf Women 2014 and it was great. It was memorable that I promoted the keysigning for that MiniDebConf, and came home with lots of signatures and papers to verify and sign... and then I was not remembering my GPG main key passphrase! so I hid under my desk for two months, and then, decided to start again (created a new key and tried to meet some Debian people in Madrid...). So I guess I should go to some (Mini)DebConf again. You are currently involved in process of becoming non-packaging Debian Developer - what made you take that step? I began translating in 2011, and since then, I enjoyed contributing in Debian (women, l10n-es, website, publicity). I'm quite regular with the translation work, and applying for DD is a plan to 'force' myself to find chunks of time to contribute more in the other areas too. I also believe that applying I may help other people to also apply or get more involved or become more visible. So here I am. Although you applied for non-packaging Debian Developer you recognize that there is still a technical learning curve in Debian even for that - what are the technical aspects a non-Developer should grasp? Well, I suppose it depends on the area you are contributing. In Publicity you find repositories in git (bits.debian.org), and subversion (Debian Project News). The website uses CVS (www.debian.org)... So you need the basics of 3 different version control systems to commit your changes (or send them to the mailing list and wait somebody to commit them). We use a mail robot to coordinate the translation work, so you need to write the subject with a certain format, and some people complains when somebody send mail in HTML (plain text is preferred). There are some other tools such as IRC and GPG that I began using just for contributing to Debian. Once that you learn them a bit and you learn how Debian works, you understand they are the great tools and you get in love (hey meetbot and KGB! hey i18n.debian.org!), but I wonder how people with no technical background, or even Computer Engineering students nowadays, accostumed to instant messaging in the mobile, fancy web interfaces and so on, look at these tools and just don't even try. How do you see future of Debian development? I don't know, Debian is huge... Some areas in which I hope we, as a community, find the way to work more: packaging (or help configuring) web applications or network services, provide LTS support, and keep on improving outreach/diversity. What are your future plans in Debian, what would you like to work on? In the Spanish team area, my plan is to go on translating the website, jump more often into translating package descriptions too, and help first-time-contributors to keep themselves involved. In the website and publicity teams, I hope I manage to put some weekly time to help with pending tasks/bugs, and serve as liaison with the other areas in which I'm involved (women, l10n, contributors...). If I become DD, I would like to create/adopt some data sources for contributors.debian.org, or convince people to do it I'm not sure if I will be able to attend DebConf some year; meanwhile, at least, I'll continue trying to help with the blog and promotion (as -publicity-team member). Why should developers and users join Debian community? What makes Debian a great and happy place? For using it: the desktop experience has improved very much in the last years, there's a clear separation between free and non-free software so the Debian users always know where are we, and there is wide documentation (in English, at least. Probably in other languages too). For getting involved: I like very much that you can lurk what almost everybody does: just join some mailing lists or IRC channels, the Debian people work in the open. So you know a bit where are you jumping in. Later you learn that everything is easier than what was looking from the outside, because you make friends and with friends everything is better.
Contributions made to Debian have many chances to reach a very wide community: Debian users, upstream projects, and the hundreds of derivatives. It's a quite horizontal, decentralized organization (that has its downsides too, but I can live with them). Is there something you would change in FLOSS ecosystem? We need much more internationalization and localization efforts. People don't need English for using libre software nowadays in their desktop, it's one of our big strenghts, but they definitely need English for using libre software for Android, or solving problems with the libre software they use in your computers/devices, or to contribute to any community. I think we need more local groups for user support/outreach, more libre-software-based translation tools and online services (replacements for Google Translator, for Transifex...),and more internationalization and localization efforts (manuals, websites... not only the software itself). If we work hard in this area, we'll gain much more users and much more contributors. Why does privacy matter to you? I have a son, at our family we interchange photos, and sometimes I have private conversations using the smartphone, mail or other internet services. I want to have the chance that the day that I (or my family) need privacy, we can have it easily. And I want that the people that really need privacy today, have proper tools at their hand. So I try to use PGP, selfhost my multimedia website, use decentralized, free software based networks and XMPP mobile apps... to help those projects to thrive. I try do my part of the network effect! You are being upset with rise of Github - why is that and what would change would you like to see? I totally agree with Mako's essay "Free software needs free tools". Yes, many nonfree web services are easier and have better features right now, but the key is that only dogfooding can change that, the same like changed it with the GNU/Linux desktop and Open/LibreOffice for example...
I would like to see more people trying to selfhost, use and promote libre software based forges, so in addition to avoid vendor lock in and win consistency in our discourse, we polish the available tools and eventually win the battle also in the technical side. You are hosting yours own instances of Mediagoblin - as it is not officially packaged in Debian yet, how do you manage it and how would you encourage others to do it? I followed the MediaGoblin documentation for its last stable release, and hanged on the IRC channel when in doubts/problems. It was not so hard, because it's well documented for Debian systems, and most of the dependencies are already packaged (in stable and testing). MediaGoblin is in its way to stretch too (thanks simonft and the rest of people working on this!). I'm documenting my adventures with selfhosting in [my blog (http://larjona.wordpress.com), but I need to write more often, and put more time in my small server (now I try to selfhost my git projects with cgit, and I want to setup an XMPP server and Etherpad Lite too). You are trying to resolve with self-hosting personal issues with services such as WhatsUp and other non-free parts of our everyday lives- what issues are you hitting on during your way and how do you resolve them? "#iloveemail", but people don't love it anymore, it seems... I've researched a bit about instant messaging to try to propose alternatives to WhatsApp to my family and friends. It seems Conversations with a community XMPP server where to create multi user chat rooms can be a replacement, so my plan is to try during this year. Meanwhile, I've setup the MediaGoblin site so I upload the photos there instead of sending them with the phone, and for the 1-to-1 chat I try to move people to Kontalk (instant messaging, GPG, photos, voice notes...). For the videocalls, I promote Jitsi or just point people to meet.jit.si/FancyNameofChatRoom. I have account and owndrive.com and will try to host Owncloud too. We'll see what happens. You are interested in radio shows - what drives you into that field and will we see soon any podcasts from Laura? I like to talk and I'm not a shy person, so the few times that in any of my social groups there was a chance to "talk in the radio", I volunteered and enjoyed. This has been, in my life, 6 or 7 times (in Spanish, talking about social activism or politics. No records, though!). Some months ago the people of "El Binario" invited me to talk at "findenegro" about pump.io and free networks (in Spanish), I accepted and had a very nice time (audios in my mediagoblin). I wish I have more free time to listen to podcasts and maybe to join some other people to participate in a program in a regular basis. OTOH, my son asks for a tale almost each day... I follow one of Gianni Rodari "The grammar of fantasy"'s approach: take some day-to-day facts and add something unexpected and crazy, and tailor a short story. Maybe I could record them and publish in my MediaGoblin... Of course their literary quality is not even near to Gianni Rodari's but people that listened to them when I was storytelling (in the metro, my mother at home, some friends...) say they are fun and interesting. Who knows! Work of Laura's son

Antonio Terceiro: rrg: visualize the require hierarchy in Ruby projects

Yesterday I was hacking on some Ruby code and getting a weird error which I thought was caused by mutually recursive require statements (i.e. A requires B, and B requires A). Later I realized that this is not an issue in Ruby, since the intepreter keeps track of what has already been required and will not enter a loop. But during the investigation I came up with something that turned out to be useful. rrg will read the source code of a Ruby project and generate a graph based on the require statements in the code; nodes represent the source files and an arrow from A to B means that A contains a require B statement. From the README:

Just run rrg at the root of your project. rrg will parse the code inside lib/, and generate a graph description in the Graphviz format. You can pipe the output to Graphviz directly, or store it in a file and process it to generate an image.

If you call rrgv instead, it will automatically process the graph with Graphviz, generate a PNG image, and open it.

Let s see some examples. First the classical analysing itself example, the require graph for rrg itself:

Not very interesting, since all of the logic is currently in the main binary and not on library code. But 1) when I do the refactorings I want to, there will be more library code and 2) while writing this I implemented also parsing scripts in bin/. Now chake which is a slightly larger project:

An even larger (but still not that big) project, gem2deb:

Note that these visualizations may not be accurate representations of the actual source code. In Ruby, nothing stops one from implementing class A::B in lib/x/y.rb, but most reasonable code will make sure that filenames and the classes namespaces actually match. If you are working on a sane codebase, though, visualizing graphs like this helps understand the general structure of the code and perceive possible improvements. The gem2deb graph gave me some ideas already and I didn t even paid much attention to it yet.

Zlatan Todori : My journey into Debian

Notice: There were several requests for me to more elaborate on my path to Debian and impact on life so here it is. It's going to be a bit long so anyone who isn't interested in my personal Debian journey should skip it. :) In 2007. I enrolled into Faculty of Mechanical Engineering (at first at Department of Industrial Management and later transfered to Department of Mechatronics - this was possible because first 3 semesters are same for both departments). By the end of same year I was finishing my tasks (consisting primarily of calculations, some small graphical designs and write-ups) when famous virus, called by users "RECYCLER", sent my Windows XP machine into oblivion. Not only it took control over machine and just spawned so many processes that system would crash itself, it actually deleted all from hard-disk before it killed the system entirely. I raged - my month old work, full of precise calculations and a lot of design details, was just gone. I started cursing which was always continued with weeping: "Why isn't there an OS that can whithstand all of viruses, even if it looks like old DOS!". At that time, my roommate was my cousin who had used Kubuntu in past and currently was having SUSE dual-booted on his laptop. He called me over, started talking about this thing called Linux and how it's different but de facto has no viruses. Well, show me this Linux and my thought was, it's probably so ancient and not used that it probably looks like from pre Windows 3.1 era, but when SUSE booted up it had so much more beautiful UI look (it was KDE, and compared to XP it looked like the most professional OS ever). So I was thrilled, installed openSUSE, found some rough edges (I knew immediately that my work with professional CAD systems will not be possible on Linux machines) but overall I was bought. After that he even talked to me about distros. Wait, WTF distros?! So, he showed me distrowatch.com. I was amazed. There is not only a better OS then Windows - there where dozens, hundreds of them. After some poking around I installed Debian KDE - and it felt great, working better then openSUSE but now I was as most newbies, on fire to try more distros. So I was going around with Fedora, Mandriva, CentOS, Ubuntu, Mint, PCLinuxOS and in beginning of 2008 I stumbled upon Debian docs which where talking about GNU and GNU Manifesto. To be clear, I was always as a high-school kid very much attached to idea of freedom but started loosing faith by faculty time (Internet was still not taking too much of time here, youth still spent most of the day outside). So the GNU Manifesto was really a big thing for me and Debian is a social bastion of freedom. Debian (now with GNOME2) was being installed on my machine. As all that hackerdom in Debian was around I started trying to dig up some code. I never ever read a book on coding (until this day I still didn't start and finish one) so after a few days I decided to code tetris in C++ with thought that I will finish it in two days at most (the feeling that you are powerful and very bright person) - I ended it after one month in much pain. So instead I learned about keeping Debian system going on, and exploring some new packages. I got thrilled over radiotray, slimvolley (even held a tournament in my dorm room), started helping on #debian, was very active in conversation with others about Debian and even installed it on few laptops (I became de facto technical support for users of those laptops :D ). Then came 2010 which with negative flow that came in second half of 2009, started to crush me badly. I was promised to go to Norway, getting my studies on robotics and professor lied (that same professor is still on faculty even after he was caught in big corruption scandal over buying robots - he bought 15 years old robots from UK, although he got money from Norway to buy new ones). My relationship came to hard end and had big emotional impact on me. I fell a year on faculty. My father stopped financing me and stopped talking to me. My depression came back. Alcohol took over me. I was drunk every day just not to feel anything. Then came the end of 2010, I somehow got to the information that DebConf will be in Banja Luka. WHAT?! DebConf in city where I live. I got into #debconf and in December 2010/January 2011 I became part of the famous "local local organizers". I was still getting hammered by alcohol but at least I was getting out of depression. IIRC I met Holger and Moray in May, had a great day (a drop of rakia that was too much for all of us) and by their way of behaving there was something strange. Beatiful but strange. Both were sending unique energy of liberty although I am not sure they were aware of it. Later, during DebConf I felt that energy from almost all Debian people, which I can't explain. I don't feel it today - not because it's not there, it's because I think I integrated so much into Debian community that it's now a natural feeling which people here, that are close to me are saying that they feel it when I talk about Debian. DebConf time in Banja Luka was awesome - firstly I met Phil Hands and Andrew McMillan which were a crazy team, local local team was working hard (I even threw up during the work in Banski Dvor because of all heat and probably not much of sleep due to excitement), met also crazy Mexican Gunnar (aren't all Mexicans crazy?), played Mao (never again, thank you), was hanging around smart but crazy people (love all) from which I must notice Nattie (a bastion of positive energy), Christian Perrier (which had coordinated our Serbian translation effort), Steve Langasek (which asked me to find physiotherapist for his co-worker Mathias Klose, IIRC), Zach (not at all important guy at that time), Luca Capello (who gifted me a swirl on my birthday) and so many others that this would be a post for itself just naming them. During DebConf it was also a bit of hard time - my grandfather died on 6th July and I couldn't attend the funeral so I was still having that sadness in my heart, and Darjan Prtic, a local team member that came from Vienna, committed suicide on my birthday (23 July). But DebConf as conference was great, but more importantly the Debian community felt like a family and Meike Reichle told me that it was. The night it finished, me and Vedran Novakovic cried. A lot. Even days after, I was getting up in the morning having the feeling I need something to do for DebConf. After a long time I felt alive. By the end of year, I adopted package from Clint Adams and Moray became my sponsor. In last quarter of 2011 and beginning of 2012, I (as part of LUG) held talks about Linux, had Linux installation in Computer Center for the first time ever, and installed Debian on more machines. Now fast forwarding with some details - I was also on DebConf13 in Switzerland, met some great new friends such as Tincho and Santiago (and many many more), Santiago was also my roommate in Portland on the previous DebConf. In Switzerland I had really great and awesome time. Year 2014 - I was also at DebConf14, maintain a bit more packages and have applied for DD, met some new friends among which I must put out Apollon Oikonomopoulos and Costas Drogos which friendship is already deep for such a short time and I already know that they are life-long friends. Also thanks to Steve Langasek, because without his help I wouldn't be in Portland with my family and he also gave me Arduino. :) 2015. - I am currently at my village residence, have a 5 years of working experince as developer due to Debian and still a lot to go, learn and do but my love towards Debian community is by magnitude bigger then when I thought I love it at most. I am also going through my personal evolution and people from Debian showed me to fight for what you care, so I plan to do so. I can't write all and name all the people that I met, and believe me when I say that I remember most and all of you impacted my life for which I am eternally grateful. Debian, and it's community effect literally saved my life, spring new energy into me and changed me for better. Debian social impact is far bigger then technical, and when you know that Debian is a bastion of technical excellence - you can maybe picture the greatness of Debian. Some of greatest minds are in Debian but most important isn't the sheer amount of knowledge but the enormous empathy. I just hope I can in future show to more people what Debian is and to find all lost souls as me to give them the hope, to show them that we can make world a better place and that everyone is capable to live and do what they love. P.S. I am still hoping and waiting to see Bdale writing a book about Debian's history to this day - in which I think many of us would admire the work done by project members, laugh about many situations and have fun reading a book about project that was having nothing to do but fail and yet it stands stronger then ever with roots deep into our minds.

Noah Meyerhans: Building OpenWRT with Docker

I've run OpenWRT on my home router for a long time, and these days I maintain a couple of packages for the project. In order to make most efficient use of the hardware resources on my router, I run a custom build of the OpenWRT firmware with some default features removed and others added. For example, I install bind and ipsec-tools, while I disable the web UI in order to save space. There are quite a few packages required for the OpenWRT build process. I don't necessarily want all of these packages installed on my main machine, nor do I want to maintain a VM for the build environment. So I investigated using Docker for this. Starting from a base jessie image, which I created using the Docker debootstrap wrapper, the first step was to construct a Dockerfile containing instructions on how to set up the build environment and create a non-root user to perform the build:

FROM jessie:latest
MAINTAINER Noah Meyerhans <frodo@morgul.net>
RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get -y install \
asciidoc bash bc binutils bzip2 fastjar flex git-core g++ gcc
util-linux gawk libgtk2.0-dev intltool jikespg zlib1g-dev make \
genisoimage libncurses5-dev libssl-dev patch perl-modules \
python2.7-dev rsync ruby sdcc unzip wget gettext xsltproc \
libboost1.55-dev libxml-parser-perl libusb-dev bin86 bcc sharutils \
subversion
RUN adduser --disabled-password --uid 1000 --gecos "Docker Builder,,," builder

And we generate a docker image based on this Dockerfile per the docker build documentation. At this point, we've got a basic image that does what we want. To initialize the build environment (download package sources, etc), I might run:

docker run -v ~/src/openwrt:/src/openwrt -u builder -t -i jessie/openwrt
 sh -c "cd /src/openwrt/openwrt && scripts/feeds update -a"

Or configure the system:

docker run -v ~/src/openwrt:/src/openwrt -u builder -t -i
jessie/openwrt make -C /src/openwrt/openwrt menuconfig

And finally, build the OpenWRT image itself:

docker run -v ~/src/openwrt:/src/openwrt -u builder -t -i
jessie/openwrt make -C /src/openwrt/openwrt -j3

The -v ~/src/openwrt:/src/openwrt flags tell docker to bind mount my ~/src/openwrt directory (which I'd previously cloned using git) to /src/openwrt inside the running container. Without this, one might be tempted to clone the git repo directly into the container at runtime, but the changes to non-bind-mount filesystems are lost when the container terminates. This could be suitable for an autobuild environment, in which the sources are cloned at the start of the build and any generated artifacts are archived externally at the end, but it isn't suitable for a dev environment where I might be making and testing small changes at a relatively high frequency. The -u builder flags tell docker to run the given commands as the builder user inside the container. Recall that builder was created with UID 1000 in the Dockerfile. Since I'm storing the source and artifacts in a bind-mounted directory, all saved files will be created with this UID. Since UID 1000 happens to be my UID on my laptop, this is fine. Any files created by builder inside the container will be owned by me outside the container. However, this container should not have to rely on a user with a given UID running it! I'm not sure what the right way to approach this problem is within Docker. It may be that someone using my image should create their own derivative image that creates a user with the appropriate UID (creation of this derivative image is a cheap operation in Docker). Alternatively, whatever Docker init system is used could start as root, add a new user with a specific UID, and execute the build commands as that new user. Neither of these seems as clean as it could be, though. In general, Docker seems quite useful for such a build environment. It's easy to set up, and it makes it very easy to generate and share a common collection of packages and configuration. Because images are self-contained, I can reclaim a bunch of disk space by simple executing "docker rmi".

19 March 2015

Lior Kaplan: CVE assignment without upstream knowledge

In the past few months I ve been dealing with aligning PHP CVE information to enable easier tracking of security fixes. The two main locations are the NEWS file which is part of each release and the changelog available on the website which is more popular (and easier to update). Usually the CVE are assigned per PHP.net security team request or with cooperation with one of the Linux distribution s teams (either PHP or security), as should be in a good ecosystem. Recently I got a few notifications issued by Debian about its PHP package, which I wasn t familiar with these CVE IDS. When checking this, I found out a few CVE assigned per 3rd party (Linux distribution, bug reporter, etc ) request without upstream knowledge. Digging deeper I found out that some CVE were assigned a month after the fixes were released, while others were only a week or two after. While this makes sure the security information is documented, it s harder to add the information after tagging and releasing. In another case, while discussing about a CVE for a specific bug, we found out one was already assigned per the reporter s request but without the our or the upstream library knowledge. Even if the issue isn t severe, upstream should get a fair chance to fix issue before making them public. Which also leads to a problem with requesting CVE IDs on a public mailing list which in some cases leads to security information leakage. We should balance transparency with some grace period for upstreams (as projects share code).
Filed under: Debian GNU/Linux, PHP

17 March 2015

Raphaël Hertzog: Freexian s report about Debian Long Term Support, February 2015

Like each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In February, 58 work hours have been equally split among 4 paid contributors. Their reports are available:

Ben Hutchings
Holger Levsen (though he did only 9h out of the 14.5h allocated, he will have to catch up in March)
Rapha l Hertzog
Thorsten Alteholz

Evolution of the situation During the last month, we gained 3 paid work hours: we re now at 61 hours per month sponsored by 28 organizations and we have one supplementary sponsor in the pipe that should bring 4 more hours. The increase is not very quick but seems to be steady. Hopefully at some point, we will have enough resources to do a more exhaustive job. For now, the paid contributors handle in priority the most popular packages used by the sponsors and there are some packages in the end of the queue which have open security issues for months already (example: CVE-2012-6685 on libnokogiri-ruby). So, as usual, we are looking for more sponsors. In terms of security updates waiting to be handled, the situation looks a little bit worse than last month: the dla-needed.txt file lists 40 packages awaiting an update (3 more than last month), the list of open vulnerabilities in Squeeze shows about 58 affected packages in total (5 less than last month). We are getting a bit more effective with CVE triage. A logo for the LTS project? Every time that I write an LTS report, I remember that it would be nice if my LTS related articles could feature a nice picture/logo that reminds people of the LTS team/initiative. Is there anyone up for the challenge of creating that logo?

Thanks to our sponsors The new sponsors of the month are in bold.

Gold sponsors:

The Positive Internet

Silver sponsors:

Bronze sponsors:

No comment Liked this article? Click here. My blog is Flattr-enabled.

16 March 2015

Daniel Kahn Gillmor: Bootable grub USB stick (EFI and BIOS for Intel)

I'm using grub version 2.02~beta2-2. I want to make a USB stick that's capable of booting Intel architecture EFI machines, both 64-bit (x86_64) and 32-bit (ia32). I'm starting from a USB stick which is attached to a running debian system as /dev/sdX. I have nothing that i care about on that USB stick, and all data on it will be destroyed by this process. I'm also going to try to make it bootable for traditional Intel BIOS machines, since that seems handy.I'm documenting what I did here, in case it's useful to other people. Set up the USB stick's partition table:

parted /dev/sdX -- mktable gpt
parted /dev/sdX -- mkpart biosgrub fat32 1MiB 4MiB
parted /dev/sdX -- mkpart efi fat32 4MiB -1
parted /dev/sdX -- set 1 bios_grub on
parted /dev/sdX -- set 2 esp on

After this, my 1GiB USB stick looks like:

0 root@foo:~# parted /dev/sdX -- print
Model:  USB FLASH DRIVE (scsi)
Disk /dev/sdX: 1032MB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 
Number  Start   End     Size    File system  Name      Flags
 1      1049kB  4194kB  3146kB  fat32        biosgrub  bios_grub
 2      4194kB  1031MB  1027MB               efi       boot, esp
0 root@foo:~#

make a filesystem and mount it temporarily at /mnt:

mkfs -t vfat -n GRUB /dev/sdX2
mount /dev/sdX2 /mnt

ensure we have the binaries needed, and add three grub targets for the different platforms:

apt install grub-efi-ia32-bin grub-efi-amd64-bin grub-pc-bin grub2-common
grub-install --removable --no-nvram --no-uefi-secure-boot \
     --efi-directory=/mnt --boot-directory=/mnt \
     --target=i386-efi
grub-install --removable --no-nvram --no-uefi-secure-boot \
     --efi-directory=/mnt --boot-directory=/mnt \
     --target=x86_64-efi
grub-install --removable --boot-directory=/mnt \
     --target=i386-pc /dev/sdX

At this point, you should add anything else you want to /mnt here! For example:

a kernel and initramfs from your own computer
debian-installer ISOs
firmware blobs for that annoying proprietary device
BIOS updates
debirf images
grub-invaders
...

And don't forget to cleanup:

umount /mnt
sync

Tags: bios, efi, grub, tip

15 March 2015

Dimitri John Ledkov: My IDE needs a makeover

Current SetupI am a Linux Distribution Engineer and work on arbitrary open source projects. Mostly I'm patching/packaging existing things, and sometimes start fresh projects.

My "IDE", or rather I shall say "toolbox" is rather sparse:

GNOME Terminal
Google Chrome
GNU Emacs
GCC toolcahin with GDB
Python3 - iPython, iPdb, pyflakes
git, GNU bazaar
There are a few things that annoy me, and should be done better these days.

Documentation
I lookup documentation mostly with Google Chrome. This includes the texinfo renderings of the docs. There are a few reasons for that. First of all my developer machine is not polluted with all the dev packages under the sun, instead I compile practically everything in a chroot. And most of the time chroots have much newer versions of everything (from gcc & automake, to boost and whatever other dependencies are in use). However I would like to have easy generic lookup builtin for common things that I lookup in the references and which have not changed for a long time:
gcc builtins & defines
glibc functions
automake/autoconf functions definitions
Given that my preferred editor is Emacs, it should be natural to use info' mode to look things up. However, the rendering there is archaic and is really hard to read. At least when visiting the HTML renderings, the function names are in bold and stand out from the rest of the description.

Ideally I would have unified place to lookup docs, instead of using Google Chrome and navigating: gnu.org, gnome.org, readthedocs.org, freedesktop.org.

Project Management
I really hate "traditional" IDEs that create and pollute the working directories with random extra files. My project management tool is VCS, thus .git should be automatically recognized as a "project". I should be able to navigate repository files, have them scanned for tab-completion and jumping to symbols and the like. At the moment, I exit the editor and use git grep to find things and open those files in the editor again. I don't use any tagging systems at the moment, ideally git repository would be scanned and Exuberant Tags (this seems to be the latest hotness in tagging space) stored inside the .git directory automatically.

"SDK" aware aka chroot support
The IDE should be aware of chroots, how to compile things in a chroot and ideally how to compile packages with sbuild, mock or obs build (these are apt, yum and zypper preferred solutions for package compilation). Most importantly to use those chroots to tag includes headers for tab completion.

Shell
Gnome Terminal is good enough for my needs. I do have a problem of too many terminal windows... I have tried Terminator (a tiling single-window / multiple-tabs terminal). However during development the things I use shell for, should be part of the IDE directly: changing projects, opening/closing/navigating/creating files, invoking build, invoking debug, "refactoring" (sed). I think I do want to try out a pull-down terminal for temporal look-ups together with a tiling "main" terminal. Or ideally ditch it all together. Emacs does provide multiple terminals, but when I did that I ended up with "inception" -> launching an instance of emacs, inside the terminal, inside emacs...

Conclusion
If anybody has tips or suggestions do share. I will investigate and experiment with all of the above, and see if I can experiment and find new cool things that work better than my current setup.

13 March 2015

Dirk Eddelbuettel: Why Drat? A Guest Post by Steven Pav

Editorial Note: The following post was kindly contributed by Steven Pav.

Why Drat? After playing around with drat for a few days now, my impressions of it are best captured by Dirk's quote:
It just works.

Demo To get some idea of what I mean by this, suppose you are a happy consumer of R packages, but want access to, say, the latest, greatest releases of my distribution package, sadist. You can simply add the following to your .Rprofile file:

drat::add("shabbychef")

After this, you instantly have access to new releases in the github/shabbychef drat store via the package tools you already know and tolerate. You can use

install.package('sadists')

to install the sadists package from the drat store, for example. Similarly, if you issue

update.packages(ask=FALSE)

all the drat stores you have added will be checked for package updates, along with their dependencies which may well come from other repositories including CRAN.

Use cases The most obvious use cases are:

Micro releases. For package authors, this provides a means to get feedback from the early adopters, but also allows one to push small changes and bug fixes without burning through your CRAN karma (if you have any left). My personal drat store tends to be a few minor releases ahead of my CRAN releases.

Local repositories. In my professional life, I write and maintain proprietary packages. Pushing package updates used to involve saving the package .tar.gz to a NAS, then calling something like `R CMD INSTALL package_name_0.3.1.9001.tar.gz`. This is not something I wanted to ask of my colleagues. With drat, they can instead add the following stanza to .Rprofile: `drat:::addRepo('localRepo','file:///mnt/NAS/r/local/drat')`, and then rely on `update.packages` to do the rest.

I suspect that in the future, drat might be (ab)used in the following ways:

Rolling your own vanilla CRAN mirror, though I suspect there are better existing ways to accomplish this.

Patching CRAN. Suppose you found a bug in a package on CRAN (inconceivable!). As it stands now, you email the maintainer, and wait for a fix. Maybe the patch is trivial, but suppose it is never delivered. Now, you can simply make the patch yourself, pick a higher revision number, and stash it in your drat store. The only downside is that eventually the package maintainer might bump their revision number without pushing a fix, and you are stuck in an arms race of version numbers.

Forgoing CRAN altogether. While some package maintainers might find this attractive, I think I would prefer a single huge repository, warts and all, to a landscape of a million microrepos. Perhaps some enterprising group will set up a CRAN-like drat store on github, and accept packages by pull request (whether github CDN can or will support the traffic that CRAN does is another matter), but this seems a bit too futuristic for me now.

My wish list In exchange for writing this blog post, I get to lobby Dirk for some features in drat:

I shudder at the thought of hundreds of tiny drat stores. Perhaps there should be a way to aggregate `addRepo` commands in some way. This would allow curators to publish their suggested lists of repos.

Drat stores are served in the `gh-pages` branch of a github repo. I wish there were some way to keep the index.html file in that directory reflect the packages present in the sources. Maybe this could be achieved with some canonical RMarkdown code that most people use.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Wouter Verhelst: New toy: Fujitsu Lifebook e734

My Lenovo x220, which I've owned for almost four years now (I remember fetching it from the supplier shortly before driving off to Banja Luka), was getting somewhat worn out. The keyboard and the screen had both been replaced at some point already, and the wwan interface had given up as well. The case was all cracked, and the NIC connector wasn't doing very well anymore either; there have been a few cases of me trying to configure the wireless network at a customer, but this being harder than it needs to be because the NIC will only work if I put in the network cable just so, and someone dropped a piece of paper onto the cable. In other words, it was time for a new one. At first I wanted to buy a Lenovo x250, but then I noticed that the Fujitsu came with an i7 4712MQ, which I liked (as today it is still quite exceptional for an ultrabook to have a quadcore processor). Fujitsu also claims up to 9 hours of battery life, but it's not clear to me whether this is supposed to be the case with the default battery only. They also have a battery for the modular bay, which I bought as well (to replace the optical drive whic I sometimes use, but only rarely), and on top of that it came with a free port replicator. Not all is well, however. In the x220, getting the WWAN interface to work involved some creative use of chat against /dev/ttyACM0 wherein I issue a few AT commands to put the WWAN interface into a particular mode, and from then on the WWAN interface is just a regular Ethernet interface on which I can do DHCP. The new laptop has a "Sierra Wireless, Inc." WWAN interface (USB id 1199:9041) which annoyingly doesn't seem to expose the ttyACM (or similar) devices, and I'm not sure what to use instead. Just trying to do DHCP doesn't work -- yes, I tried. Unfortunately, the keyboard isn't very good; it's of the bubble gum type, and I keep getting annoyed at it not picking up my keystrokes all the time. When I'm at home or at my main customer, I have a Das Keyboard Ultimate S (3rd (customer) and 4th (home) generation), so it's only a problem when I'm not at home, but it's still extremely annoying. There is a "backlight" function in that keyboard, but that's not something I think I'll ever use (hint: "das keyboard ultimate s"). The display can't do more than 1366x768, which is completely and utterly wrong for a computer -- but it's the same thing as my x220, so it's not really a regression. The "brightness" ACPI keys don't seem to work. I may have to fiddle with some ACPI settings at some point, I suppose, but it's not a major problem. When I plugged it in, I noticed that fdpowermon ignored the second battery. I had originally written fdpowermon with support for such a second battery, but as my x220 had only one, I never tested it. Apparently there was a bug, but that's been fixed now -- at least in unstable. On the good side of the equation, it has three USB3 ports in the laptop, and four in the port replicator, with no USB2; this is a major leap forwards from the one USB3 and six USB2 in the x220. A positive surprise was the CCID smartcard reader that I somehow missed while reading the specs, but which -- given my current major customer, is very welcome, indeed. Update: After having used it a few days, there were a few minor annoyances:

Audio didn't work whenever I plugged the laptop to its port replicator and used the external screen. It took me a while to figure out that the default ALSA card (i.e., card 0) is the HDMI output, whereas card 1 is the PCH output, and that since I'm using the DVI port and analog audio, I hear nothing. To fix, create a .asoundrc containing:
```
  pcm.!default  
    type hw
    card 1
   
  ctl.!default  
    type hw
    card 1
   
```

Backlight didn't work. My .config/awesome/rc.lua now contains the following lines:

  awful.key(   , "XF86MonBrightnessDown", function() awful.util.spawn("xbacklight -dec 5") end),
  awful.key(   , "XF86MonBrightnessUp", function() awful.util.spawn("xbacklight -inc 5") end),
  awful.key(  modkey  , "XF86MonBrightnessDown", function() awful.util.spawn("xbacklight -set 1") end),
  awful.key(  modkey  , "XF86MonBrightnessUp", function() awful.util.spawn("xbacklight -set 100") end),

The lines with "modkey" allow me to go to "brightness max" or "brightness min" in one go, rather than have to hit fn+f6 or fn+f7 repeatedly, which is a useful extra.

It was suggested to me that ModemManager might be able to figure out how to enable the WWAN modem. The good news is that it detects the modem, and mmcli should have a way to enable things. The bad news is that mmcli -m 0 -e just comes back with "error: couldn't enable the modem: 'timed out'" (partially translated into Dutch). I haven't had the time to look into this much yet, but it seems to be another one of those dbus complications. To be continued, I'm sure.

12 March 2015

Enrico Zini: crypttab-reuse-passwords

Reuse passwords in /etc/crypttab Today's scenario was a laptop with an SSD and a spinning disk, and the goal was to deploy a Debian system on it so that as many things as possible are encrypted. My preferred option for it is to setup one big LUKS partition in each disk, and put a LVM2 Physical Volume inside each partition. At boot, the two LUKS partition are opened, their contents are assembled into a Volume Group, and I can have everything I want inside. This has advantages:

if any of the disks breaks, the other can still be unlocked, and it should still be possible to access the LVs inside it
once boot has happened, any layout of LVs can be used with no further worries about encryption
I can use pvmove to move partitions at will between SSD and spinning disks, which means I can at anytime renegotiate the tradeoffs between speed and disk space.

However, by default this causes cryptsetup to ask for the password once for each LUKS partition, even if the passwords are the same. Searching for ways to mitigate this gave me unsatisfactory results, like:

decrypt the first disk, and use a file inside it as the keyfile to decrypt the second one. But in this case if the first disk breaks, I also lose the data in the second disk.
reuse the LUKS session key for the first disk in the second one. Same problem as before.
put a detached LUKS header in /boot and use it for both disks, then make regular backups of /boot. It is an interesting option that I have not tried.

The solution that I found was something that did not show up in any of my search results, so I'm documenting it here:

    # <target name> <source device>   <key file>   <options>
    ssd             /dev/sda2         main         luks,initramfs,discard,keyscript=decrypt_keyctl
    spin            /dev/sdb1         main         luks,initramfs,keyscript=decrypt_keyctl

This caches each password for 60 seconds, so that it can be reused to unlock other devices that use it. The documentation can be found at the beginning of /lib/cryptsetup/scripts/decrypt_keyctl, beware of the leopard . main is an arbitrary tag used to specify which devices use the same password. This is also useful to work easily with multiple LUKS-on-LV setups:

    # <target name> <source device>          <key file>  <options>
    home            /dev/mapper/myvg-chome   main        luks,discard,keyscript=decrypt_keyctl
    backup          /dev/mapper/myvg-cbackup main        luks,discard,keyscript=decrypt_keyctl
    swap            /dev/mapper/myvg-cswap   main        swap,discard,keyscript=decrypt_keyctl

Next.

Previous.