Search Results: "tar"

25 August 2013

Joey Hess: southern fried science fiction with plum sauce

Had an afternoon of steak and science fiction. Elysium is only so-so, but look what we found in a bookstore that was half religious materials and half SF, local books, and carefully hidden romance novels:
Star Wars, Star Trek and the 21st Century Christians
Best part was at the end, when I finally found one of the local asian markets Tomoko tells us about when she casually pulls out the good stuff at family gatherings. I will be back for whole ducks, fresh fish, squid, 50 lb bags of rice, tamarind paste, fresh ginger that has not sat on the shelf for 2 months because only I buy it, etc. Only an hour from home in the woods! Between the garlic scapes, bean sprouts, enoki mushrooms, etc that I got for $10 today and this week's CSA surprise of 18 inch snake beans and smoked pork knuckles, I have the epic stir fry potential.. Spock and R2D2

Jonathan Carter: Still Alive

I ve been really quiet on the blogging front the last few months. There s been a lot happening in my life. In short, I moved back to South Africa and started working for another company. I have around 10 blog drafts that I will probably never finish, so I m just declaring post bankruptcy and starting from scratch (who wants read my old notes from Ubuntu VUDS from March anyway?) For what it s worth, I m still alive and doing very well, possibly better than ever. Over the next few months I want to focus my free software time on the Edubuntu project, my Debian ITPs (which I ve neglected so badly they ve been turned into RFPs) and Flashback. Once I ve caught up there s plenty of other projects where I d like to focus some energy as well (LTSP, Debian-Edu, just to name a few). Thanks for reading! More updates coming soon.

Vincent Bernat: Boilerplate for autotools-based C project

When starting a new HTML project, a common base is to use HTML5 Boilerplate which helps by setting up the essential bits. Such a template is quite useful for both beginners and experienced developers as it is kept up-to-date with best practices and it avoids forgetting some of them. Recently, I have started several little projects written in C for a customer. Each project was bootstrapped from the previous one. I thought it would be useful to start a template that I could reuse easily. Hence, bootstrap.c1, a template for simple projects written in C with the autotools, was born.

Usage A new project can be created from this template in three steps:
  1. Run Cookiecutter, a command-line tool to create projects from project templates, and answer the questions.
  2. Setup Git.
  3. Complete the todo list .

Cookiecutter Cookiecutter is a new tool to create projects from project templates. It uses Jinja2 as a template engine for file names and contents. It is language agnostic: you can use it for Python, HTML, Javascript or C! Cookiecutter is quite simple. You can read an introduction from Daniel Greenfeld. The Debian package is currently waiting in the NEW queue and should be available in a few weeks in Debian Sid. You can also install it with pip. Bootstrapping a new project is super easy: 
$ cookiecutter https://github.com/vincentbernat/bootstrap.c.git
Cloning into 'bootstrap.c'...
remote: Counting objects: 90, done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 90 (delta 48), reused 64 (delta 22)
Unpacking objects: 100% (90/90), done.
Checking connectivity... done
full_name (default is "Vincent Bernat")? Alfred Thirsty
email (default is "bernat@luffy.cx")? alfred@thirsty.eu
repo_name (default is "bootstrap")? secretproject
project_name (default is "bootstrap")? secretproject
project_description (default is "boilerplate for small C programs with autotools")? Super secret project for humans
Cookiecutter asks a few questions to instantiate the template correctly. The result has been stored in the supersecret directory: 
.
  autogen.sh
  configure.ac
  get-version
  m4
      ax_cflags_gcc_option.m4
      ax_ld_check_flag.m4
  Makefile.am
  README.md
  src
      log.c
      log.h
      Makefile.am
  secretproject.8
  secretproject.c
  secretproject.h
2 directories, 13 files

Remaining steps There are still some steps to be executed manually. You first need to initalize Git, as some features of this template rely on it: 
$ git init
Initialized empty Git repository in /home/bernat/tmp/secretproject/.git/
$ git add .
$ git commit -m "Initial import"
[...]
Then, you need to extract the todo list built from the comments contained in source files: 
$ git ls-tree -r --name-only HEAD   \
>   xargs grep -nH "T[O]DO:"   \
>   sed 's/\([^:]*:[^:]*\):\(.*\)T[O]DO:\(.*\)/\3 (\1)/'   \
>   sort -ns   \
>   awk '(last != $1)  print ""   last=$1 ; print '
2003 Add the dependencies of your project here. (configure.ac:52)
2003 The use of "Jansson" here is an example, you don't have (configure.ac:53)
2003 to keep it. (configure.ac:54)
2004 Each time you have used  PKG_CHECK_MODULES  macro (src/Makefile.am:12)
2004 in  configure.ac , you get two variables that (src/Makefile.am:13)
2004 you can substitute like above. (src/Makefile.am:14)
3000 It's time for you program to do something. Add anything (src/secretproject.c:76)
3000 you want here. */ (src/secretproject.c:77)
[...]
Only a few minutes are needed to complete those steps.

What do you get? Here are the main features:
  • Minimal configure.ac and Makefile.am.
  • Changelog based on Git logs and automatic version from Git tags2.
  • Manual page skeleton.
  • Logging infrastructure with variadic functions like log_warn(), log_info().
logging output of lldpd

About the use of the autotools The autotools are a suite of tools to provide a build system for a project, including:
  • autoconf to generate a configure script, and
  • automake to generate makefiles using a similar but higher-level language.
Understanding the autotools can be a quite difficult task. There are a lot of bad documentations on the web and the manual does not help by describing corner-cases that would be useful if you wanted your project to compile for HP-UX. So, why do I use it?
  1. I have invested a lot of time in the understanding of this build system. Once you grasp how it should be used, it works reasonably well and can cover most of your needs. Maybe CMake would be a better choice but I have yet to learn it. Moreover, the autotools are so widespread that you have to know how they work.
  2. There are a lot of macros available for autoconf. Many of them are included in the GNU Autoconf Archive and ready to use. The quality of such macros are usually quite good. If you need to correctly detect the appropriate way to compile a program with GNU Readline or something compatible, there is a macro for that.
If you want to learn more about the autotools, do not read the manual. Instead, have a look at Autotools Mythbuster. Start with a minimal configure.ac and do not add useless macros: a macro should be used only if it solves a real problem. Happy hacking!
  1. Retrospectively, I think boilerplate.c would have been a better name.
  2. For more information on those features, have a look at their presentation in a previous post about lldpd.

Francesca Ciceri: Some things I learnt at DebConf13

My self-tagged badge

24 August 2013

Marko Lalic: PTS rewrite: Django Memory Usage


Rewriting the PTS in a fairly large Web framework such as Django is sure to cause larger memory consumption by even the more simple tasks, when compared to a couple of (loosely related) perl scripts. If anything, simply due to the fact that the entire Django machinery needs to be loaded.

However, just how large the memory consumption difference between the two can be seen in the example of the command to dispatch received package emails to users. In the new version, the command is implemented as a Django management command, whereas before it was a perl script dispatch.pl relying on the common.pl script for some additional functions.

The reason this command's memory usage is important is an incident with the current test deployment of the new PTS found at http://pts.debian.net. All package mails that the old PTS receives are also forwarded to the new instance in order to expose the new implementation to real-world mail traffic. At a certain point, over a hundred mails were received instantly causing over a hundred dispatch processes to be launched. This led to the system running out of memory and the kernel had to run the OOM Killer which brought down PostgreSQL server thereby causing the site to become unavailable.

For now, such a problem seems to have been prevented from reoccurring by setting up exim to queue messages when a certain system average load is exceeded, however it is interesting to check how large the memory consumption difference between the old and new dispatch really is.

Measurement setupIn order to measure the memory use, two messages of different sizes were used: 1KB and 1MB.

For the new PTS two cases were considered, when the database used is sqlite3 and PostgreSQL. In both cases, it was simply initialized by syncdb, meaning it contained only the default keywords found in the initial fixture and no registered users and subscriptions. The DEBUG setting was set to False.

For testing the old PTS's dispatch.pl, the database was also initialized empty.

This way, the tests should show the actual difference between the two implementations.

Since the main concern here is the maximum memory usage of the dispatch script, a simple bash script was written which takes a PID of a running process and outputs the maximum memory usage as reported by ps once the process is terminated. The script used is shown below.

#!/usr/bin/env bash

pid=$1

while ps $pid >/dev/null
do
    ps -o vsz= $ pid 
    sleep 0.1
done   sort -n   tail -n1

Old PTS Measurements
The maximum memory usage of the old PTS was:
  1. 37.8 MB for the 1 KB message
  2. 38.2 MB for the 1 MB message

New PTS Measurements

sqlite3
The maximum memory usage of the new PTS using sqlite3 was:
  1. 91.77 MB for the 1 KB message
  2. 92.59 MB for the 1 MB message

PostgreSQL

The maximum memory usage of the new PTS using PostgreSQL was:
  1. 148 MB for the 1 KB message
  2. 160 MB for the 1 MB message

Comparison and DiscussionWhen comparing the memory used by the old dispatch.pl implementation and the new PTS running sqlite3, the difference does not seem to be too large and is to be expected, since, as mentioned, the whole Django framework is loaded when executing a management command.

However, the huge difference between the two versions of the new PTS: the one using sqlite3 and the one using PostgreSQL is very surprising indeed.

Another interesting measurement is the difference between running a bare Django management command: one including only a sleep statement (in order to allow enough time to measure its memory usage) on PostgreSQL and sqlite3. In both cases, the memory usage was only about 1 MB less than what the respective maximum memory usage when processing the 1 KB message.

All this considered, the logical conclusion seems to be that either psycopg2 package and/or the Django postgresql_psycopg2 database engine use a lot more memory than the corresponding sqlite3 alternatives.

Could anyone shed some more light as to what causes this stark difference between using sqlite3 and Postgres in Django? Anything that could be done to mitigate it?

Marko Lalic: PTS Rewrite Project Status

The rewrite has already achieved some of the major goals which have been set in the beginning, as well as made some changes and improvements along the way. Some of the major points of the new system implemented so far will be presented here.

    Tasks framework
    The current PTS implementation regenerates the HTML pages for all packages a few times per day. A goal of the rewrite was to make smaller incremental updates possible so that the information displayed for each package can be much more dynamic.

    In order to allow incremental updates, it is necessary to consider the situation where updating some information should trigger an update of other information.

    This is where the tasks framework comes in. It allows developers to define a task with a list of "events" that it produces. An event can be triggered by any change to shared information or anything else the task would like to signal. Each event is identified by a simple unique string name. Events can also contain arbitrary (JSON-serializable) objects. Other than events that it produces, a task defines a list of dependencies events it depends on.

    When a single task should be executed, the tasks framework uses this information to build a dependency graph (a DAG) and executes the tasks in topological sort order, making sure each task runs only after all tasks which could raise an event it depends on have finished. It automatically makes the raised events available to the task so that it can access the extra data. Tasks for which no event was raised are skipped.

    Such execution of a task along with all its dependencies is called a Job. A Job's state is persisted to the database after each task completes its operation. This allows Jobs to be continued if there is an unexpected error causing it to terminate.

    So far, the tasks framework has already proven useful. For example, there are tasks which extract some package information to a denormalized model suited for directly displaying the data in the package page after a new package version is detected in a repository. Other examples are tasks which update standards-version warnings (after a new package version is detected) and the news generation task based on various detected changes to the packages (new source version, version removed from repository, version migrated to another repository).

    Vendor-specific customizationAnother goal of the rewrite was to make the PTS less Debian-specific. This is so that derivatives could easily deploy their own versions of the PTS. The idea is to have a number of core PTS services which alternative deployments can customize and augment.

    The simplest way vendors can customize PTS is by changing some of the local settings values. An example of this would be changing the PTS_VENDOR_NAME which leads to this name being displayed in any part of the system.

    This is fairly uninteresting since the information is always static. What is more interesting is when the vendor should provide different information based on some context, e.g. package name, received message, etc. This requirement led to the concept of the vendor app with a rules module.

    Various core parts of the PTS specify hook functions which different vendors can implement to provide customized behavior of the core component. Some examples are:
    • providing URLs to a developer information site (e.g. http://qa.debian.org/developer.php in Debian's case) based on a developer's email
    • providing a URL to a bug tracker for a particular package (based on the package type and bug category type)
    • providing additional rules to tag received package mails with keywords
    • providing additional headers to be injected in the forwarded package mails
    • providing a list of bugs which should be displayed in the bugs panel
    A settings value is set giving the full dotted path to the Python module containing the implementation of these hook functions.

    Vendor-specific featuresA step beyond customizing core features is for vendors to include their own specific functionality. The already described tasks framework allows for implementing custom data update/generation processes (optionally depending on core events). Furthermore, since a vendor-specific Django app is expected, it can contain any vendor-specific models which are managed just as the core models are (syncdb, migrations, etc. all work as expected). The last piece of the puzzle is displaying this information on the package page.

    Currently, all package information is provided in various boxes (general, bugs, links, etc.) which were named panels. The rewrite has made it possible to implement additional panels in a really elegant way.

    It is required to implement a class with some properties giving:
    • the panel's title
    • position in the page (left, center, right)
    • panel importance (higher importance panels are placed above lower importance ones in the same column)
    To generate the HTML for a package, two alternatives are allowed. The first is to provide a property which gives (explicitly marked safe) HTML output which is then included in the correct position verbatim. The second alternative, which is what should cover most needs and should be preferred, is to provide a property giving a template name and a property giving the additional context variables necessary to render this template. By using the power of Python properties, the context can be dynamically generated for each package by the panel. The PTS then makes sure to pass the correct panel's context when rendering its template and to include the rendered result in the final package page.

    Various convenience classes are implemented in the panel API, but those are better left to be read about in the panels API documentation.

    Changes from current PTSSome existing features of PTS were not simply rewritten to fit into a Django project, but also improved upon.

    One of the first such changes was reducing the number of confirmation mails sent by the email control bot to only one per received control mail (as per a wishlist PTS bug).

    Work started by Markus Wanner (see this bug) was expanded on and integrated with the new News model so that any inline GPG signatures found in the content of a news item are automatically extracted and the signers name displayed in the package page.

    The existing TODO and Probems panels have been merged into a single Action Needed panel. However, besides merging their representation in the package page, all the package issues are now represented as an ActionItem Django model which provides additional meta-data (item type, date created/updated, severity level) which could be used in the future to provide additional functionality (such as all packages with a certain issue type).

    The descriptions of the action items were also shortened from their current rendition, with the detailed description being moved to a dedicated item page. If Javascript is enabled in the user's browser, it can also be displayed in a popup.

    GUIThe graphical interface has seen some changes, however I hesitate to claim improvements. In my opinion, some of it is definitely better, more polished and more "modern" (for what it's worth).

    Still, this could benefit from more people getting involved in testing and providing feedback so that the design is polished to suit the largest number of users. For example, the padding between the panels is increased, which to me definitely looks better, however, it is possible that this introduces too much whitespace and reduces the information density below what some may expect.

    RecapOverall, I am quite pleased with where the project is currently. Most of the information provided by the current PTS has been reimplemented, with only a few smaller ones remaining. For now, since we are nearly entering the final month of GSoC, the development will switch focus from porting old features to the new PTS to adding in some new ones. Namely, user registration and account management is in the pipeline for the coming week. It entails features such as managing package subscriptions via the online interface (for multiple emails associated with the account) and sso.debian.org integration for Debian Developers.

    This post aimed to provide the most notable changes and improvements made so far to the PTS. In reality, there has been quite a lot of them and not everything is covered here. For more information, check out the PTS documentation which is updated weekly (corresponding to the weekly deployments) or the source code repository.

    The latest iteration of the PTS can always be found at http://pts.debian.net. The mail control interface bot responds to mails sent to control@pts.debian.net.

    Stay tuned for updates and be sure to check out pts.debian.net on at least a weekly basis to see new deployments in action!

    23 August 2013

    Thorsten Glaser: FrOSCon 2013, or, why is there no MirBSD exhibit?

    FrOSCon is approaching, and all MirBSD developers will attend but why s there no MirBSD exhibit? The answer to that is a bit complex. First let s state that of course we will participate in the event as well as the Open Source world. We ll also be geocaching around the campus with other interested (mostly OSS) people (including those we won for this sport) and helping out other OSS projects we ve become attached to. MirOS BSD, the operating system, is a niche system. The conference on the other hand got younger and more mainstream. This means that almost all conference visitors do not belong to the target group of MirOS BSD which somewhat is an ancient solution : the most classical BSD around (NetBSD loses because they have rc.d and PAM and lack sendmail(8), sorry guys, your attempt at being not reformable doesn t count) and running on restricted hardware (such as my 486SLC with 12 MiB RAM) and exots (SPARCstation). It s viable even as developer workstation (if your hardware is supported otherwise just virtualise it) but its strength lies with SPARC support and embedded x86 . And being run as virtual machine: we re reportedly more stable and more performant than OpenBSD. MirBSD is not cut off from modern development and occasionally takes a questionable but justified choice (such as using 16-bit Unicode internally) or a weird-looking but beneficial one (such as OPTU encoding saving us locale(1) hassles) or even acts as technological pioneer (64-bit time_t on ILP32 platforms) or, at least, is faster than OpenBSD (newer GNU toolchain, things like that), but usually more conservatively, and yes, this is by design, not by lack of manpower, most of the time. The MirPorts Framework, while technically superiour in enough places, is something that just cannot happen without manpower. I (tg@) am still using it exclusively, continuing to update ports I use and occasionally creating new ones (mupdf is in the works!), but it s not something I d recommend someone (other than an Mac OSX user) to use on a n n-MirBSD system (Interix is not exactly thriving either, and the Interix support was only begun; other OSes are not widely tested). The MirBSD Korn Shell is probably the one thing I will be remembered for. But I have absolutely no idea how one would present it on a booth at such an exhibition. A talk is much more likely. So no on that front too. jupp, the editor which sucks less, is probably something that does deserve mainstream interest (especially considering Natureshadow is using it while teaching computing to kids) but probably more in a workshop setting. And booth space is precious enough in the FH so I think that d be unfair. All the other subprojects and side projects Benny and I have, such as mir c , josef stalin, FreeWRT, Lunix Ewe, Shellsnippets, the fonts, etc. are interesting but share few, if any, common ground. Again, this does not match the vast majority of visitors. While we probably should push a number of these more, but a booth isn t it here, either. MirOS Linux ( MirLinux ) and MirOS Windows are, despite otherwise-saying rumours called W*k*p*d*a, only premature ideas that will not really be worked on (though MirLinux concepts are found in mir c and stalin). As you can see, despite all developers having full-time dayjobs, The MirOS Project is far from being obsolete. We hope that our website visitors understand our reasons to not have an exhibition booth of our own (even if the SPARCstation makes for a way cool one, it s too heavy to lift all the time), and would like to point out that there are several other booths (commercial ones, as well as OSS ones such as AllBSD, Debian and (talking to) others) and other itineries we participate in. This year both Benny and I have been roped into helping out the conference itself, too (not exactly unvoluntarily though). The best way to talk to us is IRC during regular European geek hours (i.e. until way too late into the night which Americans should benefit from), semi-synchronously, or mailing lists. We sort of expect you to not be afraid to RTFM and look up acronyms you don t understand; The MirOS Project is not unfriendly but definitely not suited for your proverbial Aunt Tilly, newbies, desktop users, and people who aren t at least somewhat capable of using written English (this is by design).

    Justus Winter: No noweb anymore...

    ... which is probably a good thing. But here is the boot log you all have been waiting for: 
    start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1] exec init proc auth
INIT: version 2.88 booting
Using makefile-style concurrent boot in runlevel S.
Activating swap...done.
Checking root file system...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
/dev/hd0s1: clean, 44693/181056 files, 291766/723200 blocks
done.
Activating lvm and md swap...(default pager): Already paging to partition hd0s5!
done.
Checking file systems...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
done.
Cleaning up temporary files... /tmp.
Mounting local filesystems...done.
Activating swapfile swap...(default pager): Already paging to partition hd0s5!
done.
df: Warning: cannot read table of mounted file systems: No such file or directory
Cleaning up temporary files....
Configuring network interfaces...Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Listening on Socket//dev/eth0
Sending on   Socket//dev/eth0
*** stack smashing detected ***: dhclient terminated
Aborted
Failed to bring up /dev/eth0.
done.
Cleaning up temporary files....
Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix.
INIT: Entering runlevel: 2
Using makefile-style concurrent boot in runlevel 2.
Starting enhanced syslogd: rsyslogd.
Starting deferred execution scheduler: atd.
Starting periodic command scheduler: cron.
Starting system message bus: dbusFailed to set socket option"/var/run/dbus/system_bus_socket": Protocol not available.
Starting OpenBSD Secure Shell server: sshd.
unexpected ACK from keyboard
GNU 0.3 (debian) (console)
login: root
[...]
root@debian:~# ifup /dev/eth0
Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Listening on Socket//dev/eth0
Sending on   Socket//dev/eth0
*** stack smashing detected ***: dhclient terminated
Aborted
Failed to bring up /dev/eth0.
root@debian:~# dhclient -v -pf /run/dhclient.-dev-eth0.pid -lf /var/lib/dhcp/dhclient.-dev-eth0.leases /dev/eth0
Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Listening on Socket//dev/eth0
Sending on   Socket//dev/eth0
*** stack smashing detected ***: dhclient terminated
Aborted
root@debian:~# dhclient -pf /run/dhclient.-dev-eth0.pid -lf /var/lib/dhcp/dhclient.-dev-eth0.leases /dev/eth0
root@debian:~# ifup /dev/eth0
Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Listening on Socket//dev/eth0
Sending on   Socket//dev/eth0
DHCPREQUEST on /dev/eth0 to 255.255.255.255 port 67
DHCPACK from 10.0.2.2
bound to 10.0.2.15 -- renewal in 34108 seconds.
ps: comm: Unknown format spec
root@debian:~# halt
Broadcast message from root@debian (console) (Fri Aug 23 19:42:19 2013):
The system is going down for system halt NOW!
INIT: Switching to runlevel: 0root@debian:~#
INIT: Sending processes the TERM signal
INIT: Sending processes the KILL signal
Using makefile-style concurrent boot in runlevel 0.
Stopping deferred execution scheduler: atd.
task c10f53f8 deallocating an invalid port 2098928, most probably a bug.
Asking all remaining processes to terminate...done.
All processes ended within 1 seconds...done.
Stopping enhanced syslogd: rsyslogd.
Deconfiguring network interfaces...Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Listening on Socket//dev/eth0
Sending on   Socket//dev/eth0
DHCPRELEASE on /dev/eth0 to 10.0.2.2 port 67
/dev/eth0 (2):
  inet address  0.0.0.0
  netmask       255.255.255.0
  broadcast     10.0.2.255
  flags         BROADCAST ALLMULTI MULTICAST
  mtu           1500
done.
Deactivating swap...swapoff: /dev/hd0s5: 177152k swap space
done.
Unmounting weak filesystems...umount: /etc/mtab: Warning: duplicate entry for device /dev/hd0s1 (/servers/socket/26)
umount: /etc/mtab: Warning: duplicate entry for device /dev/hd0s1 (/dev/cons)
umount: could not find entry for: /dev/cons
umount: could not find entry for: /servers/socket/26
done.
mount: cannot remount /: Device or resource busy
Will now halt.
store a new irq 11init: notifying pfinet of shutdown...init: notifying tmpfs swap of shutdown...init: notifying tmpfs swap of shutdown...init: notifying tmpfs swap of shutdown...init: notifying ext2fs device:hd0s1 of shutdown...init: halting Mach (flags 0x8)...
In tight loop: hit ctl-alt-del to reboot
    With some tiny patches for ifupdown I've been able to resolve network related issues. All of them? Of course not, funny thing about developing for the Hurd is that once you fix one thing, then some other thing or code path is executed that has never been run on Hurd before, and therefore something else breaks. In this case I fixed ifupdown to generate valid names for the pid file and leases file and all of the sudden dhclient starts dying. Funny thing about that is, if one drops the -v flag from the dhclient invocation as I did it above, the crash isn't triggered and once the lease file has been successfully written, it is safe to add the -v flag again. Not yet sure what goes on there, then again, looking at the source of isc-dhcp-client it is not so surprising that it crashes :/ When I first looked at ifupdown it was written in noweb, a literate programming tool. It is an interesting idea, even more so since (classic) c can be very verbose and cryptic. But it decouples the control flow from the structure of the program, which makes patching it quite a challenge since it is not as obvious where the changes have to go in. This is how ifupdown looked some weeks ago: 
    % wc --lines ifupdown.nw
6123 ifupdown.nw
% pdftk ifupdown.pdf dump_data   grep NumberOfPages
NumberOfPages: 113
    The ifupdown.nw is the noweb source, from which seven .c, four .h, two .pl and one Makefile are generated. It also contains a redicioulus amount of documentation, to the point that the authors at several points did not now what to write and just drop some nonsensical lines into the file. The source also compiles to a 113 page pdf file, that contains all of the documentation and all of the code, not at all in the order that one would expect a program to be written, but in the order the authors chose to structure the documentation. Fortunately for me the maintainer decided to drop the noweb source and to add the generated files to the source control system. This made my job much easier :) So here are the patches I published this week: I must admit that I do not know exactly what I will do next week. Obviously fixing the dhclient crash would be nice, I'll look into that. But I'm surely find some useful thing to do.

    Gregor Herrmann: DebConf 2013

    [still no pictures here, feel free to go to gallery.debconf.org]. After my report about DebCamp, & after being home for 5 days now, here are some thoughts about DebConf13: First of all, I really enjoyed being there, & this was one of the best DebConfs for me so far. An important reason for this was the place: Le Camp was in my opinion the best DebConf venue ever (I've been to), with everything & everyone in one place, & still enough space (especially outdoors space!) for work & relaxing. Kudos to the local team for finding this spot, & for sticking to it despite some criticism. In general, I'd like to say thanks to the local team for the perfect organisation of all aspects you did a great job! What else did I enjoy? As usual, meeting old friends & making new ones was an important part. It was great to see people I've worked together with on-line in person for the first time, & also to get to know new people. Debian not only produces a great operating system, it also is an awesome community! Speaking of which: I attended quite a few seesions from the Community and Team tracks, & it's good to see that lots of energy & thoughts are going into improving how we deal with new or existing contributors. Like each year, we had the "use Perl;" BoF, the annual meeting of the Debian Perl Group, where we discussed various topics around our tasks, tools, & workflow. As each year, in a very friendly & collaborative atmosphere. Besides that, I attended talks about systemd & upstart; the first one disappointed me, since it felt more like a sales pitch than a technical talk & the presenter come across alot like "I know better than you what's good for you", the latter was better but tried to avoid upstart's main issue, the infamous Canonical Contributor License Agreement; of course the audience raised this topic in the Q&A part afterwards. Rock vs. hard place Other interesting topics were what Debian can learn from Ubuntu's QA processes for its own testing migration (mail from the Release Team pending, AFAIK); lots of git stuff, & especially the "birth" of dgit, a tool to treat the Debian archive as a git remote. If this gets traction, it could be a revolution in packaging! The lintian BoF encouraged me further to try to create some pkg-perl specific checks, & then there was jenkins, piuparts, autopkgtest, etc. On Friday we celebrated Debian's 20th birthday with a huge Krem nita/Cr meschnitte/mille-feuille. & with the Poetry Night at the campfire. Usually I'd close with "See you next year in Portland!"; but since it seems like the local team has decided to scratch DebCamp (which they only said after being asked in the Q&A after their presentation), I'm not sure how attractive this really is for me We'll see. Thanks again to everyone who made this DebConf possible!

    Jose Luis Rivas: Too many authentication failures on SSH

    It seems like having several SSH keys on the same system starts giving issues with some SSH servers. The issue is that while connecting it returns: Too many authentication failures for xxxxxx. The fix for this is very simple and quick. Open up your ~/.ssh/config and put: IdentitiesOnly yes You can add that per-host or wide (without indents, even as the first line in the config file!).

    22 August 2013

    Matthew Garrett: Re: Default offerings, target audiences, and the future of Fedora

    Eric (a fellow Fedora board member) has a post describing his vision for what Fedora as an end goal should look like. It's essentially an assertion that since we have no idea who our users are or what they want, we should offer them everything on an equal footing.

    Shockingly enough, I disagree.

    At the most basic level, the output of different Special Interest Groups is not all equal. We've had issues over the past few releases where various spins have shipped in a broken state, because the SIG responsible for producing them doesn't have the resources to actually test them. We're potentially going to end up shipping F20 with old Bluetooth code because the smaller desktops aren't able to port to the new API in time[1]. Promoting these equally implies that they're equal, and doing so when we know it isn't the case is a disservice to our users.

    But it's not just about our users. Before I joined the Fedora project, I'd worked on both Debian and Ubuntu. Debian is broadly similar to the current state of Fedora - no strong idea about what is actually being produced, and a desire among many developers to cater to every user's requirements. Ubuntu's pretty much the direct opposite, with a strongly defined goal and a willingness to sacrifice some use cases in order to achieve that goal.

    This leads to an interestingly different social dynamic. Ubuntu contributors know what they're working on. If a change furthers the well-defined aim of the project, that change happens. Moving from Ubuntu to Fedora was a shock to me - there were several rough edges in Fedora that simply couldn't be smoothed out because fixing them for one use case would compromise another use case, and nobody could decide which was more important[2]. It's basically unthinkable that such a situation could arise in Ubuntu, not just because there was a self appointed dictator but because there was an explicit goal and people could prioritise based on that[3].

    Bluntly, if you have a well-defined goal, people are more likely to either work towards that goal or go and do something else. If you don't, people will just do whatever they want. The risk of defining that goal is that you'll lose some of your existing contributors, but the benefit is that the existing contributors will be more likely to work together rather than heading off in several different directions.

    But perhaps more importantly, having a goal can attract people. Ubuntu's Bug #1 was a solid statement of intent. Being freer than Microsoft wasn't enough. Ubuntu had to be better than Microsoft products on every axis, and joining Ubuntu meant that you were going to be part of that. Now it's been closed and Ubuntu's wandered off into convergence land, and signing up to spend your free time on producing something to help someone sell phones is much less compelling than doing it to produce a product you can give to your friends.

    Fedora should be the obvious replacement, but it's not because it's unclear to a casual observer what Fedora actually is. The website proudly leads with a description of Fedora as a fast, stable and powerful operating system, but it's obvious that many of the community don't think of Fedora that way - instead it's a playground to produce a range of niche derivatives, with little consideration as to whether contributing to Fedora in that way benefits the project as a whole. Codifying that would actively harm our ability to produce a compelling product, and in turn reduce our ability to attract new contributors even further.

    Which is why I think the current proposal to produce three first-class products is exciting. Offering several different desktops on the download page is confusing. Offering distinct desktop, server and cloud products isn't. It makes it clear to our users what we care about, and in turn that makes it easier for users to be excited about contributing to Fedora. Let's not make the mistake of trying to be all things to all people.

    [1] Although clearly in this case the absence of a stable ABI in BlueZ despite it having had a dbus interface for the best part of a decade is a pretty fundamental problem.
    [2] See the multi-year argument over default firewall rules and the resulting lack of working SMB browsing or mDNS resolving
    [3] To be fair, one of the reasons I was happy to jump ship was because of the increasingly autocratic way Ubuntu was being run. By the end of my involvement, significant technical decisions were being made in internal IRC channels - despite being on the project's Technical Board, I had no idea how or why some significant technical changes were being made. I don't think this is a fundamental outcome of having a well-defined goal, though. A goal defined by the community (or their elected representatives) should function just as well.

    comment count unavailable comments

    Jose Luis Rivas: Starting a Raspbery Pi without a display

    Recently (actually 3 weeks ago) I bought a Raspberry Pi for myself and it wasn't until today that I power it on by the first time. Call it RealLife for simplicity. Anyway, one of my first issues was that I do not own any display with RCA or HDMI inputs, and all I have is a router that works as my WiFi hotspot as well. Since this starter kit comes with Raspbian by default I made some research and found that the default user:password is pi:raspberry, ssh-server is on by default and the network interface eth0 tries to grab an IP automatically via DHCP. So starting my Raspberry Pi was as easy as connect a wire to my device and the router, then a quick nmap -sP 192.168.0.0/24 and there it was, under the hostname raspberry my brand new raspberry. After a ssh pi@192.168.0.X a nice raspi-conf was receiving me to start setting up my device.

    Martín Ferrari: Setting up my server: re-installing on an encripted LVM

    Very long post ahead (sorry for the wall of text), part of a series of posts on some sysadmin topics, see post 1 and post 2. I want to show you how I set up my tiny dedicated server to have encrypted partitions, and to reinstall it from scratch. All of this without ever accessing the actual server console.

    Introduction As much as my provider may have gold standards on how to do things (they don't, there are some very bad practises in the default installation, like putting their SSH key into root's authorized_keys file), I wouldn't trust an installation done by a third party. Also, I wanted to have all my data securely encrypted. I know this is not perfect, and there are possible attacks. But I think it is a good barrier to have to deter entities without big budgets from getting my data. I have done this twice on my servers, and today I was reviewing each step as he was doing the same thing (with some slight differences) on his brand new server, so I think this is all mostly correct. Please, tell me if you find a bug in this guide. This was done on my 12 /month Kimsufi dedicated server, sold by OVH (see my previous post on why I chose it), and some things are specific to them. But you can do the same thing with any dedicated server that has a rescue netboot image. The process is to boot into the rescue image (this is of course a weak link, as the image could have a keylogger, but we have to stop the paranoia at some point), manually partition the disk, set-up encryption, and LVM; and then install a Debian system with debootstrap. To be able to unlock the encrypted disks, you will have to ssh into the server after a reboot and enter the passphrase (this is done inside the initrd phase). Once unlocked, the normal boot process continues. If anything fails, you end up with an unreachable system: it might or might have not booted, the disk might or might not be unlocked, etc. You can always go back into the rescue netboot image, but that does not allow you to see the boot process. Some providers will give you real remote console access, OVH charges you silly money for that. They used to offer a "virtual KVM", which was a bit of a kludge, but it worked: another netboot image that started a QEMU connected to a VNC server, so by connecting to the VNC server, you would be able to interact with the emulated boot process, but with a fake BIOS and a virtual network. For some unspecified reason they've stopped offering this, but there is a workaround available. The bottom line is, if you have some kind of rescue netboot image, you can just download and run QEMU on it and do the same trick.

    The gritty details Start by netbooting into your rescue image. For OVH, you'd go to the control panel, in the Services/Netboot section and select "rescue pro". Then reboot your server. OVH will mail you a temporary password when it finishes rebooting. Connect to it, without saving the temporary SSH key: 
    $ ssh -oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no root@$ IP
    For the rest of the text, I am assuming you have one hard drive called /dev/sda. We start by partitioning it: 
    # fdisk /dev/sda
    Start a new partition table with o, and then create two primary partitions: a small one for /boot at the beginning (100 to 300 MB would do), and a second one with the remaining space. Set both as type 83 (Linux), and don't forget to activate the first one, as this servers refuse to boot from the hard drive without that. Create the file system for /boot, and the encrypted device: 
    # mkfs.ext4 /dev/sda1
# cryptsetup -s 512 -c aes-xts-plain64 luksFormat /dev/sda2
    The encryption parameters are the same as the ones used by the Debian Installer by default, so don't change them unless you really know what you are doing. You will need to type a passphrase for the encrypted device, be sure not to forget it! This passphrase can later be changed (or secondary passphrases added) with the cryptsetup tool. Look up the crypt device's UUID, and save it for later: 
    # cryptsetup luksDump /dev/sda2   grep UUID:
UUID:           xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    Open the encrypted device (type the passphrase again), and set up the LVM volume group: 
    # cryptsetup luksOpen /dev/sda2 sda2_crypt
# pvcreate /dev/mapper/sda2_crypt
# vgcreate vg0 /dev/mapper/sda2_crypt
    Create the logical volumes, this is of course a matter of personal taste and there are many possible variations. This is my current layout, note that I put most of the "big data" in /srv. 
    # lvcreate -L 500m -n root vg0
# lvcreate -L 1.5g -n usr vg0
# lvcreate -L 3g -n var vg0
# lvcreate -L 1g -n home vg0
# lvcreate -L 10g -n srv vg0
# lvcreate -L 500m -n swap vg0
# lvcreate -L 100m -n tmp vg0
    Some possible variations:
    • You can decide to use a ramdisk for /tmp, so instead of creating a logical volume, you would add RAMTMP=yes to /etc/default/tmpfs.
    • You can merge / and /usr in one same partition, as neither of them change much.
    • You can avoid having swap if you prefer.
    • You can put /home in /srv, and bind mount it later.
    Now, create the file systems, swap space, and mount them in /target. Note that I like to use human-readable labels. 
    # for i in home root srv tmp usr var; do 
  mkfs.ext4 -L $i /dev/mapper/vg-$i; done
# mkswap -L swap /dev/mapper/vg0-swap
# mkdir /target
# mount /dev/mapper/vg0-root /target
# mkdir /target/ boot,home,srv,tmp,usr,var 
# mount /dev/sda1 /target/boot
# for i in home srv tmp usr var; do
  mount /dev/mapper/vg-$i /target/$i; done
# swapon /dev/mapper/vg-swap
    Don't forget to set the right permissions for /tmp. 
    # chmod 1777 /target/tmp
    If you want to do the /home on /srv, you'll need to do this (and then copy to /etc/fstab): 
    # mkdir /target/srv/home
# mount -o bind /target/srv/home /target/home
    The disk is ready now. We will use debootstrap to install the base system. The OVH image carries it, otherwise consult the relevant section in the Install manual for details. It is important that at this point you check that you have a good GPG keyring for debootstrap to verify the installation source, by comparing it to a good one (for example, the one in your machine): 
    # gpg /usr/share/keyrings/debian-archive-keyring.gpg
pub  4096R/B98321F9 2010-08-07 Squeeze Stable Release Key <debian-release@lists.debian.org>
pub  4096R/473041FA 2010-08-27 Debian Archive Automatic Signing Key (6.0/squeeze) <ftpmaster@debian.org>
pub  4096R/65FFB764 2012-05-08 Wheezy Stable Release Key <debian-release@lists.debian.org>
pub  4096R/46925553 2012-04-27 Debian Archive Automatic Signing Key (7.0/wheezy) <ftpmaster@debian.org>
    Now, for the actual installation. You can use any Debian mirror, OVH has their own in the local network. In OVH's case it is critical to specify the architecture, as the rescue image is i386. I didn't notice that and had to painfully switch architectures in place (which was absolutely not possible a couple of years ago). 
    # debootstrap --arch amd64 wheezy /target http://debian.mirrors.ovh.net/debian
    After a few minutes downloading and installing stuff, you almost have a Debian system ready to go. Since this is not D-I, we still need to tighten a few screws manually. Let's mount some needed file systems, and enter the brand new system with chroot: 
    # mount -o bind /dev /target/dev
# mount -t proc proc /target/proc
# mount -t sysfs sys /target/sys
# XTERM=xterm-color LANG=C.UTF-8 chroot /target /bin/bash
    The most critical parts now are to correctly save the parameters for the encrypted device, and the partitions and logical volumes. You'll need the UUID saved before: 
    # echo 'sda2_crypt UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx none luks' \
  > /etc/crypttab
    Create the file systems table in /etc/fstab. Here I use labels to identify the devices: 
    # file system   mount point type    options             dump    pass
LABEL=root      /           ext4    errors=remount-ro   0       1
LABEL=tmp       /tmp        ext4    rw,nosuid,nodev     0       2
LABEL=var       /var        ext4    rw                  0       2
LABEL=usr       /usr        ext4    rw,nodev            0       2
LABEL=home      /home       ext4    rw,nosuid,nodev     0       2
# Alternative home in /srv:
#/srv/home      /home       auto    bind                0       0
LABEL=srv       /srv        ext4    rw,nosuid,nodev     0       2
LABEL=boot      /boot       ext4    rw,nosuid,nodev     0       2
LABEL=swap      none        swap    sw                  0       0
    You can also just use the device mapper's names (/dev/mapper/<volume_group>-<logical_volume>), be sure not to use the /dev/<volume_group>/<logical_volume> naming, as there are some initrd-tools that choked on them. 
    # file system           mount point type    options             dump    pass
/dev/mapper/vg0-root    /           ext4    errors=remount-ro   0       1
/dev/mapper/vg0-tmp     /tmp        ext4    rw,nosuid,nodev     0       2
/dev/mapper/vg0-var     /var        ext4    rw                  0       2
/dev/mapper/vg0-usr     /usr        ext4    rw,nodev            0       2
/dev/mapper/vg0-home    /home       ext4    rw,nosuid,nodev     0       2
# Alternative home in /srv:
#/srv/home              /home       auto    bind                0   0
/dev/mapper/vg0-srv     /srv        ext4    rw,nosuid,nodev     0   2
/dev/sda1               /boot       ext4    rw,nosuid,nodev     0   2
/dev/mapper/vg0-swap    none        swap    sw                  0   0
    Some tools depend on /etc/mtab, which now is just a symbolic link: 
    # ln -sf /proc/mounts /etc/mtab
    Now configure the network. You most surely can use DHCP, but you might prefer static configuration, that's personal choice. For DHCP, it is very straightforward: 
    # cat >> /etc/network/interfaces
auto eth0
iface eth0 inet dhcp
    For static configuration, first find the current valid addresses and routes as obtained by DHCP: 
    # ip address
# ip route
    And then store them: 
    # cat >> /etc/network/interfaces
auto eth0
iface eth0 inet static
    address AAA.BBB.CCC.DDD/24
    gateway AAA.BBB.CCC.254
    pre-up /sbin/ip addr flush dev eth0   true
    Note that pre-up command I added; that is to remove the configuration that is to be done by the kernel during boot (more on that later), otherwise ifupdown will complain about existing addresses. If your provider does IPv6, add it too. For OVH, the IPv6 set-up is a bit weird, so you need to add the routes in post-up. Your default gateway is going to be your /64 prefix, with the last byte replaced by ff, and then followed by :ff:ff:ff:ff. As you can see, that gateway is not in your network segment, so you need to add an explicit route to it. They have some information, but it is completely unreadable. If your IPv6 address is 2001:41D0:1234:5678::1/64, you will add: 
    iface eth0 inet6 static
    address 2001:41D0:1234:5678::1/64
    post-up /sbin/ip -6 route add 2001:41D0:1234:56ff:ff:ff:ff:ff dev eth0
    post-up /sbin/ip -6 route add default via 2001:41D0:1234:56ff:ff:ff:ff:ff
    You probably don't want the auto-configured IPv6 addresses, so disable them via sysctl: 
    # cat >> /etc/sysctl.conf
# Disable IPv6 autoconf 
net.ipv6.conf.all.autoconf = 0
net.ipv6.conf.default.autoconf = 0
net.ipv6.conf.eth0.autoconf = 0
net.ipv6.conf.all.accept_ra = 0
net.ipv6.conf.default.accept_ra = 0
net.ipv6.conf.eth0.accept_ra = 0
    To have a working DNS resolver, we can use the local server (OVH in this case): 
    # cat > /etc/resolv.conf 
search $DOMAIN
nameserver 213.186.33.99
    The most important part of a new install: choose a host name (and make the system use it). 
    # echo $HOSTNAME > /etc/hostname
# hostname $HOSTNAME
# echo "127.0.1.1 $HOSTNAME.$DOMAIN $HOSTNAME" >> /etc/hosts
    If we want to speficy the BIOS clock to use UTC: 
    # echo -e '0.0 0 0.0\n0\nUTC' > /etc/adjtime
    Set up your time zone: 
    # dpkg-reconfigure tzdata
    Configure APT with your preferred mirrors. I also prevent APT from installing recommends by default. 
    # echo deb http://ftp2.fr.debian.org/debian wheezy main contrib non-free \
  >> /etc/apt/sources.list
# echo deb http://ftp2.fr.debian.net/debian wheezy-updates main contrib non-free \
  >> /etc/apt/sources.list
# echo deb http://security.debian.org/ wheezy/updates main contrib non-free \
  >> /etc/apt/sources.list
# echo 'APT::Install-Recommends "False";' > /etc/apt/apt.conf.d/02recommends
# apt-get update
    Before installing any package, let's make sure that the initial ram disk (initrd) that is going to be created will allow us to connect. There will be no chance of using the root password during boot. Your public key is usually found in $HOME/.ssh/id_rsa.pub. 
    # mkdir -p /etc/initramfs-tools/root/.ssh/
# echo $(YOUR_PUB_RSA_KEY) > /etc/initramfs-tools/root/.ssh/authorized_keys
    If you change this, or the host key stored at /etc/dropbear/dropbear_*_host_key, the /etc/crypttab, or any other critical piece of information for the booting process, you need to run update-initramfs -u. Now we can install the missing pieces: 
    # apt-get install makedev cryptsetup lvm2 ssh dropbear busybox ssh \
  initramfs-tools locales linux-image-amd64 grub-pc kbd console-setup
    During the installation you will have to choose where to install grub, I recommend directly on /dev/sda. Also, the magic initrd will be created. We want to double check that it has all the important pieces for a successful boot: 
    # zcat /boot/initrd.img-3.2.0-4-amd64   cpio -t conf/conf.d/cryptroot \
  etc/lvm/lvm.conf etc/dropbear/\* root/.ssh/authorized_keys sbin/dropbear
etc/lvm/lvm.conf
etc/dropbear/dropbear_dss_host_key
etc/dropbear/dropbear_rsa_host_key
sbin/dropbear
root/.ssh/authorized_keys
conf/conf.d/cryptroot
    All these files need to be there. Most critically, we need to check that the cryptroot file has the right information to access the root file system: 
    # zcat /boot/initrd.img-*   cpio -i --to-stdout conf/conf.d/cryptroot
target=sda2_crypt,source=UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx,key=none,rootdev,lvm=vg0-root
    If all that was correct, now we need to tell the kernel to configure the network as soon as possible so we can connect to the initrd and unlock the disks. This is done by passing a command-line option though grub. This should match what was done in /etc/network/interfaces: either DHCP or static configuration. For DHCP, this line should be changed in /etc/default/grub: 
    GRUB_CMDLINE_LINUX="ip=:::::eth0:dhcp"
    For static configuration: 
    GRUB_CMDLINE_LINUX="ip=MY_IP_ADDR::MY_DEFAULT_GW:MY_NETMASK::eth0:none"
    It is also a good idea to disable the quiet boot and graphical boot splash, in case we need to use QEMU to fix some booting issue: 
    GRUB_CMDLINE_LINUX_DEFAULT=""
GRUB_TERMINAL=console
    And make the changes effective: 
    # update-grub2
    Having fsck fix problems automatically can be a life-saver too: 
    # echo FSCKFIX=yes >> /etc/default/rcS
    Get some very useful packages: 
    # apt-get install vim less ntpdate sudo
    Create an user for yourself, possibly make it an administrator: 
    # adduser tincho
# adduser tincho sudo
# adduser tincho adm
    This is mostly done, exit the chroot, and unmount everything. 
    # exit  # the chroot.
# umount /target/ dev,proc,sys,boot,home,srv,tmp,usr,var 
# umount /target
# swapoff -a
# lvchange -an /dev/mapper/vg0-*
# cryptsetup luksClose sda2_crypt
    Disable the netboot option from your administration panel, reboot, and hope it all goes well. If you followed every step carefully, a few minutes later you should be able to ping your server. Use this snippet to enter the password remotely: 
    $ stty -echo; ssh -o UserKnownHostsFile=$HOME/.ssh/known_hosts.initramfs \
  -o BatchMode=yes root@"$HOST" 'cat > /lib/cryptsetup/passfifo'; \
  stty echo
    It is very important that you close the pipe (with control-D twice) without typing enter. For my servers, I have a script that reads the passphrase from a GPG-encrypted file and pipes it directly into the remote server. That way, I only type the GPG passphrase locally: 
    $ cat unlock.sh 
#!/bin/sh
BASE="$(dirname "$0")"
HOST="$1"
gpg --decrypt "$BASE"/key-"$HOST".gpg   \
    ssh -o UserKnownHostsFile="$BASE"/known_hosts.initramfs -o BatchMode=yes \
        root@"$HOST" 'cat > /lib/cryptsetup/passfifo'
    It might be a good idea to create a long, impossible to guess passphrase that you can use in the GPG-encrypted file, and that you can also print and store in somewhere safe. See the luksAddKey function in the cryptsetup(8) man page. Once again, if everything went right, a few seconds later the openSSH server will replace the tiny dropbear and you will be able to access your server normally (and with the real SSH host key). Hope you find this article helpful! I would love to hear your feedback.

    21 August 2013

    Rog rio Brito: SNA Acceleration vs. UXA Acceleration

    Seeing as recent versions of Xorg have made it into Debian's testing (which I am using right now, instead of my usual sid installations), I decided to see what they would bring me in terms of performance, since they have the SNA acceleration implemented as well as the standard UXA acceleration. Well, I did not have any other tests to perform, but below are the results on my wife's laptop (a Core i5 2410M), when running gtkperf: With UXA 
    GtkPerf 0.40 - Starting testing: Mon Aug 12 20:33:20 2013
GtkEntry - time:  0.19
GtkComboBox - time:  5.36
GtkComboBoxEntry - time:  3.59
GtkSpinButton - time:  0.60
GtkProgressBar - time:  0.41
GtkToggleButton - time:  0.73
GtkCheckButton - time:  0.48
GtkRadioButton - time:  0.97
GtkTextView - Add text - time: 21.72
GtkTextView - Scroll - time:  2.29
GtkDrawingArea - Lines - time:  6.18
GtkDrawingArea - Circles - time:  6.74
GtkDrawingArea - Text - time:  4.32
GtkDrawingArea - Pixbufs - time:  0.67
 --- 
Total time: 54.25
    With SNA 
    GtkPerf 0.40 - Starting testing: Mon Aug 12 20:36:58 2013
GtkEntry - time:  0.21
GtkComboBox - time:  4.86
GtkComboBoxEntry - time:  2.67
GtkSpinButton - time:  0.59
GtkProgressBar - time:  0.24
GtkToggleButton - time:  0.61
GtkCheckButton - time:  0.51
GtkRadioButton - time:  1.00
GtkTextView - Add text - time: 17.19
GtkTextView - Scroll - time:  1.84
GtkDrawingArea - Lines - time:  2.70
GtkDrawingArea - Circles - time:  2.65
GtkDrawingArea - Text - time:  1.72
GtkDrawingArea - Pixbufs - time:  0.11
 --- 
Total time: 36.91
    So, with SNA, the timing that I get are almost 2 times lower, which is superb for a very simple /etc/X11/xorg.conf config file: 
    Section "Device"
        Identifier "Intel Graphics"
        Driver "intel"
        Option "AccelMethod" "sna"
EndSection
    I have not noticed any change, but if drawing graphics faster means that I get a bit more of battery life, then I will be super happy. I still have not tested if this works on my notebook with a Core2Duo T7250. Crossing my fingers so that it does.

    Lars Wirzenius: Yakking: a new group blog targeting new free software hackers

    Daniel, Richard, and I started a new group blog, Yakking, targeting those new to software development in the free software world.

    20 August 2013

    Maximilian Attems: Mapping the research literature

    Paperscape visualizes all the arXiv papers. The arXiv is a pioneer in Open Data publishing from 1991 and started by Paul Ginsparg for high energy physicists (hep). It soon extended to most physicists. According to it's importance (citation) each paper has a certain radius and the citations also link the papers to their position in the map. In addition there are helpful overall topics and one can click once zoomed in on each paper to know all the authors and the title. The colors correspond to each arXiv section.

    Erich Schubert: Google turning evil

    I used to be a fan of the Google company. I used to think of this as a future career opportunity, as I've received a number of invitations to apply for a job with them.
    But I am no longer a fan.
    Recently, the company has in my opinion turned to the "evil" side; they probably became worse than Microsoft ever was and the "do no evil" principle is long gone.
    So what has changed? In my perception, Google:
    • Is much more focused on making money now, than making technology happen.
    • Instead of making cool technology, it tries more and more to be a "hipster" thing and following the classic old 80-20 rule: get 80% of the users with 20% of the effort. Think of the GMail changes that received so much hate and the various services it shuts down (Code search, Reader, Googlecode downloads) and how much Google is becoming a walled garden.
    • Google leverages its search market dominance as well as Android to push their products that don't perform as good as desired. Plus for example. There is a lot of things I like about plus. In particular when you use it as a "blog" type of conversation tool instead of a site for sharing your private data, then it's much better than Facebook because of the search function and communities. (And I definitely wouldn't cry if a Facebook successor emerges). But what I hate is how Google tries hard to leverage their Android and YouTube power to force people to Plus. And now they are spreading even further, into TV (ChromeCast) and ISP (Google Fiber) markets.
    • Hangups, oops, Hangouts is another such example. I've been using XMPP for a long time. At some point, Google started operating an XMPP server, and it would actually "federate" with other servers, and it worked quite nicely. But at some point they decided they need to have more "hipster" features, like ponies. Or more likely, they decided that they want all these users to use Plus. So now they are moving it away from an open standard towards a walled garden. But since 90% of my use is XMPP outside of Google, I will instead just move away from them. Worse.
    • Reader. They probably shut this down to support Currents and on the long run move people over to Plus, too. Fortunately here, there now exist a number of good alternatives such as Feedly. But again: Google failed my needs.
    • Maps. Again an example where they moved from "works good" to "hipster crap". The first thing the new maps always greets me with is a gray screen. The new 3D view looks like an earthquake happened, without offering any actual benefit over the classic (and fast) satellite imagery. Worse. In fact my favorite map service right now is an OpenStreetmap offline map.
    • Google Glass is pure hipsterdom crap. I have yet to see an actual use case for it. People use it to show off, and that's about it. If you are serious about photos, you use a DSLR camera with a large objective and sensor. But face it: it's just distraction. If you want productive, it's actually best to turn of all chat and email and focus. Best productivity tip ever: Turn off email notifications. And turn of chat, always-on and Glass, too.
    • Privacy. When privacy-aware US providers actually recommend against anyone trusting their private data to a company with physical ties to the United States, then maybe it's time to look out for services in countries that value freedom higher than spying. I cannot trust Google on keeping my data private.
    We are living in worrisome times. The U.S.A., once proud defenders of freedom, have become the most systematic spies in the world, even on their own people. The surveillance in the Eastern Bloc is no match for this machinery built the last decade. Say hello to Nineteen Eighty-Four. Surveillance, in a mixture of multi-state and commercial surveillance has indeed become omnipresent. Wireless sensors in trash cans track your device MACs. Your email is automatically scanned both by the NSA and Google. Your social friendships are tracked by Facebook and your iPhone.
    I'm not being paranoid. These things are real, and it can only be explained with a "brainwashing" with both the constantly raised fear of terror (note that you are much more likely to be killed in a traffic accident or by a random gun crazy in the U.S.A. than by terrorists) and the constant advertisement for useless technology such as Google Glass. Why have so many people stopped fighting for their freedom?
    So what now? I will look into options to move away stuff from Google (that is mostly, my email -- although I'd really like to see a successor to email finally emerge). Maybe I can find a number of good services located e.g. in Switzerland or Norway (who still seem to hold freedom in high respect - neither the UK nor Germany are an alternative these days). And I hope that some politicians will have the courage to openly discuss whether it may be necessary to break up Google into "Boogles" (Baby-Googles), just as Reagan did with AT&T. But unfortunately, todays politicians are really bad at such decisions, in particular when it might lower their short-run popularity. They are only good at making excuses.

    Chris Lamb: Thomas Fuchs on frontend Javascript frameworks in startups

    What you end up with is building a layer cake that doesn t add any value and slows down development. Especially when you re starting out and need to stay flexible you don t want to have too much code around and Rails is great for that. But adding a JSON API layer and basically a second application that runs on the client is annihilating this advantage for you.
    Source.

    19 August 2013

    Enrico Zini: Debian Contributors right after DebConf

    Debian Contributors right after DebConf "Debian Contributor" is going to be a new hat in Debian, automatically given to anyone who contributes to Debian in some way. I kickstarted the idea with a BOF at the beginning of DebConf (see video), worked on it during the conference and announced the results so far at a talk towards the end of the conf (see video). In the end, we have a data model and an initial prototype, still not deployed. A team is forming to work on this after DebConf. This TED video came to my mind several times, while working on this. I'm excited to see a way to turn on the light and see who Debian Contributors really are. My gut feeling is that we are in for a lot of surprises, which may change the way we see our community in exciting ways. Wow.

    Gunnar Wolf: After Debian's 20th birthday, everybody's life gets back to normality...

    As I slowly read my good friends wishing each other a good trip, telling they got home safely, and the IRC channels form thick drops of a bitter-sweet etheral substance, I cannot help feeling DebConf13 is over For me as well, from the distance. Many friends gave me warm greetings, and without being there, gave me that beautiful feeling of real community that Debian has given me for ten years already, since I met in real-life many of its developers at DebConf3 in Oslo. And yes, I have stated this far too many times I have attended every DebConf since (and worked organizing most of them). This year, over 300 people were gathered in Switzerland to enjoy the always most intense weeks of the year. This year, I was unable to attend due to calendar clashes. Even so, without the stress that organizers have, and thanks to the great work of the always-loved Video Team, I think I was able to be present at more sessions than at in any of the last few years. Oh, and for the readers of this blog who were not there Do you want to follow what was presented? You can download already the videos for all of the recorded presentations (that were, due to the planned coverage and the manageable size of the Video Team, about of the total scheduled sessions). And, as always, I was able to follow many very interesting talks and take part of a couple interesting meetings/BoF sessions. I still have a bit of catchup, partly due to the timezone difference (I was only at one of the sessions during the Swiss morning, at 02:30 local time, the pkg-ruby-extras team BoF). Anyway... Not being there, I surely was an avid consumer of the photos posted in the DebConf13 gallery, and will surely follow it for some more time as some of you upload your pending material. It was clear from the beginning that, no matter what your definition of consensus is, the chosen venue was beautiful. A beautiful place between the lake and the mountains where our sportiest guys had a very good share of morning runs, cycling sessions, competition sports of different types, outright plain fun for attendees of all sizes and all species... But, hey, wait! During a chat in the course of DebConf, a friend told me a bit worried that all this beauty and fun might make our dear and very important sponsors they are paying for a geek vacation, is it so? No, not at all. Not by a long stretch. And just looking at those same galleries makes it clear and obvious. After all, it's widely known that Debian is the operating system for the gurus. Simple: It's impossible to have all those geeks without getting amazing work done, in ways that even seem clich s (this last photo had Joey Hess explaining dpkg format version 3.0 (git) ideas, sketched after waking up at 3AM on the first sketching surface available to him). After all, Debian people are famous for their inclination to use any excuse to open their computers and hack away. We can find Debianers hacking in small spaces and also hacking out in the fields. But this time, people were able to hack indoors while enjoying the nature and hack outdoors under a tree. And, yes, one of the things that makes organizing DebConf worth it is, after eleven months having low-bandwidth meetings over IRC, having the opportunity to plan for the next days face to face, in a relaxed but work-full environment. Anyway, here at home I didn't sit idly just longing over them. How could I? We are just celebrating the Debian Project's 20th anniversary! http://gwolf.org/content/jonathan-host-and-organizer-rancho-electr-nico">Jonathan, a Debian enthusiast, student at my university, and collaborator for several free software-related collectives in Mexico City, invited me to the celebration at Rancho Electr nico (which I recently mentioned in this same blog). While I was unable to stay for the whole celebration, we had a very good time; I talked about some ways on how to contribute to Debian. Although I didn't have much of a presentation prepared for it, I feel it was successful and interesting for the attendees I just hope to start seeing some of them get into any of the ways for helping Debian soon. I also stayed as a listener and ocassional commenter for a talk on the Debian Project's history and goals, and to a presentation on a nifty electronic music programming tool called Supercollider (of course, available in Debian). Now, "regular" life should continue. For some value of "regular".

    Next.

    Previous.