Search Results: "cas"

5 April 2012

Steve Kemp: So I want a backup solution

I look after a lot of systems, and most of them want identical and simple backups taking of their filesystems. Currently I use backup2l which works but suffers from a couple of minor issues. In short I want to take a full filesystem backup (i.e. Backup "/"). I wish to only exclude a few directories and mounted filesystems. So my configuration looks like this:
# List of directories to make backups of.
# All paths MUST be absolute and start with a '/'!
SRCLIST=(  / /boot -xdev )
# The following expression specifies the files not to be archived.
SKIPCOND=( -path '/var/backups/localhost' -o -path '/var/run/' -o \
    -path '/tmp'  -o -path '/var/tmp' \
    -o -path '/dev' -o -path '/spam' \
    -o -path '/var/spool/' )
The only surprising thing here is that I abuse the internals of backup2l because I know that it uses "find" to build up a list of files - so I sneakily add int "-xdev" to the first argument. This means I don't accidentally backup any mounted gluster filesystem, mounted MySQL binary/log mounts, etc. backup2l then goes and does its jobs. It allows me to define things to run before and after the backup runs via code like this:
# This user-defined bash function is executed before a backup is made
PRE_BACKUP ()
 
   if [ -d /etc/backup2l/pre.d/ ]; then
      run-parts /etc/backup2l/pre.d/
   fi
 
So what is my gripe? Well I get a daily email, per-system, which shows lots of detail - but the key thing. The important thing. The thing I care about more than anything else, the actual "success" or "fail" result is only discoverable by reading the mail. If the backup fails, due to out of disk, I won't know unless I read the middle of the mail. If the pre/post-steps fail I won't know unless I examine the output. As I said to a colleague today in my view the success or failure of the backup is the combination of each of three distinct steps: If any of the three fail I want to know. If they succeed then ideally I don't want a mail at all - but if I get one it should have:
Subject: Backup Success - $(hostname) - $(date)
So I've looked around at programs such as backup-ninja, backup-manager and they seem similar. It is a shame as I mostly like backup2l, but in short I want to do the same thing on about 50-250 hosts: In my case it is usually the rsync-step that fails. Which is horrific if you don't notice (quota exceeded. connection reset by peer. etc). The local backups are good enough for 95% of recovery times - but if the hardware is fried having the backups be available, albeit slowly, is required. Using GNU Tar incrementally is trivial. If it weren't such a messy program I'd probably be inclined to hack on backup2l - but in 2012 I can't believe I need to. (Yes, backuppc rocks. So does duplicity. So does amanda. But they're not appropriate here. Sadly.) ObQuote: "Oh, I get it. I see now. You've been training for two years to take me out, and now here I am. Whew! " - Blade II - An example of a rare breed, a sequel that doesn't suck. No pun intended.

Jon Dowland: Greg Bear Hull Zero Three

In (roughly) 2008 I read "Blood Music" my first Greg Bear novel and devoured it in a weekend, in only two or three reading sessions. That happens very rarely, and when it does I often wonder whether it's the quality of the writing or some combination of that and your personal moods and tastes at the time that is responsible. My next Bear was "Darwin's Radio" in mid-2010 which, whilst good, did not provoke the same feverish read. I might write short reviews for both at some point in the future, but right now I can't remember either well enough to do so. In the last few weeks I read "Hull Zero Three" which was quite a different beast. The novel first captured my attention due to the unusual way it was promoted: via a video, which was created using a modified Quake 2 engine. It wasn't a great video but the novelty made it interesting. The story begins with a first-person narrator waking up from a sort-of hyper sleep on a generation starship that has clearly had some serious problems. He has to run to escape danger before he has an opportunity to figure out exactly what has happened and what he is running from, or to. Something has gone horribly wrong on the ship: it's nowhere near where it should be going and nobody should be awake. The narrator's memory is fractured, fabricated and unreliable; the ship is hostile, and whilst it's mechanical, it cyclically changes and re-arranges it's hulls in an almost organic way. He is joined by a rotating cast of similarly confused refugees of varying shapes and sizes. Language and cultural barriers serve to further magnify the sense of alienation that you experience on behalf of the story teller. The troupe are persued through the corridors by a collection of lethally efficient killer "things", contrasting with the background by being organic in nature but mechanical in behaviour and appearance. The scale of the ship is necessarily massive, and Bear does an impressive job of capturing the awe you might feel when considering it. Three huge, cigar-shaped hulls, joined by "spars" to an enormous chunk of ice captured from the Oort cloud on exodus from the solar system. The ice serves as a mixture of "fuel" for the biological ship systems and as part of a shield system for the hulls from cosmic rays. However, in part due to the narrator's confusion, it was hard to follow the very abstract descriptions of the environment in the earlier stages of the story. I found myself re-reading passages to try and understand the comparisons being offered and scales presented. In common with the two earlier Bears I've read, 'body horror' features prominently in this story. It feels like a "Dead Space" or "Resident Evil" in space in a hard SF novel which seriously attempt to capture real speculative science. Like "Blood Music", I found myself compelled to read "Hull Zero Three" at every available opportunity. I finished it off in a very enjoyable reading session outdoors in the spring sunshine. It was an enjoyable read, well written but not beyond criticism, with some interesting takes on well-worn themes. Very reminiscent of the movie "Pandorum" (which was a surprisingly good movie).

3 April 2012

Axel Beckert: Tools for CLI Road Warriors: Hidden Terminals

Some networks have no connection to the outside except that they allow surfing through an HTTP(S) proxy. Sometimes you are happy and the HTTPS port (443) is unrestricted. The following server-side tools allow you to exploit these weaknesses and get you a shell on your server. sslh sslh is an SSH/SSL multiplexor. If a client connects to sslh, it checks if the clients speaks the SSH or the SSL protocol and then passes the connection to the according real port of SSL or some SSL enabled service, e.g. an HTTPS, OpenVPN, Tinc or XMPP server. That way it s possible to connect to one of these services and SSH on the same port. The usual scenario where this daemon is useful are firewalls which block SSH, force HTTP to go through a proxy, but allow HTTPS connections without restriction. In that case you let sslh listen on the HTTPS port (443) and to move the real HTTPS server (e.g. Apache) to listen on either a different port number (e.g. 442, 444 or 8443) or on another IP address, e.g. on localhost, port 443. On an Debian or Ubuntu based Apache HTTPS server, you just have to do the following to run Apache on port 442 and sslh on port 443 instead:
  1. apt-get install sslh as root.
  2. Edit /etc/default/sslh, change RUN=no to RUN=yes and --ssl 127.0.0.1:443 to --ssl 127.0.0.1:442.
  3. Edit /etc/apache2/ports.conf and all files in /etc/apache2/sites-available/ which contain a reference to port 443 (which is only /etc/apache2/sites-available/default-ssl.conf in the default configuration) and change all occurrences of 443 to 442.
  4. service apache2 restart
  5. service sslh start
Now you should be able to ssh to your server on port 443 (ssh -p 443 your.server.example.org) while still being able to surf to https://your.server.example.org/. sslh works as threaded or as preforking daemon, or via inetd. It also honors tcpwrapper configurations for sshd in /etc/hosts.allow and /etc/hosts.deny. sslh is available as port or package at least in Gentoo, in FreeBSD, in Debian and in Ubuntu. AjaxTerm A completely different approach takes AjaxTerm. It provides a terminal inside a web browser with login and ssh being its server-side backend. Properly safe-guarded by HTTPS plus maybe HTTP based authentication this can be an interesting emergency alternative to the more common but also more often blocked remote login mechanisms. AjaxTerm is available as package at least in Debian and in Ubuntu.

Happily I never were forced to use either of them myself. :-)

1 April 2012

Joey Hess: ls: the missing options

I'm honored and pleased to be the person who gets to complete ls. This project, begun around when I was born, was slow to turn into anything more than a simple for loop over a dirent. It really took off in the mid and late 80's, when Richard Stallman added numerous features, and the growth has been steady ever since. But, a glance at the man page shows that ls has never quite been complete. It fell to me to finish the job, and I have produced several handy patches to this end: The only obvious lack now is a -z option, which should make output filenames be NULL terminated for consuption by other programs. I think this would be easy to write, but I've been extermely busy IRL (moving lots of furniture) and didn't get to it. Any takers to write it? Due to the nature of these patches, they conflict with each other. Here's a combined patch suitable to be applied and tested.
diff -ur orig/coreutils-8.13/src/ls.c coreutils-8.13/src/ls.c
--- orig/coreutils-8.13/src/ls.c    2011-07-28 06:38:27.000000000 -0400
+++ coreutils-8.13/src/ls.c 2012-04-01 12:41:56.835106346 -0400
@@ -270,6 +270,7 @@
 static int format_group_width (gid_t g);
 static void print_long_format (const struct fileinfo *f);
 static void print_many_per_line (void);
+static void print_jam (void);
 static size_t print_name_with_quoting (const struct fileinfo *f,
                                        bool symlink_target,
                                        struct obstack *stack,
@@ -382,6 +383,7 @@
    many_per_line for just names, many per line, sorted vertically.
    horizontal for just names, many per line, sorted horizontally.
    with_commas for just names, many per line, separated by commas.
+   jam to fit in the most information possible.
 
    -l (and other options that imply -l), -1, -C, -x and -m control
    this parameter.  */
@@ -392,7 +394,8 @@
     one_per_line,      /* -1 */
     many_per_line,     /* -C */
     horizontal,            /* -x */
-    with_commas            /* -m */
+    with_commas,       /* -m */
+    jam            /* -j */
    ;
 
 static enum format format;
@@ -630,6 +633,11 @@
 
 static bool immediate_dirs;
 
+/* True means when multiple directories are being displayed, combine
+ * their contents as if all in one directory. -e */
+
+static bool entangle_dirs;
+
 /* True means that directories are grouped before files. */
 
 static bool directories_first;
@@ -705,6 +713,10 @@
 
 static bool format_needs_type;
 
+/* Answer "yes" to all prompts. */
+
+static bool yes;
+
 /* An arbitrary limit on the number of bytes in a printed time stamp.
    This is set to a relatively small value to avoid the need to worry
    about denial-of-service attacks on servers that run "ls" on behalf
@@ -804,6 +816,7 @@
    "escape", no_argument, NULL, 'b' ,
    "directory", no_argument, NULL, 'd' ,
    "dired", no_argument, NULL, 'D' ,
+   "entangle", no_argument, NULL, 'e' ,
    "full-time", no_argument, NULL, FULL_TIME_OPTION ,
    "group-directories-first", no_argument, NULL,
    GROUP_DIRECTORIES_FIRST_OPTION ,
@@ -849,12 +862,12 @@
 static char const *const format_args[] =
  
   "verbose", "long", "commas", "horizontal", "across",
-  "vertical", "single-column", NULL
+  "vertical", "single-column", "jam", NULL
  ;
 static enum format const format_types[] =
  
   long_format, long_format, with_commas, horizontal, horizontal,
-  many_per_line, one_per_line
+  many_per_line, one_per_line, jam
  ;
 ARGMATCH_VERIFY (format_args, format_types);
 
@@ -1448,6 +1461,9 @@
       print_dir_name = true;
      
 
+  if (entangle_dirs)
+      print_current_files ();
+
   if (print_with_color)
      
       int j;
@@ -1559,6 +1575,7 @@
   print_block_size = false;
   indicator_style = none;
   print_inode = false;
+  yes = false;
   dereference = DEREF_UNDEFINED;
   recursive = false;
   immediate_dirs = false;
@@ -1644,7 +1661,7 @@
      
       int oi = -1;
       int c = getopt_long (argc, argv,
-                           "abcdfghiklmnopqrstuvw:xABCDFGHI:LNQRST:UXZ1",
+                           "abcdefghijklmnopqrstuvw:xyABCDFGHI:LNQRST:UXZ1",
                            long_options, &oi);
       if (c == -1)
         break;
@@ -1667,6 +1684,10 @@
           immediate_dirs = true;
           break;
 
+   case 'e':
+          entangle_dirs = true;
+     break;
+
         case 'f':
           /* Same as enabling -a -U and disabling -l -s.  */
           ignore_mode = IGNORE_MINIMAL;
@@ -1697,6 +1718,10 @@
           print_inode = true;
           break;
 
+   case 'j':
+     format = jam;
+     break;
+
         case 'k':
           human_output_opts = 0;
           file_output_block_size = output_block_size = 1024;
@@ -1765,6 +1790,10 @@
           format = horizontal;
           break;
 
+   case 'y':
+     yes = true;
+     break;
+
         case 'A':
           if (ignore_mode == IGNORE_DEFAULT)
             ignore_mode = IGNORE_DOT_AND_DOTDOT;
@@ -2510,7 +2539,7 @@
       DEV_INO_PUSH (dir_stat.st_dev, dir_stat.st_ino);
      
 
-  if (recursive   print_dir_name)
+  if ((recursive   print_dir_name) && ! entangle_dirs)
      
       if (!first)
         DIRED_PUTCHAR ('\n');
@@ -2526,7 +2555,8 @@
   /* Read the directory entries, and insert the subfiles into the  cwd_file'
      table.  */
 
-  clear_files ();
+  if (! entangle_dirs)
+     clear_files ();
 
   while (1)
      
@@ -2615,7 +2645,7 @@
       DIRED_PUTCHAR ('\n');
      
 
-  if (cwd_n_used)
+  if (cwd_n_used && ! entangle_dirs)
     print_current_files ();
  
 
@@ -3464,6 +3494,10 @@
       print_with_commas ();
       break;
 
+    case jam:
+      print_jam ();
+      break;
+
     case long_format:
       for (i = 0; i < cwd_n_used; i++)
          
@@ -4418,6 +4452,24 @@
   putchar ('\n');
  
 
+static void
+print_jam (void)
+ 
+  size_t filesno;
+  size_t pos = 0;
+
+  for (filesno = 0; filesno < cwd_n_used; filesno++)
+     
+      struct fileinfo const *f = sorted_file[filesno];
+      size_t len = length_of_file_name_and_frills (f);
+
+      print_file_name_and_frills (f, pos);
+      pos += len;
+     
+  putchar ('\n');
+ 
+
+
 /* Assuming cursor is at position FROM, indent up to position TO.
    Use a TAB character instead of two or more spaces whenever possible.  */
 
@@ -4627,11 +4679,13 @@
   -D, --dired                generate output designed for Emacs' dired mode\n\
 "), stdout);
       fputs (_("\
+  -e, --entangle             display multiple directory contents as one\n\
   -f                         do not sort, enable -aU, disable -ls --color\n\
   -F, --classify             append indicator (one of */=>@ ) to entries\n\
       --file-type            likewise, except do not append  *'\n\
       --format=WORD          across -x, commas -m, horizontal -x, long -l,\n\
                                single-column -1, verbose -l, vertical -C\n\
+                               jam -j\n\
       --full-time            like -l --time-style=full-iso\n\
 "), stdout);
       fputs (_("\
@@ -4667,6 +4721,8 @@
   -i, --inode                print the index number of each file\n\
   -I, --ignore=PATTERN       do not list implied entries matching shell PATTERN\
 \n\
+  -j                         jam output together, makes the most of limited\n\
+                             space on modern systems (cell phones, twitter)\n\
   -k                         like --block-size=1K\n\
 "), stdout);
       fputs (_("\
@@ -4733,6 +4789,7 @@
   -w, --width=COLS           assume screen width instead of current value\n\
   -x                         list entries by lines instead of by columns\n\
   -X                         sort alphabetically by entry extension\n\
+  -y                         answer all questions with \"yes\"\n\
   -Z, --context              print any SELinux security context of each file\n\
   -1                         list one file per line\n\
 "), stdout);
It remains to be seen if multi-option enabled coreutils will be accepted into Debian in time for the next release. Due to some disagreements with the coreutils maintainer, the matter has been referred to the Technical Committee (Flattr me) Traditionally new ls contributors stop once enough options have been added that they can spell their name, in the best traditions of yellow snow. Once ls -richard -stallman worked, I'm sure RMS moved on other other more pressing concerns. The current maintainer, David MacKenzie, was clearly not done yet, since only ls -david -mack worked. But he was being slow to add these last few features, and ls was very deficient in the realm of spelling my name (ls -o -hss .. srsly?), so I took matter into my own hands in the best tradition of free software.

Kai Wasserb ch: kvpm in Debian now

Just a short announcement: kvpm (short for "KDE volume and partition manager") is now a part of Debian's unstable distribution, Sid, and should enter testing (Wheezy) soon. The big thank you for this goes to Benjamin, upstream author of kvpm who has become also the Debian Maintainer of his software! The story behind packaging kvpm started with Benjamin writing an e-mail to pkg-kde-extras, in which he asked for inclusion of kvpm into Debian. After an initial answer by myself about filing a RFP bug, in case he wouldn't like to package kvpm himself, a discussion in private ensued. There I explained that Debian is a do-ocracy, where things get packaged if some maintainer uses something herself or if the requesting party does the work. Benjamin took up the challenge and packaged kvpm, I helped only with feedback and finally sponsoring the upload. Why do I tell this story? Because I hope it encourages other people to do the same as Benjamin did: package the software they'd like to see become a part of Debian. We, the Debian Developers around the world, are more than happy to assist you. Just ask us (e.g. by e-mail or through IRC (#debian-mentors)).

Russ Allbery: Review: Agile Project Management with Scrum

Review: Agile Project Management with Scrum, by Ken Schwaber
Publisher: Microsoft
Copyright: 2004
ISBN: 0-7356-1993-X
Format: Trade paperback
Pages: 153
Ken Schwaber is one of the two original developers of Scrum and one of the signatories to the Agile Manifesto. Agile Project Management with Scrum is an introduction to Scrum by him that feels aimed primarily at business people (project managers, executives, line managers) rather than at technical people. There's very little about agile software development in general; the emphasis is on business process, team organization, and how the Scrum process looks to the rest of the organization. After an introductory chapter laying out the basics of Scrum, the remainder of the book is primarily case studies. Schwaber presents some principle and then discuss an anonymized case study that illustrates the principle. Each major Scrum role (ScrumMaster, Product Owner, and Team Member) gets their own chapter, interspersed with chapters on planning, reporting, and scaling. Finally, there are some concluding appendices laying out the rules of Scrum in more detail, adding a brief (and very disappointing) discussion of fixed-term, fixed-price contracts, and providing some other supplemental information. This is only the second regular book that I've read about Scrum, but my reading has been a bit out of order. We've been using Scrum for various projects for about a year and a half, I've played both team member and product owner roles, and we've been thinking hard about how to integrate Scrum more into the way we do development. My impression of this book is that it's intended as more of an introduction for people who aren't yet comfortable with Scrum or who are still struggling to make the pieces fit together. I'm therefore possibly not the target audience. Much of this book is a rehash of things someone who has studied Scrum will already know. I'm of two minds about the case study presentation style of this book. On one hand, it does help to make the points Schwaber discusses more concrete, and it's difficult to talk about business process in any other way. They're also more interesting and engaging than Schwaber's straight rule descriptions. But I had difficulty digging into most of the ones here. They're very simple and a little shallow, and I had trouble applying many of them to practical situations I might be in. I would have liked somewhat more detailed stories with more analysis and possible alternatives within the situation, rather than simple illustrations of how Scrum solves the problem that Schwaber had just said it would solve. What I did like about this book is that it provides the background and theory of Scrum as empirical process control much more clearly than any material I'd previously read. Schwaber is good at explaining what empirical process control is and why it's a superior approach, and he gave me new vocabulary and basic concepts to use to talk about Scrum. I also liked the high-level look from a management perspective without getting into the details of story points and velocity estimation (Schwaber doesn't use them at all in this book, relying instead on traditional "days of work" estimates). Much of my previous experience with Scrum was tightly focused on its advantages for estimation and responsiveness to changes, and it's nice to see some of the other advantages of Scrum (developer job satisfaction, improved focus, incremental deployment) highlighted more. Another part of this book I appreciated was the discussion of sprint length. Schwaber doesn't go into the possible alternatives, largely assuming that a month is the best interval, but the discussion of the impact and implications of a month sprint length helped me see the advantages of longer sprints. (To date, we've been doing either one week or two week sprints.) I came away with the feeling that a hybrid model where longer sprints are used for business purposes and then subdivided into shorter sprints for developer coordination might be particularly effective, although that's not an idea that Schwaber presents. This is not a particularly compelling read, particularly if, like me, you're not a huge fan of genericized case studies. Schwaber's writing is clear, but wooden and occasionally awkward, and the book is often dry. But it talks about Scrum from a management angle instead of a developer angle, which is an additional perspective that I found valuable. Possibly worth taking a look if you want more detail about Scrum from a high-level management perspective, although I don't think you're missing a lot by not reading it. Rating: 6 out of 10

30 March 2012

Joey Hess: podcasts that don't suck

My public radio station is engaged in a most obnoxious spring pledge drive. Good time to listen to podcasts. Here are the ones I'm currently liking.

Russell Coker: The Security Benefits of Automation

Some Random WTFs The Daily WTF is an educational and amusing site that recounts anecdotes about failed computer projects. One of their stories titled Remotely Incompetent concerns someone who breaks networking on a server and is then granted administrative access to someone else s server by the Data Center staff [1]! In one of the discussions about that I saw people make various claims about Data Center security, such as claiming that having their own locked room helps. My experience indicates that such things don t do much good, I have often been granted access to server rooms without appropriate checks. My experience is that security guards on site generally don t directly do any good. I once had a guard hold a door for me when I was removing a server from a DC without even bothering to ask for ID! On another occasion in the Netherlands I had a security guard who didn t speak English unlock the wrong server room for me, I used hand gestures to inform him that I needed access to the room with the big computers and he gave me the access I needed! It seems that the benefit of security guards is solely based on scaring people who don t have the confidence needed to bluff their way in. Preventing children from thieving is a good thing, On another occasion I showed ID and signed in for access to a DC owned by my employer and I used my security key to go through a locked door with a sign that promised many bad consequences if I failed to lock the door behind me. Then I discovered that the back door was wide open for the benefit of some electricians who were working in the building. Presumably the electricians who had no security training were expected to act as ad-hoc security guards if someone tried to enter through the back door presumably they would not have been good at it. When a company uses part of their own office for a server room then many of these problems disappear. But a common issue in such ad-hoc DCs is the lack of planning and procedures, I have lost count of the number of times I ve seen doors (and even windows) propped open to allow ventilation because there were too many servers for the air-conditioning to cope. The most ironic example of this is the company that had a walk-in safe (think of a small bank vault with concrete walls and thick solid steel door) used for storing servers but with it s door propped open to allow cooling. The advantage of a serious hosting company is that they will have procedures for cooling etc and will be very unlikely to do strange and silly things. Having a locked room in a DC makes some sense, but if security guards have the master keys and are allowed to use them then it might not do much good. The one time I locked my keys in such a room I had a guard let me in without verifying my ID or the claim that there were actually keys locked in the room. Presumably anyone could just claim to have forgotten their keys and get the door unlocked just like a cheap hotel. Locking a rack sounds like a good idea, but the racks I ve seen have had locks which are quite easy to pick. On the one occasion when I had to pick a lock on a rack (due to keys being too difficult to manage for the relevant people) the security guards didn t investigate, so either the security cameras were not supervised or they just didn t care about people picking locks in a shared server room. Also if you allow people to do things freely in a shared server room they could install devices to monitor network traffic. A locked cage in a server room should work well. In the one case where I worked for a company that used such a cage I found it to mostly work well apart from the few weeks when the lock was broken. One company that I worked for had scales before the door between a server room and the car-park to prevent people from stealing heavy servers. Of course that wouldn t stop people stealing hard drives full of data which is worth more than the servers! Also an over-weight colleague had to have the scales disabled for him (as they were based on absolute mass not unexpected changes in an individual s mass) which presumably means that any skinny employee could steal a 2RU server and still be below the mass threshold. How to Solve some of these Problems Computers are subject to all manner of security problems. But they tend not to do arbitrary things for no apparent reason and they will never give in to someone who is charming, attractive, or aggressive unlike humans. I have servers running on Hetzner, Linode, and the Rackspace Cloud. I am always concerned about possible security compromises. But I am not worried about someone climbing in a window of a server room or convincing a security guard to let them in through the door. All three of those hosting companies have the vast majority of interactions automated. I can change many aspects of the servers without involving ANY human interaction. Out of the three of those companies I have had some human interaction with Hetzner (who provide managed servers) when a hard drive needed to be replaced obviously replacing a disk in the wrong server would have been a significant system integrity issue even though everyone would be running RAID-1 and if Hetzner improperly disposed of the broken disk then there could be security issues but this is an unlikely mistake in the face of a rare occurrence. With Linode and the Rackspace Cloud (and the previous Slicehost hosting that was purchased by Rackspace) the most common interactions I have with employees of those companies are when my clients don t pay their bills on time and that s an administrative not a technical issue. When I do have to contact the support people about a technical issue it s usually something that s not immediately connected to the virtual server (EG a loss of routing to the DC). It seems most likely that there are a fairly small number of people who are allowed in the DCs for companies like Hetzner, Linode, and Rackspace. Those people would probably be recognised by the security guards and their work would be restricted to replacing failing hardware and not involve granting access requests. There are some unusual requests that they can process (EG one of my clients recently transferred a virtual server between business units) but even in those cases the administrative software controls who gets access. This is much better than just handing hardware access to what seems to be the correct physical server to a client. If you have software running a few computers and operating correctly then you can probably scale it up to run thousands of computers and have it still work correctly. But if you have a team of people controlling access requests and want to scale it up significantly then there are huge problems in hiring skilled people and training them correctly. There is a real risk of security flaws in such administrative software, if someone managed to exploit the automated management system for one of those three companies then they could probably gain access to the private data of any of their customers. But the risk of this seems a lot less than the risk of general incompetence among humans who perform routine and boring tasks which have the potential for great errors. Related posts:
  1. The Security Benefits of Being Unimportant A recent news item is the hacking of the Yahoo...
  2. Security Lessons from a Ferry On Saturday I traveled from Victoria to Tasmania via the...
  3. Public Security Cameras There is ongoing debate about the issue of security cameras,...

28 March 2012

Andrew Pollock: [life/americania] Four days in New Orleans

Sarah's Mum had accrued too much annual leave and had to take some time off work, so Sarah did some (very mild) arm twisting and convinced her to come over for 3 weeks, and do a 5 day cruise to Mexico out of New Orleans. Unfortunately, my annual leave situation wasn't quite so abundant, and I had a lot going on at work, so regretfully I didn't join them on the cruise, and instead went to New Orleans for a four day weekend when they returned. From all reports, the cruise was very good. Zoe handled it well, although she did say "home" a lot. One of the two stops in Mexico was to check out some Mayan ruins, which looked awesome from the photos. The other stop involved a dolphin encounter. I was incredibly envious of all that they got to do, and would have loved to have gone with them, as I've never been on a cruise ship either. I can also report that no cats were lost during this bachelor stint. I had a night flight on the Wednesday evening to get there, which was scheduled to get in at around midnight, and I'd booked a motel room near the airport for that night, and we'd booked a vacation rental home for Thursday to Sunday nights. Unfortunately, my flight ended being delayed something like 2.5 hours, so I didn't get into New Orleans until around 2am. The house we rented did the trick nicely. It was a small "shotgun" duplex in what looked like a nice neighbourhood. It was advertised as being close to the street car line, but they were doing some work on the tracks, so the street car didn't seem to be running as far down the line as it usually did, so it ended up being a bit more of a trek to get to it. It was also extremely slow, and there was a marathon on the Sunday, which closed everything down for a long time, making it a generally pretty unreliable form the transport. We ended up renting a car for Saturday and Sunday, which was something of a saga in itself, as Enterprise didn't have any cars at the location we'd booked one, so after a couple of hours cooling our heels there (Zoe was incredibly well-behaved, all things considered), they shuttled us over to another location and we ended up with a minivan instead of a compact, which for the same price, allayed our concerns about being able to transport all of our luggage to the airport on Monday morning. We had a very early morning flight on Monday morning to come back, which got into SFO at around 9am, and I went directly to work from there. Thursday We all arrived at the house, separately. It ended up taking them 2 hours to disembark the ship when it came back into port, with Customs taking an eternity to process everyone. I think we went exploring the local area that afternoon, and took a street car into the city to check out Bourbon Street, having a Cajun dinner at Remoulade. Friday In the morning, we went to check out Lafayette Cemetery Number 2. Sarah took Zoe back to the house for a nap, and Sarah's Mum and I continued back to explore the French Quarter some more, walking down the length of Royal Street (which was vastly different from Bourbon Street, just one block over). We had lunch at the French Market. The cemetery was interesting, as pretty much all of the graves were these huge above ground tombs, that seemed to have multiple family members interred in them. Apparently the cemetery filled up quite quickly courtesy of a Yellow Fever outbreak. After lunch, Sarah's Mum and I continued wandering around the French Quarter. We went and took a look at the Mississippi River, and I had an encounter with a grifter who was so good at his job I couldn't bring myself to argue with him over the $20 he diddled me out of. We tried to get to the Civil War Museum, but it closed at 4pm. We looked at the Robert E. Lee Monument, which seemed to be draped in drunks, and then I think we rendezvoused with Sarah and Zoe back on Canal Street for dinner at The Court of Two Sisters (which apparently we were under-dressed for, as Sarah and her Mum said we were getting a lot of dirty looks from other patrons). Saturday On Saturday morning, we had the aforementioned car rental experience from hell, and by the time we had the car it was lunchtime, so Zoe napped in the car after lunch on the way out to Oak Alley Plantation, where we were introduced to the delightful beverage known as the mint julep, and took a tour of the house and wandered the grounds. Sunday On Sunday, Sarah and her Mum did a swamp tour, and Zoe and I went to the zoo. As I said earlier, there was a marathon that completely closed down Saint Charles Avenue, which is where the street cars run, so after walking down to where the street cars started operating (which ended up being most of the way down South Carrollton Avenue), the driver informed me that the street cars were queuing up at the corner of South Carrollton and Saint Charles, and I should get off her street car and get on the one at the front of the queue. I did this, but the driver of the front street car informed me that she wouldn't be leaving for an hour and half. At this point, I started considering a bus instead. Zoe and I went to check out the Mississippi River, which was quite close to where we were, and then I went back, and despite a street car having left (without any passengers) the driver of the current street car couldn't tell me when she'd be leaving, so I started walking down Saint Charles Avenue. Unfortunately, Zoe's going through a phase where she wants to be carried everywhere, so I was lugging her all over the place on my hip. Sarah didn't take a stroller with her, and instantly regretted it. Lesson learned. Eventually we managed to get onto a bus, which dropped us off at Audubon Park, which had Audubon Zoo at the other end of it. There was a playground near the Saint Charles Avenue end of the park, so Zoe had a bit of a play on that, and then we continued through the park to the zoo. Mercifully, the zoo had dodgy strollers for rent, and there was no way in the world I wasn't going to rent one of them, so that made getting around with Zoe a lot easier on my back. We had a really good time at the zoo. There was some sort of a music festival on in the parklands within the zoo grounds, and that included a jumping castle, which Zoe expressed a desire to have a go on. She had a fabulous time on it. I think she probably spent about 15 minutes in there, without any tears. I was very impressed. I took a brief video of some of her antics. It was getting close to Zoe's nap time, and she was getting tired, but fortunately Sarah and her Mum were able to pick us up from the zoo after their swamp tour and Zoe got to nap back at the house. Monday We had a very early start. Unfortunately, our flight (with United) was a couple of days after United and Continental officially merged, and despite having checked in online, we had to queue up with everyone else (for an extended period of time) to drop off our checked luggage. Then there was a 45 minute line for security screening. We pulled the "toddler going to melt down" card and jumped to the head of the line, but Sarah's Mum had to wait. The flight ended up being delayed because people were stuck in the security line. Overall impressions of New Orleans Fabulous architecture. There were so many gorgeous houses on Saint Charles Avenue and the surrounding area. I'd have loved to do an architecture tour. Crap (but cheap) public transport. $3 gets you a day pass. The street cars are cute, but slow and unpredictable. The drivers were remarkably unhelpful. The buses were okay. Great food. Zoe seemed to have a liking for the spicy stuff. I gave her some fresh alligator jerky, and after some initial coughing and spluttering at the spiciness of it, she came back for more. Post-Katrina recovery. We really only saw a very small part of the city, but there were still some houses with boarded up windows, and some vacant blocks where buildings had been demolished, but largely you'd not have been able to tell that large parts of the city had been underwater, from casual inspection. I really enjoyed the trip, even though it was brief, I feel like I got a good feel for the place. We were there just after Mardi Gras, and there were still beads everywhere. Draped all over fences. Over power lines. Trees on the parade route were absolutely covered in beads. I'd have loved to have been there for Mardi Gras. My friend Brandon, who is an excellent street photographer, took some great photos that capture some of it. Photos from Sarah's cruise and our time in New Orleans are here.

27 March 2012

Timo Jyrinki: On brands, marketing and technical details

I have a few thoughts based on the yet another round of debate where marketing clashes with free software advocation and technical details. Nothing new in the debate itself, but I'm adding a couple of insights.

Marketing is not highly respected by many technical people, and neither by the people wanting more advocating than the messing up with facts and feelings that the marketing does. I'm all for advocating free software, but it's currently not something you can use for marketing to win big markets. If we advance to a world where free software is as wanted as the green values today are, it can be used in marketing as well similar to all the ecological (according to the market department at least) products today, but alas the benefits of free software are not yet as universally known. Since it doesn't say much that touches the masses, advocating has a negative marketing effect since it takes space away from the potentially "hitting" marketing moves, in those cases where you target the big masses in the first place. "Open" this and that has some marketing power in it nowadays, but it's a mess of different meanings that probably doesn't advance libre software freedoms as such. Wikipedia has probably been the biggest contributor to advancing general knowledge of software and culture libre. Disclaimer: I'm not a proper marketing person, some more professional might have better insights in this area. If I'd be a proper marketing person, I'd decorate this blog post with fancy pictures so that more people would actually read it.

Some of the marketing can be done without sacrificing any of the advocation. The fabulous Fedora campaigns, graphics, slogans and materials are a great example of those and do an important job, even though Fedora isn't reaching the big masses with it (and Fedora isn't targeted for OEMs to ship to millions either). But they do hopefully reach a lot of level people on the grassroots level. I hope as a Debian Developer that more people valuing the freedoms of free software would help also Debian as a project to reach more of its advocation potential and developers from the more proprietary world. But I'm happy that at least some free software projects have nowadays true graphical and marketing talent. Even though not as widely known, freedoms of the software users including for example privacy aspects are a potential good marketing tool toward a portion of the developer pool.

Let's not forget that Android conquered the mobile market without using the brand power of Linux. The 30+ million people who know Linux a bit deeper than just "I've heard it" already run a Linux distribution like Ubuntu, but we need hotter brands than a project name of a kernel to reach the bigger masses. Call it Ubuntu, Fedora, or something, but no matter do what it needs to ship it to millions. AFAIK mostly Ubuntu and SUSE are shipping currently via desktop/notebook OEMs, and more Ubuntu than SUSE. Others aren't concentrating on the market, which is a very difficult one. Ubuntu is doing a lot more mass population advocating for free software than Android has ever done. Note for example the time any user tries to play a video for the first time in a non-free format - Ubuntu will tell the user about the problems related to those formats and asking a permission to install a free software player for those (or buy licensed codecs), and the Ubuntu Help texts describe a lot of details about restricted formats, DRM et cetera. Not to mention what happens if the person actually wanders into the community, discovering Debian, other free software projects, free software licenses and so on. Meanwhile Android users never notice anything being wrong while watching H.264 videos or watching DRM Flash videos. Granted, like Boot2Gecko debate shows, it may be a partially similar situation on shipping desktop Linux variants as well, in order to actually ship them via partners.

Anyway, the best example of brands is MeeGo. Even the LWN editor does not get into dissecting the meaning of MeeGo on Nokia N9, because "there has been no real agreement on that in the past", but just uses the brand name as is. That is the power of brands. Technical people debate that it's not really MeeGo, it's maemo GNU/Linux with a special permission from Linux Foundation to use the MeeGo brand name, and then counter-argument with that the MeeGo is an API and maemo matches the MeeGo 1.2 API which is actually just Qt 4.7 API. And actually, as proven by the LWN example, it's not even "technical people". For most of technical people Nokia N9 is MeeGo as well. Only the people who have actually worked on it plus the couple of other people migrated from maemo and MeeGo.com communities to Mer project understand the legacy, history and the complete difference between the Maemo Harmattan platform and what MeeGo.com was. Yet at the same time like all GNU/FreeDesktop.org/Linux distributions, they are 95% same code, just all the infrastructure and packaging and polishing and history is different.

For 99.9% of people who know what MeeGo is, MeeGo is the cool Nokia smartphone, one of its kind and not sold in some of the major western markets, and/or the colorful sweet characters of meego.com shown at one time on a couple of netbooks. That is the brand, and the technical details do not matter even to the technical people unless they actually get into working on the projects directly.

To most people, the Linux brand is a mess of a lot of things, while other brands have the possibility at least to have a more differentiating and unique appearance.

25 March 2012

Gregor Herrmann: RC bugs 2012/12

thanks to lucas' last archive rebuild, this week offered some new & easy additional RC bugs :) here's the list:

Petter Reinholdtsen: Debian Edu screencast: Checking email with kmail using Kerberos authentication

The same Debian Edu developer that did the last screen cast I published, Wolfgang Schweer, has created a new screen cast showing how to set up Kmail in Debian Edu Squeze to authenticate using Kerberos, allowing users to check their local email account without providing any password. The video is embedded here in quarter size, and also available from vimeo and download as a Ogg Theora file. Check it out below. <video controls="controls" height="184" id="kmail-kerberos-movie" preload="preload" width="256"> <source src="http://ftp.skolelinux.org/skolelinux/press/screencasts/2012-03-14-Debian-Edu_Configure_Kmail_for_internal_usage.ogv" type="video/ogg; codecs="theora, vorbis""> Download video as Ogg. </video>

24 March 2012

Richard Hartmann: podcasts that don't suck

Funny, I had planned to use the same name for quite some time.. Good English German

John Goerzen: German: Suprisingly Hilarious for a 2-Year-Old

The other night, I was tucking the boys into bed. I occasionally use a few German words with them (I don t know all that many myself, but hey). They d been rather bored with it, so I had dropped it for a few months. I said to Oliver, Gute Nacht, Oliver. That means good night in German. Oliver laughed. I said it a few more times. He laughed some more. Jacob peeked down from his top bunk to see what the fun was, and pretty soon was getting into it also. Apparently I m quite the comedian, because when I said Auf Wiedersehen, Oliver burst out laughing uncontrollably. Finally he d say, Say it again! AGAIN!! And I would, and he d laugh and laugh some more. And then, I happened to say tsch ss. Jacob loved that one. He said it over and over, much to Oliver s delight. Finally I closed the door, and heard the laughter still going on behind me, as Jacob would yell out a mangled German word, and Oliver would fill the room with laughter. I was a little surprised, but figured it d end there. Not so much. The next night, I tucked them into bed like usual. I left their room, and about 10 seconds later, I heard frantic yelling from the boys room. I went back in, and poor Oliver was so upset he couldn t even say what he wanted. I finally figured out he was pleading, DAD! Say gute Nacht again! So we went through some German words again, and I left the boys much relieved. Phew. The next night, I almost repeated my horrid mistake of no German at bedtime, but Jacob stopped me before I left their room. Dad! You forgot to say tsch ss! So I did, with the boys prompting me on what words they wanted to hear. Then Jacob started asking me how to say these things in other languages. He was rather disappointed, and I think also didn t really believe me, when I only knew a few of those phrases in Spanish and French. Though he was surprised that I could come up with them in Greek. (Kali sp ra / good evening) was a hit. By this point, another day or two later, Jacob says tsch ss when I leave. Oliver wishes me gute Nacht at breakfast time. We ll have to work on that one. So, if you ever walk in the door at our house some morning, don t be surprised if our 2-year-old wishes you good night, our 5-year-old says goodbye, and they both start laughing.

23 March 2012

Russell Coker: Cheap NAS Devices Suck

There are some really good Network Attached Storage (NAS) devices on the market. NetApp is one company that is known for making good products [1]. The advantage of a NAS is that you have a device with NVRAM for write-back caching, a filesystem that supports all the necessary features for best performance (NetApp developed their own filesystem WAFL to provide the features they needed), and a set of quality hardware that has been tested and certified to work together. If you want a cheap NAS then you end up with something running Linux with GPL filesystems. This isn t a bad thing as such, but some of the best performance and data integrity features are available in ZFS (which isn t GPL) and BTRFS (which isn t ready for production use). Not to mention WAFL which has been providing ZFS/BTRFS type features for more than a decade. A cheap NAS will generally be sold without disks as this is the best way to keep costs down. Selling with disks either means selling lots of different variations (which means it can t be sold off the shelf) or selling packages that don t quite suit some customers (thus causing people to buy the device and replace the disks which means extra costs). This means that the vendors can t provide the guarantees about disk quality and suitability that NetApp can provide. One major problem with a NAS is that you typically can t get shell access. Commands such as rm -rf and cp -rl which are typically rather quick when performed locally can take ages when run over NFS. Also commands such as grep -R which can perform reasonably well over NFS will always perform better when run locally. Also tasks such as compiling big programs which require good disk speed as well as some CPU time can be run locally if you have a file server system that also has local accounts (IE a typical multi-user Unix server configuration), but a dedicated NAS will prevent that. I have never used a NetApp device due to my clients deciding (sometimes correctly and sometimes incorrectly IMHO) that they are too expensive. But when considering what my clients do the down-sides of lacking local code execution on a NetApp would be more than compensated in many cases by the significant performance and reliability advantages that they offer (see my post about the reliability issues in standard RAID implementations [2]). A server class system (with ECC RAM as a minimum criteria) running RAID-6 can be a fairly decent file server. That requires hardware RAID with NVRAM for the write-back cache for decent write performance, but when write performance isn t required software RAID does the job quite well. A Dell tower system will typically hold at least 4 disks which means 6TB of RAID-6 storage and is quite cheap it can be under $2000. also such a system can be easily expanded with extra Ethernet ports etc. NetApp doesn t sell products directly and doesn t list prices, but they do have some adverts for products being under $7500 . That s not really cheap but not THAT expensive when you consider the features. A hidden cost in running a NAS is having someone perform sysadmin work on it. For a relatively expensive device that offers significant features such as a NetApp Filer this expense probably isn t too great. But for a device that does what any PC running Linux can do it s noteworthy that more training or experimenting time is required. There are some special cases where small and cheap NAS appliances really make sense, such as the Apple Time Capsule for home network backups. But apart from that I don t think that cheap NAS appliances make sense. It seems that cheap NAS devices provide the biggest down-sides of expensive NAS devices (in terms of lacking local access and having a different administration interface to servers) while also having the biggest down-sides of PC servers (lacking the advanced features of WAFL and performance of a NetApp). Earlier today I started a process of reorganising some backups which included backups to a cheap NAS. I have been very unimpressed by the time taken to copy and rm files over NFS. I m sure that the job would have been completed hours ago if I had local root access to the NAS. Related posts:
  1. big and cheap USB flash devices It s often the case with technology that serious changes occur...
  2. Insider Threats and Small Storage Devices Danny Angus writes about the potential threat posed by small...
  3. Some RAID Issues I just read an interesting paper titled An Analysis of...

Rapha&#235;l Hertzog: People behind Debian: J rg Jaspert, FTPmaster, Debian Account Manager, and more

Photo by Wouter Verhelst

J rg is a very active contributor within Debian, and has been for a long time. This explains why he holds so many roles (FTPmaster and Debian Account Manager being the 2 most important ones) Better known as Ganneff (his IRC nick), he s not exactly the typical hacker. He has no beard and used to drink milk instead of beers. :-) Check out his interview to learn more about some of the numerous ways one can get involved in Debian, managing its infrastructure and without having to be a packager. Raphael: Who are you? J rg: My name is J rg Jaspert and I m 35 years old working for a small company doing system administration and consulting work for our customers. I m married for a little while now and sometime soon a little Ganneff will be crawling out of my wife. (Whoever didn t think of the movie Alien now is just boring). Raphael: How did you start contributing to Debian? J rg: I started using Debian somewhere around 2000, 2001. Before that I had the misfortune to try SuSE and RedHat, both with a user experience that let me fully understand why people think Linux is unusable. (Due to my work I m in the unfortunate situation to have to use RedHat on two machines. Funny how they are still utter crap and worse than bad toys). And all of this lets get a Linux running here came up because I was trying to find a replacement for my beloved OS/2 installation, which I had for some years. So after I got Debian installed, good old Potato, I got myself active on our mailing lists, starting with the German user one. A bit later I replied to a question if someone can help as staff for a Debian booth somewhere. It was the most boring event I ever visited (very nice orga, unfortunately no visitors), but I got a few important things there: The software I packaged, found me a sponsor and voila, maintainer I was. Some more packages got added and at some point my sponsor turned out to be my advocate. The NM process run around 2 months, and mid April 2002 I got THE MAIL. Raphael: Some Debian developers believe that you have too many responsibilities within Debian (DAM, FTPMaster, Debconf, Partners, Planet Debian, Mirrors, ). Do you agree that it can be problematic, and if yes, are you trying to scale down? J rg: It s DebConf, tssk. And yes, I do have some extra groups and roles. And you even only list some, leaving out all I do outside Debian. But simply counting number of roles is a plain stupid way to go. Way more interesting is how much work is behind a role and how many other people are involved. And looking at those you listed I don t see any I am a SPOF. Let s look at those you listed: DAM: Here I did start out assisting James to get the huge backload down which had accumulated over time. Nowadays I am merely the one with the longest term as DAM. Christoph Berg joined in April 2008 and Enrico Zini followed during October 2010, both very active. Especially Enrico, lately with the redesign of the NM webpages. FTPMaster: The basic outline of the FTPMaster history is similar to the DAM one. I joined as an assistant, after the oh-so-famous Vancouver meeting in 2004. Together with Jeroen, we both then got the backload down which had accumulated there. He did most of the removals while I had a fun time cleaning up NEW. And we both prepared patches for the codebase. And in 2007, as the last action as DPL, Sam made me FTPMaster. Since then I haven t been alone either. In fact we have much more rotation in the team than ever before, which is a good thing. Today we are 3 FTPMasters, 4 FTP Assistants and 1 Trainee. Though we always like new blood and would welcome more volunteers. DebConf: I am very far outside the central DebConf team. I am not even a delegate here. Currently I am merely an admin, though there are 4 others with the same rights on the DebConf machines. I ve not taken any extra jobs this year, nor will I. Probably for next year again, but not 2012. Planet: I am one of three again, but then Planet is mostly running itself. Debian developers can just edit the config, cron is doing the work, not much needed here. Occasional cleanups, every now and then a mail to answer, done. In short: No real workload attached. Mirrors: My main part here is the ftpsync scriptset. Which is a small part of the actual work. The majority of it, like checking mirrors, getting them to fix errors, etc. is done by Simon Paillard (and since some time, Raphael Geissert is active there too, you might have heard about his http.debian.net). Having said that, there is stuff I could have handled better or probably faster. There always is. Right now I have 2 outstanding things I want to do a (last) cleanup on and then give away. Raphael: You got married last year. I know by experience that entertaining a relationship and/or a family takes time. How do you manage to combine this with your Debian involvement? J rg: Oh well, I first met my wife at the International Conference on OpenSource 2009 in Taiwan. So OpenSource, Debian and me being some tiny wheel in the system wasn t entirely news to her. And in the time since then she learned that there is much more behind when you are in a community like Debian, instead of just doing it for work. Even better that she met Debian people multiple times already, and knows with who I am quarreling Also, she is currently attending a language school having lots of homework in the evening. Gives me time for Debian stuff. :) How that turns out with the baby I have no idea yet. I do want to train it to like pressing the M key, so little-Ganneff can deal with NEW all on its own (M being Manual reject), but it might take a day or twenty before it gets so far. :) Raphael: Thanks to the continuous work of many new volunteers, the NEW queue is no longer a bottleneck. What are the next challenges for the FTPmaster team? J rg: Bad link, try this one. :) Also, no longer sounds like its recent. It s not, it s just that people usually recognize the negative only and not the positive parts. Well, there are a few challenges actually. The first one, even if it sounds simple, is an ongoing one: We need Debian Developers willing to do the work that is hidden behind those simple graphs. Yes, we are currently having a great FTP Team doing a splendid work in keeping that queue reasonably small this is a/THE sisyphean task per excellence. There will always be something waiting for NEW, even if you just cleaned the queue, you turn around and there is something else back in already. Spreading this workload to more people helps not burning one out. So if one or more of the readers is interested, we always like new volunteers. You simply need to be an uploading DD and have a bit of free time. For the rest we do have training procedures in place. Another one is getting the multi-archive stuff done. The goal is to end up with ONE host for all our archives. One dak installation. But separate overrides, trees, mirrors, policies and people (think RMs, backports team, security team). While this is halfway easy to think of in terms of merging backports into main it gets an interesting side note when you think of merging security into main . The security archive does have information that is limited to few people before public release of a security announce, and so we must make sure our database isn t leaking information. Or our filesystem layer handling. Or logs. Etc. Especially as the database is synced in (near) realtime to a DD accessible machine. And the filesystem data too, just a little less often. There is also a discussion about a good way to setup a PPA for Debian service. We do have a very far developed proposal here how it should work, and I really should do the finishing touches and get it to the public. Might even get a GSoC project on it. So far for some short to middle term goals. If you want to go really long term, I do think that we should get to the point where we get rid of the classical view of a source package being one (or more) tarballs plus the Debian changes. Where a new version requires the full upload of one or more of those parts of the source package. I don t know exactly where it should end up. Sure, stuff like one central DVCS, maintainers push there, the archive generates the source tarballs and prepares the mirrors do sound good for a quick glance. But there are lots of trouble and pitfalls and probably some dragons hidden here. Raphael: The Debian repositories are managed by DAK (Debian Archive Kit) which is not packaged. Thus Debian users pick tools like reprepro to manage their package repositories. Is that how things should be? J rg: Oh, Mark Hymers wants to do a package again. More power to him if he does, though yes, DAK is not exactly a quick-and-easy thing to install. But nowadays it is a trillion times easier than the past thanks to Mark s work people can now follow the instructions, scripts and whatever they find inside the setup directory. Still, it really depends on the archive size you are managing. A complex tool like dak does not make sense for someone who wants to publish one or a dozen of his own packages somewhere. Thats just like doing a finger amputation with a chainsaw it certainly works and is fun for the one with the chainsaw but you probably end up a little overdoing it. I myself am using dpkg-scan[packages sources] from a shell script but also mini-dinstall in places (never got friend with reprepro when I looked at it). Works, and for the few dozen packages those places manage it is more than enough. Also, using dak forces you into some ways of behaviour that are just what Debian wants but might not be what a user wants. Like inability to overwrite an existing file. One of the reasons why mentors.debian.net won t work with dak. Or the use of a postgres database. Or that of gpg. Sure, if you end up having more than just a dozen packages, if you have many suites and also movement between them, then dak is sure a thing to look at. And how should things be : however the user and admins of that certain install of reprepro, mini-dinstall, dak, whatever want it. This is not one-tool-for-all land :) Raphael: What is the role of Debian Account Managers (DAM)? Do you believe that DAMs have a responsibility to shape Debian by defining limits in terms of who can join and what can be done within Debian? J rg: Quote from https://lists.debian.org/debian-devel-announce/2010/10/msg00010.html:
The Debian Account Managers (DAM) are responsible for maintaining the list of members of the Debian Project, also known as Debian Developers. DAMs are authoritative in deciding who is a member of the Debian Project and can take subsequent actions such as approving and expelling Project members.
Now, aside from this quote, my OWN PERSONAL OPINION, without wearing anything even vaguely resembling a DAM hat: DAM is the one post that is entitled to decide who is a member or not. Usually that is in the way of joining (or not), which is simple enough. But every now and then this also means acting on a request to do something about whatever behaviour of a Debian Project member. I hate that (and i think one can easily replace I with WE there). But it s our job. We usually aren t quick about it. And we don t act on our own initiative when we do, we always have (numerous) other DDs complain/appeal/talk/whatever to us first. The expulsion procedure , luckily not invoked that often, does guarantee a slow process and lots of input from others. Are we the best for it? Probably not, we are just some people out of a thousand who happen to have a very similar hobby Debian. We aren t trained in dealing with the situations that can come up. But we are THE role inside Debian that is empowered to make such decisions, so naturally it ends up with us. Raphael: You did a lot of things for Debian over the years. What did bring you the most joy? Are there things that you re still bitter about? J rg: The most joy? Hrm, without being involved in Debian and SPI I would never have met my wife.
Or my current job. Or a GR against me. Not many running around with that badge, though I m still missing my own personal Serious problems with Mr. Jaspert thread. Bad you all.
Or visited so many places. Think of all the DebConfs, QA meetings, BSPs and whatever events.
Or met so many people.
Or learned so many things I would never even have come near without being DD. Raphael: Is there someone in Debian that you admire for their contributions? J rg: Yes.
Thank you to J rg for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Note that older interviews are indexed on wiki.debian.org/PeopleBehindDebian.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook.

One comment Liked this article? Click here. My blog is Flattr-enabled.

22 March 2012

Axel Beckert: Tools for CLI Road Warriors: Tunnels

Sometime the network you re connected to is either untrusted (e.g. wireless) or castrated in some way. In both cases you want a tunnel to your trusted home base. Following I ll show you three completely different tunneling tools which may helpful while travelling. sshuttle sshuttle is a tool somewhere in between of automatic port forward and VPN. It tunnels arbitrary TCP connections and DNS through an SSH tunnel without requiring root access on the remote end of the SSH connection. So it s perfect for redirecting most of your traffic through an SSH tunnel to your favourite SSH server, e.g. to ensure your local privacy when you are online via a public, unencrypted WLAN (i.e. easy to sniff for everyone). It runs on Linux and MacOS X and only needs a Python interpreter on the remote side. Requires root access (usually via sudo) on the client side, though. It s currently available at least in Debian Unstable and Testing (Wheezy) as well as in Ubuntu since 11.04 Natty. Miredo Miredo is an free and open-source implementation of Microsoft s NAT-traversing Teredo IPv6 tunneling protocol for at least Linux, FreeBSD, NetBSD and MacOS X. Miredo includes not only a Teredo client but also a Teredo server implementation. The developer of Miredo also runs a public Miredo server, so you don t even need to install a server somewhere. If you run Debian or Ubuntu you just need to do apt-get install miredo as root and you have IPv6 connectivity. It s that easy. So it s perfect to get a dynamic IPv6 tunnel for your laptop or mobile phone independently where you are and without the need to register any IPv6 tunnel or configure the Miredo client. I usually use Miredo on my netbooks to be able to access my boxes at home (which are behind an IPv4 NAT router which is also an SixXS IPv6 tunnel endpoint) from whereever I am. iodine iodine is likely the most undermining tool in this set. It tunnels IPv4 over DNS, allowing you to make arbitrary network connections if you are on a network where nothing but DNS requests is allowed (i.e. only DNS packets reach the internet). This is often the case on wireless LANs with landing page. They redirect all web traffic to the landing page. But the network s routers try to avoid poisoning the client s DNS cache with different DNS replies as they would get after the user is logged in. So DNS packets usually pass even the local network s DNS servers unchanged, just TCP and other UDP packets are redirected until logging in. With an iodine tunnel, it is possible get a network connection to the outside on such a network anyway. On startup iodine tries to automatically find the best parameters (MTU, request type, etc.) for the current environmenent. However that may fail if any DNS server in between imposes DNS request rate limits. To be able to start such a tunnel you need to set up an iodine daemon somewhere on the internet. Choose a server which is not already a DNS server. iodine is available in many distributions, e.g. in Debian and in Ubuntu.

David Welton: I'm not good enough to work on open source software

Actually, that's not true - I've produced plenty of open source software over the years. However, in a sense, it is true: only the very best actually get paid to work on open source software full time, and I'm not one of them. People like Linus Torvalds. People like Guido van Rossum, although even he supposedly divides his time, and does not work on Python full-time. Think about that. Python is a hugely popular programming language used by many companies and individuals who get a lot of value out of it. But the creator doesn't even work on it full-time. Now, that's just one example - perhaps Guido enjoys the things that Google sends his way to work on outside of Python in any event - but I think it's representative of open source in general. Back to me: I've produced some small bits of open source code that other people find useful. Several people have even built products on Hecl that make money. But I'm not good enough to work on open source full time - I'm not one of those famous, brilliant coders who is so good that someone will find a way to pay them to work on stuff that gets given away for free. I am, however, a good enough programmer to work on people's proprietary code, and have never had too much trouble finding someone with a project they're happy to pay me for. Why is that? Because it's so much easier to funnel money back into a proprietary project. If people like the product and buy it, the company gets money, which can be used to pay the developers. With open source, millions of people might use it and get a lot of value from it, but the developer has no right to receive any of that back as cash, which he or she can use to pay for things like food and rent. So, I can code tolerably well, and I could conceivably contribute more open source code, but instead I spend my time working on proprietary code because it pays the bills. Clearly, when I can, I use open source software in these endeavors, and contribute back whenever I can, but the "secret sauce" remains closed. That's a pair of hands lost to the creation of more open source. I know I'm not alone in this, either - tons of people work on mostly proprietary projects the world over, but relatively few people get paid to work on open source code full time. So when I read about people debating the utility of copyright bring up the existence of open source as some sort of counterexample, it irritates me a bit. The right level of protection and enforcement of copyright is a complex debate that I'm not going to get into here. What I want to point out is "that which is not seen". Sure, open source exists. But how much more of it would exist it there were more money to fund work on it? How much open source software remains an idea in the developers head that does not get realized for lack of time? People often criticize the "Linux desktop" despite its extraordinary strides in recent years. Well, how much farther along would it be if there were more people paid to work on the 'boring stuff', like usability testing? Ubuntu and Redhat pay a few people to do that kind of stuff, but how many more people do Microsoft and Apple have for that kind of work? That's not to say that open source "doesn't work" or some such nonsense. It obviously works quite well, but it really shines where the currency is code, not money. Developers can and do give back lots, in terms of code, bug reports, suggestions, documentation, and so on to open source projects, which make them better for all involved. Where open source doesn't seem to work quite as well are in small, fast-moving, consumer-oriented products. My guess is that 99% of iPhone users could care less about the source code for their apps, but on the other hand, a large portion of the Emacs user base more than likely has written at least a few lines of Elisp. In any event, the point isn't to beat up on open source software, but to counter this idea that "intellectual property" is in no way shape or form necessary because the existence of open source software somehow "proves" that "things will get made just the same". Yes, maybe they will, but in lower quantities than consumers might find desirable. After all, most of us aren't good enough to work on open source software.

Russell Coker: Links March 2012

Washington s Blog has an informative summary of recent articles about corporate psychopaths [1]. Including the fact that some banks deliberately hire psychopaths. Anu Partanen wrote an insightful article for The Atlantic about the difference between Finnish and American education systems [2]. It seems that Finland has achieved great educational success by aiming for equality with no private schools and giving the teachers and principals enough responsibility to do the job properly. Ramona Pierson gave an interesting TED talk about how she recovered from being run over by a drunk driver with the help of the residents of a senior citizens home [3]. Quyen Nguyen gave a very interesting TED talk about the use of fluorescent dyes in cancer surgery [4]. They can make cancer glow one color and nerves glow with a different color which makes it a much easier task to remove ALL the cancer without cutting the smaller nerves. James Fallows wrote an interesting article for The Atlantic about the experience of having his wife s Gmail account cracked [5]. She stored EVERYTHING in her Gmail account so this was a lot worse for her than for the typical Geek who doesn t use such accounts for storing much. James describes what the attacker did, how they did it, and what needed to be done to recover. When running a mail server it s worth considering what you would do to help a user who was attacked in that way. Charles Stross has written an interesting blog post trying to predict some future psychological and social changes [6]. Matthew Wright of Beyond Zero Emissions has written an informative article about how solar panels on home roofs save everyone money [7]. Mikko Hypponen gave an interesting TED talk about different types of online attack and how they can affect us [8]. Among other things he describes how online attacks can result in people dying. Scott Rickard gave an interesting TED talk about using maths to create SONAR pings without repitition and also the worst music ever created [9]. I found it entertaining to watch Michael Lindel (the director of chamber music for the New York Symphony) play the music, he obviously didn t enjoy that performance. Sheena Iyengar gave an interesting TED talk about the way people make choices [10]. It s useful for anyone who is going to prepare a set of options for someone else to choose from. ASD Aid is a project that uses Lego to encourage kids on the Autism Spectrum to socialise [11]. They have training manuals for using Lego in therapy. Unfortunately they have not been supported by the Lego corporation. Bilal Bomani gave an interesting and informative TED talk about NASA research into renewable aviation fuel [12]. The most interesting thing to me was the way that they were aiming for a sustainable lifecycle that didn t use resources that coule be used for food and which required minimal input once it was started. Alain de Botton gave an interesting TED talk about Atheism 2.0 [13]. He suggests that we adopt some ideas from religious organisations including lectures (sermons), celebrations, and rituals. It s interesting to think of a technology conference as a pilgrimage. Homaro Cantu and Ben Roche from Moto restaurant in Chicago gave an interesting talk about some of the unusual foods that they have produced [14]. If I visit the US again I will try and go to Chicago to eat there! Related posts:
  1. Links March 2009 Cory Doctorow has written a column for The Guardian titled...
  2. Links February 2012 Sociological Images has an interesting article about the attempts to...
  3. Links January 2012 Cops in Tennessee routinely steal cash from citizens [1]. They...

Richard Hartmann: Svalbard

Svalbard The things you find when cleaning out a disk; preparing for re-installation of your laptop on a larger disk once the laptop comes back from repair... I thought I had posted this in early January, but apparently not. As it would be a shame to just throw this away, here goes: I am sitting in the Oslo airport, waiting for boarding back home to start. Seeing the sun after a week of darkness still feels strange. Inital trouble It's been a very interesting week, starting with our trip to Oslo in the other direction. We spent New Year's Eve in Oslo, timing our forced overnight stay before reaching Svalbard to coincide with something interesting. The close timing of our travel meant that there was exactly one flight to Oslo we could take. Never having heard of Air Baltic before and finding out that they are a discount airline, my gut feeling told me to be be wary. I refuse to fly with those airlines on principle, not wanting to support their business model while hurting airlines with decent treatment of customers. Unfortunately, I was forced to book with them in this case. As it turns out, my gut was right. More on that below. Our plane was late in landing and my luggage was lost. As it was New Year's Eve, the staff at Oslo airport understandably wanted to be home, not at work. Still, getting them to file a report was tedious and finding out days later that the report was incomplete was, well, not good. The express train from the airport to Oslo central station had closed early without any advance notice or local signs, presumably because of NYE. The gates were simply closed and that's that. We figured out how the bus system worked, got our tickets from a vending machine, saw the one bus to central station drive away, and proceeded to wait in the outside waiting area; at least we had a front spot in the queue. While we don't get cold easily, it was funny to see Norwegians in thin clothes stand around in the biting wind, apparently being comfortable. A young mother with a baby, who didn't anticipate being forced to wait outside at -15C, as opposed to just sitting down in a train, had no warm clothes for her son; something that was fixed by wrapping him in spare clothes from Ilona's luggage. After waiting for about thirty minutes for the next bus to appear, it parked twenty or thirty meters away from the designated parking spot. The rough queue disintegrated and if not for Ilona's leaving me with luggage and backpacks and storming off to fight the masses, said mother with baby and ourselves would have waited for another thirty minutes. More than hundred people were left stranded at the airport to celebrate there or on the bus. Meh. We hurried, by taxi, from central station to hotel to harbour and arrived about five minutes to twelve. NYE itself was nice. We stood on top of the opera house and watched a rather impressive show of fireworks through the thickening mists. Norwegian fireworks pack lot more punch than German ones; you actually feel your clothes shake when they go off. Getting to Longyearbyen The flight from Troms to Longyearbyen had free in-flight Wi-Fi and flying over the edge of satellite coverage demonstrated how far we were from everywhere else rather impressively. Arriving at Longyearbyen, I fixed the lost luggage claim with the help of an extremely nice woman working at the airport. She confirmed that Air Baltic is legally required to reimburse me after an arcane system based on a virtual IMF currency to the tune of about 1.500. That may sound like a lot, but seeing as I had most of my scuba gear with me, that's not even half of what my luggage was worth. I was forced to get by New Years with what I had on my body and went shopping the next day when I could buy at least a few things. I got by with spending about 180 by going for non-fitting bargain bin clothes, wanting to reduce impact on Air Baltic. Shortly after that, my luggage arrived, unannounced. Air Baltic refused to reimburse me even though they are legally required to, again the airport staff confirmed this. But unless I sue in the country of destination, Norway, I won't see any money. Long story short: Avoid Air Baltic if possible. They will break the law to cheat you out of money when they have a reasonable expectation of getting away with it. Update: Yep, seems they got away with it unless I take legal measures. You have been warned. Longyearbyen itself was very nice. We started off with a short, guided taxi tour around the city, seeing literally everything of it as there's not a lot of Longyearbyen to start with. Dogsledding Next day, as noted earlier, we went shopping and spent the "evening" with dogsledding which turned out to be tons of fun. We helped with harnessing the dogs which is somewhat cumbersome as the dogs are so eager to burn off their energy that they want to run all over the place, not being put into a harness and snapped onto the pull-line. Never having been dogsledding before I was a bit wary, but riding over not too rough terrain is almost trivial once you get the hang of it; listen to what the musher in front of you yells and emulate the same shouts with your own dogs. If the dogs become too fast, step onto the brake pad which simply drives spikes into the snow. If the dogs slow down while going uphill, skate with one foot to help them. That's it for speed. As the dogs are following the musher's sled anyway, steering the dogs was not a concern. Fun fact: The musher used a laser pointer to steer his dogs; simple, efficient, and presumably fun. We learned, by demonstration, that sledding dogs can eat snow, take a leak, and take a dump while running at full speed and pretty much all at the same time. The dogs are left either in cages or on long chains far away from the city as they tend to bark and howl a lot. We were surprised to learn that, even when there are seals left to hang dry as dog food nearby, there are no problems with bear attacks. Apparently even the extremely aggressive and hungry one year old males will not go near dogs. Still, while out in the ice and snow, our guide carefully flooded all crevices and cliff bases to root out female bears with young ones early. They hide their children from the wind and they will attack, dogs and all, if we get too close. That's why our musher carried rifle in his sled and flare gun on his body. Snowmobile The next "morning", we drove around by snowmobile. This turned out to be extremely boring as it was a curated trip with over half a dozen snowmobiles in our group; a stark contrast to our two-sled tour the day before. The last time I went snowmobiling, we raced each other up and down a two-star black (i.e. steep, bumpy, and curvy) ski slope, jumping several meters when racing over larger bumps and crossing streets, so riding single file at 30 km/h was... anticlimatic. Again, the guide had rifle and flare gun with him. We spent the afternoon and evening walking around the city. Ice bears, part I As the ice caves and the glacier were still closed, we decided to have another walk around the city. Having planned make it a quick tour, we lost track of time due to lack of sun and ended up walking around for seven hours. A note about that trip.. If you are alone with your wife, unarmed, climbing up a very steep and slippery mountainside over a sheet of ice with deep snow underneath and loose rocks in between, and then start shining around with your flashlight under the stilts of an abandoned mine that looks like in a horror movie, the correct answer to "What are you doing?" is never ever "Looking for ice bears". Even if it's the truth, this is not an acceptable answer. I crawled up the last part on all fours, camera and tripod in hand while Ilona stayed about two dozen meters below the mines' entrance, refusing to go another step towards the mine. As soon I made it up the creaky and shaky, for a lack of a better word, let's call it ladder, she forced me to come down again. Bleh, but I guess I deserved that. Armed photographer Next day, we got final confirmation that we would not be allowed to rent snowmobiles to explore the hinterland on our own and that other for some, and I quote, "crazy Russians", no one would even attempt to cross over to Barentsburg. Thus, we ended up renting a car for the ~20 kilometers of total road length. That turned out to be a great idea as it allowed us to get away from the light and take some very nice long exposures. It was then that I got a rental rifle, as well. There is a law against leaving the town unarmed and I was not about to test my luck too much. Turns out that, as part of Germany's WWII reparations, Norway received our all hand guns and as they still function perfectly when dirty and in cold climate which makes them still popular in Svalbard, today. The Karabiner 98 Kurz which I received is built incredibly well. It's somewhat scary inasmuch every detail is designed to make this weapon ready to fire. If you hurry or are inexperienced, you will end up with an unlocked and loaded rifle after putting in the bolt. Putting the safety in and not chambering a bullet takes conscious effort and knowledge of the weapon. This is in stark contrast to other weapons I had the chance to dissect, which all defaulted to being safe. Even other military weapons such as the AK74 and the M4A1 are inherently more secure, designed to be locked and safe. The Mauser K98... not so much. As an aside, they didn't bother to remove the Reichsadler and Swastika from the rifle, contending themselves with striking out the German registration number and stamping the rifle with a Norwegian one. I guess Norwegians don't really "care" about these signs as much as we Germans do. In a way, that's a good thing I guess, at least as long as it's an indication of indifference towards the sign, not one of forgetting or ignoring the underlying issues. Still, I was very glad to have rented the rifle. While Ilona tended to stay in the running car with heat and lights on, I went out and away from the car. Even when standing near a street, a medium snow storm will make you appreciate the four powerful arguments against being eaten by a random bear which are at the ready over your shoulder. We even went to the shooting range so I could get some practice. The procedure is very trusting, as is anything in Longyearbyen. After accessing the interior of one of the houses in a particular way which I won't specify here, you simply switch on the floodlights, put up the red flag, position a target and write your name into the guest book. Once you're done shooting, toss a few coins into the bowl next to the guest book, remove the targets and flag, turn off the lights, close the door and that's that. Unfortunately, the way in was under a few meters of snow so I couldn't get in any practice shots. Ice bears, part II Later, as I was standing on a wind-polished slate of ice taking pictures of the Seed Vault (located here), I heard an ice bear roar behind (i.e. to the north of) me. I consciously remember hearing the bear, I consciously remember facing the other way around, half-crouched, rifle raised in the direction of the roar. I also consciously remember smacking the safety off and chambering a bullet after having regained control of my body. I do not remember spinning around on a wind-polished slate of ice, so treacherous I hand to balance with my arms and didn't even lift my feet when walking over it, without losing balance or footing, nor do I remember crouching and raising the gun. Evolution really is amazing; no matter what primal chord that roar struck, it certainly saved a ton of people over the years. In my case, thankfully, there was no bear to be seen down the slope. There may be no sun or anything, but the snow reflects the starlight so you tend to see surprisingly far and as I was on on the mountainside and the roar came from down from the coast, and as I had my rifle, I decided to finish the photo session while keeping the slope in close view. In hindsight, I am still glad I decided to do that as the shots came out rather nice. Next day, we drove out to Mine 12, the farthest you can away from Longyearbyen. The dump truck transporting coal alternated between driving a full load of coal back to the city and being its own snowplow. One quick trip to get coal, one slow and empty trip to plow away snow, rinse, repeat. If not for that, our 4WD would never have made it all the way up to the mine. Neither snow storms in North America nor around the Alps prepared me for what people on Longyearbyen consider normal wind in their backyard. This is where the word wind-swept was invented. The main reason that Svalbard is inhabited at all, other than the Gulf Stream, is coal. We have been told that the coal up there is of extremely high quality and while I can't say much about that, I can say that it's hard as stone. This is nice as it does not smear when you get coal all over yourself. Just shake out your jacket and pants and you are good as new. On our way back, we met two locals who had just prepared the glacier and ice caves for tourists. Had I known that in advance, I would have tried to go with them to take pictures completely away from all artificial light. Oh well, can't have everything. Speaking of not being able to have everything, the outdoor hot tub in our hotel which integrated ice bar and BBQ grill was still under several meters of snow so we couldn't use that, either. Finally, the few divers who are in Longyearbyen didn't have time to take me onto a trip while I was there. As I already missed my opportunity to dive the Arctic circle when the one diver on Gr msey happened to be on the mainland and barely missing it by diving Str tan instead, this was kind of a bummer. On the plus side, this has given me a goal to pursue and achieve. Random notes For the rest of my notes, I will resort to a largely unsorted list of bullet points as there's just too much to talk about in prose. All in all, it was well worth it. PS: If you know anyone working with Google Maps, ask them to consider improving their coverage of the Arctic. This is a real pity. As is the rest of the Arctic and the fact that Google Earth cheats you out of the North and South Pole by stretching adjacent tiles into and over them.

Next.

Previous.