Since installing a whiteboard in our kitchen, conversations at the
Acetarium have been moving in new and interesting directions.
For example, Mika and I recently noticed that, when rotated correctly, the
skeletal formula for 2,3-dimethyl-2-butanol looks pretty friendly!
I look after a lot of systems, and most of them want identical and simple backups taking of their filesystems. Currently I use backup2l which works but suffers from a couple of minor issues.
In short I want to take a full filesystem backup (i.e. Backup "/"). I wish to only exclude a few directories and mounted filesystems.
So my configuration looks like this:
# List of directories to make backups of.
# All paths MUST be absolute and start with a '/'!
SRCLIST=( / /boot -xdev )
# The following expression specifies the files not to be archived.
SKIPCOND=( -path '/var/backups/localhost' -o -path '/var/run/' -o \
-path '/tmp' -o -path '/var/tmp' \
-o -path '/dev' -o -path '/spam' \
-o -path '/var/spool/' )
The only surprising thing here is that I abuse the internals of backup2l because I know that it uses "find" to build up a list of files - so I sneakily add int "-xdev" to the first argument. This means I don't accidentally backup any mounted gluster filesystem, mounted MySQL binary/log mounts, etc.
backup2l then goes and does its jobs. It allows me to define things to run before and after the backup runs via code like this:
# This user-defined bash function is executed before a backup is made
PRE_BACKUP ()
if [ -d /etc/backup2l/pre.d/ ]; then
run-parts /etc/backup2l/pre.d/
fi
So what is my gripe? Well I get a daily email, per-system, which shows lots of detail - but the key thing. The important thing. The thing I care about more than anything else, the actual "success" or "fail" result is only discoverable by reading the mail.
If the backup fails, due to out of disk, I won't know unless I read the middle of the mail.
If the pre/post-steps fail I won't know unless I examine the output.
As I said to a colleague today in my view the success or failure of the backup is the combination of each of three distinct steps:
Subject: Backup Success - $(hostname) - $(date)So I've looked around at programs such as backup-ninja, backup-manager and they seem similar. It is a shame as I mostly like backup2l, but in short I want to do the same thing on about 50-250 hosts:
Voting is open in the Debian Project Leader Elections 2012
So now I need to figure out who to vote for. This year I didn t take part in the discussions (all my spare time was bought, basically). The platforms are linked from the Debian Project Leader Elections 2012 page above and the key discussions were:
In (roughly) 2008 I read "Blood Music" my first Greg Bear novel and
devoured it in a weekend, in only two or three reading sessions. That happens
very rarely, and when it does I often wonder whether it's the quality of the
writing or some combination of that and your personal moods and tastes at the
time that is responsible. My next Bear was "Darwin's Radio" in mid-2010 which,
whilst good, did not provoke the same feverish read. I might write short
reviews for both at some point in the future, but right now I can't remember
either well enough to do so.
In the last few weeks I read "Hull Zero Three" which was quite a different
beast. The novel first captured my attention due to the unusual way it was
promoted: via a video, which was created using a modified Quake 2 engine. It
wasn't a great video but the novelty made it interesting.
The story begins with a first-person narrator waking up from a sort-of hyper
sleep on a generation starship that has clearly had some serious problems. He
has to run to escape danger before he has an opportunity to figure out exactly
what has happened and what he is running from, or to.
Something has gone horribly wrong on the ship: it's nowhere near where it
should be going and nobody should be awake. The narrator's memory is fractured,
fabricated and unreliable; the ship is hostile, and whilst it's mechanical, it
cyclically changes and re-arranges it's hulls in an almost organic way. He is
joined by a rotating cast of similarly confused refugees of varying shapes and
sizes. Language and cultural barriers serve to further magnify the sense of
alienation that you experience on behalf of the story teller. The troupe are
persued through the corridors by a collection of lethally efficient killer
"things", contrasting with the background by being organic in nature but
mechanical in behaviour and appearance.
The scale of the ship is necessarily massive, and Bear does an impressive job
of capturing the awe you might feel when considering it. Three huge,
cigar-shaped hulls, joined by "spars" to an enormous chunk of ice captured from
the Oort cloud on exodus from the solar system. The ice serves as a mixture of
"fuel" for the biological ship systems and as part of a shield system for the
hulls from cosmic rays. However, in part due to the narrator's confusion, it
was hard to follow the very abstract descriptions of the environment in the
earlier stages of the story. I found myself re-reading passages to try and
understand the comparisons being offered and scales presented.
In common with the two earlier Bears I've read, 'body horror' features
prominently in this story. It feels like a "Dead Space" or "Resident Evil" in
space in a hard SF novel which seriously attempt to capture real speculative
science.
Like "Blood Music", I found myself compelled to read "Hull Zero Three" at every
available opportunity. I finished it off in a very enjoyable reading session
outdoors in the spring sunshine. It was an enjoyable read, well written but not
beyond criticism, with some interesting takes on well-worn themes. Very
reminiscent of the movie "Pandorum" (which was a surprisingly good movie).
It s taken some time, but now none of my personal Linux hosts (4 in total) are running the 2.6 kernel any more.
From the start (January) my company web host on Amazon EC2 has been running a 3.x kernel. My little Acer Aspire Revo low power home server, with attached disk pack that sits in my shed in a network cabinet has run 3.x for the last 6 months or so. My Linux laptop (Dell Studio 1558) which only recently got installed (and, since removing Windows, hasn t overheated once!) went to 3.x immediately. And the last piece of the puzzel is a virtual machine I ve had for many years with Bytemark.co.uk they re now offering a 3.2 kernel in their menu of selectable kernels.
Not that 3.x is that much different than 2.6.3x; but its a line in the sand of feature and security thats easy to identify. But with nearly 15 years of looking at a 2.x kernel, its about time we moved to 3.x!
Some networks have no connection to the outside except that they allow
surfing through an HTTP(S) proxy. Sometimes you are happy and the
HTTPS port (443) is unrestricted. The following server-side tools
allow you to exploit these weaknesses and get you a shell on your
server.
sslh
sslh is an SSH/SSL multiplexor. If a client connects to sslh, it
checks if the clients speaks the SSH or the SSL protocol and then
passes the connection to the according real port of SSL or some SSL
enabled service, e.g. an HTTPS, OpenVPN, Tinc or XMPP server. That way
it s possible to connect to one of these services and SSH on
the same port.
The usual scenario where this daemon is useful are firewalls which
block SSH, force HTTP to go through a proxy, but allow HTTPS
connections without restriction. In that case you let sslh listen on
the HTTPS port (443) and to move the real HTTPS server (e.g. Apache)
to listen on either a different port number (e.g. 442, 444 or 8443) or
on another IP address, e.g. on localhost, port 443.
On an Debian or Ubuntu based Apache HTTPS server, you just have to do
the following to run Apache on port 442 and sslh on port 443 instead:
apt-get install sslh as root./etc/default/sslh, change
RUN=no to RUN=yes and --ssl
127.0.0.1:443 to --ssl 127.0.0.1:442./etc/apache2/ports.conf and all
files in /etc/apache2/sites-available/ which
contain a reference to port 443 (which is only /etc/apache2/sites-available/default-ssl.conf in
the default configuration) and change all occurrences of
443 to 442.service apache2 restartservice sslh startssh -p 443 your.server.example.org) while
still being able to surf to
https://your.server.example.org/.
sslh works as threaded or as preforking daemon, or via inetd. It also
honors tcpwrapper configurations for sshd in /etc/hosts.allow and /etc/hosts.deny.
sslh is available as port or package at least in Gentoo, in FreeBSD,
in Debian and in Ubuntu.
AjaxTerm
A completely different approach takes AjaxTerm. It
provides a terminal inside a web browser with login and ssh being its
server-side backend.
Properly safe-guarded by HTTPS plus maybe HTTP based authentication
this can be an interesting emergency alternative to the more common
but also more often blocked remote login mechanisms.
AjaxTerm is available as package at least in Debian
and in
Ubuntu.
Happily I never were forced to use either of them myself. :-)
printLn(mutable IO, String line)If we bind the the first parameter, what should be the type of the function, and especially important how do we get back the mutable parameter? Consider the partially applied form printLn1:
printLn1(String line)The mutability parameter would be lost and we could not do any further I/O (and the curried form would appear as a pure function, so a good compiler would not even emit a call to it) An answer might be to put the thing into the result when currying:
printLn2(String line) -> mutable IOBut how can we explain this? In the end result, do we maybe have to use a signature like:
printLn(String, mutable in IO io1, mutable out IO io2)We could then introduce syntax to call that as
printLn(string, mutable io)Where as the mutable io argument basically expands to io and io1 for the first call, and for later calls to io1, io2 , and so on. It can also be easily curried by allowing currying to take place on variables not declared as output parameters. We can then curry the function as:
printLn3(mutable in IO io1, mutable out IO io2)
printLn4(mutable out IO io2)
If so, we can rename mutable back to unique and make that easier by introducing the unary operator & for two locations, just like Mercury uses ! for it. We could then write calls looking like this:
printLn("Hello", &io);
printLn("Hello", io, io1);
How out parameters are explained is a different topic; we could probably say that an out parameter defines a new variable.
Another option is to forbid currying of mutable parameters entirely. This would have the advantage of maintaining the somewhat simple one parameter style.
The programming language Clean does not provide any special syntactic sugar for having mutable variables. In Clean, the function gets a unique object and returns a unique object (noted by *). For example, the main entry point in a Clean program (with state) looks like this:
Start:: *World -> *WorldIn short, the function Start gets a abstract world passed that is unique and at the end returns a new unique world. In Clean syntax, our example function would most likely have the signature:
printLn :: String *IO -> *IOYou know have to either maintain one additional variable for the new unique object, which gets a bit complicated with time. On the other hand, you can do function composition on this (if you have a function composition operator that preserves uniqueness when available, as should be possible in Clean):
printTwoLines :: *IO -> *IO
printTwoLines = (printLn "World") o (printLn "Hello")
Function composition on mutable things however, does not seem like it is needed often enough in a functional programming language with a C syntax.
People might also ask why monads are not used instead. Simon L Peyton Jones and Philip Wadler described monadic I/O in their paper Imperative Functional Programming (http://research.microsoft.com/en-us/um/people/simonpj/papers/imperative.ps.Z) in 1993, and it is used in Haskell (the paper was about the implementation in Haskell anyway), one of the world s most popular and successful functional programming languages.
While monadic I/O works for the Haskell crowd, and surely some other people, the use of Monads also limits the expressiveness of code, at least as far as I can tell. At least as soon as you want to combine multiple monads, you will have to start lifting stuff from one monad to another (liftIO and friends), or perform all operations in the IO monad, which prevents obvious optimizations (such as parallelizing the creation of two arrays) in short dependencies between actions are more strict than they have to be. For a functional language targeting imperative programmers, the lifting part seems a bit too complicated.
One of the big advantages of monads is that they are much easier to implement, as they do not require extensions to the type system and the Hindley-Milner type inference algorithm used by the cool functional programming languages. If you want uniqueness typing however, you need to modify the algorithm or infer the basic types first and then infer uniqueness in a second pass (as Clean seems to do it).
This is the third summary of my squeeze RC bug squashing. If you take a look at the bug graph you will notice that the blue line went up a bit over a two week's period. I would like to claim that confirming the samhain RC bug 618728 did cost me the time (I actually gave it the time to finish), but the real reason was that I was looking at other stuff. I came back with a vengeance though, so here is the list of the bugs squashed for squeeze since the last report:
This is my monthly summary of my Debian related activities. If you re among the people who made a donation to support my work (227.83 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects.
Dpkg
Thanks to Guillem, dpkg with multiarch support is now available in Debian sid. The road has been bumpy, and it has again been delayed multiple times even after Guillem announced it on debian-devel-announce. Finally, the upload happened on March 19th.
I did not appreciate his announce because it was not coordinated at all, and had I been involved from the start, we could have drafted it in a way that sounded less scary for people. In the end, I provided a script so that people can verify whether they were affected by one of the potential problems that Guillem pointed out. While real, most of them are rather unlikely for typical multiarch usage.
Bernhard R. Link submitted a patch to add a new status command to dpkg-buildflags. This command would print all the information required to understand which flags are activated and why. It would typically be called during the build process by debian/rules to keep a trace of the build flags configuration. The goal is to help debugging and also to make it possible to extract that information automatically from build logs. I reviewed his patch and we made several iterations, it s mostly ready to be merged but there s one detail where Bernhard and I disagree and I solicited Guillem s opinion to try to take a decision. Unfortunately neither Guillem nor anyone else chimed in.
On request of Alexander Wirt, I uploaded a new backport of dpkg where I dropped the DEB_HOST_MULTIARCH variable from dpkg-architecture to ensure multi-arch is never accidentally enabled in other backports.
One last thing that I did not mention publicly at all yet, is that I contacted Lennart Poettering to suggest an improvement to the /etc/os-release file that he s trying to standardize across distributions. It occurred to me that this file could also replace our /etc/dpkg/origins/default file (and not only /etc/debian_version) provided that it could store ancestry information. After some discussions, he documented new official fields for that file (ID_LIKE, HOME_URL, SUPPORT_URL, BUG_REPORT_URL). Next step for me is to improve dpkg-vendor to support this file (as a fallback or as default, I don t know yet).
Packaging
I packaged quilt 0.60 (we re now down to 9 Debian-specific patches, from a whopping 26 in version 0.48!) and zim 0.55.
In prevision of the next upstream version of Publican, I asked the Perl team to package a few Perl modules that Publican now requires. Less than two weeks after, all of them were in Debian Unstable. Congrats and many thanks to the Perl team (and Salvatore Bonaccorso in particular, which I happen to know because we were on the same plane during last Debconf!).
On a side note, being the maintainer of nautilus-dropbox became progressively less fun over the last months, in particular because the upstream authors tried to override some of the (IMO correct) packaging decisions that I made and got in touch with Ubuntu community managers to try to have their way. Last but not least, I keep getting duplicates of a bug that is not in my package but in the official package and that Dropbox did not respond to my query.
Book update
The translation is finished and we re now reviewing the whole book. It takes a bit more time than expected because we re trying to harmonize the style and because it s difficult to coordinate the work of several volunteer reviewers.
The book cover is now almost finalized (click on it to view it in higher definitions):
We also made some progress on the interior design for the paperback. Unfortunately, I have nothing to show you yet. But it will be very nice and made with just a LaTeX stylesheet tailored for use with dblatex.
The liberation fundraising slowed down with only 41 new supporters this month but it made a nice bump anyway thanks to a generous donation of 1000 EUR by Offensive security, the company behind Backtrack Linux. They will soon communicate on this, hopefully it will boost the operation. It would be really nice if we managed to raise the remaining 3000 EUR in the few weeks left until the official release of the book!
The work on my book dominated the month and explains my relative inactivity on other fronts. I worked much more than usual, and my wife keeps telling me that I look tired and that I should go in bed earlier but I see the end of the tunnel: if everything goes well, the book should be released in a few weeks and I will be able to switch back to a saner lifestyle.
Thanks
See you next month for a new summary of my activities.
One comment Liked this article? Click here. My blog is Flattr-enabled.
I'm honored and pleased to be the person who gets to complete ls. This
project, begun around when I was born, was slow to turn into anything more
than a simple for loop over a dirent. It really took off in the mid and
late 80's, when Richard Stallman added numerous features, and the growth
has been steady ever since. But, a glance at the man page shows that
ls has never quite been complete. It fell to me to finish the job, and
I have produced several handy patches to this end:
diff -ur orig/coreutils-8.13/src/ls.c coreutils-8.13/src/ls.c
--- orig/coreutils-8.13/src/ls.c 2011-07-28 06:38:27.000000000 -0400
+++ coreutils-8.13/src/ls.c 2012-04-01 12:41:56.835106346 -0400
@@ -270,6 +270,7 @@
static int format_group_width (gid_t g);
static void print_long_format (const struct fileinfo *f);
static void print_many_per_line (void);
+static void print_jam (void);
static size_t print_name_with_quoting (const struct fileinfo *f,
bool symlink_target,
struct obstack *stack,
@@ -382,6 +383,7 @@
many_per_line for just names, many per line, sorted vertically.
horizontal for just names, many per line, sorted horizontally.
with_commas for just names, many per line, separated by commas.
+ jam to fit in the most information possible.
-l (and other options that imply -l), -1, -C, -x and -m control
this parameter. */
@@ -392,7 +394,8 @@
one_per_line, /* -1 */
many_per_line, /* -C */
horizontal, /* -x */
- with_commas /* -m */
+ with_commas, /* -m */
+ jam /* -j */
;
static enum format format;
@@ -630,6 +633,11 @@
static bool immediate_dirs;
+/* True means when multiple directories are being displayed, combine
+ * their contents as if all in one directory. -e */
+
+static bool entangle_dirs;
+
/* True means that directories are grouped before files. */
static bool directories_first;
@@ -705,6 +713,10 @@
static bool format_needs_type;
+/* Answer "yes" to all prompts. */
+
+static bool yes;
+
/* An arbitrary limit on the number of bytes in a printed time stamp.
This is set to a relatively small value to avoid the need to worry
about denial-of-service attacks on servers that run "ls" on behalf
@@ -804,6 +816,7 @@
"escape", no_argument, NULL, 'b' ,
"directory", no_argument, NULL, 'd' ,
"dired", no_argument, NULL, 'D' ,
+ "entangle", no_argument, NULL, 'e' ,
"full-time", no_argument, NULL, FULL_TIME_OPTION ,
"group-directories-first", no_argument, NULL,
GROUP_DIRECTORIES_FIRST_OPTION ,
@@ -849,12 +862,12 @@
static char const *const format_args[] =
"verbose", "long", "commas", "horizontal", "across",
- "vertical", "single-column", NULL
+ "vertical", "single-column", "jam", NULL
;
static enum format const format_types[] =
long_format, long_format, with_commas, horizontal, horizontal,
- many_per_line, one_per_line
+ many_per_line, one_per_line, jam
;
ARGMATCH_VERIFY (format_args, format_types);
@@ -1448,6 +1461,9 @@
print_dir_name = true;
+ if (entangle_dirs)
+ print_current_files ();
+
if (print_with_color)
int j;
@@ -1559,6 +1575,7 @@
print_block_size = false;
indicator_style = none;
print_inode = false;
+ yes = false;
dereference = DEREF_UNDEFINED;
recursive = false;
immediate_dirs = false;
@@ -1644,7 +1661,7 @@
int oi = -1;
int c = getopt_long (argc, argv,
- "abcdfghiklmnopqrstuvw:xABCDFGHI:LNQRST:UXZ1",
+ "abcdefghijklmnopqrstuvw:xyABCDFGHI:LNQRST:UXZ1",
long_options, &oi);
if (c == -1)
break;
@@ -1667,6 +1684,10 @@
immediate_dirs = true;
break;
+ case 'e':
+ entangle_dirs = true;
+ break;
+
case 'f':
/* Same as enabling -a -U and disabling -l -s. */
ignore_mode = IGNORE_MINIMAL;
@@ -1697,6 +1718,10 @@
print_inode = true;
break;
+ case 'j':
+ format = jam;
+ break;
+
case 'k':
human_output_opts = 0;
file_output_block_size = output_block_size = 1024;
@@ -1765,6 +1790,10 @@
format = horizontal;
break;
+ case 'y':
+ yes = true;
+ break;
+
case 'A':
if (ignore_mode == IGNORE_DEFAULT)
ignore_mode = IGNORE_DOT_AND_DOTDOT;
@@ -2510,7 +2539,7 @@
DEV_INO_PUSH (dir_stat.st_dev, dir_stat.st_ino);
- if (recursive print_dir_name)
+ if ((recursive print_dir_name) && ! entangle_dirs)
if (!first)
DIRED_PUTCHAR ('\n');
@@ -2526,7 +2555,8 @@
/* Read the directory entries, and insert the subfiles into the cwd_file'
table. */
- clear_files ();
+ if (! entangle_dirs)
+ clear_files ();
while (1)
@@ -2615,7 +2645,7 @@
DIRED_PUTCHAR ('\n');
- if (cwd_n_used)
+ if (cwd_n_used && ! entangle_dirs)
print_current_files ();
@@ -3464,6 +3494,10 @@
print_with_commas ();
break;
+ case jam:
+ print_jam ();
+ break;
+
case long_format:
for (i = 0; i < cwd_n_used; i++)
@@ -4418,6 +4452,24 @@
putchar ('\n');
+static void
+print_jam (void)
+
+ size_t filesno;
+ size_t pos = 0;
+
+ for (filesno = 0; filesno < cwd_n_used; filesno++)
+
+ struct fileinfo const *f = sorted_file[filesno];
+ size_t len = length_of_file_name_and_frills (f);
+
+ print_file_name_and_frills (f, pos);
+ pos += len;
+
+ putchar ('\n');
+
+
+
/* Assuming cursor is at position FROM, indent up to position TO.
Use a TAB character instead of two or more spaces whenever possible. */
@@ -4627,11 +4679,13 @@
-D, --dired generate output designed for Emacs' dired mode\n\
"), stdout);
fputs (_("\
+ -e, --entangle display multiple directory contents as one\n\
-f do not sort, enable -aU, disable -ls --color\n\
-F, --classify append indicator (one of */=>@ ) to entries\n\
--file-type likewise, except do not append *'\n\
--format=WORD across -x, commas -m, horizontal -x, long -l,\n\
single-column -1, verbose -l, vertical -C\n\
+ jam -j\n\
--full-time like -l --time-style=full-iso\n\
"), stdout);
fputs (_("\
@@ -4667,6 +4721,8 @@
-i, --inode print the index number of each file\n\
-I, --ignore=PATTERN do not list implied entries matching shell PATTERN\
\n\
+ -j jam output together, makes the most of limited\n\
+ space on modern systems (cell phones, twitter)\n\
-k like --block-size=1K\n\
"), stdout);
fputs (_("\
@@ -4733,6 +4789,7 @@
-w, --width=COLS assume screen width instead of current value\n\
-x list entries by lines instead of by columns\n\
-X sort alphabetically by entry extension\n\
+ -y answer all questions with \"yes\"\n\
-Z, --context print any SELinux security context of each file\n\
-1 list one file per line\n\
"), stdout);
It remains to be seen if multi-option enabled coreutils will be accepted
into Debian in time for the next release. Due to some disagreements with
the coreutils maintainer, the matter has been referred to the Technical
Committee
(Flattr me)
Traditionally new ls contributors stop once enough options have been
added that they can spell their name, in the best traditions of yellow
snow. Once ls -richard -stallman worked, I'm sure RMS moved on other
other more pressing concerns. The current maintainer, David MacKenzie, was
clearly not done yet, since only ls -david -mack worked. But he was being
slow to add these last few features, and ls was very deficient in the realm
of spelling my name (ls -o -hss .. srsly?), so I took matter into my own
hands in the best tradition of free software.
due to some new
incoming RC bugs, this week was more devoted to fixing bugs in "our" (=
the Debian Perl
Group's) packages. here's the list:
| Publisher: | Microsoft |
| Copyright: | 2004 |
| ISBN: | 0-7356-1993-X |
| Format: | Trade paperback |
| Pages: | 153 |
I d be interested in getting an instance set up of a Django powered Git project viewer (or rather, something between GitHub and viewgit)
The ideal implementation would use something like dulwich, and be super snappy.
If anyone feels like writing this, let me know. If something like this exists, please let me know even faster.
notmuch-mutt package. It will work out of the
box with Mutt, without requiring any ~/.muttrc
fiddling~/.cache/notmuch/mutt/ by
default~/.mutt-notmuch*
manually.)/etc/Muttrc.d/ (Debian-specific) or
/etc/Muttrc My public radio station is engaged in a most obnoxious spring pledge drive.
Good time to listen to podcasts. Here are the ones I'm currently liking.
Next.