Search Results: "ed"

6 December 2013

Steve Kemp: So PaaS

I just realised a lot of my projects are deployed in the same way: This includes both Apache-based projects, and node.js projects. I'm sure I could generalize this, and do clever things with git-hooks. Right now for example I have run-scripts which look like this:
#!/bin/sh
#
# /etc/service/blogspam.js/run - Runs the blogspam.net API.
#
# update the repository.
git pull --update --quiet
# install dependencies, if appropriate.
npm install
# launche
exec node server.js
It seems the only thing that differs is the name of the directory and the remote git clone URL. With a bit of scripting magic I'm sure you could push applications to a virgin Debian installation and have it do the right thing. I think the only obvious thing I'm missing is a list of Debian dependencies. Perhaps adding soemthing like the packages.json file I could add an extra step:
apt-get update -qq
apt-get install --yes --force-yes $(cat packages.apt)
Making deployments easy is a good thing, and consistency helps..

Petter Reinholdtsen: Debian Edu interview: Klaus Knopper

It has been a while since I managed to publish the last interview, but the Debian Edu / Skolelinux community is still going strong, and yesterday we even had a new school administrator show up on #debian-edu to share his success story with installing Debian Edu at their school. This time I have been able to get some helpful comments from the creator of Knoppix, Klaus Knopper, who was involved in a Skolelinux project in Germany a few years ago. Who are you, and how do you spend your days? I am Klaus Knopper. I have a master degree in electrical engineering, and is currently professor in information management at the university of applied sciences Kaiserslautern / Germany and freelance Open Source software developer and consultant. All of this is pretty much of the work I spend my days with. Apart from teaching, I'm also conducting some more or less experimental projects like the Knoppix GNU/Linux live system (Debian-based like Skolelinux), ADRIANE (a blind-friendly talking desktop system) and LINBO (Linux-based network boot console, a fast remote install and repair system supporting various operating systems). How did you get in contact with the Skolelinux / Debian Edu project? The credit for this have to go to Kurt Gramlich, who is the German coordinator for Skolelinux. We were looking for an all-in-one open source community-supported distribution for schools, and Kurt introduced us to Skolelinux for this purpose. What do you see as the advantages of Skolelinux / Debian Edu? What do you see as the disadvantages of Skolelinux / Debian Edu? For these reasons and experience from our project, I would now rather consider using plain Debian for schools next time, until Skolelinux is more closely integrated into Debian and becomes upgradeable without reinstallation. Which free software do you use daily? GNU/Linux with LXDE desktop, bash for interactive dialog and programming, texlive for documentation and correspondence, occasionally LibreOffice for document format conversion. Various programming languages for teaching. Which strategy do you believe is the right one to use to get schools to use free software? Strong arguments are

Wouter Verhelst: POV-Ray 3.7 out!

I've been using POV-Ray off and on for the past decade or so. I've never been extremely talented with graphical stuff, but I've always liked playing around with it; and POV-Ray, with its turing-complete scene description language, appeals to me as a programmer. I've used it when I needed to do some animation; for instance, I created the FOSDEM 2013 and DebConf13 "wait screen" animations for the video team with FOSDEM. One particular downside of POV-Ray has always been the fact that their license was a custom non-free one. This was a historical accident (POV-Ray has existed for a long time, since before the popularization of FLOSS), and AIUI, the relicensing was impossible for various reasons. However, a rewrite of POV-Ray (as version 3.7) has been in the making for quite a while. Today, I noticed two things: first, POV-Ray 3.7 was released (under the AGPLv3, thereby becoming Free Software); and second, as of the 3.7 release, the POV-Ray is put into a git repository and available on github. Also, apart from it being free software now, POV-Ray 3.7 has a few new features as well. Most importantly among those (at least in my opinion), POV-Ray 3.7 is a multithreaded application, in contrast to POV-Ray 3.6 and before which wasn't. Building it seemed to have some issues with the versions of a few things that are in Debian unstable; but for one of these a fix has already been merged, and for the other a merge request is out. Now to decide whether I should package it...

Gunnar Wolf: For people in Mexico: Workshop next Wednesday! Video editing from the command line (by Chema Serralde, @joseserralde)

(Yes, yes... Maybe I should post in Spanish.. But hey, gotta keep consistecy in my blog!) General, public, open invitation Are you in Mexico City, or do you plan to be next Wednesday (December 11)? Are you interested in video edition? In Free Software? I will have the pleasure to host at home the great Chema Serralde, a good friend, and a multifacetic guru both in the technical and musical areas. He will present a workshop: Video editing from the command line. I asked Chema for an outline of his talk, but given he is a busy guy, I will basically translate the introduction he prepared for this same material in FSL Vallarta, held two weeks ago.
With the help of the commandline, you can become a multimedia guru. We will edit a video using just a terminal. This skill will surprise your friends and your couple. But the most important is that this knowledge is just an excuse to understand step by step what does a video CODEC mean, what is a FORMAT, and how video and audio editors work; by using this knowledge, you will be able to set the basis for multimedia editing, without the promises and secrets of propietary editors. How much does my file weigh and why? How to improve a video file's quality? Why cannot I read my camera's information from GNU/Linux? By the end of this workshop, we well see how some libraries help you develop your first audio and video application, what are their main APIs and uses.
Logistics Everybody is welcome to come for free, no questions asked, no fees collected. I can offer coffee for all, but if you want anything else to eat/drink, you are welcome to bring it. We do require you to reserve and confirm your place (mail me to my usual mail address). We have limited space, and I must set an absolute quota of 10 participants. Some people hide their address... Mine is quite publicly known: Av. Copilco 233, just by Parque Hugo Margain, on the Northern edge of UNAM (Metro Copilco). The course starts at 16:00, and lasts... As long as we make it last ;-) So, that said... See you there! :-D [update]: Chema sent me the list of topics he plans to cover. Copy/pasting from his mail, in Spanish:
TALLER REL MPAGO DE EDICI N AUDIOVISUAL EN L NEA DE COMANDO
Jos Mar a Serralde Ruiz, facilitador
  1. Editando como cavern cola.
    1. Manipulaci n b sica de archivos multimedia en entornos POSIX.
    2. S un Bash VJ (videojockey)
    3. Vaciando y entubando
  2. Editando como cient fico.
    1. Encabezados y fourcc
    2. 3 familias de CODECS de v deo y sus patentes
    3. 3 famlias de CODECS de audio y sus patentes
    4. Muxers, demuxers y muxes.
  3. Editando como artista.
    1. Cajas de herramientas en software libre para procesamiento de v deo.
    2. Procesamiento en tiempo real de v deo (el que se crea artista pierde)
    3. Derritiendo v deo, audio con calcetines MELT + SOX
Software necesario (sistemas operativos POSIX, windouseros acercarse con el af n de repensar sus vidas): mplayer, avconv/ffmpeg (libavcodec), melt, sox, imagemagick

5 December 2013

Daniel Kahn Gillmor: The legal utility of deniability in secure chat

This Monday, I attended a workshop on Multi-party Off the Record Messaging and Deniability hosted by the Calyx Institute. The discussion was a combination of legal and technical people, looking at how the characteristics of this particular technology affect (or do not affect) the law. This is a report-back, since I know other people wanted to attend. I'm not a lawyer, but I develop software to improve communications security, I care about these questions, and I want other people to be aware of the discussion. I hope I did not misrepresent anything below. I'd be happy if anyone wants to offer corrections. BackgroundOff the Record Messaging (OTR) is a way to secure instant messaging (e.g. jabber/XMPP, gChat, AIM). The two most common characteristics people want from a secure instant messaging program are:
Authentication
Each participant should be able to know specifically who the other parties are on the chat.
Confidentiality
The content of the messages should only be intelligible to the parties involved with the chat; it should appear opaque or encrypted to anyone else listening in. Note that confidentiality effectively depends on authentication -- if you don't know who you're talking to, you can't make sensible assertions about confidentiality.
As with many other modern networked encryption schemes, OTR relies on each user maintaining a long-lived "secret key", and publishing a corresponding "public key" for their peers to examine. These keys are critical for providing authentication (and by extension, for confidentiality). But OTR offers several interesting characteristics beyond the common two. Its most commonly cited characteristics are "forward secrecy" and "deniability".
Forward secrecy
Assuming the parties communicating are operating in good faith, forward secrecy offers protection against a special kind of adversary: one who logs the encrypted chat, and subsequently steals either party's long-term secret key. Without forward secrecy, such an adversary would be able to discover the content of the messages, violating the confidentiality characteristic. With forward secrecy, this adversary is be stymied and the messages remain confidential.
Deniability
Deniability only comes into play when one of the parties is no longer operating in good faith (e.g. their computer is compromised, or they are collaborating with an adversary). In this context, if Alice is chatting with Bob, she does not want Bob to be able to cryptographically prove to anyone else that she made any of the specific statements in the conversation. This is the focus of Monday's discussion. To be clear, this kind of deniability means Alice can correctly say "you have no cryptographic proof I said X", but it does not let her assert "here is cryptographic proof that I did not say X" (I can't think of any protocol that offers the latter assertion). The opposite of deniability is a cryptographic proof of origin, which usually runs something like "only someone with access to Alice's secret key could have said X."
The traditional two-party OTR protocol has offered both forward secrecy and deniability for years. But deniability in particular is a challenging characteristic to provide for group chat which is the domain of Multi-Party OTR (mpOTR). You can read some past discussion about the challenges of deniability in mpOTR (and why it's harder when there are more than two people chatting) from the otr-users mailing list. If you're not doing anything wrong... The discussion was well-anchored by a comment from another participant who cheekily asked "If you're not doing anything wrong, why do you need to hide your chat at all, let alone be able to deny it?" The general sense of the room was that we'd all heard this question many times, from many people. There are lots of problems with the ideas behind the question from many perspectives. But just from a legal perspective, there are at least two problems with the way this question is posed: In these situations, people confront real risk from the law. If we care about these people, we need to figure out if we can build systems to help them reduce that legal risk (of course we also need to fix broken laws, and the legal environment in general, but those approaches were out of scope for this discussion). The Legal Utility of Deniability Monday's meeting was called specifically because it wasn't clear how much real-world usefulness there is in the "deniability" characteristic, and whether this feature is worth the development effort and implementation tradeoffs required. In particular, the group was interested in deniability's utility in legal contexts; many (most?) people in the room were lawyers, and it's also not clear that deniability has much utility outside of a formal legal setting. If your adversary isn't constrained by some rule of law, they probably won't care at all whether there is a cryptographic proof or not that you wrote a particular message (In retrospect, one possible exception is exposure in the media, but we did not discuss that scenario). Places of possible usefulness So where might deniability come in handy during civil litigation or a criminal trial? Presumably the circumstance is that a piece of a chat log is offered as incriminating evidence, and the defendant is trying to deny something that they appear to have said in the log. This denial could take place in two rather different contexts: during rules over admissibility of evidence, or (once admitted) in front of a jury. In legal wrangling over admissibility, apparently a lot of horse-trading can go on -- each side concedes some things in exchange for the other side conceding other things. It appears that cryptographic proof of origin (that is, a lack of deniability) on the chat logs themselves might reduce the amount of leverage a defense lawyer can get from conceding or arguing strongly over that piece of evidence. For example, if the chain of custody of a chat transcript is fuzzy (i.e. the transcript could have been mishandled or modified somehow before reaching trial), then a cryptographic proof of origin would make it much harder for the defense to contest the chat transcript on the grounds of tampering. Deniability would give the defense more bargaining power. In arguing about already-admitted evidence before a jury, deniability in this sense seems like a job for expert witnesses, who would need to convince the jury of their interpretation of the data. There was a lot of skepticism in the room over this, both around the possibility of most jurors really understanding what OTR's claim of deniability actually means, and on jurors' ability to distinguish this argument from a bogus argument presented by an opposing expert witness who is willing to lie about the nature of the protocol (or who misunderstands it and passes on their misunderstanding to the jury). The complexity of the tech systems involved in a data-heavy prosecution or civil litigation are themselves opportunities for lawyers to argue (and experts to weigh in) on the general reliability of these systems. Sifting through the quantities of data available and ensuring that the appropriate evidence is actually findable, relevant, and suitably preserved for the jury's inspection is a hard and complicated job, with room for error. OTR's deniability might be one more element in a multi-pronged attack on these data systems. These are the most compelling arguments for the legal utility of deniability that I took away from the discussion. I confess that they don't seem particularly strong to me, though some level of "avoiding a weaker position when horse-trading" resonates with me. What about the arguments against its utility? Limitations The most basic argument against OTR's deniability is that courts don't care about cryptographic proof for digital evidence. People are convicted or lose civil cases based on unsigned electronic communications (e.g. normal e-mail, plain chat logs) all the time. OTR's deniability doesn't provide any legal cover stronger than trying to claim you didn't write a given e-mail that appears to have originated from your account. As someone who understands the forgeability of e-mail, i find this overall situation troubling, but it seems to be where we are. Worse, OTR's deniability doesn't cover whether you had a conversation, just what you said in that conversation. That is, Bob can still cryptographically prove to an adversary (or before a judge or jury) that he had a communication with someone controlling Alice's secret key (which is probably Alice); he just can't prove that Alice herself said any particular part of the conversation he produces. Additionally, there are runtime tradeoffs depending on how the protocol manages to achieve these features. For example, forward secrecy itself requires an additional round trip or two when compared to authenticated, encrypted communications without forward secrecy (a "round trip" is a message from Alice to Bob followed by a message back from Bob to Alice). Getting proper deniability into the mpOTR spec might incur extra latency (imagine having to wait 60 seconds after everyone joins before starting a group chat, or a pause in the chat of 15 seconds when a new member joins) or extra computational power (meaning that they might not work well on slower/older devices) or an order of magnitude more bandwidth (meaning that chat might not work at all on a weak connection). There could also simply be complexity that makes it harder to correctly implement a protocol with deniability than an alternate protocol without deniability. Incorrectly-implemented software can put its users at risk. I don't know enough about the current state of mpOTR to know what the specific tradeoffs are for the deniability feature, but it's clear there will be some. Who decides whether the tradeoffs are worth the feature? Other kinds of deniability Further weakening the case for the legal utility of OTR's deniability, there seem to be other ways to get deniability in a legal context over a chat transcript. There are deniability arguments that can be made from outside the protocol. For example, you can always claim someone else took control of your computer while you were asleep or using the bathroom or eating dinner, or you can claim that your computer had a virus that exported your secret key and it must have been used by someone else. If you're desperate enough to sacrifice your digital identity, you could arrange to have your secret key published, at which point anyone can make signed statements with it. Having forward secrecy makes it possible to expose your secret key without exposing the content of your past communications to any listener who happened to log them. Conclusion My takeaway from the discussion is that the legal utility of OTR's deniability is non-zero, but quite low; and that development energy focused on deniability is probably only justified if there are very few costs associated with it. Several folks pointed out that most communications-security tools are too complicated or inconvenient to use for normal people. If we have limited development energy to spend on securing instant messaging, usability and ubiquity would be a better focus than this form of deniability. Secure chat systems that take too long to make, that are too complex, or that are too cumbersome are not going to be adopted. But this doesn't mean people won't chat at all -- they'll just use cleartext chat, or maybe they'll use supposedly "secure" protocols with even worse properties: for example, without proper end-to-end authentication (permitting spoofing or impersonation by the server operator or potentially by anyone else); with encryption that is reversible by the chatroom operator or flawed enough to be reversed by any listener with a powerful computer; without forward secrecy; or so on. As a demonstration of this, we heard some lawyers in the room admit to using Skype to talk with their clients even though they know it's not a safe communications channel because their clients' adversaries might have access to the skype messaging system itself. My conclusion from the meeting is that there are a few particular situations where deniability could be useful legally, but that overall, it is not where we as a community should be spending our development energy. Perhaps in some future world where all communications are already authenticated, encrypted, and forward-secret by default, we can look into improving our protocols to provide this characteristic, but for now, we really need to work on usability, popularization, and wide deployment. Thanks Many thanks to Nick Merrill for organizing the discussion, to Shayana Kadidal and Stanley Cohen for providing a wealth of legal insight and legal experience, to Tom Ritter for an excellent presentation of the technical details, and to everyone in the group who participated in the interesting and lively discussion. Tags: chat, deniability, otr, security

Steve Kemp: Belated updates

Today I should have been heading down to York, to attend the Bytemark Christmas party. Instead I'm here in Edinburgh, because wind/storms basically shutdown the rail network in Scotland for the morning. Technically I could have probably made it, but only belatedly and only at a huge cost to my sanity. The train-station was insane with stranded people, and there seemed no guarantee the recently-revived service would continue. So instead I'm sulking at home. I had a lot of other things scheduled to do in York/London today/tomorrow, for reasons that will become apparent next week, so to say I'm annoyed is an understatement. In happier news I'm not dead.
Walking to work this morning was horrific, there was so much wind 70-100mph, that I counldn't actually cross a bridge, on Ocean Drive, because I just kept getting blown into the road. (Yeah, that's a road that is very close to the coast. Driving wind. Horrible rain. Storming sea. Fun.)
I ended up retracing my steps, and taking a detour. (PS. My boots leaked.) Not a good day. Enjoy some software instead - a trivial HTTP / XMPP bridge.

Matthew Palmer: The easy bit of software development

I m sure this isn t an original thought of mine, but it just popped into my head and I think it s something of a fundamental truth that all software developers need to keep in mind:
Writing software is easy. The hard part is writing software that works.
All too often, we get so caught up in the rush of building something that we forget that it has to work and, all too often, we fail in some fundamental fashion, whether it s doesn t satisfy the user s needs or you just broke my $FEATURE! (which is the context I was thinking of).

4 December 2013

Aigars Mahinovs: Translation management workflows

Whatever you do with translations, consider translation management issues. For example, you are developing a multilingual web site. All kinds of labels and buttons and form fields are nicely translatable with trans template tag and ugettext. You have po files that follow your code from dev to stage to production environment.
Now you add a CMS into the mix. And suddenly - you translations are in more than one place, in more than one format and follow different routes to production.
Now imagine that you need to add Chinese language to your entire site. The translator is an off-site contractor. What files would you send to him to translate? How would you generate them? How will you integrate them?
If someone adds or changes a page on production in English: how will your developers see that change? how will you know that an updated translation for Chinese is needed? how will you manage the update of the translation?
If you make a CMS and don't have at least the export_po_file and import_po_file management commands, then you are not really multilingual. It is either that or figuring out your own answer for the above questions.
I have finally found a Django-based CMS that has those - http://pythonhosted.org/django-page-cms/ . Have not really tried it yet, but I am hopeful.

Jonathan McDowell: Thoughts on SSDs and encryption

My laptop is just over 3 years old, which is about the point I start to think about a replacement. At present there's nothing that's an obvious contender so I've been looking at an SSD to prolong it by another year or two. One of the other thoughts I had is that I currently use dm-crypt under Linux to provide whole disk encryption for everything except the boot partition - I have a bunch of my personal financial and immigration documents stored that I'd prefer not to get disclosed if my laptop is stolen. Modern drives have started offered integral AES encryption options, so perhaps I could offload that to the drive (my i5 470UM lacks the hardware instructions for this). General consensus in the pub (where all the best security advice is to be found) is that no one present trusted SSD firmware authors to not use some badly chosen AES crypto mode, or leave the key lying around plain text in easily readable flash, or some other implementation mishap. So how hard would it be to retrofit reliable (or at least source verifiable and thus more reliable) crypto to an SSD? There was an impressive article recently about reverse engineering the firmware of a HDD, to the point of modifying data returned to the host and also running Linux on the controller. It seems that SSD firmware should be easier - NAND is simpler to talk to than motors and magnetic sensors, right? It's a case of gluing together a SATA interface, a NAND controller and an AES offload engine, yes? Aside from the minor matter of finding a suitable drive with an available JTAG interface, a controller with docs (or more likely that can be reverse engineered) and enough time to produce a replacement open firmware, that is. Alternatively can anyone provide some idea of how secure the available laptop SSDs on the market actually are? I'm fine with "the NSA can read your data if they want" because a determined attacker will be able to find other ways to get my data anyway, but I don't want "anyone who finds the drive can use this loophole in the firmware by wiggling some bits with jtag to dump the key and read all your data".

3 December 2013

Matthew Garrett: Subverting security with kexec

(Discussion of a presentation I gave at Kiwicon last month)

Kexec is a Linux kernel feature intended to allow the booting of a replacement kernel at runtime. There's a few reasons you might want to do that, such as using Linux as a bootloader[1], rebooting without having to wait for the firmware to reinitialise or booting into a minimal kernel and userspace that can be booted on crash in order to save system state for later analysis.

But kexec's significantly more flexible than this. The kexec system call interface takes a list of segments (ie, pointers to a userspace buffer and the desired target destination) and an entry point. The kernel relocates those segments and jumps to the entry point. That entry point is typically code referred to as purgatory, due to the fact that it lives between the world of the first kernel and the world of the second kernel. The purgatory code sets up the environment for the second kernel and then jumps to it. The first kernel doesn't need to know anything about what the second kernel is or does. While it's conventional to load Linux, you can load just about anything.

The most important thing to note here is that none of this is signed. In other words, despite us having a robust in-kernel mechanism for ensuring that only signed modules can be inserted into the kernel, root can still load arbitrary code via kexec and execute it. This seems like a somewhat irritating way to patch the running kernel, so thankfully there's a much more straightforward approach.

I modified kexec to add an additional loader and uploaded the code here. Build and install it. Make sure that /sys/module/module/parameters/sig_enforce on your system is "Y". Then, as root, do something like:
kexec --type="dummy" --address= printf "0x%x" $(( $(grep "B sig_enforce" /proc/kallsyms awk ' print "0x"$1 ') & 0x7fffffff)) --value=0 --load-preserve-context --mem-max=0x10000 /bin/true
to load it[2]. Now do kexec -e and watch colours flash and check /sys/module/module/parameters/sig_enforce again.

The beauty of this approach is that it doesn't rely on any kernel bugs - it's using kernel functionality that was explicitly designed to let you do this kind of thing (ie, run arbitrary code in ring 0). There's not really any way to fix it beyond adding a new system call that has rather tighter restrictions on the binaries that can be loaded. If you're using signed modules but still permit kexec, you're not really adding any additional security.

But that's not the most interesting way to use kexec. If you can load arbitrary code into the kernel, you can load anything. Including, say, the Windows kernel. ReactOS provides a bootloader that's able to boot the Windows 2003 kernel, and it shouldn't be too difficult for a sufficiently enterprising individual to work out how to get Windows 8 booting. Things are a little trickier on UEFI - you need to tell the firmware which virtual physical map to use, and you can only do it once. If Linux has already done that, it's going to be difficult to set up a different map for Windows. Thankfully, there's an easy workaround. Just boot with the "noefi" kernel argument and the kernel will skip UEFI setup, letting you set up your own map.

Why would you want to do this? The most obvious reason is avoiding Secure Boot restrictions. Secure Boot, if enabled, is explicitly designed to stop you booting modified kernels unless you've added your own keys. But if you boot a signed Linux distribution with kexec enabled (like, say, Ubuntu) then you're able to boot a modified Windows kernel that will still believe it was booted securely. That means you can disable stuff like the Early Launch Anti-Malware feature or driver signing, or just stick whatever code you want directly into the kernel. In most cases all you'd need for this would be a bootloader, kernel and an initrd containing support for the main storage, an ntfs driver and a copy of kexec-tools. That should be well under 10MB, so it'll easily fit on the EFI system partition. Copy it over the Windows bootloader and you should be able to boot a modified Windows kernel without any terribly obvious graphical glitches in the process.

And that's the story of why kexec is disabled on Fedora when Secure Boot is enabled.

[1] That way you only have to write most drivers once
[2] The address section finds the address of the sig_enforce symbol in the kernel, and the value argument tells the dummy code what value to set that address to. --load-preserve-context informs the kernel that it should save hardware state in order to permit returning to the original kernel. --mem-max indicates the highest address that the kernel needs to back up. /bin/true is just there to satisfy the argument parser.

comment count unavailable comments

Iustin Pop: A not so dark sky

I'm aware of how not-dark our modern city night sky is, but sometimes it still heavily surprises me how full of light it actually is. Yesterday I went about five kilometres out of Z rich; I thought at that distance, on a reasonably dark hill, it would be good enough for some night-sky shots. So I setup my tripod, only to realise I can't expose over 30 seconds because (at low ISO and fast aperture, shooting not straight up), everything is bleached out: I couldn't believe my eyes. Yes, I heard that one needs to go 100Km out of big cities, but but So I went and found that since about 2004, all of Switzerland is light polluted - even going on top of Jungfrau, for example, wouldn't completely eliminate it. I also learned about Merle Walker's equation - yeah, 200Km or more away from light sources would be good. Not here, in this quite small, quite populated country in the middle of Europe :/ I realised that as soon as my eyes got acclimated to the location I was at, I could easily see everything around me, due to the light pollution. I miss a really dark sky - I remember a couple of years back, in a different country, stopping at night on the side of the road, and being shocked at how the sky was filled with starts. I didn't have any camera with me at that time Heck, I remember seeing the Milky Way way back as a child, but nowadays when I get out of work, I can barely see 4-5 points of light in the sky. Anyway, enough with ranting Thanks to modern technology, one can recover lots of detail, even in a washed out picture. So in the end, shooting straight up, I could get some resemblance of structure (and this was only at 28mm, which is not that wide): Or alternatively, one can use the glowing light for a bit of play/contrast: I also played with stacking images via Deep Sky Stacker, but stacking - I learned to my surprise - only works to reduce noise in the final image, and not to make it "brighter" or more detailed. Live and learn The result was a bit better than not stacked, but not by much: This was a 25 5s, ISO 800, f/4, same 28mm lens. What I found surprising is the "non-star object" (to call it so) near the centre of the image - I have no idea what it is, it definitely doesn't look like a star, it could be an elliptical galaxy or so. I tried navigating back in time via Stellarium, but I don't remember the orientation of the lens, so it will remain a mystery to me. Anyway, two more pictures and higher resolutions on a Smugmug album. Feel free to leave a comment or drop me an email if you have suggestions where to take nice night sky photos in Switzerland

Craig Small: procps-ng 3.3.9

Procps version 3.3.9 was released today. As there has been some API changes and fixes which means the library has changed again. There is a fine balance between fixing or enhancing library functions and keeping the API stable, with the added problem it wasn t a terribly good one to start with. Besides the API change, the following changes were made: Tar file is at sourceforge at https://sourceforge.net/projects/procps-ng/files/Production/

Dirk Eddelbuettel: digest 0.6.4

digest version 0.6.4 is now on CRAN and in Debian. This is a pure maintenance release which should help with a build issue affecting users on Solaris. CRANberries provides the usual summary of changes to version 0.6.3. Our package is available via the R-Forge page leading to svn and tarball access, my digest page, the local directory here as well as via Debian and its mirrors.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Brett Parker: dd over ssh oddness

So, using the command:
root@new# ssh root@old dd if=/dev/vg/somedisk dd of=/dev/vg/somedisk
appears to fail, getting a SIGTERM at some point for no discernable reason... however, using
root@old# dd if=/dev/vg/somedisk ssh root@new dd of=/dev/vg/somedisk
works fine. The pull version fails at a fairly random point after a fairly undefined period of time. The push version works everytime. This is most confusing and odd... Dear lazyweb, please give me some new ideas as to what's going on, it's driving me nuts! Update: solved... A different daemon wasn't limiting it's killing habits in the case that a certain process wasn't running, and was killing the ssh process on the new server almost at random, found the bug in the code and now testing with that. Thanks for all the suggestions though, much appreciated.

Daniel Pocock: Ski season starts

The ski season has started again, I went down to Andermatt on Sunday to have some fun. Here is a video made with my camera phone:

Russell Coker: Preferring Not To

I ve just read Bartleby the Scrivener which is a short story about a scrivener who refused to work saying I d prefer not to . It reminded me of some situations in the computer industry. I ve never seen a single case where someone preferred not to work when everyone around them (colleagues and management) wanted them to work. But then the incidence of having an entire team and management wanting to work efficiently isn t nearly as common as one might imagine. In some cases it s desired that someone not work, such as a former colleague who was hired as a sysadmin but did nothing but change backup tapes (a few hours work per week). Not having him login as root improved the general reliability of the servers but it was fortunate that we never needed to restore from backups One time I had a colleague who preferred to spend most of his time in the office searching the Internet for videos of street fights. I have often told colleagues that I would prefer them to work, but in the case of a guy who s only hobby is street-fighting I decided to let it go. Managing people can be difficult, particularly for someone who doesn t like disagreements. Some managers that I ve reported to seemed to prefer not to manage in an apparent attempt to avoid disputes. One time when I complained about a colleague not even having a suitable computer to permit doing any work a manager responded with the rhetorical question what do you expect me to do? . That manager didn t do any annual reviews of staff for over a year, he only eventually did some reviews because he was told that his scheduled promotion was on hold until he got them done. I got the impression that at least two levels of management preferred not to work at that company. Sometimes it just gets weird though, such as the occasion when I was the only member of a team and the manager who was supposedly managing no-one but me never seemed to have time to have a meeting with me. But he didn t want me to bypass him and talk directly to other people in the company, so he preferred not to work and not to have anyone else do his job. Most of the companies that I ve worked for in a full-time capacity didn t seem to have any effective technical interviews (note that I ve mostly worked for financial companies and ISPs not free software companies). So it seems that anyone with minimal computer skills who wants a well paying job could just send out a CV to a bunch of recruiting agencies, get interviewed by enough companies to eventually hit one without a technical interview process and then find a job that doesn t require work. Depression The Wikipedia page about Bartleby the Scrivener [2] suggests that Bartleby was depressed. I wonder how much of the lack of performance I ve witnessed has been due to depression. There appears to be a strong correlation between work environments that cause depression and people preferring not to work. Maybe managers should be considering how to make work less depressing to try and get more effective employees (in terms of quality and quantity of work). One example of this is the sysadmin team death spiral I ve witnessed where no-one can automate solving problems (EG by cron jobs to manage resource usage and analysis tools to find minor problems before they become major problems) because everyone is dedicated to fixing things that break needlessly (EG systems crashing due to lack of disk space). When people start getting control over recurring problems and automating things then the work becomes increasingly about solving problems and less about implementing the same manual processes every day/week and it s more fun and effective for everyone. At the BoF on depression at LCA 2013 one delegate stated that many companies have people in HR who can arrange support for depressed employees. Apparently if you are depressed and you work for a company that s large enough to have a HR department then it can be beneficial to talk to HR about it. That probably works well in the case where an employee is depressed but the company is working well. But in the case where the company isn t working well it seems unlikely to help. David Graeber wrote an interesting article about Bullshit Jobs [3]. He goes a bit far, I don t think that late night pizza delivery is a bullshit job and actuaries are useful to society. But his points about the existence of useless jobs are reasonable. Management Levels I sometimes wonder whether there is some benefit in establishing social norms about working and then having management take little interest in how it happens. If a team works well together then management could just set deadlines (which would be negotiated with employees who know what s possible) and let the team work out how to do it. Then instead of having one manager for each team of ~10 people who theoretically tracks what everyone is doing you could have one manager for a dozen teams who just tracks overall team performance essentially remove a layer of management. Valve is famous for having no formal management structure and for getting things done, unfortunately that apparently allows school-style cliques to block actions [4]. But I think that the Valve experiment is useful and provides some ideas that can be used by other companies. Maybe if instead of requiring consensus of the entire company for hiring decisions they only required consensus of the team things would have worked better. Of course another down-side to such things is that hierarchical management can be good for avoiding discrimination and bullying. The article I cited about Valve compares it to high-school. It could be that Valve employees were all nice people who only hired other nice people. But if similar systems were implemented in many companies then some would surely end up being like a typical high school with all the bullying and mistreatment of minority groups that entails. Michael O. Church wrote an interesting article in which he divides employees into four categories, loser , clueless , psychopaths , and technocrats (note that he didn t invent the first three names) [5]. In his model the clueless category includes most middle-management. I think that there are some problems with Michael s model and I m not arguing for a technocracy (which is how this post might be interpreted in terms of his ideas). But I think he demonstrates some of the real problems in the way companies are managed and in his model the losers prefer not to work as long as they can get paid. Conclusion I don t have any good solutions to these problems to offer. It seems that the best we can hope for is incremental change to make work less depressing, to have the minimal amount of management, and to avoid bullshit jobs .

Antoine Beaupr : Announcing a prettier noping

I have implemented really pretty histograms in the venerable ping software, something I never thought could be improved, until I discovered prettyping.sh, something that was just begging for improvements. Which are now done. But first some history... First, in 1983 (!), there was ping and network operators rejoiced, as they could see if a host was down or not, and have all sorts of geeky statistics:
PING koumbit.net (209.44.112.66): 48 data bytes
56 bytes from 209.44.112.66: icmp_seq=0 ttl=52 time=25.076 ms
56 bytes from 209.44.112.66: icmp_seq=1 ttl=52 time=24.006 ms
56 bytes from 209.44.112.66: icmp_seq=2 ttl=52 time=24.106 ms
^C--- koumbit.net ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 24.006/24.396/25.076/0.483 ms
Then, in 2006, there was noping, and things were, well, not much better, but we had colors, and there was some rejoicing. noping before Then, in october 2013, there was prettyping.sh, and things got really flashy and "oh wow, you can do that?" There was much rejoicing. prettyping animation Then, tonight, I learned ncurses and there was much headaches. But then, I reimplemented prettyping in noping, and I am so happy that I wrote this blog post: noping now Unless you haven't figured out how cool this is, let me break it down for you:
  1. it supports IPv4 and IPv6
  2. it allows you to track multiple hosts at the same time, and compare them
  3. this allows you to easily track down failure points in a network, something for which you usually need smokeping (needs a webserver) or mtr (doesn't have colors)
  4. it allows you to track a lot (the last minute at least) of history by default
  5. it is visually easy to track, even from a distance
You may know of that hack that can make "ping" ring a bell when it receives a packet? This is better: you can see the packets latency (or when they are just dropped!) from a distance, using an intuitive color code. The code is up for review here:
git clone -b prettyping git://src.anarc.at/liboping
Thanks to the well architectured noping, the patches were not that complicated to implemented. I have contacted upstream to get those changes merged, and hopefully this will be in your favorite Debian distribution soon. Rejoice!

2 December 2013

Tim Retout: How not to parse search queries

While I remember, I have uploaded the slides from my talk about Solr and Perl at the London Perl Workshop. This talk was inspired by having seen and contributed to at least five different sets of Solr search code at my current job, all of which (I now believe) were doing it wrong. I distilled this hard-won knowledge into a 20 minute talk, which - funny story - I actually delivered twice to work around a cock-up in the printed schedule. I don't believe any video was successfully taken, but I may be proved wrong later. I have also uploaded the Parse::Yapp grammar mentioned in the talk. In case you don't have time to read the slides, the right way to present Solr via Perl is to use the 'edismax' parser, and write your code a bit like this:
my $solr = WebService::Solr->new($url);
my $s = $query->param('q');
# WebService::Solr::Query objects are useful for
# 'fq' params, but avoid them for main 'q' param.
my $options =  
 fq => [WebService::Solr::Query->new(...)];
 ;
$solr->search($s, \%options);
The key thing here is not to put any complicated parsing code in between the user and Solr. Avoid Search::QueryParser at all costs.

Tim Retout: Questhub.io

At the London Perl Workshop last Saturday, one of the lightning talks was about Questhub.io, formerly known as "play-perl.org". It's social gamification for your task list, or something like that. Buzzword-tastic! But most importantly, there seems to be a nice community of programming types to procrastinate with you on your quests. This means I can finally get to work refuting lamby's prediction about gamification of Debian development! Tasks are referred to as "Quests", and are pursued in themed "Realms", for that World of Warcraft feeling. For example, there's a "Perl" realm, and a "Lisp" realm, and a "Haskell" realm, but also non-programming realms like "Fitness" and "Japanese". Of course, part of me now wants to construct a federated version which can be self-hosted. :) Another downside of questhub currently is the lack of SSL support - your session cookies are sent in plain text. I hope this changes soon.

Thorsten Glaser: Neo900

I ve did something I surely will (financially) regret, next year, and designated the Neo900 to be the successor to my PocketPC, due to the latter having only 64 MiB RAM and Geocaching applications being quite hungry. It s got a lovely hardware keyboard, a pen display like the PocketPC (as opposed to the wishy-washy displays that Android and iPhone have), not only GPS but also GLONASS, fully free software with mostly free firmware (I m okay with that, mostly), a Ctrl key (useful in ssh and locally and my text editor; ^I is Tab, so it s useful in shell, too), WLAN, UMTS (I don t think I need LTE and would rather it have the more RAM), USB host (OTG), and lots of other nice features. In short, it s a tinkerable device: one I can not only hack at, but also hack on. Since I use a dumbphone for mobile phone anyway (pro: separate battery from the toy PocketPC/Smartphone we re talking two+ weeks of battery time when using it here, and easier use and less bugs, and a reliable fallback when I tinker too much ), this is perfect for me. I m reposting this in the wlog mostly because it s an interesting technical and OSS project, and because if 1000 people want one it will get less expensive for all of us (while here shameless plug any sponsors willing to contribute some EUR so I don t ruin myself with this, in exchange for services of some kind?). I ll probably run Debian on it (unless it goes systemd), maybe in a chroot if the native OS has functionality needed that I can t simply put into packages; they say Maemo has much better power management, but considering most use will have GPS, GLONASS and backlight on, battery isn t going to last long anyway or maybe even native I ve been wanting to know what this freesmartphone stuff my m68k (Atari VM) buildd has been happily compiling, anyway and some sort of Geocaching application (ideally a cross between something online, CacheWolf and an offline OSM (with most of Europe, but uninteresting tags stripped) and possibly access to the GS Live API but nevertheless supporting TC, NC, OC, gpsgames too), and my usual mksh(1), GNU screen, jupp(1), lynx(1), ssh(1) toolchain.) Delivery is expected for mid to end of 2014, but once it s there I ll keep you informed

Next.

Previous.