Search Results: "ron"

24 January 2011

Michal Čihař: Gammu can read and write contacts from Symbian

First day of Hackweek 6 is almost over and my progress is quite good so far. The first step wat to make Gammu connect to the Series60 applet and retrieve basic information:
$ ./gammu/gammu identify
Device               : 5C:57:C8:xx:xx:xx
Manufacturer         : Nokia
Model                : E52 (RM-469)
Firmware             : 3.0
IMEI                 : 359000000000000
That was pretty easy. If you want to try it, you need to install Series60 applet into phone (manually for now, but autoinstallation is planned for future) and configure gammu to talk to it:
[gammu]
connection = blues60
device = 5C:57:C8:xx:xx:xx
Second thing I choose to implement was manipulation with contacts. There are some oddities in the applet (like it simply crashes when giving it wrong contact ID to retrieve), which I will try to fix in future, but for now I stick with existing applet and code functionality only in Gammu. And it works:
$ ./gammu/gammu getmemory ME 215
Memory ME, Location 215
Last name            : "Gammu"
First name           : "Test"
Mobile number        : "123456"
General number       : [home]"654321"
Email address 1      : "gammu-users@lists.sourceforge.net"
0 entries empty, 1 entries filled
Tomorrow I should be able to implement calendar, let's see how will it go. PS: All code is is available in our Git repository

Filed under: English Gammu Suse 0 comments Flattr this!

23 January 2011

Wouter Verhelst: PMW vs Lilypond

A few days ago, I discovered From Punched Cards To Flat Screens, the "technical memoirs" of Philip Hazel (of exim and PCRE fame). This document, by itself a very good read, mentioned something about him having written a piece of software for the Acorn RISC OS platform, starting in the early 90s, to typeset music based on a text file; very much the way in which lilypond operates. The document also mentioned that after the demise of Acorn's platform, Philip had ported that application, PMW, to the unix platform. Being interested in music, this had me curious, so I did a bit of investigation. Lilypond is an interesting piece of software, but as I've written before, I do have a few gripes with it. Once you get beyond the basics, the lilypond syntaxis is somewhat counter-intuitive in my opinion, and there are a number of things that are just wrong with it. Upon reading through the first few examples, I quickly learned that most of the major issues that I have with lilypond's syntaxis simply do not exist with PMW, and that it also has quite a number of features which lilypond in its (in the mean time) fairly long history, still does not support. And while lilypond does have a slightly higher amount of output options, that simply does not weigh against the much higher processing speed and much more sensible input syntaxis of PMW. I've made a short comparison of the two, which shows some of their differences, and contains some notes on my opinion of the two. This table may grow in the future... I've only been playing with PMW for a few days now, but I've already decided that I'm going to ditch Lilypond.

Dirk Eddelbuettel: CRANberries is now tweeting

The CRANberries service (which reports on new and updated CRAN packages for the R language and environment) is now tweeting about new packages. Simply follow @CRANberriesFeed to receive theses messages. For the technically minded, adding this to the existing 200-line program which runs all of CRANberries was very easy. CRANberries relies only on R itself and a few Unix tools like diffstat as well as the simple blosxom txt-to-html/rss 'blog compiler'. The tweeting itself is now done by this new function
tweetNewBlogEntry <- function(curPkg, curVer, reposurl)  
    ## tests reveal that pipe(), cat(), close() is easiest
    ## to send multiple messages, may need --background option
    con <- pipe("bti --config bti.conf", "w")
    cat("New CRAN package", curPkg, "with initial version", curVer, " http://goo.gl/pgljT\n", file=con)
    close(con)
 
which simply pipes the message in Greg KH's bti program (which is now a new dependency). Special thanks to its Debian maintainer Gregor Herrmann for some helpful emails; I am using the newest release 0.29 which itself needs liboauth0. Once OAuth tokens are set-up (and see here for how to do that) all we need is the three-liner above. At this point I am not too sure what to do about updated packages. One message per updated package seems too noisy. To be seen---comments or suggestions welcome.

Joachim Breitner: A Mistake in Church s Paper

For a course at the IIT Bombay on functional programing, I was preparing a presentation on Alonzo Church s theorem, based on his original 1936 paper An Unsolvable Problem of Elementary Number Theory where he first showed that the Entscheidungsproblem is not solvable. In that paper, he states a theorem (Theorem II) If a formula has a normal form [ ] any sequence of reductions of the formula must (if continued) terminate in the normal form . This seems to be wrong. Consider the lambda expression KI , where K=( xy.x), I=( x.x) and =( x.xx)( x.xx). This has a normal form I. But because reduces to , there is a non-terminating sequence KI KI , in contradiction to Church s claim. The same example contradicts his Theorem III: If a formula has a normal form, every well-formed part of it has a normal form. , which is used in the very proof of his main result, Theorem XVIII. There is no proof for Theorem II in this paper. He cites it from the then forthcoming paper Some properties of Conversion by him and J. B. Rosser. There, proofs are given (see Theorem 2 and its Corollary) but they are quite impenetrable to me. So I was searching for some correction paper, or any other discussion of this issue, but could not find any. Does any reader of this blog know more? What happened in that times when a paper had an error in some proof? What would happen now? Or is this not an error after all, but just a subtle difference between the definitions of -calculus as we know and as they introduced it? Unrelated to this question: The professor asked me to write down a summary of (my interpretation of) the importance and impact of Church s paper with regard to G del and Hilbert s program, and how that relates to B hm s theorem. In the spirit of sharing, I have uploaded my thoughts on Church and B hm, comments are welcome. Update: Christian von Essen solved the riddle in a comment to this post: Church only considers lambda abstractions as well-formed when the bound variable actually occurs (freely) in the abstracted term. This does not allow for the K combinator and thus there is no problem.

Russell Coker: Autism, Food, etc

James Purser wrote Stop Using Autism to Push Your Own Brand of Lunacy about the organisations that use Autism for their own political agenda [1]. He references the ongoing vaccine issue (which didn t even stop when Andrew Wakefield was proven to have engaged in fraudulent research) and a campaign of PETA.PETA s ClaimsPETA claims that many autistic kids improve dramatically when put on a diet free of dairy foods [2]. They also speculate about the possibility of Autism being caused by dairy.One thing that confuses the issue is the impact of stress on people with Autism Spectrum Disorders (ASDs). Whatever you are trying to do you will probably do it less effectively if under an excessive amount of stress, and the background stress levels of people on the Autism Spectrum tend to be quite high at the best of times. So add in some extra stress from a food allergy and the ability to emulate an NT can decrease a lot.There are people who make credible claims about significant behavior improvements in Autistic children after changing their diet. Eliminate foods that trigger a minor allergic response and the behavior of NT kids will improve too.I m sure that there are some Autistic kids who have a minor intolerance to lactose or casein who s behavior would improve if they had no dairy products. There are probably just as many Autistic kids who have a minor allergy to peanuts who would get the same benefit from avoiding peanut butter. It s not inconceivable that some Autistic people may be allergic to soy or wheat and find that they can cope better if their consume more milk, cheese, and steak and less vegetables!Also some people on the Autism Spectrum have a better sense of smell than NTs, and it s fairly common to be more fussy about the appearance of food. Force children to eat food that smells bad and their behavior might deteriorate.The issues of diet and Autism are more difficult than those of vaccines because while people just imagine the vaccines to have a bad affect, there are people who are proven to get a bad affect from certain foods.Some Ideas for ResearchI think that the biggest problem with PETA s research in this regard is citing a study of only 20 individuals and only testing the removal of milk. A proper study would involve more subjects (maybe 100) and separate tests of removing milk, eggs, wheat, rice, seeds, nuts, soy, fish, spices, and food colorings and additives.It would also test various methods of reducing stress, having a more quiet environment etc.PETA do deserve some credit for leaving this URL in the comments [3], it s a study that disproves the link between milk and Autism. But it s still a small sample size of 14.Why They Target AutismIn spite of psychologists moving to define Asperger Syndrome and Autism as the same thing, the people who make such claims aren t talking about people like me. They want to make claims about people who can t refute them, such as people who fit the Rain Man stereotype. The Low Functioning Autistic (LFA) people have less ability to communicate and are easier to ignore, so if you want to make some stuff up about a group of people then they are an easy target.Parents of LFA kids are often desperate and will latch on to any idea that seems to help. They can imagine that things have improved in spite of a lack of evidence and also fail to realise that children can improve rapidly just due to their natural development, 1 month is not much time for an adult but for a 4yo it s 2% of their life!Also any supposed research that includes testimony of parents should be regarded as rubbish. If you talk to any child psychologist they will tell you about how difficult it is to convince parents that their child is on the Autism Spectrum, a Google search for autism + denial also brings up plenty of hits. Parental denial doesn t stop at refusing to accept a diagnosis, it includes wanting to reject it at any later time based on any shred of evidence that may be available. So when there is a diet change or some other supposed cause followed by some apparently better behavior they want to claim that Autism is cured.I ve written a document titled Some Suggestions for Parents of Aspies [4] which is largely based on the childhood memories and adult experiences of various Aspies. My document covers many issues and few kids on the Spectrum will benefit from all of them, but I think that there is something there that can benefit every child on the Spectrum and probably a lot of geeky NT kids too. It seems to me that when dealing with Autistic kids it s a lot better to seek advice from adults on the Autism Spectrum than from random Neuro-Typical people (NTs), particularly if those NTs happen to have an agenda to push. The sensible parents are the ones who seek advice from psychologists and people like me rather than from random unqualified NTs.

22 January 2011

Matthew Palmer: Debugging Debian Installer

I've been doing a bit of work recently on the netcfg portion of Debian Installer, with the largest work so far being adding IPv6 support. Unlike most of the rest of d-i, netcfg is written in C, and as such suffers from some difficulties in the realm of debugging. Worse, the whole d-i environment is rather bereft of the usual debugging tools (strace, lsof, etc) -- but most importantly, in this case, gdb. Thanks to some hints from Julien Cristau, I was reminded of the existence of gdbserver, a small part of gdb that allows you to debug a program running on a remote machine, using a copy of gdb running on your local machine. It's designed for debugging embedded systems, I think, because all of the heavy lifting (symbol lookups, etc) happens on your local machine, not where gdbserver is running. Here are the steps I took to get a useful gdbserver session inside d-i:
  1. Get the gdb sources (apt-get source gdb) and drop into the gdbserver tree (cd gdb-x.y.z/gdb/gdbserver)
  2. Configure gdbserver to build a static binary (LDFLAGS=-static ./configure). This is necessary because you don't want to have to faff around with library dependencies inside the d-i environment. Then build gdbserver (make).
  3. Copy gdbserver over to your d-i machine/VM (I like using netcat; nc -l -p 31310 >/bin/gdbserver on the d-i machine, and nc IPADDR 31310 <gdbserver where you built gdbserver), then make it executable (chmod /bin/gdbserver)
  4. Fire up gdbserver in d-i, attaching it to the process you wish to debug: gdbserver host:31337 --attach PID
  5. Now things get a little trickier... on your local machine, you need to have an unstripped copy of the binary you're debugging. For me, this is easy, because I'm building netcfg myself (to keep the symbols, I just run DEB_BUILD_OPTIONS=noopt dpkg-buildpackage -b -rfakeroot) but I'm not sure what you'd do if you were just tracing around for fun.
  6. Once you've got your binary, just run gdb BINARY on your host machine to start gdb running.
  7. For the "even trickier" part of the show, your local machine also needs copies of all the libraries your binary is linked against. If you don't have this, when you try to debug you'll get all sorts of "wrong library or version mismatch?" errors, and your backtraces will likely break (amongst other things). Again, for me, this is easy, because I'm only running the d-i initrd, but if you're debugging something later in the installation process you might need to unpack some udebs somewhere. All I have to do is keep the initrd build tree around (which lives in the installer tree, under build/tmp/netboot/tree). Once you've got your tree of libraries, you can continue.
  8. You need to tell gdb about that tree of libraries, by running the following in gdb: set solib-absolute-prefix /path/to/library/tree
  9. Now, tell gdb to connect to your gdbserver: target remote IPADDR:31337 -- gdb should tell you "Remote debugging using IPADDR:31337", and the gdbserver should say "Remote debugging from host LOCALIP:31337".
  10. Now, you should be able to interact with the program being debugged just like any other. Enjoy!

Mark Brown: Tracing ASoC with trace events

Kernel 2.6.38 will add support for tracing ASoC using trace points. Previously all logging for ASoC had been done using printk(), meaning that changing the active logging required a kernel rebuild and that when trace was enabled the volume of trace could easily become very disruptive to other logging within the system. Trace points solve these problems by providing a framework for enabling and disabling individual traces dynamically and separate logging infrastructure designed for easy filtering and post-processing. When using trace points it s reasonable to leave them enabled all the time, making life much easier especially when debugging intermittent problems. There s a bunch of existing documentation out there for the trace subsystem, including some in the kernel under Documentation/trace, which I won t repeat here. The ASoC events are grouped together under asoc . The raw log can be viewed by looking at the file tracing/trace in debugfs. Here s a sample output from starting playback of an MP3 on a Samsung SMDK6410 reference system using WM8580, a high performance six channel CODEC with a very simple software interface:
mplayer-1673  [000]   183.920000: snd_soc_dapm_start: card=SMDK-I2S
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=Rear val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=Center+Sub val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=Front val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT3R val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT3L val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT2R val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT2L val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT1R val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT1L val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=DAC3 val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=DAC2 val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=DAC1 val=1
mplayer-1673  [000]   183.920000: snd_soc_bias_level_start: card=SMDK-I2S val=2
mplayer-1673  [000]   183.920000: snd_soc_bias_level_done: card=SMDK-I2S val=2
mplayer-1673  [000]   183.920000: snd_soc_reg_read: codec=wm8580-codec.0-001b.27 reg=32 val=1e
mplayer-1673  [000]   183.920000: snd_soc_reg_write: codec=wm8580-codec.0-001b.27 reg=32 val=2
mplayer-1673  [000]   183.920000: snd_soc_bias_level_start: card=SMDK-I2S val=3
mplayer-1673  [000]   183.920000: snd_soc_bias_level_done: card=SMDK-I2S val=3
mplayer-1673  [000]   183.920000: snd_soc_dapm_done: card=SMDK-I2S
mplayer-1673  [000]   183.920000: snd_soc_reg_read: codec=wm8580-codec.0-001b.27 reg=13 val=10
mplayer-1673  [000]   183.920000: snd_soc_reg_write: codec=wm8580-codec.0-001b.27 reg=13 val=0
Each line represents a trace event, showing the process that initiated the event, the CPU it ran on and the time it was recorded at together with event-specific details. On this system the time is reported with 10ms accuracy and the CODEC is very simple so the timing information is not terribly informative, but more accurate timers can be really helpful in analysing what s going on with more complex CODECs. The set of events available will change over time, the key ones currently present in the kernel are: The set of events traced will change as the subsystem develops and as people gain more experience with using the trace API.

Joey Hess: command composition

Traditionally, Unix has operated by piping between commands ("find grep sort uniq"). I wonder if we are experiencing a shift to command composition being a more common model? I started thinking about this when I considered the following command line:
chronic mr run git annex get
Unpacking, that uses chronic, which arranges for quiet running of mr (unless it fails, then it shows the full output), which in turn runs git in every repository it knows about, and git is asked to run the git-annex subcommand, which in turn has a "get" subcommand of its own. So, in other words, this is a cron job that syncs large files in a set of git repos, quietly. You might want to make it run niced also, which adds 3 more commands:
chronic nice ionice -c3 trickle -d100 mr run git annex get
This is the most extreme example I know of (unless you tack in a "sudo"), but there does seem to have been a gradual shift to using more composition, and more subcommands. In git in particular, two-levels of subcommands is not unknown, for example "git remote add". Subcommands, especially as implemented in git (or earlier, in arch), are kind of a special case of command composition that adds a namespace. I'm sure that a count of commands with subcommands would show it exploded in the past 10-12 years. I'll bet a count of composable commands would also show a significant increase. Anyone have other examples of long compositions of commands?

Russell Coker: Continuously Usable Testing of SE Linux

Joey has proposed a new concept of Continuously Usable Testing for Debian [1], basically testing should be usable at all times and packages that aren t usable should be dropped. But to properly achieve this goal we need continual testing of usability.The Plan For SE LinuxTo do this for SE Linux I m setting up a Xen server which will have a number of different DomUs for testing a variety of server applications. The system has 1.5G of RAM and 160G of mirrored storage. An image of a typical server will take about 4G of disk space, so we could have something like 40 images online and ready for testing. I have already setup Squid on another system on the same LAN to cache Debian packages, so running apt-get dist-upgrade on a number of DomUs won t take that long. With 256M for a typical server image I could have 5 images running at the same time. If the hardware isn t enough then I can expand it, I hope to get some donations of DDR-266 or DDR-333 RAM (or maybe DDR-400) to upgrade the system to 4G of RAM, I can add more hard drives, and I could even install more servers.I want to have testing be very usable for SE Linux throughout the development cycle so that I don t have to rush things before release.At this stage I m not sure whether to track Unstable or Testing for this. I guess it might be best to track Testing most of the time and only track Unstable for daemons that are changing rapidly. It might get boring testing every version that comes through Unstable, but if people want to do this then I won t stop them.Setting up the TestsWhat I need are interested people who want to install server configurations for testing. If you have some favorite combination of daemons that you want tested for SE Linux support (even if it s daemons that have no current policy) then I can give you root access to a DomU to develop test cases. Ideally there would be automated tests used for most things for example testing a mail server by using swaks to deliver mail and a POP or IMAP client script to retrieve it. But some things can t be tested properly without human intervention.For the automated tests I want to script the creation of DomUs, upgrading the packages in the DomU, testing it, and then shutting down the DomU if it all works. If at any time the tests fail (or the upgrade fails) then it would wait for human intervention. That would be me fixing SE Linux problems and other people fixing the application problems. I think that discovering SE Linux issues will only be a part of this project.For the manual tests I will grant access to create and destroy the DomUs in question to people who can run the tests.I m thinking of having a couple of DomUs running permanently for things which are test candidates but also useful for the project, such as a MediaWiki instance. It really depends on the interest of people who might use such things.Also I m thinking of setting up some Ubuntu DomUs too, I probably should join Ubuntu and get involved with SE Linux there.Sharing the ImagesI have a web server in Germany with almost unlimited bandwidth and storage. For every image that is created I want to upload a version to the server in Germany to allow anyone in the world to test it. There are lots of possible ways of using this for software development. For example if you had a patched version of Apache that you wanted to test then you could download every image that had Apache installed and test that they all work. That would be easier than configuring Apache in different ways and also possibly provide better coverage.Also if someone can t figure out how to configure a daemon correctly then downloading a Xen image of a working configuration could be helpful (if a little bandwidth intensive). Note that deploying such an image in production would be a really bad idea, among other things there are lots of places where passwords are stored and you wouldn t want to risk missing one.I also plan to share the scripts used in running the Dom0 and anything else that seems useful along the way.What We NeedThe main thing we need is volunteers to configure virtual machines with their favorite daemons. Note that I don t plan to have only one daemon per DomU, if we can get multiple daemons running that don t conflict (EG file server and mail server) or multiple daemons that can interact (EG database server and a mail server or anything else that can be a database client) running on the same system then that s a good thing. So there will be some degree of interaction with other people.I m happy to accept contributions from people who aren t interested in SE Linux. But SE Linux will run on all DomUs.Finally I also need more RAM for a HP D530S, DU875PA (that s a Celeron 2.4GHz). I ll accept donations of complete systems too once my HP system gets full, preferably relatively low power systems as they will be housed in a location that s not as well ventilated as I would like (cost and availability of IP addresses were the main criteria). A laptop with a broken screen would be great!The system won t go live until Monday, but I think that probably people won t be ready to do much work with less than two days notice anyway.

Matthew Garrett: Open embedded GPUs

Luke: It's great to see open code for the 6410, but I'm pretty sure that that chip uses Samsung's own GPU design rather than being anything PowerVR based. What makes you think it's an SGX? The listing here implies that it's an unrelated part.

21 January 2011

Rapha&#235;l Hertzog: People behind Debian: Michael Vogt, synaptic and APT developer

Michael and his daughter Marie

Michael has been around for more than 10 years and has always contributed to the APT software family. He s the author of the first real graphical interface to APT synaptic. Since then he created software-center as part of his work for Ubuntu. Being the most experienced APT developer, he s naturally the coordinator of the APT team. Check out what he has to say about APT s possible evolutions. My questions are in bold, the rest is by Michael. Who are you? My name is Michael Vogt, I m married and have two little daughters. We live in Germany (near to Trier) and I work for Canonical as a software developer. I joined Debian as a developer in early 2000 and started to contribute to Ubuntu in 2004. What s your biggest achievement within Debian or Ubuntu? I can not decide on a single one so I will just be a bit verbose. From the very beginning I was interested in improving the package manager experience and the UI on top for our users. I m proud of the work I did with synaptic. It was one of the earliest UIs on top of apt. Because of my work on synaptic I got into apt development as well and fixed bugs there and added new features. I still do most of the uploads here, but nowadays David Kalnischkies is the most active developer. I also wrote a bunch of tools like gdebi, update-notifier, update-manager, unattended-upgrade and software-properties to make the update/install situation for the user easier to deal with. Most of the tools are written in python so I added a lot of improvements to python-apt along the way, including the initial high level apt interface and a bunch of missing low-level apt_pkg features. Julian Andres Klode made a big push in this area recently and thanks to his effort the bindings are fully complete now and have good documentation. My most recent project is software-center. Its aim is to provide a UI strongly targeted for end-users. The goal of this project is to make finding and installing software easy and beautiful. We have a fantastic collection of software to offer and software-center tries to present it well (including screenshots, instant search results and soon ratings&reviews). This builds on great foundations like aptdaemon by Sebastian Heinlein, screenshots.debian.net by Christoph Haas, ddtp.debian.org by Michael Bramer, apt-xapian-index by Enrico Zini and many others (this is what I love about free software, it usually adds , rarely takes away ). What are your plans for Debian Wheezy? For apt I would love to see a more plugable architecture for the acquire system. It would be nice to be able to make apt-get update (and the frontends that use this from libapt) be able to download additional data (like debtags or additional index file that contains more end-user targeted information). I also want to add some scripts so that apt (optionally) creates btrfs snapshots on upgrade and provide some easy way to rollback in case of problems. There is also some interesting work going on around making the apt problem resolver a more plugable part. This way we should be able to do much faster development. software-center will get ratings&reviews in the upstream branch, I really hope we can get that into Wheezy. If you could spend all your time on Debian, what would you work on? In that case I would start with a refactor of apt to make it more robust about ABI breaks. It would be possible to move much faster once this problem is solved (its not even hard, it just need to be done). Then I would add a more complete testsuite. Another important problem to tackle is to make maintainer scripts more declarative. I triaged a lot of upgrade bug reports (mostly in ubuntu though) and a lot of them are caused by maintainer script failures. Worse is that depending on the error its really hard for the user to solve the problem. There is also a lot of code duplication. Having a central place that contains well tested code to do these jobs would be more robust. Triggers help us a lot here already, but I think there is still more room for improvement. What s the biggest problem of Debian? That s a hard question :) I mostly like Debian the way it is. What frustrated me in the past were flamewars that could have been avoided. To me being respectful to each other is important, I don t like flames and insults because I like solving problems and fighting like this rarely helps that. The other attitude I don t like is to blame people and complain instead of trying to help and be positive (the difference between it sucks because it does not support $foo instead of it would be so helpful if we had $foo because it enables me to let me do $bar ). For a long time, I had the feeling you were mostly alone working on APT and were just ensuring that it keeps working. Did you also had this feeling and are things better nowadays ? I felt a bit alone sometimes :) That being said, there were great people like Eugene V. Lyubimkin and Otavio Salvador during my time who did do a lot of good work (especially at release crunch times) and helped me with the maintenance (but got interested in other area than apt later). And now we have the unstoppable David Kalnischkies and Julian Andres Klode. Apt is too big for a single person, so I m very happy that especially David is doing superb work on the day-to-day tasks and fixes (plus big project like multiarch and the important but not very thankful testsuite work). We talk about apt stuff almost daily, doing code reviews and discuss bugs. This makes the development process much more fun and healthy. Julian Andres Klode is doing interesting work around making the resolver more plugable and Christian Perrier is as tireless as always when it comes to the translations merging. I did a quick grep over the bzr log output (including all branch merges) and count around ~4300 total commits (including all revisions of branches merged). Of that there ~950 commits from me plus an additional ~500 merges. It was more than just ensuring that it keeps working but I can see where this feeling comes from as I was never very verbose. Apt also was never my only project, I am involved in other upstream work like synaptic or update-manager or python-apt etc). This naturally reduced the time available to hack on apt and spend time doing the important day-to-day bug triage, response to mailing list messages etc. One the python-apt side Julian Andres Klode did great work to improve the code and the documentation. It s a really nice interface and if you need to do anything related to packages and love python I encourage you to try it. Its as simple as:
import apt
cache = apt.Cache()
cache["update-manager"].mark_install()
cache.commit()
Of course you can do much more with it (update-manager, software-center and lots of more tools use it). With pydoc apt you can get a good overview. The apt team always welcomes contributors. We have a mailing list and a irc channel and it s a great opportunity to solve real world problems. It does not matter if you want to help triage bugs or write documentation or write code, we welcome all contributors. You re also an Ubuntu developer employed by Canonical. Are you satisfied with the level of cooperation between both projects? What can we do to get Ubuntu to package new applications developed by Canonical directly in Debian? Again a tricky question :) When it comes to cooperation there is always room for improvement. I think (with my Canonical hat on) we do a lot better than we did in the past. And it s great to see the current DPL coming to Ubuntu events and talking about ways to improve the collaboration. One area that I feel that Debian would benefit is to be more positive about NMUs and shared source repositories (collab-maint and LowThresholdNmu are good steps here). The lower the cost is to push a patch/fix (e.g. via direct commit or upload) the more there will be. When it comes to getting packages into Debian I think the best solution is to have a person in Debian as a point of contact to help with that. Usually the amount of work is pretty small as the software will have a debian/* dir already with useful stuff in it. But it helps me a lot to have someone doing the Debian uploads, responding to the bugmail etc (even if the bugmail is just forwarded as upstream bugreports :) IMO it is a great opportunity especially for new packagers as they will not have to do a lot of packaging work to get those apps into Debian. This model works very well for me for e.g. gdebi (where Luca Falavigna is really helpful on the Debian side). Is there someone in Debian that you admire for his contributions? There are many people I admire. Probably too many to mention them all. I always find it hard to single out individual people because the project as a whole can be so proud of their achievements. The first name that comes to my mind is Jason Gunthorpe (the original apt author) who I ve never met. The next is Daniel Burrows who I met and was inspired by. David Kalnischkies is doing great work on apt. From contributing his first (small) patch to being able to virtually fix any problem and adding big features like multiarch support in about a year. Sebastian Heinlein for aptdaemon. Christian Perrier has always be one of my heroes because he cares so much about i18n. Christoph Haas for screenshots.debian.net, Michael Bramer for his work on debian translated package descriptions.
Thank you to Michael for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

4 comments Liked this article? Click here. My blog is Flattr-enabled.

Cyril Brulebois: Debian XSF News #2

Time for a second Debian XSF News issue! I ll skip stuff related to the new katamari since I wrote about that already.

20 January 2011

Simon Josefsson: Introducing the OATH Toolkit

I am happy to announce a project that I have been working quietly on for about a year: the OATH Toolkit. OATH stands for Open AuTHentication and is an organization that specify standards around authentication. That is a pretty broad focus, but practically it has translated into work on specifying standards around deploying and using electronic token based user authentication such as the YubiKey. YubiKey OATH s most visible specification has been the HOTP algorithm which is a way to generate event-based one-time passwords from a shared secret using HMAC-SHA1. HOTP has been published through the IETF as RFC 4226. Built on top of HOTP is the time-based variant called TOTP, which requires a clock in the token. OATH do some other work too, like specifying a data format for transferring the token configuration data (e.g., serial number and shared secret) called PSKC.

The aim of my project OATH Toolkit is to provide an implementation of various OATH related technologies. I m intentionally leaving it open ended because you never know what they may specify that I find interesting. However, the primary goal has been to focus on HOTP and TOTP. Throughout 2010, the project was called HOTP Toolkit but that name made it difficult to support TOTP in a non-confusing way. During the last month, after discussion with Daniel Pocock on Dynalogin which is a potential consumer of my package, I took the time to create a fork of the HOTP Toolkit and the OATH Toolkit was born. Now what does the OATH Toolkit actually do? Primarily it provides a library called liboath that implements HOTP and TOTP. Liboath is a relatively small library, and my goal is to keep it well documented and of high quality. There is GTK-DOC generated API documentation. Of course there is a command line tool to go with it, called oathtool which makes working with HOTP/TOTP from the command line easier. It can generate and validate one-time passwords. Let s say you want to generate the first four OTP based on the dummy key 1234.
jas@latte:~$ oathtool -w4 1234
376439
299783
041392
819202
158134
jas@latte:~$
By default the tool is using HOTP, but you can switch it into TOTP mode with the totp parameter. The output OTP will now depend on the current time on your machine, unless you specify the time manually using the now parameter.
jas@latte:~$ oathtool now= 2011-01-20 15:46 UTC totp 1234
527971
jas@latte:~$
The tool can do more, check the oathtool man page for all the details. The final component of the OATH Toolkit is a PAM module pam_oath. With it, you can login to your machine using an OTP and optionally a password. Right now the user and password management is simplistic, but that should improve over time. To setup single-factor authentication for su you would create a file containing the user information and HOTP key as /etc/users.oath like this:
HOTP root 1234
Then configure PAM to use the pam_oath module like this in /etc/pam.d/su:
auth requisite pam_oath.so debug usersfile=/etc/users.oath window=20
The user file will be rewritten every time you su to hold the current state. There is a README for the PAM-module with more documentation. That s it for an intro! From the OATH Toolkit webpage we link to binary packages for Debian and Ubuntu so please try the OATH Toolkit yourself and provide feedback to the oath-toolkit-help mailing list.

19 January 2011

Gunnar Wolf: Mandate for what?

I lived in Israel between 1994 and 1996 This was just after the signature of the Oslo agreements, the Rabin-Arafat signature. 1994-95 was a beautiful time to be in Israel; there was the excitement of a prospect of real peace with the Palestinians, something that had been just seen as unattainable for the past 50 years. But we also saw the buildup of a huge tension, of real hatred. And the dream of the end of the conflict, of two people living in peace side by side was heavily shattered on November 4 1995, when prime minister Rabin was assassinated. But for the sake of where I'm trying to get to (this post is meant to be about current Mexico, not of Israel 15 years ago), lets go a couple of months back. The Israeli society is a deeply democratic one (although I understand many people feel it is losing that trait). Lots of political propaganda of all sorts, many of it very creative, are posted on the streets And not by the government or by political parties, but by citizen movements. I tried to follow it... I remember a particular sticker that I started seeing very often: Rabin has no mandate over the Golan . At first, I understod that the Golan residents rejected Rabin's authority (but, why was it posted all over the country? And what specifically was it about?), until a friend explained to me that it meant that it rather meant that the right wing all over the country felt that Rabin had no right to push for negotiations involving in any way the Golan heights return to Syria because that was not his electoral platform. So, fast-forward in time, and 12500Km to the west. If you have followed my ramblings for the past ~five years, you are aware of what I think about Mexico's government It is an illegitimate government, which got the power thanks to a dirty and illegal (as per Mexico's legal system) campaign and to voting fraud. And, according to what I have read, around 30% of the population still believe firmly the elections were stolen. But even if they were legal, valid and legitimate What has happened since 2006 should not be seen as valid. In Mexico, we don't have such a democratic culture. The society is not used to demanding anything from the politicians. We often see, as an example, that when talking about political organs, there is a divide everybody understands although makes no sense: We talk about citizens as people unaffiliated to political parties (v.gr. the 1996 Federal Electoral Institute was very well regarded as it was "el IFE ciudadano", and has since then been overtaken by the parties). Why are the parties' members not seen as citizens? Well, maybe because of the same distortion field that makes us believe our de-facto president when he stated that, from the (then) over 28000 people killed in the war campaign against crime (he oficially declared he never used the term war... Of course, he was proved wrong), only ~10% were civilians. In my book, that would mean that ~25000 soldiers have been killed But no, he means mostly drug trafficants. So, as they are not innocent bystanders, they somehow stopped being civilians? But again, I'm straying off the topic. Let us assume for a minute that the 2006 elections were clean legal, and that the campaigns were not diffamatory, and all that. And tha we had a modern government, with the stated separation between the three powers and all that. By today, the Legislative power would have surely called for the government's disolution. Why? Again, we Mexicans are not used to what a democratic system means. We can see over and over examples of people who have been voted to power and forgot/betrayed everything they campaigned for. And we are used to that being... Normal. But, who did we vote for? Given the suposition of this last paragraphs (clean elections led to our current government), did the majority of the population vote for Felipe Calder n, the person? Or did it vote for Partido Acci n Nacional, the abstract entity organized as a party? No. The vote was (should have been) for the political campaign, for the things he promised he would work for, the priorities in the direction he would give the nation. Not too long ago, we took pride of living in a peaceful country. Yes, everybody feared the crime rate in the streets, everybody knew big cities such as the one I live in (and love living in) have deep problems. But the country had good relations and good respect from the international players, and it was mostly safe. And, I don't think any of the few readers that have read this far (regardless of where they live or how much they know me) has that opinion of Mexico anymore. What is now heard inside and outside is that we have a huge organized crime (mostly due to drug traffic), and that it's taking over the country. Why did this happen? The fact is, Calder n lacked legitimacy when he reached the presidency in December 2006. And although his campaign was centered on pushing work generation (his slogan was El presidente del empleo, The president of the jobs), lifting restrictions to entreprises, opening borders, getting more money in our economy... He knew he had to please the people that would protect him from any (then, probable) insurrection: Military and police. Our army has always been more a joke than an army... But they still have their guns. So, contrary to everything he promised, the first things he signed was a salary increase to all order-keeping forces in the country, and an all-out attack against drug traffic cartels. He had to make the order-keeping forces protagonist, useful and loyal. Now, fast-forward again to 2011. The country is living the worst violence since the end of the 1910-1920 revolution. Over 30,000 people have been killed. Economy has fallen as it had not for many years (this was the worst-performing country in America in 2009). Work generation was said to be high, but then corrected to indicate it was mediocre (just about the baseline of population growth). So, the balance so far of four years of... Government? Economic disaster. Social disaster. Promises unachieved. Starting actions he had no mandate to do. Felipe Calder n has no mandate to get us in this war. Even if he had been democratically elected, he would have to step down.

Russell Coker: Free Amazon EC2 Servers

Amazon is providing free EC2 access for new customers (who have never been customers before) for one year [1]. It is 750 hours per month (enough to run non-stop for an entire month) of access to a Linux micro instance which has 613M of RAM and the ability to burst to two ECUs of compute power. The main EC2 web page [2] describes an ECU as the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor and they also describe a single core of a modern CPU as having 3.5 ECUs. So a micro instance could burst to half the CPU power of a single core. The DomU that runs my blog (as well as some web sites for friends) has been averaging less than 1% use of a CPU core over the last few months, so the CPU capacity of a micro instance should be more than adequate for most things that run on the net.The free offering only provides 15G of free data transmission and 15G of free data reception per month. For my blog server that would be more than adequate as it has sent 24.5G and received 14G over the last 75 days.The Cost of Disk IOThe offer requires that you sign up with a credit card so if you use more than the free capacity then you have to start paying. It seems that the main issue in this regard is disk IO.The only storage that is available for a micro instance is the Elastic Block Store (EBS) [3]. The main way that EC2 operates is that when you create a new virtual machine it copies the data from an existing image so you can easily create dozens or hundreds of virtual machines with local disk performance and the data is removed when the instance is shut down. EBS is essentially SAN based storage, it s persistent and operates like a regular disk.The pricing for EBS is $0.10 per allocated GB per month plus $0.10 per million IOs. Unfortunately they don t define what an IO is apart from mentioning that you can use IOSTAT to measure them. According to iostat the server running my blog is doing 0.99 tps, so that means in a 30 day month I would expect 30*24*3600*0.99 = 2.56M transactions. Iostat also tells me that my blog server has read 62075330 blocks and written 91683344 blocks over the last 75 days 16 hours of uptime, that means it would do about (62075330+91683344)/75.66*30 = 61M block transfers in a 30 day month. So if I was to run my blog server on EC2 I could be spending either $0.15 or $6.00 per month on disk IO depending on how they count it (or maybe something in between, something larger than a 512 byte block but smaller than a transaction as reported by iostat could be used). Given that the last time I checked the prices one could rent a DomU for less than $6 per month [4] the difference in possible ways of measuring IOs is very significant!The MySQL server that is the backend to my blog (as well as a few other things) seems to be averaging about 3 writes per second (and no reads during operation because the databases are small). So it might be another 5 million IOs per month for the database.It s unfortunate that Amazon haven t clearly specified what they mean when measuring IO for billing purposes. Some aspects of measurement such as whether the bills for bandwidth include Ethernet headers can be ignored as a 26 byte Ethernet header won t make much difference to the bill when the average packet size is around 400 bytes or more (from ifconfig output it seems that my blog server sends packets of an average size of 459 bytes and receives packets of an average size of 1250 bytes). But the methods of measuring disk IO could give a factor of 20 difference in the bill.Optimising for EC2If I was going to put my blog on EC2 then I would start by configuring Apache to log to a fifo and then write a daemon that stores the log data and allows my home server to poll it and get the log data. As the filesystem is already mounted with noatime it seems that writing logs is the cause of all the disk writes so if they were stored in RAM (which shouldn t be a problem with 4M of logs per day) then all those writes could be avoided. Another possible solution to this would be to make /var/log be a tmpfs and then rsync the files periodically to my home server. I don t really need to have all the logs remain on the server I just need them to remain somewhere.Amazon also offers 100,000 messages on their Simple Queue Service (SQS) for free [5]. The messages can be up to 8K in size and are stored for up to 4 days. So it seems possible to put Apache logs into SQS messages in bundles of less than 8K and then get them out later for transfer to a server outside EC2.If I was able to get my disk writes to almost zero then there s a good chance that I could get into the free zone for one year.ConclusionWould I use this service? If I was looking for new hosting for my blog then I would seriously consider it. EC2 is quite fast and well connected and depending on how they work out the billing for disk IO I could probably keep the cost close to zero.EC2 is a different way of running things so you can t just have a virtual server running and expect it to automatically restart if it goes down for any reason (a standard feature of virtual hosting companies). Amazon does have a range of tools for managing EC2 instances and they all seem to be available in the free trial. So after spending the time to learn those tools the result should be good.I think that there are two groups of people who could benefit from using this. One is hobbyists, this is a great way to learn some skills related to high-end server stuff and EC2 experience should look good on a CV. The other is companies who want to use EC2 anyway and who will just save some money that they would otherwise pay. I ve seen someone recommend the free offering from EC2 for a company that needed a small server, I think that isn t a good option as a company that only wants a single small server will be better off paying something between $5 and $20 per month for a DomU from one of the virtual hosting providers.After a year you have to pay regular prices. A micro instance costs $0.02 per hour which is $14.40 per month, SQS costs $0.10 per month for sending up to 1G of data in and at $0.01 per 10,000 SQS requests would costs $0.03 (the 4M of log data I generate per day would be 1000 requests to write and read it which would be 30,000 requests per month), the EBS for MySQL would cost $0.10 for 1G of storage and maybe $0.50 for IOs. That means $15.13 before counting bandwidth.My blog server averages just under 10G of transmitted data per month, the first Gig is free so that would cost $0.15 for each subsequent Gig which is $1.35 per month. It receives just under 6G per month which at $0.10 per gig would be $0.60. So including data transfer it would be about $17.08 per month.This is a lot more expensive than some of the cheaper virtual server offerings but admittedly the cheaper virtual offerings don t have as much RAM. Also with a blog instance running on EC2 I could easily configure it so that I could create some big instances that use the same MySQL database if a lot of extra traffic suddenly started arriving. A micro instance running MySQL on it s own could cope with a heap of load a lot more easily than the PHP code for my blog. So using bigger servers to run the PHP code while running MySQL on the same server would be a good option particularly if the bigger servers use caching.Finally if you want to run an EC2 instance for a year then you can get a reserved instance, you pay $54 per annum and the cost drops to $0.007 per hour instead of $0.02 per hour. Using a reserved instance for my blog would give a cost of $54+365*24*0.007+12*(0.10+0.03+0.10+0.50) or about $124.08 per annum. $10 per month isn t too bad. So if I migrated my blog to EC2 then I would probably keep it there after the free period expired. The ability to expand rapidly when necessary is worth paying extra. Of course I am making some assumptions such as that the performance of a micro instance doesn t totally suck as Amazon don t specify what bursting to 2ECU really means it could have some performance problems.Note that all prices in this post are in US Dollars.

18 January 2011

Russell Coker: Conferences and People on the Autism Spectrum

There have been some recent discussions about issues related to the treatment of women at Free Software conferences, I ve written posts about Aspie Social Skills and Free Software [1] and Empathy, Autism, and Geeks [2] about this. But of course discussion continued on how Aspies supposedly cause problems that no-one seems to be noticing.Lisa of Chaotic Idealism wrote an interesting post about the NT social bias [3]. In summary NTs seem to think that Autism Spectrum Disorders are only about socialisation, not realising that for many people on the Spectrum it s sensory issues that are their main problem. Social problems are exacerbated by sensory issues and other causes of stress, so I think it s worth considering ways in which conferences can be planned to be less stressful for people on the Autism Spectrum, people who have SPD [4] that isn t associated with an ASD, and NTs who just get annoyed by loud noises etc.What Autism Conferences DoAutscape is one of only two conferences for people on the Autism Spectrum and the only one which clearly documents how they plan their conference [5]. The first noteworthy thing that they do is have badge colors to indicate what level of social interaction is desired by each delegate, I don t think that this is relevant to Free Software conferences as people who don t have sufficient social skills to suit at least a green badge probably won t be attending. But I think that when attending a conference about Free Software or any other equally geeky topic (if there is an equally geeky topic) it s worth keeping in mind the fact that there are probably a lot of people who would like to talk to you but lack the social skills to start a conversation.
No initiationRedPlease do not initiate any interaction with me.
Prior PermissionYellowPlease do not initiate unless I have already given you permission to approach me on a yellow badge.
Please initiateGreenI would like to socialise, but I have difficulty initiating. Please initiate with me.
NeutralWhite (or no badge)I am able to regulate my own interaction.
The next thing that they document is a black circle badge which indicates that the wearer shouldn t be photographed. Prior to reading that web site I wasn t aware of this being an Autistic issue, I was only aware of it being an issue for women who don t want zoomed-in pictures of themselves appearing on guys web sites (even pictures that aren t up-skirt or down-blouse can be unwelcome). A conference policy that prohibited photographs that zoom in on one person without that person s consent (or parental consent in the case of minors) would probably be a good idea.Another thing about badges is that it s a really good idea to have the delegate s name on both sides of the badge if the badge is attached in a way that permits it to turn. People on the Autism Spectrum tend to have some difficulty in recognising people and in remembering names. I find it inconvenient when someone expects me to recognise them but has their badge turned around so I can t see their name. Some people get really unhappy if they think that someone doesn t recognise them.Autscape has long breaks between activities and a leisure session each afternoon to allow delegates to recover from the stress of dealing with people. It seems to me that computer conferences in some cases could do with longer breaks between sessions. I find that a lot of the benefit of a conference is in what happens outside sessions and the standard practice of publishing videos of presentations makes personal meetings a more important part of the conference.Autscape has designated quiet spaces. I think that for computer conferences which have hack-labs (which seems to be most Free Software conferences nowadays even if they aren t labelled as such) it would be good to have some lab areas designated as quiet zones. I think that it is a really good thing to meet people you ve only known by email and then play some LAN based games against them and that this should be encouraged as part of a conference, but having that sort of thing separated from people who want to do some quiet coding is a really good idea. Whether people want to do quiet coding, read email, or just escape from the stress of a conference a quiet hack-lab would be a good place for it.An issue that s related to quiet spaces is the ability to escape from social situations. One of my pet hates is corporate meetings on boats, when the boat is in motion there is no escape. Corporate meetings that are only accessible by coach are also bad. Fortunately most conferences aren t like that.The Autscape web site states that they prohibit people from wearing perfume or aftershave to help people who are sensitive to smell. My observation of Free Software conferences is that encouraging everyone to have a shower every morning would be a good idea as there are stronger smells from unwashed people than from perfume.One thing that s interesting about the Autscape web site is that they have different color schemes available and have tested it in multiple web browsers including Safari and Konqueror (which apparently don t work so well). It s interesting to note that they test with such a variety of browsers including free software ones most corporations don t do that.Sensory Processing IssuesThe main SPD issues related to conferences seem to be noise and light related.For social events one difficulty that most people on the Spectrum seem to face is in listening to one person in a noisy crowded room. As well as that sudden noises and loud background noises can be very stressful. While it s sometimes impossible to avoid crowds (which can be another problem) it is often possible to select venues that have less noise problems. A venue with carpet on the floor and soft walls (anything other than brick or concrete) will be a lot quieter than one with hard surfaces that reflect noise. It seems that a quiet venue will benefit NTs as well, there seems to be a strong correlation between the price of food at restaurants and the amount of sound absorbing material on the floor and walls people who can afford a good dining experience seem to want it quiet.In terms of visual issues the relevant problems seem to be related to sudden transitions and lecture halls that are extremely dark. The only thing that can be done by conference organisers is to seek to have the lights in the lecture halls as bright as possible without preventing the viewing of the projector screen, this is usually done anyway.For social events the whole binge drinking at crowded bars thing doesn t work too well due to noise, poor lighting, crowds, and the smell of vomit. But there s no shortage of reasons to discourage binge drinking at conferences.FoodPsychology Today has a good article about preferences for food and picky eaters [6]. Lots of people on the Autism Spectrum have similar issues. The thing to do when arranging a conference meal is to have things neat and without needless combinations. Think about making food look more like what you would expect to see in a Japanese restaurant and less like Paella. Also having some very plain food on offer is a good thing, I think that bread-rolls for dinner and ice-cream for desert makes a viable meal. But any dish with a word like hash or mixed in it s name isn t a good option. Finally some combinations are really bad, I always find rare steak and potatoes on the same plate to be rather disgusting potato that s blood-stained doesn t appeal. The majority of conference meals satisfy these criteria.Autscape provides options of eating outside the main dining hall for people who can t tolerate the noise. For a Free Software conference it might be a better idea to provide seating outside the main area for people who are going to make noise. I don t mind people who want to get really drunk at the conference dinner, but I would prefer them to be in a separate room. Also when assisting a drunk friend to leave the venue before they cause more problems it would be convenient if there was a good place to take them to. The one occasion when I had to strongly encourage a friend to leave a conference dinner to reduce his embarrassment the next day (and reduce the annoyance for everyone else) there was no good place to take him so he just came back!ConclusionI don t expect that anyone will make any significant changes to conference plans after reading this post. I have tried to focus on things that would benefit NTs as well, but there are practical issues that get in the way such as the cost and availability of venues that are large enough. But there are some smaller ideas that can be implemented with relative ease.I hope that the people who are making claims about Autism and conference behavior will refer to this in future. If you think that there is a problem with the way people on the Spectrum act at conferences then the solution will more likely involve the suggestions I make here than anything else.

John Goerzen: Backing Up to the Cloud

I m recently taking some big-picture looks at how we do things, and one thing that I think could be useful would be for us to back up a limited set of data to an offsite location states away. Prices are cheap enough for this to make it useful. Services such as Amazon S3 and Rackspace Cloud Files (I ve heard particularly good things about that one) seem to be perfect for this. I m not quite finding software that does what I want, though. Here are my general criteria:
  1. Storage fees of $0.15 per gigabyte-month or less
  2. Free or cheap ($0.20 per gigabyte or less) bandwidth fees
  3. rsync-like protocol to avoid having to re-send those 20GB files that have 20MB of changes in their entirety every night
  4. Open Source and cross-platform (Linux, Windows, Mac, Solaris ideally; Linux and Windows at a minimum)
  5. Compression and encryption
  6. Easy way to restore the entire backup set or individual files
  7. Versatile include/exclude rules
  8. Must be runnable from scripts, cron, etc. without a GUI
  9. Nice to have: block or file-level de-duplication
  10. Nice to have: support for accurately backing up POSIX (user, group, permission bits, symlink, hard links, sparse files) and Windows filesystem attributes
  11. Nice to have: a point-and-click interface for the non-Unix folks to use to restore Windows files and routine restore requests
So far, here s what I ve found. I should note that not a single one of these solutions appears to handle hard links or sparse files correctly, meaning I can t rely on them for complete system-level backups. That doesn t mean they re useless I could still use them to back up critical user data just less useful. Of the Free Software solutions, Duplicity is a leading contender. It has built-in support for Amazon S3 and Rackspace Cloud Files storage. It uses rdiff, which is a standalone implementation of the rsync binary delta algorithm. So you send up a full backup, then binary deltas from that for incrementals. That makes it bandwidth-efficient for incremental backups, and storage-efficient. However, periodic full backups will have to be run, which will make it less bandwidth-efficient. (Perhaps not incredibly *often*, but they will still be needed.) Duplicity doesn t offer block-level de-deuplication or a GUI for the point-and-click folks. But it DOES offer the most Unixy approach and feels like a decent match for the task overall. The other service relying on Free Software is rsync.net, which supports rsync, sftp, scp, etc. protocols directly. That would be great, as it could preserve hard links and be compatible with any number of rsync-based backup systems. The downside is that it s expensive really expensive. Their cheapest rate is $0.32 per GB-month and that s only realized if you store more than 2TB with them. The base rate is $0.80 per GB-month. They promise premium support and such, but I just don t think I can justify that for what is, essentially, secondary backup. On the non-Open Source side, there s JungleDisk, which has a Server Edition that looks like a good fit. The files are stored on either S3 or Rackspace, and it seems to be a very slick and full-featured solution. The client, however, is proprietary though it does seem to offer a non-GUI command-line interface. They claim to offer block-level de-duplication which could be very nice. The other nice thing is that the server management is centralized, which presumably lets you easily automate things like not running more than one backup at a time in order to not monopolize an Internet link. This can, of course, be managed with something like duplicity with appropriate ssh jobs kicked off from appropriate places, but it would be easier if the agent just handled it automatically. What are people s thoughts about this sort of thing?

Benjamin Drung: Poll: How to call the library?

Dear lazy web, I am writing a library that implements the Units Policy of Ubuntu. This library can be used to format sizes (of files, disks, memory, etc.) for displaying them and the other way around. For example, the size 12345 bytes can be formatted to 12.3 kB, 12.1 KiB, or 12.1 KB depending on the configuration. An other function will give you the possibility to convert 12.3 kB, 12.1 KiB, or 12.1 KB back to 12345 bytes. The preferred binary unit can be configured by an text file (per system and per user) and by an environment variable. The library is written in C and doesn t have any dependency. Therefore it can be used by any program on any desktop environment. Binding to other languages like C++, Python, Java are planned. Before creating a project for this library, I need a name for the library. The current prototype is called libdisplayunits, but this name reflect only the half of the project. It does not indicate that the library can be used for inputting sizes. What do you think? What is the best name for the library? Please poll and/or comment below.

<script charset="utf-8" language="javascript" src="http://static.polldaddy.com/p/3972978.js" type="text/javascript"></script> <noscript> View This Poll
survey software </noscript>

17 January 2011

Julien Viard de Galbert: Triaging bugs is not that hard !

There is a big thread on -devel that started about Forwarding bugs upstream, the idea is that it should be the maintainer s job. While it s true in general, some understaffed teams are so flooded by bugs that they just can t do it.
I ve seen more than once the X Strike Force asking a user to report upstream (pointing to the relevant BTS) and come back with the tracking URL to be attached to the bug. This was mostly for driver issue if I remember well.
And most users did it, so thanks! Then the thread evolved in lots of directions, I m not following all of it, but there was the question of the bug triaging and as I m doing it for the X packages, I want to share about my experience here. The available triaging docs The debian wiki already have a general Bug Triage page.
The thread pointed to the KDE Bug triaging page.
The X Strike Force has some pages, one on the wiki and a new one more specific to bug triaging on the alioth project s doc.
And when it comes to closing bugs with version information, there is a good read on the wiki: Bugs Version Tracking.
Finally don t hesitate to reopen the BTS documentation whenever you need to write to control@b.d.o ! Now if all those docs don t help you, then try to improve the docs as a first step ! ;) Talk with the maintainer(s) Well I did met Julien Cristau and Cyril Brulebois in person during mini-DebConf Paris/2010 that certainly helped. But we exchanged several mails, or met on IRC channel #debian-x.
But the triaging rules of the X Strike Force (linked above) were pretty clear anyway. My current process This process has evolved over time with the response of several contributors either to help or to fix to my mistakes, thanks!
  1. List bug from UDD: XSF unstable bugs sorted by date.
  2. Pick an old bug (once you answer and UDD runs, the bug moves down the list, so there are always old bugs on the top).
  3. Read the report, don t be afraid by the subject, it might be unrelated, the bug might be merged with other bugs, then reading the other bugs can help.
  4. If relevant try to reproduce it. (most bugs in X are hardware related or specific to one setup, so it s not that often that I have something to test).
  5. Decide what to do, most of the time, I will ping the submitter.
  6. I m a mutt user, and to my help the bts command as a way to open the bug mbox in mutt, that makes it really easy to answer the bug.
    So I launch bts -m show $n where $n is the bug number, and I can then answer the bug from mutt.
    The triaging procedure suggest to ping -submitter, but using mutt it s easier to just reply and add a -quiet to the bug number (this avoid spamming the debian-x with all my pings).
  7. In the message, always try to be kind and helpful, most of the time ask to test newer versions.
  8. Keep track of the pinged bugs, well I do it for my blog but it will be useful anyway, see below.
  9. When the submitter answers, just react accordingly, if the bug can be closed, then close it ! ;)
  10. If after some time, the bug are still not answered, consider closing them (I haven t started that yet, but this will probably get the bug number down faster:) ).
Note: Sometime I don t send the bug to -quiet so that other debian-x subscriber can have a look at the bug. They then can give me advices, or close the bug directly with a stronger reason than any I could have found. ;) To submitters, please keep the bug CCed when you answer, thanks. For the skeptics Yes as a newbie I did some mistakes, but I was always corrected in a nice way, with explanations. Also I try to have a conservative approach, I don t close bugs if I m not sure, I ask if I can close them, so it might take longer, but in the end the effect is the same for the bug, but not the same to the submitter ! ;) Also this work might look boring, but it s actually rewarding, most of the answers I received included a cheerful message thanking me for following up on the bug. Not to mention all I ve learned on Xorg. Conclusion As long as I have time for it, I ll continue to triage bugs and I d love to see others coming to help me on X or probably better go and help some other teams that also need help, and flood the planet with bug triaging reports ! ;) Get involved !

Rapha&#235;l Hertzog: Review of my Debian related goals for 2010

Last year I shared my Debian related goals for 2010 . I announced that I would not be able to complete them all and indeed I have not, but I have still done more than what I expected. Let s have a look. Translate my Debian book into English and get it published: NOT DONE Or rather not yet done. It s still an important project of mine and I will do it this year. When I wrote this last year, I expected to find a publisher that would take care of everything but we failed to find a suitable one so we re going to do it ourselves. Cleanup the dpkg perl API and create libdpkg-perl: DONE libdpkg-perl has been introduced with dpkg 1.15.6. Create dpkg-buildflags: DONE dpkg-buildflags has been introduced with dpkg 1.15.7. It s not widely used yet and it won t really be until debhelper 7 supports it (see #544844). It would be nice to see progress on this front this year. Ensure the new source formats continue to gain acceptance: DONE The adoption rate has been steady, it clearly slowed down since the freeze though. I have implemented quite a few features to satisfy the needs of users, like the possibility to unapply the patches after the build, or the possibility to fail in case of unwanted upstream changes. Design a generic vcs-buildpackage infrastructure to be integrated in dpkg-dev: NOT DONE I believe it s something important on the long term but it never made it to my short-term TODO list and it s unlikely to change any time soon. Continue fixing dpkg bugs faster than they are reported: PARTLY DONE We have dealt with many bugs over the year, but we still have 20 more bugs than at the start of last year (370 vs 350). We re not doing bad compared to many other Debian teams but we can still benefit from some help. Start here if you re interested. Enhance our infrastructure to ease interaction between contributors and to have a better view of how each package is maintained: NOT DONE I am convinced that we need something to have a clearer idea of the commitments made by each contributor. I don t put the same amount of care in maintaining smarty-gettext that I do on dpkg. If we had a database of the stuff that we know we don t do well enough, it s easier to point new contributors towards those. Anyway, this project is still unlikely to come to the top of my priorities any time soon. Work on the developers-reference: NOT DONE We have switched the Maintainer field to debian-policy@lists.debian.org to have more review of the changes suggested through the bug tracking system but that has not changed much on the global situation. I still hope to become more active on it sometimes this year. Maybe by trying to make it more fun and creating the text for some of the wishlist bugs as blog articles first. Rewrite in C the last Perl scripts provided by the dpkg binary package: DONE Dpkg 1.15.8 was the first version working without Perl, I announced it in July. Integrate the 3-way merge tool for Debian changelogs in dpkg-dev: DONE dpkg-mergechangelogs is part of dpkg-dev since 1.15.7. I enjoy it regularly. Unfortunately it doesn t work well for cherry-picks. Would be nice to see this fixed, anyone up to the task? :-)
Click here to subscribe to my free newsletter and get my monthly analysis on what s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.

One comment Liked this article? Click here. My blog is Flattr-enabled.

Next.

Previous.