Search Results: "cas"

3 January 2013

Tim Retout: New Year

Another year. 2012 was busy - I got moved house twice, changed jobs, and got married. In 2013, I should become a father, fingers crossed (due mid-April). Change is a familiar friend now. I just listened to Tom Armitage speaking about coding on Radio 4 - I /think/ the podcast mp3 link will work for people outside the UK, but the iPlayer probably won't. If you can get hold of it, it's worth the 20 minutes of your time. If I had to make a New Year's resolution, it would be to listen to more Radio 4 - there's such a lot of it, though. I'm going to try subscribing to some of their podcasts and listening to them on my commute - timeshifting some of the best bits. Might work.

Joey Hess: overcast

My least favorite forecast, and we've been having a lot of it lately. Since winter began (Dec 21st), my PV array has produced only 102 amp-hours of power. There have been only 3 days that could be considered sunny at all (even for a few hours). To put this in some perspective for you, Wolfram Alpha tells me that 102 amp-hours is half of a typical car battery capacity. So I've really had to scrimp & save. I disconnected my router to save power, and I plumbed the depths of the battery bank (further than it's really safe to go). And I was completely out of power for several days. So, I've bought a generator. I had to order it online, because it seems every store in the entire eastern US is completly out of stock, due to hurricane Sandy. This house has a generator shed (complete with a massive old generator that doesn't work), but I put the new, small generator in the furthest outbuilding, from where it can only barely be heard in the house. Later I'll be able to run a cable from its 12v terminal to charge the battery bank (if the sun doesn't get there first); for now it only powers my laptop. Nasty noisy smelly dirty thing, it pains me to run it. Back to the "overcast" forecast. I've heard meterologists say that one unexpected consequence of climate change seems to be that weather patterns are persisting longer, giving us this summer's weeks of neverending heat, and large, slow-moving storms. I suspect this could also explain the blankets of clouds that have been settling in for weeks at a time, on and off since mid-November. But I only have 2.5 years of not very good data. Before, I paid about as much attention to whether it was sunny as anyone else, which is to say, not really very much. Anyone know of a source of historical cloud cover data?

Michal Čihař: Free software plans for 2013

Year 2013 has just started and it's time to think how to spend my free time during this year. Weblate Weblate was my toy project number one in last year and I expect it to stay for next year as well. I have several ideas for new features, but all that depends on free time. I will most likely spend my hackweek on that as well. phpMyAdmin I think my involvement will stay at about same level at phpMyAdmin - doing few random bugfixes, mostly security ones, taking care of infrastructure for translations and wiki and mentoring few GSoC students. Gammu and Wammu Gammu and Wammu has not seen much activity from me in past year and it will probably not change. I don't use the program anymore and this pretty much limits my interest in adding new features. In case somebody active will pop up on mailing list, I'd be happy to transfer the project ownership. Mobile apps I'd like to make some progress here, I was also thinking about making some of them working on Android as well, but this involves lot of learning and time is always an issue here. Anyway this might be good challenge for hackweek as well. Debian No changes expected here :-). Others I consider pretty much anything else from my software projects abandoned or finished.

Filed under: Debian English Gammu Phpmyadmin Suse Weblate 0 comments Flattr this!

2 January 2013

Niels Thykier: Getting space for more packages

In 2011, I wrote about how small files could consume a lot of space. I meant to do a follow-up on the savings but I forgot about it until now. In 2.5.7, we started compressing some of the collected data files. Some of these are ridiculously compressable (#664794). Even better, compressing them is sometimes faster than writing them directly to the disk, so in some cases it is a pure win/win. For lintian.d.o, we also see a vast size reduction in overall size of the laboratory. I have taken a few samples occasionally. The samples were done with du(1):
$ du -csh [--apparent-size] laboratory/*
Version/date du -csh apparent-size
N/A around 20 Mar 2012 (#664794) 16G 13G
2.5.6 (Fri Apr 27 2012) 14GB N/A
2.5.6 (Mon Jun 04 2012)) N/A 12G
2.5.10.2 (Fri Sep 21 2012) 12G 8.3G
2.5.11 (Wed Jan 2 2013) 10G 6.1G
And the most awesome part of this? The comparison is quite biased against the 2.5.11 entry, which is the only entry to also process experimental (approx. 10% extra packages). Some of the early entries (2.5.6 and older ) might also have suffered from the too many links issue[1]. I only wish I had been better at collecting data points, so I could have made a proper graph of it. :) It sounds almost too good to be true, but if you look at the size of one of the linux-image packages[1], the space usage dropped from 27M to 15M between 2.5.5 to 2.5.9. Currently it is squeezed down to 14M (tested with head of the git master branch). [1] I believe is about 5-10% less binary packages processed for those runs. [2] linux-image-3.2.0-2-amd64_3.2.20-1_amd64.deb

1 January 2013

Vincent Sanders: I don't drink coffee I take tea my dear, I like my toast done on one side...

Choices, options, selections if you rather. These are what set us all apart from our fellow man, perhaps it is only the appearance of free will and individuality but our world is full of choice.

In software it might be argued programming is nothing more than making thousands perhaps millions of choices which result in a list of instructions for a machine incapable of making decisions of its own.

On a more mundane level sometimes a programmer cannot make a choice suitable for all expected use cases and a user option is created. These options have become something of a cause for arguments within certain sections of the open source software community especially amongst the groups that influence the graphical user experience.

The discussion should be nuanced and varied (perhaps that is my age, or maybe I am more diplomatic than I thought?) but there seems to be little compromise on this discussion which (from an outsiders point of view) splits into two viewpoints:


The reductionist group is currently winning the argument in the popular graphical environments and seem to be removing functionality which requires user choice wherever they can find it. This results in the absurd "joke" that the UI will become a single button, which they are trying to remove.

Personally my view is that an option should be present only when an option cannot be satisfactorily selected by the computer and even then a default suitable for as many users as possible should be picked.

You may ask why I have raised this topic at all? well over the last few days I have been trying to fix the preferences selection for the GTK port of NetSurf. The NetSurf project follows my view on this subject pretty closely but being a browser its very difficult to do the right thing for everyone

NetSurf has numerous frontends for the core browser, I use the term frontend because in some places the toolkit and OS are conflated (windows, cocoa) and not in others (gtk). For each frontend NetSurf is a native application, this is an important distinction, the windows and widgets a user interfaces with are produced by that platforms toolkit.

Old NetSurf Preferences Dialog
Old NetSurf Preferences Dialog
This is a deliberate choice unlike other browsers which render their UI themselves as web content, this is a beguiling solution when the authors wish there to be a single browsing "experience" with a common look and feel. However NetSurf looks and feels like a native browser for each frontend, which is what the project decided it wanted to achieve.

Given this the gtk frontend (is it a Gnome application? not sure of the distinction TBH) has had no dedicated maintainer for some time it has suffered from a bad case of bitrot from both GTK library changes and general neglect.

I have slowly been improving this situation, the browser can now be compiled with GTK version 2 (2.12 onwards) or 3, the menus and other UI elements have been translated for the supported languages and now the turn has come for the options dialog.

Oh, right, its "Preferences" not options, fair enough a common semantic throughout all applications does give a degree of homogeneousness but the word choice does seem to indicate the users control has been reduced. At this point I gained an education in just how unfriendly GTK and its tools have become towards the casual programmer.

The dialog I wanted to construct was a pretty standard tabbed layout which reflected the current options and allowed the user to change them in an obvious way. Given that I have constructed and equivalent interface in idiomatic manner for cocoa and windows I thought this would be straightforward, I was very wrong.

The interface construction tool is called glade which used to be the name of the UI "resource" file and interfaces. However the tool is still called glade but the interface is now GtkBuilder which has a different (but similar) XML based file format. Then we discover that despite it being an extensible file format the UI files are specifically versioned against the GTK library. Also why on earth can these resources not be compiled into my program? OK make them over-ridable perhaps. Generally it is yet another file to distribute and update in step with the executable.

New NetSurf Properties Dialog open on Main tab
New NetSurf Preferences
So because I want to support back to version 2.12 of GTK I do not get to use any of the features from 2.16 in my UI unless I load a different UI builder file...oh and GTK3? requires a completely different version of glade and the UI files which are incompatible with gtk 2. Once this was worked round with having multiple UI files I then moved on to the next issue.

The GTK model uses function callbacks on events, these are known as signals. Perfectly reasonable but because the ui files are loaded at runtime and not compiled in there must be a way for the GTKBuilder to map the textural signal names to function pointers.

The way the GTK developers have chosen to do this is to search the global function table which means any signal handler function symbol has also to be global which means unnecessary overhead for the ELF loader increasing load times.

This lookup should have been confined to a single object or even placed in an alternative section to avoid these issues, this would not have seemed especially challenging to implement as all callback handlers have to use a preprocessor to define already (G_MODULE_EXPORT).

Another thing that makes developing GTK applications worse is the documentation, this seems to be a continuous complaint about may open source projects, which is taciturn or simply missing in many cases. GTK seems to suffer dreadfully from having multiple API versions all subtly different resulting in a lot of work on the developers part to simply find what they want.

A specific example of this is the signals and under what circumstance they occur I wanted to update all the widgets with the current configuration state whenever the options window is shown (using gtk_widget_show()) so I figured that would be the "show" signal...right? nope, never did find the right one to use, ended up with "realize" which occurs once when the dialog is created, not what I wanted, but is at least consistent and works.

NetSurf with is Properties dialog open on the content tab
NetSurf with Preferences Dialog
Overall my impression of developing just one small dialog (60 widgets total exclusing labels and containers) for a GTK program is that the toolkit and its tooling is missing the facilities to make developers life easier when doing the drudge work that a good proportion of graphical program development.

It is not the case that one cannot do things, just that everything has to be done manually instead of having the tools do the work, and because of the documentation that tedium is magnified.

I did eventually reach the stage that the thought of writing the boilerplate to add another check button to enable "do not track" had me thinking "do they really need this or can I avoid the work?" perhaps that is why they are all reductionist?



Russ Allbery: Review: City of Diamond

Review: City of Diamond, by Jane Emerson
Publisher: DAW
Copyright: March 1996
ISBN: 0-88677-704-6
Format: Mass market
Pages: 624
The setting is deep into the star-faring future of humanity. Humans have settled many other worlds, thanks largely to technology from a race named the Curosa. There is a thriving network of settlement based around a sort of warp conduit network. But, standing apart from most human settlements, are three huge ships: the City of Opal, the City of Pearl, and the City of Diamond. Their inhabitants are devout members of the Redemptionist faith, a merging of Christian (primarily Catholic) belief and Curosa religion. Their ships are gifts from the now-departed Curosa, featuring drives unknown to the rest of humanity that allows them to ignore the normal transport network. They are meant to witness and spread their religion, but now, six centuries later, they primarily trade. Much of this is not apparent at the start. City of Diamond opens with Adrian Mercati, chosen heir of the current dying Protector, dodging people in what seems to be an oddly-organized city. Impudent, occasionally flippant, utterly charming, and in love with danger, he inherits the position of Protector shortly after we meet him. Emerson slowly reveals more of the nature of the ship while Adrian is shocking (and charming) the nobility and the church, and consorting with demons. Or, at least, one demon: Tal, an entirely human-appearing creature except for his eyes, who the Redemptionists claim is born without a soul. Slowly, it becomes clear that he's actually the offspring of an alien and a human, a mating of species whose children cannot socialize as human and behave in ways that seem like human sociopaths. Tal is no exception, but his interests align with Adrian's for the time. To those two viewpoint characters, Emerson adds four more: a low-rank administrator named Spider who works for Tal after Tal saved him from death by recycling, Adrian's politically-arranged bride-to-be from the rival (and even more religious) ship City of Opal, a guard from a French-descended slum on City of Opal, and a dangerously honorable and literal mercenary from an off-shoot culture of humanity that's now only a legend. This is a book that tends to sprawl. There's quite a lot going on, starting with the negotiations between City of Diamond and City of Opal to put to rest the remnants of a civil war and leading to the search for a legendary Curosa artifact on the planet they're both visiting. All of the characters have their own agendas, their own history, and their own goals, and are pursuing them in parallel and occasionally in opposition. Emerson takes the time to let all of this build. Despite that, there is very little in this book that drags or bores, and that's due primarily to the strength of the characters. Every one of the viewpoint characters is unique, bringing a different perspective to the story, and every one of them changes and grow over the course of the story. They also play off of each other in delightful ways. This is a book that makes full use of the fact that three characters make seven character interactions. For example, Tal by himself is logical but just slightly askew, Tal seen by Spider is uncomfortable and worrisome, Tal and Adrian is a dance of mutual amusement and respect, and Tal and Keylinn finding points of commonality is just delightful. I did wish that there was a bit more Keylinn and Tal and a bit less of Spider, but once Emerson built her world, introduced her characters, and got the story moving, there aren't many weak pieces. Part of the fun is figuring out the world background. There are almost no infodumps here; Emerson just starts telling stories of characters in the surroundings in which they find themselves, and the reader is left to piece things together from background description, the occasional explanation from one character to another, and parallels with long-standing SF tropes. It's not a puzzle book, but it is a book of where the reader is dropped as an alien into a fully-developed culture. In both that way and in the use of nobility, religion, and formal hierarchy against a starfaring backdrop, City of Diamond reminds me of the sort of classic space opera that led up to Star Wars but isn't seen as much these days. It's also funny. I was unsurprised to discover that Emerson is a pseudonym for Doris Egan, who (in addition to writing some other SF) is an occasional writer and producer for House. The humor isn't entirely obvious at the start, although Adrian frequently provokes a grin from the start of the book, but Tal's perceptive but obliviously blunt comments become more and more fun as the book progresses, and both Iolanthe (Adrian's bride) and Keylinn (the mercenary, sort of) are a delight. I want to see more of both of them, but sadly the book ends just as they (particularly Iolanthe) are hitting their strides. And that's the biggest problem with City of Diamond: it's a failed (or at least postponed) trilogy. As with Walter Jon Williams's Metropolitan and City on Fire, there were supposed to be more books, but apparently either the publisher lost interest or other projects took precedence. The story does come to something of a conclusion, but most of the major story arcs aren't resolved. Emerson is still bringing more guns onto the set late in the book. None of the villains truly get their just desserts, nor do most of the character arcs fully resolve, although friendships and more have formed satisfyingly by the end of the book. There is obviously far more to both these characters and this universe, including an entire ship that was never visited and an alien whose presence should prove startling. This is a great book: warm, funny, full of interesting characters, and with a nicely complex background universe. If it were complete in itself, or if the sequels had materialized, I would recommend it unreservedly. As is, you will have to weigh your dislike of unfinished stories against its stand-alone merits. But I enjoyed it a great deal. Rating: 7 out of 10

31 December 2012

Russ Allbery: Term::ANSIColor 4.01

It was too much to hope for that all that testing would catch everything in a 4.00 release, but it's ironic that the main bug was a bug in one of the test cases! BlueT - Matthew Lien - found that one of the tests malfunctioned when Test::Warn was not installed. Since I was cutting a new release anyway, I also improved the README somewhat and added POD documentation to the example script. You can get the latest version from the Term::ANSIColor distribution page.

Joey Hess: no longer a perl programmer

This year, I've gradually realized that I no longer identify as a perl programmer. For a decade and a half, perl was the language I reached for to solve any problem that didn't have a good reason to be solved in some other language. Now I only reach for it in the occasional one-liner -- and even then I'm more likely to find myself in ghci and end up with a small haskell program. I still maintain plenty of perl code, but even when I do, I'm not thinking in perl, but traslating from some internal lambda calculus. There's quite a few new-ish perl features that I have not bothered to learn, and I can feel some of the trivia that perl encourages be kept in mind slipping gradually away. Although the evil gotchas remain fresh in my mind! More importantly, my brain's own evaluation of code has changed; it doesn't evaluate it imperatively (unless forced to by an appropriate monad), but sees the gesalt, sees the data flow, and operates lazily and sometimes, I think in parallel. The closest I can come to explaining the feeling is how you might feel when thinking about a shell pipeline, rather than a for loop. Revisiting some of my older haskell code, I could see the perl thinking that led to it. And rewriting it into pure, type-driven, code that took advantage of laziness for automatic memoization, I saw, conclusively that the way I think about code has changed. (See the difference for yourself: before after ) I hear of many people who enjoy learning lots of programming languages, one after the other. A new one every month, or year. I suspect this is a fairly shallow learning. I like to dive deep. It took me probably 6 years to fully explore every depth of perl. And I never saw a reason to do the same with python or ruby or their ilk; they're too similar to perl for it to seem worth the bother. Though they have less arcania in their learning curves and are probably better, there's not enough value to redo that process. I'm glad haskell came along as a language that is significantly different enough that it was worth learning. The deep dive for haskell goes deep indeed. I'm already 5 years in, and have more to learn now than I ever did before. I'm glad I didn't get stuck on perl. But I may be stuck on haskell now instead, for the foreseeable future. I'd sort of like to get really fluent in javascript, but only as a means to an end -- and haskell to javascript compilers are getting sufficiently good that I may avoid it. Other than that, I sense adga and coq beckoning with their promises of proof. Perhaps one of these years. Of course if Bradley Kuhn is right and perl is the new cobol, I know what I'll be doing come the unix rollover in 2038. ;)

Russ Allbery: Review: Wasted Heart

Review: Wasted Heart, by Lynn Galli
Publisher: Penikila
Copyright: 2010
ISBN: 1-935611-32-1
Format: Kindle
Pages: 252
Austy is a lawyer working for the US District Attorney's office in Seattle. Six months ago, she moved there from Virginia, in large part because she fell in love with her best friend. Her married best friend. Now, she's trying (mostly unsuccessfully) to deal with the Seattle lesbian scene while becoming (successfully) the go-to lawyer for prosecuting cases with a substantial technology component. Two things are about to complicate that life: meeting and working with FBI special agent Elise Bridie, and working on a case that involves her best friend's company. Yes, that best friend. This is, as you might have guessed, a lesbian romance novel, so with that should come my "reading out of genre disclaimer": it's rather easy, when reading a genre one normally doesn't read, to find books fresh and interesting when they're unoriginal to regular genre readers, or to miss genre conventions or reactions because one has not read the previous work. You should therefore take my opinions with a grain of salt, particularly if you read more romance novels than I do (which wouldn't be hard). I grabbed this book for a few reasons: I felt like reading a warm-hearted story focused on human relationships, I wanted a reasonably fast read (since I'm obsessed with meaningless numeric milestones, like how many books I read in a year), and I wanted something with an upbeat ending. Romance generally delivers on those fronts. Lesbian romance has the added bonus of avoiding a whole bunch of gender essentialism and heterosexual romance tropes that tend to give me hives. Landing was excellent, so I thought I'd see if I could find more of that. Wasted Heart does indeed deliver, although it's not quite as good as Landing. There's a bit too much physical attraction and love based on appearance for my taste, but Galli leavens that with quite a lot of interesting psychology. Both Austy and Elise are private and cautious people, and for me that makes the romance feel more realistic. It also draws out the initial phase of getting to know the other person, which is the part of the romance that I often find the most interesting. Elise in particular is a great character maybe a little too ideal in a few places, but smart and observant in a way that's very fun to read about. Like a lot of romances, there is some annoying miscommunication and misunderstanding (predictably right around the height of the tension), but it's not quite as stupid as those problems usually are, and Galli doesn't drag it out until one starts wanting to hit the characters with blunt objects. As good or better is the community that surrounds the main characters. Galli has a full cast of believable and interesting characters, most notably including Austy's old gang from Virginia. The tensions between having a group of friends one cares about deeply, wanting to build a new life after making a major break and major decision but having difficulty sustaining that under the onslaught of those friends, and being easily tossed about by expressive and outgoing people when one is private and a bit quiet all play out here. And, refreshingly, without a lot of angst, and with quite a bit of intelligent introspection. Both Austy and her friends are smarter and more considerate than most of the characters I've seen in this sort of situation. Another good bit is the little pieces of legal drama. Galli doesn't neglect Austy's work life, and while this is certainly not a legal thriller or procedural, we do get some nice scenes of Austy doing her job. I love courtroom scenes, particularly ones (like these) that are more on the realistic side, so that was a nice bonus. Austy and Elise's jobs also lead to a somewhat predictable bit of drama, and I was a bit less enamored of that; the more I thought about it, the less well it hung together for me. But I did like that it stayed firmly on the realistic side, including a look at psychological aftereffects for real people rather than thriller protagonists. This is not the sort of book that I would tell someone to rush out and buy, but it met my criteria nicely and delivered just the sort of entertainment I was looking for. (And it's not particularly expensive on Kindle, at least in the US.) There was a bit too much focus on physical appearance and physical infatuation, and Elise might be just a little too perfect, but I enjoyed it. Note that I read the special edition, which includes a (much appreciated) epilogue that closes out the story. That seems to be the default edition for sale everywhere I look, but in case you see the older version, I recommend the special edition. Rating: 7 out of 10

Russell Coker: Modern Swap Use

A while ago I wrote a blog post debunking the idea that swap should be twice as large as RAM [1]. The issue of swap size has come up for discussion in a mailing list so it seems that it s worth another blog post. Swap Size In my previous post I suggested making swap space equal to RAM size for systems with less than 1G of RAM and half RAM size for systems with 2-4G of RAM with a maximum of 2G for any system. Generally it s better to have the kernel Oom handler kill a memory hungry process than to have the entire system go so slow that a hardware reset is needed. I wrote memlockd to lock the important system programs and the shared objects and data files they use into RAM to make it easier to recover from some bad paging situations [2], but even that won t always make it possible to login to a thrashing system in a reasonable amount of time. But it s not always good to reduce swap size. Sometimes if you don t have enough swap then performance sucks in situations where it would be OK if there was more swap. One factor in this regard is that pages mapped read-only from files are discarded when there is great memory pressure and no swap space to page-out the read-write pages. This can mean that you get thrashing among memory for executables and shared objects while there s lots of unused data pages in RAM that can t be paged out. One noteworthy corner case in this regard is Chromium which seems to take a lot of RAM if you have many tabs open for a long time. Another factor is that there is a reasonable amount of memory allocated which will almost never get used (EG all the X based workstations which have 6 gettys running on virtual consoles which will almost never be used). So a system which has a large amount of RAM relative to it s use (EG a single-user workstation with 8G of RAM) can still benefit from having a small amount of swap to allow larger caches. One workstation I run has 8G of RAM for a single user and typically has about 200M of the 1G swap space in use. It doesn t have enough swap to make much difference if really memory hungry programs are run, but having 4G of memory for cache instead of 3.8G might make a difference to system performance. So even systems which can run without using any swap can still be expected to give better performance if there is some swap space for programs that are very inactive. I have some systems with as much 4G of swap, but they are for corner cases where swap is allocated but there isn t a lot of paging going on. For example I have a workstation with 3G of RAM and 4G of swap for the benefit of Chromium. Hardware Developments Disk IO performance hasn t increased much when compared to increases in RAM size over the last ~15 years. A low-end 1988 era hard drive could sustain 512KB/s contiguous transfer rates and had random access times of about 28ms. A modern SATA disk will have contiguous transfer rates getting close to 200MB/s and random access times less than 4ms. So we are looking at about 400* the contiguous performance and about 10* the random access since 1988. In 1988 2MB was a lot of RAM for a PC, now no-one would buy a new PC with less than 4G so we are looking at something like 2000* the RAM size in the same period. When comparing with 1993 (when I first ran a Linux server 24*7) it was 4M of RAM, maybe 15ms random access, and maybe 1MB/s contiguous IO approximately double everything I had in 1988. But I ll use the numbers from 1988 because I m more certain of them even though I never ran an OS capable of proper memory management on the 1988 PC. In the most unrealistically ideal situation paging IO will be a factor of 5* more expensive now relatively than it was in 1988 (RAM size increase by a factor of 2000 divided by contiguous IO performance increase by a factor of 400). But in a more realistic situation (random IO) it will be 200* more expensive relatively than it was. In the early 90s a Linux system with swap use equal or greater than RAM could perform well (my recollection is that a system with 4M of RAM and 4M of active swap performed well and the same system with 8M of active swap was usable). But nowadays there s no chance of a system that has 4G of RAM and 4G of swap in active use being usable in any way unless you have some corner case of an application allocating RAM and not using it much. Note that 4G of RAM doesn t make a big system by today s standards, so it might be reasonable to compare 2M in 1988 to 8G or 16G today. But I think that assuming 4G of RAM for the purpose of comparison makes my point. Using Lots of Swap It is probably possible to have some background Chromium tabs using 4G of paged out RAM with good system performance. I ve had a system use 2G of swap for Chromium and perform well overall. There are surely many other ways that a user can launch processes that use a lot of memory and not use them actively. But this will probably require that the user know how the system works and alter their usage patterns. If you have a lot of swap in use and one application starts to run slowly you really don t want to flip to another application which would only make things worse. If you use tmpfs for /tmp then you need enough swap to store everything that is written to /tmp. There is no real upper limit to how much data that could be apart from the fact that applications and users generally don t want to write big data files that disappear on reboot. I generally only use a tmpfs for /tmp on servers. Programs that run on servers tend not to store a large amount of data in /tmp and that data is usually very temporary, unlike workstation users who store large video files in /tmp and leave them there until the next reboot. Are there any other common corner cases that allow using a lot of swap space without serious performance problems? My Swap Space Allocation When I setup Linux workstations for other people I generally use 1G of swap. For a system that often has multiple users (via the user-switch feature) I use 2G of swap if it only has 3G of RAM (I run many systems which are limited to 3G of RAM by their chipset). For systems that I personally use I generally allocate 4G of swap, this is to cater to excessive RAM use by Chromium and also because I test out random memory hungry programs and can t easily predict how much swap space will be needed a typical end-user desktop system is a lot more predictable than a workstation used for software development. For servers I generally allocate 512M of swap space. That s enough to page out things that aren t used and make space for cache memory. If that turns out to be inadequate then it s probably best to just let the watchdog daemon reboot the system.

30 December 2012

Neil McGovern: Fleeing Google

For some time, I have been concerned about the amount of data I store on Google. This post aims to show alternatives to the service. This is primarily for two reasons. The first is simple. I would like to avoid a single point of failure, and the risk that I could be suspended for a number of months with no recourse. The second if more political. I'm worried about the way that certain governments can subpoena information which they may not be able to collect directly. Now, don't get me wrong. Google is good at transparency. They also provide a very useful service that works incredibly well. In fact, the quality of the integration and services provided is the primary reason I have not changed previously. Calendaring is something that I naturally feel shouldn't be hard, but somehow is. Google Calendaring service is rather good, and far outstrips anything open source I've found so far. However, I've managed to migrate to Zoho which offers a very similar level of functionality. It's also very easy to extract your entire calendar from Google, which makes the transition easier. Reader is a service provided to subscribe and read RSS feeds. In the past, I've used Gregarius as a self hosted solution, but this seems to have disappeared recently. I briefly flirted with NetVibes, which works well once you turn off the frustrating 'widget' mode, but has completely failed to address the need to have native tablet apps. In the end, I settled for NewsBlur which also has the clever use of hotkeys for navigation which I found essential in Google's products. Picasa has stood me well for hosting photos for some time. And photo hosting is something I have quite a large amount of professional experience with :) Despite having previously been in competition with them, I can heartily recommend Flickr, even with it's dogmatic reliance to creating a Yahoo! email address for each account. Search still has some way to go to find a viable replacement. I've tried DuckDuckGo, but found it lacking. Much like the mistake that Apple seem to have made with the launch of Apple Maps, there seems to be a misunderstanding that search is actually HARD.
There are two camps when it comes to searching. You can be dumb, or you can be smart. By being dumb, the user can provide enough information to help narrow the results down to a single (or at least manageable) result set. This works fine, and is a traditional model. The opposite is to be smart. Simply get the results right, using a variety of metrics to work out what the user wanted in the first place. This is obviously more difficult, but infinitely more valuable, and is something that Google excels at. The problem lies when you fall in between these stalls. Unfortunately, it seems that there is no viable alternative to the big G at the moment.

29 December 2012

Russell Coker: Samsung Galaxy Camera a Quick Review

I recently had a chance to briefly play with the new Samsung Galaxy Camera [1]. The Galaxy Camera is an Android device with a 4.8 display (the same size as the Samsung Galaxy S3) that has a fairly capable camera (IE nothing like a typical phone camera). It runs Android 4.1 (Jelly Bean) and the camera has 21* zoom with a 16 megapixel sensor. Camera Features It seems that professional photographers are often annoyed when they see someone with a DSLR set in auto mode. It s widely regarded that auto mode is a waste of a good camera, although the better lenses used with DSLRs will usually give a better result than any compact camera even when it s in auto mode. The problem is that photography is quite complex, in an earlier post about digital cameras I summarised some of the technical issues related to cameras and even without any great detail it became a complex issue [2]. The Galaxy Camera has a reasonably friendly GUI for changing camera settings which even includes help on some of the terms, I expect that most people who use it will end up using most of the features which could make it a good training camera for someone who is going to move to a DSLR. A DSLR version of the Galaxy Camera could also be an interesting product. The camera also has modes such as Waterfall and Panorama , hopefully the settings for those would be exposed to the user so they could devise their own similar groups of settings. I ve seen the phone criticised for the lack of physical controls as the expert mode in software is inherently slower than manually turning dials on a DSLR. But it seems obvious to me that anyone who knows how to use the controls manually should be using a DSLR or bridge camera and anyone who doesn t already know how to do such things will be better suited by the software controls. It supports 120fps video at 720*480 resolution (with a file format stating that it s 30fps to give 1/4 speed) which could be useful. I used to have a LG Viewty smart-phone that did 120fps video but the resolution was too low to be useful. 720*480 is enough resolution to see some detail and has the potential for some interesting video, one use that I ve heard of is filming athletes to allow them to analyse their performance in slow motion. It also does 60fps video at 720p (1280*720) resolution. One down-side to the device is that the lens cover doesn t seem particularly sturdy. It s quite OK for a device that will be stored in a camera case but not so good for a device that will be used as a tablet. I didn t get to poke at the lens cover (people don t like it if you mess with their Christmas presents) but it s design is a couple of thin flaps that automatically retract when the camera is enabled which looks quite weak. I d like to see something solid which doesn t look like it will slide back if the device is treated as roughly as a phone. I think that the lack of a solid lens cover could be the one factor that prevents it from being used as a replacement for a smart phone. Apart from that a Galaxy Camera and a cheap GSM phone could perform all the functions of a high end phone such as the Galaxy S3 while also producing great pictures. It would probably make sense for retailers to bundle a cheap phone with a Galaxy Camera for this purpose. Tablet Features The device boasts WiFi Direct to allow multiple cameras and phones to talk to each other without a regular WiFi access point [3]. I didn t test this and I don t think it would be particularly useful to me, but it seems like a handy feature for less technical users. It can connect to the Internet via Wifi or 3G, supports automatic upload of pictures (it comes with Dropbox support by default like the Galaxy S3), and has a suite of photo and video editing software. I don t expect that any photo editing software that runs on an Android device would be much good (I think that you really need fine cursor control with a mouse and a high resolution screen), but it would probably be handy for sending out a first draft of photos. Most Android apps should just work, the exceptions being apps that rely on a camera that faces the user or full phone functionality. So the Galaxy Camera can do almost anything that an Android phone or tablet can do. Value The RRP for the Galaxy Camera is $599, that puts it in the same price range as a DSLR with a single lens. While that s not a bad price when compared to smart-phones (it s cheaper than the LTE version of the Galaxy S3 phone) it s still quite expensive for a camera that s not a DSLR. Fortunately Kogan is selling it for $469 and has free shipping at the moment [4]. This still makes it more expensive than some of Bridge Cameras which probably have significantly better optical features, but in terms of what the typical user can do with a camera the Galaxy Camera will probably give a much better result. The sensor in the Galaxy Camera is smaller than that in the Nokia 808 PureView [5] (1/2.3 vs 1/1.2 ) so the Nokia PureView should be able to take better pictures in some situations. Unfortunately the Nokia 808 doesn t run Android, I d probably own one if it did. Some of the reviews are rather harsh, the Verge has a harsh but fair review by Aaron Souppouris which makes a number of negative comparisons to cheaper cameras [6]. I really recommend reading Aaron s review as there s a lot of good information there. But I think that Aaron is missing some things, for example he criticises the inclusion of ebook software by saying that he wouldn t read a book on a camera. But the device is a small tablet computer which also has a compact camera included. I can easily imagine someone reading a book or playing Angry Birds on their camera/tablet while in transit to where they are will photograph something. I can also imagine a Galaxy Camera being a valuable tool for a journalist who wants to be able to write short articles and upload pictures and video when out of the office. Aaron concludes by suggesting that the Galaxy Camera is a $200 camera with $300 of editing features. I think of it as $200 in camera hardware with software that allows less skilled users to take full advantage of the hardware and the ability to do all the software/Internet things that you would do on a $450+ smart-phone. Would I Buy One? No. The Galaxy Camera is among other things a great toy, I d love to have one to play with but I can t spare $469 on one. Part of the reason for this is that my wife just bought a DSLR and is getting lessons from a professional photographer, so I really won t get better pictures from a Galaxy Camera. The DSLR on auto mode will allow me to take pictures that will usually be better than a Galaxy Camera can achieve (sometimes you just can t beat a good lens). For more demanding pictures my wife can tweak the DSLR. The 120fps video is a really nice feature, I don t know if my wife s DSLR can do that, but it s a toy feature not something I really need. I ve just bought a Galaxy S3 which is a great little tablet computer (most of the time it won t be used for phone calls). I don t need another 4.8 tablet so a significant part of the use of the Galaxy Camera doesn t apply to me. I recommend the Galaxy Camera to anyone who wants to take good photos but can t get a DSLR and lessons on how to use it properly. But if you would rather get a 35mm camera with interchangeable lenses that runs Android then it might be worth waiting. I expect that the Galaxy Camera will be a great success in the market (it s something you will love when you see it). That will drive the development of similar products, if Samsung doesn t release a 35mm Android camera soon then someone else will (for example Sony develops both high end cameras and Android phones). If my wife didn t have a DSLR then I d probably have bought a Galaxy Camera already. I will recommend it to my parents and many other people I know who want an OK camera and can benefit from a tablet, but don t know how to use a DSLR properly (or don t want to carry a bulky camera).

28 December 2012

Guido G nther: GNOME Shell and On Screen Keyboard

Running Debian on the WeTab with GNOME Shell without an external keyboard works pretty nicely. The on screen keyboard - if enabled via the accessibility menu - folds out automatically in the shell itself within text input fields. To have this within GTK+3/GTK+2 applications you need libcaribou-gtk3-module and libcaribou-gtk-module installed. For other cases I stitched together a small extension that puts a keyboard "Button" prominently into the middle of the panel. Clicking/touching it will fold it out, clicking again will hide it again. You can fetch it from
git clone git://honk.sigxcpu.org/git/gnome-shell-oskb-extension.git ~/.local/share/gnome-shell/extensions/OnScreenKeyboardButton@sigxcpu.org
and activate it via
 gsettings set org.gnome.shell enabled-extensions "['OnScreenKeyboardButton@sigxcpu.org']"
OSK Button Screenshot For kinetic scrolling in Iceweasel I'm currently using Grab and Drag which is not yet packaged for Debian. Thanks to the Debian's GNOME packaging team gnome-shell 3.6 is already available in experimental. This blog is flattr enabled.

Hideki Yamane: How to associate with distribution


I know there is a reason why you rebuild/create your own packages.

  • Unfortunately, there's no official package that is necessary to provide service to your customers
  • distro package is not suitable for your purpose by default
    • maybe since its features
    • maybe since performance issue




However, I want to ask you one question.
Is it necessary that you'd maintain it in your private repository?
Because create custom package repository is a kind of "fork"


  • Of course, you can do folk. But if you don't want to be "upstream", every time you need to follow your upstream to fix bugs, vulnerabilities and introduce new features. Important: Merge cost would become larger and larger from the time you did folk.
  • It's best that if your system can be used "out of box", but you need to modify packages for some reason (as above).
    Then, what would you do? - minimize
    • minimize numbers of private package
      = reduce maintenance cost
    • minimize diff with distro package
      = minimize diff with upstream = reduce maintenance cost

If you could minimize diff with distro, you can update and introduce new attractive features from updated distro package easily (maybe you don't need to maintain your custom package since its feature is merged to upstream).

Yes, you can get Agility.


And, if you would put your package to Debian official repository, we Debian can give you below benefits

  • build daemon with many architectures
  • open bug tracking system
  • world wide mirror network
  • security vulnerabilities tracker
  • users

  1. We can provide build system, including non-x86 arch (none of you have thought arm* architecture become important, right?) You can deal with porting as daily job, no huge jump to support newer architectures. It also brings the view to build cleaner structures in your code.
    It is much work to setup such system, but you can use it without any trouble.
  2. You can get bug report/wishlist/patches from Debian BTS (without any pay for it! :-)
    It will help you to improve software packages, it cannot be sent to you if you would put packages to closed private repository.
  3. Package is mirrored to many repositories in world wide. Robust. You can get binary/source code immediately, anywhere. No need to backup your package source.
  4. Any CVE is tracked. You can analyze security status in your system easily (it means you don't need to pay much money to build such system)
  5. Debian has huge user base. It would work as good "showcase" test (of course, without any pay for it...)



Conclusion)

It's okay to create/modify package. It's FLOSS. As you like! :)

However, to maintain custom package, you need to pay its cost (someone should work for it. If it's in only your own repository, it means you should work, no one can help it).

To maintain packages in distro can reduce its cost, can get many benefit - Less cost, more benefit - why not? Let's get involved to distribution!

Russell Coker: SIM Annoyances

My new Samsung Galaxy S3 phone takes a micro-SIM (see the Wikipedia page about Subscriber Identity Modules for details [1]). All my other mobile devices take a mini-SIM so I can t just put the SIM from my old phone in my new phone. I ve just made my second application to Virgin Mobile for a new micro-SIM, the first time they sent me a nano-SIM by mistake. Also if my new phone breaks I will have difficulty in getting a micro-SIM to run in a mini-SIM device (it should be possible, but will be difficult at least). I may even have to ask my Telco to send me a new mini-SIM to allow the use of an older phone if my Galaxy S3 breaks. The difference between micro and mini SIMs is 25*15mm vs 15*12mm. For a phone with a 4.8 display this doesn t seem to be a great benefit. It doesn t seem that the phone would be much bigger if they had designed it for a mini-SIM. If they had used a nano-SIM which is 0.09mm thinner that might have allowed them to make the entire phone thinner, I think that such thin phones are a bad idea but the reviewers seem to like thin phones with small batteries. I think that any device which is large enough to have a speaker somewhere near my ear and a microphone somewhere near my mouth will be big enough to fit a 25mm long SIM card. It seems to have been a trend in Australia in the last few years towards only selling unlocked phones on contracts (relying on the contract terms to lock the customer in) and only use locked phones for discount sales for pre-paid use. Also the free phones in Australia are considerably more expensive than buying a phone outright [2]. So I m sure that most people have a collection of older phones which in future can t be easily interchanged with new phones due to SIM card size differences. This is more of a problem due to the fact that modern phones seem to have a low quality of hardware production, the majority of Android phones owned by my relatives have failed in some substantial way well within the two year replacement period. I don t believe that I can rely on my Galaxy S3 working until I want to buy something better, I expect that my old Galaxy S (which has a hardware fault that makes it crash regularly but is mostly usable) will be pressed into service again. In terms of phone purchases, buying a phone that takes a micro-SIM seems like a bad strategy as there is an even smaller nano-SIM available. For all I know Samsung will release a Galaxy S4 or Note3 that uses a nano-SIM and give me the same upgrade problem in a couple of years time. I wish that the people who reviewed phones would pay more attention to the real world use cases than to slightly better specifications. The micro and nano SIMs seem to provide no real benefit for users. Saving a fraction of a gram in device mass or a fraction of a millimeter in one dimension might seem nice when aggregated with other small savings, but for the user it s no real benefit. Being able to interchange a bunch of random phones is a real benefit. It s also a real benefit to be able to buy a phone and expect it to just work immediately. With the current state of the market one can t buy a phone and make any assumptions about the SIM size that it will accept, at best this requires some extra research before buying and at worst this could involve some time without phone service. I think that a major feature for a review of an important device like a modern smart phone should be how quickly and easily it can be deployed for full use. When SIM size issues prevent a new device from being used properly for over a week (as has happened for me) then it seems like a design failure.

Russ Allbery: Review: Perl Best Practices

Review: Perl Best Practices, by Damian Conway
Publisher: O'Reilly
Copyright: July 2005
Printing: April 2011
ISBN: 0-596-00173-8
Format: Trade paperback
Pages: 492
I had a rather bad first reaction to this book or, rather, the existence of this book. Some time ago, I started hearing about a new program named perlcritic that served as a lint program for Perl and decided to give it a try. I was taken aback by the flood of diagnostics, but even more taken aback by the fact that most of them were apparently explained only by a reference to a book that I didn't own. It struck me wrong in an open source project and gave me a negative reaction towards the book, which lingered for years. That was exacerbated by discovering that it recommended using use English in some circumstances, something that I consider to be a horrible idea. So, when a coworker recommended it and suggested I really should read it, I was dubious. I'm glad I gave it a chance, since it turns out that nearly all of my objections were based on misunderstandings or false information. (Well, except for the use English part, but more on that in a moment.) Perl Best Practices does provide a lot of advice, explaining that flood of diagnostics, but perlcritic also warns about the same thing repeatedly, inflating a handful of issues into a lot of messages. Most of the recommendations in Perl Best Practices were things I was doing already; most of the rest are entirely sensible. Also, perlcritic does come with complete explanations of its diagnostics once you know where they're hidden (the perldoc of Perl::Critic::Policy modules whose names you can get at by insisting perlcritic divulge them). And, even more delightfully, once I got past my objections, I discovered that Perl Best Practices is one of the best books on coding style that I've read. Coding style is a notoriously difficult subject, since nearly everyone is convinced their personal style is the best (or at least that they don't want to change). Conway acknowledges that up-front, and acknowledges that some things are just arbitrary and consistency with any rule set is better than following some specific rule set. He then spends surprisingly little time on the sorts of things that dominate most coding style discussions (indentation, spacing, commenting, etc.). Instead, this is primarily a book about how to write efficient and transparent Perl code, one that focuses much more on the content and structure of that code than on how it is indented. The best part is that there are no undefended rules here. Perl Best Practices contains 256 separate guidelines, some quite general ("Provide an optimal interface, rather than a minimal one") and some quite specific ("Pass constructor arguments as labeled values, using a hash reference"), but every one of them has a supporting argument and specific reasons behind it. Conway backs up every recommendation with code samples and practical scenarios showing the pluses and minuses of different approaches. This is a book unconcerned with being an authority; rather, it reads like optional advice from a good mentor, complete with insightful discussions on each point raised. You may come away from it disagreeing with Conway's specific recommendation (and I did, in several places), but you'll almost certainly be better-informed about the problem for having read it. Plus, this book is just fun to read. Admittedly, I love this sort of thing; I spend lots of time thinking about coding style in any language, and a lot more time refactoring coding style when working on my own projects to try to achieve clarity and simplicity. But even without sharing my love of the material, I think this book will be a delight for any practicing Perl programmer who hasn't yet read it. Conway clearly has a wealth of experience and years in the trenches of working with other people's Perl code. The examples he draws on are brief, comprehensible, and illuminating. He also gave me a tour of a variety of helper modules and libraries that I didn't know existed. List::Util is the most prominant example; this incredibly helpful supply of list manipulation functions that I've always half-known I wanted but never bothered to write is now part of Perl core (as of 5.7.3), so there's no reason to avoid it. You too can stop using grep when you actually mean first, or rewriting max in ad hoc ways. That does raise two of the rare flaws in this book, though. One is that I disagree (partially) with Conway's willingness to recommend non-core modules. He is a particular champion of the Readonly module over either normal file global variables or use constant, for very good reasons. But making Readonly a prerequisite for all other Perl code that one releases seems a bit annoying (speaking as someone who used to maintain local Perl installations). Maybe this isn't as big of a deal in these days of ubiquitous Linux distributions and willingness to use the CPAN shell, but I still dislike releasing code that depends on a half-dozen (or more) other CPAN modules for reasons of coding convenience and minor readability rather than core functionality. (His advocacy of Sub::Install raised similar objections.) The second flaw is not something Conway could have done anything about: this book is from 2005, and it's now 2012. Now, the Perl ecosystem is now quite mature, so that hampers the book much less than it would a corresponding book about Ruby or even Python. I also aim for compatibility with the oldest thing out there I know someone is still running, so I'm still hesitant to require anything newer than Perl 5.8. But still, this book was written seven years ago, and Perl is now at 5.14. In the meantime, we've seen core language features like given/when added, the addition of state variables, the need for substantial reworking of how Unicode I/O is handled, the addition of the new // operator, and the new package syntax, among many other things. The Perl taught here is a bit dated, and I would love to hear Conway's opinion about when (or if) to use say. Maybe someday we'll get a second edition. There are two other points that I do have to argue with. One is that Conway does advocate, in a limited way, use English. I could almost go along with his recommendation to use it only for those variables that one has to look up, except that in nearly every case there are other options and one can take the much better approach of simply not using those variables at all. But he then writes the remaining examples in the book using $EVAL_ERROR rather than $@, and I'm sorry, but no. One could not construct a more frustrating annoyance for an experienced Perl programmer than sending them on a futile hunt for where the global variable $EVAL_ERROR is defined. One can debate the merits of Larry Wall's original decision to use variables like $_, @_, $?, $!, and $?, but for better or ill this is Perl, and marginal readability gains for inexperienced programmers do not warrant confusing experienced programmers. This is particularly true when the use English variables are completely indistinguishable from normal global variables, and not infrequently look like a global variable that a programmer might choose. The second is more of an amusement that a real disagreement. Conway rightfully points out that Perl OO, and particularly the way that Perl stores object instance data, is awkward, and none of the commonly-used techniques are particularly satisfying. He then introduces a completely new way of doing instance data that, while having some advantages, has a very strange syntax and is utterly unlike how any OO code one finds in the wild is written. While the discussion of the problem that he's trying to solve is still interesting, I think the solutions he proposes will be more of a curiosity than a style that most programmers will adopt. But, those disagreements aside, I thoroughly enjoyed this book. I found it so engrossing that I kept reading it straight through a long train ride rather than switching to fiction halfway, like I do with most programming books. I've since recommended it to several other coworkers, and we're in the process of revising our internal Perl coding style based heavily on it. It's the sort of book that I expect to re-read several times to refresh my memory of Conway's analysis. If you've been avoiding this one due to misconceptions like I had, or because you think it might be too dry, or even just because you're afraid it's outdated, I strongly recommend it. If you're worried about the price of a printed (or electronic) book on this topic and you do Perl programming as part of your job, see if your employer will buy you a copy as training. It's worth it. I've been writing Perl professionally for more than fifteen years, and I learned a ton from this book. (And if it inspires you to change your coding style, take a look at perlcritic. It requires some configuration, and getting started is quite frustrating, but it reliably detects a surprisingly high percentage of the issues discussed here and can help a great deal in making the coding style shift stick until you can build new habits.) Rating: 9 out of 10

Matthew Garrett: Secure Boot distribution support

It's after Christmas, and some number of people doubtless ended up with Windows 8 PCs and may want to install Linux on them. If you'd like to do that without fiddling with firmware settings, here are your options.

comment count unavailable comments

27 December 2012

Russ Allbery: Charity review

I'm thankfully in a place where I can afford to give money to charity, and I've slowly built up a set of charities that I give money to. But I've never been that systematic about it. I've had a vague target for how much I want to give, I add new charities when they seem neat, I'll rarely drop a charity when it doesn't seem to be effective, and I probably spread my money out too much. Recently, I followed a chain of blog references and discovered GiveWell, a site that's devoted to doing more in-depth analysis of charities than the basic financial analysis done by sites like Charity Navigator. In particular, they do literature surveys to try to dig into the effectiveness of particular interventions and take a closer look at how effectively a charity can use new donations. After reading quite a bit on their site and some related sites, I sat down yesterday and did a comprehensive review of my goals and principles and then put together a new plan. Some of that may be interesting to other people, so I'll write up that process here. This will be somewhat US-centric, since charities vary a lot by tax regime, but I think some of the general principles will still apply elsewhere. (But most of my specific examples will be US charities.) First, a few basic principles about charitable giving:
  1. Charity goals cannot be evaluated uniformly. Different types of organizations need different criteria. I've found that charities divide generally into ones that are trying to accomplish a specific task and ones that are trying to create political change. For the former, small and quiet organizations are often the best. For the latter, the charity has to get to a certain size before it's part of the conversation, and that involves a different set of tradeoffs.
  2. When you care about efficiency (number of lives saved or quality of life improvement per dollar spent), don't give money to organizations you've heard about via means other than word of mouth or third-party evaluations. This is for a simple reason: if the charity has successfully made you aware of it, that means they're advertising themselves to you. Advertising generally costs quite a bit of money. That's money they're not spending on whatever they're trying to accomplish. Excellent examples are many of the health research and education charities, such as the American Heart Association and the American Lung Association, or the massive disaster relief organizations like the American Red Cross. These are not efficient organizations. If you're trying to get the most impact for your dollar, this isn't a good place to spend it. As a special case of this, please don't ever give money to any organization that produces TV commercials showing starving children in Africa. If the commercial makes you want to give money, that's great; just don't give it to the organization on TV. TV commercials are extremely expensive, and those organizations are notorious for being some of the least efficient charities in existence (in some cases bordering on scams). You'll be lucky if 50% of your money actually goes to Africa, and the money that is spent probably isn't spent wisely.
  3. If your donation gets you a membership that comes with a glossy magazine, remember that means that the organization is spending your money producing that magazine. This is counted as program funding, not administrative costs, in sites like Charity Navigator because the charity probably counts the magazine as part of their educational mission. If you want to subscribe to the magazine and enjoy it, there's not necessarily anything wrong with this. But if you were giving them money to do something else, be aware that money is being used to produce the magazine (and the Christmas cards and the mailing labels and the tote bag) instead of doing things that you might consider program funding. For example, I stopped giving money to the Audubon Foundation because I want my environmental giving to go directly to protecting the environment, not towards printing calendars and making a glossy magazine full of bird pictures. As a counter-example, I consider the Southern Poverty Law Center to be one of the most effective US political charities, despite the fact that they produce a glossy magazine. That's because a large part of their mission is investigative reporting and research, and that magazine is how they publish that research. In this case, I both want to read the magazine and consider the magazine something that I'm happy to fund.
  4. Efficiency isn't everything. For some types of charities, the charity has to be huge in order to be effective. This is particularly true of political charities. For example, the ACLU is not a particularly efficient charity and has several fund-raising practices that I dislike. However, what the ACLU does requires that they have extensive media access and respect from legislators and the judicial system. It cannot be done effectively by a smaller and more efficient organization. Similar principles apply to a lot of political and environmental charities, and to some international aid charities such as Doctors without Borders (M decins Sans Fronti res).
If you're looking to increase the efficiency and effectiveness of your giving according to objective measures, I highly recommend reading through GiveWell's site. They recommend charities that avoid all the pitfalls above. You've probably never heard of them because they don't spend their money on advertising and marketing. However, keep in mind that efficiency isn't everything. With those principles in mind, and after doing a lot of reading on GiveWell, I redid my charity plan by dividing my giving up into five categories with different goals. Services. There are some things that I use that I think should be paid for by tax dollars but aren't, and are instead funded by charitable donations. I don't consider this charity in the same sense as the rest of this list. Rather, I'm paying for what I use plus some extra so that other people who can't afford it can use the service for free. Efficiency doesn't matter as much because there's usually only one organization supporting the thing that I personally use. Examples include my local PBS (public broadcasting) station, the non-profit that works with city government to support the local library system, and the trust that helps maintain the state parks where I go on vacation. Education. This is a special category for me since I regularly give money to the community college I attended. I know the region and the college and have some idea of what they need and what matters to the students there, so I can make specific choices. I give all the money in this category to the same place. Free software. Here, I know that my donations won't be used as efficiently in terms of quality of life as they could be if sent somewhere else, but this is my community and I want to support it because it's my community. Here, as with education, I have a lot more data and can pick and choose places that I think can use my money. Examples here are the Ada Initiative and the Free Software Foundation. Politics. This is the hardest area for efficiency, since it's so difficult to measure effectiveness in any organization where much of the goal of the organization is to persuade politicians or the general public. I try to measure effectiveness by how many concrete actions the organization takes (rather than press releases), which is easiest for organizations like the ACLU or the EFF that file lawsuits or legal documents and requests. As mentioned above, I think the standout in this area is the Southern Poverty Law Center, but political charities are so targeted that I give to multiple charities to cover my range of interests. I've found environmental charities particularly difficult to evaluate, but the one that appears the most effective to me so far is the Environmental Defense Fund, so that's where I concentrate my environmental money. I used to give to small political charities but have given up on that as basically useless, at least for national politics. I think the charity has to be of a minimum size to get anyone in government to listen to them. Poverty and Health. This is where efficiency matters much more than size. GiveWell's analysis is very interesting here, and sadly shows that most interventions either don't work or at least can't be proven to work. Here, I abandoned nearly all of my traditional large charities and have gone almost entirely with GiveWell's recommendations, with two exceptions. I still cycle money through Kiva (a microfinance support charity) because I have a bit of an emotional attachment to it, although microfinance is looking increasingly suspect as an effective charity method and I may yet drop them. And I still give money to Doctors without Borders because, despite being less efficient, I think their size and history has won them an international credibility that lets them get into areas that smaller, more efficient charities wouldn't be able to help. One of GiveWell's recommendations is my new favorite charity: GiveDirectly. All aid charities run a serious risk of having a colonialist bent, where rich countries come into poor countries and build things or tell them how to do things following the priorities of the rich countries. (This is one of the reasons why I prefer medical charities, since they're less susceptible to this.) GiveDirectly identifies the poorest people in a region (using a very transparent process) and transfers money to them directly using the M-Pesa cell system to spend however they choose, with no strings attached other than some due diligence to protect against fraud. This is refreshingly non-paternalistic and makes me far more comfortable than typical aid projects. Originally, I was going to assign money into those buckets and then break the buckets down further, but I ended up not quite doing that. The services bucket is more driven by the services I use than by an even allocation. Of the rest, it goes approximately three shares to poverty and health, two to politics, and one each to education and free software. I think I've managed to eliminate most of the places where I'm giving money to multiple organizations that do the same thing, but I'm still spreading my money rather widely and could stand to focus it more. That's hard, though; there are a lot of different things that I care about, and political charities in particular aren't very interchangeable. I'm currently not giving locally very much, which might be worth changing. The easiest way to do so would be to find a good local food bank and start donating to them (giving cash, not food). For a more comprehensive list of charities I support, see my charities link page.

Vasudev Kamath: Note to Self: Telling apt to download specific architecture information only

If you have enabled multi-arch enabled in your system and one of the third party repository you use in /etc/apt/sources.list is not providing the foreign arch you have selected then fear not just do the below for that particular repository line.
deb [arch=arch1] uri distribution component
In my case mozilla.debian.net doesn't provide binary-armel hence I just did this
deb [arch=amd64] http://localhost:9999/mdn experimental iceweasel-aurora
and apt stopped complaining about missing foreign arch. Well this is present in man page of sources.list but I just wrote here so I can find it easily next time :-).

Jamie McClelland: Live video streaming using vp8enc and webm on Debian

The vp8 video codec and webm container format (which are alternates for theora/ogg and h264/flv) are getting lots of attention, especially as more web browsers are supporting html5 - which includes a new video tag that plays video directly in a web page, rather than requiring a plugin. With Google fully behind these new formats (and converting YouTube videos) and even skype using vp8, it seems as though it could supplant h264/flash as the default video codec and container. After years under the boot of the proprietary flash video, this change could usher in a lot of exciting developments. While the shift in formats is also significant for on-demand video, I'm focusing on live streaming in this blog. There are a lot of tools needed to successfully stream a live video. Enough of them support vp8 and webm to create a live video stream, however, not all of the right versions of these tools have landed in Debian. Without any patches or changes, if you try to send a live video stream to icecast using vp8 encoding and webm, you'll get something like this:
0 jamie@animal:~$ gst-launch v4l2src ! video/x-raw,width=320,height=240 ! vp8enc ! webmmux ! shout2send ip=icecast.server port=8000 password=secret mount=/test.webm
WARNING: erroneous pipeline: could not link webmmux0 to shout2send0
1 jamie@animal:~$
I've documented the steps for patching/upgrading gstreamer and icecast2 so you can get a live video stream using vp8 encoding and the webmc container that originates on a Debian wheezy machine using gstreamer and streams to a Debian squeeze server running icecast2. On the workstation, you'll need libshout3 version 2.3.0 or higher, which is at the moment available in experimental and installable without bringing in any dependencies. In addition, you will need to bring in gstreamer1.0 from sid and you will need to rebuild gstreamer1.0-plugins-good after libshout-dev has been installed from experimental. So, begin by adding experimental and sid to your sources list by creating the files /etc/apt/sources.list.d/experimental.list and /etc/apt/sources/lists.d/sid.list with the respective contents:
deb http://mirror.cc.columbia.edu/debian experimental main
and
deb http://mirror.cc.columbia.edu/debian sid main 
deb-src http://mirror.cc.columbia.edu/debian sid main
In order to avoid upgrading your entire machine to sid (experimental packages won't upgrade by default), you'll also need to add the file /etc/apt/preferences.d/sid:
Package: *
Pin: release n=sid
Pin-Priority: 200
Then update and install:
apt-get update
apt-get install -t experimental libshout3
apt-get install gir1.2-clutter-gst-1.0 gir1.2-gst-plugins-base-1.0 gir1.2-gstreamer-1.0 gstreamer0.10-ffmpeg  gstreamer1.0-libav:amd64 gstreamer1.0-plugins-bad:amd64 gstreamer1.0-plugins-base:amd64 gstreamer1.0-plugins-base-apps gstreamer1.0-plugins-good:amd64 gstreamer1.0-pulseaudio:amd64 gstreamer1.0-tools gstreamer1.0-x:amd64 libclutter-gst-1.0-0:amd64 libgstreamer-plugins-bad1.0-0:amd64 libgstreamer-plugins-base1.0-0:amd64 libgstreamer-plugins-base1.0 libgstreamer1.0-0:amd64 libgstreamer1.0
Fortunately, the version of gstreamer1.0-plugins-good in sid includes a fix for a bug patched on November 31, 2012 that allows the use of the webm container. However, for it to work properly gstreamer1.0-plugins-good has to be built against libshout version 2.3.0 or higher (which is in experimental, not sid). Therefore, you have install libshout-dev from experimental and then rebuild gstreamer1.0-plugins-good.
sudo apt-get build-dep gstreamer1.0-plugins-good
sudo apt-get -t experimental install libshout-dev
apt-get source gstreamer1.0-plugins-good
cd gst-plugins-good1.0-1.0.4
dch -v 1.0.4-1+webmmux
fakeroot debian/rules binary
sudo dpkg -i ../gstreamer1.0-plugins-good_1.0.4-1+webmmux_amd64.deb
Now if you try the command, things should work on the client side. When streaming to an icecast2 server running squeeze, I don't get any errors, however, when opening the streaming page via the icecast URL, there is a play button, which can be clicked, but no picture is shown. That's because the version of Icecast2 in squeeze doesn't support either vp8 or webm. On the server side, you will need icecast2 version 2.4beta or higher. This version hasn't landed anywhere in Debian as of this writing. However, you can build it from source by following these steps (the patches I'm removing have all been applied to the new upstream version).
apt-get source icecast2
wget http://downloads.xiph.org/releases/icecast/icecast-2.4-beta.tar.gz
tar -xzvf icecast-2.4-beta.tar.gz
cp -r icecast2-2.3.2/debian icecast-2.3.99.0/
cd icecast-2.3.99.0/
rm debian/patches/ 1001_autotools_avoid_debian-subdir.patch,1002_add_missing_xspf_file,1003_fix_memory_leak.patch,series 
dch -v 2.3.99.0-1
fakeroot debian/rules binary
When you install it, be sure to replace the icecast2.xml configuration file and update it with your server-specific customizations. Hopefully this version of icecast2 will make it to experimental soon. And now... you can successfully stream via:
gst-launch-1.0 v4l2src ! video/x-raw,width=320,height=240 ! vp8enc ! webmmux !  shout2send ip=icecast.server port=8000 password=secret mount=/test.webm
If you want full audio and video, here's a more complete pipeline: gst-launch-1.0 v4l2src ! queue ! video/x-raw,width=320,height=240 ! tee name=tscreen ! queue ! autovideosink tscreen. ! queue ! videorate ! video/x-raw,framerate=25/2 ! queue ! vp8enc ! queue ! webmmux name=mux pulsesrc ! queue ! audioconvert ! vorbisenc ! queue ! queue ! mux. mux. ! queue ! shout2send ip=hobo port=8000 mount=jamie.webm password=secret Unfortunately, since I'm not sure how to deal with the python2.7 vs python3 dependecies, my elegant mobi python script that creates this pipeline for you using python-gtk as been reduced to a crude bash script. Another item on the todo list is opus audio encoding. gstreamer doesn't seem able to use opus in the webm container (although it seems to work with ogg). In the course of this research I found directions for getting gstreamer-1.0 installed on debian independent of packages installed on the system. If you follow the directions above, these steps should not be necessary. I'm including in case you need the latest sources for another reason. Extra steps I needed were:

Next.

Previous.