I have been asked for advice from several people recently on how to get started programming, or how to further develop a nascent interest in coding or software engineering. The people asking the questions range in age from about 10 years old to older than me. These are people that, for various reasons, are not very easily able to take computer science courses right now. One would think that, since I ve been doing this for somewhere around a quarter century (oh I do feel old now), that I d be ready to offer up some great advice. And offer some suggestions I have. But I m not convinced they re good ones. I have two main tensions. The first is that I, like many in the communities I tend to hang out in such as Debian s, have a personality that leads me to take a deep dive into details of anything that holds my interest. Whether it s Linux, Haskell, or amateur radio, I want to do more than skim the surface if I m having fun with it. Many people are not like that. They may have a lot of fun programming in Visual Basic, not really caring that other languages are out there. Or some people are not like this yet. I feel unqualified to provide good advice to people that are different from me in that way. To put it a different way: most people don t want to wait 4 years to be useful, and want to start out right away and get better over time (and I was the same way too.) The second is related. I learned programming at a time when, other than BASIC, interpreted languages were not really available to me. (Yes, they were available, but not to me.) I cut my teeth on BASIC, Pascal, and C. Although I rarely use C anymore, I can still drop into it at a moment s notice and be perfectly comfortable. I feel it was a fundamentally valuable experience, and that it would be very hard to become a great programmer without ever having lived and breathed something like C, where memory and pointers must be managed manually. Having said that, it is probably possible to become a good coder without ever having touched C. Here, then, is an edited version of some rambly advice I sent to someone recently, where learning OOP was particularly mentioned. I would welcome your comments and suggestions. I may point people that ask to this post in the future. For simply learning how to write code, Dive Into Python has long been a decent resource, though it may assume more experience than some have. I haven t read them myself, but I ve also heard good things about the How to Think Like a Computer Scientist series from Green Tea Press. They re all available as free PDF downloads, too! Eric S. Raymond s The Art of Unix Programming is another work I ve heard good things about, despite having never read it myself. A quick glance at the table of contents makes me think that even if people don t wind up working on Unix, the lessons and philosophy should be informative. It seems that many Computer Science programs are using Java for the core of their instruction, or even almost exclusively. Whether that is good or bad, I m not completely sure. It certainly gets people into OOP more deeply, but I m a right tool for the job kind of person. Despite the hype, OO like everything else isn t the right tool for every job. It is fine for people to dive straight into OO and become good programmers/engineers. However, I think it would be difficult to become a great programmer/engineer without ever having a solid understanding of a more low-level language, such as C in particular. I did my CS work when it was mostly based in C, and am glad for it. If someone never has to manage memory or pointers, I suspect they will be at a disadvantage in the long run for not being able to understand or work with the system at a more fundamental level. If a person knows C, plus some concepts of OO and Functional Programming (FP), it should be easy to pick up just about any other language out there. I used to think Python was a great first language, but during the 2.x series they added so much fluff and so many special cases that I m less enthusiastic now, though I don t know how much of that got cleaned up in 3.x. I am not too keen on Java as a first language, because too many things that should be simple aren t. I have a fondness for Haskell, and its close relationship to mathematics could make it a great first language or maybe a poor one, depending on your perspective. One other thing I think it s important for good programmers to have experience with all three major models of programming (procedural, OO, functional.) Even if a person winds up working mostly in one universe, knowledge of and experience with the others is important and informative and, in my experience, leads to better algorithms and architecture all around.

• Procedural languages: Obviously C, but also Unix shell
• OO languages: Python, Java, plenty of other fine choices
• Functional: Lisp, Scheme, Haskell (also the only lazy and pure language on this list)
Having said all that, more important than a choice of book or language is experience. I have heard people suggest that it takes 10,000 hours of practice to become a superstar at something, whatever that something is, and I wouldn t doubt it. Seth Godin discusses that a bit, with some criticism of the idea too. So that leads to the most important piece of advice: dive in to whatever your interest is. Experiment, write code, put theory into practice in a way that holds interest and excitement. People that try to do things they don t enjoy don t seem to stick with them as long or execute as well, and thus will never become great.

Last day of Hackweek VIII is over and I think Weblate is pretty much ready for 1.2 release. Today I continued to analyzing performance and did few optimizations of some frequently used code paths. There is still lot of area for improvement, but I think Weblate is now fast enough for daily usage on even quite big projects. Rest of the day was spend by looking at various questions which people have asked me and distilling that into documentation. The most extended part were frequently asked questions, which now includes some hints for setup. Of course the demo server is running latest and greatest git version, I will probably push it soon to http://l10n.cihar.com/ as well.

Filed under: English Suse Weblate 0 comments Flattr this!

The brief outage was due to a scheduled move of the servers to a separate rack and subnet dedicated to our work with the Center for Information Assurance & Cybersecurity (ciac) at the University of Washington Bothell (uwb), and a11y.com I am currently exercising the new (to us) equipment and hope to winnow the less than awesome equipment over the next quarter. I spent the last six months finding the best in breed of the surplussed DL385 and DL380 chassis we (work) were going to have recycled. The team and I were able to find enough equipment to bring up one of each with eight and six gigs of memory, respectively. These will make excellent hypervisors for provisioning embedded instances of Slackware, Fedora, RHEL, CentOS, Debian, FreeBSD, OpenSolaris, OpenIndiana, FreeDOS, etc. When I initially configured this xen paravirt environment, I failed to plan for integration with libvirt, so I am now re-jiggering the software bridges so that they are managed using virt-manager. My experience doing this in the Eastsound lab has proved to be quite rewarding. Once network, storage and memory allocation are handled by libvirt, it will be trivial to automate the process of deploying, reclaiming, and managing these Telecommunications Resources efficiently and effectively using Perl. The new equipment we racked over the last weekend all has hardware assisted virtualization and is capable of hosting operating systems which have not been patched to communicate with the host hypervisor. This provides a higher level of assurance that guests will receive the full amount of provisioned resources and reduces the likelihood of resource conflicts, should the cluster operations team mistakenly over-commit. It also makes it possible for these nodes to host Windows-based operating systems. Oh, and they will also host these not insignificant things called BIG-IP Local Traffic Manager Virtual Edition (VE) and BIG-IP GTM Virtual Edition.

The brief outage was due to a scheduled move of the servers to a separate rack and subnet dedicated to our work with the Center for Information Assurance & Cybersecurity (ciac) at the University of Washington Bothell (uwb), and a11y.com I am currently exercising the new (to us) equipment and hope to winnow the less than awesome equipment over the next quarter. I spent the last six months finding the best in breed of the surplussed DL385 and DL380 chassis we (work) were going to have recycled. The team and I were able to find enough equipment to bring up one of each with eight and six gigs of memory, respectively. These will make excellent hypervisors for provisioning embedded instances of Slackware, Fedora, RHEL, CentOS, Debian, FreeBSD, OpenSolaris, OpenIndiana, FreeDOS, etc. When I initially configured this xen paravirt environment, I failed to plan for integration with libvirt, so I am now re-jiggering the software bridges so that they are managed using virt-manager. My experience doing this in the Eastsound lab has proved to be quite rewarding. Once network, storage and memory allocation are handled by libvirt, it will be trivial to automate the process of deploying, reclaiming, and managing these Telecommunications Resources efficiently and effectively using Perl. The new equipment we racked over the last weekend all has hardware assisted virtualization and is capable of hosting operating systems which have not been patched to communicate with the host hypervisor. This provides a higher level of assurance that guests will receive the full amount of provisioned resources and reduces the likelihood of resource conflicts, should the cluster operations team mistakenly over-commit. It also makes it possible for these nodes to host Windows-based operating systems. Oh, and they will also host these not insignificant things called BIG-IP Local Traffic Manager Virtual Edition (VE) and BIG-IP GTM Virtual Edition.

With L szl B sz rm nyi we adopted mime-support, which was maintained since September 1996 by Brian White. It is the first package of standard priority that I take care of. It gives me the same feeling as when side wheels were removed from my bicycle, a long, long time ago. We transferred the source package on Alioth, and uploaded a first update allowing the extraction of information for the mailcap file directly from Desktop files. I think that the next two works will be to resolve the bug about the handling of PHP files, and to update the mime.types base, if possible in synergy with Fedora or other distributions.

The big news for ASoC in 3.5 is the first two changes here which are pretty major features for the subsystem:

• Dynamic PCM support, allowing support for SoCs with internal routing through components with tight sequencing and formatting constraints within their internal paths or where there are multiple components connected with CPU managed DMA controllers inside the SoC.
• Greatly improved support for direct digital links between off-SoC devices, providing a much simpler way of connecting things like digital basebands to CODECs.
• Much more fine grained and robust locking, cleaning up some of the confusion that crept in with multi-component.
• CPU support for nVidia Tegra 30 I2S and audio hub controllers and ST-Ericsson MSP I2S controllers
• New CODEC drivers for Cirrus CS42L52, Freescale MC13783, LAPIS Semiconductor ML26124 and Texas Instruments LM49453.

The lunch and dinner catering was outdoors which was novel. The food was generally good if a little limited for those of us with less straightforward dietary requirements.

Surprisingly my previous post on my Debconf trip has gathered more queries over its title than anything else.

Mostly my blog titles are, if I can manage it, relevant quotes or book titles. For the last post I used the title of a collection of Mark Twain scripts surrounding his trip through Nicaragua in 1866.

I have not been able to read the publication myself beyond the quoted excerpts in more modern articles because, somehow, almost 150 years after the words were written the only access to these words is to buy a very expensive and rare 1940 publication.

This appears to stem from the fact the original Clements scripts were simply not published before 1940 and hence appear to gain copyright from that date (IANAL I might be wrong here is the source I used) and thanks to the US government effectively making copyrighted works published after 1922 be "protected" forever I may never read it at all.

Rather sad really, but if you do have a copy I could borrow...oh no that is probably illegal too? better just let the words fade to dust eh?

This is one of the days where reading my everyday newspaper was worth more than just getting bitter at the news. I found this text in La Jornada, my usual newspaper. I liked it very much, and decided to translate it for a wider audience. Of course, if you can read Spanish, do yourself a favor and go to the original. It is not that the text is so easy to translate. And, after all, I'm not a native English writer. I'm trying to do a literal translation, even when disagreeing with the author. Ten theses in favor of free download of cultural goods on the Internet Enrique G. Gallegos Poet and philosopher. Currently a researcher in Universidad Aut noma Metropolitana-C

1. Historical legacy. Criticists who promote the persecution of piracy and free downloads of cultural works from the Internet argue that the patrimonial rights have been wronged; they start off suposing that a cultural work is formed from a historical void, as if there was a "nothing" to begin with, and then a "something" appeared. Nothing more fallacious: All cultural products has its precedents, and thanks to these it generates part of its best forces.
2. Opening towards the future. Human beings are projected into the future. As the main philosophies of the 20th century have explored, one of the singular characteristics of mankind is the ability to think and imagine the future. Products of culture are the best ways to think and project society, politics, love, hope, needs, failures A song, a poem, an essay, can trigger imaginary worlds with a transformative potential. To deny this to humanity with "patrimonial damage" as an argument is to mutilate man's temporal nature.
3. Recognition of the present. Cultural works generate moments of reflection, criticism and pleasure; they can trigger actions demanding commitment, solidarity and strong indignation against injustice. If every cultural work is adscribed in some way to a historic tradition, and if cultural products anre necessary to imagine other possible worlds and open our future, it's not less true that they also allow us to recognize our present by making us more sensitive to others' ideas, sensations and emotions. Without culture, there is no present nor others' presence.
4. Divulgation of the cultural heritage. If culture is "heritage of mankind", it should be also spread by any possible way. But not just spread, but actively sought for the biggest number of people to be able to effectively have access to it. In a potentially interconnected world, Internet is the most adequate world for it (despite its limitations). Therefore, penalizing free downloads means avoiding spreading it and restricting access to those groups who don't have enough economic resources.
5. Preserving cultural heritage. If we agree that cultural works are heritage of mankind, we need to take every action needed to preserve them. But preserving culture does not only mean keeping the products in museums, galleries or never-polluted drawers; it rather means to keep them in the collective memory and in the flow of constant interpretations and appropiations. Said in other words, the genuine way to preserve culture is to allow universal access to cultural goods. That's why, holding that free copies and downloads of books, music, videos, etc. over the Internet are harmful, is an incompatible argument with the obligation to preserve cultural heritage.
6. They are not goods. Cultural works are material and spiritual manifestations of mankind, concretions of its historicity; they are, also, expressions of emotions, ideas and projections of other lives and other worlds. That's why they cannot be equated to goods or inserted in patrimonialist logic. The market might want to cheat by giving a certain price to a painting by Orozco or auctioning a Baudelaire manuscript, but it will never be able to aprehend their true meaning as cultural works. And it's not that culture is not faluable, but taht its estimation criteria do not follow the rules of market, but those of the imponderable and unlimited.
7. Price disproportion. Assuming we accept the possibility that intermediaries charge for the services they perform, the value of a book, record or movie should never exceed the daily wage of a worker or employee. But this ellection would only be one more option within the effective possibility that people opt to download or freely copy the cultural work. The final decision as to which media to go to should be a sovereign resolution of the culture-interested person.
8. The principle of the most benefit. Even when copies and free downloads over Internet could generate a patrimonial "wrong" to third parties, the cultural benefit obtained by it will always be greater in as much that it carries out intensely the principle to foster, spread and access the cultural heritage of mankind. To think otherwise is to privilege the few over the many.
9. The real evil is elsewhere. Murder, women- and children-trade, poverty and misery are the real evils that ache mankind. According to the United Nations, in 2010 there were 468 thousand murders in the world; the estimation is that 3500 million people live in poverty; in their 2009 inform, the UN found over 2400 victims of "people trade", kidnapped as sex slaves; in some regions in Africa, close to 30% of children suffer acute malnutrition and six people (between children and adults) die every day from hunger. That's why pretending that free download of cultural works is an evil is clearly a tricky and deceptive speech
10. Against the intermediary-salesman. Culture does not need intermedieries that reduce cultural works to goods. Culture is too important to leavie it in the hands of salespeople that simplistly equate cultural goods with cakes. An intermediary of this kind will never understand the difference between a work of art and a disposable razor blade. What the world needs is more support from the government from every country to artists, creators and poets, as well as ensuring the conditions for the absolute mobility freedom for cultural works.

A couple week ago, during the last debconf (Debian Conference) in Managua/Nicaragua, I presented the latest developments about the inclusion of clang in the Debian architecture. To sum up (details are available in the slides and the video), the rebuild of the Debian archive with Clang 3.1 increased the number of failures from 8.8 to 12.1%. The main reason is that further checks have been added to clang. With Paul Tagliamonte as co-mentor and Alexander Pashaliyski as a GSoC student, we made great progress in bringing Clang as A new compiler in the Debian infrastructure. The various feedbacks that I had during Debconf were pretty good. It interests many people for reasons like Quality Assurance (more checks), performances, hackability or to decorrelate Debian and GCC. Build Debian with another compiler - Slides
Build Debian with another compiler - Video A nice resume of my talk has been made by Michael Larabel on Phoronix:
Decoupling GCC From Debian By Using LLVM/Clang

As I write this, DebConf12 has actually been over for more than a week. Usually during DebConf, I'm a more proficient blogger than during other times of the year, since I'm not doing as much work that I have to remain quiet about at that time, which simply means I have more to blog about. Unfortunately, however, this year the server on which this blog runs managed to become unavailable on the second day of DebCamp, an issue that I could only fix once I was back in Belgium, which meant I haven't been able to blog as much as I would've wanted to. Anyway, all I can say is that both DebConf and DebCamp have been a huge success for me again, this year. I'd started the second rewrite of ipcfg a few weeks before DebCamp, and spent most of DebCamp whipping the basic framework into shape. It's nowhere near ready yet, but I'm confident it'll get further this time than it did before. During DebConf, I was also responsible for doing the group photo. I'd done that (together with Andrew McMillan) during DebConf8 before (if you look closely, you'll see me pressing the remote which actually triggers that picture), and I still like that picture. This time around, we didn't have a beach nearby (at least not as nearby as during DebConf8), so we had to do things slightly differently. Rather than doing one large picture, I took several pictures and used Hugin to stitch them together. Unfortunately however, some things did go wrong with the group photo. For starters, the weather wasn't very helpful, in that I noticed it was about to rain. As such, the main bunch of pictures were taken slightly before the announced time of 14:30, which meant that some people had missed it. To make matters worse, I had rushed to the group and handed the camera to someone else, so that I could gimp myself onto the picture too; but that meant that some people who were arriving as these pictures were being taken thought they were on the picture, when they really weren't. I've fixed most of these by just gimping them into the picture as well, but I might've missed some of them. If you're one of them, talk to me! Preferably now. If you're not one of them (or even if you are), hold off on doing things like printing it in large format or some such. I'll re-upload improved versions onto the same location once they're ready.

A couple week ago, during the last debconf (Debian Conference) in Managua/Nicaragua, I presented the latest developments about the inclusion of clang in the Debian architecture. To sum up (details are available in the slides and the video), the rebuild of the Debian archive with Clang 3.1 increased the number of failures from 8.8 to 12.1%. The main reason is that further checks have been added to clang. With Paul Tagliamonte as co-mentor and Alexander Pashaliyski as a GSoC student, we made great progress in bringing Clang as A new compiler in the Debian infrastructure. The various feedback that I had during Debconf were pretty good. It interests many people for reasons like Quality Assurance (more checks), performances, hackability or to decorrelate Debian and GCC. Build Debian with another compiler - Slides
Build Debian with another compiler - Video A nice resume of my talk has been made by Michael Larabel on Phoronix:
Decoupling GCC From Debian By Using LLVM/Clang

At work, we have a rotation of who is on call at a given time. We have few calls, but they do happen and so it's important to ensure both that a person is available, but also that they're aware they are on call (so they don't stray too far from their phone or a computer). In the grand tradition of abusing spreadsheets, we are using google docs for the roster. It's basically just two columns, one with date and one with user name. Since the volume is so low, people tend to be on call for about a week at a time, 24 hours a day. Up until now, we've just had a pretty old and dumb phone that people have carried around, but that's not really swish, so I have implemented a small system which grabs the current data, looks up the support person in LDAP and sends SMSes when people go on and off duty as well as reminding the person who's on duty once a day. If you're interested, you can look at the (slightly redacted) script.

Giving away distromatch at last year's Fosdem I tried to inject a lot of energy into distromatch but shortly afterwards I've had to urgently rewrite the nm.debian.org website. After Lars Wirzenius GTDFH talks in Bologna and Varese I wrote a tool which, among other things, is able to scan my home dir and list how many projects I'm working on. The output was scary. Like, they are too many. Like, I couldn't even recite the list out of memory. And since I couldn't do that, I had no idea there were so many. And I kept being stressful because I couldn't manage to take care of them all properly. Now that I became conscious of the situation, it's time to deal with it like a grown up, and politely back off from some of my irresponsible responsibilities. Distromatch is one of them. It had just started as a proof of concept prototype, and I had the vision that it could be the basis for a fantastic culture of sharing and exchange of information across distributions. I need to distinguish the vision from the responsibility. I still have that vision for distromatch, but I cannot take responsibility for making it happen. So I am giving it up to anyone who has the time and resources to pick up that responsibility. Current status It works well enough as a prototype. I believe it can successfully map a large enough slice of packages, that one can prototype stuff based on it. I have for example used it to export the Debtags categories for other distros, and the resulting file looked big enough to be used for prototyping category-based features on distributions that don't have them yet. I think it also works well enough to support a few common use cases, like sharing screenshots, or doing most of the work of converting dependency lists from a distro to another. And finally, anyone can deploy it, and work on it. Existing data sources Everything I index in the Debian distromatch deployment is available at http://dde.debian.net/exports/distromatch/. The rpm-based data in there comes from an export script I wrote that runs on Sophie, but which I cannot maintain properly. This is an experimental export of Fedora and OpenSUSE data: http://tmp.vuntz.net/misc/distromatch/distromatch-opensuse-fedora.tar All existing export scripts are found in distromatch git repo on gitorious. Contacts I gathered at Fosdem At Fosdem I devoted quite some work to get contacts from all possible distributions and software repositories, so that distromatch could be hooked into them. Here is a dump of what I have collected:

• Debian: me
• OpenSuse: Vincent Untz and Adrian Schr ter
• Fedora: Tom "Spot" Callaway
• Arch: Tasser on IRC
• CPAN: contact the people of https://metacpan.org/, on irc.perl.org:#metacpan or make an issue on github
• NetBSD: ask on #netbsd on Freenode
• FreeBSD: Baptiste Daroussin (bapt)
• Mageia: Olivier Thauvin

Some of those contacts may have "expired" in the meantime: I wouldn't assume all of them still remember talking with me, although most probably still do. My commitment for the time being I am happy to commit, at the moment, to maintaining a working data export for Debian data. I can take responsibility for making it so that the Debian data for it stays up to date, and to fix it asap if it isn't the case. I hope that now someone can take distromatch over from me, and make it grow to achieve its great potential.

I can't remember exactly when, it was either pre-Zoe heart-related diagnostic imaging or during her pregnancy with Zoe, that it became apparent that Sarah had gall stones. Apparently it's not uncommon for women to develop them after pregnancy, and it turns out that Sarah is slightly more genetically predisposed to getting them to boot. So it wasn't terribly surprising when she started having some pain recently. She had another ultrasound to confirm it, and went off to see a surgeon. Apparently they care more about the symptoms than the number of stones in the gallbladder, and they don't bother removing the stones and keeping the gallbladder, so she was booked in for a cholecystectomy last Thursday. It was a pretty straightforward procedure, she was out of the operating room in just under an hour, and awake a bit over an hour after that. I've had a few friends need to have a cholecystectomy, and the photos on Wikipedia have always fascinated me, particularly this one. It seems so freaky to have a gallbladder full of that much gravel. It can't be comfortable. We're going to be able to pick up Sarah's stones on Monday, so it'll be interesting to see what they look like. Sarah's recovering now. It's one of the few surgeries she's had under general anaesthesia where she hasn't been sick afterwards, so that was a good start. The procedure was done laparoscopically, using four incisions, and to give themselves room to work, they inflate the body with carbon dioxide gas, which then has to work its way out of her body over the following days. So she's dealing with some bloating, bruising and swelling at the moment, but not doing too badly. Apparently if you're vertical for too long, the gas can work its way up to your diaphragm, which causes some shoulder pain, so the trick has been to walk around a bit, then lie down for a bit, and then rinse and repeat. She's not allowed to lift Zoe for 3-4 weeks, which presents a few challenges for us, but nothing insurmountable (hopefully). We converted Zoe's crib to a toddler bed a week before the surgery, and Zoe thought that was just the best thing ever, and has taken to it really well. I'm around at home for breakfast and dinner, so I can take care of the lifting into and out of the high chair for those meals, and Zoe can eat lunch at her little table. Zoe's also pretty good about getting into and out of the car by herself (not that Sarah can drive until she's off the painkillers) and she's normally in daycare two days a week. She's in daycare again on Monday, so Sarah won't have to deal with Zoe on her own until Tuesday, and we have backup daycare arrangements we can fall back on if need be. Zoe's been very good about Sarah not being able to pick her up, and been very gentle. It helped with the explanation that Sarah could show her all of her "owwies". The surgeon was saying that only 1 in 400 people need to alter their diet after removing their gallbladder, so we're hopeful that Sarah will do just fine without it. She'll certainly be more comfortable in the long run.

In this post I ll make a quick introduction by example on how to use Zeitgeist s Python API for good and profit. If you re interested in using Zeitgeist from C instead, see the libzeitgeist examples; to use it with C++/Qt, Trever s a web browser in 4 steps may be of interest. First things first In case you re not familiar with Zeitgeist, it may prove helpful to first read Mikkel s introduction to Zeitgeist post. If that s too much to read, just should know that Zeitgeist is an event log. Like the history in your browser, it keeps track of what websites you open at which point in time. It also keeps track of when you close them, and of what browser you used, since it s a system-wide service. Furthermore, it does the same for files, conversations, e-mails, and anything else you want to insert into it. So Zeitgeist is a database of events, and an event can be pretty much anything. But what does it look like? It s main attributes are the following:

• timestamp when did the event happen (milliseconds since Unix epoch)
• interpretation what sort of event is it (eg. opened, closed)
• manifestation why did it happen (user activity, notification )
• actor which is the primary application involved
• origin where did it come from (eg. website where you clicked the link that opened this page)

Additionally, each event has one or more subjects, which have the following attributes:

• uri
• current_uri updated URI if it changed since the event
• interpretation abstract type (document, image, video )
• manifestation how it is stored (file, remote object, website)
• origin parent folder for files, domain name for websites
• mimetype
• text a title for the event (eg. filename, website title )
• storage identifier for the storage medium of the subject (eg. local, online, pendrive X)

Retrieving recent data Okay, so let s say you want to know the last song you ve listened to (if you have a Zeitgeist-enabled music player). It s a simple as: This may need some explaining. We start by importing the Zeitgeist client and some associated data structures (such as Event and Interpretation), and create an instance of the client. The ZeitgeistClient class is a wrapper around Zeitgeist s D-Bus API which not only makes it much nicer to use, but also makes it easy to install monitors (as we will see later) and provides convenient functionality such as automatic reconnection if the connection to the engine is lost. To query for the most recent song, we just need to create a template with the restrictions we want to impose and submit it to Zeitgeist using the find_events_for_template call. If you haven t read Mikkel s post yet, please do so, as it introduces the structure of events and subjects (in short: an event has a timestamp, some other properties, and one or more subjects representing the resources -files, websites, people - involved in the event). The Python API is inherently asynchronous (if for some reason you need a synchronous API, you may still use the lower level ZeitgeistDBusInterface class), so we need to define a callback function to handle the results we receive from the Zeitgeist engine. Finally, we need to create a main loop so the asynchronous functions can run.

---

Now this was a pretty simple example. Let s make it more interesting. One song isn t much, so let s get the 5 most recent songs. Also, now we want both songs and videos. The first part is pretty easy, we just need to change the num_events parameter. For the second extension, we have to change the event template. In fact, now we need two different event templates and the find_events_for_templates function, which takes an arbitrary number of event templates and ORs them. The result is as follows: This will work, but unless you re lucky you re likely to get some duplicate line. Why is this? Well, other than that you may have used the same file twice, don t forget that what you are requesting are actually events. If you ve started playing a given song, you probably also stopped playing it, so that s actually two of them (an AccessEvent and a LeaveEvent). Since this isn t what we want, we ll change the query a bit: By requesting the most recent subjects, vs. the most recent events, we can filter out events with duplicate URI. See the ResultType documentation for other modes you can use. Note particularly the MostPopularSubjects result type. I also used the chance to introduce the storage_state parameter. This one will filter out events for files Zeitgeist knows aren t available (this mostly means online resources won t be shown if you don t have a network connnection; there s also support for handling external storage media, but because of problems with GIO this is currently disabled). Last but not least, the find_events_for_* methods also accept a timerange parameter. It defaults to TimeRange.until_now(), but you may change it to TimeRange.always() (if for some reason you re working with events in the future) or to any other time range of your choice. Here it s important to note that Zeitgeist s timestamps use millisecond precision.

---

For more advanced queries, you can use more complex combinations of events and subject templates. The rule to keep in mind here is that events are OR d and subjects are ANDed. Additionally, some field (actor, origin, mimetype, uri and current_uri) may be prefixed with an exclamation mark ( ! ) for NOT, or you may append an asterisc ( * ) to them for prefix search. You can even combine the two operators together. Here s an example of a template you could build: In my case, this template would fetch a list of the source code files I modified most recently but excluding those related to the Zeitgeist project. Working with big sets of data In case you re trying to do something crazy, you may end up with a Zeitgeist query complaining that it exceeded the memory limit. You re not supposed to do that. Instead, we provide some methods for working with large collections of events. And there you have the source code files you worked with during the last 3 months, ordered from most to least popular (popularity is measured counting the number of events; for more precision, maybe you could limit the results to events with interpretation AccessEvent). Why do we provide this mechanism instead of querying with a simple offset? Well, this avoids problems when the log changes (events are inserted or deleted). Have you ever been exploring the latest posts in some website, and as you change to the next page some of the results from the previous page show up again (because new posts have been added in the meantime)? With Zeitgeist this won t happen. Receiving information in real time At this point you re an expert at requesting all sorts of data from Zeitgeist, but now you want to show a list of the last kitten images you ve viewed, updated in real time. Don t worry, Zeitgeist can provide for this: It s important to note that on_delete won t be called when an image is deleted (that d be a newly inserted event with interpretation=DELETE_EVENT); rather, it s called when a previously inserted event is deleted (for example, using the forget recent history option in Activity Log Manager). In case you re curious: for best performance, this doesn t actually use D-Bus signals. Instead, this little call will setup a D-Bus object behind the scenes and register it with the Zeitgeist engine, so it can notify said object when (and only when) an event of its interest is registered. To stop receiving notifications for a template, you ll need the save the object returned by the install_monitor call: Pro Tip: You can use the Zeitgeist Explorer GUI to quickly try out different queries (note: it s still work in progress, so much funcionality is missing, but it does work somewhat). Contextual awesomeness: finding related events By now you re familiar with retrieving events and keeping them up to date. Now it s time for a little secret: This little query example will return up to 10 websites I used at the same time as the Vala files inside my Zeitgeist directory, considering only data from the last 6 months. Nice, huh? This is an experimental feature, and it doesn t work well when operating on big inputs, so it s usually better to use the find_related_uris_for_uris variant (which replaces the first query_templates parameter with a list of URIs). Advanced searching: the FTS extension Some people think prefix searches aren t good enough for them, and this is why the Zeitgeist engine ships by default with a FTS (Full Text Search) extension. Using the methods provided by this extension you can perform more advanced queries against subjects current_uri and text properties (unlike the name may suggest, the FTS extension doesn t index the content of the files, but just the information in the event). This is exposed as zeitgeist_index_search in libzeitgeist (the C library), but unfortunately isn t currently available in the Python API. If you still need it, you ll have to fallback to pretty much using the D-Bus interface (you still get reconnection support, though). Here s an example: The most interesting thing here is the query parameter. Quoting from the C documentation:

The default boolean operator is AND. Thus the query foo bar will
be interpreted as foo AND bar. To exclude a term from the result
set prepend it with a minus sign - eg foo -bar. Phrase queries
can be done by double quoting the string "foo is a bar". You can
truncate terms by appending a *.
There are a few keys you can prefix to a term or phrase to search
within a specific set of metadata. They are used like key:value.
The keys name and title search strictly within the text field of
the event subjects. The key app searches within the application
name or description that is found in the actor attribute of the
events. Lastly you can use the site key to search within the
domain name of the subject URIs.
You can also control the results with the boolean operators AND
and OR and you may use brackets, ( and ), to control the operator
precedence.

Modifying the log So far we ve only queried Zeitgeist for information, let s get a bit more active. You can delete events from Zeitgeist with the following query: The confirmation callback will receive a timerange going from the first to the last event. If no events were deleted (because they didn t exist), you ll get (-1, -1).

---

And now for the interesting part. If your application involves resources (files, websites, contacts, etc.) of any sort, you ll probably want to let Zeitgeist know that you re using them. It s time that you write a data-source! We start by registering the data-source. Here we go: Once that s done (and if it is enabled), we are free to send our events: If you don t know what interpretation and manifestation your subject should have, you can use the following utility methods: Event better, with Zeitgeist 0.9 you can just leave the subject (but not event!) interpretation and manifestation fields empty, and they ll be guessed the same way as if you used those utility methods. Pro Tip: You can examine all registered data-sources and toggle whether they are enabled or not using the zeitgeist-data-sources-gtk.py tool. Conclusion Wow, I m impressed if you ve got this far. By now you should have quite a good idea on how to use the Zeitgeist API, and I m looking forward to seeing what you do with it in your next awesome project. If you have any problem with Zeitgeist, feel free to visit us on IRC (#zeitgeist on irc.freenode.net), or join our mailing list. We ll also be at GUADEC next week, so if you re there make sure to say hi! In case you missed them, here are some useful links:

• Python API documentation
• C API documentation
• Zeitgeist project website
• Meta-project on Launchpad
• Bug tracker on FreeDesktop.org

---

No comments
Siegfried-Angel Gevatter Pujals, 2012. Permalink License Post tags: gnome, zeitgeist

That's a seemingly simple question I started asking various people two weeks ago. I didn't get many useful answers, but now I have experience doing it myself, and so here's a blog post brain dump. I have been trying to convert git-annex to use GHC's threaded runtime, for a variety of reasons. Naively adding the -threaded option resulted in a git-annex command that seemed to randomly freeze, but only sometimes (and, infuriatingly, never when I straced it), and a test suite that froze at a random point almost every time I ran it. Not a good result, and lacking any knowledge about gotchas with using the threaded runtime, I was at a loss for a long time (most of DebConf) about how to fix it. I now know of at least three classes of problems that enabling the threaded runtime can turn up in programs that have been developed using the non-threaded runtime. accessing a MVar after forkprocess can hang MissingH has some code similar to this, which works ok with the non-threaded runtime:

forkProcess $ do
    debugM "about to run some command"
    executeFile ...

In the above example, debugM accesses a MVar. Doing that after forkProcess can result in a MVar deadlock, as it tries to access a MVar value, that is, apparently, not accessible to the forked process. (Bug report with test case) So, using System.Cmd.Utils from MissingH is asking for trouble. I switched all my code to the newer and, apparently, threaded runtime safe System.Process. forkProcess is a massively bad idea Even when not accessing a MVar after forkProcess, it's very unsafe to use. It's easy to miss the warning attached to forkProcess, when the code seems to work. But with the threaded runtime, I've found that most any call to forkProcess will sometimes succeed, and sometimes freeze the whole program. This might only happen around 1 time in 1000. Then you'll find this warning and do a lot of head-scratching about what it really means:

forkProcess comes with a giant warning: since any other running threads are not copied into the child process, it's easy to go wrong: e.g. by accessing some shared resource that was held by another thread in the parent.

The hangs I saw could be due to laziness issues deferring code to run after the forkProcess that you'd expect to have run before it ... or who knows what else. It's not clear to me that it's possible to use forkProcess safely in Haskell code. I think it's notable that System.Process runs the whole fork/exec in C code instead. unsafe FFI calls block According to most of the documentation you'll find in eg, the Haskell wiki, Real World Haskell, etc, the only difference between the safe and unsafe imports in the FFI is that unsafe is faster, and shouldn't be used for C code that calls back into Haskell code. But the documentation is out of date. Actually, if you're using the FFI, and the foreign function can block, you need to use safe. When using unsafe, a blocking foreign function can block all threads of the program. In my case, I was using kqueue to wait for changes to files, and this indeed blocked my whole program when linked with -threaded. Marking it safe fixed this. The details are well described in this paper: http://community.haskell.org/~simonmar/papers/conc-ffi.pdf Somewhat surprisingly, this blocking only happens when using the threaded runtime. If you're using the non-threaded runtime with unsafe blocking FFI functions, your other pseudo-threads won't be blocked. This is because the non-threaded runtime has an SIGALARM timer that interrupts (most) blocking system calls. This leads to other troubles of its own (like needing to restart interrupted FFI functions, or blocking the other pseudo-threads from running if the C code ignores SIGALARM), but that's offtopic for this post. summary Converting a large Haskell code base from the default, non-threaded runtime to the threaded runtime can be quite tricky. None of the problems are the sort of thing that Haskell helps to manage either. I will be developing new programs using the threaded runtime from the beginning from now on. By the way, don't take this post to say that threading in Haskell sucks. I've really been enjoying writing threaded Haskell code. The control Haskell gives over isolation of state to threads, and the excellent and wide-ranging suite of thread communications data types (MVar, Chan, QSemN, TMVar, SampleVar, etc) have made developing a complex threaded program something I feel comfortable doing for the first time, in any language.

Panasonic's G1, G2 and G3 micro four-thirds cameras are missing a dedicated orientation sensor. With a lens featuring image stabilisation the camera makes use of its sensor. Otherwise (e.g. with the Panasonic Lumix 20mm/1.7) the Exif image orientation tags are not correct for portrait format images. Here is how to write correct tags (compatible with the camera's preview function) with exiv2: Camera rotated anti-clockwise (shutter button on top):
exiv2 -k -M"set Exif.Image.Orientation 8" -M"set Exif.Panasonic.Rotation 8" -M"set Exif.Thumbnail.Orientation 8" P1020800.JPG Camera rotated clockwise (shutter button at bottom):
exiv2 -k -M"set Exif.Image.Orientation 6" -M"set Exif.Panasonic.Rotation 6" -M"set Exif.Thumbnail.Orientation 6" P1020800.JPG

A random tip for shell script hackers. You know that with set -x you can turn on tracing, so that every command is printed before being executed. In bash, you can also customize the output prefix by setting the PS4 variable. The default is PS4='+ '. Here is an example. I wanted to "profile" a deployment script, to see why it took so long. Ordinarily, I might have sprinkled it with date calls. Instead, I merely added

set -x
PS4='+\t '

near the top. \t stands for time stamp. (The script was already using bash explicitly, as opposed to /bin/sh.) That way, every line is prefixed by a time stamp, and the logs could easily be analyzed to find a possible performance bottleneck.

Once again Debian day was somewhat compromised: this time by more DIY. I still managed a few bits and pieces, amortized over the rest of the week. I benefited from some train time this weekend. A new version of flactag and catching up the flactag packaging vcs repository. A maintainer endorsement for one of the flactag maintainers and a request to NM for another maintainer to be granted access to the collab-maint repository. More work on the musicbrainz 'transition': First, usertag tracking of packages to update, with upstream bugs where necessary. Tested and confirmed Michael Biebl's sushi patch; found and fixed a sushi bug along the way; wrote patches for sound-juicer ( merged upstream) and goobox.