Search Results: "ron"

19 March 2012

Petter Reinholdtsen: Debian Edu interview: John Ingleby

Debian Edu / Skolelinux users are spread all across the globe. The second inteview after the Squeeze release was publised is with John Ingleby, a teacher and long time Linux user in United Kingdom. Who are you, and how do you spend your days? I teach ICT part time at the Rudolf Steiner School in Kings Langley, near London, UK. Previously I worked as a technical author/trainer while my children attended the school, and I also contributed to the Schoolforge UK community with the aim of encouraging UK schools to adopt free/open source software. Five or six years ago we had about 50 schools interested in some way, but we weren't able to convert many of them into sustainable installations. How did you get in contact with the Skolelinux/Debian Edu project? Skolelinux had two representatives at an early Edubuntu meeting in London which I attended. However at that time our school network had just been installed using CentOS, LTSP 4 and GNOME. When LTSP 5 came along we switched to Edubuntu thin client servers so now we have a mixed environment which includes Windows PCs and student laptops, as well as their MacBooks and iPads. However, the proprietary systems have always been rather problematic, and we never built a GUI for the LDAP server, so when I discovered Skolelinux is configured for all these things we decided to try it. What do you see as the advantages of Skolelinux/Debian Edu? By far the biggest advantage is the Debian Edu community. Apart from that I have always believed in the same "sustainable computing" goals that Skolelinux is built on: installing Linux on computers which would otherwise be thrown away, to provide a reliable, secure and low-cost IT environment for schools. From my own experience I know that a part-time person can teach and manage a network of about 25 Linux computers, but it would take much more of my time if we had proprietary software everywhere. What do you see as the disadvantages of Skolelinux/Debian Edu? As a newcomer I'm just finding out who's who in the community and how you're organised, and what your procedures are for dealing with various things such as editing manual pages and so-on. The only English language mailing list seems to be for developers as well as users, so my inbox needs heavy pruning each day! Which free software do you use daily? Besides the software already mentioned at school we use Samba, OpenLDAP, CUPS, Nagios and Dansguardian for the network, and on the desktops we have LibreOffice, Firefox, GIMP and Inkscape. At home I use Ubuntu and an Android 4 eePad Transformer (but I'm not sure if that counts...) Which strategy do you believe is the right one to use to get schools to use free software? That's a tough question! For very many years UK schools installed and taught only proprietary software, so that at the highest levels the notion of "computer" means simply "proprietary office applications". However, schools today are experiencing budget constraints, and many are having to think hard about upgrading Windows XP. At the same time, we have students showing teachers how to use iPads, MacBooks and Android, so the choice of operating system is no longer quite so automatic. What is more, our government at last realised that we need people with programming skills, so they're putting coding back in the curriculum! And it's encouraging that the first 10,000 Raspberry Pi units sold out in 2 hours. I don't really know what strategy is going to get UK schools to use free software, but building an active community of Skolelinux/Debian Edu users in this country has to be part of it.

Benjamin Mako Hill: A Manhattan Project for Clich Collection

This weekend, I launched an extremely ambitious effort to collect evidence of extremely ambitious efforts. The result was a short program that searched the web and revealed the: [Previously in this series: frailty, the invisible hand, science as dance.]

18 March 2012

Aurelien Jarno: 10 years ago

Date: Mon, 18 Mar 2002 18:22:10 +0000
From: James Troup <troup@samosa.debian.org>
To: Aurelien Jarno <aurelien@aurel32.net>
Cc: da-manager@debian.org
Subject: New Debian maintainer Aurelien Jarno [ This is a long (automatically-generated) mail, but it contains
important information, please read it all carefully. ] Dear Aurelien Jarno! An account has been created for you on developer-accessible machines with username aurel32 . The password for this account can be found encrypted with your PGP or GPG key and appended to this message. A list of machines available to Debian developers can be found at <URL:http://db.debian.org/machines.cgi>. Please take a minute now to familiarize yourself with the Debian Machine Usage Policy, available at <URL:http://www.debian.org/devel/dmup> You have been subscribed to the debian-private mailing list as <aurel32@debian.org>. Please respect the privacy of that list and don t forward mail from it elsewhere. E-mail to <aurel32@debian.org> will be forwarded to <aurelien@aurel32.net>. To change this, please see <URL:http://db.debian.org/forward.html> Also, please subscribe to debian-devel-announce, if you haven t done so already. We strongly suggest that you use your aurel32@debian.org address for the maintainer field in your packages, because that one will be valid as long as you are a Debian developer, even if you change jobs, leave university or change Internet Service providers. If you do so, please add that address to your PGP/GPG key(s) (using gpg edit-key YOUR USER ID ) and send it to the keyring server at keyring.debian.org with gpg keyserver keyring.debian.org send-keys YOUR USER ID . You can find more information useful to developers at <URL:http://www.debian.org/devel/> (in particular, see the subsection titled Debian Developer s reference ). We suggest that you subscribe to debian-mentors@lists.debian.org. This list is for new maintainers who seek help with initial packaging and other developer-related issues. Those who prefer one-on-one help can also post to the list, and an experienced developer may volunteer to help you. You can get online help on IRC, too, if you join the channel #debian-devel on irc.debian.org. Take a look at the support section on www.debian.org in order to find out more information. You should have read these documents before working on your packages. o The Debian Social Contract
<URL:http://www.debian.org/social_contract.html> o The Debian Policy Manual
<URL:http://www.debian.org/doc/debian-policy/> If you have some spare time and want to contribute it to Debian you may wish to take a look at the Work-Needing and Prospective Packages for Debian GNU/Linux also known as WNPP that can be found at <URL:http://www.debian.org/devel/wnpp/> If you plan to make a Debian package from a not yet packaged piece of software you *must* announce your intention on the debian-devel mailing list to make sure nobody else is working on them. The machine ftp-master.debian.org is our main archive server. Every uploaded package finds it s way there (except for Packages covered by US crypto laws which go to non-us.debian.org) eventually. master.debian.org is the home of our bug tracking system. Project web pages and CVS archives are hosted on klecker.debian.org (aka cvs/www.debian.org), klecker is also our general shell server. Web pages should be placed in public_html on klecker and refered to by http://people.debian.org/~aurel32 You should use ssh to log into the machines instead of regular telnet or rlogin. Our LDAP directory is able to share ssh RSA keys among machines, please see <URL:http://db.debian.org/doc-mail.html> Otherwise when you first login a ~/.ssh directory will be created with the appropriate permissions. Please be aware of the security implications of using RSA authentication and ssh agents. Finally, please take a minute to visit <URL:http://db.debian.org/>.
Login using the password information appended to this email, and update your personal information. The information is used to maintain your accounts on various Debian machines, and also to allow other developers and general users to find out more about you. Many of the fields are only visible to other registered Debian developers. This is also the only way to change your password. The passwd program does not yet work. Welcome to the project!
The Debian New Maintainer Team

Charles Plessy: Debian-installer in a cloud.

Debian Med prepares a metapackage for multi-purpose image machines, containing packages for bioinformatics that can be used in command line or via scripts, and that do not depend on too many other packages. I would like to prepare such an image for the Amazon cloud in the most secure manner, and the simplest way is to build it automatically in order to not have to clean anything at the end of the process. I tried for months so use the Debian installer, but banged my head against walls repeatedly, as it was impossible to re-partition the volume on which the installer was started. In the end, I figured out that this was not necessary. For instance, one can start an image containing the installer on a micro instance (-t t1.micro), with an additional volume of one gibibyte (-b /dev/sdb=:1:false) to install Debian on, and preseed the installer via instance metadata (-f preseed.txt) using a file prepared in advance. When the installation finishes, the instance terminates instead of stopping (--instance-initiated-shutdown-behavior terminate), and its volumes disappear, except the one where Debian was installed (/dev/sdb=:1:false). Debian-installer in Stable is not often updated, and its size is very small. One can therefore think of releasing one machine image per zone and architecture. I did so for T ky (ap-northeast-1) on amd64. It contains the kernel, its initrd, and a GRUB 1 menu for pvgrub, that passes the following options: console=hvc0 auto=true priority=critical url=http://169.254.169.254/latest/user-data DEBIAN_FRONTEND=text. Two key pieces are missing in the resulting system. When the kernel is installed or updated, the GRUB 1 configuration file for pvgrub must be refreshed. Also, the system must be able to retrieve a public SSH key provided through the instance metadata, to allow one to log in without using a predefined password. These two functions are provided by the cloud-init package, available in Ubuntu and Fedora. I am looking for volunteers to maintain or co-maintain cloud-init in Debian.

Soeren Sonnenburg: Shogun got accepted at Google Summer of Code 2012

SHOGUN has been accepted for Google Summer of Code 2012. SHOGUN is a machine learning toolbox, which is designed for unified large-scale learning for a broad range of feature types and learning settings. It offers a considerable number of machine learning models such as support vector machines for classification and regression, hidden Markov models, multiple kernel learning, linear discriminant analysis, linear programming machines, and perceptrons. Most of the specific algorithms are able to deal with several different data classes, including dense and sparse vectors and sequences using floating point or discrete data types. We have used this toolbox in several applications from computational biology, some of them coming with no less than 10 million training examples and others with 7 billion test examples. With more than a thousand installations worldwide, SHOGUN is already widely adopted in the machine learning community and beyond. SHOGUN is implemented in C++ and interfaces to all important languages like MATLAB, R, Octave, Python, Lua, Java, C#, Ruby and has a stand-alone command line interface. The source code is freely available under the GNU General Public License, Version 3 at http://www.shogun-toolbox.org. During Summer of Code 2012 we are looking to extend the library in three different ways:
  1. Improving accessibility to shogun by developing improving i/o support (more file formats) and mloss.org/mldata.org integration.
  2. Framework improvements (frameworks for regression, multiclass, structured output problems, quadratic progamming solvers).
  3. Integration of existing and new machine algorithms.
Check out our ideas list and if you are a talented student, consider applying!

17 March 2012

Matthew Garrett: More ways for firmware to screw you

Some of my recent time has been devoted to making our boot media more Mac friendly, which has entailed rather a lot of rebooting. This would have been fine, if tedious, except that some number of boots would fall over with either a clearly impossible kernel panic or userspace segfaulting in places that made no sense. Something was clearly wrong. Crashes that shouldn't happen are generally an indication of memory corruption. The question is how that corruption is being triggered. Hunting that down wasn't terribly easy.

My first thought was that we were possibly managing to load the kernel over a region used by UEFI code. UEFI defines two types of code - boot services and runtime services. While runtime services code and data must be preserved by the OS, in theory boot services code and data is available to the OS once the firmware has exited. In practice, that's not true. It seemed entirely possible that the kernel might be ending up on top of some of that boot services code or data and getting trodden on. Grub now has code to avoid putting the kernel on boot services, so testing the latest code seemed like a good plan. But no, crashes still happened.

That pretty much ruled out the bootloader. My next thought was that executing some of the firmware code was triggering a write to some other memory that contained the kernel. Josh Boyer suggested the next trick, which was to try marking the kernel read-only to see whether anything was hitting it. x86 lets you mark pages as read-only - any attempt to write to them should take a fault. UEFI functions are executed in the context of the kernel, so share the same page tables. That let me rule this out, since everything still went just as wrong and I wasn't taking an extra fault first.

However, at this point I was reasonably happy that it wasn't the kernel itself being overwritten - faults were occurring in userspace code as well. That was a pretty strong indication that what was happening was continuing to happen once userspace had started, so it wasn't a direct response to a firmware call. I made sure of that by stubbing out all the calls that could be triggered after kernel initialisation, and saw the same failures. Once all attempts to be clever have failed, it's time to just start using brute force. The kernel lets you reserve areas of RAM by passing arguments like memmap=0xlength$0xstart to block length bytes starting at start from being used. It took a while, but I finally found a 256MB range that made a difference - reserving it resulted in the machine booting reliably, letting the OS use it resulted in occasional crashes.

Definite progress. Comparing that memory range to the EFI memory map was helpful. There were several blocks of UEFI boot services data present there, which really seemed like too much of a coincidence. By reserving each of them in turn, I'd traced it down to a single 31MB region of boot service data - that is, memory reserved by the firmware for use by the UEFI boot services. Per spec, this is available to the OS once the boot environment has been exited. Nothing other than the OS should be touching this after boot, but something clearly was. Tracking down what was far easier than I expected, although the first attempt was a failure. Setting it read-only should have triggered a fault, but didn't. That was rather confusing. But, rather than give up, I patched the kernel to fill the region with 0xff at kernel init. Then I booted the system, read it back and looked for values that weren't 0xff. I got this:
00000000  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................ 
*
001568a0  ff ff ff ff ff ff ff ff  ff ff ff ff 84 00 00 00   ................ 
001568b0  00 20 a7 ac 46 00 00 00  00 00 00 00 00 00 06 01   . ..F........... 
001568c0  c2 0b 0c 00 ff ff ff ff  ff ff ff ff ff ff ff ff   ................ 
001568d0  ff ff 0a 04 f0 03 82 0d  40 00 00 00 ff ff ff ff   ........@....... 
001568e0  ff ff 00 21 00 36 9a 80  ff ff ff ff ff ff 00 7e   ...!.6.........~ 
001568f0  00 09 43 48 41 2d 47 75  65 73 74 01 04 02 04 0b   ..CHA-Guest..... 
00156900  16 32 08 0c 12 18 24 30  48 60 6c 2d 1a 0e 18 1a   .2....$0H l-.... 
00156910  ff ff 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00156920  00 00 00 00 00 00 00 dd  09 00 10 18 02 00 10 01   ................ 
00156930  00 00 dd 1e 00 90 4c 33  0e 18 1a ff ff 00 00 00   ......L3........ 
00156940  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00156950  00 00 bd ea f8 b3 ff ff  ff ff ff ff ff ff ff ff   ................ 
00156960  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................ 
*
022fb000
That's a lot of 0xffs (around 31MB of them) with one small section that contains an 802.11 probe packet with the SSID of the hospital across the road from my house. Apple support network booting off wireless networks. It seems that the firmware brought up the wireless card, associated with this network (it's the only public one nearby) and then left the card DMAing packets into RAM. The read-only page attribute only applies to CPU-initiated accesses, so it could do this without triggering a page fault. It also explained why it was so random - whether memory corruption occurred would depend on whether a packet appeared between that memory being used by the OS and the kernel reinitialising the wireless card. It certainly explains why I couldn't reproduce it when I left the machine repeatedly rebooting on the bus home.

How do we fix this? Unsure. With luck disconnecting the UEFI driver in the bootloader should quiesce the hardware, but without testing I'm not sure of that yet. For now it's just another example of firmware managing to break expectations in deeply strange ways.

comment count unavailable comments

Kai Wasserb ch: The joy of building Wine in a non-multiarch environment

This is an updated and consolidated post for/of two previous blog entries. Before I continue, let me make it clear, that this post isn't intended to criticise anybody. It is meant as a instruction leaflet to go with my unofficial (source) package(s). The best idea, until all build dependencies of Wine have been made multiarch ready (we're almost there), is to create a base.tgz for pbuilder in which you make the required changes. So you don't have to remember applying them everytime (note, this is only needed for amd64 builds). After you created yourself a base.tgz for Wine, do the follwoing:
  1. Log into the chroot stored in your base.tgz for Wine: pbuilder --login --save-after-login --basetgz /path/to/your/wine.base.tgz
  2. Update the package list and installed packages with aptitude update && aptitude safe-upgrade unless you know, everything is updated.
  3. Install ia32-libs-dev and libjpeg-dev: aptitude install ia32-libs-dev libjpeg-dev
  4. Work around #638543:
    1. cd /usr/lib32
    2. rm libjpeg.so libjpeg.so.62* (you can also remove the libjpeg.a and libjpeg.la files)
    3. ln -s libjpeg.so.8 libjpeg.so
    4. cd /usr/include
    5. ln -s x86_64-linux-gnu/jconfig.h
  5. As we build build with GCC 4.5 (due to a problem with GCC 4.6, see WineHQ bug #22053 for more details) on amd64 (i386 isn't using GCC 4.5 anymore (since the 1.5.0-0.2 packages), because that resulted in linking errors), we hit #638418. To work around that, you need to create the symlink yourself. Execute ln -s x86_64-linux-gnu/asm asm in /usr/include.
  6. Exit the chroot, pbuilder should now update your base.tgz.
  7. Build the source package with pbuilder.
For i386 users this shouldn't be required. Not even the symlinking of the asm directory, as we don't use GCC 4.5 anylonger on i386 (due to various linking errors). Though that comes at the cost of potential issues with e.g. Steam.

16 March 2012

James Morrison: Penance driven development

Recently, I got to try out working in a couple different environments. Working in these environments helped me formalize some of the things that I like and some of the things I don't like. With these new environments I've come to realize my style of development. It's called penance driven development. It's called this because there are two important things:
1) It's better to ask forgiveness than for permissions
2) You need to be deserving of forgiveness.

So the concept is simple, whenever you break something, you first fix the symptom, then you improve the infrastructure so similar breaks should be harder to do. The cost of the penance doesn't have to be huge, but the penance has to be done, otherwise technical debt accumulates and people believe there is no downside to breaking things.

What needs to be improved depends on what is missing. For example, if an engineer broke a feature, but that feature didn't have a test, the penance should be to write a test. Even if the one that broke the feature isn't the owner or the one that wrote the feature. If someone breaks the build and no one noticed because the continuous build didn't email, the fix could be setting up the continuous build to email out build failures, or even better, run each change on a build bot before allowing commits.

I think I'll have to start a penance of the day blog/twitter stream.

I've been practicing this style of development for most of my career, and it's similar if less precise than the 5 whys . As mentioned in the The Lean Startup, the cost of the penance should be proportional to the cost of the break.

Christian Perrier: [life] No more teens at home...

So, since Wednesday, we have no more teenagers among our children. Indeed, our youngest, Magali, turned 20 and we happily celebrated that in a japanese restaurant in Paris : Magali is living in a (very) small flat in Paris as she attends preparation courses for exams meant to enter physical therapist schools. She's currently working very hard to prepare these and is left with only one week before the first exam. We're 100% confident that she'll succeed. Really, the days where our little shrimp was crawling across her bed, turning clockwise and finally sleeping in the width of the bed....do not seem so far. And still, these folks we call "our kids" are now grown-up persons living their own lives : Jean-Baptiste (some of my friends reading this blog have met him at DebConf 10) will soon enter profesionnal life after graduating an engineering school in electronics, at 24. Sophie, now 22, will resume her social worker school in Bordeaux in September and will probably start working as social worker ("assistante sociale", in French) afterwards. And we hope that our little shrimp (which her sister calls "mini-poop") starts a physical therapist school next year. So, well, more and more, Elisabeth and I are old parents and weeks where we're indeed both alone at home are now common (not to mention weeks where only one of us is alone either because she has professionnal commitments on some other side of the planet or I have some free software event on another side of the planet). It is indeed really likely that this year's holidays will be the first ones where we have a distant travel...which we'll be doing alone. Holidays in Nicaragua (you probably guess why Nicaragua) are much awaited and will be enjoyed a lot, but still...:-) So, well, to all those of you who read this and are parents, or plan to become parents, be happy. The wonder that it is lasts for years and it's very hard to describe precisely the link that connects parents and their children. At least the one we have with our children, which both us and them are proud of. And that pride and this link have no price. I really wish you guys to develop the same with your newborns (particularly those of you who very very very recently became parents anbd, yes I mean you who know who you are...).

15 March 2012

Erich Schubert: ELKI applying for GSoC 2012

I've submitted an organization application to the Google Summer of Code 2012 for ELKI - an open source data mining framework I'm working on.
ELKI
I hope I can get spots for 1-2 students to help implementing additional state of the art methods, to allow for even more broad comparisons. Acceptance notification will be tomorrow. I have no idea how high out chances are of getting accepted. We're open source, as part of the university we have a proven record of educating students (in fact, around two dozen have by now contributed to ELKI, although not all of this has been "released" yet). We're rather small compared to e.g. Gnome, Debian or Apache and just a few years old. But I believe we are trying to fill an important gap in the research and open source interaction: way too many algorithms get published in science, but are never made available as source code to test and compare. This is where we try to step in: make many largely overlooked methods available. Of course we also have k-means (who hasn't?), but there is much more than just k-means! And there are plenty of methods often cited in scientific literature, yet nobody seems to have a working implementation of them ...

While most people equal data mining to prediction and classification (both of which are actually more machine learning topics), ELKI is strong on cluster analysis and outlier detection as well as index structures. Plus, it is much more flexible than other frameworks. For example, we allow almost arbitrary combinations of algorithms and distance functions. So with our tool, you can easily test your own distance function by plugging it into various algorithms. Otherwise we could just have used Weka.
The other reason why we did not extend Weka (or used R) is that we actually wanted to not just implement some algorithms, but actually be able to study the effects of index structures on the algorithms. And in my opinion, this is actually the key difference between true data mining and ML, AI or "statistics". In ML and statistics, the key objective is the result quality. Which is often rather easy to measure, too. For "full" data mining, one also needs to consider all the issues of actually managing the data, indexing it to accelerate computations. Plus, the prime objective is to discover something new, that you did not know before.
Of course these things cannot be completely separated. You can of course discover patterns and rules in the data that will allow you to make good predictions. But data mining is not just prediction and classification!

The next release of ELKI, 0.5, will be released in early april. ELKI 0.4 is already available in Debian testing and Ubuntu. The next release has a focus on comparing clustering results, including an experimental visualization for clustering differences, which will be presented at the ICDE 2012 conference.
On the big to-do list is a lot of stuff. In particular, ELKI is currently mostly useful for research, as it is too difficult to use for the average business guy wanting to do data mining. You can think of it more as a prototyping system: try out what works for you, then implement that within your larger project in an integrated and optimized way. The other big thing that I'm unhappy with is the visualization speed. SVG is great for print export, but Batik can be really slow, and the XML-DOM-API isn't exactly "accessible" to students wanting to add new visualizations. Right now, it is all useful and okay for my kind of experiments, but it could be so much more if we could solve the speed (and memory) issues of the pure SVG based approach. I'd love to see something much faster here, but with a SVG export for editing and printing.

Kenshi Muto: Squeeze amd64/i386 d-i images with Linux kernel 3.2.4 bpo, and WPA support

After a long silence, I updated the bpo Debian-installer for Debian GNU/Linux 6.0.4, Squeeze. You know, you can take it from http://kmuto.jp/debian/d-i/.

Debian Med: Bits from Debian Med team (Posted by Andreas Tille)

Hi,

in this bits:
  1. Debian Med Bug Squashing Advent Calendar 2011
  2. Anniversary of Debian Med
  3. Second Debian Med sprint (Southport, 27th-29th January 2012)
  4. Mentoring of Month (MoM)
  5. DDs who came to Debian because of Debian Med
  6. Future plans
  7. General lessons learned

Debian Med Bug Squashing Advent Calendar 2011In December last year Thorsten Alteholz has started a nice QA initiative which might be interesting for other teams next Advent. The Debian Med team was able to fix about 70 bugs in this time. Thanks to Thorsten for this nice piece of motivation and thanks to everybody who took part in the bug squashing.

Anniversary of Debian MedOn Mon, 7 Jan 2002 the Debian Med project was first officially announced. I submitted a short blog posting about this and perhaps you might like to see a long sequence of talks about this topic.

Second Debian Med sprint (Southport, 27th-29th January 2012)In end of January 2012 the Debian Med team has met to the second sprint. As last year I would call this a very successfull event and I would recommend other teams to instanciate such meetings as well. Feel free to read my more verbose report.

Mentoring of Month (MoM)I have started this project for the following reasons:
I made a short summary how the first MoM project worked (see at bottom).

DDs who came to Debian because of Debian MedAfter realising that several members of the Debian Med team finally became DDs I made a little survey to find out about their reasons to become DD / DM. I came to the conclusion that a Blend could be a nice entry point for people to join Debian because newcomers can identify themselves with a known topic (the scope of the Blend - in this case medicine and bioinformatics) first and learn Debian rules in a team with common interest. This perfectly fits my expectation which I had from the beginning 10 years ago and I would be very happy if other Blends would follow this example to be nice, inviting and try to *actively* ask people for cooperation (see some simple rules which I learned in this process below). In the teammetrics GSOC project some graphs were created where you can see the level of contribution of these people (and other team members).

Future plansCurrently some heavy work regarding bringing bibliographic references about packages straight into package information is going on. This topic is specifically interesting in Biology because programs are frequently connected to some publication about the methods used inside the code. This topic is as well relevant to Debian Science and DebiChem. Thanks to the patient work done by Charles Plessy we now have about 70 packages featuring debian/upstream files featuring bibliographic references and there is ongoing work to move these data to UDD to enable further usage. We are in the process of final polishing the format and finishing scripts for the import. If people are interested to join this effort this would be the right moment to raise their hand.

General lessons learned
  1. Do not let wait anybody who wants to do work.
  2. Newcomers are frequently shy - try to invite them kindly and patiently.
  3. Tell people verbosely about your project - it is astonishing how less people know and what wrong assumptions they make about your project.

Kind regards
Andreas.

14 March 2012

Erich Schubert: Google Scholar, Mendeley and unreliable sources

Google Scholar and Mendeley need to do more quality control.
Take for example the article
A General Framework for Increasing the Robustness of PCA-Based Correlation Clustering Algorithms
Hans-Peter Kriegel, Peer Kr ger, Erich Schubert, Arthur Zimek
Scientific and Statistical Database Management (SSDBM 2008)
(part of my diploma thesis).
Apparently, someone screwed up entering the data into Mendeley and added the editors to the authors. Now Google happily imported this data into Google Scholar, and keeps on reporting the authors incorrectly, too. Of course, many people will again import this incorrect data into their bibtex files, upload it to Mendeley and others...
Yet, neither Google Scholar nor Mendeley has an option for reporting such an error. They don't even realize that maybe Springerlink - where the DOI points to - is the more reliable source.
On the contrary, Google Scholar just started suggesting to me that the editors are coauthors ...
They really need to add an option to fix such errors. There is nothing wrong with having errors in gathered data, but you need to have a way of fixing them.

Richard Hartmann: Open Source Days 2012, the aftermath

Open Source Days Open Source Days were nice, especially the meating(sic) of and talking to old friends and new people. I got to discuss Mercurial with two of its developers which was very interesting indeed. It's funny how similar git and Mercurial are in some regards and how different in others. Overall, it feels a bit as if Mercurial is not quite as distributed as git. Its local, sliding revision index feels like disaster waiting to happen, to me. On the other hand, Mercurials ability to not check out large files to your local repository sounds very git-annex-ish, which is nice. Choice is good :) My talk about how to gitify your life (slides, no video) went extremely well. Within my time-slot, there were a total of six talks, and lunch(!), in parallel. About 300 people attended OSD on Sunday and of those, many were at booths, stands, eating, etc. My personal turnout was 50-70 listeners so I was well above mathematical average and from what I heard, my talk was the most-visited one during that time. Add the fact that several people asked me to put up the resource links up again after the talk so they could take pictures and I would say this talk really was a success. Ego-stroking? Yes. Vanity? Most likely. Being able to see that people can really want to learn about what you have to say? Priceless. It's a real pity that there was no video recording, but it's obviously too late to do anything about that. Joey Hess asked me to at least make an audio recording on my laptop, but that turned out to be a lot of useless white noise with faint mumbling in between. Maybe I should at least get a small microphone or recorder so I will always have an audio copy. I already did order a presentation clickie with built-in laser pointer so I won't have to rely on possibly non-existent conference infra, any more. If there's any interest, I may re-record the talk as audio, else I will just try to get another speaker slot at another conference, with video recording, and link that. Denmark Barring the airport, central station, bus, hotel, venue, and metro, I did not see anything of Copenhagen. That's a pity, but I didn't want to leave the conference early. To continue my recent tradition of dumping random observations about places I visit onto my blog, here goes:

13 March 2012

Don Armstrong: Debbugs forcemerge support

One of the earliest features I wrote for the Debian bug tracking system (Debbugs) after joining the team was support for forcibly merging bugs. Originally, merging two bugs required that the bugs be in exactly the same state before merging them; forcemerge removed this requirement. Unfortunately, the way I originally implemented this was shortsighted, and merely forced the merge partners to have the same values as the merge master. This meant that owners, blocking bugs, and many other things were silently changed, which meant that people weren't notified of changes, and bugs could end up in an inconsistent state. A while ago, I decided to fix this by calculating the changes required to actually merge the bugs, making those changes, and then merging the bugs normally; thus, doing everything that a maintainer would normally have done for them. This necessitated abstracting out the entire control apparatus into the Debbugs::Control module. Now that it's complete, you can do the following:
> forcemerge 1 2
Bug #1 [foo] new title
Bug #2  Done: foo@bugs.something  [foo] foo
Unset bug forwarded-to-address
Severity set to 'wishlist' from 'grave'
3 was blocked by: 2
3 was not blocking any bugs.
Removed blocking bug(s) of 3: 2
2 was blocked by: 4
2 was not blocking any bugs.
Removed blocking bug(s) of 2: 4
Bug reopened
Removed annotation that bug was owned by bar@baz.com.
Removed indication that 2 affects bleargh
Removed tag(s) unreproducible and moreinfo.
Merged 1 2
> thanks
Stopping processing here.
and bug 2 now is merged with 1 and matches the state of 1. [The above is the control output from the appropriate bit of the 06_mail_handling.t test.] This change also means that I'll be able to finally write support for control@ operations at submit@ time. Also, all of the bug modifications that happen at submit@ or nnn@ time (setting title, found, etc.) will be implemented as calls to Debbugs::Control so we can eventually keep a postgresql database updated in addition to the flatfile database.

Martin-&#201;ric Racine: Ubuntu: LTS to LTS+1 upgrade leaves cruft

Over the past few weeks, I upgraded my Ubuntu hosts to the Precise release due to be published next month. While the upgrade on my desktops from Oneiric to Precise went mostly well, the upgrade of my terminal server from Lucid to Precise was more daunting. In short: too many packages leave an undue amount of cruft behind, namely tons of obsolete configuration files that are not correctly handled by maintainer scripts, to either delete obsolete files or to rename them to the currently valid filename. At Debian, these oversights are periodically monitored using piuparts and reported to the package maintainer, but Ubuntu doesn't seem to offer this sort of automated Quality Assurance process yet. Perhaps this would be something worth developing? PS: For those who wonder how I spotted those obsolete configuration files, I simply enabled the FLAUSCH environment variable before running upgrade-system.

12 March 2012

Pablo Lorenzzoni: Migrating from Mephisto to WordPress

Just as I promised yesterday, I pushed a new git repo with my fork of the tool I used to migrate my old Mephisto blog to this new WordPress one. I forked because the tool have not worked the first time. First of all, I was missing uuidtools gem, and to install it would be a pain inside the jail system I used to run my blog. Too much trouble just to get a UUID we can get by other means so I just added an environment variable UUIDGEN anyone can use to point to a tool to do the job. I know this have performance implications, but I am not talking about 10-thousand entries Then, I found out that, for some odd reason I still have to understand, WordPress was cutting my articles everytime it read a character. I could study the subject, but I just added a #gsub in mephisto-to-wxr code and moved on. I was about to remove it from the repo, but I left it there since it could help other people. Also, since there might be other similar occurrences, leaving it there serves as a heads up. Also, I added support for Categories and Tags to mephisto-to-wxr, that seemed to be limitedly accepted (I translated Mephisto Sections into WordPress Categories). All other activities were just clean-up. That tool generated a .WXR with all the articles and comments from my Mephisto blog. Everything I had to do was import it using WordPress import tool.

11 March 2012

Mohammed Sameer: Justin TV without flash!

Justin TV is a streaming service that uses the broken flash concept in order to allow you to view channels.

Unfortunately it's the only source of Arabic movies, series and shows.

I wrote a GStreamer source that retrieves the channel information and uses libRTMP to connect to the Justin streaming server. The rest is handled by GStreamer ;-)

If you want to watch http://www.justin.tv/ammartv for example, you simply do

gst-launch-0.10 playbin uri=jtv://ammartv

The 2nd plugin is a Totem plugin that indirectly utilizes the GStreamer source.
Given a language code (ar or en or ...), it will download the list of channels in that language and shows them in Totem's side panel. One can then easily double click on a channel and simply watch it. I originally wanted to do the Totem plugin in python but the inability to do asynchronous callback based HTTP requests prevented me :-(

Be warned that the code is of course a bit buggy and needs a lot of cleanups, it's been years since I coded anything in Gtk and that Justin might find out and either block the source somehow or even change the protocol but let's wait until that happens ;-) read more

Russell Coker: USB Flash Storage

For some years I have had my Internet gateway/firewall system in a cupboard in my bedroom. While I don t mind some computer noise (I ve slept near a server for most of the last 22 years) it s good to have it as quiet as possible so getting rid of the hard drive is desirable. I considered buying an IDE flash drive, but I d like to continue my trend of not paying for hardware so I chose to use some USB flash devices that HP was giving away at a seminar (thanks HP as an aside the P3 system is an old Compaq Desktop system). So I ve got one 4G USB device for root and one for Squid. For the past few months I ve had /var/spool/squid be a USB flash device. I considered using RAID-0 for that filesystem because the computer is a P3 and only has USB 1.2 and thus a maximum theoretical transfer rate of 1.5MB/s and a maximum real-world rate of about 1MB/s. But my ADSL connection doesn t seem able to sustain much more than 1MB/s and Squid doesn t write data synchronously so in all my tests the USB speed hasn t affected HTTP performance. One issue that has delayed my move to all USB is the difficulty of booting as the P3 system in question doesn t support booting from USB. I considered creating a boot CD that loads the kernel from the USB device, but that seemed a little painful and also relies on the CD-ROM drive working which isn t a great idea for a system that runs 24*7 in a dusty cupboard. I ended up using GRUB on the IDE hard drive to load the kernel and initrd and then mount a USB device as root, this seems to work and the command hdparm -S6 /dev/sda in /etc/rc.local makes the hard drive go to sleep once the system is booted. The only technical parts of the process were putting in the UUIDs of the filesystems in /etc/fstab (because I can t be sure which USB device will be found first) and creating a new initramfs with modules for USB storage listed in /etc/initramfs-tools/modules so that a USB device could be the root filesystem. The firewall system is now a bit quieter and based on my last tests of hard drive power use will probably dissipate about 5-7W less heat. The next thing to do is wait and see if it keeps running or falls over. ;) Related posts:
  1. flash for main storage I was in a discussion about flash on a closed...
  2. Flash Storage and Servers In the comments on my post about the Dell PowerEdge...
  3. IDE DMA and Flash I ve just been working with a Flash device used as...

10 March 2012

Gustavo Noronha Silva: WebKitGTK+ Debian packaging repository changes

For a while now the git repository used for packaging WebKitGTK+ has been broken. Broken as in nobody was able to clone it. In addition to that, the packaging workflow had been changing over time, from a track-upstream-git/patches applied one to a import-orig-only/patches-not-applied one. After spending some more time trying to unbreak the repository for the third time I decided it might be a good time for a clean up. I created a new repository, imported all upstream versions for series 1.2.x (which is in squeeze), 1.6.x (unstable), and 1.7.x (experimental). I also imported packaging-related commis for those versions using git format-patch and black magic. One of the good things about doing this move, and which should make hacking the WebKitGTK+ debian package more pleasant and accessible can be seen here:
kov@goiaba ~/s/debian-webkit> du -sh webkit/.git webkit.old/.git
27M webkit/.git
1.6G webkit.old/.git
If you care about the old repository, it s on git.webkit.org still, named old-webkit.git. Enjoy!

Next.

Previous.