Search Results: "andree"

29 October 2013

Soeren Sonnenburg: Shogun Toolbox Version 3.0 released!

Dear all, we are proud to announce the 3.0 release of the Shogun Machine-Learning Toolbox. This release features the incredible projects of our 8 hard-working Google Summer of Code students. In addition, you get other cool new features as well as lots of internal improvements, bugfixes, and documentation improvements. To speak in numbers, we got more than 2000 commits changing almost 400000 lines in more than 7000 files and increased the number of unit tests from 50 to 600. This is the largest release that Shogun ever had! Please visit http://shogun-toolbox.org/ to obtain Shogun. News Here is a brief description of what is new, starting with the GSoC projects, which deserve most fame: Screenshots Everyone likes screenshots. Well, we have got something better! All of the above projects (and more) are now documented in the form of IPython notebooks, combining machine learning fundamentals, code, and plots. Those are a great looking way that we chose to document our framework from now on. Have a look at them and feel free to submit your use case as a notebook! FGM.html GMM.html HashedDocDotFeatures.html LMNN.html SupportVectorMachines.html Tapkee.html bss_audio.html bss_image.html ecg_sep.html gaussian_processes.html logdet.html mmd_two_sample_testing.html The web-demo framework has been integrated into our website, go check them out. Other changes We finally moved to the Shogun build process to CMake. Through GSoC, added a general clone and equals methods to all Shogun objects, and added automagic unit-testing for serialisation and clone/equals for all classes. Other new features include multiclass LDA, and probability outputs for multiclass SVMs. For the full list, see the NEWS. Workshop Videos and slides In case you missed the first Shogun workshop that we organised in Berlin last July, all of the talks have been put online. Shogun in the Cloud As setting up the right environment for shogun and installing it was always one of the biggest problems for the users (hence the switching to CMake), we have created a sandbox where you can try out shogun on your own without installing shogun on your system! Basically it's a web-service which give you access to your own ipython notebook server with all the shogun notebooks. Of course you are more than welcome to create and share your own notebooks using this service! *NOTE*: This is a courtesy service created by Shogun Toolbox developers, hence if you like it please consider some form of donation to the project so that we can keep up this service running for you. Try shogun in the cloud. Thanks The release has been made possible by the hard work of all of our GSoC students, see list above. Thanks also to Thoralf Klein and Bj rn Esser for the load of great contributions. Last but not least, thanks to all the people who use Shogun and provide feedback. S ren Sonnenburg on behalf of the Shogun team (+ Viktor Gal, Sergey Lisitsyn, Heiko Strathmann and Fernando Iglesias)

24 June 2013

Vincent Sanders: A picture is worth a thousand words

When Sir Tim made a band the first image on the web back in 1992, I do not imagine for a moment he understood the full scope of what was to follow. I also do not imagine Marc Andreessen understood the technical debt he was about to introduce and that fateful day in 1993 when he proposed the img tag allowing inline images.

Many will argue of course that the old adage of my title may not apply to the web, where if it were true, every page view would be like reading a novel! and many of those novels would involve cats or one pixel tall colour gradients.

Leaving aside the philosophical arguments for a moment it takes web browser author a great deal of effort to quickly and efficiently put those cat pictures in front of your eyeballs.
NavigatingImages are navigated by a user in a selection of ways, including:
Whatever the method, the resulting object is subject to the same caching operations as any content object within a browser. These caching and storage operations are not specific to images however images are one of the most resource intensive objects a browser must regularly deal with (though javascript and stylesheet sources are starting to rival it at times) because they are relatively large and numerous.
Pre render processing
When the image object fetch is completed the browser must process the image for rendering which is where this gets even more complicated. I shall use how this works in NetSurf as I know that browsers internals best, but operation is pretty similar in many browsers.

The content fetch will have performed basic content sniffing to determine the received objects mime type. This is necessary because a great number of servers are misconfigured and a disturbingly large number of images served as png are really jpegs etc. Indeed sometimes you even get files served which are not images!

Upon receipt of enough data to decode the image header, for the detected mime type, the images metadata is extracted. This metadata usually includes things like size and colour depth.

If the img tag in the original source document omitted the width or height of the image the entire document render may have to be reflowed at this point to allow the correct spacing. The reflow process is often unsightly and should be avoided. Additionally at this stage if the image is too large to handle or an unhandled format the object will be replaced with the "broken" image icon.

Often that will be everything that is done with the image, when I added "lazy" image conversion to NetSurf we performed extensive profiling and discovered well over 40% of images on visited pages are simply never viewed by the user but that small (100 pixel on a side) images were almost always displayed.

This odd distribution comes down to how images are used in modern web pages, they broadly fall into two categories of "decoration" and "content" for example all the background gradients and sidebar images etc. are generally small images used as decoration whereas the cat picture above is part of the "content". A user may not scroll down a page to see content but almost always gets to "view" the decoration.
Rendering
Created by Andrea R used under CC Attribution-NonCommercial-ShareAlike 2.0 licence
The exact browser heuristics used differ as to when the render operation is performed but they all have a similar job to perform. When i say render here this may be possibly as an "off screen" view if they are actually on another tab etc. Regardless the image data must be converted from the source data (a PNG, JPEG etc.) into a format suitable for the browsers display plotting routines.

The browser will create a render bitmap in whatever format the plotting routines require (for example the GTK plotters use a Cairo image surface) , use an image library to unpack the source image data (PNG) into the render bitmap (possibly performing transforms such as scaling and rotation) and then use that bitmap to update the pixels on screen.

The most common transform at render time is that of scaling, this can be problematic as not all image libraries have output scaling capabilities which results in having to decode the entire source image and then scaling from that bitmap.

This is especially egregious if the source image is large (perhaps a multi megabyte jpeg) but the width and height are set to produce a thumbnail. The effect is amplified if the user has set the image cache size limit to a small value like 8 Megabytes (yes some users do this apparently their machines have 32MB of RAM and they browse the web)

In addition the image may well require tiling (for background gradients) and quite complex transforms (including rotation) thanks to CSS 3. Add in that javascript can alter the css style and hence the transform and you can imagine quite how complex the renderers can become.
CachingThe keen reader might spot that repeated renderings of the source image (e.g. because window is scrolled or clipped) result in this computationally expensive operation also being repeated. We solve this by interposing a render image cache between the source data and the render bitmaps.

By keeping the data in the preferred format, image rendering performance can be greatly increased. It should be noted that this cache is completely distinct from the source object cache and relates only to the rendered images.

Originally NetSurf used to perform the render conversion for every image as it was received without exception, rather than at render time, resulting in a great deal of unnecessary processing and memory usage. This was originally done for simplicity and optimising for "decoration" images.

The rules for determining what gets cached and for how long are somewhat involved and the majority of the code within the current implementation NetSurf uses is metrics and statistic generation to produce better decisions.

There comes a time at which this cache is no longer sufficient and rendering performance becomes unacceptable. The NetSurf renderer errs on the side of reducing resource usage (clearing the cache) at the expense of increased render times. Other browsers make different compromises based on the expected resources of the user base.
FinallyHopefully that gives a reasonable overview to the operations a browser performs just to put that cat picture in front of your eyeballs.

And maybe next time your browser is guzzling RAM to plot thousands of images you might have a bit more appreciation to exactly what it is up to.

27 October 2012

Soeren Sonnenburg: Shogun at Google Summer of Code 2012

The summer came finally to an end and (yes in Berlin we still had 20 C end of October), unfortunately, so did GSoC with it. This has been the second time for SHOGUN to be in GSoC. For those unfamiliar with SHOGUN - it is a very versatile machine learning toolbox that enables unified large-scale learning for a broad range of feature types and learning settings, like classification, regression, or explorative data analysis. I again played the role of an org admin and co-mentor this year and would like to take the opportunity to summarize enhancements to the toolbox and my GSoC experience: In contrast to last year, we required code-contributions in the application phase of GSoC already, i.e., a (small) patch was mandatory for your application to be considered. This reduced the number of applications we received: 48 proposals from 38 students instead of 70 proposals from about 60 students last year but also increased the overall quality of the applications. In the end we were very happy to get 8 very talented students and have the opportunity of boosting the project thanks to their hard and awesome work. Thanks to google for sponsoring three more students compared to last GSoC. Still we gave one slot back to the pool for good to the octave project (They used it very wisely and octave will have a just-in-time compiler now, which will benefit us all!). SHOGUN 2.0.0 is the new release of the toolbox including of course all the new features that the students have implemented in their projects. On the one hand, modules that were already in SHOGUN have been extended or improved. For example, Jacob Walker has implemented Gaussian Processes (GPs) improving the usability of SHOGUN for regression problems. A framework for multiclass learning by Chiyuan Zhang including state-of-the-art methods in this area such as Error-Correcting Output Coding (ECOC) and ShareBoost, among others. In addition, Evgeniy Andreev has made very important improvements w.r.t. the accessibility of SHOGUN. Thanks to his work with SWIG director classes, now it is possible to use python for prototyping and make use of that code with the same flexibility as if it had been written in the C++ core of the project. On the other hand, completely new frameworks and other functionalities have been added to the project as well. This is the case of multitask learning and domain adaptation algorithms written by Sergey Lisitsyn and the kernel two-sample or dependence test by Heiko Strathmann. Viktor Gal has introduced latent SVMs to SHOGUN and, finally, two students have worked in the new structured output learning framework. Fernando Iglesias made the design of this framework introducing the structured output machines into SHOGUN while Michal Uricar has implemented several bundle methods to solve the optimization problem of the structured output SVM. It has been very fun and interesting how the work done in different projects has been put together very early, even during the GSoC period. Only to show an example of this dealing with the generic structured output framework and the improvements in the accessibility. It is possible to make use of the SWIG directors to implement the application specific mechanisms of a structured learning problem instance in python and then use the rest of the framework (written in C++) to solve this new problem. Students! You all did a great job and I am more than amazed what you all have achieved. Thank you very much and I hope some of you will stick around. Besides all these improvements it has been particularly challenging for me as org admin to scale the project. While I could still be deeply involved in each and every part of the project last GSoC, this was no longer possible this year. Learning to trust that your mentors are doing the job is something that didn't come easy to me. Having had about monthly all-hands meetings did help and so did monitoring the happiness of the students. I am glad that it all worked out nicely this year too. Again, I would like to mention that SHOGUN improved a lot code-base/code-quality wise. Students gave very constructive feedback about our (lack) of proper Vector/Matrix/String/Sparse Matrix types. We now have all these implemented doing automagic memory garbage collection behind scenes. We have started to transition to use Eigen3 as our matrix library of choice, which made quite a number of algorithms much easier to implement. We generalized the Label framework (CLabels) to be tractable for not just classification and regression but multitask and structured output learning. Finally, we have had quite a number of infrastructure improvements. Thanks to GSoC money we have a dedicated server for running the buildbot/buildslaves and website. The ML Group at TU Berlin does sponsor virtual machines for building SHOGUN on Debian and Cygwin. Viktor Gal stepped up providing buildslaves for Ubuntu and FreeBSD. Gunnar Raetschs group is supporting redhat based build tests. We have Travis CI running testing pull requests for breakage even before merges. Code quality is now monitored utilizing LLVMs scan-build. Bernard Hernandez appeared and wrote a fancy new website for SHOGUN. A more detailed description of the achievements of each of the students follows:

26 April 2012

Soeren Sonnenburg: GSoC2012 Accepted Students

Shogun has received an outstanding 9 slots in this years google summer of code. Thanks to google and the hard work of our mentors ranking and discussing with the applicants - we were able to accept 8 talented students (We had to return one slot back to the pool due to not having enough mentors for all tasks. We indicated that octave gets the slot so lets hope they got it and will spend it well :). Each of the students will work on rather challenging machine learning topics. The accepted topics and the students attacking them with the help of there mentors are We are excited to have them all in the team! Happy hacking!

19 September 2010

Asheesh Laroia: When "free software" got a new name

On January 30, 1998, Wired.com gushed about the ethical underpinnings of the free software movement. The movement was growing:
Netscape's bold move last week [was] to free the source code of its browser is a prime-time endorsement for no-cost, build-it-yourself software.
The free software movement was in its second decade. In the first decade, corporations learned to tolerate it. In the late '90s, a transition was underway. Red Hat was one of the first companies ostensibly founded on free software principles. But as free software grew, some were concerned that its name was holding it back. The article explains with a link to a page within gnu.org:
But "free software" is an ambiguous term - there is a difference in meaning between the cultures of PC-based proprietary systems and the Net-centric UNIX worlds.
Michael Stutz, the author of the piece, surveyed the writing of Eric S. Raymond and interviewed luminaries like Bob Young, Russell Nelson, and Marc Andreessen. The article is about the creation of a new term for the freely-reusable code produced by the free software movement.
As proponents of free software often point out, while this software can be free-of-cost - that is, gratis - the real issue is about freedom, or human liberty. So it is really freed software.
Yes, that's right -- freed software. The emphasis is in the original. Most of us know the names Eric S. Raymond and Russ Nelson as people involved early-on in the Open Source Initiative. I guess January 1998 is before they decided on the "open source" name. Today, the community is divided into people who think it's important to say "free software" and the rest who call it "open source." We'd all agree with the following statement from the article:
"Freed software is a big win for society in general," said Russell Nelson.
And that's today's random page from the history books.

3 November 2008

Andree Leidenfrost: EeePC 901 with Ubuntu Intrepid Ibex

I have finally bought an EeePC 901, mainly to use for travelling.

I had been holding off for some weeks because I had hoped (in vain) that the Linux version with 20GB SSD and without a Windows XP license would become available here in Australia.

Anyway, I decided to try putting brand new Ubuntu Intrepid Ibex on it and I must say it worked really well. Here is what I did:
Everything else is pretty much ok. Apart from smaller fonts and makingg windows go above the top panel (with: gconftool-2 --set /apps/compiz/plugins/move/allscreens/options/constrain_y --type bool 0), I have not really made any changes.

I am typing this on the little keyboard of the 901 and it is actually quite usable even though I would not want to type pages and pages on it. (Somehow the Blogger editor behaves a bit funny, but I guess this has nothing to do with the hardware but if anything with Firefox 3.)

So far, I think this is an excellent little machine - it even plays Big Buck Bunny in 720p without any problems. ;-)

15 March 2008

Isaac Clerencia: Please, Dems, don t mess up this one: vote for Obama

I am a huge political junkie and I have been closely following Europe and US politics for more than 5 years. After this time and several disappointments, I have to admit that Obama is the first truly inspirational politician I have seen (listen to the Yes We Can song based on his New Hampshire speech). Some of his detractors dismiss his speeches as lousy, empty or vague, but you just have to listen to a couple of them to see that he is genuinely smart (such as the one about faith and atheism or the interview at Google). He doesn’t only take the right stance on most issues that I care about (Iraq, foreign policy, ethics, net neutrality, …), but he does it in a sincere way. Obama just gets it. I have got this impression from watching several of his speeches and interviews, but Marc Andreessen had the chance to spend an hour and a half with him a year ago and got the very same feeling. The main argument against Barack Obama nowadays is his alleged lack of experience. “Watch how I run my campaign”, Barack said to Marc when inquired about that. It’s obvious that running a primary campaign isn’t the same as being POTUS, but being the president’s wife isn’t exactly the same either. So if we compare the Obama and Hillary campaigns we can easily see Hillary’s experience as an “old school” politician. She overstates, lies, accepts loads of money from lobbies (because “they represent people too”, haha), resorts to fear-mongering (”Obama is not a muslim, …, as far as I know”, 3 a.m. ad), sides with McCain if needed to get some extra votes, surrounds herself with nitwits, … . To summarize, she uses every dirty trick she has learned in these years in Washington, and that’s exactly what I am so sick of right now. On the other hand, Obama’s campaign hasn’t just been one of the best organized and executed campaigns I have ever seen, but also the cleanest one. He has managed to overcome double-digit Hillary leads in most states without having to resort to any of these experienced politician’s dirty tricks. If I have to trust one of both to run a country, the decision is obvious. If I didn’t manage to convince you, I hope Lawrence Lessig and xkcd’s Randall Munroe do. Obama is leading and almost there, he just needs the final push. Please, do the right thing.

9 June 2007

Isaac Clerencia: How to hire great people: drive, curiosity and ethics

I’ve just come across this article from Marc Andreessen about hiring. It raises some interesting points about the hiring process, which he bases in three criteria. Drive
I like to see what someone has done. Not been involved in, or been part of, or watched happen, or was hanging around when it happened. If you’re a programmer: the open source project to which you’ve made major contributions. Something. If you can’t find anything — if a candidate has just followed the rules their whole lives, showed up for the right classes and the right tests and the right career opportunities without achieving something distinct and notable, relative to their starting point — then they probably aren’t driven. And you’re not going to change them. Motivating people who are fundamentally unmotivated is not easy.
Curiosity
Sit a programmer candidate for an Internet company down and ask them about the ten most interesting things happening in Internet software. REST vs SOAP, the new Facebook API, whether Ruby on Rails is scalable, what do you think of Sun’s new Java-based scripting language, Google’s widgets API, Amazon S3, etc. If the candidate loves their field, they’ll have informed opinions on many of these topics. That’s what you want.
Ethics
Pick a topic you know intimately and ask the candidate increasingly esoteric questions until they don’t know the answer.
They’ll either say they don’t know, or they’ll try to bullshit you.
Guess what. If they bullshit you during the hiring process, they’ll bullshit you once they’re onboard.
He also links to Software Interview, a site which lists questions and answers that several companies use in their hiring processes.

6 May 2007

Andree Leidenfrost: Etch Upgrade - Sweet As

I upgraded my little file & print server and firewall (in UML) VIA C3 box on the weekend.

Everything went perfectly smooth, it even worked to upgrade via the apt-proxy that's running on the box. Super cool!

The only issue I had was non-working HTTP connections via my Telstra Bigpond cable connection after the upgrade of the firewall UML. In the end I fixed it by forcing the same MTU on the cable-facing as on the LAN-facing interface. It may be related to NAT, but it worked with the same 2.6.18 kernel before the upgrade, so maybe it is an iptables thing.

Andree Leidenfrost: Mondo Rescue 2.2.2 Packages Ready

New packages for amd64 and i386 are in unstable now. I did have to fix a few things here and there to get it to work. The biggest change is internal, however, I've finally moved to using upstream's install.sh script for the Debian package.

I've tested with kernels 2.6.18-4-amd64 (amd64) and 2.6.18-4-k7 (i386) and the usual assortment of nasties, i.e. NFS, AFS, LVM & RAID using NFS as restore media and amd64 and i386 as platforms.

Looks like 2.2.3 will be hot on its heels because of a bzip2-related issue with 2.2.2. (I normally use gzip, i.e. '-G' option, because it is heaps faster and compression rates are really only slightly worse.)

5 March 2007

Andree Leidenfrost: IA64 Support for Mondo Rescue!

The first fruit of my work on Debian IA64 packages for Mondo Rescue are now on http://people.debian.org/~andree/packages/. It took a bit longer than I had anticipated and there was a bit of an interruption while I stayed on beautiful Norfolk Island for two weeks.

Everything should be fine and dandy apart from the fact that restores will currently miserably fail on IA64 because parted2fdisk expects an older version of parted than what Etch and Sid contain.

So, why publish IA64 packages at all if they don't even work, I hear you scream. Well, firstly I thought I give a bit of an update and show that things are actually moving forward. Secondly, maybe I'm lucky and someone is interested in helping out with getting parted2fdisk to work with newer versions of parted or (better) in getting mondo migrated to using parted natively on all platforms (pending Bruno's approval).

Other than that, the packages have a few user-visible improvements and fixes across all supported architectures and they bring the Debian packages a fair bit closer to upstream - see the changelogs for details.

As usual, check out the new packages, let me know if something doesn't (or does) work, get involved! ;-)

25 January 2007

Holger Levsen: Melbourne...

Andree, it's been announced in the LCA closing ceremony: Melbourne will be brilliant! :)

Currently Melbourne is not so brilliant: I have fever and lay in bed. And except for the really beautiful botanic garden I have not seen much yet, and I somewhat doubt I'll see a lot more this time.

Andree Leidenfrost: linux.conf.au 2007 Gleanings

I was fortunate enough to attend lca2007 last week and to meet Bruno Cornec, the Mondo Rescue project leader, for the first time in the flesh.

The conference was pretty cool, and I found the quality of the program to be generally impressingly high: Excellent topics, superb presentation skills, humor and genuine passion for the topic at hand. The atmosphere and organisation were great, too - extremely well done, 7 Team!

My personal two highlights were:
Apart from the conference as such, it was super cool to meet Bruno. We had a number of good conversations and did some nice hacking, burning the midnight oil a couple of times. And HP gave me a (used) zx2000 (which Bruno carried all the way from France to Sydney!) to enable me to work on the (Debian) Itanium port of Mondo Rescue - thank you HP and Bruno! Bruno's talk was well received, and it was great to actually meet people that use the software that I spend quite a bit of time on. :-)

Finally, there is the Linus phenomenon (so you can tell this is my first
Linux conference) - he amazingly is basically left in peace: People who know him obviously talk to him and he participates vividly in discussions and so forth, but everyone else does the business as usual thing - no autographs, no photos, no nothing. I was (positively) impressed and refrained from asking him to sign my copy of Just for Fun (just kidding).

Finally, finally, I propose to find an alternative for the term 'awesome' next year. ;-)

14 January 2007

Andree Leidenfrost: Debian Pre-Release of Mondo Rescue 2.2.1, Take 2

mindi-2.21~r1021-2 and mondo-2.21~r1021-2 are now on http://people.debian.org/~andree/packages/ with the following changes:
  • petris works again during restore (self-inflicted, oh well).
  • Restore of ISO images and friends should now work when gzip (i.e. '-G') is used.
  • The network interfaces should be fine now when booting into a restored system for the first time.
Other than that:
  • The crash in mondorestore when nuking from tape has disappeared. I have no idea what caused it or why it went away again, though...
  • Kernel 2.6.18-3-k7 hangs when 'acpi=off' is specified (which is the default as per mindi's ADDITIONAL_BOOT_PARAMS, so restore fails with this kernel). I have filed bug #406809 which may or may not be related to #389931.
  • The issue with booting a (NTFS) Windows partition failing after a restore appears to be normal as per the ntfsclone manpage:
    Usually, Windows will not be able to boot, unless you copy, move or restore NTFS to the same partition which starts at the same sector on the same type of disk having the same BIOS legacy cylinder setting as the original partition and disk had.
  • I have tried a few things playing with parted and ntfsresize, but so far the only thing that works reliably in order to get Windows to boot is resizing the partition using gparted. I still have to figure out what it is that gparted does differently. (If you know, please tell me!)
Off to bed now so that I'm fine and dandy when I pick up Bruno from the airport in the morning for lca2007. :-)

8 January 2007

Andree Leidenfrost: Debian Pre-Release of Mondo Rescue 2.2.1

Preliminary Debian packages for Mondo Rescue 2.2.1 are available at the usual place: http://people.debian.org/~andree/packages/. (r1021 is the SVN revision of the actual 2.2.1 release.)

The new version comes with quite an impressive list of fixes and enhancements as can be seen on the Mondo Rescue website. The new '-G' option which allows for using gzip as the compressor is my personal favourite as it really speeds things up! Also, I've finally gotten around to fixing #222065 and there is no more unsolicited creation of directories in /var/cache anymore (neither submitted upstream yet).

Unfortunately, there are a few issues:
  • Links in directories that actually are links themselves are not resolved correctly (fixed in the Debian package.).
  • nuke restore from tape makes mondorestore crash (at least on amd64). The workaround is to use automatic or interactive mode.
  • Windows 2000 does not boot after a restore (at least on amd64). The data is restored but there is something wrong with how fdisk creates the partition (I believe). This actually also happens with 2.2.0 as well, so it is not a regression. I have to look into this more. The workaround is to use gparted (or similar) and resize the windows partition/filesystem by a few MB.
  • petris doesn't start when restoring (again at least on amd64). Surely the most minor issue I noticed so far...
Anyway, apart from the above, things seem to be ok. Give it a try if you like! If you do find more issues, let us know via the mailing list and - even better - send a patch. ;-)

Other than that, (belated) Happy New Year to everyone! :-)

26 December 2006

Andree Leidenfrost: Mondo Rescue Debian News

I've used the Christmas time to do some work on Mondo Rescue:

I've uploaded mindi-2.20-2 which fixes RC bug #403454 and important bug #404315. Luckily, I only had to apply patches provided by Matija & Bruno - thanks guys! I've asked for the new version to be allowed into Etch but haven't heard anything yet (maybe I should have waited a few days first or something?).

Also, I've packaged the new pre-release version of Mondo Rescue. You can grab it from here: http://people.debian.org/~andree/packages/. I would much appreciate feedback on how things are working on Etch. So, if you take them for a spin, please let me know how it went, good or bad! The upcoming new release fixes a substantial number of bugs and comes with some notable improvements, so it should definitely be worth checking out.

Finally, when doing some testing I came across a problem with the stock Debian 2.6.18 kernel when used on a Mondo Rescue recovery media. It sometimes already hangs while booting and sometimes starts the restore only to hang later during formatting or even later during restore. This seems to only happen on i386 whereas amd64 is fine. It may also be restricted to my hardware (although it hangs in Qemu as well). 2.6.17 is fine btw. If anyone has an idea what could be causing this, that would be great as I haven't got the foggiest at the moment (there is no message whatsoever, it just hangs).

3 December 2006

Andree Leidenfrost: Upgrade Testing with QEMU

I have been really busy until two weeks ago [Shameless plug: If you need a (technical) project manager and/or hands-on Basis person to get your SAP R/3 system upgraded to ECC6, get in touch.]. Which means I've been a bit slack - so slack in fact that Andreas Barth had to resolve an RC bug in one of my packages - mea culpa.

Ridden by guilt I decided to finally do what I've been wanting to for some time: Putting the steps together to do a Debian test upgrade using QEMU. I chose QEMU because it is free and readily comes with Debian. It is slower than VMware, at least without the non-free kernel module, but still usable on somewhat reasonable hardware. The actual upgrade steps given below are independent of the virtualisation technology used, so they will apply to VMware as well.

Without further ado, here goes:
All in all, things seem to work ok. However, this is far from hitting the friendly green upgrade button and it just happens. So, I thought I try the next best thing to the friendly green upgrade button which is synaptic. Doing an upgrade with synaptic does actually work quite smoothly. It needs to be restarted a few times, leaves some cruft in terms of obsolete and orphan packages and the reboot doesn't really work from within Gnome, but other than that it is ok. Most notably, it didn't leave me with an unusable system in the middle of it all.

Robert Collins made some interesting remarks about the challenges of upgrades last week when we had dinner with Martin Krafft and a number of other great people. It looks like the Ubuntu folks are working on improving update-manager but also the underlying infrastructure to smooth out the upgrade process. Maybe there could be an opportunity to work together on this and achieve a situation where upgrades become as smooth as installs are now due to the fantastic work of the d-i people.

9 October 2006

Andree Leidenfrost: New Upstream Mondo Rescue Packages

I have just uploaded mindi and mondo 2.20 (version numbers are now the same for mindi and mondo).

This release marks a bit of a milestone in that we can ship the pristine upstream source in Debian for the first time because the busybox binaries were removed upstream. This together with the numerous bug fixes and stability improvements and combined with the fact that there are no revolutionary new features (despite the version number jump) made me decide to try and push the new upstream version into etch.

'Try' because although I've tested things quite thoroughly on both amd64 and i386, there is always a residual risk (one of my favourite alliterations ;-) ) of something bad slipping through. I think that taking this risk is justified, though, because 2.20 should be a definite improvement over 2.09. Bruno and I worked really hard to get this released in time for etch (Thank you, Bruno, for making my priority your priority!).

While testing, I found #389931, #389729 and #390653 (Thanks for your great help, Russ!). I've also found that archiving to tape doesn't work for me (still gathering more information, so no bug report yet), but it also fails with 2.09, i.e. there is no regression.

Main testing was performed on:
Oh, and this also contains the fix for RC bug #391127.

People seem to be using Mondo Rescue on 'vintage' versions of Debian which is presumably due to Mondo Rescue being disaster recovery software. So, I've made a change to post-nuke suggested by Augustin Amann (Thank you, Augustin!) to make people using the latest packages on sarge happier. I'm still in two minds about adding versioned depends on grep and binutils to make the life of woody (!) users a bit easier as well.

8 October 2006

Andree Leidenfrost: Freedom vs. Engineering, Part II

Ingo: What you describe is still an engineering problem, not one of freedom. If you can't do it yourself, you could find someone to do it for you. The point is that all the bits and pieces are freely accessible.
If you truly believe that the dependencies on tmpfs and udev should only be recommends, then I think you should file bug reports against the affected packages.

PS: I've had trouble with package dependencies as a user myself. I've also had a bug filed against one of my packages where I did reduce a depends to a recommends. So, I think I understand where you are coming from and generally agree that dependencies can be a chore. My point is merely that this is not about freedom but about engineering.

PPS: Seems like the PS turned out as long as the post. ;-)

Andree Leidenfrost: RE: Being forced into non-freeness of choice by Debian packages

Ingo: Maybe I am a bit over-sensitive here, but whilst I feel your pain, your complaint is a pure engineering issue and has nothing to do with (software) freedom. You have all the freedom to take the software packages you mention and change them so you don't have to use udev or tmpfs.

Next.