Search Results: "xam"

18 March 2011

Adnan Hodzic: Debian CUT, a new rolling release?

This post is also available on/was written for OMG! Ubuntu

It looks like 2011 started well for Debian. The project won awards in two out of seven categories at the Linux New Media Awards 2011 ( Best Open Source Server Distribution and Outstanding Contribution to Open Source/Linux/Free Software ). Just recently Internet.com declared Debian the most influential distribution ever, stating that ~63% of all distributions now being developed come ultimately from Debian. However, my intention for this article is not solely to praise Debian for its recent awards, but rather to focus on a new project, Debian CUT. Don t be surprised if you haven t heard about CUT; it seems most Debian community hasn t either. Then again, maybe it s because it is only labelled as unofficial/development so far.
A bit of history One of the greatest criticisms of Debian is that its release cycles are too long. Debian stable release is seen as often as Ubuntu s LTS release. As a server solution this doesn t present a problem at all, it can even seen as a pro. However, for desktop use and for your average Joe who needs to have the latest software and is unable to get it, this may well present a problem. Of course he can always turn to backports to get what he needs but by the time you have finished reading this very sentence, Joe has already moved to Ubuntu. For those who are completely unaware how things work within Debian, let me try to shed some light. Debian has 3 main branches: Unstable , is a branch used mainly by developers, where the latest changes to the software they are working on are made. Usually, after approximately 10 days this software is pushed to testing , branch which is going to become next Debian stable release. For comparison, in the process of a new Ubuntu release at one point in this process, changes from testing/unstable are frozen where Ubuntu fixes bugs until they are ready to release new Ubuntu release. Debian s testing branch is said by some to be more stable then most of the stable distributions out there. Last but not least is stable branch, which follows the (in)famous mantra it s released when it s ready , although last two releases have been released in a more timely manner. Okay, so what is Debian CUT? The idea stretches back more then 2 years, to Joey Hess proposing the idea of Constantly Usable Testing . For users/developers who can t wait ~2 years to see new Debian release, they usually turn to Debian testing branch. To clarify, testing images are released weekly, but most of us in Debian don t recommend you jump to testing by installing it from weekly image, but rather by upgrading from stable. To make the situation even worse, these builds frequently don t work due to all the changes that have been pushed from unstable. But this is where Constantly Usable Testing idea comes in, where the installer is always installable and you re already using what s going to be next stable release. You wouldn t have to worry about whether the next update is going to break your system, but would get regular security updates, while the big updates would come in shape of new testing snapshot versions. Frequency of release of these snapshots is ought to be on monthly basis, so you can plan on seeing the next release on April 6th (exactly one month after the current release, provided of course if there are no showstopper bugs). To sum it all up, you wouldn t have to wait ~2 years before new Stable release shows up, nor (in case you have already switched to testing) would you have to worry whether your next update will break your system. You would get your updates in form of timely (ie: monthly) snapshots until it s time for major milestone which would turn out to be new Debian release. Even though this might be long term goal, in these early stages no one can guarantees this. Rolling release? The simplest way to explain this concept is that there are two types of Linux distributions: one with milestone version numbers and other where new updates keep coming through their rolling release cycles. Some of the rolling distributions are Gentoo and Arch Linux, while on the other side you have RedHat and Suse. Novell announced that OpenSuse is moving to rolling release with project codename Tumbleweed. It s also worth mentioning that Debian already has its rolling counters and perfect examples are: LMDE and aptosid. It is a way to continuously develop software, a way that best fits Debian as a platform where software is continuously developed. Especially because most of its greatest values could get lost with the release of that big stable version . Perfect case of that could be the almost drop of Chromium in Squeeze. Another great example could be Android, which is criticised most for its versioning. Companies are refusing to release new Android versions to their current phones, because they want you to buy new phone with latest Android version, even though your current one is perfectly capable of running latest Android. Personally, I m sure the future will reveal that rolling releases are the future, towards which all Linux platforms should be heading. Perhaps the definition of rolling releases I gave earlier is why I believe this project has such bright future ahead of it. Debian would be somewhat of a hybrid of this definition, it would be rolling release until it s time to mark a big milestone with a version number, and before you know it, we re rolling again. Unofficial Debian Monthly Testing Snapshot Release (version 2011.03 final) Michael Gilbert took on a big responsibility, trying experimentally to prove the feasibility of such a project. He has released a first wheezy snapshot installer (versioned 2011.03) as a test for the development community to try out and evaluate. Please read official announcement along with download links. While you can use this snapshot to have constantly working installer, after it is installed you could move to testing altogether if you wish. Conclusion As I have already mentioned, this project is still in the development/experimental stage, which makes all of this its very conception. Also, apart from rumors of Ubuntu becoming rolling release, if Ubuntu is to get its rolling release this how it s going to get it Smile If you d like to find out more about Debian CUT, I suggest you read Raphael Hertzog s article A constantly usable testing distribution for Debian , and watch the video of Joey Hess CUT BoF on DebConf10 in NYC. Of course, if you d like to get involved into this project, please use the project s mailing list. If you have any comments, please do share, as I d be happy to answer any of your questions.

16 March 2011

Thorsten Glaser: HOWTO OpenID Delegate with Launchpad Login Service

Rhonda suggested I document how to use the LLS (Launchpad Login Service their implementation of an OpenID provider) as Delegate, which basically means, you can put something up on your webpage, which can be a simple static (X)HTML page like mine (a /index.htm is especially nice, a /~user/index.htm works too), and use its URI and not https://launchpad.net/~me to login. For example, this often hides the LLS from view e.g. in blog comments, such as those where Canonical is being criticised but it s also yours, easier to type and to change if you switch service providers. The basic idea is to go to your Launchpad user page and view its page source. Look for openid relation links in the header on Rhonda s the value we re looking for is cyLQbcp , and you see it several times.

Now you put this on your web page:

	<!-- begin: OpenID delegation to LP -->
	 <link rel="openid.server" href="https://login.launchpad.net/+openid" />
	 <link rel="openid.delegate" href="https://login.launchpad.net/+id/cyLQbcp" />
	 <link rel="openid2.provider" href="https://login.launchpad.net/+openid" />
	 <link rel="openid2.local_id" href="https://login.launchpad.net/+id/cyLQbcp" />
	<!-- end: OpenID delegation -->

Of course, insert your, not Rhonda s, ID. Do note that we don t copy the X-XRDS-Location tag (that breaks things for some unknown reason), but otherwise, what we insert on our page is pretty much a copy of the info on the user page (maybe it s a Delegate page, too?). As usual, try at your own risk, bug Canonical if it breaks. It works with AO3, Gerrit Code Review, and others though (interestingly enough, better in Lynx than GUI browsers because I stay logged in across Lynx sessions (and just have to confirm sending my information to the accessing site), whereas I have to re-login to the LLS in every GUI browser session). As with the LLS generally, to access a site which is not recognised is expected and worked on with low urgency (mostly cosmetical, I think).

15 March 2011

Michal &#268;iha&#345;: phpMyAdmin translation status

Last weekend we've released phpMyAdmin 3.4.0-beta4, hopefully last beta and there are still lot of translations missing. The only new complete translation compared to last week is German, so we now have 13 translations at 100%: There are still lot of translations which would need some love, for example: I'm happy that not only translation status is improving, but some translators also reviewed older translations, so that it is more consistent. The great work in this area was done in Russian. And of course there is still great ongoing effort on Urdu translation. You can help make translation to your language better! If your language is missing just let me know (either in comments or by email).

Filed under: English Phpmyadmin 0 comments Flattr this!

Colin Watson: Wubi bug 693671

I spent most of last week working on Ubuntu bug 693671 ("wubi install will not boot - phase 2 stops with: Try (hd0,0): NTFS5"), which was quite a challenge to debug since it involved digging into parts of the Wubi boot process I'd never really touched before. Since I don't think much of this is very well-documented, I'd like to spend a bit of time explaining what was involved, in the hope that it will help other developers in the future. Wubi is a system for installing Ubuntu into a file in a Windows filesystem, so that it doesn't require separate partitions and can be uninstalled like any other Windows application. The purpose of this is to make it easy for Windows users to try out Ubuntu without the need to worry about repartitioning, before they commit to a full installation. Wubi started out as an external project, and initially patched the installer on the fly to do all the rather unconventional things it needed to do; we integrated it into Ubuntu 8.04 LTS, which involved turning these patches into proper installer facilities that could be accessed using preseeding, so that Wubi only needs to handle the Windows user interface and other Windows-specific tasks. Anyone familiar with a GNU/Linux system's boot process will immediately see that this isn't as simple as it sounds. Of course, ntfs-3g is a pretty solid piece of software so we can handle the Windows filesystem without too much trouble, and loopback mounts are well-understood so we can just have the initramfs loop-mount the root filesystem. Where are you going to get the kernel and initramfs from, though? Well, we used to copy them out to the NTFS filesystem so that GRUB could read them, but this was overly complicated and error-prone. When we switched to GRUB 2, we could instead use its built-in loopback facilities, and we were able to simplify this. So all was more or less well, except for the elephant in the room. How are you going to load GRUB? In a Wubi installation, NTLDR (or BOOTMGR in Windows Vista and newer) still owns the boot process. Ubuntu is added as a boot menu option using BCDEdit. You might then think that you can just have the Windows boot loader chain-load GRUB. Unfortunately, NTLDR only loads 16 sectors - 8192 bytes - from disk. GRUB won't fit in that: the smallest core.img you can generate at the moment is over 18 kilobytes. Thus, you need something that is small enough to be loaded by NTLDR, but that is intelligent enough to understand NTFS to the point where it can find a particular file in the root directory of a filesystem, load boot loader code from it, and jump to that. The answer for this was GRUB4DOS. Most of GRUB4DOS is based on GRUB Legacy, which is not of much interest to us any more, but it includes an assembly-language program called GRLDR that supports doing this very thing for FAT, NTFS, and ext2. In Wubi, we build GRLDR as wubildr.mbr, and build a specially-configured GRUB core image as wubildr. Now, the messages shown in the bug report suggested a failure either within GRLDR or very early in GRUB. The first thing I did was to remember that GRLDR has been integrated into the grub-extras ntldr-img module suitable for use with GRUB 2, so I tried building wubildr.mbr from that; no change, but this gave me a modern baseline to work on. OK; now to try QEMU (you can use tricks like qemu -hda /dev/sda if you're very careful not to do anything that might involve writing to the host filesystem from within the guest, such as recursively booting your host OS ... [update: Tollef Fog Heen and Zygmunt Krynicki both point out that you can use the -snapshot option to make this safer]). No go; it hung somewhere in the middle of NTLDR. Still, I could at least insert debug statements, copy the built wubildr.mbr over to my test machine, and reboot for each test, although it would be slow and tedious. Couldn't I? Well, yes, I mostly could, but that 8192-byte limit came back to bite me, along with an internal 2048-byte limit that GRLDR allocates for its NTFS bootstrap code. There were only a few spare bytes. Something like this would more or less fit, to print a single mark character at various points so that I could see how far it was getting: 
	pushal
	xorw	%bx, %bx	/* video page 0 */
	movw	$0x0e4d, %ax	/* print 'M' */
	int	$0x10
	popal
In a few places, if I removed some code I didn't need on my test machine (say, CHS compatibility), I could even fit in cheap and nasty code to print a single register in hex (as long as you didn't mind 'A' to 'F' actually being ':' to '?' in ASCII; and note that this is real-mode code, so the loop counter is %cx not %ecx): 
	/* print %edx in dumbed-down hex */
	pushal
	xorw	%bx, %bx
	movb	$0xe, %ah
	movw	$8, %cx
1:
	roll	$4, %edx
	movb	%dl, %al
	andb	$0xf, %al
	int	$0x10
	loop	1b
	popal
After a considerable amount of work tracking down problems by bisection like this, I also observed that GRLDR's NTFS code bears quite a bit of resemblance in its logical flow to GRUB 2's NTFS module, and indeed the same person wrote much of both. Since I knew that the latter worked, I could use it to relieve my brain of trying to understand assembly code logic directly, and could compare the two to look for discrepancies. I did find a few of these, and corrected a simple one. Testing at this point suggested that the boot process was getting as far as GRUB but still wasn't printing anything. I removed some Ubuntu patches which quieten down GRUB's startup: still nothing - so I switched my attentions to grub-core/kern/i386/pc/startup.S, which contains the first code executed from GRUB's core image. Code before the first call to real_to_prot (which switches the processor into protected mode) succeeded, while code after that point failed. Even more mysteriously, code added to real_to_prot before the actual switch to protected mode failed too. Now I was clearly getting somewhere interesting, but what was going on? What I really wanted was to be able to single-step, or at least see what was at the memory location it was supposed to be jumping to. Around this point I was venting on IRC, and somebody asked if it was reproducible in QEMU. Although I'd tried that already, I went back and tried again. Ubuntu's qemu is actually built from qemu-kvm, and if I used qemu -no-kvm then it worked much better. Excellent! Now I could use GDB: 
(gdb) target remote   qemu -gdb stdio -no-kvm -hda /dev/sda
This let me run until the point when NTLDR was about to hand over control, then interrupt and set a breakpoint at 0x8200 (the entry point of startup.S). This revealed that the address that should have been real_to_prot was in fact garbage. I set a breakpoint at 0x7c00 (GRLDR's entry point) and stepped all the way through to ensure it was doing the right thing. In the process it was helpful to know that GDB and QEMU don't handle real mode very well between them. Useful tricks here were: Single-stepping showed that GRLDR was loading the entirety of wubildr correctly and jumping to it. The first instruction it jumped to wasn't in startup.S, though, and then I remembered that we prefix the core image with grub-core/boot/i386/pc/lnxboot.S. Stepping through this required a clear head since it copies itself around and changes segment registers a few times. The interesting part was at real_code_2, where it copies a sector of the kernel to the target load address, and then checks a known offset to find out whether the "kernel" is in fact GRUB rather than a Linux kernel. I checked that offset by hand, and there was the smoking gun. GRUB recently acquired Reed-Solomon error correction on its core image, to allow it to recover from other software writing over sectors in the boot track. This moved the magic number lnxboot.S was checking somewhat further into the core image, after the first sector. lnxboot.S couldn't find it because it hadn't copied it yet! A bit of adjustment and all was well again. The lesson for me from all of this has been to try hard to get an interactive debugger working. Really hard. It's worth quite a bit of up-front effort if it saves you from killing neurons stepping through pages of code by hand. I think the real-mode debugging tricks I picked up should be useful for working on GRUB in the future.

14 March 2011

Mike Hommey: Avoiding dependencies upon recent libstdc++

Mozilla has been distributing Firefox builds for GNU/Linux systems for a while, and 4.0 should even bring official builds for x86-64 (finally, some would say). The buildbots configuration for these builds uses gcc 4.3.3 to compile the Firefox source code. With the C++ part of gcc, it can sometimes mean side effects when using the C++ STL. Historically, the Mozilla code base hasn t made a great use of the STL, most probably because 10+ years back, portability and/or compiler support wasn t very good. More recently, with the borrowing of code from the Chromium project, this changed. While the borrowed code for out-of-process plugins support didn t have an impact on libstdc++ usage, the recent addition of ANGLE had. This manifests itself in symbols version usage These are the symbol versions required from libstdc++.so.6 on 3.6 (as given by objdump -p): And on 4.0: This means Firefox 4.0 builds from Mozilla need the GLIBCXX_3.4.9 symbol version, which was introduced with gcc 4.2. This means Firefox 4.0 builds don t work on systems with a libstdc++ older than that, while 3.6 builds would. It so happens that the system libstdc++ on the buildbots themselves is that old, which is why we set LD_LIBRARY_PATH to the appropriate location during tests. This shouldn t however be a big problem for users. Newer gcc, new problems As part of making Firefox faster, we re planning to switch to gcc 4.5, to benefit from better (as in working) profile guided optimization, and other compiler improvements. We actually attempted to switch to gcc 4.5 twice during the 4.0 development cycle. But various problems made us go back to gcc 4.3.3, the main contender being the use of even newer libstdc++ symbols: GLIBCXX_3.4.14 was added in gcc 4.5, making the build require a very recent libstdc++ installed on users systems. As this wouldn t work for Mozilla builds, we attempted to build with -static-libstdc++. This options makes the resulting binary effectively contain libstdc++ itself, which means not requiring a system one. This is the usual solution used for builds such as Mozilla s, that require to work properly on very different systems. The downside of -static-libstdc++ is that it makes the libxul.so binary larger (about 1MB larger). It looks like the linker doesn t try to eliminate the code from libstdc++ that isn t actually used. Taras has been fighting to try to get libstdc++ in a shape that would allow the linker to remove that code that is effectively dead weight for Firefox. Why do we need these symbols? The actual number of symbols required with the GLIBCXX_3.4.14 version is actually very low: With the addition of the following on debug builds only: The number of symbols required with the GLIBCXX_3.4.9 version is even lower: It however varies depending on the compiler version. I have seen other builds also require std::ostream& std::ostream::_M_insert(double). All these are actually internal implementation details of the libstdc++. We re never calling these functions directly. I m going to show two small examples triggering some of these requirements (that actually generalize to all of them). The case of templates 
#include <iostream>
int main()  
    unsigned int i;
    std::cin >> i;
    return i;
This example, when built, requires std::istream& std::istream::_M_extract<double>(double&), but we are effectively calling std::istream& operator>>(unsigned int&). It is defined in /usr/include/c++/4.5/istream as: 
template<typename _CharT, typename _Traits>
class basic_istream : virtual public basic_ios<_CharT, _Traits>  
    basic_istream<_CharT, _Traits>& operator>>(unsigned int& __n)  
        return _M_extract(__n);
And _M_extract is defined in /usr/include/c++/4.5/bits/istream.tcc as: 
template<typename _CharT, typename _Traits> template<typename _ValueT>
        basic_istream<_CharT, _Traits>&
        basic_istream<_CharT, _Traits>::_M_extract(_ValueT& __v)  
            (...)
And later on in the same file:
extern template istream& istream::_M_extract(unsigned int&);
What this all means is that libstdc++ actually provides an implementation of an instance of the template for the istream (a.k.a. basic_istream<char>) class, with an unsigned int & parameter (and some more implementations). So, when building the example program, gcc decides, instead of instantiating the template, to use the libstdc++ function. This extern definition, however, is guarded by a #if _GLIBCXX_EXTERN_TEMPLATE, so if we build with -D_GLIBCXX_EXTERN_TEMPLATE=0, we actually get gcc to instantiate the template, thus getting rid of the GLIBCXX_3.4.9 dependency. The downside is that this doesn t work so well with bigger code, because other things are hidden behind #if _GLIBCXX_EXTERN_TEMPLATE. There is however another (obvious) way to for the template instantiation: instantiating it. So adding template std::istream& std::istream::_M_extract(unsigned int&); to our code is just enough to get rid of the GLIBCXX_3.4.9 dependency. Other template cases obviously can be worked around the same way. The case of renamed implementations 
#include <list>
int main()  
    std::list<int> l;
    l.push_back(42);
    return 0;
Here, we get a dependency on std::_List_node_base::_M_hook(std::_List_node_base*) but we are effectively calling std::list<int>::push_back(int &). It is defined in /usr/include/c++/bits/stl_list.h as: 
template<typename _Tp, typename _Alloc = std::allocator<_Tp> >
class list : protected _List_base<_Tp, _Alloc>  
    void push_back(const value_type& __x)  
        this->_M_insert(end(), __x);
_M_insert is defined in the same file: 
template<typename ... _Args>
void _M_insert(iterator __position, _Args&&... __args)  
    _List_node<_Tp>* __tmp = _M_create_node(std::forward<_args>(__args)...);
    __tmp->_M_hook(__position._M_node);
Finally, _M_hook is defined as follows: 
struct _List_node_base  
    void _M_hook(_List_node_base * const __position) throw ();
In gcc 4.4, however, push_back has the same definition, and while _M_insert is defined similarly, it calls __tmp->hook instead of __tmp->_M_hook. Interestingly, gcc 4.5 s libstdc++ exports symbols for both std::_List_node_base::_M_hook and std::_List_node_base::hook, and the code for both methods is the same. Considering the above, a work-around for this kind of dependency is to define the newer function in our code, and make it call the old function. In our case here, this would look like: 
namespace std  
    struct _List_node_base  
        void hook(_List_node_base * const __position) throw ();
        void _M_hook(_List_node_base * const __position) throw ();
     ;
    void _List_node_base::_M_hook(_List_node_base * const __position) throw ()  
        hook(__position);
which you need to put in a separate source file, not including <list>. All in all, with a small hack, we are able to build Firefox with gcc 4.5 without requiring libstdc++ 4.5. Now, another reason to switch to gcc 4.5 was to use better optimization flags, but it turns out it makes the binaries 6MB bigger. But that s another story.

Colin Watson: Wubi bug 693671

I spent most of last week working on Ubuntu bug 693671 ("wubi install will not boot - phase 2 stops with: Try (hd0,0): NTFS5"), which was quite a challenge to debug since it involved digging into parts of the Wubi boot process I'd never really touched before. Since I don't think much of this is very well-documented, I'd like to spend a bit of time explaining what was involved, in the hope that it will help other developers in the future. Wubi is a system for installing Ubuntu into a file in a Windows filesystem, so that it doesn't require separate partitions and can be uninstalled like any other Windows application. The purpose of this is to make it easy for Windows users to try out Ubuntu without the need to worry about repartitioning, before they commit to a full installation. Wubi started out as an external project, and initially patched the installer on the fly to do all the rather unconventional things it needed to do; we integrated it into Ubuntu 8.04 LTS, which involved turning these patches into proper installer facilities that could be accessed using preseeding, so that Wubi only needs to handle the Windows user interface and other Windows-specific tasks. Anyone familiar with a GNU/Linux system's boot process will immediately see that this isn't as simple as it sounds. Of course, ntfs-3g is a pretty solid piece of software so we can handle the Windows filesystem without too much trouble, and loopback mounts are well-understood so we can just have the initramfs loop-mount the root filesystem. Where are you going to get the kernel and initramfs from, though? Well, we used to copy them out to the NTFS filesystem so that GRUB could read them, but this was overly complicated and error-prone. When we switched to GRUB 2, we could instead use its built-in loopback facilities, and we were able to simplify this. So all was more or less well, except for the elephant in the room. How are you going to load GRUB? In a Wubi installation, NTLDR (or BOOTMGR in Windows Vista and newer) still owns the boot process. Ubuntu is added as a boot menu option using BCDEdit. You might then think that you can just have the Windows boot loader chain-load GRUB. Unfortunately, NTLDR only loads 16 sectors - 8192 bytes - from disk. GRUB won't fit in that: the smallest core.img you can generate at the moment is over 18 kilobytes. Thus, you need something that is small enough to be loaded by NTLDR, but that is intelligent enough to understand NTFS to the point where it can find a particular file in the root directory of a filesystem, load boot loader code from it, and jump to that. The answer for this was GRUB4DOS. Most of GRUB4DOS is based on GRUB Legacy, which is not of much interest to us any more, but it includes an assembly-language program called GRLDR that supports doing this very thing for FAT, NTFS, and ext2. In Wubi, we build GRLDR as wubildr.mbr, and build a specially-configured GRUB core image as wubildr. Now, the messages shown in the bug report suggested a failure either within GRLDR or very early in GRUB. The first thing I did was to remember that GRLDR has been integrated into the grub-extras ntldr-img module suitable for use with GRUB 2, so I tried building wubildr.mbr from that; no change, but this gave me a modern baseline to work on. OK; now to try QEMU (you can use tricks like qemu -hda /dev/sda if you're very careful not to do anything that might involve writing to the host filesystem from within the guest, such as recursively booting your host OS ...). No go; it hung somewhere in the middle of NTLDR. Still, I could at least insert debug statements, copy the built wubildr.mbr over to my test machine, and reboot for each test, although it would be slow and tedious. Couldn't I? Well, yes, I mostly could, but that 8192-byte limit came back to bite me, along with an internal 2048-byte limit that GRLDR allocates for its NTFS bootstrap code. There were only a few spare bytes. Something like this would more or less fit, to print a single mark character at various points so that I could see how far it was getting: 
	pushal
	xorw	%bx, %bx	/* video page 0 */
	movw	$0x0e4d, %ax	/* print 'M' */
	int	$0x10
	popal
In a few places, if I removed some code I didn't need on my test machine (say, CHS compatibility), I could even fit in cheap and nasty code to print a single register in hex (as long as you didn't mind 'A' to 'F' actually being ':' to '?' in ASCII; and note that this is real-mode code, so the loop counter is %cx not %ecx): 
	/* print %edx in dumbed-down hex */
	pushal
	xorw	%bx, %bx
	movb	$0xe, %ah
	movw	$8, %cx
1:
	roll	$4, %edx
	movb	%dl, %al
	andb	$0xf, %al
	int	$0x10
	loop	1b
	popal
After a considerable amount of work tracking down problems by bisection like this, I also observed that GRLDR's NTFS code bears quite a bit of resemblance in its logical flow to GRUB 2's NTFS module, and indeed the same person wrote much of both. Since I knew that the latter worked, I could use it to relieve my brain of trying to understand assembly code logic directly, and could compare the two to look for discrepancies. I did find a few of these, and corrected a simple one. Testing at this point suggested that the boot process was getting as far as GRUB but still wasn't printing anything. I removed some Ubuntu patches which quieten down GRUB's startup: still nothing - so I switched my attentions to grub-core/kern/i386/pc/startup.S, which contains the first code executed from GRUB's core image. Code before the first call to real_to_prot (which switches the processor into protected mode) succeeded, while code after that point failed. Even more mysteriously, code added to real_to_prot before the actual switch to protected mode failed too. Now I was clearly getting somewhere interesting, but what was going on? What I really wanted was to be able to single-step, or at least see what was at the memory location it was supposed to be jumping to. Around this point I was venting on IRC, and somebody asked if it was reproducible in QEMU. Although I'd tried that already, I went back and tried again. Ubuntu's qemu is actually built from qemu-kvm, and if I used qemu -no-kvm then it worked much better. Excellent! Now I could use GDB: 
(gdb) target remote   qemu -gdb stdio -no-kvm -hda /dev/sda
This let me run until the point when NTLDR was about to hand over control, then interrupt and set a breakpoint at 0x8200 (the entry point of startup.S). This revealed that the address that should have been real_to_prot was in fact garbage. I set a breakpoint at 0x7c00 (GRLDR's entry point) and stepped all the way through to ensure it was doing the right thing. In the process it was helpful to know that GDB and QEMU don't handle real mode very well between them. Useful tricks here were: Single-stepping showed that GRLDR was loading the entirety of wubildr correctly and jumping to it. The first instruction it jumped to wasn't in startup.S, though, and then I remembered that we prefix the core image with grub-core/boot/i386/pc/lnxboot.S. Stepping through this required a clear head since it copies itself around and changes segment registers a few times. The interesting part was at real_code_2, where it copies a sector of the kernel to the target load address, and then checks a known offset to find out whether the "kernel" is in fact GRUB rather than a Linux kernel. I checked that offset by hand, and there was the smoking gun. GRUB recently acquired Reed-Solomon error correction on its core image, to allow it to recover from other software writing over sectors in the boot track. This moved the magic number lnxboot.S was checking somewhat further into the core image, after the first sector. lnxboot.S couldn't find it because it hadn't copied it yet! A bit of adjustment and all was well again. The lesson for me from all of this has been to try hard to get an interactive debugger working. Really hard. It's worth quite a bit of up-front effort if it saves you from killing neurons stepping through pages of code by hand. I think the real-mode debugging tricks I picked up should be useful for working on GRUB in the future.

Erich Schubert: Google Circles rumor

The media are spreading a rumor about a social networking platform by Google, called Circles.
So far, these things have been debunked.
I do not believe that Google could succeed with a "Facebook clone". The market is already taken, Facebook is too big and had successfully been taking away international markets from local clones. For example in Germany, the other social networks used to be a lot bigger, but Facebook overtook them in just a few months, and the people I know pretty much quit the other networks then.
And I assume Google is aware that unless they have a strong strategy, the network would go the Wave-Buzz route.
At least here in Germany, people are surprisingly concerned by the amount of data Google might have on them, while at the same time they give it to Facebook for free. This is probably due to the media attention received with StreetView. It's not fair, but that's life.
However, "social network" is a broad term. Just think of what people actually use facebook for:
  • Games (often even not with their social circle!)
  • Microblogging
  • Photo sharing
  • Email
  • Automatic address book
Now, if you look closely at this list, Google pretty much has all of these, if we ignore the browser games (and they are all over the internet by now, Facebook and without!), then there is just one key ingredient missing for Google: the address book thing.
I believe this "Google circles" thing will largely by a "smart address book" that helps people managing their social contacts in a social network style. And obviously, this can be integrated in various products such as Mail, Picasa, Buzz, ...
If Google manages to launch a "contact manager" that makes it really easy for people to manage their social circles, this can be quite a killer. Facebook has "lists" but they are awful to use. It likes to "hide" friends to reduce the amount of information it throws at you. But it doesn't really organize it for you, for example by social circles or topics. These days for example Facebook could split the news feed into "Japan" and "everything else" I guess.

Rapha&#235;l Hertzog: Missing firmware in Debian? Learn how to deal with the problem

You know it already, since Debian 6.0 non-free firmware are no longer provided by a standard Debian installation. This will cause some troubles to users who need them. I m thus going to do a small overview on the topic and teach you what you need to know to deal with the problem. What are firmware and how are they used? From the user s point of view, a firmware is just some data that is needed by some piece of hardware in order to function properly. The driver for that hardware typically loads the firmware on the device as part of its initialization. In the Linux kernel, the drivers are all using a standardized interface (request_firmware) to retrieve the firmware before sending it to the device. Thanks to this standardization, it s possible to either embed the firmware in the kernel or to have it request the firmware from user-space when needed. Debian (like most distributions) has selected the latter option. Thus when the kernel needs a firmware, it sends out a request to user-space. udev is getting the request with the name of the firmware, and in its default configuration (see /lib/udev/rules.d/80-drivers.rules) it executes /lib/udev/firmware.agent in response. Where are firmware stored? firmware.agent is a simple shell script that tries to locate a firmware before sending it back to the kernel through a sysfs entry. It looks into the following directories: Firmware provided by packages are thus usually in /lib/firmware and you can use /usr/local/lib/firmware for manually installed firmware. How do I know whether I need a firmware? First of, you can notice messages from the kernel telling you that it tried to load a firmware but it failed. They look like this: 
e100: eth0: e100_request_firmware: Failed to load firmware "e100/d101m_ucode.bin": -2
But you might be informed sooner. When you install a new version of the Linux kernel with the official Debian packages, the post-installation script will go through all loaded modules (those listed by lsmod) and it will verify whether this module as provided by the newly installed kernel might require firmware files. This information can be retrieved with modinfo: 
$ modinfo -F firmware /lib/modules/2.6.32-5-amd64/kernel/drivers/net/e100.ko
e100/d102e_ucode.bin
e100/d101s_ucode.bin
e100/d101m_ucode.bin
If one (or more) of those firmware is (are) not yet available on the system, you will get a warning message similar to this one:
update-initramfs will also generate a similar warning on the terminal: 
update-initramfs: Generating /boot/initrd.img-2.6.32-5-amd64
W: Possible missing firmware /lib/firmware/e100/d102e_ucode.bin for module e100
W: Possible missing firmware /lib/firmware/e100/d101s_ucode.bin for module e100
W: Possible missing firmware /lib/firmware/e100/d101m_ucode.bin for module e100
The Debian installer also detects when you have hardware that might require a missing firmware file. You have the option to supply the missing files on a USB stick (either directly or through the corresponding package). How do I find and install the missing firmware? Now that you have the name of the firmware file that you want, it s relatively easy to identify the package that provides the required file. You can use apt-cache search <filename> because the firmware packages embed the list of firmware files in their description. You can also use apt-file (provided by the package of the same name) or the web interface at packages.debian.org. 
$ apt-cache search d101m_ucode.bin
firmware-linux-nonfree - Binary firmware for various drivers in the Linux kernel
$ apt-file search d101m_ucode.bin
firmware-linux-nonfree: /lib/firmware/e100/d101m_ucode.bin
If the above commands return nothing, you probably need to enable the non-free repository in your /etc/apt/sources.list (you can also enable it within synaptic). And you also want to run sudo apt-file update to have the latest information. Now you can install the right package, in the example above it was firmware-linux-nonfree. How do I install all firmware just to be sure I don t miss any? There s no meta-package depending on all firmware packages so there s no easy answer to this question. Furthermore not all firmware packages respect the naming convention firmware-* (there s for example zd1211-firmware). So your best bet is to look up all the packages with a generic search like this one: 
$ apt-file --package-only search /lib/firmware/
atmel-firmware
[...]
And then install them all. Are there DVD or CD images with non-free firmware? Yes. Debian provides an unofficial netinst image for i386/amd64/powerpc with the non-free firmware, you can find it here: http://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/. I provide complete DVD sets with firmware, and Multi-Arch CD/DVD with firmware, it s over there in my DVD shop: http://raphaelhertzog.com/go/debian-cd/ (i386/amd64 only) When using those installation discs, the Debian-installer will be able to find the required firmware immediately. There s no need for you to provide them on a USB stick. Other questions? I think I covered the most important things you have to know about firmware on Debian. But should you still have a question, feel free to share it in the comments so that I can improve this article.
Click here to subscribe to my free newsletter and get my monthly analysis on what s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.

6 comments Liked this article? Click here. My blog is Flattr-enabled.

Russell Coker: Effective Computers for Schools

Sam Varghese has written an article that is very skeptical of the educational value of the OLPC project [1].Are Laptops Any Good for Schools?Sam cites an article in the New York Times by Winnie Hu about schools removing laptop programs due to a lack of success [2]. Winnie s article gives an example of a school shutting a program because of getting new teachers who lacked computer skills and some examples of schools which had issues with the repair cost. The solution to that would be computers that are more robust and easier to use by all accounts the OLPC systems are solidly constructed and easy to use!It is claimed that using laptops doesn t increase test scores. Using only test scores to compare educational methods is a way to lose, probably any school which does nothing other than try to increase test scores is worth avoiding. One example from Winnie s article is a report by Mr. Warschauer of students at a middle school in Yarmouth, Me., who used their laptops to create a Spanish book for poor children in Guatemala and debate Supreme Court cases found online . What value can you place on having students develop books for poor children in other countries and debate Supreme Court cases? If this sort of program became popular then it would lead to the countries which do it becoming better places to live in future decades!Mr. Warschauer also claims that If the goal is to get kids up to basic standard levels, then maybe laptops are not the tool. But if the goal is to create the George Lucas and Steve Jobs of the future, then laptops are extremely useful . I would go further than that, I think that providing the educational environment that involves international charity work and analysis of important court cases would lead to students having the skills to become good engineers, designers, and artists who work for people like George Lucas and Steve Jobs. No matter what you might do the vast majority of children will not grow up to be like George Lucas and Steve Jobs, there is a very limited number of positions for such people. But those people employ a huge number of creative people who do interesting and enjoyable work for good pay.I don t think that Winnie s article supports criticism of the OLPC project. I don t think that Winnie s article even contains sufficient evidence to match the headline.Playing GamesSam also cites an article about the misuse of school computers for playing games [3]. Of course there are ways of limiting access to games based on time etc. For a Linux system you could have a root cron job that runs chmod on /usr/games at various times, and preventing users from easily running programs that they install isn t difficult.Not that games are all bad, the flash-based games on www.physicsgames.net do teach kids some things about physics although I admit that much of that could be learned by playing with balls, skate-boards, etc.The Problem with the OLPC ProjectI think that the policy change to make OLPC systems only available to disadvantaged children was a mistake. The give one get one program was a great idea, maybe not the most effective way of getting funding but good for getting developers. While the new Sugar on a Stick project to put the OLPC GUI on a bootable USB device [4] it doesn t compare well to having dedicated hardware IMHO.Minimal Use of ComputersI have previously written about the weight of school bags and how laptops can alleviate the risk of health problems related to carrying heavy text books [5]. Since that time ebook readers and tablets have become incredibly cheap. Ebook readers are really light and the recent tablets are also very light (particularly the smaller ones). It seems to me that every school should at least be moving towards every student having an ebook reader and to use ebooks for all books that are part of the curriculum.At the moment there are a bunch of tablets on sale for about $150, that includes tablets that can access the Internet via Wifi or via 3G and a cheap 3G plan costs less than $150 per annum. If tablets were used for some of the computer tasks related to schools then a lot of the difficulty of repairing hardware would be removed. Tablets don t offer as many options for messing up the software configuration and if the hardware breaks it s easy to transfer the data onto a new tablet from the cloud with a maximum cost of $150 to replace the hardware.Also a cloud-based computing model could permit students to access all the same data from school and home while using desktop computers at both locations.Current SuccessNatalie Craig wrote an interesting article for The Age about The Lab a drop-in computer center for kids on the Autism Spectrum run by Dale Linegar and Stefan Schutt [6]. The fact that kids who have communication difficulties can find it easier to communicate electronically should be fairly obvious to everyone in the free software community, but the creation of an organisation to support such kids is noteworthy I think that such centers should be funded by the government and run in every city.Salman Khan gave an interesting TED talk about his project the Khan Academy [7] which is an online video-based teaching system that evolved from some Youtube videos he produced to tutor his cousins. Salman pointed out that his cousins said that they would rather watch his videos than have him teach them in person, it seems that they liked being able to pause and rewind the talk as well as avoiding the pressure of human interaction. The Khan Academy has videos in Adobe flash format, but conveniently they have download links so those of us who don t have the Flash plugin can still view the content [8].One of the interesting things about the Khan Academy is that in school use the videos are being assigned to kids as homework and the class time is used for teaching children to do worked examples what has traditionally been homework.So it seems that there are real examples of special-needs kids and average kids benefiting from different types of electronic learning.ConclusionI think that almost everything about the education system is broken. I also think that in many ways the education system has reached a local maximum, so there is no small way of improving things, and adding computers without making any of the significant changes needed to fix the big problems probably can t do a lot of good. In spite of this computers can provide some real benefits and I expect that those benefits can be better than the benefits of other potential ways of spending the money (IE it s worth the opportunity cost). I also have no doubt that almost anything can give a negative result if done badly enough, so I don t interpret examples of computer use failing in a school as anything other than evidence of a failing school.While the Khan Academy has some results that seem very positive I think that s only the first step of what needs to be done. It also seems to be based around users who have good Internet access which often isn t the case in places where the OLPC is being deployed. This implies that we need to get systems like the Khan Academy designed to operate in a disconnected manner, maybe with a remote server that s not connected to the Internet. This should be possible if funding is available.Sam suggested that there needs to be a scientific study of the effectiveness of the OLPC. I think that every project which involves significant amounts of public funding should be researched to ensure that it s an effective use of resources. 400,000 laptops is going to cost something more than $40,000,000 so it seems reasonable to devote a few person-years of research to determine how effective they are and which of the possible ways of using an OLPC will give the best results.I am confident that a good study of the effectiveness of the OLPC would demonstrate that it provides real educational benefits when correctly incorporated into an educational program. I also expect that such a study would show some significant differences in the effectiveness of various ways of using them. But we shouldn t be relying on confidence, we need some test results.

13 March 2011

Mark Brown: Changing core code

One of the biggest differences between working on most other OSs and working upstream on drivers for the Linux kernel is that elsewhere the core is usually a fixed thing that has been released and can t really be changed, even if source is available (which may not even be the case). If whatever subsystem you re working in can t cope with the thing you re trying to express then one of the things it s really useful to be good at is working out ways to implement the behaviour that s required behind the back of the subsystem. It s very common to see such code in drivers submitted by hardware vendors, particularly those that are new to Linux, as this is such a big mindset shift. With kernel development the approach is much more to make sure that the core can cope with whatever needs expressing. It s not always done in the core for example, sometimes the issue is due to unusually limited hardware and very likely to never come up again but it s very unusual to see code merged that doesn t at least fit in with the design of the subsystem. This post is inspired by a brief exchange with Thomas Gleixner on linux-kernel who articulated the advantages of doing things this way very well: 
 - It's usually simpler and faster as the core code has all the
   necessary information. So that's even an argument which managers
   might understand.
 - Such workarounds, when not caught, tend to spread themself
   magically because driver writers checkout the existing code of
   similar devices and copy/paste/modify^Wuglify them over and over.
 - In the worst case such workarounds make the core maintainence
   harder and in some cases impossible, because they silently imply
   semantics on the core w/o the core maintainer knowing about them.
The last point is the most important one to me as a subsystem maintainer I really don t want to be merging code that is going to create issues with ongoing development of the subsystem. This is all another facet of the policy on stable APIs: APIs are only worth working around if they re fixed, and the assumptions that end up embedded in a workaroud are as much part of the API as the explicit interfaces. It s things like this that make Linux such a pleasure to work on.

11 March 2011

Pietro Abate: On equivalent debian versions

For one of our experiments, we ended up analyzing all versions mentioned in a Debian Packages file if mentioned in a constraint or in a version field of a package. It seems that there are a lot of DDs that like to use strange 'formatting' when writing down a versioned constraint. This is might be not a choice done directly by the debian maintainer, but just a consequence of a particular version schema from upstream. The debian policy is rigorous with relation to the algorithm to compare version . 
 The strings are compared from left to right.
 First the initial part of each string consisting entirely of non-digit characters is determined. These two parts 
 (one of    which may be empty) are compared lexically. If a difference is found it is returned. The lexical 
 comparison is a comparison of ASCII values modified so that all the letters sort earlier than all the non-letters 
 and so that a tilde sorts before anything, even the end of a part. For example, the following parts are in sorted 
 order from earliest to latest: ~~, ~~a, ~, the empty part, a.[34]
 Then the initial part of the remainder of each string which consists entirely of digit characters is determined. 
 The numerical values of these two parts are compared, and any difference found is returned as the result of 
 the comparison. For these purposes an empty string (which can only occur at the end of one or both version 
 strings being compared) counts as zero.
 These two steps (comparing and removing initial non-digit strings and initial digit strings) are repeated until a
 difference is found or both strings are exhausted.
I think the important part is about the numerical comparison The numerical values of these two parts are compared, and any difference found is returned as the result of the comparison . Our tool finds a number of examples of equivalent versions : For example version '0.00001-1' is from package libbenchmark-progressbar-perl. I don't know why so many equivalent way of writing a version string are used. Probably they appear in completely unrelated packages I'm pretty sure no harm is intended :) However, to avoid confusion, it might be a good idea to settle on a non-normative canonical representation of versions. Like no '0' in front of dots and hyphen... Maybe we could add a warning in lintian ? If somebody is interested I can generate a full report that associates each version to a package name.

Guido G nther: git-buildpackage tip #4: creating remote repositories

In order to ease creation of remote repositories to publish your Debian packaging work git-buildpackage ships gbp-create-remote-repo: If you run 
gbp-create-remote-repo --remote-url-pattern='ssh://git.debian.org/home/users/agx/public_git/%(pkg)s.git'
from inside a git repository it will create a remote repository on git.debian.org in the given directory, push your debian, upstream and pristine-tar branches into it and set up branch tracking so you can easily use gbp-pull or git pull to update your local repository from there. '%(pkg)s' will be replaced by the source packages name. If you want to be more explicit you can use: 
gbp-create-remote-repo --remote-url-pattern='ssh://git.debian.org/home/users/agx/public_git/foo.git'
so no substitution will take place. In case you only want to push to that repository but not pull from there you can use the --no-track option so gbp-create-repo won't set up any branch tracking. If you specify the remote-url-pattern via gbp.conf calling: 
gbp-create-remote-repo
is enough to create a repo on your favorite git server. The default is to create new repositories in the collab-maint repository on alioth.debian.org. This feature is available in your $PATH since 0.5.16 (it was shipped in examples/ before) and was inspired by the Debian OCaml Maintainers dom-new-git-repo work. This blog is flattr enabled.

9 March 2011

Rog rio Brito: (Some) Solutions with Chromium

In my previous post, I was pointing out some of my frustrations with Chromium/Chrome. A very kind reader, Timo Juhani Lindfors, gave me answers to many of the problems that I pointed out and got me straight on some issues:
  • I can disable full zoom in Iceweasel (i.e., I only want to zoom the text, not the images). According to Chromium's project BTS, it seems that this will be a WONTFIX bug, despite many people asking for it (and it having a lot of duplicates).
Timo told me that there is an extension called "Font Size Increase" (ID: ombpcpigmndepfckcifdblemkabaoihk) that adds a button on the side of the URL bar that you can click to increase the size of the text, without affecting the size of the images. Highly recommended. Not that I need this one as much as "Font Size Increase", but there is also a "Font Size Decrease" (ID: mpajngnpcmjjeoflljdjpnehcfaldcia) that, as the name implies, decreases the size of the text. It is nice to have this one when you clicked on the button to increase the size of the font one or two times too many.
  • In Iceweasel, I have a way (without resorting to many extensions or hackish solutions) to use the equivalent of Firefox's "View > Page Style > No Style" (i.e., to ignore any CSS sheet that the document author may have specified on the page).
Timo also told me that there is a very cool extension called "Chrome Page Style" (ID: ddpkgkegfklikkmfmneldonhldahhacb) that allows you to alternate between stylesheets provided by the page (see Joey Hess's ikiwiki's CSS Market for an example of that). It also has the possibility of disabling all the CSS'es that a page may have, which, in many cases around the web actually improves the readability of the pages, for many "fancy" things detract from readability, especially if you have poor eyesight. Although I had said that I didn't want to resort to extensions, this was a simple solution enough that made me consider revisit my requirements. I do, though, wonder why some frequently used extensions are not built into the programs from stock.
  • In Iceweasel, I can press Ctrl+Q and make the damn browser quit, instead of me having to use the mouse/trackpad/pointing device. This is probably a lack of my understanding on how to associate key bindings to Chromium, but the problem still stands.
Of course, Timo also set me straight and told me about Ctrl+Shift+Q for that purpose, which I had missed completely.
  • Similarly, when there is only one tab open, pressing Ctrl+W in Chromium does not close the tab (say, presenting me a blank tab or the list of most visited sites), nor it closes the application.
As a poor writing of my part, Ctrl+W does close the tabs. But when there is only one tab present, it doesn't close the window. I'm OK with that, since I now know the way to close the window. Thanks again Timo. Oh, and one point that may not be known by many is that using about:flags in Chrome does something analogue to Firefox's about:config (even with the warning). And tweaking with it you can try to see if you can enable hardware acceleration of some parts of your pages, as present in newer releases of Chromium. So, there I am: way more satisfied than with earlier with Chromium, but still having Firefox/Iceweasel as my primary browser. Good to have two usable browsers now.

8 March 2011

Evgeni Golov: Desktop in a shell: irssi

Continuing my series about my desktop in a shell, in this chapter I will present you my irssi setup.Irssi is the most customized part of my setup, using many scripts from the irssi-scripts Debian package and some from the internet , handling 6 irc networks with a total of 34 channels plus JabberXMPP, ICQ, Twitter and identi.ca. My custom irssi settingsI like my conversations to look like Buddy, you rock instead of Buddy: you rock , so:
/set completion_char ,I also like *long highlighted text*, so:
/set emphasis_multiword onHide mIRC and ANSI colors when turned on. This can be used to eliminate angry fruit salad syndrome in some channels. (Quote from irssi.org/documentation/settings, do I have to say more?):
/set hide_colors onWhen talking in more than one channel, it sometimes comes handy to repeat something from the past said in THAT channel, not searching though all:
/set window_history onTell irssi to shutup when away, noone will hear/see the beep anyways:
/set beep_when_away offI want my screen to inform me about new messages, so I need this:
/set bell_beeps onBeep for query text, notices, dcc, dcc text and hilight:
/set beep_msg_level MSGS NOTICES DCC DCCMSGS HILIGHMy used irssi scriptsactive_notice.pl (from irssi-scripts)Show notices in the active window or the corresponding query instead of the status-window.
No further configuration needed.adv_windowlist.pl (from anti.teamidiot.de)Some really smart windowlist, see my previous post about adv_windowlist.pl, I won t explain its features and settings here again.autorealname.pl (from irssi-scripts)Show peoples real-name when they join (my $nick $name memory is poor :()
No further configuration needed.chansort.pl (from irssi-scripts)Automatically sort my channels alphabetically, grouped by network.
Enable via /set chansort_autosort on<Update>kaimi asked me yesterday, whether chansort could ignore the network and just sort ALL channels in alphabetical order. Well, it could not, until yesterday ;) Fetch either the patch against chansort.pl or my chansort-ng.pl and set chansort_ignore_network to ON. You can also set chansort_ignore_windowtype to mix channels and queries if you like to.</Update>hack-whois-in-current-window.pl (from dgl.cx)Display /whois in the current window or the corresponding query instead of the status-window.
No further configuration needed.keepnick.pl (from irssi-scripts)Restore my nick after it is free again (like after a ping timepout).
Set the nick via /keepnick [-net chatnet] [nick]nickserv.pl (from irssi-scripts)Automatically authorize me on networks with NickServ.
Enable by settings services address, nick and password via:
/nickserv addnet freenode NickServ@services.
/nickserv addnick freenode Zhenech mypass
screen_away.pl (from irssi-scripts)Set away when screen is detached.
Will be enabled automatically, but you maybe want to tweak screen_away_message :)seen.pl (from irssi-scripts)Add a /seen nick command to lookup when and where nick was online the last time.
No further configuration needed.trackbar.pl (from irssi-scripts)Draw a line when you switch irssi windows so you can see which part of the conversation you have already seen.
No further configuration needed.BitlBeeI m using BitlBee as my irssi-to-every-im gateway of choice ;) But while it sounds easy, my setup isn t. I am using BitlBee only to connect to jabber.die-welt.net, my Jabber/XMPP server. There I have a pyICQt instance running, to connect to ICQ via Jabber (yes, BitlBee would support that directly, but that would fail on me when I would like to connect from a different Jabber-client). Twitter and identi.ca are handled via Jabber too: identi.ca does support this by themself, for Twitter I use the tweet.im service.
But besides of my strange setup, BitlBee is easy: apt-get install bitlbee, connect irssi to localhost:6667, you will be force-joined to &bitlbee, where you do:
register somepassword
account add Jabber you@jabberserber.example.com jabberpassword
account on 0
and done, you@jabberserver.example.com is online :)The all awaited screenshot (from my last irssi-post):

And the whole ~/.irssi/config: http://files.die-welt.net/irssi-config.txt, read it carefully while waiting for the next post about newsbeuter :)

7 March 2011

Craig Small: JFFNMS at RC2, ncurses at 5.8

After some reports back about jffnms 0.9.0rc1 I have now updated it to rc2. Thanks for all who gave me information about how it worked in YOUR setup. I cannot be sure but I d say the second RC will be the last until the release itself. Sven has also given me the nod and ncurses 5.8 migrated into unstable. We ve had one report that the new version of ncurses might not play well with stfl (see #616711 ) but generally speaking it should work ok. Finally, congratulations to the debian project on winning two categories at the Linux New Media Awards. It was especially good to hear the presentation by Karsten Gerloff who is president of the Free Software Foundation Europe. ncurses bug update It seems that the ncurses bug is more serious and is to do with newwin() function in the library. If you get crashes when a program starts and its linked to ncurses 5.8 (even if it is not a Debian system) you may have this problem. It doesn t happen to all ncurses programs, as the stfl example code and mutt work ok. Y9VW3CNYRFF6

6 March 2011

Eugene V. Lyubimkin: cupt: 2.0.0~beta1

Cupt v2 has reached a beta stage.

Main news of Cupt v2 comparing to Cupt v1 are grouped here.

Notable changes since 2.0.0~alpha3:



Full changelog is available in debian/changelog, as usual.

binary packages for i386, source package and README

Thorsten Glaser: mksh gains recursive parser for command substitutions $( )

mksh-current has just gained an experimental recursive parser for command substitutions, fixing RedHat BZ#496791 and decades-old complaints about the pdksh codebase, compared to AT&T ksh93. (GNU bash could also do the example, but not some other things mksh(1) parses fine now.)

This means that things like the following work now. 

	# POSIX, should  always  work
	echo $(case 1 in (1) echo yes;; (2) echo no;; esac)
	# POSIX optional, works now in mksh, works in GNU bash
	echo $(case 1 in 1) echo yes;; 2) echo no;; esac)
	# GNU bash seems to choke on comments ending with backslash
	# a comment with " ' \
	x=$(
	echo yes
	# a comment with " ' \
	)
	# No non-recursive COMSUB parser can pass all of the above
	# tests and these below at the same time (some extensions)
	echo $(typeset -i10 x=16#20; echo $x)
	echo $(typeset -Uui16 x=16#$(id -u)
	) .
	echo $(c=1; d=1
	typeset -Uui16 a=36#foo; c=2
	typeset -Uui16 b=36 #foo; d=2
	echo $a $b $c $d)
	# the  #  is especially tricky, that s why the above cases

Next on my TODO is the complete rewrite of the read built-in command, as well as its documentation. I think that the (reduced) goals for mksh R40 will have been met by then, except porting to LynxOS and MPE, but we re working on it, and re-testing Syllable and Plan 9). Of course, a release implies testing on a lot of the supposedly supported platforms, so it won t be out immediately . Though, associative arrays have been removed from the R40 goals, so that I can at least get a new release out. Note that Debian and OpenSuSE Buildservice users have been provided with somewhat well-tested mksh-current snapshots for a while already, and Gentoo users can use the live ebuild ; there s always compiling from source too

5 March 2011

Mike Hommey: More graphs for the Debian Bug Tracking System

I have been maintaining Debian Bug Tracking System graphs for a few years, now, though not very actively. They initially were available on people.debian.org/~glandium/bts/, but there have been some recent changes. A while ago, I started experimenting with brand new graphs on merkel.debian.org/~glandium/bts/, and when merkel was announced to be dying a few months ago, I got in touch with the QA team to know what to do with them, and it was decided we d put them on qa.debian.org. I unfortunately didn t follow up much on this and only recently actually worked on the migration, which took place 2 weeks ago. The result is that the graphs have officially moved to qa.debian.org/data/bts/graphs/, and links on the Package Tracking System have been updated accordingly. There is now also an additional graph tracking all open bugs in the BTS, across all packages: Today, I added a new feature, allowing to consolidate data for multiple arbitrary packages in a single graph. Such graphs can be generated with the following URL scheme (please don t over-abuse of it):
http://qa.debian.org/data/bts/graphs/multi/name0,name1,name2,etc.png
As an example, here is a graph for all the bugs on the packages I (co-)maintain:
http://qa.debian.org/data/bts/graphs/multi/dehydra,diggler,iceape,iceweasel,libxml2,libxslt,livehttpheaders,mozilla-dom-inspector,nspr,nss,pyxpcom,venkman,vmfs-tools,webkit,xulrunner,zfs-fuse.png
And here are the bugs affecting Mozilla-related packages:
http://qa.debian.org/data/bts/graphs/multi/iceape,icedove,iceowl,iceweasel,nspr,nss,xulrunner.png
I guess the next step is to allow per-maintainer consolidation through URLs such as
http://qa.debian.org/data/bts/graphs/by-maint/address.png
Update: per-maintainer consolidation has been added. (Hidden message here: please help triaging these bugs)

4 March 2011

Steve Kemp: A final update on redisfs

I think I'm done with the redis filesystem for the moment. It does everything I need it to do, although I am curious to see how much faster it could go if it were to use non-blocking writes that isn't a major concern. There is only one missing feature I'm planning to play with - and that is the ability to implement snapshots. As a refresher redis is a key+value store, which mostly uses system memory. I've built a simple FUSE filesystem on top of that (now with symlink support!) and so all the file contents, meta-data, and similar is stored in memory. Implementing snapshots could be done by two different routes:
Copying all keys and their values, under a new name.
For example right now, by default, all the filesystem entries in the root directory are stored beneath the key "SKX:/" - where "SKX" is the key prefix. Assume I copy each existing key, and the associated value(s), giving them a new prefix such as "SKX2:" I can mount the filesystem against that prefix - and we've got a point-in-time snapshot.
Serialising all keys & values
Redis has a primitive which allows you to determine the names of keys at runtime. Given that all my filesystem keys have a prefix ("SKX:" by default) it wouldn't be difficult to find them, and serialise them. This would require more effort to re-import and re-mount, but it should be portable across hosts.
Anyway assuming I get this right we'll have a filesystem which is replication-friendly and snapshot-able. A fun combination. ObQuote: "We had been everywhere. We had really seen nothing. " - Lolita

Cyril Brulebois: Debian XSF News #7

Time for a seventh Debian XSF News issue!

Next.

Previous.