Search Results: "ron"

23 May 2011

David Welton: Summary: Built To Sell - Creating A Business That Can Thrive Without You

<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="http://rcm.amazon.com/e/cm?lt1=_blank&amp;bc1=000000&amp;IS2=1&amp;bg1=FFFFFF&amp;fc1=000000&amp;lc1=0000FF&amp;t=dedasys-20&amp;o=1&amp;p=8&amp;l=as4&amp;m=amazon&amp;f=ifr&amp;ref=ss_til&amp;asins=B004IYISQW" style="width: 120px; height: 240px; float: left; margin-right: 20px; margin-bottom: 8px;"></iframe> Many small business owners start a business with the idea of greater freedom in mind, yet end up chaining themselves to something that takes even more dedication than a regular job. This book discusses the dos and don'ts that business owners need to be conscious of in order to create a business that is independent of its owner(s) in order to be able to sell it. It does so in an easy to read format that's a bit of a gimmick - a fictional story illustrating the authors points. Like most business books, the core idea isn't that hard to explain. the book even includes a handy summary of the main points, which I'll paraphrase here:
  1. Focus on one thing you can do really well; even a service business can be 'productized' by creating a standard service that is the focus of the business.
  2. If your business revolves around one or two key clients, the risk inherent in that approach will lower the value or scare of potential buyers completely.
  3. Put a process in place, from sales through production.
  4. Don't "be" your company. If the company is all about you, what's a potential buyer really getting if you sell it and leave?
  5. By focusing on products, you can charge up front rather than having poor cash flow. If you pay people to work on a project for 3 months, and don't get paid for another month, you are, by the end, out 4 months salary while waiting for the payment.
  6. Say no to projects outside the scope of your business. Only by focusing narrowly can you really excel at what you do.
  7. Spend time doing some research and calculations to estimate your potential market size; buyers will want to know this.
  8. If you have a business that has sales people, hire at least two so that they'll compete with one another.
  9. You want people who are good at selling products, not services; the latter will want to tweak your offering for each and every client, rather than selling it as-is and trying to find how it can meet the customer's needs that way, which is the best strategy for a product.
  10. If you were previously running a more 'generic' services company, and you switch to a more productized approach, be prepared to take a hit the year when you switch.
  11. Potential acquirers will want to see at least a couple of years of steady growth with the new model after making the switch.
  12. If you grow, you'll need a management team that can work without you. Put an incentive system into place to reward their loyalty and results.
  13. When looking for an adviser to sell your company, aim for one where you will not be the largest or smallest client.
  14. If your adviser really only has one company in mind to sell yours to, they may be trying to sell you off cheap as a favor.
  15. Consider how much you could grow with the resources of a buyer. Think big, and show them what kind of growth could happen with the right backing.
  16. Think and speak like a product business with 'customers' rather than 'clients'.
  17. Stock options are more complex than simpler options like bonuses that are paid out over a period of time, in terms of ways to give people an incentive to stay.
All in all, it was an enjoyable read, with solid points. Even if you have no intention of selling your business, thinking of the business itself as a sort of "product" that is not dependent on you to work correctly is a sensible way to go about creating and growing a business. He's certainly not, nor claims to be the first one to discuss this idea, and indeed references the well known E-Myth Revisited which focuses less on selling a business, and more on how to go about extricating you and your skills from the business.

Christian Perrier: 2011 week 20 Debian work

This week, I continued triaging bugs in the samba source package. The result is interesting Many bugs are tagged "moreinfo", most of them being old bugs, many likely to be user error....or something fixed for quite a while. So, I expect the bug count to decrease again. Moreover, I did a call for translations 10 days ago, so about 10 bugs are indeed transaltion updates which will be closed in next upload. Samba upstream also released their 3.6.0rc1 version this week, and 53 hours after the annoucement, it landed in Debian experimental. I pushed tasksel 3.02 in git. Indeed, I upload it quite a while ago and forgot to push the release and tags. I'm not entirely used to git, as it seems..:-) On l10n front, I launched a Smith review for ntfs-3g and launched a call for translations matlab-support (and got thanks from maintainer). Finally, after last week push, the French team completed to translation of package descriptions for the top500 packages in popcon. As a consequence, I moved the target further and we nos have "Popcon600" on the stats page. French now reached rank 3 in the DDTP.

John Goerzen: Baptism

Today I was baptized and joined the Mennonite Church. I imagine this might surprise some of you for various reasons, so let me provide a bit of background. I ve had a lot of people, both in person and in comments responding to my blog post, express surprise with statements I have made. The view of Christianity that many people have is of a group that devalues scientific inquiry and places a lot of emphasis on things like opposition to gay marriage, evolution, and abortion, and enjoys political leaders that say bring it on. While I know some Christians that fall into that mold, there are quite a few that don t as well. Some churches, such as the one we attend, have a surprisingly diverse set of people and yet still function and get along well. Today they accepted me with joy. Nobody was concerned that I started my statement with a reference to ancient Greek philosophy, wound up suggesting that the church ought to make sure to make illegal immigrants, gays and lesbians, and prostitutes feel welcome, and embraced both religion and scientific inquiry, feeling them complimentary. Mennonites practice adult baptism rather than infant baptism. A traditional age for people to be baptized is during high school, though emphasis is placed more on the individual than their age, so it s not unheard-of for someone to do so a bit later in life as I have. Part of the baptism involves the candidate sharing their faith story. These are typically intensely personal, widely varied, heartfelt, and honest. Some people s stories involve struggles with depression, physical disabilities, or the place of religion in their lives, while others reflect little struggle at all. Mine involves letting go of a lot of things, and also seeing some things, such as serious intellectual inquiry or existential questions about God (such as is there a god? ), as a positive rather than a negative feature of a Christian life. But it also involves a recognition and deep respect for those that don t approach things in this way. Though given publicly in a church, baptism testimonies are rarely published or shared more broadly. But I m going to share mine here. I have edited it only very lightly to remove a few local references that wouldn t make sense out of the context of this community. This was delivered in front of a rather different audience than is likely to be reading this post, so if you have questions, do feel free to ask in the comments. But before I begin, I just have to pass along a joke a friend told me: John, you picked quite a day for your baptism the day after the rapture. Baptism Testimony John Goerzen May 22, 2011 Those of you that know me well will probably not be surprised that I will begin my Christian baptism testimony with a story about an ancient Greek philosopher, and also touch on the philosophical nature of truth. These are key parts of my story. As legend goes, Socrates famously said, I know that I know nothing in other words, he believed that nothing could be known with absolute certainty. The Greek Oracle thought to be infallibly wise said that Socrates, the man that thought he knew nothing, was the wisest man in all Athens. An interesting paradox, and one that sheds light on my own religious story. My story involves coming to grips with the understanding that I know very little, that no person can ultimately know much about God, and finding a way to make peace with that situation. Growing up in this community, I thought I knew some things about faith and Christianity. At a certain age, it all seemed so simple. We took the Bible to be literally true. We marveled in Sunday School at how the apostles could sometimes appear so blind. And, modeled both implicitly and explicitly, was this notion: the stronger our faith, the fewer inner questions or doubt we have about the nature of God, the literal accuracy of the entire Bible, or our relationship to God. Those I perceived held up as examples never seemed to question any of these things, and showed outwardly, at least complete certainty about them. Moreover, holding certain intellectual beliefs was key to Christian identity, and even more importantly, to eternal life. This model has quite obviously worked well for many people for many years. The good that has happened, and continues to happen, from people that have that kind of certainty is manifest all around us. And yet, it didn t seem to work out for me. I m not the kind of person that accepts a lot of things at face value. It is helpful to be able to examine and challenge ideas and even more helpful to have other people challenge my ideas. From well before I was in high school, I was questioning some things about the Bible, God, and religion in general. My thoughts ranged from the impact of evolution on religion to the apparently vengeful God of the Old Testament to the very existence of God. In the Christian context, I perceived having these questions as a personal failing, something that I ought to repress. The more I tried to repress them, the more troublesome they became. Why, for instance, should a loving merciful God decide whether to let us into heaven based on whether we hold certain intellectual beliefs? In high school, I participated in the catechism class here at this church, and was frustrated because it didn t tackle deeper meanings or the kinds of questions I had. I wasn t yet able to articulate all my thoughts and questions very well, and I probably had an overactive case of teenage cynicism. As a result, I didn t get baptized like most others my age did. As I learned more about the early history of Judaism and Christianity, I only found more reasons to question the model of faith I thought I had received the one in which Biblical literalism and a divine guarantee of sorts was key. More recently, I gradually became aware that the model of Christianity I had in mind was one of many views. Christians, Mennonites, and even this church are incredibly diverse groups, and in retrospect, I am surprised that it took me so long to realize this. Three major steps led me to baptism. The first step was the realization that, whatever our understanding of the literal accuracy of the Bible, literal truth is often inferior to metaphorical truth. As an example, many of us have read the works of Shakespeare. They are fictional, but the reason they have been revered for so many centuries is that they are true. They teach us things about ourselves and our world in a way that no history book can. By placing such an emphasis on literal truth in the Bible, I was missing out on the message right there for me. By concerning myself with creationism or evolution, I missed out on thinking about what the story in Genesis meant for the Jews, and what it could mean for me. Evolution ceased to be a threat to religion; it became simply a tool for learning about a different sort of truth than we get from religion. N. T. Wright mentions an incident that illustrates this point. A woman attending a religious conference asked the speaker, a prominent theologian, Is it true that the serpent in Genesis actually spoke like you and I speak? The theologian answered, It doesn t matter whether the serpent could speak. What matters is what the serpent said. The second step I took towards being baptized was realizing where the real core of Christianity lies. It s not some debate about Genesis, but rather the death to one s old self, and the rebirth and continual remaking of oneself in the example of Jesus. Given that, a lot of questions seem unimportant or even irrelevant. Good can be expressed in many ways, and if one person achieves a remarkably Christian life via a literal understanding of the Bible, and another via a more metaphorical reading, then it is my place only to affirm both and say that they both got it right. If we say that the task of remaking ourselves is like climbing a mountain, then what matters is not how we are climbing the mountain, but simply that we are climbing it. I used to equate faith with an intellectual belief. I have come to see that was a narrow view. Faith, to me now, is more about vision: do we see the world like Christ did? Where do our loyalties and our trust lie in God or in human institutions? What are our goals in life? We humans have failed to understand God, and probably always will. I too share in that incomplete understanding, but I have come to accept that it is OK. I know enough to know that I want my loyalty to lie in Jesus, to know what kind of vision of the world I want to have, and I have learned to accept that intellectual questions can even be a form of meditation, enlightenment, and prayer. The third step toward baptism was moving past my own shortcomings. For a long time, I thought I didn t believe the right things, didn t believe them strong enough, wasn t certain enough about God, didn t pray enough, didn t read enough, didn t understand enough, didn t love enough and ultimately, that I wasn t good enough. I request baptism today understanding that, despite the various imperfections I still have, and we all have, everyone is good enough and deserving of love and peace. God s love is for everyone. No exceptions! Following Jesus boils down to this: I too must be an agent of love and peace to everyone, without exception. My vision must be centered around the fact that we are to emulate the God that loves the entire world, sinners all, and therefore I should as well. I hope I can show others the kind of love that has been shown me. I hope, too, to share with this congregation and the global church in the vision of love. I hope that we can continually strive to re-focus on Christ s vision. As an example, we all know that many different viewpoints about whether homosexuality is sinful exist in Mennonite Church USA. These opinions are deeply held and personal for many, and have been discussed over and over and over. But ultimately, they aren t terribly relevant to the church s mission. The example of Jesus unites us all: he embraced everyone. He accepted criminals and prostitutes and showed them love and kindness. Our difficult task, which is also my difficult task, is to show this same love to absolutely everyone, regardless of our varying opinions about them and their conduct. The forgotten and repressed of our day perhaps criminals, homeless, Muslims, addicts, gays and lesbians, illegal immigrants, ethnic minorities, and still prostitutes deserve the love of Christ s church and all its members. My hope is that any one of the earth s 7 billion people could step through the doors of our church, or any church, and immediately feel Christ s love, and the unconditional, non-judgmental, welcome and love of Christ s followers, no matter what. This prayer from Ephesians 3 sums up my hope and my vision, for myself and everyone:
I pray that out of his glorious riches he may strengthen you with power through his Spirit in your inner being. I pray that Christ may dwell in your hearts through faith. And I pray that you, being rooted and established in love, may have power, together with all the Lord s people, to grasp how wide and long and high and deep is the love of Christ. And I pray that you will know this love that surpasses knowledge that you may be filled to the measure of all the fullness of God. (3.16-19)

22 May 2011

Christian Perrier: 2011 week 20 Debian work

This week, I continued triaging bugs in the samba source package. The result is interesting Many bugs are tagged "moreinfo", most of them being old bugs, many likely to be user error....or something fixed for quite a while. So, I expect the bug count to decrease again. Moreover, I did a call for translations 10 days ago, so about 10 bugs are indeed transaltion updates which will be closed in next upload. Samba upstream also released their 3.6.0rc1 version this week, and 53 hours after the annoucement, it landed in Debian experimental. I pushed tasksel 3.02 in git. Indeed, I upload it quite a while ago and forgot to push the release and tags. I'm not entirely used to git, as it seems..:-) On l10n front, I launched a Smith review for ntfs-3g and launched a call for translations matlab-support (and got thanks from maintainer). Finally, after last week push, the French team completed to translation of package descriptions for the top500 packages in popcon. As a consequence, I moved the target further and we nos have "Popcon600" on the stats page. French now reached rank 3 in the DDTP.

Thomas Girard: Introducing CORBA

What is CORBA? The wikipedia page says it all:
The Common Object Request Broker Architecture (CORBA) is a standard
defined by the Object Management Group (OMG) that enables software
components written in mutiple computer languages and running on
multiple computers to work together (i.e., it supports mutiple
platforms).
The standard was created twenty years ago, in 1991. The technology can be used in many programming languages: Python, Java, C++, C, etc. Hence you can write a software in your favorite programming language, exposing services that others will be able to call using a different language. (Nowadays you can do the same with WebServices. This article is about CORBA.) All CORBA specifications are available on [OMGSPEC]; latest version is [CORBA3.1]. [MCHALE] book is a very good way to learn CORBA.
How does it work? CORBA is language agnostic because it relies on its own language: the Interface Definition Language (IDL for short). The IDL defines the contract between the client (software calling the service) and the server (software implementing the service). The service is exposed through a remote object. IDL is used to define:
  • modules, which are called namespaces in C++ (or packages in Java)
  • structs, which are like C structs: they can contain data only (the data part of the contract)
  • interfaces, which contain methods specification (the service part of the contract)
  • exceptions that can be raised by a method
  • typedefs which are type aliases
  • unions, la C
  • enums, enumerations
  • constants
IDL comes with existing types:
  • char, octet
  • string, wstring
  • short, long, long long (signed if unspecified, unsigned if prefixed with unsigned)
  • float, double, long double
  • arrays
  • sequences
  • any which can be any existing or user-defined type
From this contract, the following code will be generated using a dedicated tool:
  • client-side code, called stubs
  • server-side code, called skeletons
So to call a CORBA remote service in a given programming language, we will generate stubs from the IDL, then write code relying on these stubs. To provide a CORBA service in a given progamming language, we will generate skeletons from the IDL, then extend them with our implementation. The dispatching and networking machinery is handled by the Object Request Broker (ORB for short). That's the most important part of the CORBA system: it's the component that hides away remoting aspects to make distant objects appear as if they were embedded within the local, client-side code.
Telecom Log Service example The Telecom Log Service [TLOG] is an OMG specified service that defines two interfaces:
  • Log, to add, search and delete records
  • LogMgr, a Log container to lookup a given log, or list all contained logs
The specification details five sub-interfaces of the log service, grouped into three different approaches:
  • basic implementation. This is the service depicted below.
  • event-based implementation. The Event Service [EVENT] is another CORBA service specification. Combining both services means that software emitting events can be plugged to an Event Log without modification.
  • notify-based implementation. The Notification Service [NOTIF] is another CORBA service specification, built on top of the Event Service, with added filtering facility. Again, combining these two services provide easier integration with existing software.
Here is the UML representation of key base interfaces (in yellow), basic implementation (in blue) and important date structures (in green): http://thomas.g.girard.free.fr/CORBA/tls.png Here is the simplified IDL for these parts:
#include <TimeBase.idl>  // IDL can include each other
#pragma prefix "omg.org" // extra prefix to use in front of type names

// DsLogAdmin definition
// Because of the #pragma prefix above, the complete name of this module
// is omg.org/DsLogAdmin
module DsLogAdmin
 
        // E X C E P T I O N --------------------------------------------------
        exception InvalidGrammar  ;                    // empty exception
        exception InvalidConstraint  ;                 // empty exception
        exception LogFull   short n_records_written;  ; // one attribute
        exception LogOffDuty  ;                        // empty exception
        exception LogLocked  ;                         // empty exception
        exception LogDisabled  ;                       // empty exception

        // T Y P E   A L I A S ------------------------------------------------
        typedef unsigned long LogId;
        typedef sequence<LogId> LogIdList; // LogIdList is a sequence of LogId
        typedef unsigned long long RecordId;
        typedef string Constraint;
        typedef TimeBase::TimeT TimeT; // TimeT type defined in TimeBase module
        typedef sequence<any> Anys;    // Anys is the name of a sequence of any

        // S T R U C T U R E S ------------------------------------------------
        struct NVPair  
                string name;
                any    value;
         ; // <-- Mandatory semi-colon
        typedef sequence<NVPair> NVList; // Give a name to a sequence of NVPair

        struct LogRecord  
                RecordId id;
                TimeT    time;
                NVList   attr_list;
                any      info;
         ;
        typedef sequence<LogRecord> RecordList;
        // I N T E R F A C E S ------------------------------------------------
        // Iterator interface specification
        interface Iterator  
                // get() method:
                // - two input parameters
                // - returning a list of LogRecord
                // - possibly raising InvalidParam exception
                RecordList get(in  unsigned long position,
                               in  unsigned long how_many)
                        raises (InvalidParam);
                void destroy();
         ;  // <-- Mandatory semi-colon

        // Forward declaration
        interface LogMgr;
        interface Log
         
                LogMgr my_factory();
                LogId id();
                // retrieve() method:
                // - two input parameters
                // - one output parameter
                // - returning a list of LogRecord
                // This method will actually return two objects: the
                // RecordList and the Iterator
                RecordList retrieve(in  TimeT from_time,
                                    in  long how_many,
                                    out Iterator i);
                RecordList query(in  string grammar,
                                 in  Constraint c,
                                 out Iterator i)
                        raises (InvalidGrammar, InvalidConstraint);
                unsigned long delete_records(in  string grammar,
                                             in  Constraint c)
                        raises (InvalidGrammar, InvalidConstraint);
                void write_records(in  Anys records) 
                        raises (LogFull, LogOffDuty, LogLocked, LogDisabled);
         ;
        typedef sequence<Log> LogList; // Give name LogList to sequence of Log

        // inteface BasicLog derives from interface Log
        interface BasicLog : Log  
                void destroy();
         ;
        interface LogMgr
         
                LogList list_logs();
                Log find_log(in LogId id);
                LogIdList list_logs_by_id();
         ;
        // inteface BasicLogFactory derives from interface LogMgr
        interface BasicLogFactory : LogMgr
         
                BasicLog create(
                        in  LogFullActionType full_action,
                        in  unsigned long long max_size,
                        out LogId id)
                        raises (InvalidLogFullAction);
                BasicLog create_with_id(
                        in  LogId id,
                        in  LogFullActionType full_action,
                        in  unsigned long long max_size)
                        raises (LogIdAlreadyExists, InvalidLogFullAction);
         ;
 ; // <-- Mandatory semi-colon
The complete IDL file for the Basic Telecom Log Service is available here.
[CORBA3.1] http://www.omg.org/spec/CORBA/3.1/
[MCHALE] http://www.ciaranmchale.com/corba-explained-simply/
[OMGSPEC] http://www.omg.org/technology/documents/spec_catalog.htm#Middleware
[TLOG] http://www.omg.org/spec/TLOG/
[EVENT] http://www.omg.org/spec/EVNT/
[NOTIF] http://www.omg.org/spec/NOT/

Russell Coker: Preventing the End of the World

Some crazy people led by Harold Camping who call themselves Christian claimed that the world was going to end yesterday [1]. Harold had previously done this in 1994 so he s got form for predictions that don t come true. The bible says that the day of the Lord so cometh as a thief in the night [2], so obviously Jesus can t return on a day when billboards all around the world are predicting him! Maybe rather than being wrong, Harold has in fact prevented the end of the world by predicting it correctly! Now if a church like Harold s can make predictions separated by slightly less than 17 years (let s round it off to 6000 days) then we only need 6000 such churches scheduled to predict different days to cause the rapture to be indefinitely deferred. If the churches were to predict which week the world would end then we would only need something less than 900 such churches or maybe 1800 so we have redundancy. It seems that Harold Camping makes his money by scaring gullible people into selling everything they own and giving the money to him [3]. If we are to achieve the goal of having someone forecast the end of the world every day then it would be good if such people were forced to raise money more often than every 17 years. A good solution to this might be a tax on end of days religious investments, money that was given to a religious corporation or sole trader following a scare about the end of the world could be subject to a 100% tax rate if not spent within 1 year. Then Harold would be forced to make a prediction for every year and thus give us greater protection from the end of the world. </Satire> But seriously we need to tax religious organisations in the same way as other companies. A religious organisation that gives money to help the needy can use that as a tax deduction, and operating expenses such as paying salaries of staff are also tax deductable. So the serious religious groups would have a tax rate that is fairly close to zero anyway. When a religious leader wants to have 27 Rolls Royces they should be forced to pay tax on all that money, and additionally they should be compelled to publish a statement of company accounts for everyone who gave money just as corporations are compelled to provide information to stock-holders. Currently various religious organisations are denouncing Harold and his followers to try and save Christianity from ridicule (here s one example [4]). It seems to me that if they want to avoid ridicule they should advocate that religious organisations be forced to open their books and pay tax on their profits so that people like Harold can t use religion as a money-making scam.

21 May 2011

Vincent Bernat: Network lab with User Mode Linux

To setup a virtual network lab, you get a lot of alternatives. You can for example use GNS3, Netkit, Marionnet or VNUML. Some of those tools, like GNS3, allow to add some closed source equipments like a Cisco 7200 or a Juniper router. All those tools are a great way to setup your network lab. Look at them! If you want to setup a virtual network lab for educational purpose, one of those should fit your purpose. However, none of those solution were a perfect match for me. I did not want to maintain some root filesystem. I wanted my lab to start in a few seconds. I wanted to keep all configuration files (including the ones for the virtual hosts) into one subdirectory of my home and be able to modify them while the lab was running. I also wanted to be able to plug some Cisco router using Dynamips/Dynagen. None of the listed solution above matched all those criteria. Therefore, I setup my own lab script with User Mode Linux. This is not a complete solution, but is more like a home-made solution to match one particular need. You cannot use the final result without tweaking it. Again, look at the other solutions first. We will go step by step to understand how the lab was built. If you are in a hurry, you can look at the result which is published on GitHub.

User Mode Linux User Mode Linux (or UML) is a Linux kernel running as a process instead of running on top of some processor. With Debian, you can install it using apt-get install user-mode-linux. You get a linux command that will run your Linux kernel as a simple process. No need to be root.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
$ linux
Core dump limits :
    soft - 0
    hard - NONE
Checking that ptrace can change system call numbers...OK
Checking syscall emulation patch for ptrace...OK
Checking advanced syscall emulation patch for ptrace...OK
Checking for tmpfs mount on /dev/shm...nothing mounted on /dev/shm
Checking PROT_EXEC mmap in /tmp/user/500/...OK
Checking for the skas3 patch in the host:
  - /proc/mm...not found: No such file or directory
  - PTRACE_FAULTINFO...not found
  - PTRACE_LDT...not found
UML running in SKAS0 mode
Adding 5632000 bytes to physical memory to account for exec-shield gap
Initializing cgroup subsys cpuset
Linux version 2.6.32 (2.6.32) (root@tito) (gcc version 4.4.5 (Debian 4.4.5-10) ) #2 Thu Jan 27 12:49:46 UTC 2011
[...]
console [mc-1] enabled
Couldn't stat "root_fs" : err = 2
Failed to initialize ubd device 0 :Couldn't determine size of device's file
registered taskstats version 1
VFS: Cannot open root device "98:0" or unknown-block(98,0)
Please append a correct "root=" boot option; here are the available partitions:
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(98,0)
You get a Linux kernel that will just panic because it does not find a disk to mount to run init from it. Usually, you build some root file system and give it as an argument to your UML kernel.

Basic setup However, you can also use your host filesystem as a root filesystem. We also specify /bin/sh as init instead of /sbin/init. This will be the first process to be started by our UML kernel.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
$ linux init=/bin/sh rootfstype=hostfs
[...]
Linux version 2.6.32 (2.6.32) (root@tito) (gcc version 4.4.5 (Debian 4.4.5-10) ) #2 Thu Jan 27 12:49:46 UTC 2011
[...]
VFS: Mounted root (hostfs filesystem) readonly on device 0:12.
IRQ 3/console-write: IRQF_DISABLED is not guaranteed on shared IRQs
IRQ 2/console: IRQF_DISABLED is not guaranteed on shared IRQs
IRQ 10/winch: IRQF_DISABLED is not guaranteed on shared IRQs
/bin/sh: can't access tty; job control turned off
# uname -a
Linux (none) 2.6.32 #2 Thu Jan 27 12:49:46 UTC 2011 x86_64 GNU/Linux
# echo $$
1
You may want to setup a bit your environment:
1
2
3
4
5
6
7
8
9
# hostname -b R1
# export TERM=xterm
# export PATH=/usr/local/bin:/usr/bin:/bin:/sbin:/usr/local/sbin:/usr/sbin
# mount -t proc proc /proc
# mount -t sysfs sysfs /sys
# mount -t tmpfs tmpfs /var/run -o rw,nosuid,nodev
# mount -t tmpfs tmpfs /var/log -o rw,nosuid,nodev
# mount -o bind /usr/lib/uml/modules /lib/modules
# mount -t hostfs hostfs /home/bernat/mylab -o /home/bernat/mylab
We setup /proc and /sys correctly. We also turn /var/run and /var/log as tmpfs filesystems. This will allow most daemons to run correctly: they will be able to write runtime data in /var/run and to write logs in /var/log. You can even launch your favorite syslog daemon. The line about /lib/modules is to ensure that our UML kernel is able to access its kernel modules. For example, we can now use modprobe tun. The line about /home allows to mount the home directory of your host inside UML. This was already the case, but it was mounted as read-only. One important thing to notice is that while you seem to be root inside UML, you did not run linux as root. Therefore, when the UML kernel tries to write a file it will still be bound by the permissions granted by your host kernel. See this example:
1
2
3
4
# mount -o remount,rw /
# touch /tmp/test1
# touch /etc/test1
touch: cannot touch  /etc/test1': Permission denied
This is a great way to ensure that your lab will not destroy your host. Do not run linux as root. What is interesting with such a setup is that you share the filesystem with your host. You can modify some configuration file on your host and your UML will see the modification right away. You can also modify a file inside UML and see the modification on your host too.

Console & job control There is a slight problem with this environment. You may be used to it if you already started your own GNU/Linux system with init=/bin/sh. The problem is that you don t have job control. Job control allows you to put tasks in background and interrupt them while they are running. It is job control that allows you to stop a program with ^C. Here, you don t have job control. See:
1
2
# cat
^C^C^C^C
You are stuck. The only way to go out is to kill the linux process on your host. Job control is a great mystery for me. I don t know how to enable it by hand. However, getty is able to enable job control for you. Just type this command:
1
# exec getty -n -l /bin/sh 38400 /dev/tty0
You get job control. We keep /bin/sh, but you may prefer /bin/bash since on most systems, /bin/sh is rather basic.

Writable root filesystem Let s get back to our root filesystem. Most daemons can run just fine with our setup. For example, we can start Nginx, a popular web server:
1
2
3
# mkdir /var/log/nginx
# /etc/init.d/nginx start
Starting nginx: nginx.
However, it will use the configuration file in /etc/nginx. You don t want to modify this directory for all your labs. The best way to handle this is to start Nginx with a custom configuration file:
1
2
# /etc/init.d/nginx stop
# nginx -c /home/bernat/mylab/nginx.conf
While keeping the configuration files related to your lab in your home is an important objective (all your lab related files are in the same place and you can modify them at will), it may be more convenient to still use /etc/init.d/nginx script to control Nginx. Moreover, some daemons do not allow to specify a configuration file. You will then need to modify your root filesystem. This is a pretty common problem with Live CD and it is solved by using an union mount. This is a special kind of mount that will merge two filesystems into one. One of the two filesystems can be read-only and all the changes will be reported to the second one. We will use AUFS. Ensure that you have aufs-tools package installed. Let s start from scratch for this example.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
$ linux init=/bin/sh rootfstype=hostfs
[...]
# mount -n -t proc proc /proc
# mount -n -t sysfs sysfs /sys
# mount -o bind /usr/lib/uml/modules /lib/modules
# mount -n -t tmpfs tmpfs /tmp -o rw,nosuid,nodev
# mkdir /tmp/ro /tmp/rw /tmp/aufs
# mount -n -t hostfs hostfs /tmp/ro -o /,ro
# mount -n -t aufs aufs /tmp/aufs -o noatime,dirs=/tmp/rw:/tmp/ro=ro
# exec chroot /tmp/aufs /bin/bash
It seems that AUFS with hostfs is rather fragile. We could have merged our root filesystem with some directory in our lab using hostfs like this:
1
2
3
4
# mount -n -t hostfs hostfs /tmp/rw -o /home/bernat/mylab/root,rw
# mount -n -t hostfs hostfs /tmp/ro -o /,ro
# mount -n -t aufs aufs /tmp/aufs -o noatime,dirs=/tmp/rw:/tmp/ro=ro
# exec chroot /tmp/aufs /bin/bash
However, the kernel panics on most operations. Therefore, /tmp/rw stays in a tmpfs filesystem. This means that any change to the root filesystem will be lost. I also was unable to issue a correct pivot_root before chroot. This is not really important here. UPDATED: It seems that there are other drawbacks to this setup. For some reason, if you have a separate /usr partition on your host system, you won t be able to write to it until you specify the noxino option to AUFS. Another issue is that AUFS does not notice most changes on the hostfs filesystem. You can fix this with option udba=inotify. However, this is incompatible with the noxino option. Therefore, you need to choose what is best for your lab. You can also mount some parts of your home with hostfs (for example, the source directory of the piece of software that you would like to test). Let s consider that Nginx configuration is in /home/bernat/lab/nginx. We simply uses a symbolic link:
1
2
# rm -rf /etc/nginx
# ln -s /home/bernat/mylab/nginx /etc/nginx
Since we lose the modifications to the root filesystem, the symbolic link should be done each time we start the lab. However, we can modify directly the configuration files in the host or in the UML.

Networking We know how to start our UML. Let s see how we can add some wires to it. UML supports several network adapters. The two most useful are TAP and VDE backend.

TAP A TAP interface is a virtual network kernel device which can be used by userland applications to inject layer 2 frames in it. If the kernel sends frames in a TAP device, they will be received by the application listening to it. This also works the other way around. A TAP interface is a regular ethernet device for the kernel. This means that you can bridge it or use tcpdump on it. The drawback of such interfaces is that you need to be root to create them. The application using them does not need to be run as root. I usually use this nifty shell function to setup TAP interfaces:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
__add_to_bridge()  
    # Optionally, add it to given bridge
    [ -z "$2" ]    
        [ -f /sys/class/net/$2/brforward ]    
            sudo brctl addbr $2
            sudo brctl stp $2 off
            sudo ip link set $2 up
         
        [ -f /sys/class/net/$2/brif/$1 ]    
            # We need to check if it is in another bridge
            bridge=$(echo /sys/class/net/*/brif/$1 2> /dev/null   \
                sed 's+/sys/class/net/\([^/]*\)/.*+\1+') 2> /dev/null
            [ -n "$bridge" ] && \
                sudo brctl delif $bridge $1
            sudo brctl addif $2 $1
         
     
 
tap()  
    sudo tunctl -b -u $(whoami) -t $1 > /dev/null
    sudo ip link set up dev $1
    __add_to_bridge $1 $2
 
It will create the interface and put it into a bridge. For example, you can create two interfaces linked together through a bridge to allow two UML to talk each other:
1
2
$ tap tap-R1 br-R1R2
$ tap tap-R2 br-R1R2
We can setup two UML which will use those interfaces. The first one is setup like this:
1
2
3
4
$ linux init=/bin/sh rootfstype=hostfs eth0=tuntap,tap-R1
[...]
# ip link set up dev eth0
# ip addr add 192.168.0.1/24 dev eth0
The second one:
1
2
3
4
5
6
7
8
9
$ linux init=/bin/sh rootfstype=hostfs eth0=tuntap,tap-R2
[...]
# ip link set up dev eth0
# ip addr add 192.168.0.2/24 dev eth0
# ping 192.168.0.1
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
Warning: time of day goes back (-13832us), taking countermeasures.
64 bytes from 192.168.0.1: icmp_req=20 ttl=64 time=0.633 ms
64 bytes from 192.168.0.1: icmp_req=21 ttl=64 time=0.157 ms
If you are running a firewall on your host, ensure that you let your host forward those packets. By default, bridging code forwards the frames through Netfilter.

VDE A VDE switch is a software emulation of a regular network switch. It is a userland component and does not need to be run as root. You need to install vde2 package to make use of it. You can run a switch using vde_switch command. You will get a console by pressing Enter. From this console, you can configure the switch: add ports, configure VLAN, etc. Like for TAP interfaces, let s setup two UML connected to this switch. The first one is like this:
1
2
3
4
$ linux init=/bin/sh rootfstype=hostfs eth0=vde
[...]
# ip link set up dev eth0
# ip addr add 192.168.0.1/24 dev eth0
And the second one:
1
2
3
4
5
6
7
8
9
$ linux init=/bin/sh rootfstype=hostfs eth0=vde
[...]
# ip link set up dev eth0
# ip addr add 192.168.0.2/24 dev eth0
# ping 192.168.0.1
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
64 bytes from 192.168.0.1: icmp_req=1 ttl=64 time=2.93 ms
Warning: time of day goes back (-8101us), taking countermeasures.
64 bytes from 192.168.0.1: icmp_req=2 ttl=64 time=0.503 ms

The lab So far, we had a look at what can be used to setup a lab. We need to put everything together into some script to setup the lab in a quick way. For each lab that I want to setup, I copy my latest script to a new location and adapt to the topology I want. I have put everything described below in GitHub. Feel free to browse and fork. Our lab will be about setting up a redundant VPN between two remote sites of the same company. This company is using OSPF as an IGP on each site and we will use BGP to let each site know the routes advertised in the other site. We will only use VDE switches for the layer 2 part. Redundant VPN lab Let s implement this lab. You should grab the sources first:
1
2
$ git clone https://github.com/vincentbernat/network-lab.git
$ cd network-lab/lab-redundant-vpn
You have a setup script which will setup the whole lab for you. You also have one directory for each UML. Those directories only contain configuration for the various daemons used inside UML. To make this lab work, ensure that you have quagga (Quagga, an Internet routing daemon supporting both OSPF and BGP) and racoon packages.

UML configuration First look at the end of script. There is something like this:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
case $$ in
    1)
        # Inside UML. Three states:
        [...]
        ;;
    *)
        TMP=$(mktemp -d)
        trap "rm -rf $TMP" EXIT
        check_dependencies
        setup_screen
        # Setup switches
        setup_switch site1
        setup_switch site101
        setup_switch internet
        # Start VM
        start_vm R1 eth0=vde,$TMP/switch-site1.sock
        start_vm R2 eth0=vde,$TMP/switch-site101.sock
        start_vm V1 eth0=vde,$TMP/switch-site1.sock eth1=vde,$TMP/switch-internet.sock
        start_vm V2 eth0=vde,$TMP/switch-site1.sock eth1=vde,$TMP/switch-internet.sock
        start_vm V3 eth0=vde,$TMP/switch-site101.sock eth1=vde,$TMP/switch-internet.sock
        start_vm V4 eth0=vde,$TMP/switch-site101.sock eth1=vde,$TMP/switch-internet.sock
        start_vm I1 eth0=vde,$TMP/switch-internet.sock
        display_help
        cleanup
        screen -X quit
        ;;
esac
The script will check its PID ($$). If it is 1, this means it has been invoked as init. Otherwise, it means it has been launched by the user. We will see the init part later. When called, we first check some dependencies, then we setup screen. Everything is run inside a screen session. No multiple windows. The next step is to setup the various switches needed for our lab. We take one switch for each site and one switch for Internet . The function that setups the switch just invokes vde_switch with the help of start-stop-daemon to record the PID (to shutdown the lab properly). Then, we start our various UML. For each of them, we need to specify which network adapters we need. For R1 and R2, we don t use a network adapter to connect to the internal network. We will use a dummy0 interface. To emulate Internet, we use another UML, I1. For each UML, our script will be used as init. In this case, its PID will be 1. As we have seen earlier, the script checks if it is called as init. If it is the case, it will setup the UML. During this setup, host specific configuration will be applied:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
echo "[+] Setup UML"
sysctl -w net.ipv4.ip_forward=1
case $ uts  in
    R1)
        modprobe dummy
        ip link set up dev dummy0
        ip addr add 192.168.15.1/24 dev dummy0
        ip addr add 192.168.1.10/24 dev eth0
        setup_quagga
        ;;
    R2)
        modprobe dummy
        ip link set up dev dummy0
        ip addr add 192.168.115.1/24 dev dummy0
        ip addr add 192.168.101.10/24 dev eth0
        setup_quagga
        ;;
    V1)
        ip addr add 192.168.1.11/24 dev eth0
        ip addr add 1.1.2.1/24 dev eth1
        ip route add default via 1.1.2.10
        setup_quagga
        setup_racoon 1.1.2.1 1.1.1.1 192.168.0.0/19-192.168.100.0/19
        ;;
    V2)
        ip addr add 192.168.1.12/24 dev eth0
        ip addr add 1.1.2.2/24 dev eth1
        ip route add default via 1.1.2.10
        setup_quagga
        setup_racoon 1.1.2.2 1.1.1.2 192.168.0.0/19-192.168.100.0/19
        ;;
    V3)
        ip addr add 192.168.101.13/24 dev eth0
        ip addr add 1.1.1.1/24 dev eth1
        ip route add default via 1.1.1.10
        setup_quagga
        setup_racoon 1.1.1.1 1.1.2.1 192.168.100.0/19-192.168.0.0/19
        ;;
    V4)
        ip addr add 192.168.101.14/24 dev eth0
        ip addr add 1.1.1.2/24 dev eth1
        ip route add default via 1.1.1.10
        setup_quagga
        setup_racoon 1.1.1.2 1.1.2.2 192.168.100.0/19-192.168.0.0/19
        ;;
    I1)
        ip addr add 1.1.1.10/24 dev eth0
        ip addr add 1.1.2.10/24 dev eth0
        ;;
esac
After all this, we drop to a shell to allow you to issue additional commands interactively.

Testing We need to wait one minute after the lab has been started. We can then check the routing table of various equipments. For example, here is R1 routing table (you need to type vtysh to get Quagga console):
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
R1# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
       I - ISIS, B - BGP, > - selected route, * - FIB route
C>* 127.0.0.0/8 is directly connected, lo
O   192.168.1.0/24 [110/10] is directly connected, eth0, 00:25:42
C>* 192.168.1.0/24 is directly connected, eth0
O   192.168.15.0/24 [110/10] is directly connected, dummy0, 00:25:42
C>* 192.168.15.0/24 is directly connected, dummy0
O>* 192.168.115.0/24 [110/20] via 192.168.1.11, eth0, 00:02:24
  *                           via 192.168.1.12, eth0, 00:02:24
We can see that if this router wants to contact R2, it has learned a multipath route with OSPF and can then sends its packets through V1 or V2, as expected. Here is the routing table of V3:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
V3# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
       I - ISIS, B - BGP, > - selected route, * - FIB route
K>* 0.0.0.0/0 via 1.1.1.10, eth1
C>* 1.1.1.0/24 is directly connected, eth1
C>* 127.0.0.0/8 is directly connected, lo
O   192.168.15.0/24 [110/20] via 192.168.101.14, eth0, 00:04:26
B>* 192.168.15.0/24 [20/20] via 192.168.1.11 (recursive via 1.1.1.10), 00:07:16
O   192.168.101.0/24 [110/10] is directly connected, eth0, 00:07:30
C>* 192.168.101.0/24 is directly connected, eth0
O>* 192.168.115.0/24 [110/20] via 192.168.101.10, eth0, 00:07:18
We see that to contact R1, V3 knows two routes. One with BGP through the VPN. This is the one that will be used. The other one is through V4 but will not be used unless the BGP route is lost. We can ping R2 from R1:
1
2
3
4
R1# ping -I 192.168.15.1 192.168.115.1
PING 192.168.115.1 (192.168.115.1) from 192.168.15.1 : 56(84) bytes of data.
Warning: time of day goes back (-459129us), taking countermeasures.
64 bytes from 192.168.115.1: icmp_req=1 ttl=62 time=1.10 ms
Let s suppose we break the link between V1 and the Internet (using ip link set down eth1 on V1). The previous multipath route on R1 is now a single route through V2. Moreover, V3 has lost its BGP route to V1 and is now using its OSPF route through V4:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
V3# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
       I - ISIS, B - BGP, > - selected route, * - FIB route
K>* 0.0.0.0/0 via 1.1.1.10, eth1
C>* 1.1.1.0/24 is directly connected, eth1
C>* 127.0.0.0/8 is directly connected, lo
O>* 192.168.15.0/24 [110/20] via 192.168.101.14, eth0, 00:10:13
O   192.168.101.0/24 [110/10] is directly connected, eth0, 00:13:17
C>* 192.168.101.0/24 is directly connected, eth0
O>* 192.168.115.0/24 [110/20] via 192.168.101.10, eth0, 00:13:05
We could enhance the resiliance of such a setup by adding a dedicated link between V1 and V2 and between V3 and V4 and running OSPF on this link (or iBGP). This would allow us to support several outages on our network.

Conclusion The script used to setup this lab can be adapted to setup other labs with little effort. Please, note that it depends on the configuration of your environment. However, the whole lab is contained in a single directory and is very small. You can email it to your friends or publish it on GitHub. It can also be extended to support Dynagen to include some Cisco equipments in your lab (for example, to experiment with VRF or MPLS).

Craig Small: JFFNMS 0.9.0 Released

After 3 release candidates jffnms is now at version 0.9.0. Both the web frontend and backend (engines) have had extensive re-work done to them to cleanup and tighten up the code. There should be a lot less warnings and errors from PHP when you set to a higher error level.

Tollef Fog Heen: Upgrading Alioth

A while ago, we got another machine for hosting Alioth and so we started thinking about how to use that machine. It's a used machine and not massively faster than the current hardware, so just moving everything over wouldn't actually get us that much of a performance upgrade. However, Alioth is using FusionForge, which is supposed to be able to run on a cluster of machines. After all, this was originally built for SourceForge.net, which certainly does not run on a single host. So, a split of services is what we'll do. This weekend, we're having a sprint in Collabora's office in Cambridge, actually implementing the split and doing a bit of general planning for the future. Last afternoon (Friday), European time, we started the migration. The first step is to move all the data off the Xen guest on wagner, where Alioth is currently hosted. This finished a few minutes ago; it turns out syncing about 8.5 million files across almost 400G of data takes a little while. The new host is called vasks and will host the database, run the main apache and be the canonical location for the various SCM repositories. We are not decomissioning wagner, but it'll be reinstalled without Xen or other virtualisation which should help performance a bit. It'll host everything that has lower performance requirements such as cron jobs, mailing lists and so on. I'll try to keep you all updated and feel free to drop by #alioth on irc.debian.org if you have any questions.

20 May 2011

Russ Allbery: git-pbuilder 1.20

I've been sadly neglecting this script, but tonight I finally found some time to catch up on all the suggestions and bug reports that Guido G nther has been letting me know about (and I greatly appreciate that!). This version merges in all the good ideas people have had. This version removes the -i and -I flags passed to dpkg-source when the package format is 3.0, since with 3.0 packages, dpkg-source both already ignores Git files by default and ignores various other things that were not included in the pattern. It also offers much better configuration methods for the ignore patterns, which git-pbuilder was overriding. So that's all skipped now for 3.0 packages. git-pbuilder still passes in the ignore flags for 1.0 packages, since they don't ignore Git files by default. The environment variable BUILDER can now be set to choose the builder, which is still cowbuilder by default. The other supported option is qemubuilder, and for it git-pbuilder selects an appropriate configuration file. Additional flags for the builder can be set in the GIT_PBUILDER_OPTIONS environment variable and will be passed along. There are also a few more minor fixes: better documentation, always printing out what we're doing at the start of a run, and preserving the exit status of pdebuild and exiting with the same status. These changes are based on work by Guido G nther, Yaroslav Halchenko, and Beno t Knecht. You can get the latest version from my scripts page.

17 May 2011

Vincent Bernat: Debian on Thinkpad Edge 11"

This has been several months since I didn t use much my laptop, an Asus S5N: it was crippled by a short battery life and was hit by several bugs, most of them targeting the i855GME graphic chipset thanks to the use of GEM and KMS in the latest Xorg. I did not want to invest too much in a laptop and I have found a rebate from Lenovo for Thinkpad Edge series. Voil , I have a Thinkpad Edge 11 .

Hardware Thinkpad Edge is a entry-level notebook from Lenovo. In France, it is available for around 500 , thanks to the usual parity between dollar and euro. Thinkpad Edge 11" enclosure

Hardware configuration Lenovo do not sell directly in France. Therefore, you cannot configure your laptop as you want. There is two or three configurations available. This can be a bit frustrating if you want something specific. Thinkpad Edge 11" parts Here is the configuration that I bought:
  • Intel Core i3 380UM at 1.33 GHz CPU.
  • 11.6 LED screen, with a resolution of 1366x768.
  • 4 GB RAM.
  • 320 GB hard drive.
  • Wireless network (Centrino Wireless-N) and bluetooth.
  • Broadband modem (WWAN)
It s a small laptop but it is not as light as it should be (1.5 kg or 3.3 pounds). Apart from the power supply, you don t get any extra. My Asus was shipped with a wireless mouse, a transport bag, a sleeve, an USB key and a CD burner. On one side, you get two USB ports and on the other side, one USB port, an HDMI port, a network outlet and a speaker jack. The VGA connector is placed on the rear. Thinkpad Edge 11"

BIOS First step: checking BIOS settings. WiMAX can be disabled as well as any other component that you may not need. The keyboard can be configured so that function keys are available directly instead of having to use the Fn key. There is an option called Computrace from AbsoluteSoftware which seems targeted at find back your laptop in case it gets stolen or just let the FBI know where you are. I have disabled it.

Debian installation Debian can be installed from an USB stick. This is really easy now. You grab the ISO image you want to use (I take the one for AMD64 in testing because I will run unstable) and to copy it on the USB stick:
1
# dd if=debian-testing-amd64-netinst.iso of=/dev/sdb
The previous way of making the USB stick bootable, installing a kernel, an initrd and the ISO image is deprecated (and does not work). Moreover, using testing to install Debian allows me to use a 2.6.37 kernel and circumvent some problems that may have happen with an older kernel. During the installation, I choose a guided partitioning using encrypted LVM. I choose to have several partitions but I keep /usr in the root partition since having a separate /usr is less and less supported, especially with piece of software like udev. The partitioning tool in Debian installer do not allow to resize volumes, you need to destroy them and recreate them as well as their content. However, this is still faster than writing a patch for this. When the installation is finished, the laptop boots on a 2.6.38 kernel, asks for the passphrase to decrypt the LUKS volume and display the login prompt a few seconds later.

Software configuration Most of the hardware seems to be supported. No obscure commands to run to get a working network. Therefore, I start by setting up my usual environment.

Setting up the desktop First, I change /etc/apt/sources.list to get unstable:
deb http://ftp.fr.debian.org/debian/ unstable main
deb-src http://ftp.fr.debian.org/debian/ unstable main
deb http://ftp.fr.debian.org/debian/ experimental main contrib non-free
I also want that everything that is installed from experimental to be upgraded if an update is available in experimental. This is done by creating a file /etc/apt/preferences.d/experimental:
Package: *
Pin: release experimental
Pin-Priority: 101
Then, I install the following packages:
  • zsh, my favourite shell with my hand-made maturating configuration. I should put it on GitHub one day.
  • mg, is the editor that I use to quickly edit a configuration file. I don t like vi. I have used jed for a long time but its default configuration can be cumbersome and it misses some emacs keybinding.
  • emacs, my work environment. I use it as a text editor and mail reader (with Gnus). I don t use it as a browser nor as a terminal. As for zsh, I have my configuration files ready to be copied anywhere.
  • fvwm, my window manager since quite some time. I use fvwm-crystal on top of it. My configuration is described in another post. I also use conky.
  • rxvt-unicode, a terminal. It has a very good terminal emulation and some fancy features like transparency (using compositing or not). It is light and fast and allows to search in the backbuffer. It can use a client/server system that will close all your windows when the server crashes. This almost never happens.
  • unison, a synchronisation tool that allows me to keep the same files on my different computers.
  • gdm3 is the only display manager that is able to talk correctly to ConsoleKit and get the rights set correctly without adding yourself to a lot of groups.
Nowadays, I use the kernel from Debian and I do not have to run my own configuration for it. The glory days where I had to recompile the kernel before plugging a new USB toy have passed. I remove rpcbind and nfs-common. I use ferm to setup a simple firewall. Here is my ferm.conf:
domain (ip ip6)  
  table filter  
chain INPUT  
    policy DROP;
    mod state state INVALID DROP;
    mod state state (ESTABLISHED RELATED) ACCEPT;
    interface lo ACCEPT;
    proto tcp dport ssh ACCEPT;
    
chain OUTPUT  
    policy ACCEPT;
    mod state state (ESTABLISHED RELATED) ACCEPT;
    
chain FORWARD  
    policy DROP;
    
     
 
unison will then sync my configuration files and my documents. After this step, my environment is ready.

Hardware configuration This is usually the most troublesome part. However, looking at the output of lspci and lsusb, everything seems pretty easy to get supported. Here is lspci output:
00:00.0 Host bridge: Intel Corporation Core Processor DRAM Controller (rev 02)
00:02.0 VGA compatible controller: Intel Corporation Core Processor Integrated Graphics Controller (rev 02)
00:16.0 Communication controller: Intel Corporation 5 Series/3400 Series Chipset HECI Controller (rev 06)
00:1a.0 USB Controller: Intel Corporation 5 Series/3400 Series Chipset USB2 Enhanced Host Controller (rev 06)
00:1b.0 Audio device: Intel Corporation 5 Series/3400 Series Chipset High Definition Audio (rev 06)
00:1c.0 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 1 (rev 06)
00:1c.1 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 2 (rev 06)
00:1c.5 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 6 (rev 06)
00:1d.0 USB Controller: Intel Corporation 5 Series/3400 Series Chipset USB2 Enhanced Host Controller (rev 06)
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev a6)
00:1f.0 ISA bridge: Intel Corporation Mobile 5 Series Chipset LPC Interface Controller (rev 06)
00:1f.2 SATA controller: Intel Corporation 5 Series/3400 Series Chipset 4 port SATA AHCI Controller (rev 06)
00:1f.3 SMBus: Intel Corporation 5 Series/3400 Series Chipset SMBus Controller (rev 06)
00:1f.6 Signal processing controller: Intel Corporation 5 Series/3400 Series Chipset Thermal Subsystem (rev 06)
03:00.0 Network controller: Intel Corporation Centrino Wireless-N 1000
09:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 03)
ff:00.0 Host bridge: Intel Corporation Core Processor QuickPath Architecture Generic Non-core Registers (rev 02)
ff:00.1 Host bridge: Intel Corporation Core Processor QuickPath Architecture System Address Decoder (rev 02)
ff:02.0 Host bridge: Intel Corporation Core Processor QPI Link 0 (rev 02)
ff:02.1 Host bridge: Intel Corporation Core Processor QPI Physical 0 (rev 02)
ff:02.2 Host bridge: Intel Corporation Core Processor Reserved (rev 02)
ff:02.3 Host bridge: Intel Corporation Core Processor Reserved (rev 02)
And lsusb output (I disabled the broadband modem in the BIOS and therefore, it does not appear here):
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 8087:0020 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 002: ID 8087:0020 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 003: ID 0bda:0158 Realtek Semiconductor Corp. USB 2.0 multicard reader
Bus 001 Device 004: ID 0a5c:217f Broadcom Corp. Bluetooth Controller
Bus 002 Device 003: ID 5986:01a7 Acer, Inc Lenovo Integrated Webcam
There is a dedicated page for the Thinkpad Edge 11 on ThinkWiki. This wiki is one of the best source of information when you get a Thinkpad. The kernel as shipped by Debian kernel team contains every module that you may need. One of them, thinkpad_acpi will handle most of the perks under the hood. I do not use a complete desktop like Gnome. I usually prefer to use command line tools and put them in .xsession for persistence. There is usually some graphical utility to do the same thing. There are however some configuration steps that are now far easier to do with a graphical tool. Since I use X all the time, I just rely on those tools when necessary.

BIOS upgrade From lshw output, I see that I already have the latest BIOS available. No need to update then. In case you need to update, yours, ThinkWiki has a documentation to upgrade. You can find memdisk image in syslinux package.

Graphic card The graphic card is integrated into the CPU. Intel usually ships graphic cards well supported by open source drivers. ThinkWiki tells us:
A very recent Linux distribution with kernel 2.6.33 and Intel Xorg driver 2.11 or newer is recommended. Note that XVideo (Xv) playback at certain frame sizes does not work properly unless you have at least version 2.12 of the Intel Xorg driver.
With Debian Wheezy, you get version 2.15. Therefore, configuring the graphic card is just a matter of installing xserver-xorg-video-intel and that s all. No /etc/X11/xorg.conf needed. HDMI and VGA outputs are detected without configuring anything. They can be enabled with xrandr. I did not try if it was possible to use both of them at the same time.
Screen 0: minimum 320 x 200, current 1366 x 768, maximum 8192 x 8192
LVDS1 connected 1366x768+0+0 (normal left inverted right x axis y axis) 256mm x 144mm
   1366x768       60.0*+   50.0  
   1360x768       59.8     60.0  
   1024x768       60.0  
   800x600        60.3     56.2  
   640x480        59.9  
VGA1 disconnected (normal left inverted right x axis y axis)
HDMI1 disconnected (normal left inverted right x axis y axis)
DP1 disconnected (normal left inverted right x axis y axis)
Compositing works without any problem. I use only transparency and therefore, I don t know what the speed is on a desktop with all effects turned on. Regarding video acceleration, XVideo is supported but the graphic card also features hardware decoding with i965-va-driver package. Here is the output of vainfo:
libva: libva version 0.32.0
libva: va_getDriverName() returns 0
libva: Trying to open /usr/lib/dri/i965_drv_video.so
libva: va_openDriver() returns 0
vainfo: VA API version: 0.32
vainfo: Driver version: i965 Driver 0.1
vainfo: Supported profile and entrypoints
  VAProfileMPEG2Simple            : VAEntrypointVLD
  VAProfileMPEG2Main              : VAEntrypointVLD
  VAProfileH264Baseline           : VAEntrypointVLD
  VAProfileH264Main               : VAEntrypointVLD
  VAProfileH264High               : VAEntrypointVLD
With Debian, VLC can make use of this if you put ffmpeg-hw=1 in your ~/.config/vlc/vlcrc. We are then greeted by this:
[0x2629040] avcodec decoder: Using VA API version 0.32 for hardware decoding.

Network The network card works out of the box. ThinkWiki tells us that there can be stability problems with 2.6.36 kernel. During the installation, a 2.6.37 was used without any problem. I am now running a 2.6.38 and never got a glitch. It is likely that the described problem was solved with 2.6.37.

UltraNav UltraNav is the combination of the usual touchpad and the trackpoint which is a small joystick in the center of the keyboard and is well known to Thinkpad users since most of them have one. The trackpoint is a great input device. Nowadays, all input devices are configured with the help of xinput command. Here is the output of xinput list:
  Virtual core pointer                      id=2    [master pointer  (3)]
      Virtual core XTEST pointer                id=4    [slave  pointer  (2)]
      SynPS/2 Synaptics TouchPad                id=11   [slave  pointer  (2)]
      TPPS/2 IBM TrackPoint                     id=12   [slave  pointer  (2)]
  Virtual core keyboard                     id=3    [master keyboard (2)]
  Virtual core XTEST keyboard               id=5    [slave  keyboard (3)]
  Power Button                              id=6    [slave  keyboard (3)]
  Video Bus                                 id=7    [slave  keyboard (3)]
  Power Button                              id=8    [slave  keyboard (3)]
  Integrated Camera                         id=9    [slave  keyboard (3)]
  AT Translated Set 2 keyboard              id=10   [slave  keyboard (3)]
  ThinkPad Extra Buttons                    id=13   [slave  keyboard (3)]
The command xinput list-props will give you additional information on each input device listed above. You can then configure any settings using xinput set-prop. For example, if you want to enable scrolling by using the edges of the touchpad and enable palm detection, you can use the following commands:
1
2
$ xinput set-int-prop "SynPS/2 Synaptics TouchPad" 'Synaptics Edge Scrolling' 8 1 1 0
$ xinput set-int-prop "SynPS/2 Synaptics TouchPad" 'Synaptics Palm Detection' 8 1
If you want to disable the touchpad and only use the trackpoint, use the following command:
1
$ xinput set-int-prop "SynPS/2 Synaptics TouchPad" 'Synaptics Off' 8 1
Each property has a type and a size. There is no place for guessing since every property is listed in man 4 synaptics. The same applies for the trackpoint but the correct manual page is man 4 evdev. For example, if you want to enable scrolling using the middle button, use:
1
2
$ xinput set-int-prop "TPPS/2 IBM TrackPoint" "Evdev Wheel Emulation" 8 1
$ xinput set-int-prop "TPPS/2 IBM TrackPoint" "Evdev Wheel Emulation Button" 8 2
All those commands can be tossed in ~/.xsession.

Keyboard hot keys Thinkpad laptops usually have a lot of hot keys. This is not the case of the Thinkpad Edge which only features a few of them:
  • volume control
  • brightness control
  • webcam and external screen toggle
  • wireless toggle
  • multimedia keys
All those keys trigger an event in one of the keyboard listed in xinput list. Therefore, your window manager or any other application can intercept them and act upon. You can use evtest to check which key triggers which event on which keyboard. The acpid daemon listen on those keyboards and can also trigger some actions. For example, it will run ibm-wireless.sh when you hit the wireless toggle. By default, this will enable or disable your wireless card. But you can configure the script to use a multistate toggle and disable or enable the bluetooth card as well.

Sound card As for the network card, the sound card works out of the box. I use PulseAudio because it allows per-application volume control and per-application sound card selection. It comes handy if you have some bluetooth headset. To make PulseAudio the default application for most applications, I put this in my ~/.asoundrc:
pcm.!default  
  type pulse
 
ctl.!default  
  type pulse
 

Wireless card A firmware is needed to make this card work. On Debian, you need to install firmware-iwlwifi package. Then, iwlagn module needs to be loaded. You can check with rfkill list that the wireless card has not been disabled. iw list will then give the status:
Wiphy phy0
Band 1:
    Capabilities: 0x872
        HT20/HT40
        Static SM Power Save
        RX Greenfield
        RX HT20 SGI
        RX HT40 SGI
        No RX STBC
        Max AMSDU length: 3839 bytes
        No DSSS/CCK HT40
    Maximum RX AMPDU length 65535 bytes (exponent: 0x003)
    Minimum RX AMPDU time spacing: 4 usec (0x05)
    HT RX MCS rate indexes supported: 0-15, 32
    TX unequal modulation not supported
    HT TX Max spatial streams: 1
    HT TX MCS rate indexes supported may differ
    Frequencies:
        * 2412 MHz [1] (14.0 dBm)
        * 2417 MHz [2] (14.0 dBm)
        * 2422 MHz [3] (14.0 dBm)
        * 2427 MHz [4] (14.0 dBm)
        * 2432 MHz [5] (14.0 dBm)
        * 2437 MHz [6] (14.0 dBm)
        * 2442 MHz [7] (14.0 dBm)
        * 2447 MHz [8] (14.0 dBm)
        * 2452 MHz [9] (14.0 dBm)
        * 2457 MHz [10] (14.0 dBm)
        * 2462 MHz [11] (14.0 dBm)
        * 2467 MHz [12] (14.0 dBm) (passive scanning, no IBSS)
        * 2472 MHz [13] (14.0 dBm) (passive scanning, no IBSS)
    Bitrates (non-HT):
        * 1.0 Mbps
        * 2.0 Mbps (short preamble supported)
        * 5.5 Mbps (short preamble supported)
        * 11.0 Mbps (short preamble supported)
        * 6.0 Mbps
        * 9.0 Mbps
        * 12.0 Mbps
        * 18.0 Mbps
        * 24.0 Mbps
        * 36.0 Mbps
        * 48.0 Mbps
        * 54.0 Mbps
max # scan SSIDs: 20
Supported interface modes:
     * IBSS
     * managed
     * monitor
I don t know why, but 802.11a channels have been disabled. I don t have interest in them and therefore I did not search why this happens.
iwlagn 0000:03:00.0: Detected Intel(R) Centrino(R) Wireless-N 1000 BGN, REV=0x6C
iwlagn 0000:03:00.0: device EEPROM VER=0x15d, CALIB=0x6
iwlagn 0000:03:00.0: Device SKU: 0X9
iwlagn 0000:03:00.0: Valid Tx ant: 0X1, Valid Rx ant: 0X3
iwlagn 0000:03:00.0: Tunable channels: 13 802.11bg, 0 802.11a channels
iwlagn 0000:03:00.0: irq 44 for MSI/MSI-X
iwlagn 0000:03:00.0: loaded firmware version 128.50.3.1 build 13488
Wireless is one of the thing that make me use graphical applications like Network Manager. Command line tools are too low level to quickly connect to some random wireless network. There was a large debate in debian-devel@ldo to determine if Network Manager was ready to replace ifup and ifdown commands, even for servers. It seems that this is not something impossible and therefore, I think it has become flexible enough to ensure most of the configuration tasks I need.

Bluetooth The bluetooth chipset is hosted on the USB bus. hci_usb module will take care of it. It supports AD2P and can be used to listen to music with a compatible headset. hciconfig will tell us if the chipset is working. Again, bluetooth is easier to configure with a graphical tool. BlueZ, the bluetooth stack in Linux, uses DBus to communicate with other applications. The command line tools have been deprecated and therefore, you need to fight with some DBus binding to pair your computer to your headset. gnome-bluetooth package comes with an applet which allows you to quickly pair to any device. If you install pulseaudio-module-bluetooth package, any bluetooth headset should work out of the box.

Broadband (WWAN) To configure the broadband modem, the most difficult thing to do is to grab the appropriate firmware. ThinkWiki comes with various instructions to get it. I did not keep my Windows installation and therefore, I grabbed the Qualcomm Gobi 2000 Wireless WAN driver and I unpacked it with some Windows host with msiexec as instructed (I did have to execute the installation program first, otherwise, msiexec tells that the file is invalid). I choosed the default firmware since my laptop is unlocked. ThinkWiki page explains which firmware to choose in which situation. The package gobi-loader needs to be installed. The firmware files amss.mbn, apps.mbn and UQCN.mbn should be placed in /lib/firmware/gobi. They will be loaded by udev. To avoid to reboot, I used this command:
1
# /lib/udev/gobi_loader -2000 /dev/ttyUSB0 /lib/firmware/gobi
To configure the 3G connection, I just used Network Manager. Ensure with rfkill list that your WWAN card is not blocked. I did not test the GPS part that needs to be enabled on Windows.

Webcam Nowadays, USB webcam are UVC device and there is a driver to support this class of devices. Check that the uvcvideo module is correctly loaded and you should have something like this in your dmesg:
uvcvideo: Found UVC 1.00 device Integrated Camera (5986:01a7)
cheese is a small application that can be used to check if the webcam operates as expected.

Card reader Nothing to do. This works out of the box.

Suspend to memory and disk Install the uswsusp package which will also pull pm-utils. You will get pm-suspend and pm-hibernate commands. pm-suspend will enable suspend to memory. On resume, the system takes a whole minute to become operational. I get the following message:
r8169 0000:09:00.0: eth0: unable to apply firmware patch
PM: resume of devices complete after 60786.035 msecs
PM: Finishing wakeup.
Installing firmware-realtek package solves this issue. It should also be solved by a future kernel. I also have this message when I come back from sleep:
[drm:i915_hangcheck_ring_idle] *ERROR* Hangcheck timer elapsed... render ring idle [waiting on 9297, at 9297], missed IRQ?
The X server is then slow as hell. The message can be fixed by using i915.semaphores=1 as a kernel parameter but the X server is still slow. It seems that running powertop restores the original speed. This is a known problem. On sleep, the battery loses 2% every 6 hours. This allows you to keep your laptop on sleep for several days if needed. pm-hibernate enable suspend to disk. The suspend part works well but on resume, the kernel freezes after reading back the image into memory. There are some documentation to debug this problem but I prefer to wait for 2.6.39 which may just solves this issue.

Power management Let s see on how we could enhance battery life. First, cpufrequtils package will slow down the processor when it is not used by using the ondemand governor. The frequency will drop from 1.33 GHz to 666 MHz. Several tools are then available:
  • pm-utils package comes with scripts to enable some powersaving features. Those scripts are run on ACPI events (for example, when you unplug the AC adapater). There is no way to put the laptop to sleep after two hours of inactivity for example. You need something like sleepd to handle this part. I have found nothing that could dim the screen on inactivity.
  • laptop-mode-tools is also a set of script to be run on ACPI events. It is therefore somewhat superseded by pm-utils.
  • gnome-power-manager is a daemon that makes use of upowerd. Despite its name, you can run it outside of the Gnome environment. It will put your laptop to sleep on inactivity and will dim the screen when needed. All other functionalities are handled to upowerd. This seems the way to go on today Linux desktops.
Behing the scene, upowerd will call at some point pm-powersave from pm-utils. This allows to run scripts when unplugging the AC adapter. There are some scripts shipped in /usr/lib/pm-utils/power.d. You can add your own in /etc/pm/power.d. Some scripts are configurable. For example, if you create /etc/pm/config.d/local and put SATA_ALPM_ENABLE=true in it, this will enable SATA ALPM mode on battery. You can check what needs to be done with powertop or go to a website like Less Watts to get some tips on what to configure to get a better battery life. Without adding any script, I get 5 hours of battery life with the laptop doing almost nothing. Lenovo tells that you should get 6 hours of battery life. I think that on moderate workloads, you should get 4 hours.

Conclusion This Lenovo Edge has a good value for money. It features a great keyboard for a laptop of its size. It could be lighter and thinner. Its battery life is pretty good. Almost everything works out of the box with GNU/Linux. I hope that 2.6.39 kernel will solve the remaining issues like the suspend on disk issue.

16 May 2011

Sylvestre Ledru: Status of LLVM in Debian

Here is some updates on the LLVM world into Debian.
LLVM and CLANG getting better and better after each release. After some lag, it is now time to have all these packages up-to-date into the archive. Here is an update of their status in Debian: Deprecated stuffs:

John Sullivan: Send bricks to Nintendo

Brick Nintendo before they brick you! Connectivity can be a force for autonomy and freedom, or it can be used for control and subjugation. The Nintendo 3DS is an example of the latter. Okay, yeah, I just said a cute video game console is a threat to personal freedom. Hear me out. If you haven't read the highlights of Nintendo's Terms of Service for the 3DS yet, you should do that. On one hand, it's "just" a games console. On the other hand, it's a computer, marketed to children, which monitors your location (based on proximity to wifi access points), uploads your photos and messages to Nintendo's servers (giving them a license to re-use that information in various ways), and tracks your activity. Now that's entertainment. If you wanted your Nintendo computer to stop doing those things, you could try to install some 3rd-party "homebrew" software on it. Except then, according to the Terms, Nintendo would "brick" your device render it inoperable, when the modified software is detected. Would you accept Ford disabling your car because you had a mechanic friend fix a problem using some aftermarket parts? Microsoft breaking your computer hardware because you installed Firefox? The ability to stop your electronics from spying on you is an important freedom to defend, even if you don't ever plan on purchasing a Nintendo device. Such terms, and the Digital Restrictions Management software that enforces them, are a problem in all areas of digital life. It's the same fundamental strategy as Apple controlling what you can install on the iPhone or iPad, or Sony sending police officers to a programmer's house because he showed other people how to modify their own computers to make them able to run non-Sony-approved software. If we don't set a strong precedent against this behavior, it will become more prevalent. If we lose the ability to install what we like on our computers, then we lose the ability to use those computers in any way that isn't pre-approved by a corporation. We lose it not only because we ourselves can't install some software maybe we hate changing anything on our computers at all but because the people who write the software we rely on lose that freedom, which ends up impacting even the most non-technical of computer users. Companies like to portray anyone arguing for this freedom as criminals just wanting to illegally copy software. But that's obviously not true -- a person who follows all laws, even the most absurd, might still want to install something to stop Nintendo from receiving copies of her photos or reporting on her location. Companies use the threat of illegal copying to justify massive infringements on real personal freedoms. People often say, "well, just don't buy it then". I agree don't buy it. But don't think this is merely a consumer issue. Terms like Nintendo's are backed by the force of criminal law the Digital Millennium Copyright Act in the US for example which can put you in jail for helping people to modify their own computers. So this is more than a mere preference issue, it's also one of law and ethics. It's not a free market when one side has coercive state resources in play to prop up its arbitrary business model. To help send this message, I just donated to send 5 bricks to Nintendo. I hope you'll send one or five too. The campaign has a goal of sending 200 bricks by the end of today (Monday, May 16th). Nintendo clearly wants to get into the brick-making business, so surely they'll appreciate the extra inventory. (Disclosure: While I do work for the Free Software Foundation and am paid partly to run the Defective by Design campaign, I really did donate, and support this campaign personally.)

Lars Wirzenius: TDDD, more thoughts

My previous blog entry about Test Driven Distro Development (TDDD) was rather rambling. Here is the condensed version, plus some further thoughts. I would like to automatically determine whether "testing" is minimally ready to be released, at any given point in time. To implement this, I would like to see Debian get tools that would allow us to define specific minimal functionality that should work in a Debian release, and automatically verify that it does. I would like us to use these tools to determine whether a release is ready for release, with manual override from the release team. Ideally, we would use the tools to prevent breakages from moving into "testing", so that it gets closer to always being releasable. Getting test coverage sufficiently high that we can replace the release team's judgement abour releasablity will never happen: Debian is rather too large and and complex for that. However, we can reap much benefit from even relatively small coverage. Here is my initial suggestion for the minimal criteria that should be satisfied, at all times, in "testing". They are aimed at making sure that even if something else breaks, at least the basic stuff works well enough that one can log in and fix things. These are minimal criteria. They're not meant to verify that everything in Debian works. After these work, adding more coverage is not just possible, but reasonably easy in many cases. Want to make sure DNS gets configured right? Write a test that logs in and does a lookup inside the test system. After the tools have evolved, writing a simple test like that should be only a line or two. The tests I am talking here are system tests, or acceptance tests for the whole system, not just for individual packages. Unit tests would test individual functions or methods or classes; integration tests would test groups of them, or possibly entire programs; system tests test groups of programs/packages. The point is not to verify that every package works in every possible case, but that the system as a whole works. Most package-specific functional tests should be part of the package's own test suite. We should also run those, but that's somewhat orthogonal to the vision I'm trying to articulate here. At least the following scenarios should be tested (the above tests should be run after each scenario): It may be useful to run the tests against "unstable" as well as "testing". However, rejecting an upload to "unstable" may be less wise than preventing packages from being copied from "unstable" to "testing". Exactly how this the development processes should adopt TDDD is an interesting question, but a reasonable toolset should be adaptable to anything we want to do. (Indeed, it would be possible to structure the toolset so that it isn't specific to developing a distro: a "system-test" tool that a sysadmin could use to verify that a server works would probably be useful to some people. Perhaps such a tool exists and could be adapted for TDDD?) Running these tests for security updates to "stable" would obviously also be a possibility that should perhaps be considered. Running these tests would happen something like this: I would further like to see Debian make it possible to have isolated branches for development. This would help with, for example, large transitions that take a long time. The transition could be prepared in a branch of its own, and when it's ready to be pushed into unstable, the automatic tests outlined above could be used to verify the transition doesn't break anything too critical. (If the tests fail, the push fails; if necessary, that can be overridden manually, but that should be rare.) In my view, branches would greatly benefit Debian development, but they're not necessary for starting to implement TDDD. My condensed versions are still too long.

Vincent Bernat: Migrating from Trac to GitHub

I was hosting my projects on several Trac instances on a dedicated server. However, since several months, spam was pourring.

Trac & spam

How to fight spam with Trac There is not built-in mechanism in Trac to fight spam. However, plugins enable to setup several ways to stop spam (or at least, reduce it). One of them is Akismet, a web service who can tell you if some message is a spam or not. It is able to catch most of the spam. A bayesian filter complement this service. A learning step is mandatory and can become cumbersome when the vast majority of the tickets you receive is just spam. I used those two mechanisms on my Trac instances. Another path is to require an account to post a ticket. Creating an account each time you want to create a ticket for some software is burdensome. I did not want to do this to the poor souls that have found a bug in one of my projects. An alternative is to setup a public account and adding somewhere in the page some instructions on how to use it when you want to open a ticket but don t want to open an account.

Removing spam Despite Akismet and the bayesian filter, I still got two or three spams a day. I catched them in my RSS flow or in my email if they were not classified as spam. There are two steps to remove the spam:
  • tell the bayesian filter by using the administration panel in the web interface,
  • remove the spam from the database using some SQLite commands directly on the server hosting the Trac instance.
Indeed, Trac does not propose the possibility to remove a ticket. Of course, there are plugins, but you must find them, install them, find how to configure them, etc. Cumbersome.

GitHub GitHub is a popular source code hosting service which uses Git as a backend. This is a proprietary platform but a lot of open source projects have migrated to it, thanks to its features and the good performances. Everybody is on GitHub. This was the ideal opportunity.

Features First, GitHub allows to host Git repository. You push your Git repositories. There is a nice and fast web interface to browse them. However, I still keep an instance of cgit. Then, GitHub is very popular because they drive people to fork projects. With one single click, you can fork your favorite project. You get a copy which is synchronised with the forked project. There is also a wiki (accepting many markup syntaxes). This wiki is also available through Git. At last, GitHub offers a minimalistic ticket system. You open a ticket, you make some comments, you can add a tag or two and you can close it. That s all. Quite light. But enough for small projects like mine. However, there is more! Do not open a ticket to send a patch. There is no way to attach files to tickets. You need to fork the project, create a branch with your patch and request a pull for your branch. This will appear as a ticket but your commits will be linked to it. The author can then review and comment your branch, ask for some changes and if he wants to integrate it, a simple merge is sufficient. He can even do it using the web interface. Convenient. Socially, you can comment commits, tickets, projects and subscribe to a lot of events. You can receive and answers tickets by mail. However, an accound is mandatory to open a ticket. Since there is a lot of projects on GitHub, this account should be useful for a lot of software. To cut a long story short, even if GitHub is a proprietary platform, it offers nice and interesting features. More and more projects migrate from Google Code to GitHub even if there is less features on the latter.

Migration I have searched for tools to migrate automatically tickets from Trac to GitHub. For example, there is SD. At last, I have done it manually. GitHub API is well documented and there exists bindings in various languages including Python but it is a very limited API. You can t choose the number of the ticket nor its date. I have also rewritten README files to use the markdown markup and avoid the use of wiki pages when possible. I have also put some URL redirections with the help of nginx. In conclusion, even if maintaining Trac instances was not a lot of work, one thing that I do not have to do anymore. And no spam on GitHub, I assume.

15 May 2011

Steve Kemp: Images transitioned, and mysql solications.

Images Moved
So I've retired my old picture hosting sub-domain, and moved all the files which were hosted by the dynamic system into a large web-root. This means no more uploads are possible, but each link continues to work. For example: Happily the system generated "random" links, and it was just a matter of moving each uploaded file into that static location, then removing the CGI application. The code for the new site has now been made public, although I suspect there will need to be some email-pong if anybody wishes to use it. Comments welcome.
MySQL replacements?
Lets pretend I work for a company which has dealings with many MySQL users. Lets pretend that, even though it is true, such that I don't have to get into specifics. Let us pretend that we have many many hundreds of users who are very happy with MySQL, but that we have a few users who have "issues". That might be:
  • mysqld segfaulting every few months, with no real idea why.
    • Transactions are involved. So are stored proceedures.
    • MySQL paid support might have been perfect, or it might have lead to "yup, its a bug. good luck rebuilding with this patch. let us know how it turns out kthxbai."
    • Alternatively it might not have been re-producable.
  • Master-Master and Master-Slave setups being "unreliable" such that data inconsistencies arise despite MySQL regarding them as being in sync.
    • Good luck resolving that when you have two almost-identical "mysqldump" outputs which are 6Gb each and which cause "diff" to exit with "out of memory" even on a 64Gb host.
    • Is it possible to view differences in table-data, via the binary records? That'd be a fun project .. for a masochist.
  • Poor usage of resources.
  • Heavy concurrancy caused by poorly developed applications in a load-balanced environment, leading to stalling queries. (Wordpress + Poor wordpress plugins I'm looking at you; you're next on my killfile).
To compound this problem some of these installations may or may not be running Etch. let us pretend they are not, just to simplify things. (They mostly arent' these days, but I'm sure I could think of one or two if I tried) So, in this hypothetical situation what would you recommend? I know there are new forks aplenty of MySQL. Drizzle et al. I suspect most of the forks will be short-lived - lots of this stuff is hard and non-sexy. I suspect the long-lived forks are probably concentrating on edge-cases we've not hit (yet), or on sexy exciting things like new storage engines and going nosql like all the cool kids. Realistically going down the postgresql road is liable to lead to wholly different sets of problems, and a significant re-engineering of several sites, applications and tools with no proof of stability. Without wanting to jump ship entirely, what, if any, are our options?
PS. MySQL I still mostly love you, but my two most recent applications were written to use redis instead. Just a coincidence... I swear. No, put down that axe. Please can't we just talk about it?/p> ObQoote: "I've calculated your chance of survival, but I don't think you'll like it. " - Hitchhikers Film.

Kai Wasserb ch: Chronicle 4.6-2 fixes another UTF-8 encoding issue

Today I was playing with some RDF information, initially triggered by my entry about Josephine. While at it I decided to add some information explicitly to the head tag, including
<meta property="cc:attributionName" content="Kai Wasserb ch" />
<meta property="dc:creator" content="Kai Wasserb ch" />
Obviously I wanted to add those two tags to my templates, so Chronicle adds that to all files. Unfortunately this resulted in an encoding error of the " ". Some digging in the source of chronicle showed, that HTML::Template-new() was called without ensuring, that the tempalte files were parsed as UTF-8 encoded. The solution was to add a filter. The just uploaded version 4.6-2 contains the required patch and if you don't use the Debian packages you won't be left out for long, as Steve already applied the fix upstream.

Christian Perrier: 2011 week 19 Debian work

That was a damn busy week. It was mostly centered about attending SambaXP, the annual Samba user and developers conference, in G ttingen, Germany. The only free software conference I attend with expenses paid by my employer, Onera. This year was the 10th edition and, as last year, to the "who was there for the nth edition" game, I won by staying alone as they asked who attended all editions of the conference. :-) That was a great week, with time spent with people as interesting as Andrew Tridgell, Jeremy Allison, John Terpstra, Volker Lendecke, Kai Blin, to name a few. A good opportunity, again, to get input from our packaging work for that big piece of software, as well as getting visibility about the future of Samba. I also had a great, even if short talk, with the kind Karolin Seeger, the release manager of Samba for 3 years now. We talked about....children, as she's now a mother since last year (with a non negligible impact on her professionnal life, as often in Germany). Great meeting, too, with Brad Kuhn, from the Software Freedom Conservacy, who had a keynote about GPL licence enforcment activities. It becomes more and more sure that Samba3 and Samba4 will reconverge together after the Samba Team releases Samba 3.6. It brings plans for our packaging work: I think we'll stick with having samba 3.6 in wheezy while the brand new shiny Samba4 probably stays separate in some way. Our users (and /me first) clearly need stability in the file and print services first. Of course, I did some packaging work there: samba 3.6.0pre3 was uploaded to experimental, about 10 days after its official announcement. I also worked on the samba *binary* package bugs, triaging them as usual. We now have 51 bugs opened against the samba binary package: 18 unclassified, 11 moreinfo (several likely to be closed as unreproducible or user error), 1 wontfix, 8 with a pending patch and 13 forwarded upstream. I'm also thinking about a possible way to ask about SMB2 support in samba: it won't be activated by default in 3.6 (mostly because us, distros, requested for that and, by "us", I mean Debian, RHEL, SuSe and their derivatives, so quite a large consensus). Still, it would be good to put some light on SMB2 support and a debconf question about it could be a solution (not shown by default and defaulting to no SMB2). I also worked quite extensively on packages maintained by No l K the, Ralf Treinen and me, aka "the pkg-running team". I did setup a git repo for my new "garmin-ant-downloader" package, that allows downloading track files from Garmin Forerunner 405 GPS watches (guess what is the brand and model of mine!). My first packaging git repository! Thanks to Ralf for his advice and help in this. I triaged bugs in the other two packages we maintain: pytrainer (more bugs forwarded upstream) and garmin-forerunner-tools (which was later uploaded by Ralf). I also setup a team mailign list so, now, we're a real team...:-) Few activity on the l10n front: a few Smith reviews are in progress and I completed 1 or 2 French translations, and reviewed some others. Regular activity, then. The only specific stuff is that I'm now pushing harder for the French DDTP effort, doing many reviews and translations there. We try to reach 100% in the "popcon500" packages. Later, we'll try to head at reaching the hieghts reached by the Italian and German teams, who are, on this l10n activity, way ahead from us. Finally, during the SambaXP conference, and as usual (except last year because of too heavy work duties), I visited my German friends, living "close to" G ttingen, accomppanied by Luk Claes and his friend and colleague Ivo, who were also attending SambaXP. Great barbecue at Andreas and Kathrin Tille's place, facing the Wernigerode castle at sunset. And the best sp tzle ever at Meike and Alex Reichle's place in Hildesheim, with a french touch on the salad's dressing as well as great Chilean wine brought by Meike's coworker Wolfram. Always a great time to see these good friends even if that means driving a few hours (and being flashed....twice!...by german speed cameras on the way to and back Andreas place!) To complete the week, I ran a 34km/800m+ trail today in the Rambouillet forest, completing it in 3h31. I'll probably blog separately about running updates as it is now quite some time that I didn't. Guess what? I'll be sleeping well tonight...

Kai Wasserb ch: LWP::Authen::OAuth in NEW

I've uploaded liblwp-authen-oauth-perl to Debian today, [UPDATE: currently the package is waiting in NEW (please note, that the previous link might not work, in case the package has entered the archive already or the page hasn't been generated yet). and is now in the archive. Thanks a lot to the FTP Masters for the quick processing!] LWP::Authen::OAuth allows to easily integrate OAuth authentication into Perl programs and libraries. Some of you might already guess, why I have an interest in OAuth for Perl: as a user (and maintainer) of Chronicle I'd find it neat, if there could be some Flattr integration (especially a automatic submission of new posts to Flattr). This sent me searching for an Perl implementation of the Flattr API. Unfortunately this turned up nothing, therefore I'm now planning to write some sort of Net::Flattr Perl module, which I can then use in Chronicle for one task or another. Net::Flattr (preliminary name, but probably the final one, as the implementation of the Twitter API is called Net::Twitter) is going to be a library, which should offer the same functionality as Flattr's PHP library. For now there are some scetches to this extent on my hard drive, but nothing showable. I hope I find the time soon to write this but it shouldn't be too difficult. ;-) Apart from my particular use case, you can use LWP::Authen::OAuth for any other OAuth-requiring API. I hope the package helps you to build more cool tools for and on your Debian system!

John Goerzen: Napoleon, Winter in May, and Silly String

This is just a diary of a random day sort of post. Nothing all that particularly unusual, but then perhaps if I only blog about the unusual, the typical gets lost. This week started out with temperatures as high at 99F. Today we didn t even reach 60, and tonight will be cold almost to freezing. It s Kansas, so while we remark on the strange weather, we re not particularly surprised by it. This morning, Jacob and Oliver went over to play across the street. Jacob then went with Terah to a birthday party for one of his classmates, while Oliver helped plant some trees with his grandparents. I was at home this morning, which gave me a small but welcome break from the past week: we had something going on 6 out of the last 7 evenings, and little time to do anything else. So I got out the Kindle and read from War and Peace. I started at the kitchen table while eating some granola, and then sat on the couch. The day was cold, windy, and overcast, and I like to use natural light, so there were only a few places to sit with enough light. I sat there reading the account of Napoleon s capture of Moscow in 1812, the characters lives uprooted by those events, and stories of violent mobs forming. And, of course, Tolstoy s essays on the mistake of overstating the importance of famous figures in history were common. I am now about 3/4 through War and Peace, which means I ve read 1,044 pages and have (only) 348 to go. As I got up to try to find slippers or a blanket for my cold feet at one point, I remembered a conversation I had while eating lunch this week. I was reading on the Kindle while reading, and a person I knew happened to walk past. She asked what I was reading. I said simply, War and Peace. There was this stunned silence, so I added, for fun. It s great! Which is true. She said, Oooookay, John, you just enjoy that then and went on her way. This isn t the first time I ve had a conversation like that about this book. It seems to be held up as some example of long, dry, literature that people are made to read but don t want to. I think that only long is accurate here. Anyhow, pretty soon Oliver arrived and, much to my surprise, said Doh! (meaning no ) when I asked him if he was ready for lunch. Instead he stood there looking at me sweetly. Taking a guess, I said, Oliver, would you like to watch train videos? Yeah yeah yeah yeah! So we did. He particularly enjoyed the steam engines today, and eventually was ready for lunch at 12:30. Jacob and Terah got home just as he was finishing. Oliver got his nap, and I played with Jacob for a bit. Ever since he had heard of the British wedding in preschool, he s been interested in weddings, so I showed him pictures from Terah and me at our wedding. This was a big hit. Then he went with Terah to decorate a Sunday School room at church, so I read a bit more Tolstoy. Tolstoy s characters sometimes are prone to not just changes in mood but changes in outlook and worldview changing from cold, logical, and uncaring to warm and loving, or from despair at the vanity and shallowness of humanity to elation at the simple honesty of a peasant and renewed trust in the human spirit. At first I marveled at this tendency in certain characters, but on reflection, and in consideration of the timeline of the novel, which spans several years, it didn t seem quite so strange. Things can happen to change the mood of individuals, or even a country; I can remember the national mood changing after 9/11, and again after Obama s election, for instance. I also felt a contrast with some of the characters who, due to their decisions or circumstances, lost things that mattered while chasing things that weren t as important. I felt fortunate to be at this place, with Jacob, Oliver, and Terah, and decided that we all ought to do something fun in the evening, but hadn t quite figured out what. And about that, the clouds started to break and Oliver woke up from his nap. I went up to his room, and much to his delight, played peekaboo with him for quite awhile. Eventually he was done with that game, but spotted my phone, so I helped guide his finger to launch Youtube and watch a train video over the wifi. This was certainly a Big Deal to him. After Terah and Jacob got back, I figured it was time figure out something fun to do. And what could be more fun, I decided on the spur of the moment, than a drive into town to pick up some cans of Silly String for the boys, neither of which had ever used it before. So we went to a restaurant, then bought the surprise toy (I hadn t told them what it would be beforehand), then went to an ice cream place, and finally back home. Oliver loved it. He needed my help to press the nozzle to make it come out, but still had a great time and shrieked with delight. Jacob then tried out his can, and stood in one place spraying his bright green strings all over the place. Then he took to running around the yard spraying string randomly in every direction until he finally completely emptied out the can. And after that, he gathered up some of the different colors of string and rolled them up into a ball which he was very proud of. A nice day.

Next.

Previous.