Search Results: "cas"

22 May 2010

Rog rio Brito: Changing Some Configuration from Defaults

Two of my most frequently used tools are, hands down, Emacs and bash. Having a pleasant configuration of one s toolbox is necessary for long-term productivity. However, newer versions of the programs (any programs) don t always have good, comfortable defaults. That is the case with Emacs 23 and bash 4. With Emacs 23, by default, the scratch buffer shows some text that, if you want to quickly enter text to be saved, has to be deleted. This can be tolerated for some invocations of the editor. But at some point, it gets on one s nerves. Luckily, Emacs is so fully customizable that this is not a problem, unless you have to hunt down the right knobs and you are an e-lisp illiterate. That is my case. After reading an admittedly small amount of documentation, some good lines that ought to be put into one s .emacs file are:
(setq inhibit-startup-echo-area-message "rbrito")
(setq initial-scratch-message nil); show an empty scratch buffer
(setq inhibit-startup-message t); remove splash screen
Another line that may make things more comfortable (especially if you, like me, use Emacs s dired as a filemanager to navigate your folders):
(defalias 'yes-or-no-p 'y-or-n-p)
But configuring Emacs is not always that hard. Seeing a change of behaviour with your shell is scarier. When bash 4 was released, one thing that got implemented is the echo of some characters from the terminal to the screen. In practice and with my usage pattern, this meant that whenever I was typing a command line, changed my mind and pressed C-c, I got a ^C printed on the screen, usually overwriting one or two characters of what I had typed. And this prevented me from automatically copying and pasting the command that I had typed. Looking for an answer to that was much harder than to see how to change a parameter on Emacs: every search that I tried on Google with the words bash and Ctrl-C or some variants of that eventually lead me to how do I trap a Ctrl-C on my shell script? . Decided to see what changed, I read the documentation of readline. It didn t help, because I was reading something from version 5. I, then, downloaded the source code and started looking for SIGINT. Then, perusing the code (and the changelog), I saw that with bash 4 and readline 6, a new configuration option was introduced: the echo-control-characters option of readline. Of course, after knowing this, stopping the undesired behaviour was a simple matter of inserting set echo-control-characters off in my .inputrc and think about how much time I spent with that uncomfortable behaviour in some of my preferred tools. As an addendum to the previous paragraphs (written on February of this year, but only now published), the Debian package of Emacs 23.2, the newest version as of this writing, comes with some configuration that makes it ignore the settings that are used with fontconfig. I did not spend any time trying to see the cause of the problem, but adding a simple
Emacs*font:			Mono
to my .Xdefaults file did the trick and restored back the ability to use DejaVu Sans Mono at the right size on my system.

Russell Coker: A School IP Project

The music industry seems fairly aggressive in taking legal action against children when they break the licence terms of copyright material. I think it would be good to teach children about how the IP industry really works. It seems to me that you could have a school project that involves an entire year level (maybe 100 students depending on the size of the school) each of whom can produce copyright material (everything they do in art and English classes would be suitable as a start). Then they could register their work (make digital photographs and then store them in a school database that records the entry date) and sue anyone who infringes their work. Every student would receive licence fees for their work, but if they are sued for infringement they would have to pay all revenue plus damages. Other students could work as lawyers and take a portion of the proceeds of any successful law suit, and finally some students could run recording companies and spend their time hunting for infringing work for the purpose of launching legal action. In terms of the licence fees paid, this could be done by just allocating a fixed value per item to each student as a way to get the system running without regard to the fact that some students just aren t able to create good art. It could however have a large portion of the value coming from what other students choose to spend, every student gets to spend $10 per week on art and they can choose from the database what they want to buy copies of. The most popular art could then be printed on every notice-board in the school as an incentive for students to vote with their play-money for something that they don t mind seeing all the time. It s obvious that popularity would be a significant factor in the success of some artists, but that s OK, a casual review of the chart topping music reveals that it s quite obviously not created by the world s best musicians so it seems that rewarding popularity rather than skill just adds some realism. One possibility would be to allow the students to elect representatives to create their own IP laws. It would be interesting to see how the IP laws voted on by representatives of the students (who are all in some way involved with the process of creating, buying, selling, and distributing artistic products) differ from those which we have foisted upon us in the real-world. Also an interesting possibility would be to allow corruption in the election process and observe how the results differ from year levels where corruption is not permitted. I expect that teaching children how political corruption works would be a little controversial, but it s nothing that they can t learn from reading news reports about what the entertainment industry is really doing. Really being a corrupt politician for a school project shouldn t be as bad as playing a murderer in a school play! Naturally this couldn t be done with real money, but giving higher marks at the end of the year to the students who accumulate the most play money would be quite reasonable. I don t think that there would be a problem with giving higher marks to a student who succeeded through political corruption as long as they gave a good written report of how they did so and the implications for society. Please note that I am not suggesting this for a subject that is used for university entrance, I think it would be a good project for years 8-10 which in Australia have no relevance to university entrance. So the marks would just be letters on a bit of paper that might make parents happy or unhappy and otherwise mean nothing. I anticipate responses from people who believe that educating children about how the world works is not appropriate for a school. Such people are never going to convince me, but if anyone thinks that they can make a good point to convince some of the readers then I encourage them to write it up in the comments section if it s short or on their own blog if it s longer.

20 May 2010

Aigars Mahinovs: So, what is Microsoft Azure and how it compares to Amazon s AWS

We had a Microsoft salesperson stop at our offices today to tell people about wonders of cloud computing and Microsoft s Azure will save us all. The following is a short and non-exhaustive list of what Azure does not do according to their own experts: The Microsoft presenter also repeatedly engaged in shady tactics when presenting his information: So to summarize, Microsoft has build a cloud of Windows-like machines which can run your programs (as long as they are in .Net or Silverlight) and which does not have even a third of the features of such de-facto market leader as Amazon AWS. They try giving away their product for free and only get 14k applications in 6+ months. Note: Amazon has at least 300 000 public applications and God only knows how many more private ones. They will grow, due to companies locked-into Microsoft s way of thinking buying into their marketing, but if you have a head on your shoulders, you should avoid it as a trap, which it actually is. All that can be done on Azure, can also be done on Amazon. The opposite is not always true. In addition, if you will want have a mixed environment with both Windows and Linux servers, you will not be able to deploy Linux servers to Azure (or they will be very slow) and you ll have to put your Linux servers into Amazon (or another) cloud and pay double traffic fees for every byte of traffic between your own servers. Microsoft Azure prices are exactly the same as Amazon EC2 prices with Windows instances. However on Amazon you actually get fully capable Windows Server instances. And if you don t need Windows, you can get a much better price on Amazon. The only companies that can safely use Azure are companies that only have Windows servers AND plan to only have Windows servers for the foreseeable future. Knowing how dominant Linux is in the server market and how many Windows-only shops actually have a few Linux servers in the back room, there is very little future for Azure, until they wise up and make Linux instances first class citizens in their cloud. If not because their customers need them, then because their customers think that they at least might need them in the future. Update: Amazon now released a Reduced Redundancy storage for S3 that provides the exacts same level of storage guarantees that regular Microsoft Azure SLA (99.9%) for two thirds of the price, and also Amazons now explicitly says that their regular S3 is Designed for 99.999999999% Durability , which is worlds above what Microsoft provides. Now in this case Amazon again is cheaper than MS. Also Amazon bandwidth prices are 33% lower compared to Azure. In Azia the difference is simply staggering (0.45$ vs 0.19$ per Gb).

19 May 2010

Dominique Dumont: Improve config upgrade Ep 02 Minimal model for OpenSsh s sshd_config

Hello Currently Config::Model can upgrade /etc/ssh/sshd_config file because Sshd model is complete. All 80 parameters mentioned in sshd_config man pages are described there. The whole model (including doc is about 1000 lines long). If one just want configuration upgrade capability, requiring a complete model is too much work. The aim of this GSoC project is to find a way to provide upgrades at a reasonable cost. Ep 01 focused on what feature was missing from Config::Model to provide a model that could handle unknown parameters in most cases to enable upgrades without a complete model. This covered the use case of an unmaintained model facing software evolution. Now, let s go on with the OpenSsh example and let s see what would be the minimal model that would enable upgrades from older sshd_config version to a more recent one. (Note that this blog is based on upgrade requirements mentioned in sshd_config man page). This will cover the use case of a maintained model targeted for upgrades. From last blog, we know that this minimal model must feature: Let s tackle the structure of this minimal model by re-using the current model structure. We will need 2 classes:
  1. Sshd: to handle all leaf parameters a the the Match parameter
  2. Sshd::MatchElement: to handle parameters within a Match block
Sshd::MatchElement will need to handle unknown leaf parameters (exceptd Match parameters), so its declaration should be like: 'Sshd::MatchElement' =>
accept =>
match_not => '/Match/i'' ,
type => 'leaf',
value_type => 'string'


Note that there s no element declaration. The accept structure means that Config::Model will accept any parameter that does not contain match in its name and will handle it as a string parameter. (Note that this feature is not yet implemented in Config::Model) Now, we must declare the main class: Sshd. It will feature the a wider accept statement and element declarations for the following parameters: Here we go: Any unknown parameter will be accepted by Sshd configuration class: accept =>
type => 'leaf',
value_type => 'string'
,
Here are the element that require a special treatment: Match just refer to the class declared above
Match =>
type => 'hash',
index_type => 'uniline',
cargo =>
type => 'node',
config_class_name => 'Sshd::MatchElement'
,
KeepAlive is the obsolete parameter. Parameter with status 'deprecated' are handled by Config::Model but will not be written back in the configuration file. Hence re-using the deprecated information elsewhere is more than recommended.
KeepAlive =>
type => 'leaf',
value_type => 'enum',
status => 'deprecated',
choice => [ 'no', 'yes' ]
,
Here's the new parameter. It's value is computed from KeepAlive.
TCPKeepAlive =>
type => 'leaf',
value_type => 'enum',
choice => [ 'no', 'yes' ],
migrate_from =>
formula => '$keep_alive',
variables => keep_alive => '- KeepAlive'
,
,
Now that we have the model, we must figure out how to read and write value with minimal coding efforts. Due to the mismatch between the syntax of the file (flat structure) and the actual structure (special Match parameter), a dumb parser/writer cannot be used. A parser that takes into account Match is required. Either dedicated Perl code (like the parser/writer delivered with Config::Model::OpenSsh) or use Augeas and its sshd lenses. In summary, we ve seen that the model required to perform upgrades is about 30 lines instead of the 1000 lines required for a complete model. On the other hand, the syntax of sshd_config makes the parser and writer harder to write. But, normally, this parser/writer should not evolve during the life of OpenSsh project. Now the question that I cannot answer is: Can a packager handle the extra work to maintain this minimal model ? Feedback is welcome here ;-) Next episode will study how upgrades based on Config::Model can be applied on files with other syntaxes. All the best

Russell Coker: Google Chrome and SE Linux

Google Chrome saying aw snap when it crashes [107108.433300] chrome[12262]: segfault at bbadbeef ip 0000000000fbea18 sp 00007fffcf348100 error 6 in chrome[400000+27ad000] When I first tried running the Google Chrome web browser [1] on SE Linux it recursively displayed the error message in the above picture, it first displayed the error and then displayed another error while trying to display a web page to describe the error. The kernel message log included messages such as the above message, it seems that some pointers are initialised to the value 0xbbadbeef to make debugging easier and more amusing. V8 error: V8 is no longer usable (v8::V8::AddMessageListener()) When I ran Chrome from the command-line it gave the above error message (which was presumably somewhere in the 8MB ~/.xsession-errors file generated from a few hours of running a KDE4 session). type=AVC msg=audit(1274070733.648:145): avc: denied execmem for pid=12833 comm= chrome scontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=process
type=SYSCALL msg=audit(1274070733.648:145): arch=c000003e syscall=9 success=no exit=-131938567594024 a0=7fd863b41000 a1=40000 a2=7 a3=32 items=0 ppid=1 pid=12833 auid=4294967295 uid=1001 gid=1001 euid=1001 suid=1001 fsuid=1001 egid=1001 sgid=1001 fsgid=1001 tty=pts4 ses=4294967295 comm= chrome exe= /opt/google/chrome/chrome subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null)
type=ANOM_ABEND msg=audit(1274070733.648:146): auid=4294967295 uid=1001 gid=1001 ses=4294967295 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 pid=12833 comm= chrome sig=11 V8 is the Google Javascript system which compiles JavaScript code and thus apparently needs read/write/execute access to memory [2]. In /var/log/audit/audit.log I saw the above messages (which would have been in the kernel message log as displayed by dmesg if I didn t have auditd running). The most relevant parts are that execmem access was requested and that it was by system call 9. From linux/x86_64/syscallent.h in the strace source I discovered that system call 9 on the AMD64 architecture is sys_mmap. Does anyone know a good way to discover which system call has a given number on a particular architecture without reading strace source code? Attempts to strace the Google Chrome process failed, Chrome gave the error Failed to move to new PID namespace after clone() failed. Clone was passed the flag 0 20000000 which according to /usr/include/linux/sched.h is CLONE_NEWPID. It seems that programs which create a new PID namespace (as Google Chrome does) can t be straced as the clone() call fails. It s a pity that Chrome doesn t have an option to run without using this feature, losing the ability to strace it really decreases my ability to find and report bugs in the program I m sure that the Google developers want people like me to be able to help them find bugs in their code without undue effort. Anyway the solution to this problem to allow it to run on the SE Linux Targeted configuration is to run the command chcon -t unconfined_execmem_exec_t /opt/google/chrome/chrome which causes the Chrome browser to run in the domain unconfined_execmem_t which is allowed to do such things. Of course we don t want Chrome processes to run unconfined, I think that the idea I had in 2008 for running Chrome processes in different SE Linux contexts is viable and should be implemented [3]. As a general rule if you are running a program from the command-line on SE Linux with the Targeted configuration (the default and most common configuration) then any time you see an execmem failure logged to the kernel message log or the audit subsystem then you can change the context of the program to unconfined_execmem_exec_t to make the problem go away. Note that this isn t necessarily a good thing to do, sometimes it s best to change the program to not require such access. But it seems that in this case the design of V8 requires write/execute memory access to pre-compile JavaScript code.

18 May 2010

Sam Hartman: Moonshot Bar BOF Thursday March 20 at 9 PM; specs available

At IETF 77, we re having a get together to discuss federated authentication beyond the web. The meeting will be in the Mahattan room starting at 9 PM US Pacific time. I think audio streaming will be available; I will post a link closer to the meeting time. In the last entry, I mentioned that a preliminary spec would be available; see the preliminary EAP GSS-API mechanism. A use case paper and slide set are being reviewed internally and should be ready early next week. We may even have preliminary versions of the binding between RADIUS and SAML available before IETF. There have been a number of great discussions on the moonshot-community list and with others interested in the broader area.

Sam Hartman: Two SASL mechanisms for Federated Authentication

There are two other approaches that are likely to come up tonight; see this message for details. These mechanisms require significantly lower infrastructure than Moonshot, but do not provide all the benefits. One question is whether there is a continuum of use-cases depending on what level of investment in client changes are made.

Sam Hartman: Internet2 Moonshot Briefing Paper

Please see here for a briefing paper including snapshots of all our specs as well as an updated use case paper. This paper was presented at the end of April at the Internet2 Spring Members meeting. This is a great snapshot of Project Moonshot at the end of last month.

Andrew Pollock: [life] Zoe, 2 weeks later

The fact that it's only two weeks feels totally surreal. I think it's the feeding approximately every 3 hours. That's approximately 100 interactions that we've had with her in only 2 weeks. We had her two week weight check at the paediatrician's today. She's now weighing 6 lbs 12 oz (3061 grams), so she's made back her birth weight and then some. We're still largely bottle feeding her with pumped breast milk. The paediatrician said that once she's 7 pounds she should have enough strength to breastfeed exclusively, so we'll get a lactation consultant involved if that isn't the case in another week or so. Other than that, everything's going swimmingly. She eats, sleeps and poops. She sleeps amazingly well. I keep telling myself it won't last, but we're making the most of not needing to tip-toe around the house. In the last week, we had a new stove installed, which involved the contractor using a hammer drill to install the anti-tip brackets. He also used a drop-saw to modify one of the drawers. Didn't bother her. We had some shutters installed in the kitchen, which involved a lot of cordless drilling. Also didn't bother her. The gardeners for the complex were mowing the lawn and using a leaf blower right outside her window. Not a peep. Today, after the paediatrician's appointment, we dropped in to Palo Alto Animal Services, the animal shelter where Sarah has volunteered for the last 4.5 years, to show them Zoe. They threw a bit of a surprise baby shower, and gave us a ton of baby clothes, which was very nice and totally unexpected. Zoe slept in her car seat on the table in the lunch room while a good dozen people or so made a lot of noise around her, and she was oblivious to it all. Sarah's Mum arrives on Wednesday. Tomorrow will be a busy day of final cleaning up in preparation.

17 May 2010

Dirk Eddelbuettel: Rcpp 0.8.0

Romain and I are happy to announce the release of Rcpp version 0.8.0. It has been uploaded to CRAN. A Debian upload is delayed until the now-required inline package is accepted into Debian. The source package is also available from here. This release brings a number of changes that are detailed below. Of particular interest may be the much more robust treatment of exceptions, the new classes for data frames and formulae, and the availability of the new helper function cppfunction for use with inline. Also of note is the new support for the 'LinkingTo' directive with which packages using Rcpp will get automatic access to the header files. An announcement email went to the r-packages list (ETH Zuerich, Gmane); Romain also blogged about the release. The full NEWS entry for this release follows below:
0.8.0   2010-05-17
    o   All Rcpp headers have been moved to the inst/include directory,
	allowing use of 'LinkingTo: Rcpp'. But the Makevars and Makevars.win
	are still needed to link against the user library.
    o   Automatic exception forwarding has been withdrawn because of
	portability issues (as it did not work on the Windows platform).
	Exception forwarding is still possible but is now based on explicit
	code of the form:
	
	  try  
	    // user code
	    catch( std::exception& __ex__)  
	    forward_exception_to_r( __ex___ ) ;
	   
	
	Alternatively, the macro BEGIN_RCPP and END_RCPP can use used to enclose
	code so that it captures exceptions and forward them to R. 
	
	  BEGIN_RCPP
	  // user code
	  END_RCPP
	
    o   new __experimental__ macros 
	
	The macros RCPP_FUNCTION_0, ..., RCPP_FUNCTION_65 to help creating C++
	functions hiding some code repetition: 
	
	  RCPP_FUNCTION_2( int, foobar, int x, int y) 
	    return x + y ;
	   
	The first argument is the output type, the second argument is the 
	name of the function, and the other arguments are arguments of the C++ 
	function. Behind the scenes, the RCPP_FUNCTION_2 macro creates 
	an intermediate function compatible with the .Call interface and handles 
	exceptions
	
	Similarly, the macros RCPP_FUNCTION_VOID_0, ..., RCPP_FUNCTION_VOID_65 
	can be used when the C++ function to create returns void. The generated
	R function will return R_NilValue in this case.
	
	  RCPP_FUNCTION_VOID_2( foobar, std::string foo ) 
	    // do something with foo
	   
	  
	The macro RCPP_XP_FIELD_GET generates a .Call compatible function that
	can be used to access the value of a field of a class handled by an 
	external pointer. For example with a class like this: 
	
	  class Foo 
		public:
			int bar ;
	   
	
	  RCPP_XP_FIELD_GET( Foo_bar_get, Foo, bar ) ;
	  
	RCPP_XP_FIELD_GET will generate the .Call compatible function called
	Foo_bar_get that can be used to retrieved the value of bar.
	
	
	The macro RCPP_FIELD_SET generates a .Call compatible function that 
	can be used to set the value of a field. For example:
	
	  RCPP_XP_FIELD_SET( Foo_bar_set, Foo, bar ) ;
	
	generates the .Call compatible function called "Foo_bar_set" that 
	can be used to set the value of bar
	
	
	The macro RCPP_XP_FIELD generates both getter and setter. For example
	
	  RCPP_XP_FIELD( Foo_bar, Foo, bar )
	  
	generates the .Call compatible Foo_bar_get and Foo_bar_set using the 
	macros RCPP_XP_FIELD_GET and RCPP_XP_FIELD_SET previously described
	
	  
	The macros RCPP_XP_METHOD_0, ..., RCPP_XP_METHOD_65 faciliate 
	calling a method of an object that is stored in an external pointer. For 
	example: 
	
	  RCPP_XP_METHOD_0( foobar, std::vector<int> , size )
	
	creates the .Call compatible function called foobar that calls the
	size method of the std::vector<int> class. This uses the Rcpp::XPtr<
	std::vector<int> > class.
	
	The macros RCPP_XP_METHOD_CAST_0, ... is similar but the result of
	the method called is first passed to another function before being
	wrapped to a SEXP.  For example, if one wanted the result as a double
	
	   RCPP_XP_METHOD_CAST_0( foobar, std::vector<int> , size, double )
	
	The macros RCPP_XP_METHOD_VOID_0, ... are used when calling the
	method is only used for its side effect.
	
	  RCPP_XP_METHOD_VOID_1( foobar, std::vector<int>, push_back ) 
	
	Assuming xp is an external pointer to a std::vector<int>, this could
	be called like this :
	
	  .Call( "foobar", xp, 2L )
	
    o	Rcpp now depends on inline (>= 0.3.4)
	
    o   A new R function "cppfunction" was added which invokes cfunction from
	inline with focus on Rcpp usage (enforcing .Call, adding the Rcpp
	namespace, set up exception forwarding). cppfunction uses BEGIN_RCPP
	and END_RCPP macros to enclose the user code
    o	new class Rcpp::Formula to help building formulae in C++
    
    o   new class Rcpp::DataFrame to help building data frames in C++
	
    o   Rcpp.package.skeleton gains an argument "example_code" and can now be
	used with an empty list, so that only the skeleton is generated. It
	has also been reworked to show how to use LinkingTo: Rcpp
	
    o   wrap now supports containers of the following types: long, long double,
	unsigned long, short and unsigned short which are silently converted
	to the most acceptable R type.
	
    o	Revert to not double-quote protecting the path on Windows as this
	breaks backticks expansion used n Makevars.win etc
        
    o   Exceptions classes have been moved out of Rcpp classes,
	e.g. Rcpp::RObject::not_a_matrix is now Rcpp::not_a_matrix
As always, even fuller details are in Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

Kees Cook: yay for barriers

I find it surreal to have people guessing at my motivations when they could just ask me. On top of all that, I find it weird that people spend so much time with in-fighting. I just want my system not to suck. Some time ago (during in the Ubuntu Karmic development cycle maybe in September 2009), I started having giant problems with my build system. All I/O would start to stall, wait times would surge, and usually my entire system would just go unresponsive with the disk light on solid. This scared the crap out of me, and it wasn t entirely obvious what was triggering it. No one else seemed to be seeing it. I managed to start tracking things using latencytop , and saw stuff like liferea going crazy. As I eliminated more and more things, I eventually settled on it being a problem with umount, and I reported an Ubuntu bug. It seemed to look like an upstream bug that no one else but the reporter could reproduce either. Since no one else was seeing this issue, and it seemed related to LVM snapshots, I migrated off of snapshots, and started using aufs overlays for my builds. For a while, it seemed like things went away. It didn t, and I started hitting it again. I opened a new (now famous) bug in Ubuntu, since now snapshots weren t in the picture, and I didn t want to confuse the earlier history. I managed to find a relatively minimal test-case too. A few other people commenting on the bug were seeing the problem now too, but it was less pronounced for them. As an aside, this wasn t a just wait a few seconds longer kind of issue on my system. A single umount would last 30-40 minutes. And when I m doing parallel builds of security updates, this would turn into my system being unavailable for hours at a stretch. Since none of the kernel developers I was in contact with were able to track down the root cause, I asked Ted Ts o in email if he could just quickly peek in on this for me, since I figured he d be in a good position to confirm or deny it. I didn t want to start wasting upstream time with this if it wasn t reproducible (see earlier upstream kernel bug). To my great relief, Ted found a few minutes to check it, and was able to immediately confirm it and give me a viable work-around ( sync; umount ) for the time being. I confirmed the work-around, and went off to do other things. A while later, Ted came back to deliver a bit of a rant, the purpose of which was not clear to me, but I ultimately ignored it I didn t seem directed at me. I just wanted my system operating normally, and he d done me a favor to check in on it and got me a work-around. More time passes, and I eventually get caught in another I/O-wait melt-down. On investigation, it seemed that the in-kernel work-around in the Ubuntu kernel totally back-fired on me in some cases, rendering even the user-space work-around useless. After investigating the Ubuntu-specific work-around, I re-read Ted s rant in the course of researching what had happened during this bug s triage. It seemed that Ted was basically saying:
this is an upstream problem
RedHat hasn t run into it and he didn t know why I figured I should confirm for myself if Fedora was affected, so downloaded and installed Fedora to double-check there. Since I was able to reproduce it there, I opened an upstream bug, linking back to the original Ubuntu bug, and then went to open a bug in the Fedora tracker, linking back to upstream. And it seems to be these actions that everyone has jumped on. I will now bore you with the reality of my motivations: I wanted to fix the bug so no one would end up experiencing the same pain I d been through over the last 6 months. The bug was, from my perspective, a serious issue. Since I d managed to reproduce it in another distro, it was my duty as a Free Software developer to report it to them. And, in what I felt was an unambiguous gesture, I made sure to include the link to the upstream kernel bug. Reproducing it in Ubuntu, in Fedora, and with a stock kernel had me confident that it was an upstream issue. While Ted did correctly suspect the issue was upstream, I really didn t want to just open an upstream bug and have it be ignored. I wanted some additional proof of reproduction, which I got when I tested it on Fedora. So, I m rather saddened that so many people spent so much time questioning my motivations, making fun of Canonical, or doing anything other than trying to just simply solve this problem. I m totally disinterested in inter-distro fighting. Instead, I continue to assume we re all on the same team, fighting a philosophical battle against close-source software. And in that regard, I think it s still true. If I ignore the rants and jeering, I come away thankful for all the people that spent time trying reproduce the issue at Canonical, at RedHat, and in the larger community. I m hugely thankful that Ted made some time to let me know I wasn t crazy, and there was actually a problem. I m thankful for having some work-arounds, and I m thankful that the root cause was eventually ferreted out, with some possible solutions. I m even thankful that some people on the LWN thread saw that, far from malicious, I was trying to be helpful with the bug. I just wanted my filesystem not to eat my computer. And I was hoping other people could maybe help me, since I m not a filesystem expert. The drama around this bug is pathetic, and now by talking about it for almost 1000 words, I m just as guilty.

Russell Coker: Timing Processes

One thing that happens periodically is that I start a process from an interactive shell, discover that it takes longer than expected, and then want to know how long it took. Basically it s a retrospective need to have run time whatever that I discover after the process has been running for long enough that I don t want to restart it. My current procedure in such situations is to run ps from another session to discover when it started and then type date to display when it ends. A quick test with strace showed that bash uses the wait4() system call to determine when a process ends, but passes NULL as the last parameter. If it passed the pointer to a struct rusage then it would have the necessary data. I think it would be a really good feature for a shell to allow you to type something like echo $TIME_LAST_CMD to see how long the last command took. For the common case where you aren t interested in that data it would only involve an extra parameter to the wait4() system call, a small amount of memory allocated for it, and to store yet another environment variable in it s list. A quick Google search didn t show any way of filing wishlist bugs against Bash and I don t think that this is a real bug as such so I haven t filed a bug report. If anyone reads my blog and has some contact with the Bash people then please pass this idea along if you think it s worthy of being included.

Matt Zimmerman: Ubuntu 10.10 (Maverick) Developer Summit

I spent last week at the Ubuntu Developer Summit in Belgium, where we kicked off the 10.10 development cycle. Due to our time-boxed release cycle, not everything discussed here will necessarily appear in Ubuntu 10.10, but this should provide a reasonable overview of the direction we re taking. Presentations While most of our time at UDS is spent in small group meetings to discuss specific topics, there are also a handful of presentations to convey key information and stimulate creative thinking. A few of the more memorable ones for me were: The talks were all recorded, though they may not all be online yet. Foundations The Foundations team provides essential infrastructure, tools, packages and processes which are central to the development of all Ubuntu products. They make it possible for the desktop and server teams to focus on their areas of expertise, building on a common base system and development procedures. Highlights from their track: Desktop The desktop team manages both Desktop Edition and Netbook Edition, on a mission to provide a top-notch experience to users across a range of client computing devices. Highlights from their track: Server/Cloud The server team is charging ahead with making Ubuntu the premier server OS for cloud computing environments. Highlights from their track: ARM Kiko Reis gave a talk introducing ARM and the corresponding opportunity for Ubuntu. The ARM team ran a full track during the week on all aspects of their work, from the technical details of the kernel and toolchain, to the assembly of a complete port of Netbook Edition 10.10 for several ARM platforms. Kernel The kernel team provided essential input and support for the above efforts, and also held their own track where they selected 2.6.35 as their target version, agreed on a variety of changes to the Ubuntu kernel configuration, and created a plan for providing backports of newer kernels to LTS releases to support deployment on newer hardware. Security Like the kernel team, the security team provided valuable input into the technical plans being developed by other teams, and also organized a security track to tackle some key security topics such as clarifying the duration of maintenance for various collections of packages, and the ongoing development of AppArmor and Ubuntu s AppArmor profiles. QA The QA team focuses on testing, test automation and bug management throughout the project. While quality is everyone s responsibility, the QA team helps to coordinate these activities across different teams, establish common standards, and maintain shared infrastructure and tools. Highlights from their track include: Design The design team organized a track at UDS for the first time this cycle, and team manager Ivanka Majic gave a presentation to help explain its purpose and scope. Toward the end of the week, I joined in a round table discussion about some of the challenges faced by the team in engaging with the Ubuntu community and building support for their work. This is a landmark effort in mating professional design with free software community, and there is still much to learn about how to do this well. Community The community track discussed the usual line-up of events, outreach and advocacy programs, organizational tools, and governance housekeeping for the 10.10 cycle, as well as goals for improving the translation of Ubuntu and related resources into many languages. One notable project is an initiative to aggressively review patches submitted to the bug tracker, to show our appreciation for these contributions by processing them more quickly and completely.

Michael Casadevall: Mailing Lists and Newsreaders ....

So for awhile, I've been questioning on why very few FOSS programs use newsgroups (or USENET on moderated groups) as their primary means of communication between developers. I personally find email to be a clunky way dealing with mass-mailings, and while switching to alpine and mutt have helped, I feel newsgroups would work better in place of mailing lists.

In this spirit, I'm going to try to replace reading most of my public facing mailing lists with Gmane and see if my opinion actually holds true; if newsgroups are superior to bog-standard mailing lists, and post back in some time. I'm currently looking at various GUI and console based newsreaders to see which works best to meet my needs.

I also need to blow the dust off the memories and remember how to setup leafnode so I can mimic OfflineIMAP. Any suggestions are welcome.

16 May 2010

Russell Coker: Noise Canceling Headphones and People Talking

The Problem I was asked for advice on buying headphones to protect students who have medical conditions that make them sensitive to noise, such headphones would have to allow them to hear human voices. Due to the significant differences in hearing issues (including physical damage and sensory issues) it seems unlikely that getting identical headphones for all students will give an ideal result. The person who asked me the question didn t explain what type of students are being taught. If it s an adult education class then getting everyone the ideal headset wouldn t be particularly difficult. If however it s the special needs class in a high school then students would probably want the most shiny headphones rather than the ones that are a best match to their hearing issues. Also some combinations of hearing problems and ambient noise can t be addressed by such headsets. A friend who developed Noise Induced Hearing Loss from shooting tells me that he really can t stand brass instruments. But the high frequencies from such instruments tend not to be filtered well by noise canceling headphones, so any student who has such a problem would probably need hearing aids that filter out high frequencies I believe that such hearing aids are available but don t have any particular knowledge about them. Test Results I did a quick test on my Bose QC-15 noise canceling headphones [1] which cost me $320US including tax and my cheap Bauhn headphones from Aldi [2] which cost me $69AU (and which apparently could later be purchased on special for $35AU to clear stock). I found that when not playing music they seemed to perform about equally well in terms of allowing me to hear people speaking, although I admit that just having a conversation with the nearest people wasn t the most scientific test. When I was playing music I found that the Bose headset made it significantly more difficult to hear people speak than the Bauhn headset. This is an advantage for the Bose for it s intended use, and I expect that students who need a headset for medical reasons won t want to listen to music while studying so it s never a disadvantage. In both cases, if the headphones are used for just canceling unwanted noise the speaker shouldn t need to raise their voice significantly to be heard. In some situations the noise canceling headphones make it easier for someone with good hearing to hear what people are saying, for example a conversation in a car or plane could probably be held at a lower volume if all people involved were wearing suitable noise canceling headphones. If however the students have damaged hearing then I can t make any prediction as to whether the teacher could speak at a lower volume or whether they would be required to use a higher volume if the students wore such headphones. The Brookstone on-ear headphones that I tested [3] seem particularly noteworthy in this regard due to the way they canceled the melody of the store background music and just left the singing. If someone wants to buy headphones for people with physical damage to their ears then the Brookstone product is really worth investigating. If however the target market happens to be people on the Autism Spectrum then they may hate anything that presses on their ears (as I do) in which case the Brookstone product can t be considered. The Brookstone price of $150US (presumably $160 including tax) was also the best price I saw when shopping in the US but I presume that I could have found something with a similar quality and price to Bauhn in the US if I looked hard enough. Conclusion The big advantage of the Bose for this use is that it blocks a wider range of frequencies than some other noise canceling headsets. They all work really well on regular low frequency noise such as car engine noise (whether a car passenger or a pedestrian) but to stop certain higher frequencies such as those from air conditioning systems the Bose wins hands down. I guess this may depend on what noise is to be blocked, if a class was held in the same room every time and noise canceling headsets were purchased specifically for that class then it would probably make sense to ensure that the acoustic capabilities of the headsets match the unwanted background noise and the hearing issue that each student has. Here s an Amazon link: Bose QuietComfort 15 Acoustic Noise Cancelling Headphones I ve been reading about Sensory Processing Disorder. I m sure that some children are doing poorly in the default school system because they either have an undiagnosed case of SPD or who don t have enough symptoms to get a diagnosis. I think it would make a good experiment to try noise canceling headphones on some of the difficult children, I wouldn t expect a high success rate but if it worked in as little as 5% of cases and did no harm to the children who didn t benefit then it would be worth doing.

14 May 2010

Matthew Garrett: Using qemu to instrument Windows

Part of the problem that we face in providing Linux hardware support is that we're lucky if there's a spec, and even if there's a spec there almost certainly isn't a test suite. Linux still isn't high on the list of things that vendors test with, so as a result hardware and firmware tend to be written to work with Windows rather than some more ideal notion of what a spec actually says.

This can lead to all kinds of problems. If the spec says we can do something and Windows never does that, we'll often find that there's at least one piece of hardware out there somewhere that breaks in response. But sometimes there'll be several different workarounds, and picking the wrong one may break some other system instead. It's helpful to know what Windows is actually doing, if only because that's the only configuration most systems have been tested with.

The problem is that doing this at the hardware level is next to impossible. I'm sure there are people out there who salivate at the possibility of working out i8042 initialisation sequences by hooking up oscilloscopes to their mouse, but I'm not one of them. There's a much easier alternative involving qemu.

The qemu source base is reasonably large and complex, but that's ok - we don't need to care about most of it. For our purposes we really only want to trace accesses to given bits of hardware. There's three main types of hardware access we're likely to care about: io ports, memory mapped io and pci configuration space. io ports are easy. Each piece of qemu that performs hardware emulation will call register_ioport_read or register_ioport_write. Just grep for those, find the ports that correspond to your hardware and then edit the functions to dump the information you want. PCI configuration space will be handled via pci_default_read_config or pci_default_write_config unless the driver overrides them. Finally, for PCI devices mmio will be handled via pci_register_bar - the last argument is the function called when the memory region is accessed.

All of which makes it pretty easy to watch register reads and writes performed by Windows when it's starting up. Suspend/resume is also an interesting problem, but sadly one that's harder to work with. First of all, you need at least version 0.6c of vgabios in order to indicate to Windows that it's possible to suspend at all. Secondly, for Vista or later you'll also need a WDDM driver for the graphics. Sadly there isn't one for the Cirrus that qemu emulates, which is unsurprising given how ancient it is. So I've had to perform my testing under XP, which was enough to give me an indication as to what Windows does with the SCI_EN bit on resume (answer: Ignores both the bit of the spec that says it should already be enabled and the bit of the spec that says it should never be written by hand). Nice, but if someone would like to write a WDDM driver for Cirrus it'd make my life easier.

The other thing I've been testing is keyboard controller probing. Some Macs deal badly with you banging on the keyboard controller, which is dumb but on the other hand they don't claim to have one. Linux will look at your ACPI tables and use any keyboard controllers it finds there, but if there isn't one it'll go on to try probing the legacy locations. Adding some debug code to the read and write functions of the keyboard driver in qemu and then editing the DSDT in Seabios to remove the declarations for the keyboard showed that Windows will only probe if there's a device with a recognised keyboard PNP ID. No keyboard for you if that's missing. So we probably need to emulate that behaviour as well.

The main reason to do this work is to try to reduce the number of DMI tables in the kernel. These are cases where we alter the kernel's behaviour based on the identity string of the hardware, allowing us to work around functional differences. The problem with this approach is that Windows generally works fine on these machines without knowing anything special about them, and chances are that the tables aren't exhaustive - there may well be some other piece of hardware that's also broken, but the user just gave up and went back to Windows instead of ever letting us know. Using qemu to work out how Windows actually behaves gives us the opportunity to fix things up in a way that will (with luck) work on all machines.

Lucas Nussbaum: Receive Ubuntu bugs by mail with the Debian PTS

It is now possible to subscribe to Ubuntu bugmail for the packages you care about, without having to use Launchpad (and subscribe on a per-package basis there). This is implemented as a new opt-in Package Tracking System keyword: derivatives-bugs. To subscribe for all your packages, use keyword [email] + derivatives-bugs (as documented in the Developers Reference). You might also want to subscribe to derivatives (Ubuntu diff, etc. also opt-in). Of course, if other derivative distributions are interested in providing such data, don t hesitate to contact me or the Debian QA team. Also, if you are like me and never remember about subscribing to packages you maintain, you can use that UDD script to check for missing subscriptions.

13 May 2010

Aigars Mahinovs: Hacker Neo caught in Latvia

A scandal has been brewing in Latvia over the last half year and yesterday the activity spiked shocking the media and some IT people in the country. I ll go back and explain what happened first, what is happening now and why this could have a heavy impact on IT and journalists in Latvia. At the end of last year, there were rumours that the IT system of Latvia s Internal Revenue System was hacked and millions of documents had been downloaded by multiple organizations. Shortly thereafter more details on the glaring security hole became public (after it was closed). There is a full electronic interface to give all reports to the IRS electronically (at http://eds.vid.gov.lv) and as part of that system you could also view and export monthly report summaries about your organization into XML and PDF files. After the system checked that you are authorized to access the report, you were redirected to the URL to actually download the report by report ID (as a single param in a GET request). Unfortunately, report IDs were predicable and the script that gave the reports for download did not check if you were authorized to get that report. It did not even check if were logged into the system. There were suspicions that the authorization was disabled on purpose to allow to leak data on purpose, but apparently it was an error of forgetting to disable debug code in production environment. The error was discovered only because the firewall administrator noticed an unexplained stable increase of traffic, especially during night hours when typically the traffic fully stopped. Apparently a single hacker (who later identified himself as Neo to the press) discovered the flaw and wrote a script to just try all possible report ids and get as much data out as possible. This had been going on for months, before someone noticed. After the flaw was discovered and a bit of time passed, Neo made his first move he published the list of top salaries in a governmental company, that clearly showed that the top leadership of this company failed to cut their salary by 40%, like everyone elses during harsh budget cuts of 2009. He stripped the names and ids of the specific employees, but named the company which made it pretty easy to figure out who was who. The society was outraged that the top managers in a government owned company failed to comply with the strict pay cut that everyone else in government had to endure. But after a few weeks the outrage subsided and no action followed from the government or law enforcement. Neo continued to release documents detailing salaries of top managers in different Latvian government companies. And each time after short outrage, nothing happened. Neo gave an interview where he said that he was disappointed in the passivity of the Latvian people in face of such blatant injustices. After a few month Neo went silent, promising to return before parliamentary elections this fall. However, this week a new development shocked everyone in the middle of the night two police SWAT teams went into action: one detained Ilm rs Poik ns, a researcher in artificial intelligence at the University of Latvia s Computer Science department and another raided the home of a Latvian TV journalist Ilze Nagle who interviewed Neo. Poik ns confessed of being Neo the next day and was released (with travel restrictions, pending trial) today. Politicians reacted immediately opposition demanded the resignation of the Interior Minister over such blatant disregard of freedom of press and another politician (who is also a famous lawyer) Aleksejs Loskutovs volunteered to defend Neo pro-bono (on Twitter, no less). Almost all Latvian online media have the arrest of Neo and the raid on the home of a journalist as main stories of the day. As a legal titbit, we also know that Neo is being charged with breaking statutes 145 and and 244p2 of the criminal law. Statute 145 is hard to find applicable in this situation as talks about actions done by people authorized (..) to access [private] information . Statute 244p2 will also be hard to pin down as it mentions influencing system resources of (an IT system) and if such action caused severe harm . It looks like the first part talks about at least a DoS attack (which did not happen in this case) and also there was no measurable harm from these leaks. Also Neo was careful to strip all personally identifying information (such as names, social security numbers and addresses of the employees in question), so it will be hard to pin him on that. Also no actual breaking or other modification of an IT system occurred. And no specialized software was used beyond a trivial script such as :
for i in range(0,7000000):
    wget('https://eds.vid.gov.lv/getRep.aspx?id='+str(i))
A lot of commentators on the Internet likened the situation to walking trough an unlocked door and stealing something. I think that analogy is very incorrect there was no door, and nothing went missing after the action. I came up with a different analogy there was this corridor with a lot of doors in IRS, locked steel doors. You were instructed to go to a room with a specified number and given a key to that room to unlock it and see your secret info. However, that corridor opened out to the street on one end, oh and also the walls of the rooms with all the secrets were transparent. So Neo walked into the corridor, looked at some of the secrets, wrote them down (to remember them better) and then went out and discussed the worst examples abuses of power he saw. In the end IRS had to learn their lesson if you have to put naked photos of yourself on the Internet (or something equally embarrassing), then make damn sure you password protect that, but if you don t then don t cry that someone hacked you and stole you pictures. What other people think:
http://freespeechlatvia.blogspot.com/2010/05/neo-released-under-restrictions.html We ll see how the story develops soon.

Jonathan McDowell: Random election related numbers

We appear to have a government again, which is always helpful. Let's see how they do. While all the deliberation was going on Dad and I had a ponder about exactly what your chances of voting for a winner were. The Guardian helpfully have the results dataset available, so I nabbed that. They may have updated it since I did; it certainly seemed to be a bit off compared with the BBC. Anyway.

29,577,337 - total votes cast.
13,982,219 - total votes cast for winning MPs.
7,279,220 - total votes cast for winning MPs in the new government (ie Conservative or LibDem).

So there was a 47.27% chance of a vote being for a winning MP, but only 24.61% chance that a vote was not only for a winning MP but also one that ended up being part of the coalition.

Another interesting number; 220 seats were won with 50% or more of the vote, 540 with more than 40%. That's higher than I expected.

11 May 2010

Martin Meredith: An hour in the life of my mouse

A mini-meme that has been spreading round the IT team at my workplace is IOGraph a small Java program that tracks and draws the status of your mouse over a period of time. Lines are movement, circles are places where the mouse has stopped The bigger the circle, the longer it was there. I have 2 monitors, which explains the weird dimensions! An hour in the life of my mouse

Next.

Previous.