Search Results: "will"

17 July 2011

Thorsten Glaser: mksh R40b; joe 3.1jupp16 and 2.8jupp2 released

mksh R40b (nowadays with filled in user s caveats (for R40, too!) and packager s upgrade hints) has just been released. This is a should-have upgrade, fixing a number of admittedly some obscure bugs, changing things begun in R40, improving upon others. Thanks to the PLD Linux guys for spotting all these errors; thanks to them and phpnet.org both for adopting mksh so well. I have also fixed a bug in nroff(1) which will lead to an even nicer looking HTML manpage mksh(1) (after the next rebuild and upload of a MirBSD snapshot scheduled RSN). jupp 3.1.16 took on the task of merging Debian joe changes (aiming at an upload). I also split the jupprc file into three versions (2.8 generic/DOS, 3.1+jupp and 3.7/Unix) because of the differences in the baseline executables making rc files partially mutually incompatible (think Insert key), annoyingly warning (think syntax, hmsg), or less usable (joe s new menu system). jupp 2.8.2 is a companion to jupp 3.1.16 mostly because of the new help window character map Binaries for jupp should be updated RSN too. Considering Banja Luka is arriving quickly, the r in RSN should be taken with a few grains of salt. I ve also scheduled working on the pcc Debian package for the next future; updating lynx and maybe others like OpenSSH in MirBSD is also due; cvs(1) will receive more of my time, but before the next Upload I d like to fix LP#12230 once verified. Builds for Debian/m68k are also still running. I note I did in fact not manage to make a new base image, yet (but 2.6.39 kernels miss a patch, anyway, so waiting for 3.0 is ok). It s still using gcc-4.4 because nobody tests gcc-4.6 and gcj-4.6 FTBFS due to SIGSEGV, but that s ok in my books. rsyslog is broken but sysklogd works. The #ksh Freenode page finally got a well-deserved link to Planet Commandline. Throw more my way! Acronyms and translations, too. (Got Norwegian and Rumanian covered in the meantime. No idea whether any RTL languages will work in that beast. But I m young and need the money) Since I m writing a wlog entry anyway let me thank Gunnar for a nice summary on the current Free Culture discussion; my comments on Nina s site seem to be eaten, but let me support it fully, although, of course, I normally use a copycenter style licence, which is specifically written for general works of authorship under copyright law, not limited to software. I did in fact have that in mind. Maybe some people will like it (it s less than one Kibibyte long) either generally or just for their everyday random musings (they can then keep CC-BY-SA for the big works if they so desire). Wouter, grass background makes green headlines illegible. I ve never liked, and never installed manually, cups either. (Benny tells me that Apple s new version refuses to talk with a non-Apple cups, kinda defeating the whole idea I think.) Port 9100 is JetDirect (probably with an HP in front and some subset of trailing) and just nice. (Being able to talk ESC/P with your printer like print '\033K\x07\0\x3E\x81\x99\xA5\xA5\x81\x3E' >/dev/lpa too rocks though, IMHO. Yes, mine can, and I still can. /dev/lpa is BSD.)

Kai, thanks for your vimrc lines:

	:highlight TrailWhitespace ctermbg=red guibg=red
	:match TrailWhitespace /\s\+$\  \+\ze\t/

Automatic removal is harmful, though I just fell into the trap since jupprc contains needed whitespace at EOL but manual removal (bound to ^K] in jupp) rocks. And I like that your solution uses such strong a colour vim users are the single most represented offender group for actually leaving the redundant whitespace at EOL there, and it should hurt their eyes. (Sadly there is some vehement disagreement preventing them from inclusion in grml-etc-core but that s why I re-post them here.) Ah, and jupp can of course display whitespace visibly (although it uses / , replacing the arrow with if no UTF-8, not ), accessible with ^Ov. Steve, want to put up a checklist for sites? We can crowdsource the testing to maybe get some interesting results Some other people would get more comments if they were idling in IRC (Freenode) or allow comments on their blog, specifically without too high an entrance barrier OpenID is ok, but many other things, and ECMAscript, are not; but I can t really say that loud because our wlog is static HTML compiled from a flat plaintext data source so it doesn t allow such either. I often forget what I wanted to add if I can t get it out quickly enough (especially at work). Sowwy Me like the cat picture postings (Amayita, Tiago, Gracias!).

16 July 2011

Matthew Palmer: MySQL replication and crash recovery

The question was recently asked, How do I perform crash recovery on a MySQL master/slave database cluster? Because replication is async, a transaction that has been committed to the master may not be able to leave the master before the crash happens. The short answer is, you don t. Asynchronous replication, by it s very nature, is prone to these sorts of issues, and MySQL doesn t make life easier in general by using query replay. Some of the issues aren t massive: for instance, if a slave crashes, the master will keep replication logs for the slave until it comes back up (as long as the slave comes back before the master decides some of the replogs are too old and starts deleting them, of course), so as long as you can live without a master for a while, you can recover and continue along your merry way. Promoting a slave to be a master, on the other hand, is a situation frought with peril. Here, you ve just got to hope hope that your chosen slave is consistent with the master s state at death (because it s async, you have no guarantees about that), and that all the other slaves have the same ideas about what counts as the current state of the data. If your newly-promoted slave managed to apply an update that another slave didn t, that slave will be out of sync with reality (until such time as the now-dead master comes back and replays all those queries hahahaha). To even guarantee that any slaves have a consistent view of the data as compared to the new master, you ve got to rereplicate everything from the new master, because MySQL s where are you up to? counter is master-specific. I ve heard people who have to deal with this sort of thing these days say that the Maatkit tools are handy for dealing with various issues of this type (monitoring, repairing the database when it gets inconsistent). However, I prefer to keep right the hell away from MySQL replication altogether, after a year of having to gently coax an insane MySQL tree replication setup to do the right thing for any extended period, and watching a group of DBAs at Engine Yard go slowly insane. My preferred alternatives are: Never discount the value of the Jaws approach to scaling ( we re gonna need a bigger boat^Wmachine ) hardware is so ridiculously cheap, relative to the clueful sysadmin time needed to make (and keep) MySQL replication running, that you really need to be at the huge end of town (where other scaling issues are going to bite you in the arse first) before spending too much time on your database is worth the investment. Even then, you ll get more bang for your buck from: than you ever will from the constant, mind-numbing grind that is keeping MySQL replication going.
  1. The term comes from a time when a colleague near-screamed This slave cluster keeps fucking me! at a particularly intransigent set of MySQL slaves.

15 July 2011

Alexander Reichle-Schmehl: About Debian, The Hurd and Linux or in short: Yes, we will still have a Linux kernel

A lot of online magazines are currently reporting about Debian, its port to the Hurd kernel and plans for the next release. However, there seem to be quite some misunderstandings. One online magazine took the cake by titling Debian 7.0 Wheezy: Erste Pl ne f r Hurd statt Linux-Kernel (rough translation: Debian 7.0 Wheezy: First plans for Hurd instead of Linux kernel), and a colleague already asked me, if we are really going to drop the Linux kernel. So let's clarify one thing: The Debian Project does not plan to drop its port to the Linux kernel (nor its two ports to the FreeBSD kernel for what it's worth). Apparently it all started with a short status report and if you read it, you'll just read that some people are trying and planing to get Debian 7 (aka wheezy) to be released with an additional port to the Hurd. It is not yet clear if they will achieve their goal, nor did anyone ever mention anything about replacing the Linux or the FreeBSD kernel. So, calm down, nothing changed, just someone talking about the possibility of adding yet another port to the next release. Please note the adding.

Jon Dowland: puppet packages

Puppet lets you install packages on your system, like this: 
package   'foo': ensure => installed
You can also ask it to make sure that the package is kept up-to-date: 
package   'foo': ensure => latest
What does "latest" mean? How does this interact with a mixed repository sources.list on Debian? It turns out that both ensure => installed and ensure => latest honour pinning. For example: if you have a stable system; sid in your sources.list and pinning set up; either of the above will install stable's version of foo. I think ensure => latest is less useful than it might initially appear. If you install a package foo and use ensure => latest, but foo depends on packages bar and baz, it will only ensure that foo is up-to-date, not bar or baz. Similarly any other packages that you have not installed with puppet will not be kept up-to-date unless you name them explicitly. If you want to automatically take all updates from your distributor, you need to use another mechanism to do so.

Guillaume Mazoyer: Travel to DebConf 11

internet.pngMy flights tickets are booked so I guess:

DC11_banner.png
I m really happy to go to Banja Luka. This will be my first DebConf and not my last I hope. I ll be there to meet incredible people that make Debian what it is but I will also give a talk with Tom Marble and Sylvestre Ledru about my Google Summer of Code project at Debian. This talk should be the 2011-07-30 at 15:00:00.

Dirk Eddelbuettel: MCMC and faster Gibbs Sampling using Rcpp

Sanjog Misra, who uses Rcpp for Monte Carlo Markov Chain (MCMC) analyses in quantitative marketing, kindly set me a short example of Rcpp use. The example is based on a blog post by Darren Wilkinson which itself discusses and compares the suitability of R, Python, Java or C for MCMC analysis, using the Gibbs sampler as a concrete example. Darren's post is worth checking out: he stresses the rather pragmatic aspects of how fast and/or easy it is to write the code, rather than just the mere runtime. As such, he is not too concerned with a speed advantage of Python over R which he sees at a factor of around 2.4, leaving him to continue to prototype in R. Similarly, with C 'only' being faster than Java by a factor of two, he prefers Java for the numerically more demanding parts. We do of course advocate the use of Rcpp to combine the best aspects of R and C++, respectively. This Gibbs sampler example provides a nice backdrop. So working with Darren's example, consider the same Gibbs sampler for a bivariate distribution (and apologies for the lack of latex typesetting on my blog) 
f(x,y) = k x^2 exp( -x y^2 - y^2 + 2y - 4x)
where the conditional distributions are 
f(x y) = (x^2)*exp(-x*(4+y*y))               ## a Gamma density kernel
f(y x) = exp(-0.5*2*(x+1)*(y^2 - 2*y/(x+1))  ## a Gaussian kernel
Sanjog then spotted and corrected a small error in the variance expression in Darren's derivation; this is now acknowledged on Darren's website. Full details are in the R script now committed in SVN. The R code for the Gibbs sample can therefore be written as follows below. Note that this uses thinning to minimize serial correlation in the conditional densities--which renders the computation more demanding as 'N times thin' draws have to be generated: 
## Here is the actual Gibbs Sampler
## This is Darren Wilkinsons R code (with the corrected variance)
## But we are returning only his columns 2 and 3 as the 1:N sequence
## is never used below
Rgibbs <- function(N,thin)  
    mat <- matrix(0,ncol=2,nrow=N)
    x <- 0
    y <- 0
    for (i in 1:N)  
        for (j in 1:thin)  
            x <- rgamma(1,3,y*y+4)
            y <- rnorm(1,1/(x+1),1/sqrt(2*(x+1)))
         
        mat[i,] <- c(x,y)
     
    mat
A second variant can be computed using the R bytecode compiler which appeared with the recent release of R 2.13.0 (and which we analysed in this blog post from April). This is as easy as 
## We can also try the R compiler on this R function
RCgibbs <- cmpfun(Rgibbs)
Thanks to Rcpp and the inline package, we can also write a C++ variant that can be built and launched from R with ease. The C++ code is assigned to an R text variable gibbscode. (And we used to typeset such code on the blog as a character string, ie in a faint red color---but have now switched to highlight it as if it were freestanding C++ code. It really is passed as a single string to R which then uses the cxxfunction() to compile, link and load a C++ function built around the code. See previous posts on the inline package for more.) 
## Now for the Rcpp version -- Notice how easy it is to code up!
gibbscode <- '

  using namespace Rcpp;   // inline does that for us already
  // n and thin are SEXPs which the Rcpp::as function maps to C++ vars
  int N   = as<int>(n);
  int thn = as<int>(thin);
  int i,j;
  NumericMatrix mat(N, 2);
  RNGScope scope;         // Initialize Random number generator
  // The rest of the code follows the R version
  double x=0, y=0;
  for (i=0; i<N; i++)  
    for (j=0; j<thn; j++)  
      x = ::Rf_rgamma(3.0,1.0/(y*y+4));
      y = ::Rf_rnorm(1.0/(x+1),1.0/sqrt(2*x+2));
     
    mat(i,0) = x;
    mat(i,1) = y;
   
  return mat;             // Return to R
'
# Compile and Load
RcppGibbs <- cxxfunction(signature(n="int", thin = "int"),
                         gibbscode, plugin="Rcpp")
It is noteworthy how the code logic is essentially identical between the basic R version, and the C++ version. Two nested loops control draws of x from a Gamma distribution, conditional on y, as well as draws of y from a Normal, conditional on x. Add a small amount of parameter passing to obtain the parameters N and thin, allocation of a results matrix and setup of the random number generator state to remain consistent with R, as well as a return of the matrix---and that is all. As Darren's code uses the GNU GSL in its C variant, I also became interested in seeing how a C/C++ hybrid variant using our RcppGSL package would fare. The code is below. 
gslgibbsincl <- '
  #include <gsl/gsl_rng.h>
  #include <gsl/gsl_randist.h>
  using namespace Rcpp;  // just to be explicit
'
gslgibbscode <- '
  int N = as<int>(ns);
  int thin = as<int>(thns);
  int i, j;
  gsl_rng *r = gsl_rng_alloc(gsl_rng_mt19937);
  double x=0, y=0;
  NumericMatrix mat(N, 2);
  for (i=0; i<N; i++)  
    for (j=0; j<thin; j++)  
      x = gsl_ran_gamma(r,3.0,1.0/(y*y+4));
      y = 1.0/(x+1)+gsl_ran_gaussian(r,1.0/sqrt(2*x+2));
     
    mat(i,0) = x;
    mat(i,1) = y;
   
  gsl_rng_free(r);
  return mat;           // Return to R
'
## Compile and Load
GSLGibbs <- cxxfunction(signature(ns="int", thns = "int"),
                        body=gslgibbscode, includes=gslgibbsincl,
                        plugin="RcppGSL")
This code is similar to the version using just Rcpp, but we need to allocate space for the GSL random generator object (and later release that memory allocation), and we do of course call the GSL functions. This also necessitates declaring the function using the two header files listed as arguments for the include argument of cxxfunction(). So how is the performance? The script (now committed in Rcpp's SVN repo as examples/RcppGibbs/RcppGibbs.R, and also part of the next release) creates the output below when running in non-interactive mode (whereas interactive mode creates a few charts too): 
edd@max:~/svn/rcpp/pkg/Rcpp/inst/examples/RcppGibbs$ r RcppGibbs.R
Replication # 1 complete 
Replication # 2 complete 
Replication # 3 complete 
Replication # 4 complete 
                    N=1000 N=5000 N=10000 N=20000
Elasped Time (R)     0.124  2.650  10.511  41.773
Elasped Time (RC)    0.081  1.916   7.594  30.280
Elapsed Time (Rcpp)  0.003  0.076   0.306   1.221
Elapsed Time (Rgsl)  0.002  0.049   0.196   0.784
SpeedUp Rcomp.       1.530  1.380   1.380   1.380
SpeedUp Rcpp        41.330 34.870  34.350  34.210
SpeedUp GSL         62.000 54.080  53.630  53.280
               test replications elapsed  relative user.self sys.self
4  GSLGibbs(N, thn)           10   7.845  1.000000      7.84        0
3 RcppGibbs(N, thn)           10  12.218  1.557425     12.22        0
2   RCgibbs(N, thn)           10 312.296 39.808286    311.98        0
1    Rgibbs(N, thn)           10 420.953 53.658764    420.59        0
The first block is a hand-computed comparison using four sets of parameters; the second block uses the excellent rbenchmark package to compare ten runs at twenty-thousand draws. We see a nominal increase in performance due to the bytecode compiler, saving roughly 38 percent which seems appropriate given that the R code mostly controls the loops; actual work in undertaken in the already compiled RNG functions. The Rcpp variant is about 34 times faster than the pure R code. Sanjog reported higher increases on his OS X machines (and Darren's post echos a similar order of magnitude). However, on my i7 running Linux I always obtained an improvement in the mid- to high thirties. That is certainly already a rather pleasant result. What surprised and stunned me at first was that the GSL solution scores an improvement of around 53 times (close to the factor of 60 reported by Darren). A closer look at the code (shown above) makes it clear that there are very few compute-intensive operations outside of the RNG draws. I validated this further with a second study timing just one million draws each from a Gaussian and Gamma using R via Rcpp, and using the GSL. It turns out that the R code is about 2.5 times slower for random draws from the Gamma distribution than the GSL. Inspection of the source code---in files src/nmath/rgamma.c for R and randist/gamma.c for the GSL shows why. R uses a much more complex (and presumably more precise) algorithm. I may follow up with Martin Maechler to see if we can illustrative the numerical benefits of this more expensive approach---this blog entry is getting long enough already. So to sum up: Gibbs sampling is a somewhat resource-heavy Monte Carlo Markov Chain method for investigating multivariate distributions. R excels at quick and simple explorations, albeit at somewhat slower execution speed. The Rcpp package can help by providing easy means to accelerate simulations by a significant factor. The example discussed here is now in SVN repository for Rcpp and will be part of the next release.

Updated 2011-07-16: Darren has just updated his comparison; fixed two typos here too.

14 July 2011

Kai Wasserb ch: Vim: automatically delete whitespaces and CRs

Yesterday's blog entry garnered a lot of attention and some responses (in the form of e-mails, which is totally fine). Two of them deserve, as far as I'm concerend, their own little entry in the mini-series I kicked off. So, here we go: First up was reader Sylvain "ythier" Hitier, who suggested adding 
set list lcs=eol: ,tab: -,trail:
to your .vimrc and making use of the :list option. The above line will highlight tabulators, trailing whitespaces and the end of a line. And it'll allow you to differentiate the three.
A warning though: you would want to deactivate this, in case you want to do copy and pasting with your mouse. [UPDATE] Sylvain just sent me a little key binding for C&P he's using to toggle the highlighting: 
function CutPaste()
  setlocal paste! nu! list!
endfunction
map <F10> :call CutPaste()
[/UPDATE] The second really nice idea came by courtesy of Thilo Six to my attention: 
" automatically delete trailing whitespace & Dos-returns     2
fun! <SID>MyDeleteTrailingWhitespace()
  if ! &bin
    let l:l = line(".")
    let l:c = col(".")
    silent! :%s/[\r \t]\+$//
    call histdel("search", -1)
    call cursor(l:l, l:c)
  endif
endfun
autocmd BufWritePre,FileWritePre * call <SID>MyDeleteTrailingWhitespace()
The above snippet will automatically delete trailing whitespace and carriage returns as added by a lot of Windows programs.
Again a little warning: the function is executed automatically, so be sure you have only replacements included in the regular expression you really want to get deleted! Thanks a lot for all the input I've received!

Matthew Garrett: Booting with EFI

One of the ways in which EFI is *actually* better than BIOS is its native support for multiple boot choices. All EFI systems should have an EFI system partition which holds the OS bootloaders. Collisions are avoided by operating system vendors registering a unique name here, so there's no risk that Microsoft will overwrite the Fedora bootloader or whatever. After installing the bootloader the OS installer simply sets an NVRAM variable pointing at it, along with a descriptive name and (if they want) sets the default boot variable to point at that. The firmware will then typically provide some mechanism to override that default by providing a menu of all the configured variables.

This obviously doesn't work so well for removable media, where otherwise you'd have an awkward chicken and egg problem or have to force people to drop to a shell and run the bootloader themselves. This is handled by looking for EFI/boot/boot(architecture).efi, where architecture depends on the system type - examples include bootia32.efi, bootia64.efi and bootx64.efi. Since vendors have complete control over their media, there's still no risk of collisions.

Why do we care about collisions? The main reason this is helpful is that it means that there's no single part of the disk that every OS wants to control. If you install Windows it'll write stuff in the MBR and set the Windows partition as active. If you install Linux you'll either have to install grub in the MBR or set the Linux partition as active. Multiple Linux installations, more problems. It's very, very annoying to handle the multiple OS case with traditional BIOS.

This was all fine until UEFI 2.3 added section 3.4.1.2 of the spec, which specifies that in the absence of any configured boot variables it is permitted for the firmware to treat the EFI system partition in the same way as removable media - that is, it'll boot EFI/boot/bootx64.efi or whatever. And, if you install Windows via EFI, it'll install an EFI/boot/bootx64.efi fallback bootloader as well as putting one in EFI/microsoft.

Or, in other words, if your system fails to implement the boot variable section of the specification, Windows will still boot correctly.

As we've seen many times in the past, the only thing many hardware vendors do is check that Windows boots correctly. Which means that it's utterly unsurprising to discover that there are some systems that appear to ignore EFI boot variables and just look for the fallback bootloader instead. The fallback bootloader that has no namespacing, guaranteeing collisions if multiple operating systems are installed on the same system.

It could be worse. If there's already a bootloader there, Windows won't overwrite it. So things are marginally better than in the MBR days. But the Windows bootloader won't boot Linux, so if Windows gets there first we still have problems. The only solution I've come up with so far is to have a stub bootloader that is intelligent enough to scan the EFI system partition for any other bootloaders and present them as a menu, and for every Linux install to just blindly overwrite bootx64.efi if it already exists. Spec-compliant firmware should always ignore this and run whatever's in the boot variables instead.

This is all clearly less than optimal. Welcome to EFI.

( Comments Comment on this)

Rapha&#235;l Hertzog: How to squash Debian release critical bugs

Squashing release critical (RC) bugs is one of the most helpful way to contribute to Debian. Because those are the bugs that we need to get rid of in order to release the next stable version. Let s see how you can help. The process is really simple:
  1. Find a RC bug that you can fix
  2. Prepare a fixed package
  3. Send the patch to the BTS
  4. Get your fixed package uploaded
I have already covered points 2 and 3 in a former article: How to prepare patches for Debian packages. So I m not going to repeat myself in this article, instead I ll cover how to find a release critical bug to fix and how to get your package uploaded. Find a RC bug with UDD The Ultimate Debian Database (UDD) provides a neat interface to query the bug tracking system: http://udd.debian.org/bugs.cgi. The form contains many options to customize the query, we ll try explain some. First, at the top, you can decide to list bugs based on whether or not they affect testing/stable/unstable. Indeed, thanks to version tracking the BTS knows if a bug applies to the version of the package present in a given distribution. I suggest you pick a choice that includes sid because that s where you re going to push your fixed package. If sid is not affected by the bug, you can t do much about it (except if you want to join the release team and help ensure updates are flowing from unstable to testing). I usually go for wheezy and sid because that filters out packages which are not in testing (and are thus not blocking the next release). Then you have lots of filters to further reduce the list of bugs. It s important to reduce the noise as you typically have several hundreds of RC bugs at a given time. I suggest you the following settings for a start, but you re free to try other combinations depending on what you want to achieve (I ll give some examples later): I usually sort the results by source package so that I immediately see if several RC bugs are affecting the source package. And I might add popularity contest information to get a feel of whether the package is widely used or not. You can see the results of the suggested query here. Other interesting queries: Find a RC bug with rc-alert The downside of the above method is that you get bugs on packages that you don t know at all and that you might not care about. Some of them might even be better removed instead of fixed. With the rc-alert command (provided by the devscripts package), you will get a list of RC bugs
on packages that are installed on your computer. It should be much more restrictive. And you have command line options to further filter the list just like with the UDD form. Bonus feature: you can also filter by debtags allowing you to restrict the bugs to packages which are implemented in programming languages that you know. 
$ rc-alert --include-dists TU --exclude-tags P+ --debtags implemented-in::perl,implemented-in::python
Package: gwibber
Bug:     608724
Title:   gwibber bypasses certificate checking when providing the login/password for OAuth
Flags:   [     S  ] (security)
Dists:   [STU] (stable, testing, unstable)
Debtags: implemented-in::python, interface::x11, role::program, uitoolkit::gtk, works-with::im, x11::application
[...]
Get your fixed package uploaded Usually sending the patch to the BTS is more than enough. The maintainer or another Debian developer will pick it up at some point. But if you re interested in becoming a Debian maintainer/developer, it might be worth to push your changes more quickly and get credited by a proper upload with your name. Instead of just preparing a patch, you should also prepare a fixed package following the rules of a Non-Maintainer Upload. Put this package somewhere online (for example on mentors.debian.net). When you send your patch to the BTS, include the link to the updated package and express your wish to get this update sponsored. If you haven t heard anything from the maintainer after 2-3 days, try to find a sponsor on debian-mentors@lists.debian.org. NMUs are easier to review than completely new packages and many Debian developers like to kill RC bugs, so there s a good chance that you ll find someone. Now happy RC bug-squashing! And if you really enjoy it, you can subscribe to debian-bugs-rc (~3000 mails per month) :-)
If you want to read more articles like this one, click here to subscribe to my free newsletter. You can also follow me on Identi.ca, Twitter and Facebook.

No comment Liked this article? Click here. My blog is Flattr-enabled.

13 July 2011

Wouter Verhelst: 'grass' theme

I got tired of my previous website theme, so I created a new one. And thanks to the miracles of CSS, I didn't have to throw out the old there's still a 'choose your layout' option in the menu. Playing with CSS is fun, and this was an ideal opportunity to learn the new features of CSS3. I've used some of those, and did so in such a way that the theme should still be usable under a browser that doesn't speak CSS3 yet (I hope, though I didn't test). I'm calling it the 'grass' theme if you go check it out, you'll see why and it's now the default. While I was at it, I found out about this little features of CSS that I didn't know about earlier, but could be useful none the less: Using background: inherit is not the same as background: transparent. The latter will render all elements on the site (including floating elements) on top of eachother, while the former will inherit the background of parent elements, but will 'hide' elements that have a lower z-index value on the same location. This is useful if you have position:fixed or position: absolute elements that might overlap with floating elements on small screens. I'm sure this is old hat to anyone who's done some serious web development (me, serious web development? hah!), but it's news to me. One thing I haven't found is whether it's possible to use an alpha layer using CSS3. I looked for quite a while and didn't find it, so I suppose the answer is that it can't be done, and you'll have to use a PNG file with alpha values, or some such. Oh well. In other news, . But I guess that's not a surprise to anyone. Leaving by car, in three days. That might be a surprise. Still have to pack a shitload of things, and also find some things. No surprise there nothing better for me to do something than an uncomfortably close deadline. Oh well.

Lior Kaplan: An RTL issue in Ubuntu s installation

Earlier this week I saw a request for help from the Israeli community to fix an RTL issue in the Ubuntu installation. Although the request is in Hebrew, the screenshots speak for themselves and show the cropped text on the right side of the screen. There s a combination of a few problems:
  1. Some of the text has LTR directionality, probably due to sentences beginning with an English word. This could be worked around by some translation changes or forcing directionality with RLM characters.
  2. The text is aligned to the left. This might be due to the sames reasons above. Maybe it s also possible to override the alignment of RTL languages regardless of the directionality.
  3. The text is cropped on the left side. Seems that just mirroring the English positions isn t enough to get good results. (I didn t check the code to see if that s actually what is done, but that s one of the common ways I saw to handle support for RTL programs).
LP bug #798768 was reported a month ago, but it seems to repeat LP bug #560114 reported in April 2010. Any help with these issues will be more than welcome. Of course, I ll be glad to help in any possible way (sorry, I don t write code) and I m sure this also applies to the members of the local community.
Filed under: Ubuntu

Steve McIntyre: Project Harmony?

So, the "Harmony Project" launched their set of contributor agreements and tools last week. Colour me unimpressed... There's a claim on their website that they are a "community-centered group", but I don't see any list of people and organisations who contributed to this work. That bothers me. Regarding their aim to "assist organisations which use contribution agreements", I don't think that there is anything of value here for the Free Software community at all. Free Software developers don't need contribution agreements, and in my opinion encouraging their use like this is only going to cause further splintering of the community. We've managed for a very long time without them, why start now? As a developer, I personally don't believe in contribution agreements at all. If I contribute code to a project, it will be under the terms of a good Free Software license or not at all. That's all that's needed. There's a fair body of opinion out there on this - see pieces from Bradley Kuhn, Richard Fontana and Dave Neary for more discussion. What do you think?

Daniel Baumann: Debian Live 6.0.2 Images

Two days before the 6.0.2 squeeze point release a user reported that usb-hdd images are not bootable for him. We could not reproduce that for several days but thanks to David Endler, we found out a few days ago that the reason for this is the following commit: Setting partition flags after creating the filesystem, workaround for bug in parted where we result in having the partition type set to linux otherways. The original problem was that in some situation the code resulted in having a wrong partition type and that can let to confusion for some bioses. For other reasons, the applied workaround has the side effect to cause parted to not set the bootable flat at all anymore and by that creating an even bigger problem - most crappy bioses these days require the bootable flag to be set and refuse to boot from a device otherwise (Missing Operating System). Or in other words, the fix solved the original problem but created another one that is affecting much more bioses. Rather than doing a hasty and suboptimal build of 6.0.2 with the commit reverted (and wasting resources for a 6.0.2a build some time later), we have looked into fixing the original problem properly without causing side effects. Images for the 6.0.2 squeeze point release will therefore be published in the next couple of days.

MJ Ray: SPI Annual General Meeting 2011

Software in the Public Interest (SPI) will hold its Annual General Meeting today: Wednesday, 13th July 2011 at 20:30 UTC (21:30 UK, 22:30 CET). SPI is the developer association that supports debian, OpenWrt and many other projects. SPI meetings are held on irc.spi-inc.org (the OFTC network). The agenda for the meeting is available at http://www.spi-inc.org/meetings/agendas/2011/2011-07-13/ There are currently no resolutions on the agenda, but there should be some annual reports.

12 July 2011

John Goerzen: First Look at Google Plus

Seems like everybody s writing Google+ reviews these days, but most of the ones I ve found seem to be fluff pieces. I haven t used it long, but have some initial impressions to share. First, an analogy. Facebook reminds me of the sleazy guy selling stuff from his car down the street. They do things like change privacy defaults when they think it s good for Facebook, rather than good for you. Google+ reminds me of an Apple product. It s beautiful, easy to learn, but locks everything down and is lacking some critical features. So here goes with the details. The good I think that this can be best summed up by: it s not Facebook. Facebook has, perhaps, set an incredibly low bar but still, this holds. Facebook is the only website I can ever remember using that changes things so much, so often, and so completely that I keep having this feeling of not knowing how to do things. Try maintaining a Facebook page or two and you ll especially feel my pain then. But even the basics: how many times have I accidentally posted a partial comment because I pressed Enter to start a new paragraph (that has only been the button used for that for, hmm, let s see now, decades), and instead it posted the comment. Principle of least surprise, anyone? Google+ has a simple and, on the surface anyway, intuitive interface. However, it does get muddy; more on that below. I m going to spend a lot more space on the bad below, but don t let that diminish my excitement about having a viable alternative to Facebook. I am keen to get rid of that monster. The Bad I ll start of this section with the fact that Google+ is a tightly-controlled walled garden. There is no way to take a copy of your status updates, comments, etc. and back them up with your own devices. If your Google account goes away, so do all your updates about your kids first words. Facebook does have this feature these days. It has broken half the time, but it exists, works, and I use it. Twitter doesn t have the feature built in, but its API makes it pretty easy; you could easily use my twidge program to do this, for instance. But whatever you put in G+ stays in G+. There is no autoposting it to twitter or Facebook, no backups, nothing. I m disappointed in that, given Google s prior attitude about openness to individual data ownership. (Update 7/12/2011: There is a way to get a backup, which I hadn t noticed; see comments below.) Almost as bad, there s also no way to get data in. So I now have identi.ca, twitter, Facebook, and now also G+ accounts. When I post an item in identi.ca, it autoposts to twitter, and then it autoposts from twitter to Facebook. Handy one place to notify people of my public activities. My uploads to Flickr and blog posts also auto-post to Facebook, so people I m friends with there get a picture of what I m doing outside of Facebook without me manually having to link it in. Not so with G+. There is no way to even add an RSS feed for my blog to auto-post to G+ as there is with Facebook. If you use G+, the only way to get stuff into it is with a keyboard. Poor integration with other Google apps is also an issue. G+ is nowhere near Google s first social app. They also have Blogger, Google Reader, Buzz, Gmail, etc. So here s the rub. These aren t integrated well, and when they have tried to integrate them, they ve done an exceedingly poor job of it. As an example: Buzz is a twitter-like service for posting updates. Very similar to what G+ does, right? Well, the best you can do is link your Buzz account to a separate tab on your profile. You can t even have your Buzz updates flow straight into G+, from what I can see. You have a separate list of followers in Buzz from G+. Reader is even worse; it has a sharing feature, but it s not G+ aware, so it goes to a reader inbox. But what takes the cake is the integration with Gmail Contacts. Sounds easy? Not entirely. When you add someone to G+, apparently it sometimes links the record with their entry in Contacts. And the help gives you the handy warning that when you remove them from G+, you have the option of removing them from Contacts, which could impact what you see on a mobile device. It is unclear what happens when it doesn t link the record, or how it decides which record to link (some people have more than one person in a family sharing phone numbers and email addresses, for instance.) One very nice feature of G+ is you can add email addresses, even if they aren t G+ members. Then when you post updates to the circle you ve added them to, G+ offers to send them an email with an update. Very handy. Except the undocumented part is that if you select Your circles (meaning all your circles), rather than individually ticking the box by each circle, they don t get the email. And they also don t get it if you set it Public. You have to select their specific circle, which is a significant difference from how people with G+ accounts are treated. And that s just an example. There are a ton of things that work one way 95% of the time, but have non-obvious exceptions. Some of these exceptions are documented in the help, and some aren t. I m still confused about the integration with Picasa. G+ help says that you can upload unlimited numbers of photos for free (though they will be downscaled at a certain point). Picasa has definite storage limits. But G+ uploads are showing up in Picasa. Do I really get free storage by uploading to the same place via a different tool? Very odd. And finally, a gripe about web standards. I ve been using Firefox/Iceweasel 3.5, which came out in Debian squeeze and is still supported there. It is also pretty darn new by corporate IT standards. And Google Plus refuses to let me log in with it, saying it s incompatible. Perhaps it can t do some fancy animation, but then again I don t really care. Seems Google has forgotten the old RFC adage: be liberal in what you accept and conservative in what you generate. If you can detect that my browser won t animate something right, then you could give me a stripped-down version of the page rather than an error message.

Gunnar Wolf: Free Software must migrate to become Free Culture

Cineast and Free Culture activist Nina Paley wrote some days ago a rantifesto on why the FSF has a double standard: Why are the Freedoms guaranteed for Free Software not guaranteed for Free Culture?, by not following its own very strict rules on software when it comes to culture as a whole. Her post was widely circulated, and got (at least) one reply by fellow Debian Developer Wouter Verhelst, largely agreeing with her, and an anti-rantifesto by Joe Brockmeier Which was promptly answered again by Wouter with a very fun and inspired post, written from the right angle: From the viewpoint of a person who is both a programmer and a musician, and understands the concepts at hand. I'd love to write a longer, better thought post But I'm tired and frankly stressed by many things, so I am just echoing their very interesting discussion to other people who might want to read it. I have been thinking and writing bits on that subject over the last couple of months. An example of that was the talk I gave at the Senate ~6 weeks ago. Following that talk, I wrote a short article for Revista Z calo (a widely circulated magazine mainly dealing with Mexican politics and social issues) called simply Software libre, cultura libre (full text available, but in Spanish only You can try reading an automated translation if it suits you). I wrote the article, mind you, with very limited time, and I'll be the first to recognize the prose was quite poor this time :( Anyway My point is that our nature is to share culture, to build it in a collaborative fashion, and having the Internet as a practically zero-cost, zero loss medium with which we can interchange our creativity with other like-minded people will naturally boost creativity. Free Software emerged before other Free Culture groups just because programmers had privileged access to Internet in the 80s and early 90s; as network access and digital creation tools have got to more people, it's just natural for all kinds of free culture to grow. Software is just a form of knowledge. Code is just a notation for a certain kind of ideas, just as the mathematical or musical notations. I believe (and hope) it's just unavoidable for us all to eventually switch to a mainly free cultural creation system.

Jaldhar Vyas: Google^wBraincells Summer^wChaturmasa of Code

Barring a miracle, it looks like I won't be going to Debconf 11 this year which is a shame as that is a part of the world I have yet to visit. Anyway it got me thinking, I've neglected a lot of my Open Source activities this past year or so what with struggling to stay in good financial shape. As Chaturmasa started today in which we take up additional religious duties, I thought it might be a good time to recommit to getting all my projects up to speed again. So during the next four months I aim to (in no particular order) Hopefully by writing this down here in public, I will be able to shame myself into getting most if not all of this done.

11 July 2011

Thorsten Glaser: [Update] Hilfe gesucht: Mitarbeiter berwachung

Deutsche Version dieses Postings read the English Version instead Der Leiter der Abteilung Systemadministration einer Firma, welche in diesem Artikel ungenannt bleiben soll, wo der Zufall es will, da ich f r sie arbeite, m chte uns Mitarbeiter dazu zwingen, das Feature der persistenten Historie (also einen niedergeschriebenen Verlauf der abgesetzten Befehle) der Shell (Kommandozeile) auf allen Systemen zu benutzen. Nun hat die mksh dies vern nftigerweise, w hrend immer noch verf gbar, standardm ig aus Gr nden der Privatsph re deaktiviert, und ich finde, dies sollte sich nicht ndern. Es kann ja nicht sein, da man mir beim Arbeiten ber die Schulter guckt; was kommt als n chstes, Video berwachung der B ros? (Seine Argumentation l uft darauf hinaus, da er nachvollziehen/verstehen kann, wie ich (ha! ich! nicht die anderen, nein, ich!) ein Problem das letzte Mal gel st habe; es ist aber nichts anderes als rekonstru eren zu k nnen, wer wann was eingetippt/gemacht hat. Naja, Argumentationen sind eh relativ, andere jagen Flugzeuge in ihre eigenen Verteidigungsministerien, welche gerade zuf llig umgebaut werden ) Ich habe dies nat rlich in die F hrungsetage eskaliert, und unser Gesch ftsf hrer versteht meine Bedenken und m chte nun ein Gespr ch anraunen, um eine L sung zu finden. Die meisten meiner Kollegen im Bereich Systemadministration denken leider nichtmals dr ber nach, obschon ich in anderen Bereichen Gleichgesinnte gefunden habe. Einen Betriebsrat haben wir auch gro er Fehler, ich wei nicht (aber ich s e eh nicht drin, ich hasse Politik jeder Art). Dies l t mich in dringender Not, Argumente wider diese Total berwachung aller (insbesondere auf der Kommandozeile arbeitenden also besonders mich) Kollegen zu finden, idealerweise solche, die durch deutsches Recht gest tzt sind. Wenn ihr solche habt, bitte mailt die schnell an meine eMail-Adresse, welche da nicht junk@mirbsd.org lautet, sondern mit einem tg vor dem Klammeraffen (dies dient der Abwehr unerw nschter Werbemails). Vielen Dank bereits im Voraus! Update 12.07.2011: Wir haben geredet, und wie viele kommentierten keine L sung auf rechtlicher Ebene gefunden, sondern stattdessen, nachdem wir festgestellt hatten, da mir dieses Problem sehr am Herzen liegt, nach anderen L sungen f r das eigentliche Ziel des Chefadmins gesucht; dies z hlt als ein Entgegenkommen auf Seiten der Firma. Ich habe zugestimmt, meine L sungen in kleine Skripte zu verpacken und ins VCS zu committen f r das Herausfinden, wann was auf einem Server ge ndert wurde, im Fehlerfall, denken wir uns gerade ohnehin was aus. Danke an alle, die kommentiert haben!

Thorsten Glaser: [Update] Big Brother , or: Help wanted arguments against monitoring employees

English version of this posting lese stattdessen die deutsche Version The head system administrator of a henceforth unnamed company I happen to be employed at wants to force us to use the shell s persistent history feature disabled by default in mksh as is sensible, for privacy reasons on all systems, so he can retrace what we do (of course, his wording was more like, so he can reconstruct how I fixed a problem the last time it occurred yeah, sure). Now I ve escalated this to the boss, since this just cannot happen, what ll be next, video monitoring our offices or what and the boss said he sees my concerns and would like a meeting to get an agreement between all parties involved. Most sysadmin coworkers actually don t even think about this, they are sheeple, really; but my project manager (for when I m not a sysadmin but a programmer) and the leaderess of our QA department agree with me, luckily. Now I m in dire need of arguments, especially German-Legalese, to fight this total surveillance of employees, mostly targetted at these who use the command line almost exclusively, which I m a prime example of. Please mail me at my address, which is not junk@mirbsd.org but with tg instead of the waste part here to catch address grabbers (sending mail to that address yields a 24-hour total blacklist of your IPv4 address I ve yet to get enough spam via IPv6 to say filtering it is worth the cost). Thanks in advance!

Update 2011-07-12: We talked, and it was agreed to not force a shell history; instead I will now put problem solutions into smallish scripts and commit them into the sysadmin SCM. It was not solved using legal arguments, as many people commented would have been difficult; instead, they acknowledged that I was concerned with this, and we searched for a different solution to the head sysadmin s original concerns (apparently, to figure out how to fix things without reinventing the fix every time, and to find out what was changed on a server in case it breaks).

Rapha&#235;l Hertzog: 7 tips to file useful Debian bug reports and get your problem solved

Filing bug reports is the most common way for users to contribute. Even if it s not too difficult, I ll give you some advice to improve the quality of your reports. After all, when you go out of your way to report a bug, it is in the hope to see it fixed so let s see how we can make this more likely. 1. Try to reproduce the bug If you can t reproduce the bug, it s next to impossible to find the root cause and thus to fix it. In that case, I would suggest you to wait until you experienced the bug multiple times. Maybe you ll be able to find something that triggers it (or that makes it more likely to encounter it). If the application has a debug/verbose mode, it might be a good idea to enable it until you experience the bug a second time. The generated log might be helpful for the developer to understand what happens exactly. So don t file bug reports straight away unless you can reproduce it. The exception to the rule is when the application gives some useful information like a core-dump, a back-trace or an error message. Obviously if the bug happens during an upgrade, it s difficult to reproduce it (unless you have multiple computers) but you should still report it. Be sure to include all the relevant information (versions of packages before and after the upgrade, logs of the upgrade, etc.). 2. Do your best to identify the faulty package When you report a bug to Debian, you must assign it to a package. While there are pseudo-packages useful for problems which are not directly attributable to a real package, in most of the cases you should report a bug against the specific package that seems to be the cause of the problem you encountered. In turn this often requires you to attribute the problem to a file (for example the executable of the application that triggers the bug). Once you have a filename you can use dpkg -S to identify the corresponding package. 
$ dpkg -S /usr/bin/hamster-time-tracker
hamster-applet: /usr/bin/hamster-time-tracker
Note that reportbug accepts a filename as parameter and will do the above conversion for you. If you only know the name of the application (but not the filename of the associated executable), you can use dpkg -S with a pattern to let it return all possible matches: 
$ dpkg -S hamster
hamster-applet: /usr/share/applications/hamster-applet.desktop
hamster-applet: /usr/share/gnome/help/hamster-applet/es/statistics.page
[ ]
hamster-applet: /usr/bin/hamster-time-tracker
[ ]
Or you can also verify in the list of installed packages: 
$ dpkg -l "*hamster*"
Desired=Unknown/Install/Remove/Purge/Hold
  Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
 / Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
 / Name            Version         Description
+++-===============-===============-==============================================
ii  hamster-applet  2.32.1-1        time tracking applet for GNOME
3. Verify that the bug is not already reported and/or fixed If there s a newer version of the software available, it s a good idea to try to reproduce the problem with this version too. Because the developers tend to care only about the latest version, they will want to reproduce it with this version, and they will be annoyed if the problem that you reported is already fixed. That s why bug reports of users of testing/unstable tend to be more useful than bug reports of stable users. In any case, you want to verify that the bug has not yet been reported: filing a duplicate bug is useless and only generates more work for the developers to merge both bugs together. On the opposite, it s highly appreciated to add supplementary information to an existing bug report, even a simple confirmation that the bug still exists on a newer version is useful. Note that reportbug will automatically show you the list of open bugs before allowing you to submit a new one. 4. Use reportbug While the Debian bug tracking system allows anyone to submit a new bug with a simple mail, you should really use a dedicated program like reportbug (or reportbug-ng) because it will automatically include lots of useful information in the generated report (version of dependencies, current architecture, etc.) and will assist you in all the steps. 5. Describe the problem so that the developer can reproduce it Ideally your report should include everything required so that the developer can reproduce the problem on his system. If a given document triggers the bug, attach it. Describe the steps required to reproduce the bug in great details just like you would explain it to your grand-ma. Explain how you expected the program to react and what happened instead. You can learn much more on how to draft a good bug report in this article: How to report bugs effectively. It s a bit long but well worth it if you intend to report bugs and thus interact with developers. 6. Be kind and willing to help When you draft a bug report, keep in mind that you re writing to a volunteer free software developer and not to a customer service. You should be respectful and follow the usual rules of courtesy. Developers attention is scarce and should not be wasted. Be willing to help, if the developer starts investigating your problem, he might need your help to get supplementary information (in particular if he can t reproduce it) and you should be ready to provide it. Thus it s important to keep whatever you need to reproduce the problem. In some cases, the Debian maintainer might be overworked and you can offer your help to forward the bug to the upstream bug tracker, it s always appreciated. If you re reasonably confident that the problem is not Debian-specific, you can do it straight away and set the forwarded field to the URL of the upstream bug report (for example with bts forwarded <bug> <url>). 7. Use the correct severity The Debian bug tracking system lets you set the initial severity of the bug report (in decreasing severity: critical, grave, serious, important, normal, minor, wishlist). Pick the correct severity according to the official definitions but don t misread them. In particular, don t over-inflate the severity: for instance if you lost some data due to a misuse of the software, it s not critical (i.e. rm -rf * doesn t warrant a critical bug against rm). If you use only a tiny part of a software, and that part doesn t work, the package might be unusable for you but it s not unusable for everybody, so it doesn t warrant the grave severity. The important severity is often a good choice in those cases. Do not under-estimate the severity either, if a problem is important enough that it must be fixed before the next stable release (for example a regression compared to the previous release), pick a release-critical severity (i.e. at least serious ). The maintainer and the release manager can always lower the severity if they do not agree with your initial judgment. And now, happy bug-reporting!
Follow me on Identi.ca, Twitter and Facebook. Or subscribe to this blog by RSS or by email.

6 comments Liked this article? Click here. My blog is Flattr-enabled.

Next.

Previous.