Search Results: "spa"

29 March 2011

Steve Langasek: Multiarch Monomania

So the other day, I was able to do this in an Ubuntu natty amd64 chroot for the first time.
# cat > /etc/apt/apt.conf.d/multiarch-me
APT::Architectures   "amd64"; "i386";  ;
^D
# cat >> /etc/dpkg/dpkg.cfg
foreign-architecture i386
^D
# apt-get update
# apt-get install flashplugin-installer:i386
Reading package lists... Done
Building dependency tree       
Reading state information... Done
[...]
The following NEW packages will be installed:
  flashplugin-installer:i386 flashplugin-nonfree:i386 gcc-4.5-base:i386
  libatk1.0-0:i386 libavahi-client3:i386 libavahi-common-data:i386
  libavahi-common3:i386 libc6:i386 libcairo2:i386 libcomerr2:i386
  libcups2:i386 libdatrie1:i386 libdbus-1-3:i386 libdrm2:i386
  libegl1-mesa:i386 libexpat1:i386 libfontconfig1:i386 libfreetype6:i386
  libgcc1:i386 libgcrypt11:i386 libgdk-pixbuf2.0-0:i386 libgl1-mesa-glx:i386
  libglib2.0-0:i386 libgnutls26:i386 libgpg-error0:i386 libgssapi-krb5-2:i386
  libgtk2.0-0:i386 libgtk2.0-common libice6:i386 libjasper1:i386
  libjpeg62:i386 libk5crypto3:i386 libkeyutils1:i386 libkrb5-3:i386
  libkrb5support0:i386 libnspr4:i386 libnspr4-0d:i386 libnss3:i386
  libnss3-1d:i386 libpango1.0-0:i386 libpcre3:i386 libpixman-1-0:i386
  libpng12-0:i386 libselinux1:i386 libsm6:i386 libsqlite3-0:i386
  libstdc++6:i386 libtasn1-3:i386 libthai-data libthai0:i386 libtiff4:i386
  libudev0:i386 libuuid1:i386 libx11-6:i386 libx11-data libx11-xcb1:i386
  libxau6:i386 libxcb-dri2-0:i386 libxcb-render0:i386 libxcb-shape0:i386
  libxcb-shm0:i386 libxcb-xfixes0:i386 libxcb1:i386 libxcomposite1:i386
  libxcursor1:i386 libxdamage1:i386 libxdmcp6:i386 libxext6:i386
  libxfixes3:i386 libxft2:i386 libxi6:i386 libxinerama1:i386 libxrandr2:i386
  libxrender1:i386 libxt6:i386 libxxf86vm1:i386 x11-common zlib1g:i386
0 upgraded, 78 newly installed, 0 to remove and 3 not upgraded.
Need to get 15.1 MB/15.6 MB of archives.
After this operation, 48.9 MB of additional disk space will be used.
Do you want to continue [Y/n]?
It is a truly heady experience, after so many years of talking about the need to properly support multiarch in Debian and Ubuntu, to see support for cross-installation of packages come to fruition. If you've talked to me any time in the past couple of weeks and noticed it's a little hard to get me to change the subject, well, that's why. Many who have grown accustomed to Debian and Ubuntu's lack of support for installing i386 packages on amd64 (or vice versa) may wonder what the fuss is about. (Whereas others who are well versed in distributions such as Red Hat and SuSE may laugh and wonder what took us so long...) So maybe a few words of explanation are in order. If you've ever installed ia32-libs on an amd64 machine anywhere; if you've ever noticed a bug where ia32-libs didn't work right because of wrong system paths, or had to file a request for another library to be added to ia32-libs because it wasn't included in the set of libraries Debian decided to package up in this grotesque, all-in-one 32-bit compatibility bundle; if you've ever decided not to install a 64-bit OS on your perfectly 64-bit-capable hardware because of concern that you wouldn't be able to run $random32-bitonly_application; multiarch is for you. If you've gotten stuck maintaining a lib32foo "biarch" package in Debian due to popular demand, multiarch is definitely for you. :) If you've ever cross-compiled software for a different Debian architecture than the one you were running, multiarch is for you. If you've ever wanted to run binaries for a different architecture under emulation, and found it awkward to set up the library dependencies, multiarch is for you, too. Because although the .deb world may be a little late to the party, we're also naturally taking things much further than anyone's done with rpm. Multiarch won't just give you the ability to install 32-bit libs on 64-bit systems; it'll give you the ability to install libs for any known architecture on any system. And a whole lot of pain just falls out of the equation in the process. A cross-compiling environment looks the same as a native-compiling environment. An emulated system looks the same as a native system. We can start to seriously consider cross-grading systems from one architecture to another. And all this is happening now. The groundwork is there in Ubuntu natty. Wheezy will be the release that brings multiarch to Debian. When dpkg 1.16.0 is uploaded to unstable real soon now, the bootstrapping will begin. I am immensely grateful to everyone who's helped make multiarch a reality - to Tollef, Matt and others for seeding the vision; Aurelien, Matthias and Arthur for their work to ready the toolchain; David and Michael for the apt implementation; Guillem and Raphael for the dpkg implementation, and Linaro's support to help make this possible; and the many other developers who've helped to refine this design over the years in numerous other BoFs, sessions and mailing list threads. I'm excited to find out what the Debian community will do with multiarch now that it's upon us. Christian, maybe you should start a pool for how long it will take before all the libraries shipped in ia32-libs have been converted to multiarch and we can drop ia32-libs from the archive?

28 March 2011

Raphaël Hertzog: People behind Debian: Bdale Garbee, chair of the technical committee

Bdale is a long-time Free Software believer, he has been contributing even before Debian existed in the prehistoric era of free software. :-) Anyone who went to a big Free Software conference has seen one of his colorful t-shirts. Or maybe you have heard the story where he got his beard shaved by Linus Torvalds to raise funds to protect the Tasmanian Devil. More seriously Bdale has played and continue to play a number of important roles in the Debian community. He also represents one of the biggest corporate sponsors (both for DebConf and for the servers that Debian owns): Hewlett Packard. My questions are in bold, the rest is by Bdale. Who are you? I made my first personal contribution of source code to what we now call Free Software in 1979. I started with HP in 1986 and for nearly a decade have served the company as Chief Technologist for Open Source & Linux. I am president of Software in the Public Interest, which is the umbrella organization providing legal and financial existence for Debian in the USA. I also represent users, developers, and Debian interests on a number of boards including at the Linux Foundation and the Freedombox Foundation. I m happily married with two children. Many people in Debian have met some or all of my family. They all joined me for Debconf in Edinburgh, and my daughter Elizabeth also attended in Caceres and New York. I joined Debian in 1994. I ve been responsible for a number of packages essential to our base system continuously since that time. But I ve also contributed to the project in many other ways over the years. I ran the first server that was fully dedicated to Debian. Ideas of mine influenced the development of project infrastructure, from the early design of our mirror network to structuring the archive around a package pool . I started or made significant early contributions to 5 ports of Debian to non-i386 architectures. I served as Debian Project Leader (DPL) in 2002-2003, was acting Secretary for a while, and have served on the Technical Committee for a number of years. Over the years, I ve also had some interesting hobbies. I helped design, build, and program pieces of various amateur radio satellites. I enjoy making physical things, and have many tools for working in wood and metals. My son and I are very active in the world of high power model rockets. And with my partner (and fellow Debian developer!) Keith Packard I m now running a small business making and selling open hardware and open source avionics for hobby rockets. You can read more about that at http://altusmetrum.org. You re the chair of the Debian technical committee. Can you quickly explain the role of this committee? I think many people assume the Technical Committee has a larger role in Debian than it really does. Section 6 of Debian s constitution defines the official role of the Technical Committee. Most importantly, the committee exists as a last resort place to resolve technical conflicts between Debian developers that they are unable to resolve by themselves. Most of the power in Debian is left in the hands of individual developers, who are usually able to collaborate with each other to make good technical decisions. So the Technical Committee s resolution process has only rarely been needed, which I think is a very good thing. From my point of view, the technical committee is not working. In many cases, the committee does not take any (timely) decision and just waits until the underlying situation has evolved to a point where the intervention of the committee is no longer needed. Do you agree with this and how can you explain it? I think it s very important for all of us to remember that everyone working on Debian does so voluntarily, and people who volunteer their time generally deserve a measure of respect and appreciation for their efforts. No issue is brought to the Technical Committee unless resolving it is expected to be really difficult, or at least contentious. And often, the issues brought to the committee have been as much or more about personality than technology. That makes some of them really hard to solve. So I do not agree that the technical committee is not working. It seems to me that the decisions that bog down and take a long time are the ones where arguments start out or become emotional instead of technical. In this context, if committee members can help lead public and private discussions in a way that causes a situation to evolve to the point where a decision is no longer needed, that may be healthier for the project in the long term than a quick vote that satisfies some contributors at the expense of others. The last important change that was made to try to revive the committee was the addition of two new members (Don Armstrong and Russ Allbery). Is there anything else that could be tried? The biggest improvement I could personally wish for is something people sending issues to the committee can help with. As the ultimate technical decision making body for a project whose output is mostly software, the more a request can be put in terms of a decision about source code, the easier it will be for us to make a decision. That won t always be possible, but when we re forced to try and dream up alternatives and then figure out whether anyone would actually be willing to write the code to implement those alternatives, the process takes a lot longer than choosing between competing patch sets or deciding whether a patch should be included. Besides your role in the technical committee, you have held the role of mediator/facilitator/advisor on numerous occasions. Because you re an old wise bearded guy who travels a lot and knows many Debian contributors I would like to thank you for all this work that few people notice. Are there been times where this has been a real burden for you? Thank you for mentioning this. I ve put a lot of my heart into Debian over the years, largely because it s a project and a community that continues to amaze and inspire me. I feel fortunate to have been able to meet and work on Debian with so many outstanding people from around the world. Many are now my friends, with all the silly and serious things being a friend implies. I ve been asked for and have given advice many times. I ve helped celebrate birthdays, marriages, new jobs, and the arrival of children. Sadly, I have also found myself having to try and find the right words to mark the loss of some of these friends The only time any of this feels like a burden is when there s some important problem that many people care about, that I m working behind the scenes to help fix, but can t really talk about publicly without causing more harm than good. It s distressing to have people think you don t care or aren t helping, when really you re doing everything you possibly can just not in a publicly visible way. Of course I understand that this is an impossible situation. If you can t see what s happening, there s no way to know if something is happening or not. That s why I advocate doing as much as possible in Debian, and SPI, and everywhere else I contribute in as open a way as possible. You have been Debian Project Leader and you promoted the vision of Debian as the Universal Operating System. What does universal mean for you? The biggest thing to me at the time was the idea that Debian could be anything. Those who chose to work on Debian would ultimately determine what Debian became. I also wanted to make sure we thought about as broad a set of potential users and collaborators as possible. But this vision provided a framework for pursuing a whole range of worthwhile increases in Debian s scope of utility, some of which I articulated in my DPL platforms, some others put forward. Internationalization, porting to more supported architectures, our inclusive and evolving approaches to accepting new developers and new packages, and so forth. I think this vision has served us well, and it pleases me that it has stayed a part of our collective thinking for so long. We re again in Debian s electoral period, what do you think of the work done by the current DPL? I m very happy with what I ve observed of Stefano s activities during his first year as DPL. He has an obvious enthusiasm for Debian, communicates well both in one to one interactions and in front of a crowd, and I think represents Debian very well. It is interesting that he s running unopposed for re-election this year. I choose to interpret that as evidence he s doing a good job, the project is running well, and nobody feels the need to try and take the job away From him. I m glad he s willing to continue in this role for another year. What s the most important thing that Debian should achieve in the wheezy timeframe? I don t yet have a very crisp personal wish-list for wheezy. But I would certainly like to see multiarch support finally completed! I m also very interested to see what comes from the CUT work. You have been an early supporter of multiarch , a project to allow easy installation of foreign architecture packages. It s on good track for Wheezy. Do you think it s an important milestone? My original motivation for requesting multiarch support was to enable support for 32-bit x86 binaries on ia64 Itanium systems, in the time leading up to the sarge release. I ended up creating the ia32-libs package, which I m not proud of. The emergence of 64-bit extensions to x86 (the amd64 architecture) made this a much broader issue. Today, I run a 64 bit kernel and a 32 bit user space on my notebook. There are problems with just moving entirely to 64 bit but I would like to be able to run some applications that work with large data sets in full 64 bit mode!
Thank you to Bdale Garbee for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

4 comments Liked this article? Click here. My blog is Flattr-enabled.

Peter Van Eynde: how to grow a zfs file system

I'm experimenting with zfs at home, for the moment on top of my md/lvm setup, and I ran out of disk space. Growing the lv is pretty easy:

frost:~# lvextend --size +110G /dev/new-vg/zfs-test
  Extending logical volume zfs-test to 120.00 GiB
  Logical volume zfs-test successfully resized
frost:~# zpool list
NAME       SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
zfs-pool  9.94G  9.78G   161M    98%  1.00x  ONLINE  -


Hmm it did not notice the 110GB extra, so I did:

frost:~# zpool export zfs-pool
frost:~# zpool import zfs-pool
frost:~# zpool list
NAME       SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
zfs-pool   120G  9.78G   110G     8%  1.00x  ONLINE  -


so simply doing an import/export is enough.

I'm looking at zfs to have a better idea of what btrfs will mean in the future for us.

This entry was originally posted at http://pvaneynd.dreamwidth.org/139208.html. Please comment there using OpenID.

26 March 2011

Dirk Eddelbuettel: Branford Marsalis and Terence Blanchard

Wonderful jazz concert at CSO yesterday in classic double-header plus intermission format. First up was Branford Marsalis (ss,ts) with his quartet featuring Joey Calderazzo (p), Eric Revis (b) and Justin Faulkner (dr) and they simply blew the roof off the place. Best set I heard in a while: dynamic, joyful, playful, modern, whathaveyou. Really really nice. The second set was the Terence Blanchard (tp) quintet featuring Brice Winston (ts), Fabian Almazan (p), Joe Sanders (b) and Kendrick Scott (dr) who also played a nice set, but as far as I am concerned did not gel as a much as the spectacular first set. Great evening.

Thorsten Glaser: Various joys

I m online again. (In case you didn t notice, duh ) Seems as if we (the Telco/ISP guy and me) just needed to look at it hard enough for it to go away first he could dial in, using my account data, which I probably should change now, then herc with ppp(8) and pppoe(8) was working (although at about 50 KiB/s down, he showed me 508 KiB/s a rate I had never achieved with his WiXP), then I took my notebook, which worked with pppoe(4). Now herc s working again. (Maybe altq(9) can explain the slowdown? Hm, from debian.netcologne.de I get 500 so it looks okay.)
But eurynome isn t, oh the joy. Luckily, gecko2 who administers its host system just woke up. Things we do want to see: the Telco/ISP guy accepting that I run MirBSD on a P-233MMX box with Hercules graphics card and a 9 monitor with no comment other than considering its age (and that it usually runs 24/7) as partial cause for the bug. Thanks, Netcologne! Things we don t want to see:
Mar 26 10:40:02 blau /bsd: signal 11 received by (screen:16857) UID(2999) EUID(2999), parent (screen:19111) UID(2999) EUID(2999)
Suddenly the Dungeon collapses!! - You die... (luckily, I get it about once a year only) ObCoffeeSpices: Marrakech (Cumin, Allspice, Cumin Aroma) though, due to its relative strength compared to the others, the only coffee spice I have left. And another hint: pre-warming the coffee cup with hot water, so it doesn t cool down too fast with the amounts of milk I put in, rocks. I just wore the Squeeze Release (FOSDEM, Spacefun) T-Shirt to the bakery and got asked by a neighbour: Oh, a Debian fan? Developer, even now imagine the typical informed interested guy talk for a conference booth of your OS of choice here. How proud he was to get his wife and himself Windows -free at home; how he likes to tinker a bit (if he s got any time left), which has become harder with Windows; how his time constraints have him at OpenSuSE currently but asked how squeeze is; and the usual complaints at places like $ork where they have to use Windows and MSIE (apparently you can t centrally manage Firefox, eh, good someone tells me, because that s what we do ). Wow. Anyway, it s spring, so people, wear your shirts. (Hrm, what do I make of the fact that this is my only Debian shirt although I m thinking how to get Tartan Trousers if money were no issue and nobody had ever commented on my various BSD, FOSDEM, FrOSCon, etc. wear )

25 March 2011

Amaya Rodrigo: Friday cat blogging


Vi & Clara
Originally uploaded by Amayita.

Dirk Eddelbuettel: R inside Qt: A simple RInside application

The RInside package makes it pretty simple and straightforward to embed R, the wonderful statistical programming environment and language, inside of a C++ application. This uses both the robust embedding API provided by R itself, and the higher-level abstractions from our Rcpp package. A number of examples are shown on this blog both here and here as well as on the RInside page; and the source package actually contains well over a dozen complete examples which cover anything from simple examples to parallel use via MPI for parallel computing. Beginning users sometimes ask about how to use RInside inside larger projects. And as I had meant to experiment with embedding inside of the powerful Qt framework anyway, I started to dabble a little. A first result is now in the SVN sources of RInside. My starting point was the classic tkdensity demo that comes with R itself. It is a good point of departure as Tcl/Tk makes it very portable---in fact it should run on every platform that runs R---and quite expressive. And having followed some of the GUI experiments around R over the years, I have also seen various re-implementations using different GUI frameworks. And so I am adding mine to this body of work: Example of embedding R via RInside into a Qt C++ application: density estimation for a mixture The problem I addressed first was actual buildability. For the RInside examples, Romain and I provide a Makefile that just works by making calls to R itself to learn about flags for R, Rcpp and RInside such that all required headers and libraries are found. That is actually relatively straightforward (and documented in our vignettes) but a little intimidating at first---which is why a ready-made Makefile is a good thing. Qt of course uses qmake and the .pro files to encode / resolve dependencies. So task one was to map what our Makefile does into its variables. Turns out that wasn't all that hard:
## -*- mode: Makefile; c-indent-level: 4; c-basic-offset: 4;  tab-width: 8; -*-
##
## Qt usage example for RInside, inspired by the standard 'density
## sliders' example for other GUI toolkits
##
## Copyright (C) 2011  Dirk Eddelbuettel and Romain Francois
TEMPLATE =              app
HEADERS =               qtdensity.h 
SOURCES =               qtdensity.cpp main.cpp
QT +=                   svg
## comment this out if you need a different version of R, 
## and set set R_HOME accordingly as an environment variable
R_HOME =                $$system(R RHOME)
## include headers and libraries for R 
RCPPFLAGS =             $$system($$R_HOME/bin/R CMD config --cppflags)
RLDFLAGS =              $$system($$R_HOME/bin/R CMD config --ldflags)
RBLAS =                 $$system($$R_HOME/bin/R CMD config BLAS_LIBS)
RLAPACK =               $$system($$R_HOME/bin/R CMD config LAPACK_LIBS)
## if you need to set an rpath to R itself, also uncomment
#RRPATH =               -Wl,-rpath,$$R_HOME/lib
## include headers and libraries for Rcpp interface classes
RCPPINCL =              $$system($$R_HOME/bin/Rscript -e \'Rcpp:::CxxFlags\(\)\')
RCPPLIBS =              $$system($$R_HOME/bin/Rscript -e \'Rcpp:::LdFlags\(\)\')
## for some reason when building with Qt we get this each time
## so we turn unused parameter warnings off
RCPPWARNING =           -Wno-unused-parameter 
## include headers and libraries for RInside embedding classes
RINSIDEINCL =           $$system($$R_HOME/bin/Rscript -e \'RInside:::CxxFlags\(\)\')
RINSIDELIBS =           $$system($$R_HOME/bin/Rscript -e \'RInside:::LdFlags\(\)\')
## compiler etc settings used in default make rules
QMAKE_CXXFLAGS +=       $$RCPPWARNING $$RCPPFLAGS $$RCPPINCL $$RINSIDEINCL
QMAKE_LFLAGS +=         $$RLDFLAGS $$RBLAS $$RLAPACK $$RCPPLIBS $$RINSIDELIBS
## addition clean targets
QMAKE_CLEAN +=          qtdensity Makefile
The double dollar signs and escaping of parentheses are a little tedious, but hey it works and expands the compiler and linker flags such that everything <emjust works="Works">. The code itself is pretty straightforward too. We instantiate the RInside object as well as the main Qt application object. We then instantiate a new object of class QtDensity that will launch the main widget; it is given a reference to the RInside object.
// -*- mode: C++; c-indent-level: 4; c-basic-offset: 4;  tab-width: 8; -*-
//
// Qt usage example for RInside, inspired by the standard 'density
// sliders' example for other GUI toolkits
//
// Copyright (C) 2011  Dirk Eddelbuettel and Romain Francois
#include <QApplication>
#include "qtdensity.h"
int main(int argc, char *argv[])
 
    RInside R(argc, argv);  		// create an embedded R instance
    QApplication app(argc, argv);
    QtDensity qtdensity(R);
    return app.exec();
 
The definition of the main object is pretty simple: a few private variables, and a few functions to interact with the GUI and get values from the radio buttons, slider or input field---as well as functions to update the chart or re-draw the random variables.
// -*- mode: C++; c-indent-level: 4; c-basic-offset: 4;  tab-width: 8; -*-
//
// Qt usage example for RInside, inspired by the standard 'density
// sliders' example for other GUI toolkits
//
// Copyright (C) 2011  Dirk Eddelbuettel and Romain Francois
#ifndef QTDENSITY_H
#define QTDENSITY_H
#include <RInside.h>
#include <QMainWindow>
#include <QHBoxLayout>
#include <QSlider>
#include <QSpinBox>
#include <QLabel>
#include <QTemporaryFile>
#include <QSvgWidget>
class QtDensity : public QMainWindow
 
    Q_OBJECT
public:
    QtDensity(RInside & R);
private slots:
    void getBandwidth(int bw);
    void getKernel(int kernel);
    void getRandomDataCmd(QString txt);
    void runRandomDataCmd(void);
private:
    void setupDisplay(void);    // standard GUI boilderplate of arranging things
    void plot(void);            // run a density plot in R and update the
    void filterFile(void);      // modify the richer SVG produced by R
    QSvgWidget *m_svg;          // the SVG device
    RInside & m_R;              // reference to the R instance passed to constructor
    QString m_tempfile;         // name of file used by R for plots
    QString m_svgfile;          // another temp file, this time from Qt
    int m_bw, m_kernel;         // parameters used to estimate the density
    QString m_cmd;              // random draw command string
 ;
#endif
Lastly, no big magic in the code either (apart from the standard magic provided by RInside). A bit of standard GUI layouting, and then some functions to pick values from the inputs as well as to compute / update the output. One issue is worth mentioning. The screenshot and code show the second version of this little application. I built a first one using a standard portable network graphics (png) file. That was fine, but not crisp as png is a pixel format so I went back and experimented with scalable vector graphics (svg) instead. One can create svg output with R in a number of ways, one of which is the cairoDevice package by Michael Lawrence (who also wrote RGtk2 and good chunks of Ggobi). Now, it turns out that Qt displays the so-called SVG tiny standard whereas R creates a fuller SVG format. Some discussion with Michael reveals that one can modify the svg file suitably (which is what the function filterFile below does) and it all works. Well: almost. There is a bug (and Michael thinks it is the SVG rendering) in which the density estimate does not get clipped to the plotting region.
// -*- mode: C++; c-indent-level: 4; c-basic-offset: 4;  tab-width: 8; -*-
//
// Qt usage example for RInside, inspired by the standard 'density
// sliders' example for other GUI toolkits -- this time with SVG
//
// Copyright (C) 2011  Dirk Eddelbuettel and Romain Francois
#include <QtGui>
#include "qtdensity.h"
QtDensity::QtDensity(RInside & R) : m_R(R)
 
    m_bw = 100;                 // initial bandwidth, will be scaled by 100 so 1.0
    m_kernel = 0;               // initial kernel: gaussian
    m_cmd = "c(rnorm(100,0,1), rnorm(50,5,1))"; // simple mixture
    m_R["bw"] = m_bw;           // pass bandwidth to R, and have R compute a temp.file name
    m_tempfile = QString::fromStdString(Rcpp::as<std::string>(m_R.parseEval("tfile <- tempfile()")));
    m_svgfile = QString::fromStdString(Rcpp::as<std::string>(m_R.parseEval("sfile <- tempfile()")));
    m_R.parseEvalQ("library(cairoDevice)");
    setupDisplay();
 
void QtDensity::setupDisplay(void)   
    QWidget *window = new QWidget;
    window->setWindowTitle("Qt and RInside demo: density estimation");
    QSpinBox *spinBox = new QSpinBox;
    QSlider *slider = new QSlider(Qt::Horizontal);
    spinBox->setRange(5, 200);
    slider->setRange(5, 200);
    QObject::connect(spinBox, SIGNAL(valueChanged(int)), slider, SLOT(setValue(int)));
    QObject::connect(slider, SIGNAL(valueChanged(int)), spinBox, SLOT(setValue(int)));
    spinBox->setValue(m_bw);
    QObject::connect(spinBox, SIGNAL(valueChanged(int)), this, SLOT(getBandwidth(int)));
    QLabel *cmdLabel = new QLabel("R command for random data creation");
    QLineEdit *cmdEntry = new QLineEdit(m_cmd);
    QObject::connect(cmdEntry,  SIGNAL(textEdited(QString)), this, SLOT(getRandomDataCmd(QString)));
    QObject::connect(cmdEntry,  SIGNAL(editingFinished()), this, SLOT(runRandomDataCmd()));
    QGroupBox *kernelRadioBox = new QGroupBox("Density Estimation kernel");
    QRadioButton *radio1 = new QRadioButton("&Gaussian");
    QRadioButton *radio2 = new QRadioButton("&Epanechnikov");
    QRadioButton *radio3 = new QRadioButton("&Rectangular");
    QRadioButton *radio4 = new QRadioButton("&Triangular");
    QRadioButton *radio5 = new QRadioButton("&Cosine");
    radio1->setChecked(true);
    QVBoxLayout *vbox = new QVBoxLayout;
    vbox->addWidget(radio1);
    vbox->addWidget(radio2);
    vbox->addWidget(radio3);
    vbox->addWidget(radio4);
    vbox->addWidget(radio5);
    kernelRadioBox->setMinimumSize(260,140);
    kernelRadioBox->setMaximumSize(260,140);
    kernelRadioBox->setSizePolicy(QSizePolicy::Fixed, QSizePolicy::Fixed);
    kernelRadioBox->setLayout(vbox);
    QButtonGroup *kernelGroup = new QButtonGroup;
    kernelGroup->addButton(radio1, 0);
    kernelGroup->addButton(radio2, 1);
    kernelGroup->addButton(radio3, 2);
    kernelGroup->addButton(radio4, 3);
    kernelGroup->addButton(radio5, 4);
    QObject::connect(kernelGroup, SIGNAL(buttonClicked(int)), this, SLOT(getKernel(int)));
    m_svg = new QSvgWidget();
    runRandomDataCmd();         // also calls plot()
    QGroupBox *estimationBox = new QGroupBox("Density estimation bandwidth (scaled by 100)");
    QHBoxLayout *spinners = new QHBoxLayout;
    spinners->addWidget(spinBox);
    spinners->addWidget(slider);
    QVBoxLayout *topright = new QVBoxLayout;
    topright->addLayout(spinners);
    topright->addWidget(cmdLabel);
    topright->addWidget(cmdEntry);
    estimationBox->setMinimumSize(360,140);
    estimationBox->setMaximumSize(360,140);
    estimationBox->setSizePolicy(QSizePolicy::Fixed, QSizePolicy::Fixed);
    estimationBox->setLayout(topright);
    QHBoxLayout *upperlayout = new QHBoxLayout;
    upperlayout->addWidget(kernelRadioBox);
    upperlayout->addWidget(estimationBox);
    QHBoxLayout *svglayout = new QHBoxLayout;
    svglayout->addWidget(m_svg);
    QVBoxLayout *outer = new QVBoxLayout;
    outer->addLayout(upperlayout);
    outer->addLayout(svglayout);
    window->setLayout(outer);
    window->show();
 
void QtDensity::plot(void)  
    const char *kernelstrings[] =   "gaussian", "epanechnikov", "rectangular", "triangular", "cosine"  ;
    m_R["bw"] = m_bw;
    m_R["kernel"] = kernelstrings[m_kernel]; // that passes the string to R
    std::string cmd1 = "Cairo(width=6,height=6,pointsize=10,surface='svg',filename=tfile); "
                       "plot(density(y, bw=bw/100, kernel=kernel), xlim=range(y)+c(-2,2), main=\"Kernel: ";
    std::string cmd2 = "\"); points(y, rep(0, length(y)), pch=16, col=rgb(0,0,0,1/4));  dev.off()";
    std::string cmd = cmd1 + kernelstrings[m_kernel] + cmd2; // stick the selected kernel in the middle
    m_R.parseEvalQ(cmd);
    filterFile();               // we need to simplify the svg file for display by Qt 
    m_svg->load(m_svgfile);
 
void QtDensity::getBandwidth(int bw)  
    if (bw != m_bw)  
        m_bw = bw;
        plot();
     
 
void QtDensity::getKernel(int kernel)  
    if (kernel != m_kernel)  
        m_kernel = kernel;
        plot();
     
 
void QtDensity::getRandomDataCmd(QString txt)  
    m_cmd = txt;
 
void QtDensity::runRandomDataCmd(void)  
    std::string cmd = "y <- " + m_cmd.toStdString();
    m_R.parseEvalQ(cmd);
    plot();                     // after each random draw, update plot with estimate
 
void QtDensity::filterFile()  
    // cairoDevice creates richer SVG than Qt can display
    // but per Michaele Lawrence, a simple trick is to s/symbol/g/ which we do here
    QFile infile(m_tempfile);
    infile.open(QFile::ReadOnly);
    QFile outfile(m_svgfile);
    outfile.open(QFile::WriteOnly   QFile::Truncate);
    
    QTextStream in(&infile);
    QTextStream out(&outfile);
    QRegExp rx1("<symbol"); 
    QRegExp rx2("</symbol");    
    while (!in.atEnd())  
        QString line = in.readLine();
        line.replace(rx1, "<g"); // so '<symbol' becomes '<g ...'
        line.replace(rx2, "</g");// and '</symbol becomes '</g'
        out << line << "\n";
     
    infile.close();
    outfile.close();
 
What the little application does is actually somewhat neat for the few lines. One key features is that the generated data can be specified directly by an R expression which allows for mixtures (as shown, and as is the default). With that it easy to see how many points are needed in the second hump to make the estimate multi-modal, and how much of a distance between both centers is needed and so on. Obviously, the effect of the chosen kernel and bandwidth can also be visualized. And with the chart the being a support vector graphics display, we can resize and scale at will and it still looks crisp. The code (for both the simpler png variant and the svg version shown here) is in the SVN repository for RInside and will be in the next release. Special thanks to Michael Lawrence for patiently working through some svg woes with me over a few emails.

Update: Some typos fixed.

Update 2: Two URLs corrected.

Joey Hess: getting out of cell phone contracts, redux

Exactly a year after I switched from pre to droid, I broke another cell phone contract. So there seem to be three fool-proof ways to get out of a cell phone contract in the US, despite all the hype about them involving signing away your firstborn.
  1. Switch to a different US cell phone company, and do it by going into a store and finding a hungry salesman in person, and making his sale contingent on him finding a way to break your old contract. What I did last year.
  2. Simply move to a part of the map that doesn't have cell phone coverage. (You will have seen these parts of the map from a plane.) It probably helps to have your cell phone drop off the edge of their map repeatedly for some months, presumably they can look this stuff up. Call up technical support, go thru some mild rigamarole as they try to configure your phone to magically see cell towers that have several mountains and/or the curvature of the Earth blocking line of site to you. Magic words: "please note on account: no cell service"
  3. Sell your contract on one of the existing cell contract exchanges, to someone who wants cell service with a shorter than usual contract. (Means giving up your number.) I haven't tried this one, maybe next time?
Unlike my Palm Pre which didn't survive being cut off the cell net, my Android phone is still fairly happy. While I'd really like to turn it into a proper Debian box, with X etc, I may just use it as a camera, web browsing (on wifi) and emergency 911 system. Pity though about the GPS stuff needing an internet connection when the phone has many gigabytes of free space that could hold a useful map. (If you want to phone me now, the number is 1-423-IKIWIKI. I only receive voice mail, and your call data and transcript will be absorbed by the Google Borg.)

Joey Hess: weird git tricks

I wrote this code today to verify setup branch pushes on Branchable. When I was writing it I was just down in the trenches coding until it worked, but it's rather surprising that what it does to git does work. The following code runs in git's update hook. The fast path of the hook (written in C) notices that the user is committing a change to the setup branch, and hands the incoming git ref off to the setup verifier.
                # doing a shared clone makes the setupref, which has
                # not landed on any branch, be available for checkout
                shell("git", "clone", "--quiet", "--shared",
                        "--no-checkout", repository($hostname), $tmpcheckout);
                chdir($tmpcheckout)   error "chdir $tmpcheckout: $!";
                shell("git", "checkout", "--quiet", $setupref, "-b", "setup");
I got lucky here, since I initially passed --shared only to avoid the overhead of a clone of the site's entire git repository (which can be quite large, since Branchable doesn't have any real limits on site size). Without the --shared, the clone wouldn't see the incoming ref at all. In the setup branch is an ikiwiki.setup file, and we only want to allow safe changes to be committed to it. Ikiwiki has metadata about which configurations are safe. Checking that and various other amusing scenarios (what if someone makes ikiwiki.setup a symlink etc) takes a hundred lines of fairly hairy code, but that doesn't matter here. Eventually it decides the setup file is ok as-is, or it's already died with an error message.
                # Check out setup file in toplevel. This is slightly tricky
                # as the commit has not landed in the bare git repo yet --
                # but it is available in the tmpcheckout.
                shell("git", "pull", "-q", $tmpcheckout, "setup");
                # Refresh or rebuild site to reflect setup changes.
                print STDERR "Updating site to reflect setup changes...\n";
                shell("ikiwiki", "-setup", "ikiwiki.setup", "-v",
                        ($rebuild_needed ? ("-rebuild") : ("-refresh", "-wrappers"))
                );
When this code runs there are three repositories, each with a different view of the setup branch. The main bare repository is waiting for the hook to succeed before it updates the ref to point to what was pushed. The temporary clone has what was pushed already checked out. And the site's home directory still has the old version of the setup branch checked out. Possibly even a version that has diverged from what's in the bare repository. It's rather odd that the update hook goes and causes that latter repository to be updated, before the change has finished landing in the bare repository. But it does work; it ensures that if there is some kind of bizzare merge problem the user doing the push sees it, and I probably won't regret it. The result certianly is nice -- edit ikiwiki.setup file locally, commit and push it, and ikiwiki automatically reconfigures itself and even rebuilds your whole site if you've changed something significant.

23 March 2011

Kumar Appaiah: Review of "Linux Shell Scripting Cookbook"

I have been reading Linux Shell Scripting Cookbook by Sarath Lakshman, published by Packt, for a while. While most people I know learn shell scripts themselves, I was looking to refresh my concepts a little as well as have a reference lying around on the table for fast access. Since I was asked to review it for someone else, I thought I might as well post the review here.First of all, let me remark by saying that shell scripting is something learned more on a need basis than as a tool to solve the main problem. People would seldom write shell scripts as standalone programs (exceptions exist). However, what makes shell scripting invaluable to know is the fact that knowing some tricks can save several minutes, or hours, of work by automating and simplifying certain tasks, generally (but not restricted to) file management and data processing. Linux Shell Scripting Cookbook does go quite far in pursuing this goal, and is appropriate for both beginners who are looking to gain dexterity in shell scripting, as well as intermediate users who wish to polish their skills. The book also can double up as a quick reference, though I would argue that the "Advanced Bash Scripting Guide" would suit that more.At the outset, the author clarifies that the focus will be on Bash. This, people may or may not like, but the fact that bash has become ubiquitous in terms of the available shells on Unix-like systems today, starting out with bash is not a bad thing to do. Besides, learning other shell scripting languages while knowing bash isn't too hard, since the paradigm remains the same.Let's go through the books from the aspects I found most relevant.OrganizationThe book is organized into chapters based more on utility than scripting concepts themselves, although the language aspects are brought onto the reader gradually. For instance, the examples in the first chapter focus more on the basic data elements (variables, arrays, functions etc.) as well as operators (for numbers, files etc.), and all the examples demonstrate simple usage of these concepts, and he further chapters build upon these in a gradual manner.At the same time, if the reader has some familiarity with shell scripting and needs to only refresh or learn a certain concept, he/she needs to just read the relevant chapter. It is not too difficult to grasp the examples of the later chapters, provided some basic shell knowledge is assumed.Content and PresentationA positive trait in the presentation of this book is that it is all based on practical everyday examples which, with minor adaptation, can be used by many for their own daily tasks. For instance, there are several examples which describe searching for and processing files, which, I'd imagine, many users would want to do on a regular basis. Thus, providing realistic examples allows the book to double its utility. The language and approach used is simple and conversational, and the presentation is very clear, with each idea being described as a problem statement followed by a "How to do it" section with the actual code, and ending with a discussion of the nitty-gritties of the code. It is easy to go for a quick scan for those in a hurry, while those who with to read in more detail will not be disappointed either.The book also covers a wide array of applications. For instance, there are examples on automating fetching web pages and processing them, demonstrations of parsing and simplifying and even some queries around databases wrapped around in shell. It also spans to utilities and tasks connected to statistics, backups, compression, version control and many more.Breadth and DepthThe book goes into a fair amount of detail in terms of describing the shell scripting concept under consideration. The examples used go into a fair amount of detail in order to describe to the user all the aspects involved in the method or command being used. The concepts described are fairly complete, and would be sufficient for the reader to use immediately or with just a little bit of fine tuning. In terms of breadth, the book covers most of the features of shell scripting while also describing the various facilities the shell provides access to in a Unix-like environment. Thus, the book does not disappoint in this front either.In summary...In summary, probably the only thing I'd have liked to see more of is some emphasis on how to write more efficient shell scripts. Granted, most of the shell scripts described in the book are very simple and succinct, but a some words on how loops can be made better, or how to spot situations where pipes are not needed to solve a problem etc. might have been a nice addition. Some explanation of differences with dash, tcsh, zsh etc. might also have been nice, since a lot of users have different default shells. But all this isn't going to prevent me from giving this book a high rating, since it delivers quite well on the promises it makes at the beginning.This is definitely a good book to have near your desk, and kudos to the author for having taken the effort to put it together. I would highly recommend it to the beginner and occasional shell user for a thorough read, and to an intermediate to have on his/her desk for borrowing the cool scripting ideas and applications the author has written in this book.

Kapil Paranjape: John Willard Milnor: Abel Prize 2011

The Abel prize for 2011 has been awarded to John Willard Milnor.[1] In response to a query from Dr. R. Ramachandran who writes about Science and Science Policy for The Hindu'' newspaper group, I wrote the following short piece which others may find interesting as well. Milnor is well-known for his work on topology and geometry. He has also made significant contributions to algebra and even number theory. In addition, he has written a number of books which are loved by graduate students in Mathematics all over the world. One of the striking early results of Milnor was the example he gave of a seven dimensional space which is topologically a sphere but its geometric (differentiable) structure is different. This was the first example of an "exotic sphere". A nice way to state his main result (due to Ajit Sanzgiri) is that "Groups of homotopy spheres are homotopy groups of spheres". Milnor received the Fields' Medal in 1962. In addition, the work of a number of later Fields' medallists such as Donaldson, Thurston, Mori and Voevodsky can be seen as having roots in the work of Milnor.
A more personal perspective The first time I came across the name Milnor was when I heard that the only dimensions in which one can do algebra with division is 1, 2, 4 and 8; I was told that an "easy" proof was based on Characteristic Classes on which Milnor had written a nice book. In later years, I read a number of his other books like Topology from a differentiable viewpoint'', Morse theory'', Isolated singularities of complex hypersurfaces'' and Algebraic K-theory''. These books not only explained the results and definitions, but laid the foundations of my geometric intuition --- the same is probably true for many others in my generation. When I joined TIFR, Raghunathan was full of praise for the work of Milnor and how his deep ideas on differential topology would "lead somewhere". One of the first lectures in our graduate seminar was by Ajit Sanzgiri on Milnor's paper on exotic spheres --- the title of the talk was Groups of homotopy spheres are homotopy groups of spheres''. When Srinivas taught me (algebraic) K-theory, the only available reference text was Milnor's book (Milnor K-theory forms a crucial component of Voevodsky's early 21st century work that won him his Fields Medal!); since then Srinivas has written a more modern and comprehensive book on the topic. Later, when A. J. Parameswaran started work on his Ph.D. under the guidance of Srinivas, AJP and I read Milnor's book on isolated singularities together --- as a prelude to the sequel to Milnor's book (by Looijenga). Much of the modern work on the algebraic theory of singularities (which forms a crucial component of Mori's Fields medal winning work on terminal three dimensional singularities) starts with the notion of "Milnor number" and "Milnor fibrations". If the impact of a mathematician is to be measured not only by his own fantastic results but the great results of others that grow out of his work, then Milnor is certainly one of the greats of the latter half of the twentieth century. Much of the topology and geometry that I have used in my work relies on simpler versions of Milnor's results that were proved by his predecessors. So (unfortunately!) I cannot quote a result that I have proved which actually uses a theorem of Milnor's.
Groups of homotopy spheres are homotopy spheres'' I think the idea of joining two spaces by a tube to make a new space (called connected sum) is due to Whitney or Kervaire. This leads the "algebra of spaces" or to give it its mathematical name "the cobordism group". Milnor showed us how to perform calculations with this group. Another group studied by topologists combines the different ways in which a sphere of one dimension wraps around'' a sphere of a (possibly) different dimension; this is called a homotopy group of the second sphere. Milnor's brilliant idea was to show that in certain cases, the two groups obtained are the same. Said pithily, "Groups of homotopy spheres (exotic spheres)" are the same as "homotopy groups of spheres".
Some remarks A number of people have remarked that what we need in order for institutes like IISER to grow are teaching researchers''. The system of universities in the USA has thrown up a few shining examples --- Richard Feynman in Physics and John Milnor in Mathematics are two names that immediately spring to mind.

[1]The last time I wrote about the Abel Prize was to contribute to a guest post to Rahul Basu's science blog: The Far Side. Unfortunately, Rahul is no longer among us. This post is dedicated to the memory of Rahul Basu.

22 March 2011

Pietro Abate: Xen brouter setup

A while ago I received a new desktop machine (8 cores, 8Gb of memory ...) at work. Since for the moment I kinda happy to work on my laptop using an external screen, I decided to put the hw to a good use and to explore a bit more some more exotic (at least for me) xen features. In particular I spend half a day playing with different xen network settings. The bridge model, that should work out of the shelf, is the easiest one. To setup this up, you basically need to specify a couple of options in the xend-config file and you're done. This is the "default" network configuration and is should work out of the box in most situations. Using this method, since all VMs' interfaces are bridged together (surprise !) with the public interface, your network card is left in promiscuous mode (not a big problem if you ask me...). Once your VMs are up, you can then decided to use your default dhcp server, autoconf your VMs with ipv6 only, or do nothing as you please. An other popular method, albeit a bit more complex, is to setup a natted network using the script network-nat (this one is an evolution of the third method that is 'network-routed') . I played with it, but since I wanted to have all my DomU on the same subnet, this setup wasn't satisfying for me. In particular, by default, 'network-nat' assigns a different subnet to each DomU. Using the natted set up you can also configure a local dhcp server to give private IPs to your VMs all done transparently by the xen network scripts. I've noticed that there is a bug in the xen script that does not make it very squeeze friendly. Since the default dhcp server in squeeze is isc-dhcp and few configuration files got shuffled in the process (notably /etc/dhcp3.conf is not /etc/dhcp/dhcp.conf) , the script needs a little fix to work properly. I'll report this bug sometimes soon... Goggling around I found a different setup that is called brouter, that is a hybrid between a bridge configuration and a routed configuration. This is the original (??) article well hidden in an old suse wiki.
 
I've done few modifications here to add natting. So basically, all virtual interfaces connected each to one DomU are linked together by a bridge (xenbr0). The bridge with address 10.0.0.1 is also the router of the subnet. All DomU are configured to used dhcp that assigns a new ip and specifies the router of the subnet. The dhcp server is configured to answer requests only on the xenb0 interface avoiding problems on the public network... routing is configured using iptables :
iptables -t nat -A POSTROUTING -o $ netdev -j MASQUERADE
iptables -A FORWARD -i $ bridge -j ACCEPT
echo 1 >/proc/sys/net/ipv4/ip_forward
/etc/init.d/isc-dhcp-server restart
Note that since the dhcp server is configured to give addresses only on the virtual network, we need to restart it after creating the bridge interface, otherwise isc-dhcp-server will refuse to run. Mum says that I should configure the bridge in /etc/network/interfaces to make the dhcp server happy at startup, but I felt a bit lazy, so I let to task to xen... In the next episode, I'll add ipv6 connectivity to the virtual subnet and then start playing with puppet... ipv6 is almost done, puppet... I started with the doc... The complete script from the suse wiki and with my modifications is below (only lightly tested):
#!/bin/sh
#============================================================================
# Default Xen network start/stop script.
# Xend calls a network script when it starts.
# The script name to use is defined in /etc/xen/xend-config.sxp
# in the network-script field.
#
# This script creates a bridge (default xenbr$ vifnum ), gives it an IP address
# and the appropriate route. Then it starts the SuSEfirewall2 which should have
# the bridge device in the zone you want it.
#
# If all goes well, this should ensure that networking stays up.
# However, some configurations are upset by this, especially
# NFS roots. If the bridged setup does not meet your needs,
# configure a different script, for example using routing instead.
#
# Usage:
#
# vnet-brouter (start stop status) VAR=VAL *
#
# Vars:
#
# bridgeip Holds the ip address the bridge should have in the
# the form ip/mask (10.0.0.1/24).
# brnet Holds the network of the bridge (10.0.0.1/24).
#
# vifnum Virtual device number to use (default 0). Numbers >=8
# require the netback driver to have nloopbacks set to a
# higher value than its default of 8.
# bridge The bridge to use (default xenbr$ vifnum ).
#
# start:
# Creates the bridge
# Gives it the IP address and netmask
# Adds the routes to the routing table.
#
# stop:
# Removes all routes from the bridge
# Removes any devices on the bridge from it.
# Deletes bridge
#
# status:
# Print addresses, interfaces, routes
#
#============================================================================

#set -x

dir=$(dirname "$0")
. "$dir/xen-script-common.sh"
. "$dir/xen-network-common.sh"

findCommand "$@"
evalVariables "$@"

vifnum=$ vifnum:-0
bridgeip=$ bridgeip:-10.6.7.1/24
brnet=$ brnet:-10.6.7.0/24
netmask=$ netmask:-255.255.255.0
bridge=$ bridge:-xenbr$ vifnum

##
# link_exists interface
#
# Returns 0 if the interface named exists (whether up or down), 1 otherwise.
#
link_exists()

if ip link show "$1" >/dev/null 2>/dev/null
then
return 0
else
return 1
fi



# Usage: create_bridge bridge
create_bridge ()
local bridge=$1

# Don't create the bridge if it already exists.
if [ ! -d "/sys/class/net/$ bridge /bridge" ]; then
brctl addbr $ bridge
brctl stp $ bridge off
brctl setfd $ bridge 0
fi
ip link set $ bridge up


# Usage: add_to_bridge bridge dev
add_to_bridge ()
local bridge=$1
local dev=$2
# Don't add $dev to $bridge if it's already on a bridge.
if ! brctl show grep -wq $ dev ; then
brctl addif $ bridge $ dev
fi


# Usage: show_status dev bridge
# Print interface configuration and routes.
show_status ()
local dev=$1
local bridge=$2

echo '============================================================'
ip addr show $ dev
ip addr show $ bridge
echo ' '
brctl show $ bridge
echo ' '
ip route list
echo ' '
route -n
echo '============================================================'
echo ' '
iptables -L
echo ' '
iptables -L -t nat
echo '============================================================'



op_start ()
if [ "$ bridge " = "null" ] ; then
return
fi

create_bridge $ bridge

if link_exists "$bridge"; then
ip address add dev $bridge $bridgeip
ip link set $ bridge up arp on
ip route add to $brnet dev $bridge
fi

if [ $ antispoof = 'yes' ] ; then
antispoofing
fi
iptables -t nat -A POSTROUTING -o $ netdev -j MASQUERADE
iptables -A FORWARD -i $ bridge -j ACCEPT
echo 1 >/proc/sys/net/ipv4/ip_forward
/etc/init.d/isc-dhcp-server restart


op_stop ()
if [ "$ bridge " = "null" ]; then
return
fi
if ! link_exists "$bridge"; then
return
fi

ip route del to $brnet dev $bridge
ip link set $ bridge down arp off
ip address del dev $bridge $bridgeip
##FIXME: disconnect the interfaces from the bridge 1st
brctl delbr $ bridge
/etc/init.d/isc-dhcp-server restart


case "$command" in
start)
op_start
;;

stop)
op_stop
;;

status)
show_status $ netdev $ bridge
;;

*)
echo "Unknown command: $command" >&2
echo 'Valid commands are: start, stop, status' >&2
exit 1
esac

Michal &#268;iha&#345;: phpMyAdmin translation status

We're heading to release candidates for phpMyAdmin 3.4.0, so the translators have less time to make the translations better. There are still 13 translations at 100%: There are still lot of translations which would need some love, for example: Recently, new people started to work on Bulgarian and Brazilian Portuguese translations, so we will hopefully have these two on the list soon as well. You can help make translation to your language better! If your language is missing just let me know (either in comments or by email).

Filed under: English Phpmyadmin 1 comments Flattr this!

21 March 2011

Craig Small: Getting around the WordPress add image bug

WordPress currently has an annoying bug where you cannot add images easily using the in-built editor. Instead of a pop-up being shown with the image setting details, you are sent to another page. Once you choose the image size etc, you go to a blank page. Until that s fixed, there is a work-around. It s not exactly pretty but it does work. I have assumed you have uploaded your images to the media library first. First, type up what you want your blog entry to say. The save it as a draft which is the button in the blue circle in the screenshot. Then click the add image icon which will bring up the warning to go to the image selection page. This page should be a pop up on the same screen but is not (and is the bug). On the image selection page choose Image library which is the blue circle and edit the meta-data such as the name caption etc. Once you are happy with your decisions, click the Insert into Post button (red circle) which goes to a blank screen. The blank screen actually does have data in it. You will need to view source which will show something like:
<script type="text/javascript"><!--mce:0--></script>
The stuff in the win.send_to_editor is what you want. You will also need to change the backslash-quotes to plain quotes, so the code I would use is
<a rel="attachment wp-att-367" href="http://enc.com.au/2011/03/getting-around-the-wordpress-add-image-bug/tomatoes2/"><img class="alignright size-medium wp-image-367" title="tomatoes2" src="http://enc.com.au/wp-content/uploads/2011/03/tomatoes2-300x230.jpg" alt="" width="300" height="230" /></a>
You then enter this information back into your post (click back a few times in your browser). Also, make sure you have your editor set for HTML and not Visual for it to work. With that small bit of HTML, I have a nice set of home-grown tomatoes, or whatever else you want.

Craig Small: Silly C errors in gjay

I have been working on gjay to add support for Music Player Daemon where I had what initially looked like a strange problem. When WITH_MPDCLIENT was defined, the program would crash in all sorts of wierd places when the main program structure had this:
#ifdef WITH_MPDCLIENT
struct mpd_connection *mpdclient_connection;
#endif /* WITH_MPDCLIENT */
But would work fine when it was:
struct mpd_connection *mpdclient_connection;
#ifdef WITH_MPDCLIENT
#endif /* WITH_MPDCLIENT */
I tried changing the structure to just void *blah to see if it made a difference and it didn t. The program would crash every time. The answer was pretty simple in the end. The WITH_MPDCLIENT is defined in the file config.h and not every c source file was including it. Needless to say, they should! So half the program was using one version of the structure and the other was using another; no wonder the whole program was a mess because anything beyond this entry in the structure would be a few bytes out. With that little insanity out of the way i can get back to making gjay work with mpd.

18 March 2011

Christine Spang: on transparency

I found the following quote from Stefano's DPL platform interesting:
When faced with the dilemma, I've favored ditching some DPL tasks and communicating or taking notes about the others, instead of the other way around.
It takes someone who really knows Debian to realize that sometimes communicating about what's being done is more important than doing more.

Adnan Hodzic: Debian CUT, a new rolling release?

This post is also available on/was written for OMG! Ubuntu

It looks like 2011 started well for Debian. The project won awards in two out of seven categories at the Linux New Media Awards 2011 ( Best Open Source Server Distribution and Outstanding Contribution to Open Source/Linux/Free Software ). Just recently Internet.com declared Debian the most influential distribution ever, stating that ~63% of all distributions now being developed come ultimately from Debian. However, my intention for this article is not solely to praise Debian for its recent awards, but rather to focus on a new project, Debian CUT. Don t be surprised if you haven t heard about CUT; it seems most Debian community hasn t either. Then again, maybe it s because it is only labelled as unofficial/development so far.
A bit of history One of the greatest criticisms of Debian is that its release cycles are too long. Debian stable release is seen as often as Ubuntu s LTS release. As a server solution this doesn t present a problem at all, it can even seen as a pro. However, for desktop use and for your average Joe who needs to have the latest software and is unable to get it, this may well present a problem. Of course he can always turn to backports to get what he needs but by the time you have finished reading this very sentence, Joe has already moved to Ubuntu. For those who are completely unaware how things work within Debian, let me try to shed some light. Debian has 3 main branches: Unstable , is a branch used mainly by developers, where the latest changes to the software they are working on are made. Usually, after approximately 10 days this software is pushed to testing , branch which is going to become next Debian stable release. For comparison, in the process of a new Ubuntu release at one point in this process, changes from testing/unstable are frozen where Ubuntu fixes bugs until they are ready to release new Ubuntu release. Debian s testing branch is said by some to be more stable then most of the stable distributions out there. Last but not least is stable branch, which follows the (in)famous mantra it s released when it s ready , although last two releases have been released in a more timely manner. Okay, so what is Debian CUT? The idea stretches back more then 2 years, to Joey Hess proposing the idea of Constantly Usable Testing . For users/developers who can t wait ~2 years to see new Debian release, they usually turn to Debian testing branch. To clarify, testing images are released weekly, but most of us in Debian don t recommend you jump to testing by installing it from weekly image, but rather by upgrading from stable. To make the situation even worse, these builds frequently don t work due to all the changes that have been pushed from unstable. But this is where Constantly Usable Testing idea comes in, where the installer is always installable and you re already using what s going to be next stable release. You wouldn t have to worry about whether the next update is going to break your system, but would get regular security updates, while the big updates would come in shape of new testing snapshot versions. Frequency of release of these snapshots is ought to be on monthly basis, so you can plan on seeing the next release on April 6th (exactly one month after the current release, provided of course if there are no showstopper bugs). To sum it all up, you wouldn t have to wait ~2 years before new Stable release shows up, nor (in case you have already switched to testing) would you have to worry whether your next update will break your system. You would get your updates in form of timely (ie: monthly) snapshots until it s time for major milestone which would turn out to be new Debian release. Even though this might be long term goal, in these early stages no one can guarantees this. Rolling release? The simplest way to explain this concept is that there are two types of Linux distributions: one with milestone version numbers and other where new updates keep coming through their rolling release cycles. Some of the rolling distributions are Gentoo and Arch Linux, while on the other side you have RedHat and Suse. Novell announced that OpenSuse is moving to rolling release with project codename Tumbleweed. It s also worth mentioning that Debian already has its rolling counters and perfect examples are: LMDE and aptosid. It is a way to continuously develop software, a way that best fits Debian as a platform where software is continuously developed. Especially because most of its greatest values could get lost with the release of that big stable version . Perfect case of that could be the almost drop of Chromium in Squeeze. Another great example could be Android, which is criticised most for its versioning. Companies are refusing to release new Android versions to their current phones, because they want you to buy new phone with latest Android version, even though your current one is perfectly capable of running latest Android. Personally, I m sure the future will reveal that rolling releases are the future, towards which all Linux platforms should be heading. Perhaps the definition of rolling releases I gave earlier is why I believe this project has such bright future ahead of it. Debian would be somewhat of a hybrid of this definition, it would be rolling release until it s time to mark a big milestone with a version number, and before you know it, we re rolling again. Unofficial Debian Monthly Testing Snapshot Release (version 2011.03 final) Michael Gilbert took on a big responsibility, trying experimentally to prove the feasibility of such a project. He has released a first wheezy snapshot installer (versioned 2011.03) as a test for the development community to try out and evaluate. Please read official announcement along with download links. While you can use this snapshot to have constantly working installer, after it is installed you could move to testing altogether if you wish. Conclusion As I have already mentioned, this project is still in the development/experimental stage, which makes all of this its very conception. Also, apart from rumors of Ubuntu becoming rolling release, if Ubuntu is to get its rolling release this how it s going to get it Smile If you d like to find out more about Debian CUT, I suggest you read Raphael Hertzog s article A constantly usable testing distribution for Debian , and watch the video of Joey Hess CUT BoF on DebConf10 in NYC. Of course, if you d like to get involved into this project, please use the project s mailing list. If you have any comments, please do share, as I d be happy to answer any of your questions.

DebConf team: DebConf11: Short insiders summary of recent developments in DC11 organization (Posted by Vedran Omeragic)

Recently there has been some major developments in DebConf11 organization. In short, I will list some of the most important.

Website

Official DC11 website has finally been completed and now contains a great deal of information for those interested in attending this years conference. Some of the most notable changes include:

Navigation

Frontpage now contains a small menu with links to most important parts of website, like About, Registration, Contact Sidebar menu is now divided in 3 categories:

  • The Conference Links to pages which are directly connected with the Conference like About DC, Debian Day, Registration, Call for Papers, Talk Schedules, Payments Most of these links are hidden at the moment, and will be until the registration is open, to avoid any unnecessary confusion.
  • More Information Basic information for attendees like Important Dates, Accommodations, Venue, Visa Everything you need to know about your stay in Banja Luka.
  • Practical While these pages have almost nothing to do with the Conference itself, they do provide a vast information about city, getting to Bosnia, some practical information, as well as the Map of the whole city made my me :)

New Pages added
  • How Can I Help This page contains some basic information for anyone who wants to help out. Weather by volunteering, sponsoring, or just by adding a banner on their website
    DC11: How Can I Help?
  • Contact Instead of just sending visitors to mailto, we replaced it by a page which contains ALL contact information, emails AND phone numbers (which will be added just before DebCamp starts).
    DC11: Contact
  • Sponsorship Just like with the contact page, we made a page to be more user friendly to any interested parties or individuals, and contains most important info, such as sponsorship levels and contact address.
    DC11: Sponsorship
  • About DebConf11 Actually it s just a page with a content copied from frontpage (which is now used for something else).
    DC11: About DebConf11

New features:
  • Documentation Is now used to store all important documents and records for public use and sharing, like brochures, official government documents, important public mails, as well as the recordings of all IRC meeting logs. Can be found at:
    DC11: Documentation
  • Frontpage: Navigation To ensure the main goal of website, information availability, frontpage has been redesigned to provide maximum accessibility as well as easy access to any part of the website. DC11: Frontpage
  • Frontpage: Latest News To keep the public and attendees well informed of any important events and developments, we decided to add latest news section. Note however, that if you want to stay completely informed of any changes and updates, we still recommend subscribing to our mailing list, which can be found under contact page.
    DC11: Frontpage
  • Map of Banja Luka Because of lack of data on Google Maps and Open Street Map, as well as some other minor local maps, we made a small map of almost whole city and added around 60 or 70 locations which we consider to be of importance for our attendees. Such locations include hotels, parking spots, restaurants, bars, clubs, petrol stations, taxi stations, etc. This page uses simple JavaScript and is not dragable . Therefore we added Index Location to auto-locate the needed locations, as well as the legend to clarify the category of locations. Pointing your mouse over a certain location will open a small bubble with some info on that location/object. We should also point out that some of this data may not be 100% correct (ie working hours may be off ). Map is available at:
    DC11: Banja Luka City Map
Bosnian language

Site is now available on Bosnian language for native speaking attendees, should they have problems with English version. Every page is available on both languages, which can be changed by clicking on ba/en flag in upper left corner, or DC11: Bosnian Version.

Design

As you may have noticed, the official page received a small face lift. New header with the Government building, a small token of appreciation for all the help the Government is providing; as well as some minor changes in CSS and color scheme. While talking about design, we d like to thank Leandro G mez for his help with the header.

Sponsorship Sponsorship Levels

As agreed on the meeting, which took place on 22. of February, [Meeting logs], following sponsorship levels will be used for this years conference:

  • Steel < 1.000
  • Bronze > 1.000
  • Silver > 5.000
  • Gold > 10.000
  • Platinum > 20.000

Main reason for lowering amounts is due to last years results of high increase in the same. As for the benefits of these levels, the only thing changed is t-shirts / bags places; we believe t-shirts are much more noticeable than bags, and have therefore been promoted to Silver, while bags have been downgraded to Bronze. For a full list of requirements and benefits, please visit: DC11: Sponsorship Sponsorship Brochure

Sponsorship brochures have finally been completed and are available for a public distribution. The brochures are available in 3 qualities: low, high and original. Low and high quality brochures, ~6Mb and ~8Mb respectively, are for normal distribution to our sponsors, while original quality, at around 240Mb, is intended for maximum quality printing. Just as with the website, each quality is available in both Bosnian and English. Current version of the brochure is 1.1 and is most likely the final one. These brochures are available at Documentation page: DC11: Documentation >> Documents made by Local Team Or download them directly:

Also, I d like to thank Mirosal Remetic for making a great template and Pablo Duboue for some very valuable suggestions regarding the structure of the brochure.

Visas and Customs Rules

Many questions have been raised regarding Bosnian visa regime. We got in contact with the embassy and they clarified who can enter our country with(out) Visa. The list can be found on DC11 Wiki: Visa regime. It also states, which countries have a privilidge of ID-only entrance. If your country is not listed, that means you will need visa. If you want to know what exactly you need to get one, please visit the this official Government page and chose your country from a drop-down menu. As for the visa itself, as well as the list of Customs Rules, please read DC11: Visa.

Also, thanks to Darjan Prtic for getting the list from the Government officials.

So, what s next?

If you think, that is it for now, you are dead wrong. At this very moment following is being organized and developed:

  • Registration Registration Team is currently hacking penta and modifying it for this years needs and conditions. We expect to have registrations open by the end of March / beginning of April.
  • Travel and getting to Banja Luka Being one of the greatest obstacle, Travel Team is working hard on finding best routes for attendees and mapping the city
  • Sponsorship With final decision and agreement for sponsorship levels and brochures ready for public distribution, Sponsorship Team is now actively looking for new sponsors, global and local, preparing various promotional material, as well as planning our advertising strategy.

Final word

Well, these are pretty much the highlights of some recent and some future developments of DebConf11 organizational team. Mind you, there s a lot more going on, but I tried to give a small summary from the inside. Should you be interested in attending, helping out, sponsoring or just have some questions, feel free to ask them on our mailing list at: debconf11-localteam@lists.debconf.org. I should inform you that this is a public mailing list and its archives are available for a broader public. If you do not wish for your mail to be made public, please direct any enquiry directly to me trough the Contact Form or any other member of Core Local Team, Adnan Hodzic or Velimir Iveljic.

Russell Coker: Some Postfix Scripts for dealing with Outbound Spamming

I ve just written some small scripts to help me manage spam emergencies on a mail server. I ve been doing this thing with a bit of manual effort for a while, but after having done it once from a phone I want to optimise it a bit to reduce painful typing.My observation is that when a system I run is used as an outbound mail relay for spamming I will notice this reasonably quickly by the queues getting big. Some portion of mail that is queued is delayed due to general network issues and anti-spam measures that neither accept nor reject mail cause a typical spam message to be more likely to be queued than a typical non-spam message. So I look for accounts that send large amounts of mail.The biggest mail server I run (and the only one to have an outbound spam problem) is configured to not allow users to fake their sender address (IE use an address from gmail.com or another server) unless they pay extra. So the spam I deal with tends to have a valid origin email address that I can use.Queue CountingThe following command provides a sorted list of the accounts that have the most mail in the queue. This usually means a maximum of 2 or 3 spammers at the end of the list:mailq grep ^[A-F0-9] cut -c 42-80 sort uniq -c sort -n tailThe following function declaration makes the command get-top-sender find the sender with the largest number of queue entries (which has been a spammer every time I ve checked such things) and assign their email address to the environment variable $TOP_SENDER. It has to be a shell function so that the environment variable will be set in the context of the shell and available to child processes.function get-top-sender() TOP_SENDER_LINE=$(mailq grep ^[A-F0-9] cut -c 42-80 sort uniq -c sort -n tail -1) ; export TOP_SENDER=$(echo $TOP_SENDER_LINE cut -f2 -d\ ) ; TOP_COUNT=$(echo $TOP_SENDER_LINE cut -f1 -d\ ); echo "Top Sender is $TOP_SENDER with $TOP_COUNT messages" ; Viewing the QueueThe following script uses the first parameter or the environment variable $TOP_SENDER to specify the user who s mail should be read. In the normal course of events user email won t be read by the sysadmin, but if we are to determine whether a slew of email is spam or not there s no other way. So far I haven t seen a large number of queued messages not be spam, but I expect it ll happen eventually and the user will be happy that I checked it instead of locking their account and deleting all the queued mail. For readers who don t do sysadmin work, this is why we always have entries in the Terms of Service about the possibility of your email being read to fix technical problems or to investigate possible breaches of the ToS from your end.#!/bin/bash
set -e
mkdir -p ~/tmp/$$
cd ~/tmp/$$
if [ "$1" != "" ]; then
TOP_SENDER="$1"
fi
if [ "$TOP_SENDER" = "" ]; then
echo "Specify the sender on the command-line or in \$TOP_SENDER"
exit 1
fi
for n in $(mailq grep ^[A-F0-9].*$TOP_SENDER cut -c 1-10) ; do
FILE=$(echo $n sed -e "s/^\(.\)/\/var\/spool\/postfix\/deferred\/\1\/\1/" -e "s/ .*$//")
if [ -f "$FILE" ]; then
postcat "$FILE" > $n
else
echo "File $FILE missing, probably active"
fi
done
less *
rm -rf ~/tmp/$$Note that this needs to be done with a single less command so that I can terminate it by pressing q and easily go to the next file with :n . It does waste some server resources by running postcat on all queue files relating to the user in question, but that doesn t matter, servers are supposed to be powerful enough to cope with some inefficiency in processing uncommon operations.The Final CommandAfter that I have a script that connects by ssh to all outbound mail relays and deletes bad messages from the queues and then connects to the database server to lock the account that was used for sending the mail. I m not publishing it because it s specific to the servers I run.So when NAGIOS reports that the queue is too large I have one script to find the most likely culprit, one script to view the queued email from that account, and a final script to lock the account and purge the spam.I ve also added a reminder of the command names to /etc/motd as I don t want to be running ls and set on a phone to discover command names that I ve forgotten.Any suggestions for improvements to this will be welcome.

16 March 2011

Lars Wirzenius: QR codes for backup?

This is probably an idea obvious to everyone, and probably someone's already implemented this, but just in case they haven't, I'll throw this idea out there for someone to hack on. QR codes are two-dimensional bar codes. You can take arbitrary data and encode it into a barcode, which you can then print out. Later, you can scan it in and decode the barcode, and get the original data back. barcode-1.png Given that printed paper is fairly durable and unlikely to go suddenly bad, like hard disks do, this can be used for backups. I did a quick experiment. I put 2000 bytes per barcode and printed it out in various sizes: 10, 20, 30, and 70 mm squares. Then I scanned them in. The 70 mm one was the only one that could be decoded, the others were too small and smudged. It should be possible to fit 12 squares of just under 70 mm on an A4 sized page. That's 24000 bytes of raw data. What could you fit in that space? Or several pages of that? It's not dense enough for storing large amounts of data (images, audio, video), but there's a number of important things that are not so large. Perhaps secret keys, or your Revelation keyring file? For ease of use, it would be good to have a tool to take arbitrary binary data, compress it if possible, divide it into chunks of suitable size, number each chunk, and output a PDF with all the QR codes laid out optimally, ready for printing.

Next.

Previous.