Search Results: "ana"

10 March 2013

Hideki Yamane: great IOPS device and if we'd have it...?


I know some folks work at Fusion IO, maybe you know, one of the leader of PCIe flash storage system. And they say their products are "great", since their device put amazing IOPS as they press released (9,000,000 IOPS, and Wikipedia says "SAS HDDs put 200 IOPS". If you have a SSD in your laptop/desktop, you'd love it. It puts almost 10,000 IOPS or so). Such PCIe SSD can bring "Moon shot", change the rules for managing CPU, Memory, Disk and Network for servers, IMHO.


Then, I'm dreaming - if we Debian would have such MAGICAL device, what is the most efficient way to improve/accelerate/Kaizen our project? To use mailing list server? BTS? UDD? CI?

Dear lazyweb, please let me know what is the bottleneck of our development, and great IOPS capacity will fix it.

(and please don't say about things based on human resource, since PCIe SSD devices cannot improve such things, you know ;)

9 March 2013

Lars Wirzenius: Can the world be saved by coding?

I love programming. I was born to code. I want to save the world by writing code. I feel very strongly about software freedom. As I get older, and my eyes open to see more of the evil in the world, I become more concerned about freedom in general. Software freedom is, at least traditionally, about permission use, study, modify, and re-distribute software. The Free Software Foundation expresses this as the four essential freedoms. Debian expanded on that, and produced the Debian Free Software Guidelines. This is no longer enough. It is not enough to have all the freedom when you using your own computer, to have the source code to every bit of code that runs on your hardware. We live in the era of the Internet. Much of what we use computers for involves communication over the Internet, and the Internet is being actively used to curtail the freedom of people. For example, governments and corporations do large-scale surveillance of everyone, by eavesdropping on private communications, gathering enormous databases of personal data, and by analysing everything they can in order to find patterns and make conclusions both at the statistical level and about individuals. This ruins privacy. Without privacy, democracy cannot survive. Another example: in the name of various strawmen, such as terrorism, copyright violations, drugs, and child pornography, governments and corporations are collaborating in limiting private people's communications. I'm a Finnish citizen living in the UK. Both countries are among the closest ones to an ideal democractic nation state. Both countries arbitrarily block access to websites based on lists provided by private organisations, assuming that those organisations produce accurate lists of sites that contain copyright violations. As a result, both countries blocked, for example, a site where musical artists promoted their own works, bypassing the large media corporations that fund the list-making organisation. These issues transcend software freedom, though they interact with it. Software freedom is a necessary requirement for freedom, in a world where almost everything is done with, or controlled by, the use of computers. Software freedom is not enough to prevent censorship or surveillance: even if all the software in the world were free, including the national firewalls of China, Finland, and the UK, this would not prevent those countries from censoring and surveilling their citizens. The firewall would run free software, but that does not give the citizens the freedom to disable, or modify, the firewall systems. Fixing these issues is not a coding task. It is a job for politics, and it is going to take a long time, I fear. In the mean time, is there something a hacker can do to improve things? My main hobby project is Obnam, my backup program. Does that help people to protect their freedom? I think it does, in a small way: Obnam supports online, encrypted backups (and de-duplication even for encryption is used), which means they can make backups to servers anywhere on the Internet without having to fear their data gets read and analysed by hostile entities such as their own government, other governments, large corporations, or criminals. Obviously this does not help them if their government prohibits the use of encryption, or requires key escrow, or mandates backdoors to all encryption methods. But it's a step in the right direction. I don't claim Obnam will save the world, but a million small such steps by a thousand individual hackers, even without any particular organisation or guidance, will make a big impact. What step can you take?

8 March 2013

Richard Hartmann: Release Critical Bug report for Week 10

By using highly advanced and secret analytical tools (i.e. stupid correlation), I am guessing that Wheezy will be released in about six weeks. Week 16... mark my words. The UDD bugs interface currently knows about the following release critical bugs: How do we compare to the Squeeze release cycle?
Week Squeeze Wheezy Diff
43 284 (213+71) 468 (332+136) +184 (+119/+65)
44 261 (201+60) 408 (265+143) +147 (+64/+83)
45 261 (205+56) 425 (291+134) +164 (+86/+78)
46 271 (200+71) 401 (258+143) +130 (+58/+72)
47 283 (209+74) 366 (221+145) +83 (+12/+71)
48 256 (177+79) 378 (230+148) +122 (+53/+69)
49 256 (180+76) 360 (216+155) +104 (+36/+79)
50 204 (148+56) 339 (195+144) +135 (+47/+90)
51 178 (124+54) 323 (190+133) +145 (+66/+79)
52 115 (78+37) 289 (190+99) +174 (+112/+62)
1 93 (60+33) 287 (171+116) +194 (+111/+83)
2 82 (46+36) 271 (162+109) +189 (+116/+73)
3 25 (15+10) 249 (165+84) +224 (+150/+74)
4 14 (8+6) 244 (176+68) +230 (+168/+62)
5 2 (0+2) 224 (132+92) +222 (+132/+90)
6 release! 212 (129+83) +212 (+129/+83)
7 release+1 194 (128+66) +194 (+128/+66)
8 release+2 206 (144+62) +206 (+144/+62)
9 release+3 174 (105+69) +174 (+105/+69)
10 release+4 120 (72+48) +120 (+72/+48)
11 release+5
12 release+6
13 release+7
14 release+8
15 release+9
16 release+10
17 release+11
18 release+12
Graphical overview of bug stats thanks to azhag:

7 March 2013

Vincent Sanders: Man cannot discover new oceans unless he has the courage to lose sight of the shore

Continuing with my whirlwind introduction to NetSurf Development now is the time to start examining the code, how its arranged and how to interact with the existing developers.

The way the NetSurf source is structured is around the idea of the frontends each being a native browser. While this implies that there are nine separate browsers that happen to share a common code base the separation is not quite that well defined.

Each frontend provides the OS entry point (the main() function in a c program) and calls out to standard browser initialisation entry function netsurf_init() and then starts running the browsers main dispatch loop with netsurf_main_loop() when that exits the frontend cleans up with netsurf_exit().

The frontends provide a large selection of functions which are called from the core code. These routines run from running the event scheduler through to rendering graphics and text.
Finding your way aroundThe browsers directory layout is fairly shallow consisting of some Makefiles, the nine frontend directories and eleven others.

The Makefiles are GNU make and represent a pretty straightforward linear build system. We do not use recursive make or autotools. There are plans to use the core buildsystem that all the other components use.

The frontend directories contain the code for the frontend and the makefile fragments to build them which are included by the top level Makefile.

In addition there is:
desktop
This contains the non frontend specific code that actually behaves like a browser. For example desktop/netsurf.c contains the three primary functions we outlined in the introduction. You will also find much of the function and data structures interfaces the frontend must provide. It is unlikely someone new to the project will need to change anything in here (there be dragons) but is an important set of routines.
utils
Here you will find the utility and compatibility interfaces, things like url handling, logging, user messages, base64 handling. These are utility interfaces that do not justify splitting out their functionality to a separate library but are useful everywhere. Changing an interface in here would likely result in a major refactor.
For example one quirk is the logging macro was created before varadic C preprocessor macros were universal so it must be called as LOG(("hello %s world", world_type)) e.g. with double brackets. Fixing this and perhaps improving the logging functionality would be "nice" but the changes would be massive and potentially conflict with ongoing work.
content
This contains all the core code to handle contents i.e. html, css, javascript, images. The handling includes retrieval of the resources from URI , correct caching of the received objects and managing the objects state. It should be explicitly stated that the content handlers are separate and use this core functionality. The actual content handlers to deal with object contents such as the routines to decode image files or render html are elsewhere.
image
This is where the content handlers for the various image types are kept. The majority of these image types (jpeg, png, webp, gif, bmp) use a standard library to perform the actual decode (libjpeg, libpng etc.). One special feature used by most image handlers is that of a decoded image cache which is distinct and separate from the content cache.
The decoded image cache manages decoding of the source images (the jpegs and pngs) into frontend specific "render" bitmaps. For example the gtk frontend keeps the decoded images as a cairo surface ready for immediate plotting.
The cache uses a demand based (when the browser actually displays the image) just in time strategy which has been carefully balanced, with real world input, to reduce the overhead of unnecessary image decoding and processing against memory usage for the render bitmaps.
css
The css content handlers provide for the processing of css source text and use the NetSurf libcss library to process into a bytecode suitable for applying style selection at render time.
javascript
The javascript handlers (strictly speaking this should be named the "script" directory as all types of script are handled here) provide basic functionality to bind a javascript engine to the rest of the browser, principally the Document Object Model (DOM) accessed with libdom. The only engine that currently has bindings is Spidermonkey from the Mozilla foundation.
render
This is the heart of the browser containing the content handler for html (and plain text). This handler deals with:
  • Acquiring the html.
  • Running the base parser as data arrives which generates the DOM and hence DOM events from which additional resource (stylesheets etc.) fetches are started.
  • Deal with script loading
  • Constructing the box model used for layout
  • Performing the Layout and rendering of the document.
Because this module has so many jobs to do it has inevitably become very complex and involved, it is also the principle area of core development . Currently NetSurf lacks a dynamic renderer so changes made by scripts post document load event are not visible. This also has the side effect that the render is only started after the DOM has finished construction and all the resources on a page have completed their fetch which can lead to undesirable display latency.
Docs
Documentation about building and using NetSurf. If anyone wants a place to start improving NetSurf, this is it, it is very incomplete. It must be noted this is not where dynamically generated documentation is found. For the current Doxygen output the best place to look is the most recent build on the Continuous Integration system.
resources
These are runtime resources which are common to all frontends. To be strictly correct they may be the sources which get converted into runtime resources e.g. The FatMessages file which is teh message text for all frontends in all languages, this gets processed at build time into separate files ready.
!NetSurf
This is another resources directory and technically the resources for the riscos frontend. The naming and reliance on this directory are historical. To allow the RISC OS frontend to be run directly from the source directory and an inability of RISC OS to process symbolic links most common runtime resources end up in here and linked to from elsewhere.
test
These are some basic canned test programs and files, principally to test elements of the utils and perform specific exercise of various javascript components.
Getting started
Once a developer has a checked out working build environment and can run the executable for their chosen frontend (and maybe done some web browsing :-) it is time to look at contributing.

If a developer does not have a feature or bug in mind when they begin the best way to get started is often to go bug hunting. The NetSurf bugtacker has lots to choose from unfortunately. Do remember to talk to us (IRC is the best bet if you are bug hunting) about what you are up to but do not be impatient. Some of those bugs are dirty great Shelob types and are not being fixed because even the core developers are stumped!

When first getting going I cannot recommend reading the code enough, this seems to be a skill that many inexperienced open source developers have yet to acquire, especially if they are from a predominately proprietary development background. One wonderful feature of open source software is you get to see all of it, all the elegant nice code and all the "what the hell were they thinking" code too.

One important point is to use your tools well the source is in git, if you learn how to use git well you will gain a skill that is readily portable, not just for NetSurf. And not just revision control tools, learn to use your debugger well and tools like valgrind. Those skills will replay the time spent learning them themselves many times over.

When using git one thing to remember is commit early and often, it does not matter if you have lots of junk commits and dead ends, once you have something viable you can use 
git rebase --interactive
and rewrite it all into a single sensible set of commits. Also please do remember to develop on a branch, or if you did not 
git pull --rebase
is your friend to avoid unnecessary merges.
Playing nicely with othersThe NetSurf community is a small band of unpaid volunteers. On average we manage to collectively put in, perhaps, ten hours a week with the occasional developer weekend where we often manage over twenty hours each.

The result is that developer time is exceptionally valuable, add in a mature codebase with its fair share of technical debt and you get a group who, when they get to work on NetSurf, are incredibly busy. To be fair we are just like every other small open source project in that respect.

What this means to a new contributor is that they should try and do their homework before asking the obvious questions. The documentation is there for a reason and in spite of its obvious shortcomings please read it!

When asking questions it should be noted that currently the majority of active contributors are in the Europe so if you visit the IRC channel or post questions to lists the time difference is something to keep in mind.

I carefully said contributor above and not developer, users trying the CI builds and reporting results are welcome...as long as they report useful bugs to the bug tracker. Simon Tatham has produced an excellent resource on this subject.

Also we are always happy to receive translations to new languages (diff against the FatMessages file would be outstandingly useful but anything is welcome), artwork, documentation. Just recall what I mentioned about busy developers. Surest way to get us to see something is the development mailing list, you will probably get a reply, though I will not promise how fast!

Some of the more common mistakes when interacting with the community are:
In conclusion
Hopefully this has been of some use although I had hoped to cover more and provide deeper insights and advice on the codebase but there is only so much generalisation to be done before it is just easier for the developer to go read the code for themselves.

I look forward to lots of new contributions :-) though I fear this may all end up as more of a crib sheet for next time we do GSOC, time will tell.

Vincent Sanders: The way to get started is to quit talking and begin doing

When Walt Disney said that he almost certainly did not have software developers in mind. However it is still good advice, especially if you have no experience with a piece of software you want to change.

Others have written extensively on the topic of software as more than engineering and the creative aspects, comparing it to a craft, which is a description I am personally most comfortable with. As with any craft though you have to understand the material you have to work with and an existing codebase is often a huge amount of material.

While developing NetSurf we get a lot of people who talk a lot about what we "ought" or "should" do to improve the browser but few who actually get off their collective posteriors and contribute. In fact according to the ohloh analysis of the revision control system there are roughly six of us who contribute significantly on a regular basis and of those only four who have interests beyond a specific frontend.

It has been mentioned by a couple of new people who have recently compiled the browser from source that it is somewhat challenging to get started. To address this criticism I intend to give a whirlwind introduction to getting started with the NetSurf codebase, perhaps we can even get some more contributors!

This first post will cover the mechanics of acquiring and building the source and the next will look at working with the codebase and the Netsurf community.

This is my personal blog, the other developers might disagree with my approach, which is why this is in my blog and not on the NetSurf website. That being said comments are enabled and I am sure they will correct anything I get wrong.
ResourcesNetSurf has a selection of resources which are useful to a new developer:
Build environmentThe first thing a new developer has to consider is their build environment. NetSurf supports nine frontends on several Operating Systems (OS) but is limited on the build environment that can be used.

The developer will require a Unix like system but let's be honest, we have not tried with anything other than Linux distributions in some time or MAC OS X for the cocoa frontend because its a special snowflake.

Traditionally at this point in this kind of introduction it would be traditional to provide the command line for various packaging systems to install the build environment and external libraries. We do have documentation that does this but no one reads it, or at least it feels like that. Instead we have chosen to provide a shell fragment that encodes all the bootstrap knowledge in one place, its kept in the revision control system so it can be updated.

To use: download it, read it (hey running random shell code is always a bad idea), source it into your environment and run ns-apt-get-install on a Debain based system or ns-yum-install on Fedora. The rest of this posting will assume the functionality of this script is available, if you want to do it the hard way please refer to the script for the relevant commands and locations.

For Example:
$ wget http://git.netsurf-browser.org/netsurf.git/plain/Docs/env.sh
$ less env.sh
$ source env.sh
$ ns-apt-get-install

Historically NetSurf built on more platforms natively but the effort to keep these build environments working was extensive and no one was prepared to do the necessary maintenance work. This is strictly a build setup decision and does not impact the supported platforms.

Since the last release NetSurf has moved to the git version control system from SVN. This has greatly improved our development process and allows for proper branching and merging we previously struggled to implement.

In addition to the core requirements external libraries NetSurf depends on will need to be installed. Native frontends where the compiled output is run on the same system it was built on are pretty straightforward in that the native package management system can be used to install the libraries for that system.

For cross building to the less common frontends we provide a toolchain repository which will build the entire cross toolchain and library set (we call this the SDK) direct from source. This is what the CI system uses to generate its output so is well tested.
External LibrariesNetSurf depends upon several external development libraries for image handling, network fetching etc. The libraries for the GTK frontend are installed by default if using the development script previously mentioned.

Generally a minimum of libcurl, libjpeg and libpng are necessary along with whatever libraries are required for the toolkit.
Project Source and Internal Libraries
One important feature of NetSurf is that a lot of functionality is split out into libraries. These are internal libraries and although technically separate projects, releases bundle them all together and for development we assume they will all be built together.

The development script provides the ns-clone function which clones all the project sources directly from their various git repositories. Once cloned the ns-make script can be used to build and install all the internal libraries into a local target ready for building the browser.

For Example:

$ source env.sh
$ ns-clone
$ ns-make-libs install
Frontend selectionAs I have mentioned NetSurf supports several windowing environments (toolkits if you like) however on some OS there is only one toolkit so the two get conflated together.

NetSurf currently has nine frontends to consider:
Building and running NetSurfFor a developer new to the project I recommend that the gtk version be built natively which is what I describe here.

Once the internal libraries have been installed, building NetSurf itself is as simple as running make.

For Example:
$ source env.sh
$ ns-make -C $ TARGET_WORKSPACE /$ NS_BROWSER  TARGET=gtk

Though generally most developers would change into the netsurf source directory and run make there. The target (frontend) selection defaults to gtk on Linux systems so that can also be omitted Once the browser is built it can be run from the source tree to test.

For Example:
$ source env.sh
$ cd $ TARGET_WORKSPACE /$ NS_BROWSER 
$ ns-make
$ ./nsgtk

The build can be configured by editing a Makefile.config file. An example Makefile.config.example can be copied into place and the configuration settings overridden as required. The default values can be found in Makefile.defaults which should not be edited directly.

Logging is enabled with the command line switch -v and user options can be specified on the command line, options on the command line will override those sourced from a users personal configuration, generally found in ~/.netsurf/Choices although this can be compile time configured.

6 March 2013

Keith Packard: composite-swap

Composite and Swap Getting it Right Where the author tries to make sure DRI3000 is going to do what we want now and in the future DRI3000 The basic DRI3000 plan seems pretty straightforward:
  1. Have applications allocate buffers full of new window contents, attach pixmap IDs to those buffers and pass them to the X server to get them onto the screen.
  2. Provide a mechanism to let applications know when those pixmaps are idle so that they can reuse them instead of creating new ones for every frame.
  3. Finally, allow the actual presentation of the contents to be scheduled for a suitable time in the future, generally synchronized with the monitor. Let the client know when this has happened in case they want to synchronize themselves to vblank.
The DRI3 extension provides a way to associate pixmap IDs and buffers, and given the MIT-SHM prototype I ve already implemented, I think we can safely mark this part as demonstrably implementable. That leaves us with a smaller problem, that of taking pixmap contents and presenting them on the screen at a suitable time and telling applications about the progress of that activity. In the absence of compositing, I m pretty sure the initial Swap extension design would do this job just fine, and should resolve some of the known DRI2 limitations related to buffer management. And, I think that goal is sufficient motivation to go and implement that. However, I wanted to write up some further ideas to see if the DRI3000 plan can be made to do precisely what we want in a composited world. The Composited Goal To make sure we re all on the same page, here s what I expect from the Swap extension in a composited world:
  1. Application calls Swap with new window pixmap
  2. Compositor hears about the new pixmap and uses that to construct a new screen pixmap
  3. Compositor calls Swap with new screen pixmap
  4. Vertical retrace happens, executing the pending swap operation
  5. Compositor hears about the swap completion for the screen
  6. Application hears about the swap completion for its window
In particular, applications should not hear that their swap operations are complete until the contents appear on the screen. This allows for applications to throttle themselves to the screen rate, either doing double or triple buffering as they choose. I didn t add steps here indicating buffers going idle or being allocated, because I think that should all happen behind the scenes from the application s perspective. Many applications won t care about the swap completion notification either, but some will and so that needs to be visible. Redirected Swaps? Owen Taylor suggested that one way of getting the compositor involved would be to have it somehow redirect Swap operations, much like we do with window management operations today. I think that idea may be a good direction to try:
  1. Application calls Swap with new window pixmap
  2. Swap is redirected to compositor, passing along the new window pixmap
  3. Compositor constructs a new screen pixmap using the new window pixmap
  4. Compositor calls Swap on the screen and the window, passing the new screen pixmap and the new window pixmap. When the screen update occurs, the screen and the window both receive swap completion events.
This has the added benefit that the X server knows when the compositor is expecting window pixmaps to change like this the compositor has to explicitly request Swap redirection. Window Pixmap Names and GEM Buffer Handles One issue that swapping window pixmaps around like this brings up is how to manage existing names for the window pixmap. Right now, applications expect that window pixmaps will only change when the window is resized. If the Swap extension is going to actually replace the window pixmap when running with a suitable compositor, then we need to figure out what the old names will reference. Are there non-compositor applications using NameWindowPixmap that matter to us? How about non-compositor applications using TextureFromPixmap to get a GEM handle for a window pixmap? For now, I m very tempted to just break stuff and see who complains, but knowing what we re breaking might be nice beforehand. Idling Pixmaps When an application is done drawing to a window pixmap and has passed it off to the X server for presentation, we d like for that pixmap to be automatically marked as discardable as soon as possible. This way, when memory is tight, the kernel can come steal those pages for something critical. Of course, applications may not want to let the server mark the pixmap as idle after being used, so a flag to the Swap call would be needed. Ideally, the pixmap would become idle immediately after the pixmap contents have been extracted. In the absence of a compositor, that would probably be when the Swap operation completes. With a compositor running, we d need explicit instruction from the compositor telling us that the window pixmap was now idle : 
 
    SwapIdle
    drawable: Drawable
    pixmap: Pixmap
Furthermore, the application needs to know that the pixmap is in fact idle. I think that we ll need a synchronous X request that marks a buffer as no longer idle and have that return whether the buffer was discarded while idle. It doesn t seem sufficient to use events here as the application will need to completely reconstruct the pixmap contents in this case. This reply could also contain information about precisely what contents the pixmap does contain. 
 
    SwapReuse
    drawable: Drawable
    pixmap: Pixmap
       
    valid: BOOL
    swap-hi: CARD32
    swap-lo: CARD32
Pixmap Lifetimes and Triple Buffered Applications If we redirect the Swap operation and send the original application window pixmap ID to the compositor, what happens when the application frees that pixmap before the compositor gets around to using the contents? Surely the Compositor must handle such cases, and not just crash. However, I m fine with requiring that the application not free the pixmap until told by the compositor.

5 March 2013

Tanguy Ortolo: Suspend your computer from a non-mainstream desktop environment

Battery icon UPower for regular users Major desktop environments usually provide a user-friendly menu to suspend your computer. Internally, if seems to use something called UPower, which uses something called PolicyKit, which in turn may or may not use another piece called ConsoleKit (do not ask me what all these pieces are and how they relate to each other, I do not know and I do not want to know). For the regular user, the result is the following: when they click on that button, it suspends their computer.UPower for command line users However, if you are using a more unusual desktop environment, for instance a simple, light window manager, you will not have such a button. And no, UPower does not provide a simple command line tool to suspend your computer (they probably consider that command line is out of fashion ). Well, in case you need it, here is a way to suspend your computer from a command line or whatever: just alias it or bind it to a key shortcut. It uses some dark DBus magic: 
$ dbus-send --system --print-reply --dest='org.freedesktop.UPower' \
    /org/freedesktop/UPower org.freedesktop.UPower.Suspend
If you prefer to hibernate your computer, that is, to suspend it, not to RAM but to disk: 
$ dbus-send --system --print-reply --dest='org.freedesktop.UPower' \
    /org/freedesktop/UPower org.freedesktop.UPower.Hibernate
As you can see, for some reason it requires some duplication of information: the name org.freedesktop.UPower appears three times in that command line: in the destination , object path and interface member , whatever that could mean. Again, do not ask me why, I know nothing about that and I do not want to. Notes
  1. Actually, UPower does provide a command line tool called upower, but the only thing it does (monitoring power sources) is not really useful for that matter.

Lisandro Damián Nicanor Pérez Meyer: My Debian freeze experience (so far)

This is the first freeze in which I'm involved with upload rights. And it turned out to be a quite interesting ride so far, so I thought it would be nice to write about it.

As some of you may know, I'm a part of the Qt/KDE team. Before the freeze I was mostly involved in leaf packages, with some patch here or there, nothing fancy. And then the freeze came...

Bugs in Qt

...and bugs appeared in Qt. But they didn't get solved, even if the patches were there. Due to personal reasons, the manpower in Qt/KDE land decreased below normal levels (which were already low).

I took the time to review them, apply them in a local branch, build and test the fixes. I did a Qt upload before, but it was a team-consented one. This time there was not much reaction in our IRC channel as it used to be, so I was doubting if going ahead or not. I asked Ana, my great friend and former sponsor, for an opinion on the subject, and she gave me a really important advice: the patches were looking good and there is one really big true: if something get's broken, it can be fixed with a later upload.

You might be asking yourself why I was that afraid of doing the upload. Well, when one maintains such a medular package for many users one has to be careful And I also got used that those "big ones" like Qt where normally handled by hand skilled people. Do not take me wrong here, it's not that those people where keeping them for themselves, it's knowing that one does not has the same skills nor experience as them.

But again, no one was able to upload and I had the chance and will to do another upload if needed, so off it went. That was Qt 4:4.8.2-2.

Then new experiences followed: asking a buildd maintainer for a giveback, asking the Release Team for an unblock (more on this later), etc. While sponsoring me, Ana gave me another excellent advice which I always keep in my mind:

You can't know **everything** about Debian.

And that also includes a not so technical skill: communicating with other teams. But finally we got this new version of Qt in testing. Cool :-)

Of course, new bugs appeared, and my lack of skills (and sometimes, time) where replaced by team work: Pino looking at patches and Sune contacting upstream. The eleven uploads that followed are a nice example of team work, even if I was the one who signed and did the uploads. Whoever uses Qt must know that these wonderful people (including those who are not so active nowadays like Modestas or Fathi) have done lots to bring the better to their users.Thank you guys!

Be careful, they might bite you back!

Coming back to the non-technical skills, sometimes you have to communicate with other teams in Debian. And each team is (naturally) a separate world: possibly different people, different goals, etc. Of course, we share the goal to make Debian the best experience we can, but we do not necessarily agree on the paths to achieve so.

During the freeze, there is a team that gets lots of pressure, and not by chance: the Release Team. They handle a very important task, which is to ride the freeze to get to a release. OK, that's what everyone knows. Now, one thing is knowing that and another is really understanding what does that means.

Of course I was in the first group. From the outside, communicating with the RT was a kind of "special art", and not an easy one. I have even been advised to not ask for more than one or two unblocks per weekend, as they might "bite me back". So I put on my flamesuit on and... launched reportbug release.debian.org.

Now I'm really happy to say that my experience was far from what I described above. And yes, I had the chance to even disagree on some stuff. But remember: non-technical skills, a.k.a. social skills. Once I started to know what was going on inside the RT (joining #debian-release was a big help for that) I learnt some nice tips to approach them. Please allow me to list some of them:

  • Remember: you are the maintainer of the package, they are like gatekeepers that are there to help us coordinate to do a release. But they don't maintain the code, you do that. So try to be verbose when needed, explain the changes and don't forget a nice diff. They need to understand what is going on: they can't read your mind.
  • They are human beings too: not everyday might be their best day (the same goes for you too!). And they are under the pressure of a release. Be patient, that finally pays off.
  • Does your changes seem not so clear? try to improve them.
  • The package has a lot of changes but you really feel they are needed? Try to explain that as good as you can.
  • Try to put yourself in their position: do we really want this? If in doubt, there is a nice way to know what they think: a pre-approval bug.
I want to make a stop in this last point. A pre-approval bug it's an unblock bug in which you edit the subject to add "pre-approval" in it. Easy, isn't it? It gives you the opportunity to know what the RT thinks before doing the upload. In other words: it gives you the chance to communicate and do things in the best possible way for all the parts involved.

I've have also seen pre-approval bugs that were really not needed. But to learn where the threshold of what can be directly uploaded and what deserves a pre-approval bug is you need to know the guidelines the RT gives you. Do you still have doubts? fire a pre-approval bug and try to be clear.

Of course, this are all fruits of my experience with the RT during this time. If the RT thinks different from what I'm writing here, please stand up: we are hear to listen to you and learn :-)

As a side note, I think I should file a wishlist bug to include the pre-approval bug option in reportbug. Yes, I'm lazy :-)

Summing up

Overall this was a very nice and positive experience. We are not done yet. Are we really done at some point? Let's hope not, because this is where the fun comes from :-)

Russell Coker: Kogan Mobile

Kogan Mobile is the newest virtual telco in Australia [1]. They resell Telstra 3G (not NextG or LTE) and while their coverage isn t as good as the full Telstra service it s more than adequate for my needs as they provide 3G coverage to 97% of the population and 2G+3G coverage to 98.5%. Their coverage is probably a lot better than Three who had the worst record of network coverage in Australia yet managed to always provide coverage where I wanted it I was a happy Three customer for more than 6 years. Kogan s main selling point is that they offer unlimited calls to Australian mobile phones and land-lines and unlimited SMS for a pre-paid fee of only $300 per annum ($25 per month on average). My parents have been getting unpleasantly large phone bills which have considerably more than $25 of calls to mobile phones every month so an obvious solution for them is to sign up for a Kogan mobile phone and use it for all such calls. There are other ways my parents could save money on calls (such as VOIP) but a mobile phone is easiest and offers other benefits such as running Android apps (when compared to using a non-smart phone). 6G of Data! Kogan also offers 6G of data per month, the down-side to this is that they bill in 1MB increments per session . I was worried that this might be per TCP connection or something else silly but I decided to sign my parents up for it as they aren t going to use a lot of data (they claim that they don t want to use the Internet on their phone but I know better). I ve done some tests on the SIM I got for my parents. For testing purposes I installed the Kogan SIM in my wife s new Nexus 4 and had it provide Wifi net access to my phone while we were playing Ingress. So far after 2 days which involved a reasonable amount of Ingress (I reached level 7) as well as all the usual stuff that happens in the background for two phones (checking email, news, weather, etc) Kogan considers that 177MB have been used out of the 6144MB for the month, which means that even with what is an unusual amount of traffic for us the account in question still isn t going to use half the quota for the month. Now that 10 days have elapsed with less intense usage Kogan considers that a total of 1373MB have been used.
days received sent Kogan
2 127.24 41.54 177
10 959.39 299.6 1373
Kogan also don t seem to mention whether they bill for transmitted data. I used the 3G Watchdog app to measure the amount data transferred, the above table has the amounts of data that 3G Watchdog considers were sent and received along with the amount that is listed by the Kogan Android app. My past experience with 3G Watchdog and Virgin Mobile is that it s usually quite accurate but has been over-reporting the data transfers recently (I think that Virgin is only billing me for downloads while 3G Watchdog counts uploads). So the relatively small difference between the 3G Watchdog report and what Kogan thinks I ve done means that either rounding the session up to the nearest meg doesn t make any significant difference (which would imply that a session can be a long time) or that Kogan isn t counting uploaded data and the session rounding up only adds about 40% to the total recorded transfer. My current plan with Virgin Mobile gives me 1.5G per month of quota, so as long as Kogan s rounding doesn t increase the recorded data transfer by a factor of 4 I will still be able to transfer more data with Kogan while paying less. One disadvantage of using Kogan is that I might have to tweak programs like my email program to poll less frequently to avoid excessive session charges (a program polling every 5 minutes would use up the 6G quota in 21 days if each poll counted as a session) although current tests indicate that this won t be necessary. But the up-side is that there are no extra fees with Kogan, they merely restrict data access for my use and that of most people I know it s better to have data access cut off than to receive a large bill. The Kogan Android App Kogan has an Android app that will give the status of your account and allow you to change the plan etc. This is quite nice but one major disadvantage is that it s also a sales app for the Kogan online store. This is bad for the user as some aspects of what I consider the core functionality are limited (for example there s no way to force a poll of the data usage count or determine how current the data is). But there s an obvious advantage to Kogan in providing a way to sell their goods that is going to be used by every customer of Kogan Mobile. The sales part of the app isn t very functional IMHO, it doesn t seem to have basic functionality such as sorting a list of items by price. Benefits of Kogan 6G of data is a lot! $300 per annum is quite cheap, anyone who makes any serious use of phones will be paying more than that in Australia. Lack of extra fees means that there is little need to restrict net access. I can risk getting cut off near the end of the month but I can t risk the potential for hundreds of dollars in excess fees. The Kogan app shows me the data used so I will probably uninstall 3G Watchdog, having one less program running is a good thing. You get a free SIM (value $5) when you buy a phone from Kogan. Disadvantages of Kogan They are new to the Telco business and admit that their customer service is lacking due to unexpected demand. If you order a SIM now they state that it will arrive in April. Apparently they are deliberately delaying orders because they can t cope with demand. The included call quota doesn t include international calls. While unlimited free calls in Australia is great if you make many international calls then this could end up costing you more. Other mobile telcos such as Lebara offer good deals for International calls, it could be an option to use a Lebara SIM with an old non-smart phone while using Kogan for your smart phone. I am concerned about the lack of detail about how data is accounted. If the definition of a session changes then 6G could turn out not to be enough. As Kogan is reselling a Telstra service it is possible that Telstra could change the deal without Kogan being able to stop them. Conclusion I will move my phone and my wife s phone to Kogan ASAP. My general idea is to sign us up for Kogan about 2 weeks apart, so if one phone runs out of the 6G data quota then the other phone can be used as a Wifi access point for 2 weeks. If the phones don t both have their quota end at the same time then there is less chance of both phones running out during a high traffic month.
Related posts:
  1. Australian Mobile Phone Costs I previously wrote about the changes to bundling prices by...
  2. Choosing an Australian Mobile Telco for use with Android Since playing with the IBM Seer augmented reality software [1]...
  3. Dual SIM Phones vs Amaysim vs Contract for Mobile Phones Currently Dick Smith is offering two dual-SIM mobile phones for...

4 March 2013

Michal Čihař: Nice OpenStreetMap project

For quite some time, I'm contributing to OpenStreetMap. Mostly covering areas where I live or I know well, but sometime I randomly update areas where I travel as well. The one important bit was always missing for me - nice frontend to render many of hidden features. Now this space seems to be slowly filled in by map1.eu. This quite new project (first announcement has been two weeks ago) renders quite nice maps for Europe. The map is primarily intended for hiking or cycling, so it includes things like hiking routes or cycle routes. In Czech republic we have quite great maps on mapy.cz, but I don't know anything what covers whole Europe. The project still has some issues, but is progressing nicely as you can see in today's announcement of 0.2 release (in Czech). Of course you can help, either by code or by money, which is needed to run rendering servers.

Filed under: English Maps 0 comments Flattr this!

3 March 2013

Gregor Herrmann: DPL game

when francesca started her DPL game, I also started thinking about people I'd like to encourage to stand as a DPL candidate. unfortunately my initial "short list" contained 18 names, which doesn't fit the game's rules very well. after waiting a bit, 9 of them were mentioned by others already, so I can leave them out here (which also means no public hug, sorry, please complain to the inventor of the game!). from the remaining 9, I've now chosen my Fantastic Four (in alphabtic order of first names): in my opinion, the DPL task is mostly about managing, & those four people are in my experience all very good at organising projects & dealing with people, & are therefore well suited for this task. let's see if they feel motivated to take on the challenge! (the nomination period has already started, so don't hestitate!)

Petter Reinholdtsen: Frikanalen - Complete TV station organised using the web

Do you want to set up your own TV station, schedule videos and broadcast them on the air? Using free software? With video on demand support using free and open standards? Included a web based video stream as well? And administrate it all in your web browser from anywhere in the world? A few years now the Norwegian public access TV-channel Frikanalen have been building a system to do just this. The source code for the solution is licensed using the GNU LGPL, and available from github. The idea is simple. You upload a video file over the web, and attach meta information to the file. You select a time slot in the program schedule, and when the time come it is played on the air and in the web stream. It is also made available in a video on demand solution for anyone to see it also outside its scheduled time. All you need to run a TV station - using your web browser. There are several parts to this web based solution. I'll mention the three most important ones. The first part is the database of videos and the schedule. This is written in Django and include a REST API. The current database is SQLite, but the plan is to migrate it to PostgreSQL. At the moment this system can be tested on beta.frikanalen.tv. The second part is the video playout, taking the schedule information from the database and providing a video stream to broadcast. This is done using CasparCG from SVT and Media Lovin' Toolkit. Video signal distribution is handled using Open Broadcast Encoder. The third part is the converter, handling the transformation of uploaded video files to a format useful for broadcasting, streaming and video on demand. It is still very much work in progress, so it is not yet decided what it will end up using. Note that the source of the latter two parts are not yet pushed to github. The lead author want to clean them up a bit more first. The development is coordinated on the #frikanalen IRC channel (irc.freenode.net), and discussed on the frikanalen mailing list. The lead developer is Benjamin Bruheim (phed on IRC). Anyone is welcome to participate in the development.

2 March 2013

Raphaël Hertzog: My Free Software Activities in February 2013

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (78.31 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Debian packaging I wanted to update publican to the latest upstream release but I stopped after a few hours of work during which I filed two bugs that a modicum of testing should have caught before release. So I decided to wait for the next minor release. I uploaded python-django 1.4.4 and 1.4.5, new upstream maintenance and security releases which thus went into wheezy. I also prepared a stable update of Django (1.2.3-3+squeeze5) which required me to backport the last 2 sets of security patches. I uploaded a new revision of wordpress to fix a problem with TinyMCE (#700289) and to update/add many translation files (#697208). Bug reporting and misc fixes Live-build issue. I experienced some intermittent failures when building HDD live images with live-build on armel. Daniel Baumann directed me to the problematic piece of code (the oversizing of the image size was not enough) so I committed a small fix by increasing the oversizing factor to 6%. Live-config issue. I also reported another issue that I diagnosed in live-config (#701788), namely that the script which setups sudo was failing when the default user is root. git-buildpackage issue. I filed #700411 after noticing that git-import-orig imported the debian directory provided by upstream. Those directories are not used with 3.0 (quilt) source package and their presence in the upstream branch is thus harmful: any change to the upstream debian directory will result in conflicts when you merge a new upstream release in your packaging branch. rubygems integration. Later I had to package a bunch of ruby applications that were using Bundler and I wanted to reuse as many packaged ruby modules that I could. But for this, those modules had to provide the required rubygems meta-information. I filed #700419 to request those on rake-compiler and with the help of C dric Boutillier (and others on #debian-ruby), we identified a bunch of ruby modules which could get those with a simple recompilation. I filed bin-nmu requests in #700605. Misc bugs. simple-cdd offers to select profiles to install but I noticed that the associated debconf template was not translated (#700915). The startup scripts (provided by initscripts) in charge of activating the swap are supposed to handle a noswap kernel command line option to disable swap. In #701301, I reported that the option was not working correctly if quiet was present first in the command line due to spurious break statements. Debian France Administrative work. We were late for some legal procedures so I wrote the report of the last general assembly and sent it to the Tribunal d instance of Sarreguemines to record the changes in the administrative board. I also completed the special register of the association, it s a notebook that is legally required and that must document any important change in the governance structure of the association (new members of the board, headquarters change, new bylaws, etc.). Galette developments. Debian France is funding a few enhancements to the Galette free software that we re using to manage the association. I am in touch with the Galette developer to answer his questions and ensure that his work will meet our needs. Librement I have been looking for talented developers who have a genuine interest in my Librement project. I want to fund the initial development of the project but I don t have the means to fund it entirely. So I really wanted to find developers who would find an interest beside the money that I would pay. I got in touch with the team of developers from Scopyleft and they look like very good candidates. But they re heavy users of the Scrum development method and asked me to play the role of product owner . So I started to describe the project with user stories (i.e. create the backlog in the Scrum jargon), you can have a look at them here on trello.com. If you re interested by the topic of free software funding, feel free to review and to send me your comments. My goal is clearly to have a minimal viable product with the first iteration(s) that I fund and then use the platform itself to fund further developments of the project. Thanks See you next month for a new summary of my activities.

5 comments Liked this article? Click here. My blog is Flattr-enabled.

1 March 2013

Daniel Pocock: Monitoring JBoss, Tomcat and other application servers with JMXetric

JMXetric 1.0 was finally released in February, let's have a quick look at how conveniently it can be integrated with an application server. A more thorough analysis of gmetric4j and JMXetric can be found in the sections I contributed to the new O'Reilly book Monitoring with Ganglia General comments about JVM instrumentation and the JVM boot classpath Monitoring agents (there are a number of them out there) are loaded at an early stage by the JVM and it needs to have access to any of their dependencies through the boot classpath. JMXetric explicitly declares it's dependencies in MANIFEST.MF:
Boot-Class-Path: oncrpc-1.0.7.jar gmetric4j-1.0.2.jar
 As no absolute path is specified, the JVM will look for those JARs in the same directory where the jmxetric.jar is located. Furthermore, the JVM will expect the exact filenames to match. If other filenames are used (e.g. for a newer version of gmetric), then the JVM command line needs to specify the filenames and locations, for example:
-Xbootclasspath/a:/opt/ganglia/jmxetric/gmetric4j-1.0.3.jar
 Starting JBoss 7 with JMXetric To start with, unpack a vanilla JBoss distribution:
$ mkdir ~/opt
$ cd ~/opt
$ tar xzf ~/Downloads/jboss-as-7.1.0.Final.tar.gz
 Now get the JMXetric code and put it, together with dependencies, in the same directory:
$ mkdir -p ~/opt/ganglia/jmxetric
$ cd ~/opt/ganglia/jmxetric
$ wget http://search.maven.org/remotecontent?filepath=info/ganglia/jmxetric/jmxetric/1.0.2/jmxetric-1.0.2.jar
$ wget http://search.maven.org/remotecontent?filepath=info/ganglia/gmetric4j/gmetric4j/1.0.2/gmetric4j-1.0.2.jar
$ wget http://search.maven.org/remotecontent?filepath=org/acplt/oncrpc/1.0.7/oncrpc-1.0.7.jar
 Set up a config file for JMXetric:
$ mkdir -p ~/opt/ganglia/jmxetric/etc
$ cd ~/opt/ganglia/jmxetric/etc
$ wget https://raw.github.com/ganglia/jmxetric/master/etc/jmxetric.xml
$ cp jmxetric.xml jmxetric-jboss.xml
$ vi jmxetric-jboss.xml
 and insert the following code: 
                <!-- report web server request count -->
		<mbean name="jboss.as:subsystem=web,connector=http" pname="http">
			<attribute name="requestCount" type="int32"/>
		</mbean>
Finally, tell JBoss about the agent:
$ cd ~/opt/jboss-as-7.1.0.Final
$ vi bin/standalone.conf
 and apply the following changes to standalone.conf: 
--- bin/standalone.conf
+++ ../jboss-as-7.1.0.Final.for-jmxetric/bin/standalone.conf
@@ -4,6 +4,7 @@
 ##                                                                          ##
 ##############################################################################
 
+JBOSS_MODULES_SYSTEM_PKGS="org.jboss.logmanager"
 #
 # This file is optional; it may be removed if not needed.
 #
@@ -48,6 +49,8 @@
 #
 if [ "x$JAVA_OPTS" = "x" ]; then
    JAVA_OPTS="-Xms64m -Xmx512m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000"
+   JAVA_OPTS="$JAVA_OPTS -Djava.util.logging.manager=org.jboss.logmanager.LogManager -Xbootclasspath/p:$JBOSS_HOME/modules/org/jboss/logmanager/main/jboss-logmanager-1.2.2.GA.jar -Xbootclasspath/p:$JBOSS_HOME/modules/org/jboss/logmanager/log4j/main/jboss-logmanager-log4j-1.0.0.GA.jar -Xbootclasspath/p:$JBOSS_HOME/modules/org/apache/log4j/main/log4j-1.2.16.jar"
+   JAVA_OPTS="$JAVA_OPTS -javaagent:$HOME/opt/ganglia/jmxetric/jmxetric-1.0.2.jar=host=jboss-host.example.org,port=8649,wireformat31x=true,config=$HOME/opt/ganglia/jmxetric/etc/jmxetric-jboss.xml"
    JAVA_OPTS="$JAVA_OPTS -Djboss.modules.system.pkgs=$JBOSS_MODULES_SYSTEM_PKGS -Djava.awt.headless=true"
    JAVA_OPTS="$JAVA_OPTS -Djboss.server.default.config=standalone.xml"
 else
Note that we are adding some of the core JBoss logging code to the boot classpath - that is because JMXetric depends on the logger, and like many monitoring agents, it is impacted by JBoss bug AS-3681. Adding these extra classpath settings is the workaround. Now go ahead and start JBoss, you should see a log message informing you that JMXetric is operational: 
$ cd ~/opt/jboss-as-7.1.0.Final
$ chmod a+x bin/*.sh
$ bin/standalone.sh
=========================================================================
  JBoss Bootstrap Environment
  JBOSS_HOME: ...
.
.
.
=========================================================================
JMXetricAgent instrumented JVM, see https://github.com/ganglia/jmxetric
12:59:34,395 INFO  [info.ganglia.gmetric4j.GMonitor] Setting up 1 samplers
12:59:34,536 INFO  [org.jboss.modules] JBoss Modules version 1.1.1.GA
12:59:34,617 INFO  [org.jboss.msc] JBoss MSC version 1.0.2.GA
12:59:34,653 INFO  [org.jboss.as] JBAS015899: JBoss AS 7.1.0.Final "Thunder" starting
.
.
.
I hope this example is useful for you. Please feel free to share recipes you develop for collecting useful JVM metrics, and also any custom charts/reports through collaboration with the Ganglia community mailing list, the Ganglia wiki or your own blog.

Thomas Goirand: Openstack Grizzly (2013.1~g3) available

This post is just a status update for the Openstack packaging, after the next version froze last week. The Openstack bi-annual summit of next April will take place this year in Portland, Oregon, and if everything goes as plan, Grizzly will be released just before the summit. Grizzly will be out a bit before the next Ubuntu in April, as releases are following the one of Ubuntu. Openstack uses town names for it s release names: Austin, Bextar, Cactus (2011.1), Diablo (2011.2), Essex (2012.1), Folsom (2012.2), Grizzly (2013.1). I started to seriously work on the Openstack packaging in October, and never stopped working on its packaging until now. Slowly, but surely, preparing all the packages and their dependency python modules. One package at a time, working all this every day. Folsom works now pretty well and can be used in production, and I maintain it for security (along with Essex, which is in Wheezy). Then Grizzly was frozen last week, on the 22nd of February, with the G3 release. As I already worked on packaging the G2 release in January, managing the packaging of G3 was fast. On late Sunday, I had a repository with Grizzly built, and its corresponding python (build-)dependencies. But while just building your own repository is easy, having all the dependencies in Debian is a lot more work. As of today, if I include all the python modules, I have (at least) touch around 50 packages in total when working on Openstack. Many of them were simply built from scratch. The only python dependency that needs an upgrade in Experimental, so that all dependencies are satisfied, is a new version of pep8. The rest of is new python modules that were not in Debian, and which are currently waiting in the NEW queue for ftp masters approval: python-pecan, python-tablib, python-wsme and websockify. Some of these python modules have been waiting there for a long time, like python-pecan (it s been waiting in the NEW queue for more than 35 days now), some like websockify and python-wsme have been uploaded only this week. I really hope it can be possible to have all of Grizzly in Debian before the next Openstack summit (this depends mainly on the ftp-masters). Note that I do not intend to apply security patches in Grizzly until it is released as the new Openstack stable. So use my private Grizzly repository as your own risk. I intend to fix this by working on some constant integration to have nightly builds, like many people are doing with Openstack. If you want to try it out: deb http://archive.gplhost.com/debian grizzly main

28 February 2013

Erich Schubert: ELKI data mining in 2013

ELKI, the data mining framework I use for all my research, is coming along nicely, and will see continued progress in 2013. The next release is scheduled for SIGMOD 2013, where we will be presenting the novel 3D parallel coordinates visualization we recently developed. This release will bear the version number 0.6.0.
Version 0.5.5 of ELKI is in Debian unstable since december (Version 0.5.0 will be in the next stable release) and Ubuntu raring. The packaged installation can share the dependencies with other Debian packages, so they are smaller than the download from the ELKI web site.
If you are developing cluster analysis or outlier detection algorithm, I would love to see them contributed to ELKI. If I get a clean and well-integrated code by mid june, your algorithm could be included in the next release, too. Publishing your algorithms in source code in a larger framework such as ELKI will often give you more citations. Because it is easier to compare with your algorithm then and to try it on new problems. And, well, citations counts are a measure that administration loves to judge researchers ...
So what else is happening with ELKI:
  • The new book "Outlier Analysis" by C. C. Aggarwal mentions ELKI for visual evaluation of outlier results as well as in the "Resources for the Practioner" section and cites around 10 publications closely related to ELKI.
  • Some classes for color feature extraction of ELKI have been contributed to jFeatureLib, a Java library for feature detection in image data.
  • I'd love to participate in the Google Summer of Code, but I need a contact at Google to "vouch" for the project, otherwise it is hard to get in. I've been sending a couple of emails, but so far have not heard back much yet.
  • As the performance of SVG/Batik is not too good, I'd like to see more OpenGL based visualizations. This could also lead to an Android based version for use on tablets.
  • As I'm not an UI guy, I would love to have someone make a fancier UI that still exposes all the rich functions we have. The current UI is essentially an automatically generated command line builder - which is nice, as new functionality shows up without the need to modify UI code. It's good for experienced users like me, but hard for beginners to get started.
  • I'd love to see integration of ELKI with e.g. OpenRefine / Google Refine to make it easier to do appropriate data cleaning and preprocessing
  • There is work underway for a distributed version running on Hadoop/YARN.

27 February 2013

Julien Danjou: OpenStack Ceilometer and Heat projects graduated

The OpenStack Technical Committee have voted these last weeks about graduation of Heat and Ceilometer, to change their status from incubation to integrated. The details of the discussion can be found in the TC IRC meetings logs for the brave. The results are:
Approve graduation of Heat (to be integrated in common Havana release)?
yes: 10, abstain: 1, no: 1
Approve graduation of Ceilometer (to be integrated in common Havana release)?
yes: 11, abstain: 1
Therefore both projects have been graduated from Incubation to Integrated status. That means that Heat and Ceilometer will be released as part as OpenStack for the next release cycle Havana, due in Autumn 2013. For people being curious, we the Ceilometer team put up a nice wiki page about our status and what we think we were ready to jump. For the curious, The OpenStack Technical Committee charter has some explanations about the incubation and integration process. What about Grizzly? Both projects will be released with Grizzly too, obviously, since they already follow the release process of OpenStack. What about core?
The question that has been raised several times to me is if that means the projects are becoming Core projects. The answer is no, because how to become a Core project is still under discussion and is more a matter for the Board of Directors than the Technical Committee. But this is definitely a step in this direction. Anyway, from a technical point of view, this means both projects are now onboard with other OpenStack components so you can enjoy them!

26 February 2013

Ian Campbell: LinaroConnect 2013, Talks and Demos

My colleague Stefano Stabellini and I are in the process of putting the finishing touches to our presentation which we are giving at Linaro Connect next week in Hong Kong. If you are there come and see Introducing XEN on ARM on Monday at 1100, we'll be running through some of the basics of Xen as well as the specifics of how this translates into use on the ARM virtualised platform and talking about our status and roadmap going forward. As well as that we will also be demoing Xen on ARM, both on v7 hardware platforms and on v8 software models, at Demo Friday. I've been mostly working on the 64-bit port recently and the initial set of 45(!) patches was committed at the end of last week --- just in time! The other main Xen on ARM event at this years Connect is a keynote on the Tuesday morning given by Lars Kurth (Xen.org Community Manager) and Mark Heath (XenServer VP of Engineering, and also my boss). Other than that my schedule seems to be running over with other interesting sounding talks and there are certainly going to be plenty of people to catch up with, but hopefully I'll have enough time to put the finishing touches onto ARMv8 64-bit guest support in the hackrooms like I'm planning... Looking forward to seeing some of you there!

Russ Allbery: Review: Witches Abroad

Review: Witches Abroad, by Terry Pratchett
Series: Discworld #12
Publisher: Harper
Copyright: 1991
Printing: February 2008
ISBN: 0-06-102061-3
Format: Mass market
Pages: 350
Witches Abroad is the twelfth book in the Discworld series, but there's no strong reason why you couldn't start here. It's the second book about the three witches (Granny Weatherwax, Nanny Ogg, and Magrat Garlick), following Wyrd Sisters, and the third book about Granny Weatherwax, but it doesn't assume any knowledge of the previous books or of Discworld in general. Desiderata Hollow is dying. This is not a surprise; she's known for a long time exactly when she was going to die, and that time has come. She's not particularly bothered by it, but it does pose one difficult problem. She's a fairy godmother, as well as a witch, and she's balancing another fairy godmother because they come in pairs. With her dead, Lilith seems likely to win their long struggle, and that would be a bad thing. She needs to pass her wand on to a successor, but she wasn't the planning sort. At least she claims; this comment of hers turns out to hide an elegant bit of manipulation that leaves her wand in the hands of Magrat and sends the three witches on their way to the city of Genua to confront Lilith. There are two halves to Witches Abroad. The first half is the setup described above and their trip across Discworld. The second half is a sort of retelling of Cinderella, as foreshadowed early on by Desiderata. But the whole book is about stories and story shapes and their head-on collisions with the practical confidence of Nanny Ogg and the endless stubbornness of Granny Weatherwax. I think the three witches are one of Pratchett's best inventions, and Granny Weatherwax the best of the three. She has many of the characteristics that would normally relegate her to a supporting mentor figure: older, very knowledgeable, remarkably good at getting her way and at manipulating situations, and not particularly interested in going on adventures. But Pratchett turns this on end at every opportunity. Granny Weatherwax doesn't act like a mentor at all: she doesn't explain things to Magrat, or anyone else; she's almost constantly bickering with the other witches; and she's a fount of irritated and exasperated opinions about the world. She's one of the best irascibly competent characters in fiction, and she steals every scene she's in, refusing to become a supporting character and staying at the center of the story, despite not being a viewpoint character. (Pratchett wisely keeps her out of the viewpoint role for most of the story, since much of her knowledge and "headology" would be less effective if the reader knew what she was thinking.) The dynamics between the three witches are the best part of this book, and I love all three of them. But I'm also a sucker for fractured fairy tales in general, and Witches Abroad is full of them. Their opponent loves stories and has gathered most of her power from reinforcing and manipulating stories, which means that the density of random stories keeps increasing as they approach Genua. That provides numerous opportunities for the witches to stumble through, analyze, take apart, twist, or untwist numerous familiar stories, ranging from Red Riding Hood to Sleeping Beauty. Pratchett keeps things moving right along, not belaboring connections and not explaining too much, which makes it very easy to keep turning pages. The story does get a bit slower in Genua. There's quite a bit of scene-setting and figuring things out, and while Pratchett maintains the steady stream of humor, it isn't quite as compelling without the ever-changing context. He introduces a bit of voodoo magic, which I found to be the least interesting of the multiple styles of magic that show up here. But there are some absolutely brilliant bits with Nanny Ogg's cat, a wonderfully annoyed Cinderella, and an inversion of the traditional story that kept surprising me. I thought the ending was a bit of an anticlimax, but it does fit the nature of the characters. Despite a few plot short-comings, this is my favorite Discworld book up to this point in the series. Definitely recommended, and not a horrible starting point. Followed by Small Gods in the chronological sense and (later) by Lords and Ladies in the plot sense. Rating: 8 out of 10

25 February 2013

Francesca Ciceri: Three Harry Potter crossovers you want to read

...at least, if you are into fan fictions and Harry Potter. I love fan fictions, and I'll shamelessly admit that I like Harry Potter too, but I usually avoid crossovers. That is because however interesting and exciting is - in theory - to have your favourite characters from different fandoms together, it often ends bad. So, it's a public duty of every fanfic reader to recommend good crossovers, if they stumble upon them. Here are some I found recently (all the quotes are from the stories' summaries). If you read any of these and want to talk about it or if you have more crossovers to recommend, drop me a mail. For more fan fiction recommendations, I suggest you to check this detailed index on tvtropes.org.

Next.

Previous.