Search Results: "will"

11 August 2014

Sylvestre Ledru: clang 3.4, 3.5 and 3.6 are now coinstallable in Debian

Clang is finally co installable on Debian. 3.4, 3.5 and the current trunk (snapshot) can be installed together. So, just like gcc, the different version can be called with clang-3.4, clang-3.5 or clang-3.6. /usr/bin/clang, /usr/bin/clang++, /usr/bin/scan-build and /usr/bin/scan-view are now handled through the llvm-defaults package. llvm-defaults is also now managing clang-check, clang-tblgen, c-index-test, clang-apply-replacements, clang-tidy, pp-trace and clang-query. Changes are also available on llvm.org/apt/.
The next step will be to manage also llvm-defaults on llvm.org/apt to simplify the transition for people using these packages. So, with:
# /etc/apt/sources.list
deb http://llvm.org/apt/unstable/ llvm-toolchain main
deb http://llvm.org/apt/unstable/ llvm-toolchain-3.4 main
deb http://llvm.org/apt/unstable/ llvm-toolchain-3.5 main
$ apt-get install clang-3.4 clang-3.5 clang-3.6
$ clang-3.4 --version
Debian clang version 3.4.2 (branches/release_34) (based on LLVM 3.4.2)
Target: x86_64-pc-linux-gnu
Thread model: posix
$ clang-3.5 --version
Debian clang version 3.5.0-+rc2-1~exp1 (tags/RELEASE_350/rc2) (based on LLVM 3.5.0)
Target: x86_64-pc-linux-gnu
Thread model: posix
$ clang-3.6 --version
Debian clang version 3.6.0-svn214990-1~exp1 (trunk) (based on LLVM 3.6.0)
Target: x86_64-pc-linux-gnu
Thread model: posix

Ian Donnelly: dpkg Woes

Hi Everybody, The original plan for my Google Summer of Code project involved creating a merge tool for Elektra including a kdb merge command, that part is all finished and integrated into the newest release of Elektra, 0.8.7 already. The next step was to patch dpkg to add an option for conffile merging. The basic overview of how this would work is: Obviously, all of these things require patching dpkg, although not in too large away. Luckily, we found a previous attempt to patch dpkg for a three-way merge hook, however it is a few years old and was never included in dpkg. Felix decided to update this patch to work with the new versions of dpkg and cleaned up a bit of redundancy, he then resubmitted it to see if the dpkg maintainers would be interested in such a patch, and to start a dialogue about including all the patches we need. Unfortunately there has been no response from the dpkg team, any many bugs on their tracker have been sitting unanswered for years. Additionally, dpkg s source can be quite complex and it would require a ton of effort for us to make all these patches with no guarantee of integration. As a result, we have decided to go another route (which I will be posting about soon). I just wanted to update anybody interested on the progress of my Google Summer of Code Project by letting them know the experience with dpkg. We have come up with a new solution that is already looking great and you will hear a lot more about it this final week of Google Summer of Code. Stay Tuned!
- Ian S. Donnelly

10 August 2014

Matthew Garrett: Birthplace

For tedious reasons, I will at this stage point out that I was born in Galway, Ireland.

comment count unavailable comments

Ian Wienand: Finding out if you're a Rackspace instance

Different hosting providers do things slightly differently, so it's sometimes handy to be able to figure out where you are. Rackspace is based on Xen and their provided images should include the xenstore-ls command available. xenstore-ls vm-data will give you a handy provider and even region fields to let you know where you are.
function is_rackspace  
  if [ ! -f /usr/bin/xenstore-ls ]; then
      return 1
  fi
  /usr/bin/xenstore-ls vm-data   grep -q "Rackspace"
 
if is_rackspace; then
  echo "I am on Rackspace"
fi
Other reading about how this works:

Russ Allbery: Orphaning some software packages

As mentioned in my previous message, I'm orphaning various software packages that I was maintaining as part of my job at Stanford, or that for some other reason I'm no longer using directly. My goal is to free up some time and space for new work projects at my new employer, for other hobbies, and to take better care of other projects that I'm not orphaning. The following software packages are now orphaned, and marked as such on their web pages: I'm also stepping down from Debian package maintenance for the OpenAFS and Shibboleth packages, and have already notified the relevant communities. For the Debian packages, and for the above software packages, I will continue to provide security support until someone else can take them over. WebAuth is going to be in a state of transition as noted on its page: My successor at Stanford will be continuing maintenance and development, but that person hasn't been hired yet, and it will take some time for them to ramp up and start making new releases (although there may be at least one interim release with work that I'm finishing now). It's therefore not strictly orphaned, but it's noted that way on my software pages until someone else at Stanford picks it up. Development of the other packages that I maintain should continue as normal, with a small handful of exceptions. The following packages are currently in limbo, since I'm not sure if I'll have continued use for them: I'm not very happy with the current design of either kadmin-remctl or wallet, so if I do continue to maintain them (and have time to work on them), I am likely to redesign them considerably. For all of my packages, I've been adding clones of the repository to GitHub as an additional option besides my personal Git repository server. I'm of two minds about using (and locking myself into) more of the GitHub infrastructure, but repository copies on GitHub felt like it might be useful for anyone who wanted to fork or take over maintenance. I will be adding links to the GitHub repositories to the software packages for things that are in Git. If you want to take over any of the orphaned software packages, feel free. When you're ready for the current software pages to redirect to its new home, let me know.

Ian Donnelly: How-To: kdb merge

Hi Everybody, As you may know, part of my Google Summer of Code project involved the creation of merge tools for Elektra. The one I am going to focus on today is kdb merge. The kdb tool allows users to access and perform functions on the Elektra Key Database from the command line. We added a new command to this very useful tool, the merge command. This command allows a user to perform a three-way merge of KeySets from the kdb tool.
The command to use this tool is:
kdb merge [options] ourpath theirpath basepath resultpath

The standard naming scheme for a three-way merge consists of ours, theirs, and base: This works very similarly for KeySets, especially ones that consist of mounted conffiles. For mounted conffiles: If the user is just trying to accomplish is a three-way merge using any two arbitrary keysets that share a base,
it doesn t matter which ones are defined as ours or theirs as long as they use the correct base KeySet.
In kdb merge, ourpath, theirpath, and basepath work just like ours, theirs, and base except each one represents the
root of a KeySet. Resultpath is pretty self-explanatory, it is just where you want the result of the merge to be saved under.
It s worth noting, resultpath should be empty before attempting a merge, otherwise there can be unintended consquences. As for the options, there are a few basic options:

-i --interactive which attempts the merge in an interactive way
-t --test which tests the proposed merge and informs you about possible conflicts
-f --force which overwrites any Keys in resultpath
Additionally there is an option to specify a merge strategy, which is very important. The other option, strategy is:
s --strategy which is used to specify a strategy to use in case of a conflict

The current list of strategies are:
preserve the merge will fail if a conflict is detected
ours the merge will use our version during a conflict
theirs the merge will use their version during a conflict
base the merge will use the base version during a conflict
If no strategy is specified, the merge will default to the preserve strategy as to not risk making the wrong decision. If any of the other strategies are specified, when a conflict is detected, merge will use the Key specified by the strategy (ours, theirs, or base) for the resulting Key. An example of using the kdb merge command:
kdb merge -s ours system/hosts/ours system/hosts/theirs system/hosts/base system/hosts/result
For more in depth examples, please see the full tutorial on our GitHub Documentation. -Ian S. Donnelly

9 August 2014

Russell Coker: Being Obviously Wrong About Autism

I m watching a Louis Theroux documentary about Autism (here s the link to the BBC web site [1]). The main thing that strikes me so far (after watching 7.5 minutes of it) is the bad design of the DLC-Warren school for Autistic kids in New Jersey [2]. A significant portion of people on the Autism Spectrum have problems with noisy environments, whether most Autistic people have problems with noise depends on what degree of discomfort is considered a problem. But I think it s reasonable to assume that the majority of kids on the Autism Spectrum will behave better in a quiet environment. So any environment that is noisy will cause more difficult behavior in most Autistic kids and the kids who don t have problems with the noise will have problems with the way the other kids act. Any environment that is more prone to noise pollution than is strictly necessary is hostile to most people on the Autism Spectrum and all groups of Autistic people. The school that is featured in the start of the documentary is obviously wrong in this regard. For starters I haven t seen any carpet anywhere. Carpeted floors are slightly more expensive than lino but the cost isn t significant in terms of the cost of running a special school (such schools are expensive by private-school standards). But carpet makes a significant difference to ambient noise. Most of the footage from that school included obvious echos even though they had an opportunity to film when there was the least disruption presumably noise pollution would be a lot worse when a class finished. It s not difficult to install carpet in all indoor areas in a school. It s also not difficult to install rubber floors in all outdoor areas in a school (it seems that most schools are doing this already in play areas for safety reasons). For a small amount of money spent on installing and maintaining noise absorbing floor surfaces the school could achieve better educational results. The next step would be to install noise absorbing ceiling tiles and wallpaper, that might be a little more expensive to install but it would be cheap to maintain. I think that the hallways in a school for Autistic kids should be as quiet as the lobby of a 5 star hotel. I don t believe that there is any technical difficulty in achieving that goal, making a school look as good as an expensive hotel would be expensive but giving it the same acoustic properties wouldn t be difficult or expensive. How do people even manage to be so wrong about such things? Do they never seek any advice from any adult on the Autism Spectrum about how to run their school? Do they avoid doing any of the most basic Google searches for how to create a good environment for Autistic people? Do they just not care at all and create an environment that looks good to NTs? If they are just trying to impress NTs then why don t they have enough pride to care that people like me will know how bad they are? These aren t just rhetorical questions, I d like to know what s wrong with those people that makes them do their jobs in such an amazingly bad way.

Steve Kemp: Rebooting the CMS

I run a cluster for the Debian Administration website, and the code is starting to show its age. Unfortunately the code is not so modern, and has evolved a lot of baggage. Given the relatively clean separation between the logical components I'm interested in trying something new. In brief the current codebase allows: It crossed my mind the other night that it might make sense to break this code down into a number of mini-servers - a server to handle all article-related things, a server to handle all poll-related things, etc. If we have a JSON endpoint that will allow: Then we could have a very thin shim/server on top of that whihc would present the public API. Of course the internal HTTP overhead might make this unworkable, but it is an interesting approach to the problem, and would allow the backend storage to be migrated in the future without too much difficulty. At the moment I've coded up two trivial servers, one for getting user-data (to allow login requests to succeed), and one for getting article data. There is a tiny presentation server written to use those back-end servers and it seems like an approach that might work. Of course deployment might be a pain.. It is still an experiment rather than a plan, but it could work out: http://github.com/skx/snooze/.

Russ Allbery: A new challenge

Those reading this journal may have noticed that my rate of posting has dropped a bit in the past few years, and quite a lot in the past year. One of the major reasons for this was work, which had been getting more bureaucratic, more stressful, less trusting, and more fearful. After this got drastically worse in the past six months, I finally decided enough was enough and took advantage of a good opportunity to do something different. I will be joining Dropbox's site reliability engineering team in a week and a half (which means that I'll be working on their servers, not on the product itself). It will take a few months to settle in, but hopefully this will mean a significant improvement to my stress levels and a lot of interesting projects to work on. I'm taking advantage of this change to inventory the various things I'm currently committed to and let go of some projects to make more space in my life. There are also a variety of software projects that I was maintaining as part of my job at Stanford, and I will be orphaning many of those packages. I'll make another journal post about that a bit later. For Debian folks, I am going to be at Debconf, and hope to meet many of you there. (It's going to sort of be my break between jobs.) In the long run, I'm hoping this move will let me increase my Debian involvement. In the long run, I expect most of my free software work, my reviews, and the various services I run to continue as before, or even improve as my stress drops. But I've been at Stanford for a very long time, so this is quite the leap into the unknown, and it's going to take a while before I'm sure what new pattern my life will fall into.

8 August 2014

Jan Wagner: Monitoring Plugins Debian packages

You may wonder why the old good nagios-plugins are not up to date in Debian unstable and testing. Since the people behind and maintaining the plugins <= 1.5 were forced to rename the software project into Monitoring Plugins there was some work behind the scenes and much QA work necessary to release the software in a proper state. This happened 4 weeks ago with the release of the version 2.0 of the Monitoring Plugins. With one day of delay the package was uploaded into unstable, but did hit the Debian NEW queue due the changed package name(s). Now we (and maybe you) are waiting to get them reviewed by ftp-master. This will hopefully happen before the jessie freeze. Until this will happen, you may grab packages for wheezy by the 'wheezy-backports' suite from ftp.cyconet.org/debian/ or 'debmon-wheezy' suite from debmon.org. Feedback is much appreciated.

Ian Donnelly: The Line Plug-In

Hi Everybody, As you may have noticed I wrote a new plug-in for Elektra called line . I used it for a lot of examples in my tutorial, How-To: Write a Plug-In. The line plug-in is a very simple storage plug-in for Elektra. This plug-in stores files into the Elektra Key Database creating a new Key for each line and setting the string value of the Key to the string value of the line of that file. So if we have a file called hello.txt :

Hello
World!

And we mount it to kdb like so: kdb mount ~/line.txt system/hello_line line. The output of kdb ls system/hello_line the output would be:

system/hello_line/#1
system/hello_line/#
2

With the getString values of #1 and #2 being Hello and World! respectively. If this seems like a very simple plug-in, that s because it is. Obviously, this plug-in isn t a great showcase for the robustness of Elektra, any data structure could store a file line by line relatively easy, so why did we add a line plug-in at all? The answer is that we included a line plug-in to allow any line-based file to use functions of Elektra, particularly the new Merge function. My Google Summer of Code project is to allow for automatic three-way merge of Debian conffiles during package upgrades as opposed to the current prompt and manual merging a user must do if a conffile is edited. Using Elektra and the new mergecode we can mount a conffile with the best plug-in for it, (the ini plug-in for Samba s smb.conf for instance) and that allows for a very powerful merging ability with a lot more success than a simple diff merge. However, there are a lot of conffiles that don t use any particular standard (such as ini, xml, or JSON) to store data. That is where the line plug-in comes in. We can still mount these files using the line plug-in and attempt a merge. Of course it is much more likely to have conflicts, and this type of merge is still susceptible to many of the same flaws as regular file merges (such as not being able to detect when a line has been moved), but it simple cases, the merge may succeed which would reduce the overall number of times a user would be prompted during an upgrade. Basically, I wrote a line plug-in for Elektra as a fallback for conffile merges when we can t mount the conffile in any more meaningful way. While merges using KeySets that were mounted using line are more likely to fail than other, more specialized plug-ins, there are cases that these merges will succeed and the user will not have to deal with a confusing prompt. The whole point of my Google Summer of Code project is to make upgrading packages and dealing with conffiles much smoother and easier than it is now by including a three-way mere and this line plug-in will help with this goal. Sincerely,
Ian S. Donnelly

Ian Donnelly: How-To: Write a Plug-In (Part 3, Coding)

Hi Everybody! Hope you have been enjoying my tutorial on writing plug-ins so far. In Part 1 we covered the basic overview of a plug-in. Part 2 covered a plug-in s contract and the best way to write one. Now, for Part 3 we are going to cover the meat of a plug-in, the actual coding. As you should know from reading Part 1, there are five main functions used for plug-ins, elektraPluginOpen, elektraPluginGet, elektraPluginSet, ELEKTRA_PLUGIN_EXPORT(Plugin), where Plugin should be replaced with the name of your plug-in. We are going to start this tutorial by focusing on the elektraPluginGet because it usually is the most critical function. As we discussed before, elektraPluginGet is the function responsible for turning information from a file into a usable KeySet. This function usually differs pretty greatly between each plug-in. This function should be of type int, it returns 0 on success or another number on an error. The function will take in a Key, usually called parentKey which contains a string containing the path to the file that is mounted. For instance, if you run the command kdb mount /etc/linetest system/linetest line then keyString(parentKey) should be equal to /etc/linetest . At this point, you generally want to open the file so you can begin saving it into keys. Here is the trickier part to explain. Basically, at this point you will want to iterate through the file and create keys and store string values inside of them according to what your plug-in is supposed to do. I will give a few examples of different plug-ins to better explain. My line plug-in was written to read files into a KeySet line by line using the newline character as a delimiter and naming the keys by their line number such as (#1, #2, .. #_22) for a file with 22 lines. So once I open the file given by parentKey, every time a I read a line I create a new key, let s call it new_key using dupKey(parentKey). Then I set new_keys s name to lineNN (where NN is the line number) using keyAddBaseName and store the string value of the line into the key using keySetString. Once the key is initialized, I append it to the KeySet that was passed into the elektraPluginGet function, let s call it returned for now, using ksAppendKey(return, new_key). Now the KeySet will contain new_key with the name lineNN properly saved where it should be according to the kdb mount command (in this case, system/linetest/lineNN), and a string value equal to the contents of that line in the file. MY plug-in repeats these steps as long as it hasn t reached end of file, thus saving the whole file into a KeySet line by line. The simpleini plug-in works similarly, but it parses for ini files instead of just line-by-line. At their most simple level, ini files are in the format of name=value with each pair taking one line. So for this plug-in, it makes a lot of sense to name each Key in the KeySet by the string to the left of the = sign and store the value into each key as a string. For instance, the name of the key would be name and keyGetString(name) would return value .
As you may have noticed, simpleini and line plug-ins work very similarly. However, they just parse the files differently. The simpleini plug-in parses the file in a way that is more natural to ini file (setting the key s name to the left side of the equals sign and the value to the right side of the equals sign.) The elektraPluginGet function is the heart of a storage plug-in, its what allows Elektra to store configurations in it s database. This function isn t just run when a file is first mounted, but whenever a file gets updated, this function is run to update the Elektra Key Database to match. We also gave a brief overview of elektraPluginSet function. This function is basically the opposite of elektraPluginGet. Where elektraPluginGet reads information from a file into the Elektra Key Database, elektraPluginSet writes information from the database back into the mounted file. First have a look at the signature of elektraLineSet: elektraLineSet(Plugin *handle ELEKTRA_UNUSED, KeySet *toWrite, Key *parentKey) Lets start with the most important parameters, the KeySet and the parentKey. The KeySet supplied is the KeySet that is going to be persisted in the file. In our case it would contain the Keys representing the lines. The parentKey is the topmost Key of the KeySet at serves several purposes. First, it contains the filename of the destination file as its value. Second, errors and warnings can be emitted via the parentKey. We will discuss error handling in more detail later. The Plugin handle can be used to persist state information in a threadsafe way with elektraPluginSetData. As our plugin is not stateful and therefore does not use the handle, it is marked as unused in order to supress compiler warnings. Basically the implementation of elektraLineSet can be described with the following pseudocode:

open the file
if (error)

ELEKTRA_SET_ERROR(74, parentKey, keyString(parentKey));

for each key

write the key value together with a newline

close the file
The fullblown code can be found at https://github.com/ElektraInitiative/libelektra/blob/master/src/plugins/line/line.c As you can see, all elektraLineSet does is open a file, take each Key from the KeySet (remember they are named #1, #2 #_22) in order, and write each key as it s own line in the file. Since we don t care about the name of the Key in this case (other than for order), we just write the value of keyString for each Key as a new line in the file. That s it. Now, each time the mounted KeySet is modified, elektraPluginSet will be called and the mounted file will be updated. We haven t discussed ELEKTRA_SET_ERROR yet. Because Elektra is a library, printing errors to stderr wouldn t be a good idea. Instead, errors and warnings can be appended to a key in the form of metadata. This is what ELEKTRA_SET_ERROR does. Because the parentKey always exists even if a critical error occurres, we append the error to the parentKey. The first parameter is an id specifying the general error that occurred. A listing of existing errors together with a short description and a categorization can be found at https://github.com/ElektraInitiative/libelektra/blob/master/src/liberror/specification. The third parameter can be used to provide additional information about the error. In our case we simply supply the filename of the file that caused the error. The kdb tools will interprete this error and print it in a pretty way. Notice that this can be used in any plugin function where the parentKey is available. The elektraPluginOpen and elektraPluginClose functions are not commonly used for storage plug-ins, but they can be useful and are worth reviewing. elektraPluginOpen function runs before elektraPluginGet and is useful to do initialization if necessary for the plug-in. On the other hand elektraPluginClose is run after other functions of the plug-in and can be useful for freeing up resources. The last function, one that is always needed in a plug-in, is ELEKTRA_PLUGIN_EXPORT. This functions is responsible for letting Elektra know that the plug-in exists and which methods it implements. The code from my line function is a good example and pretty self-explanatory:

Plugin *ELEKTRA_PLUGIN_EXPORT(line)

return elektraPluginExport("line",
ELEKTRA_PLUGIN_GET, &elektraLineGet,
ELEKTRA_PLUGIN_SET, &elektraLineSet,
ELEKTRA_PLUGIN_END);

There you have it! This is the last part of my tutorial on writing a storage plug-in for Elektra. Hopefully you now have a good understanding of how Elektra plug-ins work and you ll be able to add some great functionality into Elektra through development of new plug-ins. I hope you enjoyed this tutorial and if you have any questions just leave a comment! Happy coding!
Ian S. Donnelly

7 August 2014

Jordi Mallach: A pile of reasons why GNOME should be Debian jessie s default desktop environment

GNOME has, for some reason or another, always been the default desktop environment in Debian since the installer is able to install a full desktop environment by default. Release after release, Debian has been shipping different versions of GNOME, first based on the venerable 1.2/1.4 series, then moving to the time-based GNOME 2.x series, and finally to the newly designed 3.4 series for the last stable release, Debian 7 wheezy . During the final stages of wheezy s development, it was pointed out that the first install CD image would not longer hold all of the required packages to install a full GNOME desktop environment. There was lots of discussion surrounding this bug or fact, and there were two major reactions to it. The Debian GNOME team rebuilt some key packages so they would be compressed using xz instead of gzip, saving the few megabytes that were needed to squeeze everything in the first CD. In parallel, the tasksel maintainer decided switching to Xfce as default desktop was another obvious fix. This change, unannounced and two days before the freeze, was very contested and spurred the usual massive debian-devel threads. In the end, and after a few default desktop flip flops, it was agreed that GNOME would remain as the default for the already frozen wheezy release, and this issue would be revisited later on during jessie s development. And indeed, some months ago, Xfce was again reinstated as Debian s default desktop for jessie as announced:
Change default desktop to xfce.
This will be re-evaluated before jessie is frozen. The evaluation will
start around the point of DebConf (August 2014). If at that point gnome
looks like a better choice, it ll go back as the default.
Some criteria for that choice will include:
* Popcon numbers for gnome on jessie. If gnome installations continue to
  rise fast enough despite xfce being the default (compared with, say
  kde installations), then we ll know that users prefer gnome.
  Currently we have no data about how many users would choose gnome when
  it s not the default. Part of the reason for switching to xfce now
  is to get such data.
* The state of accessability support, particularly for the blind.
* How well the UI works for both new and existing users. Gnome 3
  seems to be adding back many gnome 2 features that existing users
  expect, as well as making some available via addons. If it feels
  comfortable to gnome 2 (and xfce) users, that would go a long way
  toward switching back to it as the default. Meanwhile, Gnome 3 is also
  breaking new ground in its interface; if the interface seems more
  welcoming to new users, or works better on mobile devices, etc, that
  would again point toward switching back.
* Whatever size constraints exist for CD or other images at the time.
--
Hello to all the tech journalists out there. This is pretty boring.
Why don t you write a story about monads instead?
Joey Hess in dfca406eb694e0ac00ea04b12fc912237e01c9b5.
Suffice to say that the Debian GNOME team participants have never been thrilled about how the whole issue is being handled, and we ve been wondering if we should be doing anything about it, or just move along and enjoy the smaller amount of bug reports against GNOME packages that this change would bring us, if it finally made it through to the final release. During our real life meet-ups in FOSDEM and the systemd+GNOME sprint in Antwerp, most members of the team did feel Debian would not be delivering a graphical environment with the polish we think our users deserve, and decided we at least should try to convince the rest of the Debian project and our users that Debian will be best suited by shipping GNOME 3.12 by default. Power users, of course, can and know how to get around this default and install KDE, Xfce, Cinnamon, MATE or whatever other choice they have. For the average user, though, we think we should be shipping GNOME by default, and tasksel should revert the above commit again. Some of our reasons are: In short, we think defaulting to GNOME is the best option for the Debian release, and in contrast, shipping Xfce as the default desktop could mean delivering a desktop experience that has some incomplete or rough edges, and not on par with Debian quality standards for a stable release. We believe tasksel should again revert the change and be uploaded as soon as possible, in order to get people testing images with GNOME the sooner the better, with the freeze only two months away. We would also like that in the future, changes of this nature will not be announced in a git commit log, but widely discussed in debian-project and the other usual development/decision channels, like the change of init system happened recently. We will, whichever the final decision is, continue to package GNOME with great care to ensure our users get the best possible desktop experience Debian can offer.

Niels Thykier: Recent improvements to Britney2

As mentioned by Rapha l Hertzog, I have been spending some time on improving Britney2. Just the other day I submitted a second branch for review that I expect to merge early next week. I also got another set patches coming up soon. Currently, none of them are really user visible, so unless you are hosting your own version of Britney, these patches are probably not all that interesting to you. The highlights:
  1. Reduce the need for backtracking by finding semantically equivalent packages.
  2. Avoid needing to set up a backtrack point in some cases.
    • This has the side-effect of eliminating some O(e^n) runtime cases.
  3. Optimise installability testing of packages affected by a hinted migration.
    • This has the side-effect of avoiding some O(e^n) runtime cases when the post-hint state does not trigger said behaviour.
    • There is a follow-up patch for this one coming in the third series to fix a possible bug for a corner-case (causing a valid hint to be incorrectly rejected when it removed an uninstallable package).
  4. Reduce the number of affected packages to test when migrating items by using knowledge about semantically equivalent packages.
    • In some cases, Britney can now do free migrations when all binaries being updated replace semantically equivalent packages.
  5. (Merge pending) Avoid many redundant calls to sort_actions() , which exhibits at least O(n^2) runtime in some cases.
    • For the dataset Rapha l submitted, this patch shaves off over 30 minutes runtime. In the particular case, each call to sort_actions takes 3+ minutes and it was called at least 10 times, where it was not needed.
    • That said, sort_actions have a vastly lower runtime in the runs for Debian (and presumably also Ubuntu, since no one complained from their side so far).
The results so far: After the first patch series was merged, the Kali dataset (from Rapha l) could be processed in only ~2 hours. With the second patch series merged, the dataset will drop by another 30-50 minutes (most of which are thanks to the change mentioned in highlight #5). The third patch series currently do not have any mention-worthy performance related changes. It will probably be limited to bug fixes and some refactoring. Reflections: The 3 first highlights only affects the new installability tester meaning that the Britney2 instances at Ubuntu and Tanglu should be mostly unaffected by the O(n^2) runtime. Although those cases will probably just fail with several AIEEE s. :) The 5th highlight should equally interesting to all Britney2 instances though. For me, the most interesting part is that we have never observed the O(n^2) behaviour in a daily sid -> testing run. The dataset from Rapha l was basically a stable -> testing/sid run, which is a case I do not think we have ever done before. Despite our current updates, there is still room for improvements on that particular use case. In particular, I was a bit disheartened at how poorly our auto hinter(s) performed on this dataset. Combined they only assisted with the migration of something like 28 items . For comparison, the main run migrated ~7100 items and 9220 items were unable to migrate. Furthermore, the Original auto hinter spend the better part of 15 minutes computing hints at least it results in 10 items migrating. Links to the patches:

Lars Wirzenius: On ticketing systems

I don't really like any of the ticketing systems I've ever needed to use, whether they've been used as bug tracking systems, user support issue management systems, or something else. Some are not too bad. I currently rely most on debbugs and ikiwiki. debbugs is the Debian bug tracking system. See https://www.debian.org/Bugs/ for an entry point. It's mostly mail based, with a read-only web interface. You report a bug by sending an email to submission address, and (preferably) include a few magic "pseudo-headers" at the top of your message body ot identify the package and version. There's tools to make this easier, but mostly it's just about sending an e-mail. All replies are via e-mails as well. Effectively, each bug becomes is own little dedicated mailing list. This is important. A ticket, whether it is a bug report or a support request, is all about the discussion. "Hey I have this problem..." followed by "Have you tried..." and so forth. Anything that makes that discussion easier and faster to have is better. It is my very strong opinion, and long experience, that the best way to have such a discussion is over e-mail. A lot of modern ticketing systems are web based. They might have an e-mail mode, perhaps read-only, but that's mostly an afterthought. It's a thing bolted onto the side of the system because people like me whinge otherwise. I like e-mail for this for several reasons. For these reasons, I strongly prefer ticketing systems in which e-mails are the primary form of discussions, and e-mail is a first class citizen. I don't mind if there's other ways to participate in the discussion, but if I have to use something else than e-mail, I tend not to be happy. I use ikiwiki to provide a distributed, shared notebook on bugs. It's a bit cumbersome, and doesn't work well for discussions. I think we can improve on the way debbugs works, however. I've been thinking about ticketing systems for Obnam (my backup program), since it gaining enough users that it's getting hard to keep track of discussions with just an e-mail client. Here's what I want: I will eventually have this. I'm not saying I'm working on this, since I don't have enough free time to do that, but there's a git repository, and some code, and it imports e-mails automatically now. Some day there may even be a web interface. (This has been a teaser.)

Ian Donnelly: How-To: Write a Plug-In (Part 2, The Contract)

Hi Everybody! This is the second installment of my tutorial, How-To: Write a Plug-In. Previously, in Part 1, I went over the basic interface of an Elektra Storage Plug-In. For this installment, I will be focusing on one of the most important aspects of a plug-in, its contract. The contract describes how a plug-in may behave. The main part of the contract describes to Elektra how it will modify the KeySet returned. This is what allows Elektra to work with various types of plug-ins. This guide will focus on one type of plug-in: the storage plug-in. We already have a published specification for plug-ins on our repository but I will reiterate the details here. The first part of the contract you will need is the system/elektra/modules/plugin (where plugin represents the name of your plug-in). All this key does is let elektra know that your plug-in exists. The next key is system/elektra/modules/plugin/exports. This key lets Elektra know that your plug-in exports symbols to Elektra. Below this key you need to create keys for each function you want to export, in my line plug-in I implemented functions for get and set so I will create two keys as follows:
keyNew ("system/elektra/modules/line/exports/get",
KEY_FUNC, elektraLineGet, KEY_END),
keyNew ("system/elektra/modules/line/exports/set",
KEY_FUNC, elektraLineSet, KEY_END),
So far we have told Elektra that our plugin exists and will be exporting functions for getting and setting keys. The next set of keys are the system/elektra/Plugin/infos keys. These are pretty self-explanatory. The first three children of this key, author, license, and description are pretty self explanatory. The infos/provides key tells Elektra the type of plug-in that is being used, this will be storage for any storage plug-in (which is what this tutorial focuses on). To be precise, it tells Elektra which abstract functionality is provided. This allows other plugins to reference abstract functionality instead of declaring a hard dependency to one specific plugin. For example plugins could state that they want to be placed before any other plugin doing the abstract functionality conversion instead of referencing for example the keytometa plugin directly. The info/placements key tells Elektra which functions of the plug-in it should call. Since I implemented get and set in the line plug-in, I would set this key to getstorage setstorage . Next is infos/needs and infos/recommends, infos/needs should contain a list of plug-ins or providers that your plug-in needs to work (see the info/provides key above). Otherwise this key needs to be set to be blank . infos/recommends works exactly like needs except that a plug-in that needs another plug-in won t form a valid backend if the other plug-in doesn t exist but if it only recommends having the other plug-in, it will still work in the others absence. Since Elektra 0.8.7, the best practice for creating a plug-in involves storing the info keys at the very top of a file named REAMDE.md in the same directory as the plugin source. For instance, in my line plugin, the first few lines of README.md are as follows:
- infos = Information about LINE plugin is in keys below
- infos/author = Ian Donnelly <ian.s.donnelly@gmail.com>
- infos/licence = BSD
- infos/needs =
- infos/provides = storage
- infos/placements = getstorage setstorage
- infos/description = Very simple storage plug-in which stores each line from a file as a key
The rest of the README.md file should contain an English description of the plug-in. You can include an introduction to the plug-in, its purpose, any shortcomings it may have, and examples of how to use it. This README.md file is standard for GitHub which will automatically be displayed when a user is browsing your plug-in s directory in the repo. It is written in Markdown. In order to generate the contract from this file you must edit the CMakeLists.txt file in your plug-in source directory to include the following lines:
generate_readme (plugin)
add_includes (elektra-full $ CMAKE_CURRENT_BINARY_DIR )
include_directories ($ CMAKE_CURRENT_BINARY_DIR )

where you substitute plugin with the name of you plugin. This will generate a file called readme_plugin.c (once again, substitute plugin), in your plugin s build directory when you run the make command. The best way to declare the contract is within it s own function, elektraPluginContract. This makes it easier to manage the contract and is much more readable. This is the contract declaration for my line plug-in:

static inline KeySet *elektraLineContract()

return ksNew (30,
keyNew ("system/elektra/modules/line",
KEY_VALUE, "line plugin waits for your orders", KEY_END),
keyNew ("system/elektra/modules/line/exports", KEY_END),
keyNew ("system/elektra/modules/line/exports/get",
KEY_FUNC, elektraLineGet, KEY_END),
keyNew ("system/elektra/modules/line/exports/set",
KEY_FUNC, elektraLineSet, KEY_END),
#include "readme_line.c"
keyNew ("system/elektra/modules/line/infos/version",
KEY_VALUE, PLUGINVERSION, KEY_END),
KS_END);

Next, you will need to make sure to actually set the contract. This is always done in the elektraPluginGet function since this is technically the only REQUIRED function in a plug-in. We just create a new KeySet, set it to our contract, and append it to the returned KeySet. You can see how this is done for line (note that this code is at the very beginning of elektraLineGet):

if (!strcmp (keyName(parentKey), "system/elektra/modules/line"))

KeySet *moduleConfig = elektraLineContract();
ksAppend(returned, moduleConfig);
ksDel(moduleConfig);
return 1;


Note that the plugin contract is nothing special, but just a KeySet interpreted by the resolver forming an Elektra backend. Elektra will request the keys below system/elektra/modules/plugin (where plugin is the pluginname) as soon as it wants to know something about that plugin. Notice also the line #include "readme_line.c". This line, pulls in the contents of readme_line.c in my plug-in s build directory, which due to the lines we added in CMakeLists.txt will contain the contract as defined at the top of README.md. Any changes to the keys in README.md will be reflected in the plug-in next time it is built. Part 2 of this tutorial focuses on the plug-in contract by explaining what each element does and explaining how to implement a contract. For more information about plug-ins and more detailed examples I highly suggest reading Markus Thesis for Elektra. Chapter 4, which starts on page 61 goes into much more detail on Plug-Ins and explains the various features of the contracts. Look for Part 3 soon!

6 August 2014

Julian Andres Klode: Configuring an OpenWRT Router as a repeater for a FRITZ!Box with working Multicast

Since some time, those crappy Fritz!Box devices do not support WDS anymore, but rather a proprietary solution created by AVM. Now what happens if you have devices in another room that need/want wired access (like TVs, Playstations) or if you want to extend the range of your network? Buying another Fritz!Box is not very cost efficient What I did was to buy a cheap TP-Link TL-WR841N (can be bought for 18 euros) and installed OpenWRT on it. Here s how I configured it to act as a WiFi bridge. Basic overview: You configure OpenWRT into station mode (that is, as a WiFi client) and use relayd to relay between the WiFi network and your local network. You also need igmpproxy to proxy multicast packages between those networks, other UPnP stuff won t work. I did this on the recent Barrier Braker RC2. It should work on older versions as well, but I cannot promise it (I did not get igmpproxy to work in Attitude Adjustment, but that was probably my fault). Note: I don t know if it works with IPv6, I only use IPv4. You might want to re-start (or start) services after the steps, or reboot the router afterwards. Configuring WiFi connection to the FRITZ!Box Add to: /etc/config/network
config interface 'wwan'
	option proto 'dhcp'
(you can use any other name you want instead of wwan, and a static IP. This will be your uplink to the Fritz!Box) Replace wifi-iface in: /etc/config/wireless:
config wifi-iface
	option device 'radio0'
	option mode 'sta'
	option ssid 'FRITZ!Box 7270'
	option encryption 'psk2'
	option key 'PASSWORD'
	option network 'wwan'
(adjust values as needed for your network) Setting up the pseudo-bridge First, add wwan to the list of networks in the lan zone in the firewall. Then add a forward rule for the lan network (not sure if needed). Afterwards, configure a new stabridge network and disable the built-in DHCP server. Diff for /etc/config/firewall
@@ -10,2 +10,3 @@ config zone
 	list network 'lan'
+	list network 'wwan'
 	option input 'ACCEPT'
@@ -28,2 +29,7 @@ config forwarding
 
+# Not sure if actually needed
+config forwarding
+	option src 'lan'
+	option dest 'lan'
+
 config rule
Add to /etc/config/network
config interface 'stabridge'
	option proto 'relay'
	option network 'lan wwan'
	option ipaddr '192.168.178.26'
(Replace 192.168.178.26 with the IP address your OpenWRT router was given by the Fritz!Box on wlan0) Also make sure to ignore dhcp on the lan interface, as the DHCP server of the FRITZ!Box will be used: Diff for /etc/config/dhcp
@@ -24,2 +24,3 @@ config dhcp 'lan'
        option ra 'server'
+       option ignore '1'
Proxying multicast packages For proxying multicast packages, we need to install igmpproxy and configure it: Add to: /etc/config/firewall
# Configuration for igmpproxy
config rule
	option src      lan
	option proto    igmp
	option target   ACCEPT
config rule
	option src      lan
	option proto    udp
	option dest     lan
	option target   ACCEPT
(OpenWRT wiki gives a different 2nd rule now, but this is the one I currently use) Replace /etc/config/igmpproxy with:
config igmpproxy
	option quickleave 1
config phyint
	option network wwan
	option direction upstream
	list altnet 192.168.178.0/24
config phyint
	option network lan
	option direction downstream
	list altnet 192.168.178.0/24
(Assuming Fritz!Box uses the 192.168.178.0/24 network) Don t forget to enable the igmpproxy script:
# /etc/init.d/igmpproxy enable
Optional: Repeat the WiFi signal If you want to repeat your WiFi signal, all you need to do is add a second wifi-iface to your /etc/config/wireless.
config wifi-iface
	option device 'radio0'
	option mode 'ap'
	option network 'lan'
	option encryption 'psk2+tkip+ccmp'
	option key 'PASSWORD'
	option ssid 'YourForwardingSSID'
Known Issues If I was connected via WiFi to the OpenWRT AP and switch to the FRITZ!Box AP, I cannot connect to the OpenWRT router for some time. The igmpproxy tool writes to the log about changing routes. Future Work I ll try to get the FRITZ!Box replaced by something that runs OpenWRT as well, and then use OpenWRT s WDS support for repeating; because the FRITZ!Box 7270v2 is largely crap loading a page in its web frontend takes 5 (idle) 20 seconds (1 download), and it s WiFi speed is limited to about 20 Mbit/s in WiFi-n mode (2.4 GHz (or 5 GHz, does not matter), 40 MHz channel). It seems the 7270 has a really slow CPU.
Filed under: OpenWRT

5 August 2014

Riku Voipio: Testing qemu 2.1 arm64 support

Qemu 2.1 was just released a few days ago, and is now a available on Debian/unstable. Trying out an (virtual) arm64 machine is now just a few steps away for unstable users:

$ sudo apt-get install qemu-system-arm
$ wget https://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-arm64-disk1.img
$ wget https://cloud-images.ubuntu.com/trusty/current/unpacked/trusty-server-cloudimg-arm64-vmlinuz-generic
$ qemu-system-aarch64 -m 1024 -cpu cortex-a57 -nographic -machine virt -kernel trusty-server-cloudimg-arm64-vmlinuz-generic \
-append 'root=/dev/vda1 rw rootwait mem=1024M console=ttyAMA0,38400n8 init=/usr/lib/cloud-init/uncloud-init ds=nocloud ubuntu-pass=randomstring' \
-drive if=none,id=image,file=trusty-server-cloudimg-arm64-disk1.img \
-netdev user,id=user0 -device virtio-net-device,netdev=user0 -device virtio-blk-device,drive=image
[ 0.000000] Linux version 3.13.0-32-generic (buildd@beebe) (gcc version 4.8.2 (Ubuntu/Linaro 4.8.2-19ubuntu1) ) #57-Ubuntu SMP Tue Jul 15 03:52:14 UTC 2014 (Ubuntu 3.13.0-32.57-generic 3.13.11.4)
[ 0.000000] CPU: AArch64 Processor [411fd070] revision 0
...
-snip-
...
ubuntu@ubuntu:~$ cat /proc/cpuinfo
Processor : AArch64 Processor rev 0 (aarch64)
processor : 0
Features : fp asimd evtstrm
CPU implementer : 0x41
CPU architecture: AArch64
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 0

Hardware : linux,dummy-virt
ubuntu@ubuntu:~$
The "init=/usr/lib/cloud-init/uncloud-init ds=nocloud ubuntu-pass=randomstring" is ubuntu cloud stuff that will set the ubuntu user password to "randomstring" - don't use "randomstring" literally there, if you are connected to internets... For more detailed writeup of using qemu-system-aarch64, check the excellent writeup from Alex Bennee.

Hideki Yamane: one init system rules all...

At a linuxfoundation blog "A Haiku Poem Dedicated to Systemd" says...
List-units, start, stop
enable and disable
goodbye to init
But I prefer is
Systemd: All your process are belong to us
or

WE ARE THE SYSTEMD:
You will be assimilated - logging/dhcp/ntp... Resistance is futile



Andrew Pollock: [geek] Lifehacking with NFC, Tasker and HabitRPG

Oh man, I'm such a geek... Being a single parent has required a considerable amount of self-discipline on my behalf. I find I do best in an environment with routine to make sure stuff gets done. One of the things I did to try and make sure stuff got done was to form positive habits. To help and make this a bit more "fun", I started using HabitRPG, an online role-playing game based on habits. I've only ever used it in a half-arsed manner, and as a result, I die a lot. Often I'll do my dailies, but then forget to mark them off. Sometimes I just fall off the habit wagon altogether. And die. One of my dailies is to scoop my cat's litterboxes. For whatever reason, I find this a tad onerous, and had found myself falling out of the habit of doing it on a daily basis. My cat deserves better than this, so I wanted to get back on the habit wagon, but make it a bit more "fun" as well. I love NFC, I've been a big NFC weenie for a long time. It's a solution looking for a problem. I have a huge collection of NFC tags, and I've finally found a problem I could use them with. I wanted to make it more frictionless to mark a task as completed in HabitRPG. I didn't want to have to take out my phone, unlock it, open the HabitRPG app and check off the daily task. I just wanted to wave my phone at an NFC tag. Here's how I did it. The inimitable Paul Fenwick, who first inspired me to use HabitRPG, has a way more complicated set up to achieve something similar. The Site Reliability Engineer in me wanted the least number of moving parts and third-parties to have to get from my phone to HabitRPG. After some hunting around, I found this wiki page on integrating Tasker with HabitRPG's API, so based on that breadcrumb, I got hacking. I'd not used Tasker before. I was already using AutomateIt to do some other, reasonably dumb automation on my phone, though. Tasker is a little bit obtuse, and it took me a couple of days of casual fiddling with it to wrap my head around it's usage model. The fact that it can run arbitrary JavaScript, and that NFC Tools integrates with it is where the real awesome lies. So based off the wiki, I crafted a couple of bits of JavaScript, one to mark daily tasks as complete on HabitRPG, and one to query HabitRPG to see if they have been marked as complete. For the former, it was trivial, using NFC Tools, to write an NFC tag that then runs the Tasker task to call the HabitRPG API to mark a daily task as complete. That goal was now complete. The equally satisfying part was also having Tasker do a time-based conditional nag based on the state of the daily task in HabitRPG. So now if it gets to 4:45pm and I haven't scooped the litterboxes, my phone literally tells me I'd better go do it. I've also done the same thing with putting the dishwasher on. I've stuck an NFC tag on the dishwashing powder bottle lid, and I get a conditional reminder before bed time. It used to be an unconditional reminder with AutomateIt, and it was dumb, because I rarely forget to put the dishwasher on. Now, I can use HabitRPG to keep state on it instead. The hunk of JavaScript to update HabitRPG is very simple:
function mark_completed()   
  var http = new XMLHttpRequest();
  http.open("POST", http_post_url, false);
  http.setRequestHeader("x-api-user", global('HabitrpgUserid'));
  http.setRequestHeader("x-api-key", global('HabitrpgApiToken'));
  http.send();
  return(http.responseText); 
 
try   
  var result = mark_completed(); 
  catch(e)   
  var error = e.message; 
 
All of the interesting stuff is defined in variables preceding the JavaScript in the task definition. Here's a screenshot of the Task in Tasker that tells a thousand words: Screenshot of Tasker task that uses HabitRPG API to mark a task complete Similarly, to query HabitRPG, I'm using:
function query_task()   
  var http = new XMLHttpRequest();
  http.open("GET",http_get_url,false);
  http.setRequestHeader("x-api-user", global('HabitrpgUserid'));
  http.setRequestHeader("x-api-key", global('HabitrpgApiToken'));
  http.send();
  var p = JSON.parse(http.responseText);
  return p['completed'];
 
try   
  var task_completed = query_task(); 
  catch(e)   
  var error = e.message; 
 
It's only slightly trickier because you have to parse the JSON blob that comes back. Again, a screenshot. Screenshot of Tasker task that uses HabitRPG API to query task state Now maybe I'll stop dying so much in HabitRPG, my cat will have the clean toilet she deserves, and I'll stop getting reminded about putting the dishwasher on when I already have. Better living through automation, that's my motto.

Next.

Previous.