Search Results: "rul"

4 April 2013

Thomas Goirand: Git packaging workflow

Seeing what has been posted recently in planet.d.o, I would like to share as well my thoughts and work-flow, and tell that I do agree with Joey Hess on many of his arguments. Especially when he tells that Debian fetishises upstream tarballs. We re in 2013, at the age of Internet, and more and more upstream authors are using Git, and more and more they don t care about releasing tarballs. I ve seen some upstream authors who simply stopped doing so completely, as a Git tag is really enough. I also fully agree than disk space and network speed isn t much of a problem these days. When there are tags available, I use the following debian/gbp.conf:
[DEFAULT]
upstream-branch = master
debian-branch = debian
upstream-tag = %(version)s
compression = xz
[git-buildpackage]
export-dir = ../build-area/
On many of my packages, I now just use Git tags from upstream if they are available. To make it more easy, I now nearly always use the following piece of code in my debian/rules files:
DEBVERS         ?= $(shell dpkg-parsechangelog   sed -n -e 's/^Version: //p')
VERSION         ?= $(shell echo '$(DEBVERS)'   sed -e 's/^[[:digit:]]*://' -e 's/[-].*//')
DEBFLAVOR       ?= $(shell dpkg-parsechangelog   grep -E ^Distribution:   cut -d" " -f2)
DEBPKGNAME      ?= $(shell dpkg-parsechangelog   grep -E ^Source:   cut -d" " -f2)
DEBIAN_BRANCH   ?= $(shell cat debian/gbp.conf   grep debian-branch   cut -d'=' -f2   awk ' print $1 ')
GIT_TAG         ?= $(shell echo '$(VERSION)'   sed -e 's/~/_/')
get-upstream-sources:
        git remote add upstream git://git.example.org/proj/foo.git   true
        git fetch upstream
        if ! git checkout master ; then \
                echo "No upstream branch: checking out" ; \
                git checkout -b master upstream/master ; \
        fi
        git checkout $(DEBIAN_BRANCH)
make-orig-file:
        if [ ! -f ../$(DEBPKGNAME)_$(VERSION).orig.tar.xz ] ; then \
                git archive --prefix=$(DEBPKGNAME)-$(GIT_TAG)/ $(GIT_TAG)   xz >../$(DEBPKGNAME)_$(VERSION).orig.tar.xz ; \
        fi
        [ ! -e ../build-area ] && mkdir ../build-area   true
        [ ! -e ../build-area/$(DEBPKGNAME)_$(VERSION).orig.tar.xz ] && cp ../$(DEBPKGNAME)_$(VERSION).orig.tar.xz ../build-area   true
Packaging a new upstream VERSION now means that I only have to edit the debian/changelog, do ./debian/rules get-upstream-source so that I get new commits and tags, then git merge -X theirs VERSION to import the changes, then finally invoke ./debian/rules make-orig-file to create the orig.tar.xz. My debian branch is now ready for git-buildpackage. Note that the sed with the GIT_TAG thing is there because unfortunately, Git doesn t support the ~ char in tags, and that most of the time, upstream do not use _ in version numbers. Let s say upstream is releasing version 1.2.3rc1, then I simply do git tag 1.2.3_rc1 1.2.3rc1 so that I have a new tag which points to the same commit as 1.2.3rc1, but that can be used for the Debian 1.2.3~rc1-1 release and the make-orig-file. All this might looks overkill at first, but in fact it is really convenient and efficient. Also, even though there is a master branch above, it isn t needed to build the package. Git is smarter than this, so even if you haven t checked out upstream master branch from the upstream remote, make-orig-file and git-buildpackage will simply continue to work. Which is cool, because this means you can store a single branch on Alioth (which is what I do).

2 April 2013

Wouter Verhelst: Switching away from mutt

After having used mutt for about a decade, I decided it's time I move away from it, for various reasons that I won't go into here. I've been evaluating thunderbird^Wicedove now for a few weeks, but I can already see I won't keep using it forever, because I find too many issues with it:
  1. I can't tell the thing that I never ever ever want to write HTML-formatted emails. While there is an option "Compose messages in HTML format" in the "Account settings" dialog, that only seems to impact the behaviour of the client upon "new mail", not its reply behaviour, forcing me to go to Options->Format and switch it off.
  2. To make matters worse, the HTML editor and the plain-text editor are not the same dialog. That means it is not possible to switch on HTML formatting if it wasn't on originally (not a problem for me, but I can imagine it being a problem for others), but also that the fugly quoting style (vertical line next to the text) of the HTML editor is not removed when you switch from HTML to plain-text. Further on, it has issues when you want to remove white space, where it will remove entire quoting levels rather than just an empty line. This is horrible, horrible behaviour.
  3. While there is support for multiple email addresses connected to the same IMAP account, it does appear that there is no support for automatically choosing the right email address based on user-defined rules, such as "which mail folder are we currently in". Having to remember that I need to select the right mail address is tiresome and error-prone; e.g., when I'm in a Debian-related mail folder, I want my mail client to use my Debian mail address automatically. It appears the only way in which icedove can automatically select sender addresses is by checking the recipient list in the To: and Cc: headers for any of its configured mail addresses, and using that to decide what to put in the from: header.
Any one of the above would eventually be enough for me to grow tired of icedove and switch to something else; but the three combined mean that I'll be looking for something else fairly soon. Suggestions are, of course, welcome.

29 March 2013

Rémi Vanicat: New printer: Samsung SCX-3400

I've just buy a new printer/scanner, and as the documentation found on the web on how to setup the printer falsely instruct me to download some third-party driver, I will explain here how to set it up: By the way, Dear lazy web, I was wondering. In gimp, when I ask it to create a new image using Xsane, the menu give me the choice between the "Device Dialog" or my webcam. Is there any way to have my scanner there?

27 March 2013

Bits from Debian: Debian Project Leader elections 2013: interview with Moray Allan

We have asked Moray Allan, one of the three candidates for DPL elections 2013, to tell our readers about himself and his ideas for the Debian Project. You can also read the interviews to the other two candidates: Lucas Nussbaum and Gergely Nagy.
Please tell us a little about yourself. I'm Moray Allan, from Edinburgh in Scotland. I'm 32. After working in academic research for a few years, I'm now working freelance on a wide mixture of topics, with recent projects in Indonesia, Romania and Kuwait. When I'm not working, I'm likely to be found walking through a city or the countryside, or otherwise relaxing at home reading a novel in French or Spanish. What do you do in Debian and how did you started contributing? In recent years, most of my Debian time was taken up organising the annual Debian conferences. But I still have a load of packages, mostly connected to an upstream Linux-on-handheld-computers project I was working on before I joined Debian to create packages for it. Why did you decide to run as DPL? I've been involved in Debian for about 10 years now, including working for the last few years in DebConf in a way similar to how the DPL acts within overall Debian. Previously I'd ruled out running due to lack of time, but currently I'm in a more flexible work situation. It seems the right time to put myself forward, and see if the ideas in my platform interest project members. Three keywords to summarise your platform. Transparency, communication, openness. (Three ways I'd like us to think about teams in Debian.) What are the biggest challenges that you envision for Debian in the future? I think the biggest challenges are for free software in general.
End-users are moving to more closed hardware -- will our software be able to run on the phones and tablets people are shifting towards? At the same time, end-users and server users are moving to "the cloud", and often depending more heavily on non-free infrastructure outside their own control. What are, in your opinion, the areas of the project more in need of technical and/or social improvements? In my platform I give a few ideas about teams and delegations, coordination and mediation, and both internal and external communication, including more organised fundraising. These are areas where I think relatively simple changes can give big benefits. Why should people vote for you? I have proven leadership experience within Debian, as I've been working on coordination and mediation tasks for some years already. At the same time, I do regular packaging work, and work in other parts of Debian like the press and publicity teams, so I'm in touch with the experience of normal Debian contributors. People should vote for me if they support my platform, which is about coordination-level changes that I would have no mandate or authority to push through unless I am elected. Name three tools you couldn't stay without. APT, emacs, ssh. What keep you motivated to work in Debian? I've used Debian on all my computers for a long time, and by now working on the distribution myself feels a natural part of that.
Fortunately I'm constantly positively surprised by Debian and by the Debian community. Are there any other fields where you call yourself a geek, besides computers? Certainly history (such as the eastern Mediterranean region in late antiquity), languages (including dead ones) and music (especially Josquin to Monteverdi).

Benjamin Mako Hill: The Institute for Cultural Diplomacy and Wikipedia

A month ago, Mark Donfried from the Institute for Cultural Diplomacy (ICD) an organization dedicated to promoting open dialogue sent me this letter threatening me with legal action because of contributions I ve made to Wikipedia. Yesterday, he sent me this followup threat. According to the letters, Donfried has threatened me with legal action because I participated in a discussion on Wikipedia that resulted in his organization s article being deleted. It is not anything I wrote in any Wikipedia article that made Donfried so upset although Donfried is also unhappy about at least one off-hand comment I made during the deletion discussion on a now-deleted Wikipedia process page. Donfried is unhappy that my actions, in small part, have resulted in his organization not having an article in Wikipedia. He is able to threaten me personally because unlike many people I edit Wikipedia using my real, full, name. Donfried s letter is the latest step in a saga that has been ongoing since last June. It has been a frustrating learning experience for me that has made me worried about Wikipedia, its processes, and its future. In Wikipedia, debates can be won by stamina. If you care more and argue longer, you will tend to get your way. The result, very often, is that individuals and organizations with a very strong interest in having Wikipedia say a particular thing tend to win out over other editors who just want the encyclopedia to be solid, neutral, and reliable. These less-committed editors simply have less at stake and their attention is more distributed. The ICD is a non-profit organization based in Berlin. According to its own website, a large part of the organization s activities are based around arranging conferences. Its goals peace, cultural interchange, human rights are admirable and close to my heart. Its advisors and affiliates are impressive. I had never heard of the ICD before their founder, Mark Donfried, emailed me in April 2012 asking me to give a keynote address at their conference on The 2012 International Symposium on Cultural Diplomacy & Human Rights. I replied, interested, but puzzled because my own research seems very far afield of both cultural diplomacy (which I had never heard of) and human rights. I replied saying:
What would you like me to talk about I ask because I don t consider myself an expert in (or even particularly knowledgeable about) cultural diplomacy. Did someone else refer you to me?
Donfried replied with a long message seemingly copy and pasted thanking me for considering attending and asking me for details of my talk. I replied again repeating text from my previous email and asking why he was interested in me. Donfried suggested a phone call to talk about details. But by this point, I had looked around the web for information about the ICD and had decided to decline the invitation. Among things I found was a blog post by my friend James Grimmelmann that suggests that, at least in his case, the ICD had a history of sending unsolicited email and an apparently inability to take folks off their email lists even after repeated requests. I also read the Wikipedia article about the ICD. Although the Wikipedia article was long and detailed, it sent off some internal Wikipedian-alarm-bells for me. The page read, to me, like an advertisement or something written by the organization being described; it simply did not read to me like an encyclopedia article written by a neutral third-party. I looked through the history of the article and found that the article had been created by a user called Icd_berlin who had made no other substantive edits to the encyclopedia. Upon further examination, I found that almost all other significant content contributions were from a series of anonymous editors with IP addresses associated with Berlin. I also found that a couple edits had removed criticism when it had been added to the article. The criticism was removed by an anonymous editor from Berlin. Criticisms on the article included links to a website called Inside the ICD which was a website that mostly consisted of comments by anonymous people claiming to be former interns of the ICD complaining about the working conditions at the organization. There were also many very positive descriptions of work at the ICD. A wide array of pseudonymous users on the site accused the negative commenters of being liars and detractors and the positive commenters of being ICD insiders. I also found that there had been evidence on Wikipedia also removed without discussion by an anonymous IP from Berlin of an effort launched by the youth wing of ver.di one of the largest trade unions in Germany to campaign for good internships at the ICD. Although details of the original campaign have been removed from ver.di s website, the campaigned ended after coming to an agreement with the ICD that made explicit a set of expectations and created an Intern Council. Although the article about ICD on Wikipedia had many citations, many were to the ICD s own website. Most of the rest were to articles that only tangentially mentioned the ICD. Many were about people with ICD connections but did not mention the ICD at all. As Wikipedia editor, I was worried that Wikipedia s policies on conflict of interest, advertising, neutrality, and notability were not being served by the article in its state. But as someone with no real experience or knowledge of the ICD, I wasn t sure what to do. I posted a request for help on Wikipedia asking for others to get involved and offer their opinions. It turns out, there were several editors who had tried to improve the article in the past and had been met by pro-ICD editors reverting their changes. Eventually, those editors lost patience or simply moved on to other topics. By raising the issue again, I kicked off a round of discussion about the article. At the termination of that discussion, the article was proposed for deletion under Wikipedia s Articles for Deletion policy. A new Wikipedia editor began working enthusiastically to keep the article by adding links and by arguing that the article should stay. The new user edited the Wikipedia article about me to accuse me of slander and defamation although they removed that claim after I tried to explain that I was only trying to help. I spent quite a bit of time trying to rewrite and improve the article during the deletion discussion and I went link by link through the many dozens of citations. During the deletion discussion, Mark Donfried contacted me over email and explained that his representatives had told him that I was working against the ICD in Wikipedia. He suggested that we meet. We had a tentative plan to meet in Berlin on an afternoon last July but, in the end, I was too busy trying to submit my thesis proposal and neither of us followed up to confirm a particular time within the time window we had set. I have still never met him. My feeling, toward the end of the deletion discussion on Wikipedia, was mostly exasperation. Somewhat reluctantly, I voted to delete the article saying:
Delete This AFD is a complete mess for all the reasons that the article itself is. Basically: there are a small number of people who seem to have a very strong interest in the Institute for Cultural Diplomacy having an article in Wikipedia and, from what I can tell, very little else. Hessin fahem, like all the major contributors to the page, joined Wikipedia in order to participate in this issue. This article has serious problems. I have posted a detailed list of my problems on the article talk page: primary sources, conflict of interest for nearly all substantive contributions and reading like an advert are the biggest issues. My efforts to list these problems were reverted without discussion by an anonymous editor from Berlin. I have seen no evidence that the Institute for Cultural Diplomacy satisfies WP:ORG but I agree that it is possible that it does. I strongly agree with Arxiloxos that articles should always be fixed, and not deleted, if they are fixable. But I also know that Wikipedia does not deserve this article, that I don t know to fix it, and that despite my efforts to address these issues (and I ll keep trying), the old patterns of editing have continued and the article is only getting worse. This ICD seems almost entirely based around a model that involves organizing conferences and then calling and emailing to recruit speakers and attendees. A large number of people will visit this Wikipedia article to find out more about the organization before deciding to pay for a conference or to join to do an internship. What Wikipedia shows to them reads like an advert, links almost exclusively to of pages on the organizations websites and seems very likely to have been written by the organization itself. We are doing an enormous disservice to our readers by keeping this page in its current form. If somebody wants to make a serious effort to improve the article, I will help and will happily reconsider my !vote. But after quite a bit of time trying to raise interest and to get this fixed, I m skeptical this can be addressed and my decision reflects this fact. mako 05:18, 12 June 2012 (UTC)
I concluded that although the organization might be notable according to Wikipedia s policies and although the Wikipedia article about it might be fixable, the pattern of editing gave me no faith that it could be fixed until something changed. When the article was deleted, things became quiet. Several months later a new article was created again, by an anonymous user with no other edit history. Although people tend to look closely at previously deleted new pages, this page was created under a different name: The Institute of Cultural Diplomacy and was not noticed. Deleted Wikipedia articles are only supposed to be recreated after they go through a process called deletion review. Because the article was recreated out of this process, I nominated it for what is called speedy deletion under a policy specifically dealing with recreated articles. It was deleted again. Once again, things were quiet. In January, it seems, the Inside the ICD website was threatened with a lawsuit by the ICD and the maintainers of the site took it down with the following message:
Apparently, the ICD is considering filing a lawsuit against this blog and it will now be taken down. We completely forgot about this blog. Let s hope no one is being sued. Farewell.
On February 25, the Wikipedia article on ICD was recreated once again out of process and by a user with almost no previous edit history. The next day, I received an email from Mark Donfried. In the message, Donfried said:
Please note that the ICD is completely in favor of fostering open dialogue and discussions, even critical ones, however some of your activities are raising serious questions about the motives behind your actions and some even seem to be motives of sabotage, since they resulted in ICD not having any Wikipedia page at all. We are deeply concerned regarding these actions of yours, which are causing us considerable damages. As the person who initiated these actions with Wikipedia and member of the board of Wikipedia [1], we would therefore request your answer regarding our questions below within the next 10 days (by March 6th). If we do not receive your response we will unfortunately have to consider taking further legal actions with these regards against you and other anonymous editors.
I responded to Donfried to say that I did not think it was prudent to speak with him while he was threatening me. Meanwhile, other Wikipedia editors nominated the ICD article for deletion once again and unanimously decided to delete it. And although I did not participate in the discussion, Donfried emailed again with more threats of legal action hours after the ICD article was deleted:
[A]s the case of the ICD and its presentation on the Wikipedia has seriously worsened in recent days, we see no alternative but to forward this case (including all relevant visible and anonymous contributors) to our legal representatives in both USA and Europe/Germany as well as to the authorities and other corresponded organizations in order to find a remedy to this case.
Donfried has made it very clear that his organization really wants a Wikipedia article and that they believe they are being damaged without one. But the fact that he wants one doesn t mean that Wikipedia s policies mean he should have one. Anonymous editors in Berlin and in unknown locations have made it clear that they really want a Wikipedia article about the ICD that does not include criticism. Not only do Wikipedia s policies and principles not guarantee them this, Wikipedia might be hurt as a project when this happens. The ICD claims to want to foster open dialogue and criticism. I think they sound like a pretty nice group working toward issues I care about personally. I wish them success. But there seems to be a disconnect between their goals and the actions of both their leader and proponents. Because I used my real name and was skeptical about the organization on discussion pages on Wikipedia, I was tracked down and threatened. Donfried insinuated that I was motivated to sabotage his organization and threatened legal action if I do not answer his questions. The timing of his first letter the day after the ICD page was recreated means that I was unwilling to act on my commitment to Wikipedia and its policies. I have no problem with the ICD and I deeply regret being dragged into this whole mess simply because I wanted to improve Wikipedia. That said, Donfried s threat has scared me off from attempts to improve the ICD articles. I suspect I will not edit ICD pages in Wikipedia in the future. The saddest part for me is that I recognize that what is in effect bullying is working. There are currently Wikipedia articles about the ICD in many languages. For several years, ICD has had an article on English Wikipedia. For almost all of that period, that article has consisted entirely of universally positive text, without criticism, and has been written almost entirely by anonymous editors who have only contributed to articles related to the ICD. In terms of the ICD and its article on Wikipedia, I still have hope. I encourage Donfried and his representatives to create accounts on Wikipedia with their full names just like I have. I encourage them to engage in open dialogue in public on the wiki. I encourage them go through deletion review, make any conflicts of interest they have unambiguously clear, and to work with demonstrably non-conflicted editors on Wikipedia to establish notability under Wikipedia s policies. The result still can be an awesome, neutral, article about their organization. I have offered both advice on how to do this and help in that process in the past. I have faith this can happen and I will be thrilled when it does. But the general case still worries me deeply. If I can be scared off by threats like these, anybody can. After all, I have friends at the Wikimedia Foundation, a position at Harvard Law School, and am close friends with many of the world s greatest lawyer-experts on both wikis and cyberlaw. And even I am intimidated into not improving the encyclopedia. I am concerned by what I believe is the more common case where those with skin in the game will fight harder and longer than a random Wikipedian. The fact that it s usually not me on the end of the threat gives me lots of reasons to worry about Wikipedia at a time when its importance and readership continues to grow as its editor-base remains stagnant.
[1] It s a minor mistake but worth pointing out that I am not on the board of Wikipedia ; I am on its advisory board which carries no power or responsbility within the organization. Sometimes, the foundation asks for my advice and I happily give it.

24 March 2013

Petter Reinholdtsen: Typesetting a short story using docbook for PDF, HTML and EPUB

A few days ago, during a discussion in EFN about interesting books to read about copyright and the data retention directive, a suggestion to read the 1968 short story Kod mus by Tore ge Bringsv rd came up. The text was only available in old paper books, and thus not easily available for current and future generations. Some of the people participating in the discussion contacted the author, and reported back 2013-03-19 that the author was OK with releasing the short story using a Creative Commons license. The text was quickly scanned and OCR-ed, and we were ready to start on the editing and typesetting. As I already had some experience formatting text in my project to provide a Norwegian version of the Free Culture book by Lawrence Lessig, I chipped in and set up a DocBook processing framework to generate PDF, HTML and EPUB version of the short story. The tools to transform DocBook to different formats are already in my Linux distribution of choice, Debian, so all I had to do was to use the dblatex, dbtoepub and xmlto tools to do the conversion. After a few days, we decided to replace dblatex with xsltproc/fop (aka docbook-xsl), to get the copyright information to show up in the PDF and to get a nicer <variablelist> typesetting, but that is just a minor technical detail. There were a few challenges, of course. We want to typeset the short story to look like the original, and that require fairly good control over the layout. The original short story have three parts/scenes separated by a single horizontally centred star (*), and the paragraphs do not contain only flowing text, but dialogs and text that started on a new line in the middle of the paragraph. I initially solved the first challenge by using a paragraph with a single star in it, ie <para>*</para>, but it made sure a placeholder indicated where the scene shifted. This did not look too good without the centring. The next approach was to create a new preprocessor directive <?newscene?>, mapping to "<hr/>" for HTML and "<fo:block text-align="center"><fo:leader leader-pattern="rule" rule-thickness="0.5pt"/></fo:block>" for FO/PDF output (did not try to implement this in dblatex, as we had switched at this time). The HTML XSL file looked like this:
<?xml version='1.0'?> 
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version='1.0'>
  <xsl:template match="processing-instruction('newscene')">
    <hr/>
  </xsl:template>
</xsl:stylesheet> 
And the FO/PDF XSL file looked like this:
<?xml version='1.0'?> 
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version='1.0'>
  <xsl:template match="processing-instruction('newscene')">
    <fo:block text-align="center">
      <fo:leader leader-pattern="rule" rule-thickness="0.5pt"/>
    </fo:block>
  </xsl:template>
</xsl:stylesheet> 
Finally, I came across the <bridgehead> tag, which seem to be a good fit for the task at hand, and I replaced <?newscene?> with <bridgehead>*</bridgehead>. It isn't centred, but we can fix it with some XSL rule if the current visual layout isn't enough. I did not find a good DocBook compliant way to solve the linebreak/paragraph challenge, so I ended up creating a new processor directive <?linebreak?>, mapping to <br/> in HTML, and <fo:block/> in FO/PDF. I suspect there are better ways to do this, and welcome ideas and patches on github. The HTML XSL file now look like this:
<?xml version='1.0'?> 
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version='1.0'>
  <xsl:template match="processing-instruction('linebreak)">
    <br/>
  </xsl:template>
</xsl:stylesheet> 
And the FO/PDF XSL file looked like this:
<?xml version='1.0'?> 
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version='1.0'
  xmlns:fo="http://www.w3.org/1999/XSL/Format">
  <xsl:template match="processing-instruction('linebreak)">
    <fo:block/>
  </xsl:template>
</xsl:stylesheet> 
One unsolved challenge is our wish to expose different ISBN numbers per publication format, while keeping all of them in some conditional structure in the DocBook source. No idea how to do this, so we ended up listing all the ISBN numbers next to their format in the colophon page. If you want to check out the finished result, check out the source repository at github (future/new/official repository). We expect it to be ready and announced in a few days.

22 March 2013

Francesca Ciceri: One device to rule them all

demotivational poster Am I the only one who logically inferred this from the Ubuntu for Android's tagline "One device to rule them all"?

21 March 2013

Daniel Pocock: Migrating to Xen Cloud Platform (XCP Xen-API) on Debian wheezy

I recently decided to upgrade my server and host it in a new location (turns out that leaving the UK hosting was a smart move too, as new UK laws to rein in the press are more likely to punish and censor private web sites and bloggers) As part of the upgrade, starting off with Debian 7 (wheezy) seemed to make a lot of sense. I've been using Xen as a standard server virtualization platform since Debian 5 (lenny), and upgrading through Debian 6 (squeeze) without any major problems. I had been managing it with the traditional xm toolstack without any major concerns. Wheezy benefits from supporting Xen Cloud Platform with a standard set of packages, and it has been promoted heavily as a key technology in wheezy. XCP appealed to me for various reasons: The gotchas Reading through that list, it seems like there is no reason not to use XCP. In fact, there are major differences between the way xl/xm tools manage a host and the way XCP manages a host. While xl/xm offer the flexibility to manually configure networking and storage any way you like, XCP is more like a cookie-cutter virtualization system. It needs to be: without having low level control over networking and storage on the physical host, it could not offer things like hot-migration. Some people might get a shock, however, when moving to such a locked-down platform, as it takes away some of the things you would normally control manually on a Debian server:
  • XCP takes over networking. Your /etc/network/interfaces file must be almost empty - you can still set up firewall rules from a separate script, but don't plan on having any customised IP address configuration or routing.
  • XCP still has some bugs, such as requiring a quick config hack to use any subnet size other than /24, or another hack to make HVM networking function for any non-Linux VM. Hopefully these will be fixed before the official wheezy release.
  • XCP management tools don't support IPv6 - although this doesn't prevent using IPv6 in a VM. If you were planning to use dom0 as an IPv6 gateway for the domUs, it won't work: however, you can set up a domU bridged to the physical ethernet so it can act as an IPv6 gateway for the other VMs.
  • XCP takes over your storage and creates it's own PV, VG and LVs. This can inadvertently lead to the creation of a nested volume group if you are not aware of the exact way this works.
Making it work Despite these gotchas, I have managed to get it working, doing all the things that were possible on regular Xen + Debian squeeze. I would suggest that anybody attempting this read through all the bugs of severity important or serious and the updated README.Debian file (patch in bug 702337) to get off to a quick start without repeating some of the problems faced by early adopters like myself.

20 March 2013

DebConf team: DebConf13 matching fund (Posted by Didier Raboud)

As part of the DebConf13 fundraising effort, a generous sponsor, Brandorr Group, has proposed to start a matching fund in USD for DebConf13; in place through the end of April 30th. The rules are quite simple: For details, please see our Monetary support from individuals page, which also has a short url, for convenience: http://deb.li/dc13donate. Help welcome As always, the DebConf team is looking for volunteers. Some jobs need technical skills, but many DebConf tasks are about working to deadlines on non-technical issues (e.g. fundraising, budgeting, talk scheduling). You can see more information about some of the jobs to be done on the DebConf wiki. Please do think about getting involved and sharing your ideas with us, to help us make DebConf an even more useful event for Debian in the future. We look forward to welcoming you to Vaumarcus!

19 March 2013

Gergely Nagy: Rebuttals, or the lack of thereof

It is customary during Debian Project Leader elections to write a rebuttal of the other candidate's platform. However, this is very hard to do this year, as there are a lot of overlaps between platforms, goals and ideas this time around. It is hard to rebute another platform, without also rebuting one's own. So instead of trying to highlight where I disagree (there's very little of that, if at all), I'd like to point out where I found each platform lacking.Mind you, these notes are primarily based off of the platforms, because I'm sadly still behind on my debian-vote@ reading list. Furthermore, due to the high level of similarity between platforms, I will not always single out one or the other, but mention what I found lacking, or troublesome in general instead. So unless stated otherwise, my thoughts below apply to both Moray's and Lucas' platforms.The thing I miss most from both platforms is the curious lack of plans about non-packaging project members (something which I strongly emphasize in my own platform). Similarly, even though Moray does mention encouraging more local events, I find those few paragraphs a little bit lacking; and Lucas does not even mention local teams in his platform, even though they are a very, very important part of the wider Debian community, and a terrific tool in our asset. They can further our reach by long miles! Considering their importance, little's said about either local teams or local events, even when the candidate does explicitly recognise their importance.On the other hand, Lucas writes about innovation within Debian (which I found lacking in Moray's platform) - but while he does bring up the topic, I sadly found that section of his platform vague, a mistake I made last year myself, and which I tried to avoid this time, by explicitly listing a few of my ideas on how I'd imagine inspiring & motivating people (which, in turn, would foster innovation within the project).From what I see - and I may be wrong here, and I am obviously biased - both platforms are reasonably explicit when it comes to easier topics, things that we know to work, and which we'd all continue doing in the very same spirit as our predecessor. But when it comes to experimenting, to new ideas, I found both platforms too vague. There are exceptions, of course, Moray's intern idea for example is sufficiently well explained, as far as it is possible within the limits of a platform. I found no similar thing in Lucas' platform, however, most of the ideas he explained were technical things, and those are the areas where we're fairly good at, where we already have a reasonably chance of improving - and thus, an easier topic. I was suprised by Lucas' platform, because even though he lists the need for experimentation important, the lack of truly experimental ideas was lacking - I found more in Moray's platform.All in all, I found the discussions on debian-vote@ much more useful so far, than the platforms, a lot of things were asked and answered there that the platforms did not touch.

18 March 2013

Gerfried Fuchs: Backports Integrated Into The Main Archive

Dear users and supporters of the backports service! The Backports Team is pleased to announce the next important step on getting backports more integrated. People who are reading debian-infrastructure-announce will have seen that there was an archive maintenance last weekend: starting with wheezy-backports the packages will be accessible from the regular pool instead of a separate one, and all backports uploads will be processed through the regular upload queue (including those for squeeze-backports and squeeze-backports-sloppy). For Users What exactly does that mean for you? For users of wheezy, the sources.list entry will be different, a simple substitute of squeeze for wheezy won't work. The new format is:
deb http://ftp.debian.org/debian/ wheezy-backports main
So it is debian instead of debian-backports, and offered through the regular mirror network. Feel invited to check your regular mirror if it carries backports and pull from there. For Contributers What does it mean for contributing developers? Uploads for backports are no longer to be pushed to backports-master but to ftp.upload.debian.org, like any other regular package. Also, given that the packages are served from the same archive install there is no need to include the original tarball in the upload any longer because the archive knows it (Squeeze and beyond). Also, given that the upload goes to the same upload queue, there is only one keyring used anymore, so no more pain with expired or replaced keys. We though still keep the rule of adding your UID to an ACL list (this also includes DM additions). This is mostly only to give us the chance to remind you that uploads to backports are directly available for installation onto stable systems and you thus should take special care there. We carefully tried to take over the old ACLs, in case you can't upload anymore, please tell us so we can look into the issue. I've mentioned wheezy-backports (and squeeze-backports-sloppy) a few times here already, and you might wonder when it will be available. Technically, it is available from now on. Practically, while you could already upload to it, the set up of the buildd network is more painful than expected, so please allow the Buildd Team some days for setting them up. The upload rules for wheezy-backports are the same: packages that are in the next suite are accepted. Given that Jessie isn't created yet, we want you to think about whether the package you want to upload will go into Jessie final, and that you are taking a closer look once Jessie is created and the package entered there about the upgradeability. For the time until the suite is available, you can see this as relaxed upload rule. The same goes for squeeze-backports-sloppy: packages from two suites after Squeeze are acceptable, which turns it into the same relaxed rule as wheezy-backports above. Please also keep in mind that uploads to squeeze-backports-sloppy usually should be accompanied by uploads to wheezy-backports so people are able to upgrade from squeeze-backports-sloppy to wheezy with wheezy-backports. Thanks Finally, we want to thank the FTP-Master Team for their fine work on making this happen. The documentation on backports-master has been updated, and in case of any doubt or question, feel free to ask them on either the debian-backports mailinglist, or in case of sensitive topics ask us directly. Enjoy!
Rhonda for the Backports Team

/debian permanent link Comments: 1 Flattr this

16 March 2013

Stefano Zacchiroli: bits from the DPL for February 2013 and a half

Dear project members, here's another report of DPL activities, this time for a period longer than usual (February + 1st week of March), so that the next one will be at the very end of the current DPL term. Highlights Appointments DPL helpers Two more DPL helpers IRC meetings have happened, minutes and logs of both are available. Assets Events Past At the beginning of February, I've attended FOSDEM 2013, together with many other Debian people. I didn't have any specific talk this year, but it's been a chance to talk F2F about several ongoing issues (see logs), and help mediating in some conflicts. I've also accepted the invitation to participate in the GNOME Advisory Board meeting, together with Laurent Bigonville of our GNOME team. No report of that has been prepared as of yet (sorry about that), but we have both reported "live" to the rest of the team on IRC. Future Miscellaneous A couple of months ago I've mentioned that I had filed an application, as Debian representative, to participate in a working table to define software procurement rules for the Italian public administration. Good news: my application has been accepted, together with those of other well-known FOSS communities and organizations (e.g. KDE, FSFE). I'll keep you posted of how it goes. Let's go back to elect a new DPL and release Wheezy now,
Cheers.
PS the day-to-day activity logs for February and March 2013 are available at the usual place master:/srv/leader/news/bits-from-the-DPL.txt.20130 2,3

15 March 2013

Jon Dowland: list filtering

The classic procmail recipe for (reasonably) safe automatic mailing list filtering is
:0
* ^List-Id:[^<]+<\/[a-z0-9-.]+
$MAILDIR/MATCH/
This relies on mailing lists being configured such that the first part of the ID, up to the first dot, is a good identifier for the list. this is not always the case (*.lists.fedoraproject.org, I'm looking at you). Removing the dot from the expression above means you end up with unambiguous, albeit long, folder names (devel.lists.fedoraproject.org). If you access this mail over IMAP, you have another problem: the dots create a nested folder heirarchy that probably doesn't make any sense (devel lists fedoraproject org). It's possible to change the configuration of IMAP clients and servers to use a different delimiter, but this is not an oft travelled road, and I don't feel like finding the pot holes. Better to substitute the dot for something else. I'd recommend an ASCII character, Unicode seems to throw up a whole bunch of client problems. Here's a variation that solves the problem:
:0
* ^List-Id:[^<]+<\/[a-z0-9-.]+
 
  :0
  subst=  echo "$MATCH"   tr . :
  :0
  $MAILDIR/$subst/
 
At the cost of elegance, you can add rules to give friendlier names to lists from folks who have well-defined list-ids (integration left as an exercise to the reader):
:0
* ^List-Id:.*debian.*lists\.debian\.org
subst=  echo "$MATCH"   sed 's/\([A-Za-z0-9-]\+\)\..*/\1/'
Finally, there are some "lists" you probably don't want to file in their own individual folders. Inject these into the top-most rule:
* ! ^List-Id.*groups\.facebook\.com
Finally thanks to Brett for showing me some exim filter snippets that would let me implement the above in a (slightly) nicer looking way. Sadly I think I'm stuck with procmail for some other features I use, but it's been a while since I looked.

12 March 2013

Richard Hartmann: Gitify your life

These Open Source Days were kind of a mixed experience. The social part Things started off quite nicely; the speaker dinner was very pleasant and the people at my table interesting to talk with. The way back to the hotel and breakfast on Saturday were even more interesting; talking with Amelia Andersdotter, a member of the European Parliament for the Swedish Pirate Party, about anti-trust measures, privacy protection, lobbyists, and creating the right incentives for companies to actually protect user data was fun and engaging. My own claim to fame is that I told her that while Microsoft is willing to sign UEFI payloads for x86, they refuse to do so for ARM which she didn't know. From my understanding, their interpretation is that any anti-trust rulings apply to PCs only, and not general computing devices like tablets. It will be interesting to see how that plays out in the long run, especially once ARM processors start appearing in laptops and desktops as main CPUs where Microsoft clearly still has a near monopoly. It was fascinating to discuss all those issues with someone who's actively involved with legislation, mainly because her style of talking was markedly different from what I am used to. One point which she stressed repeatedly is that the most important skill within the context of political work in the EU is to be able to track down, read, and then reference all the content that's being produced in Brussels and Strasbourg. Maybe that tidbit is useful to people who want to try themselves at political activism. She nicely, and maybe unknowingly, emphasized that point by always referring to specific papers, directives, and studies after making any point. Statement of opinion, reference to supporting document. Statement of opinion, reference to supporting document. Statement of opinion, reference to supporting document. Rinse, repeat. It's no secret that I'm continuously disappointed by the political process in general and politicians in particular. Yet, I have to say that this encounter had a lasting and positive impression on me. Amelia has an incredible amount of energy and genuinely cares about the issues which too many other politicians ignore completely. It's good to know someone like this is on our side. If you're Swedish, please inform yourself about her and consider voting in her favour. Other than that, I met several other people, new and old. This year's OSD felt a lot more relaxed and there were more informal and spontaneous conversations going on in hallways and at tables. Even though I failed to have any intense discussion of Mercurial vs git this year, all those conversation were still really interesting. The "workshop" I already knew that the room assignment for my workshop on Saturday afternoon would be a last-minute thing. What I didn't expect was that the final confirmation by Copenhagen Business School would be given on Friday evening. Mixed with a very cumbersome process to update the slides of the CBS-owned and -operated information beamer in the lobby, this resulted in the official schedule showing my room as "Workshop" instead of "SP 214". The official schedule repeatedly stated that people had to pre-register for the workshop which was a left-over from last year where they actually had to plan the amount of hardware kits needed; the registration itself was was very well hidden behind a rather generic link you saw exactly once after purchasing your ticket. Add to this that my room was literally in the farthest corner of the building and on the top floor and things start to turn sour real quick. All other rooms were either close to the main entrance or on the ground floor. Helpers at the front desk had a lot of people asking where my room was and gave out directions, yet... ...the grand total of six people who arrived were all telling me how they couldn't find the room and almost gave up on finding it. As more people asked for directions than turned up at the workshop, this means that even with directions, the room was really hard to find. Myself, it took me two attempts to find the room after which I was led there by one of the helpers who wasn't completely sure we were going in the right direction, himself. Still, I managed to get everyone who had a laptop and access to a UNIX-based system set up with working mr and vcsh repositories they can work from as well as a demo git-annex repository to play with. Sadly, not everyone had a laptop, or access to UNIX, or the tools installed, or even an idea where to install them from, even though I had noted these "requirements" in the description. Next time I do this workshop, I will need to make sure to include information on what distributions have the relevant packages and how to find and install the software on Max OS X in the workshop description. Feedback from the very few people who actually managed to find me was very good and one of the attendants asked if I would be willing to hold the workshop again at another conference in Denmark. I am still awaiting details on this, but I would obviously enjoy having the chance to reach more people. At least I fared better than the guys who did the workshop on "Binary exploitation 101" in yet another room. Two guys held a workshop with a total attendance of... one... I know the organizers of OSD were really unhappy with how this turned out and I believe them when they say that something like this will never happen again. It's the second year that this team is running the conference, running conferences is an incredible amount of work, and things were smoother this year than last year. Iterate and improve, etc. The talk Saturday evening, Alex offered me the chance to hold a reparation talk on Sunday morning to compensate for the botched workshop. Said talk was scheduled for the first slot and not really announced (but the schedule on both www and the beamer was changed over night). Additionally, this was merely an updated and extended version of the talk which I already held at OSD 2012 so at least part of the visitors knew most of what I had to say, already. To be honest I didn't expect too much, but more than twenty people turned up which is a very decent turnout, all things considered. Same as with the workshop, feedback was extremely positive and I was told afterwards by Bryan that several people mentioned my talk as one of the highlights, or even the highlight, of OSD when he was talking to them. To make things even better this talk has, finally, been videotaped; I will update this blog with a link as soon as the video is online.

Russ Allbery: Review: Commitment Hour

Review: Commitment Hour, by James Alan Gardner
Series: League of Peoples #2
Publisher: Eos
Copyright: April 1998
ISBN: 0-380-79827-1
Format: Mass market
Pages: 343
This is the second book in Gardner's League of Peoples universe, following Expendable, but it is largely unrelated to that novel apart from taking place in the same universe. Expendable follows the portion of humanity that has left Earth and embraced the universe and the advanced technology of the League. Committment Hour is set on Earth, in one of the remaining low-tech settlements that have become a backwater of humanity. Earth is governed by the Spark Lords, who appear to be a sort of scientific artistocracy empowered by the League of Peoples (Gardner gets into this more in later books in the series), but the community in this book is largely left alone. The protagonist of Committment Hour is Fullin, a resident of Tober Cove. The Tober Cove community has a very unusual interaction with gender. Every year, all inhabitants under the age of twenty (starting at the age of two) are taken away by what are viewed locally as gods. When they return, they have the opposite gender. They live their childhoods alternating gender every year, including a mandatory pregnancy (created on one of those trips by the gods) and childbirth. Then, at the age of twenty, rather than being gender-swapped like the other children, they choose the gender that they'll live with for the rest of their life. Fullin is (at least at the start of this book) currently male, but as the book opens on the eve of the day of his final gender committment, he's still undecided on what gender he'll choose. A review of this sort of book immediately requires a significant caveat: this story is all about social constructions of gender and about the implications of one specific transgender scenario. As a cisgender man who fits comfortably within society's normal gender construction, I'm not going to do the best job of judging how well it deals with those issues. It was on the long list for the Tiptree Award, which is a fairly good sign that it does okay, and a quick Internet search didn't uncover any significant dislike, but it's quite possible that I would miss places where it fell short. That said, I found Gardner's presentation of gender fascinating, in large part because Committment Hour is told from deeply inside Fullin's perspective and Gardner treats the Tober Cove gender swapping mostly as the unremarked normal. Fullin and the other inhabitants are quite aware that gender doesn't work for them the way that it does for everyone else, but they consider their gender process entirely sensible and can't imagine how weird life would be without the chance to experience both genders and to have carried a child. And Gardner has several aces up his sleeve that let him, by the end of the book, position Tobin Cove's experience as a very specific and local experience that would not necessarily generalize to gender in society at large, which I thought was an ingenious way to avoid overreaching and excessive generalization. This is not a gender utopia; Fullin's world is rather patriarchal and strongly gender essentialist. The village was, in the past, ruled by a sort of religious fantatic who laid down very firm rules about the roles of men and women, and the village largely stuck to them, but since anyone can choose which gender they want to become, the gender essentialism is also undermined in an interesting way. It becomes more akin to a choice of careers rather than an innate quality, albeit with only two choices and with a permanent decision at the age of 20. Gardner complicates the situation further by introducing a neuter gender (at least that's what the characters call it; by physical description, hermaphrodite would be a more accurate description). People actually have three choices on their twentieth birthday, and although the Patriarch who ruled Tober Cove outlawed and banished all "neuts," it's still a choice. Near the start of Committment Hour one of those who chose the "neut" gender returns to Tober Cove in the company of a Spark Lord (thus making "it" the pronoun used by the natives untouchable and somewhat immune to the normal local prejudice). Committment Hour follows Fullin and his close friend and possible future wife or husband through their last day before their committment hour, a day complicated by the incessant curiosity of the observing Spark Lord and his companion. Gardner has some rather neat world building and some revelations in store; this is the sort of book where, by the end, the curtain has been fully pulled back and the audience gets to peer into the machinery. But most of the book is about the social and societal structure of Tober Cove, about how it feels to swap back and forth between genders (and how specific that is to Tober Cove's unique situation), and how this complicates interpersonal interactions. It's an immersive sort of book that puts the reader in Fullin's head and shows his (and her) comfort with the Tober Cove culture and discomfort with the life choices that he faces as part of committment. As with everything by him I've read, Gardner's tone in writing is unique and can take a bit of getting used to. His stories tend to have a light, almost flippant attitude with frequent touches of absurdist humor, which can give the erroneous impression that he isn't taking his story seriously. But there is depth beneath that tone. Fullin may treat some questions flippantly, but he gives them serious thought and works through the consequences. And I was delighted by how Fullin completely scuttled the Spark Lord's arrogant assumption of superiority around his scientific understanding of things Fullin treats as mythological. Committment Hour is different fare than the typical SF novel, with a lot of focus on an unusual domesticity and small village politics. The technology and the world building are mostly saved for the climax, although are quite satisfying when they finally come. It's primarily a thought-provoking treatment of gender that avoids overreaching by grounding the story in a specific situation. I found it a bit slow in places, and occasionally wanted the characters to stop being stupid and stop dithering, but I ended up enjoying it rather more than I expected. Mildly recommended. Rating: 7 out of 10

11 March 2013

Gunnar Wolf: So you want to be a leader

So we are at the beginning of this year's Debian Project Leader elections. And yes, after Stefano's long and (IMO) very successful DPL term, I feel as my voting machinery is somewhat stuck; it will not be so easy to get it back up to speed. Anyway, I have glanced over the three platforms, but only actually read 1.5 from the three DPL platforms. I know that whoever succeeds, I will be quite happy with the results. This year there are three runners for the post. I have worked in several teams with two of them, and would love to know better the third. In the same order as presented in the vote:
Gergely Nagy
I have not yet worked with Gergely, but enjoy reading him. The closest I have been to working with him was sketching a packaging tutorial during DebConf11, in Bosnia. Sadly, I was quite busy, and he picked up the complete workload And correspondingly, got the credits. I can say that Gergely has a very important quality, the ability to put in seductively easy words the most complex processes. So, yes, being the Debian Leader post a public-facing one, I am certain he has one of the important qualities.
Moray Allan
We have worked together organizing DebConf for many years, first loosely as orga-team members, and starting two years ago (and together with Holger Levsen) as formal delegates. I think our team is quite well balanced, and Moray plays an important role. Holger and I are sometimes anxious to take measures, measures that IMO would have proven disastrous more than once. Moray is often the voice of reason. Given that another one of the DPL's roles is to mediate in social conflicts and keep Debian working smoothly (or something close to it ;-) ), that is also a very important trait of a DPL, and I'm also sure Moray would shine as a good leader.
Lucas Nussbaum
I have long been part of the pkg-ruby-extras team (although I am way less active than what I used to, where Lucas often dazzles us with his intense streaks of activity. Among this group of three, I see Lucas firstly as the most technically oriented, the biggest implementer. Also, as the proactive bug-finder and team-herder. And yes, Lucas is maybe the most enthusiastic about the (always) important Making Debian Sexy point. So, if elected, I'm sure this facet will also make him shine
So, it's not that I'm trying to bribe our next DPL with sweet nice words about how interesting a person or how good a friend he is, but am trying to look at the election process as something different. It seems for me that we are going to choose which Debian do we want to pursue for this starting period. Now, for our soon-to-be-ex-DPL Stefano: As many will surely tell you (or already have): You rock. I truly enjoyed your DPL term, and there is much we should adopt and learn from your personality and leadership. And, although it has waned over the past few years, many people tend to publish their (stated?) vote during the campaigning period. I (think I) have never done so, and this time I will surely not do so. Choosing a DPL involves personal feelings, sympathies, and many non-objective things. And although I know nobody will feel hurt if I don't put them in the first place, I prefer not to expose such issues. I can only assure you that this year, "None of the above" will sink to the bottom of my ballot.

Daniel Pocock: Firewalling IPv6

With my new ISP providing native IPv6 transit, enabling IPv6 in Debian is pretty trivial, just update /etc/networking/interfaces:
iface eth0 inet6 static
	pre-up modprobe ipv6
	address 2001:67c:1388::2
	netmask 64
	gateway 2001:67c:1388::1
if you have a Linux router, enable packet forwarding:
echo 1 > /proc/sys/net/ipv6/conf/all/forwarding
and add AAAA records to DNS:
danielpocock.com.        IN      AAAA    2001:67c:1388:1000::9
Yes, that really is enough to get up and running. Apache was already configured to listen on IPv6 and just needed a restart. However, it is not quite secure: the configuration above routes all traffic on all ports. If somebody has the time to port scan my 64 billion trillion possible IPv6 addresses (every IPv6 customer is allocated a /48 as standard), they might end up connecting to some service that should not be public. ip6tables The ip6tables tool for IPv6 does just about everything that iptables does for IPv4. It is tempting to just rewrite an existing firewall script, adding an extra rule for the IPv6 version of every host. For example, my web server is 195.8.117.9, so I copy the 9 off the end and assign the IPv6 address 2001:67c:1388:1000::9 to the same server and add the ip6tables firewall rule:
iptables --append FORWARD -p tcp -d 195.8.117.9 --dport http -j ACCEPT
ip6tables --append FORWARD -p tcp -d 2001:67c:1388:1000::9 --dport http -j ACCEPT
Notice that my IPv6 addressing simply involves re-using the same last byte as the IPv4 equivalent? This is a very easy way to start. However, it is not quite so simple. IPv6 requires a couple of extra rules, in the simplest form:
# Allow traffic for link-local addresses:
ip6tables -A INPUT -s fe80::/10 -j ACCEPT
# Normally, link-local packets should NOT be forwarded and don't need an
# entry in the FORWARD rule.
# However, when bridging in Linux (e.g. in Xen or OpenWRT), the FORWARD rule is
# needed:
ip6tables -A FORWARD -s fe80::/10 -j ACCEPT
# Allow all ICMPv6 traffic (optional for IPv4, mandatory for IPv6):
ip6tables -A INPUT -p icmpv6 -j ACCEPT
ip6tables -A FORWARD -p icmpv6 -j ACCEPT
This works, but it turns out that it is less than ideal. Beyond that, the firewall can be significantly enhanced by rate-limiting the ICMPv6 packets and various other mechanisms as demonstrated with a practical netfilter example in Appendix B of RFC 4890 The question that arises, how to enable wide use of IPv6 without every sysadmin having to familiarise themselves with all these details of filtering ICMPv6 packets? Debian firewall packages My next step will be to look over the existing firewall packages in Debian to see which of them may answer that question. It would be really useful to hear feedback from anybody else who has already undertaken such an evaluation. If none of them are suitable, and if permitted by the licensing terms of RFC 4890, I may well wrap their sample code into a Debian package that can be used as a foundation for people building their own custom firewall scripts.

10 March 2013

Hideki Yamane: great IOPS device and if we'd have it...?


I know some folks work at Fusion IO, maybe you know, one of the leader of PCIe flash storage system. And they say their products are "great", since their device put amazing IOPS as they press released (9,000,000 IOPS, and Wikipedia says "SAS HDDs put 200 IOPS". If you have a SSD in your laptop/desktop, you'd love it. It puts almost 10,000 IOPS or so). Such PCIe SSD can bring "Moon shot", change the rules for managing CPU, Memory, Disk and Network for servers, IMHO.


Then, I'm dreaming - if we Debian would have such MAGICAL device, what is the most efficient way to improve/accelerate/Kaizen our project? To use mailing list server? BTS? UDD? CI?

Dear lazyweb, please let me know what is the bottleneck of our development, and great IOPS capacity will fix it.

(and please don't say about things based on human resource, since PCIe SSD devices cannot improve such things, you know ;)

8 March 2013

Sylvain Le Gall: Always test your HD first

I just received a WD 500GB Blue, to replace the hard drive of my wife's computer. First thing after unpacking: start to test. And I was right to do it... there are bad blocks on it. WD 500GB DOA The story: I tend to do proactive replacement of hard drive, to avoid losing too much data or be in hurry when the current one fails. There are two ways to monitor a hard drive: Logcheck is pretty straightforward to install. It scans your log every two hours and send you a report of what is happening. It is not a very precise tool and you have to tune it a little bit to just send you what is relevant (install extra rules to ignore what you know is not of interest). Whenever you start to see log entry like that:
Jan 24 18:16:08 foo kernel: [ 1965.343980] ata5.00: exception Emask 0x50 SAct 0x39 SErr 0x800 action 0x6 frozen
Jan 24 18:16:08 foo kernel: [ 1965.343991] ata5.00: irq_stat 0x08000000, interface fatal error
Jan 24 18:16:08 foo kernel: [ 1965.344001] ata5: SError:   HostInt  
Jan 24 18:16:08 foo kernel: [ 1965.344036] ata5.00: failed command: READ FPDMA QUEUED
Jan 24 18:16:08 foo kernel: [ 1965.344055] ata5.00: cmd 60/08:00:18:31:44/00:00:1a:00:00/40 tag 0 ncq 4096 in
Jan 24 18:16:08 foo kernel: [ 1965.344059]          res 40/00:2c:e6:c2:fd/00:00:26:00:00/40 Emask 0x50 (ATA bus error)
Jan 24 18:16:08 foo kernel: [ 1965.344071] ata5.00: status:   DRDY  
Jan 24 18:16:08 foo kernel: [ 1965.344081] ata5.00: failed command: WRITE FPDMA QUEUED
It is a good time to think about changing your hardrive -- but it is maybe too late. Smartmontools (aka smartd) is dedicated tool to monitor hard drives and do a good job. I think it is not installed by default in Debian, but it should be. It scans your hard drive for SMART capabilities and monitor the health of the HD using internal tools. In the case of bad blocks, you will start to see entry like that:
Feb 17 14:05:17 bar smartd[1268]: Device: /dev/sda [SAT], 1 Currently unreadable (pending) sectors
Feb 17 14:35:17 bar smartd[1268]: Device: /dev/sda [SAT], 1 Currently unreadable (pending) sectors
Obviously when you see this log, it is also a good time to change your hard drive. In the case of my wife's HD, I just got data from logcheck. It means that the error is not that important (transient failure, something is wrong but the HD can cope with it). But I still decided to get a new one for my wife. Whenever, I receive a new drive, the first thing I do is to check it for errors. You can do that using the program badblocks in write mode. It takes ages to test (count up to 1 day for 1TB on USB), but at the end you know that you have a good candidate -- where it is worth install your data. You just have to follow this procedure
  1. dmesg grep sd
  2. try to find what drive is the one you want to test in the output of 1.
  3. cfdisk /dev/sdX, sdX being the drive you want to test
  4. check that what you see in cfdisk is what you expect to test: right name and capacity for the drive
  5. sudo badblocks -wvs /dev/sdX
  6. run sudo tail -f /var/log/syslog in parallel just in case
  7. wait
If some errors appears in /var/log/syslog, you know something bad is happening. Whenever you have a single failing block, don't think it is ok. It is NOT ok for a HD to start its life with failing blocks. In this case, repack the drive and send it for replacement ASAP. In the case of my hard drive: smartd mail:
The following warning/error was logged by the smartd daemon:
Device: /dev/sda [SAT], 15 Currently unreadable (pending) sectors
syslog entries:
Mar  2 20:52:57 foo kernel: [ 8317.419715] ata4.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x0
Mar  2 20:52:57 foo kernel: [ 8317.419724] ata4.00: irq_stat 0x40000008
Mar  2 20:52:57 foo kernel: [ 8317.419732] ata4.00: failed command: READ FPDMA QUEUED
Mar  2 20:52:57 foo kernel: [ 8317.419747] ata4.00: cmd 60/80:00:00:af:08/00:00:28:00:00/40 tag 0 ncq 65536 in
Mar  2 20:52:57 foo kernel: [ 8317.419750]          res 41/40:00:65:af:08/00:00:28:00:00/40 Emask 0x409 (media error) <F>
Mar  2 20:52:57 foo kernel: [ 8317.419758] ata4.00: status:   DRDY ERR  
Mar  2 20:52:57 foo kernel: [ 8317.419764] ata4.00: error:   UNC  
Mar  2 20:52:57 foo kernel: [ 8317.423959] ata4.00: configured for UDMA/133
I am not blaming any particular brand (like Western Digital), all computer parts I have ever bought had to follow the same procedure and it is a known fact that computer parts as a non-zero percentage of chance to be DOA (dead on arrival) or after a few weeks. But as a consumer you should be aware of that and take action to avoid spending 10h configuring your computer to see it failing after a week... The waste of time to test is a win on the long term.

4 March 2013

Bastian Blank: Using SECCOMP to filter sync operations

Linux includes a syscall filter since a long time. It was restricted to a pre-defined set of syscalls. Since some versions Linux got a more generic filter. Linux can use a BPF-filter to define actions for syscalls. This allows a fine granular specification on which syscalls to act. Also it supports different outcomes assigned to the filter. This filter can be used to filter out sync operations. Debian already got a tool to do this called eatmydata. It is pretty limited as it uses a shared library to drop the library functions. It needs to be available at all times, or it will not do anything. I wrote a small tool that asks the kernel to filter out sync operations for all children. It sets a filter with all currently supported sync-like operations and makes them return success. However it can't filter the O_SYNC-flag from the open-syscall, so it just makes it return an error. It executes the command given on the command-line after that. This is just a proof of concept, but lets see.
/*
 * Copyright (C) 2013 Bastian Blank <waldi@debian.org>
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 *
 * 1. Redistributions of source code must retain the above copyright notice, this
 *    list of conditions and the following disclaimer.
 * 2. Redistributions in binary form must reproduce the above copyright notice,
 *    this list of conditions and the following disclaimer in the documentation
 *    and/or other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
 * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */
#define _GNU_SOURCE 1
#include <errno.h>
#include <fcntl.h>
#include <seccomp.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#define filter_rule_add(action, syscall, count, ...) \
  if (seccomp_rule_add(filter, action, syscall, count, ##__VA_ARGS__)) abort();
static int filter_init(void)
 
  scmp_filter_ctx filter;
  if (!(filter = seccomp_init(SCMP_ACT_ALLOW))) abort();
  if (seccomp_attr_set(filter, SCMP_FLTATR_CTL_NNP, 1)) abort();
  filter_rule_add(SCMP_ACT_ERRNO(EINVAL), SCMP_SYS(open), 1, SCMP_A1(SCMP_CMP_MASKED_EQ, O_SYNC, O_SYNC));
  filter_rule_add(SCMP_ACT_ERRNO(0), SCMP_SYS(fsync), 0);
  filter_rule_add(SCMP_ACT_ERRNO(0), SCMP_SYS(fdatasync), 0);
  filter_rule_add(SCMP_ACT_ERRNO(0), SCMP_SYS(msync), 0);
  filter_rule_add(SCMP_ACT_ERRNO(0), SCMP_SYS(sync), 0);
  filter_rule_add(SCMP_ACT_ERRNO(0), SCMP_SYS(syncfs), 0);
  filter_rule_add(SCMP_ACT_ERRNO(0), SCMP_SYS(sync_file_range), 0);
  return seccomp_load(filter);
 
int main(__attribute__((unused)) int argc, char *argv[])
 
  if (argc <= 1)
   
    fprintf(stderr, "usage: %s COMMAND [ARG]...\n", argv[0]);
    return 2;
   
  if (filter_init())
   
    fprintf(stderr, "%s: can't initialize seccomp filter\n", argv[0]);
    return 1;
   
  execvp(argv[1], &argv[1]);
  if (errno == ENOENT)
   
    fprintf(stderr, "%s: command not found: %s\n", argv[0], argv[1]);
    return 127;
   
  fprintf(stderr, "%s: failed to execute: %s: %s\n", argv[0], argv[1], strerror(errno));
  return 1;
 

Next.

Previous.