Search Results: "xam"

3 April 2013

Sylvain Le Gall: Sekred a password helper for puppet.

Puppet is a nice tool but it has a significant problem with passwords: This lead to complex situation and various workaround that more or less work: Most of these workarounds are complex, don't allow you to share the password you have set easily and most of the time are stored in another place than the target node. So I have decided to create my own solution: sekred (LGPL-2.1). The idea of sekred is to generate the password on the target node and made it available to the user that needs it. Then the user just have to ssh into the host and get the password. Pro: Cons: Let see some concrete examples. Setting mysql root password This is a very simple problem. When you first install mysql on Debian Squeeze, the root password is not set. That's bad. Let set it using sekred and puppet.
node "mysqlserver"  
  package  
    ["mysql-server",
     "mysql-client",
     "sekred"]:
      ensure => installed;
   
  service  
    "mysqld":
      name => "mysql",
      ensure => running,
      hasrestart => true,
      hasstatus => true;
   
  exec  
    "mysql-set-root-password":
      command => "mysqladmin -u root password $(sekred get root@mysql)",
      onlyif => "mysql -u root",  # Trigger only if password-less root account.
      require => [Service["mysqld"], Package["mysql-client", "sekred"]];
   
 
And to get the root password for mysql, just login into the node "mysqlserver":
$> sekred get root@mysql
Cie8ieza
Setting password for SSH-only user This example is quite typical of the broken fully automated scenario with passwords: - you setup a remote host only accessible through SSH - you create a user and set its SSH public key to authorize access - your user cannot access its account because SSH prevent password-less account login! In other word, you need to login into the node, set a password for the user and mail him back.... That defeats a little bit the "automation" provided by puppet. Here is what I do with sekred:
define user::template ()  
  user  
    $name:
      ensure => present,
      membership => minimum,
      shell => "/bin/bash",
      ....
   
  include "ssh_keys::$name"
  # Check password less account and set one, if required.
  $user_passwd="$(sekred get --uid $name $name@login)"
  exec  
    "user-set-default-password-$name":
      command => "echo $name:$user_passwd   chpasswd",
      onlyif => "test \"$(getent shadow $name   cut -f2 -d:)\" = \"!\"",
      require => [User[$name], Package["sekred"]];
   
 
So the command "test \"$(getent shadow $name cut -f2 -d:)\" = \"!\"" test for a password-less account. If this is the case, it creates a password using sekred get --uid $name $name@login and set it through chpasswd. Note that $user_passwd use a shell expansion that will be evaluated when running the command only, on the host. The --uid flag of sekred assign the ownership of the password to the given user id. So now the user (foo) can login into the node and retrieve its password using sekred get foo@login. Try it! Sekred was a very short project but I am pretty happy with it. It solves a long standing problem and helps to cover an extra mile of automation when setting up new nodes. The homepage is here and you can download it here. Feel free to send patches, bugs and feature requests (here, login required).

Petter Reinholdtsen: Isenkram 0.2 finally in the Debian archive

Today the Isenkram package finally made it into the archive, after lingering in NEW for many months. I uploaded it to the Debian experimental suite 2013-01-27, and today it was accepted into the archive. Isenkram is a system for suggesting to users what packages to install to work with a pluggable hardware device. The suggestion pop up when the device is plugged in. For example if a Lego Mindstorm NXT is inserted, it will suggest to install the program needed to program the NXT controller. Give it a go, and report bugs and suggestions to BTS. :)

Joey Hess: upstream git repositories

Daniel Pocock posted The multiple repository conundrum in Linux packaging. While a generally good and useful post, which upstream developers will find helpful to understand how Debian packages their software, it contains this statement:
If it is the first download, the maintainer creates a new git repository. If it has been packaged before, he clones the repository. The important point here is that this is not the upstream repository, it is an independent repository for Debian packaging.
The only thing important about that point is that it highlights an unnecessary disconnect between the Debian developer and upstream development. One which upstream will surely find annoying and should certainly not be bothered with. There is absolutely no technical reason to not use the upstream git repository as the basis for the git repository used in Debian packaging. I would never package software maintained in a git repository upstream and not do so. The details are as follows: So using the upstream git repository is valuable. And there is absolutely no technical value, and plenty of potential friction in maintaining a history-disconnected git repository for Debian packaging.

1 April 2013

Russ Allbery: Review: Echo

Review: Echo, by Jack McDevitt
Series: Alex Benedict #5
Publisher: Ace
Copyright: November 2010
Printing: November 2011
ISBN: 1-937007-00-6
Format: Mass market
Pages: 367
This is the fifth book in the ongoing Alex Benedict series, but reading the previous volumes is generally optional. While there are references to earlier events (and indeed I resorted to Wikipedia at one point to refresh my memory of the plot of the the first book in the series), they aren't critical to the plot. They mostly are background for the reactions of other characters to Benedict and his firm. Echo opens with the now-familiar prologue, giving us a fleeting and limited glimpse into the mystery that Benedict and Chase Kolpath will be pursuing for the rest of the book. This time, it features the explorer Somerest Tuttle, who has spent most of his life looking for intelligent alien life. (In the far future setting of these books, humanity has spread through a large region of the galaxy but has only encountered one other alien race.) In the twilight of his career and life, a friend who works as a tour guide for astronomical tours comes to him with a story about something she saw. What, we don't know, and won't for most of the book. But, whatever it is, Tuttle never goes public with it, which is directly contrary to what everyone would expect him to do with any news about alien life. Benedict and Kolpath get involved some years later, after Tuttle's death, when a stone slab with unknown markings that was part of his estate shows up on the future equivalent of Craigslist, free to the first person who will haul it away. Benedict finds that interesting enough to go pick up, but then is really hooked when someone else goes to considerable lengths to try to make the slab disappear before anyone can get a good look at it. This, slowly, leads into poking around Tuttle's life and connections, the star tour industry, and the possible origins of the slab. If you've read any of the previous books in this series, Echo is more of the same, and I mean that in a good way. I think McDevitt is at his best when writing this sort of cross between scientific puzzle, mystery, and non-military adventure, full of people who care deeply about knowing and understanding things and don't like to be thwarted in their efforts. There is McDevitt's trademark halting and red-herring-ridden investigation that succeeds in being engrossing and compelling, plenty of opportunities to guess what's really going on (a bit easier this book than in some others), and a bit of action that I thought was better-handled than the somewhat strained action scenes of The Devil's Eye. The book moves right along, despite multiple reversals in the progress of the investigation, and the final revelation does indeed justify the strange behavior of the various people Benedict encounters. As with McDevitt in general and this series in particular, there is one caveat: the astronomical details are often interesting, but the world building in general is not terribly believable. Despite supposedly taking place millennia into the future, the society is a straight transplant (projection would imply more change than exists) of middle-class American culture, with bonus air cars, space ships, and slightly better computers. Similarly, alien ecologies suffer from severe Star Trek problems: all the planets feel like copies of Earth with a few minor tweaks. This just goes with the territory, and if you're up to the fifth book of the series, you've probably gotten used to it, but the unrealistic lack of divergence is actually used as a plot point here. That strained my suspension of disbelief a bit. That said, this sort of extremely familiar social context does give the books an oddly calm, comfortable feeling. It's tempting to describe McDevitt as cozy science fiction. There's a bit of danger, a bit of suspense, but one is fairly certain that everything will work out in the end and the reader will leave the book knowing the solutions to most of the mysteries raised without having to work very hard at understanding the society or the puzzle. If that's what you're in the mood for (and I was when I read Echo), McDevitt delivers reliably. If you like the Alex Benedict series in general, recommended, as this is an excellent example of the type. Rating: 8 out of 10

31 March 2013

Nathan Handler: Debian Mobile

For one of my classes, the final project is to work on a personal project for four weeks. Since I have never done mobile programming before, I decided I would use this project as a chance to teach myself the basics of Android development. Paul Tagliamonte recently blogged about a Google Summer of Code project idea for a Debian Android application. It was this post that motivated me to create a Debian Android application for my final project. We are only in the middle of week one, but my plan is to have at least a basic BTS and PTS interface done by the end of the project. Then, depending on a few things, I hope to turn it into a Google Summer of Code project or simply continue developing it on my own. As I mentioned, I am only a few days into the project. One of the first hurdles I encountered was interacting with the SOAP interfaces available for the PTS and BTS. There are a few examples available on the wiki, but there are not any for Java. A Google search also failed to return any clear and simple results. As a result, I decided I would create a Python script that would accept requests from the Java Android application via a REST API and pass them on to the SOAP API before returning the result as JSON. Before I could finish writing the script, Paul mentioned that he had already written a similar program a few months ago. I immediately stopped working on my script and began preparing a patch for Paul's program to add support for some missing PTS SOAP functions as well as the entire BTS. After a few hours, I had the ability to query the PTS and BTS via a REST API from Java. While I normally prefer to use VIM for my programming, I decided to follow Google's recommendation and use Eclipse. I was able to follow their guide to get the SDK setup and a simple demo application running. One of the few hurdles I encountered was when I wanted to make web requests. Due to the ability for such requests to stall, you are required to perform them in a separate thread. This only becomes aparant when you try to test your application on an actual device; the emulator will happily run the process in the main thread. Regardless, I was eventually able to modify the demo application to allow me to make requests to my Python REST server and then parse and display the result. Here are some screenshots of the application running. Main Screen API Functions Information Entered Result If you have any feature suggestions, questions, or comments, feel free to send me an email (nhandler@debian.org). The source code for the Android application is currently not available, but I plan to put it up in a git repository sometime in the near future. The source code for the Python script that provides a REST API to access the PTS and BTS SOAP interface is available on GitHub.

30 March 2013

Joey Hess: Goodreads vs LibraryThing vs Free software

Four years ago I started using Goodreads to maintain the list of books I've read (which had lived in a flat text file for a decade+ before that). Now it's been aquired by Amazon. I doubt it will survive in its current form for more than 2 years. Anyway, while Goodreads has been a quite good way to find what my friends are reading, I've been increasingly annoyed by the quality of its recommendations, and its paucity of other features I need. It really doesn't seem to help me keep up with new and interesting fiction at all, unless my friends happen to read it. So I looked at LibraryThing. Actually, I seem to have looked at it several times before, since it had accounts named "joey", "joeyh", and "joeyhess" that were all mine. Which is what happens to me on sites that lack Openid or Browserid. Digging a little deeper this time, I am finding its recommendations much better than Goodreads' -- although it seems to sometimes recommend books I've already read. And it has some nice features like tracking series, so you can easily tell when you've read all the books in a series or not. The analytics overall seem quite impressive. The UI is cluttered and it seems to take 5 clicks to add and rate a single book. It supports half stars. Overall I get the feeling this was designed for a set of needs that doesn't quite match mine. For example, it seems it doesn't have a single database entry per book; instead each time I add a book, it seems to pull in data from primary sources (library of congress, Amazon cough) and treat this as a separate (but related) entry somehow. Weird. Perhaps this makes sense to say, librarians. I'm willing to adjust how I think about things if there's an underlying reason that can be grasped. There's a quite interesting thread on LibraryThing where the founder says:
Don't say we should open-source the code. That would be a nightmare! And I have limited confidence in APIs. LibraryThing has the book geeks, but not so much the computers geeks.
I assume that the nightmare is that there would be dozens of clones of the site, all balkanized, with no data transfer, no federation between them. Except, that's the current situation, as every Goodreads user who is now trying to use LibraryThing is discovering. Before I ever started using Goodreads, I made sure it met my minimum criteria for putting my data into a proprietary silo: That I could get the data back out. I can, and have. LibraryThing can import it. But the import process loses data! And it's majorly clunky. If I want to continue using Goodreads due to its better UI, and get the data into LibraryThing, for its better analytics, I have to do periodic dumps and loads of CSV files with manual fixups. This is why we have standards. This is why we're building federated social networks like status.net and the upcoming pump.io that can pass structured data between nodes transparently. It doesn't have to be a nightmare. It doesn't have to rely on proprietary APIs. We have the computer geeks. Thing is, sites like GoodReads and LibraryThing need domain-specific knowledge, and communities to curate data, and stuff like that. Things that work well in a smallish company. (LibraryThing even has a business model that makes sense, yearly payments to store more books in it.) With free software, it's much more appealing to sink the time we have into the most general-purpose solution we can. Why build a LibraryThing when we could build something that tracks not only books but movies and music? Why build that when we could build a generic federated network for structured social data? And that's great, as infrastructure, but if that infrastructure is only used to build a succession of proprietary data silos, what was the point? So, could some computer & book geeks please build a free software alternative to these things, focused on books, that federates using any of the fine APIs we have available? Bear in mind that there is already a nice start at a comprehensive collection of book data in the Open Library. I'd happily contribute to a crowd funded project doing this.

Evgeni Golov: Opera, standards and why I should have stayed in my cave

So you probably heard that I have that little new project of mine: QiFi the pure JavaScript WiFi QR Code Generator. It s been running pretty well and people even seem to like it. One of its (unannounced) features is a pretty clean stylesheet that is used for printing. When you print the result will be just the SSID and the QR code, so you can put that piece of paper everywhere you like. That works (I tested!) fine on Iceweasel/Firefox 10.0.12 and Chromium 25.0. Today I tried to do the same in Opera 12.14 and it failed terribly: the SSID was there, the QR code not. And here my journey begins First I suspected the CSS I used was fishy, so I kicked all the CSS involved and retried: still no QR code in the print-out. So maybe it s the QR code library I use that produces a weird canvas? Nope, the examples on http://diveintohtml5.info/canvas.html and http://devfiles.myopera.com/articles/649/example5.html don t print either. Uhm, let s Google for opera canvas print And oh boy I should not have done that. It seems it s a bug in Opera. And the proposed solution is to use canvas.toDataURL() to render the canvas as an image and load the image instead of the canvas. I almost went that way. But I felt that urge need to read the docs before. So I opened http://www.w3.org/html/wg/drafts/html/master/embedded-content-0.html#dom-canvas-todataurl and https://developer.mozilla.org/en-US/docs/DOM/HTMLCanvasElement and started puking:
When trying to use types other than image/png , authors can check if the image was really returned in the requested format by checking to see if the returned string starts with one of the exact strings data:image/png, or data:image/png; . If it does, the image is PNG, and thus the requested type was not supported. (The one exception to this is if the canvas has either no height or no width, in which case the result might simply be data:, .)
If the type requested is not image/png, and the returned value starts with data:image/png, then the requested type is not supported.
Really? I have to check the returned STRING to know if there was an error? Go home HTML5, you re drunk! Okay, okay. No canvas rendered to images then. Let s just render the QR code as a <table> instead of a <canvas> when the browser looks like Opera. There is nothing one could do wrong with tables, right? But let s test with the basic example first: Yes, this is 2013. Yes, this is Opera 12.14. Yes, the rendering of a fucking HTML table is wrong. Needles to say, Iceweasel and Chromium render the example just fine. I bet even a recent Internet Explorer would That said, there is no bugfixworkaround for Opera I want to implement. If you use Opera, I feel sorry for you. But that s all. Update: before someone cries ZOMG! BUG PLZ!!! , I filled this as DSK-383716 at Opera.

Hideki Yamane: using clang with pbuilder/cowbuilder


Today, Junich Uekawa has added some compiler support hook to pbuilder (thanks!), so you can test your package with clang easily.

How to use)

  1. install newest pbuilder
  2. enable hook script "D65various-complier-support" to your pbuilder hook directory
  3. specify "CHOOSE_COMPILER" variables and build


$ cp /usr/share/doc/pbuilder/examples/D65various-complier-support /your/hook/dir/
$ sudo CHOOSE_COMPILER=clang cowbuilder --build yourpackage_1.00-1.dsc


clang maybe little bit strict than gcc4.7, and its outputs looks good, have fun :)

29 March 2013

Richard Hartmann: Release Critical Bug report for Week 13

Hallowed fornicating k lusk tur, Chiroptera Sapiens Sapiens! Look at that corrected bug count... It seems week 16, or even week 15, will really be the week of the penguin! The UDD bugs interface currently knows about the following release critical bugs: How do we compare to the Squeeze release cycle?
Week Squeeze Wheezy Diff
43 284 (213+71) 468 (332+136) +184 (+119/+65)
44 261 (201+60) 408 (265+143) +147 (+64/+83)
45 261 (205+56) 425 (291+134) +164 (+86/+78)
46 271 (200+71) 401 (258+143) +130 (+58/+72)
47 283 (209+74) 366 (221+145) +83 (+12/+71)
48 256 (177+79) 378 (230+148) +122 (+53/+69)
49 256 (180+76) 360 (216+155) +104 (+36/+79)
50 204 (148+56) 339 (195+144) +135 (+47/+90)
51 178 (124+54) 323 (190+133) +145 (+66/+79)
52 115 (78+37) 289 (190+99) +174 (+112/+62)
1 93 (60+33) 287 (171+116) +194 (+111/+83)
2 82 (46+36) 271 (162+109) +189 (+116/+73)
3 25 (15+10) 249 (165+84) +224 (+150/+74)
4 14 (8+6) 244 (176+68) +230 (+168/+62)
5 2 (0+2) 224 (132+92) +222 (+132/+90)
6 release! 212 (129+83) +212 (+129/+83)
7 release+1 194 (128+66) +194 (+128/+66)
8 release+2 206 (144+62) +206 (+144/+62)
9 release+3 174 (105+69) +174 (+105/+69)
10 release+4 120 (72+48) +120 (+72/+48)
11 release+5 115 (74+41) +115 (+74/+41)
12 release+6 93 (47+46) +93 (+47/+46)
13 release+7 50 (24+26) +50 (+24/+26)
14 release+8
15 release+9
16 release+10
17 release+11
18 release+12
Graphical overview of bug stats thanks to azhag:

Olivier Berger: Managing Python code with UTF-8 (french chars) in org-mode + babel + minted for LaTeX export

The goal of this article is to illustrate how to manage Python code which includes comments in UTF-8 characters inside a latin-1 source org-mode for LaTeX export. Note that I ve pasted in wordpress the HTML generated by org-mode, so I hope it isn t broken too much. My typical use case is a french lecture on Python where the text is written in french, as well as some of the code comments and examples We ll use org-mode s babel module to include and manage the Python
examples. The goal is to write the source of the Python programs
directly in the same org source as the class book s text, and to extract them into a subdir (with the tangle feature), so that they can be shipped to the students to experiment with. The minted LaTeX environment is used, for babel, to make the Python syntax highlighting.

1.1 Issues
  • The source org-mode is in latin-1 so that it compiles with pdflatex
  • The examples source-code will be in UTF-8 so that Python 2.7 executes them well on a modern Linux desktop
  • Minted relies on pygmentize wich doesn t seem to handle UTF-8 so well

1.2 Solution
We ll show how to patch minted s use of pygmentize to take advantage
of its conversion capacity to convert the Python sources from UTF-8 to
latin-1 at document rendering time (LaTeX compilation). It s a hack, but works. The current document s source shows how this works

1.2.1 Customization
The org document should contain the following headers :
#+LANGUAGE:  fr
#+LaTeX_HEADER: \usepackage[latin1] inputenc 
#+LaTeX_HEADER: \usepackage[french] babel 
#+LaTeX_HEADER: \usepackage color \usepackage minted 
and the footers :
# Local Variables:
# coding: latin-1
# org-src-preserve-indentation: true
# tab-width: 4
# End:
Also note that pdflatex should use the -shell-escape option necessary for minted Now, the document should also include the following patch for minted :
#+LATEX_HEADER: \makeatletter
#+LATEX_HEADER: \renewcommand\minted@pygmentize[2][\jobname.pyg] 
#+LATEX_HEADER:   \def\minted@cmd pygmentize -l #2 -f latex -F tokenmerge
#+LATEX_HEADER:     \minted@opt gobble  \minted@opt texcl  \minted@opt mathescape 
#+LATEX_HEADER:     \minted@opt startinline  \minted@opt funcnamehighlighting 
#+LATEX_HEADER:     \minted@opt linenos  -P "verboptions=\minted@opt extra "
#+LATEX_HEADER:     -O encoding=UTF-8,outencoding=iso-8859-1 -o \jobname.out.pyg #1 
#+LATEX_HEADER:   \immediate\write18 \minted@cmd 
#+LATEX_HEADER:   % For debugging, uncomment:
#+LATEX_HEADER:   %\immediate\typeout \minted@cmd 
#+LATEX_HEADER:   \ifthenelse \equal \minted@opt@bgcolor 
#+LATEX_HEADER:     
#+LATEX_HEADER:     \begin minted@colorbg \minted@opt@bgcolor 
#+LATEX_HEADER:   \input \jobname.out.pyg 
#+LATEX_HEADER:   \ifthenelse \equal \minted@opt@bgcolor 
#+LATEX_HEADER:     
#+LATEX_HEADER:     \end minted@colorbg 
#+LATEX_HEADER:   \DeleteFile \jobname.out.pyg 
#+LATEX_HEADER: \makeatother
The important change is the addition of the -O encoding=UTF-8,outencoding=iso-8859-1 option of pygmentize, that will convert the UTF-8 source code to latin-1. Hopefully most of the characters will convert fine, for a document written in french.

2 Examples

2.1 Basic Python
This is a generic verbatim example, which doesn t use babel / minted.
$ python
Python 2.7.3 (default, Jan  2 2013, 16:53:07) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print "Hello, world!"
Hello, world!
>>>

2.2 Python source code for french speakers
Here, we first include the source code, so that it can be exported to
an examples/ subdir, with the following (note that the #+ BEGIN_src / #+ END_src should be changed to get rid of the space
character between + and BEGIN / END, and that the :shebang and following text should be on the BEGIN_SRC line I seem to have not quoted
these properly in this document):
#+name: helloworld
#+ BEGIN_src python :tangle examples/helloworld.py :noweb yes \\
:shebang #!/usr/bin/python :padline no :exports none
# -*- coding: utf-8 -*-
# Ceci est un exemple d'affichage d'une cha ne accentu e
name = raw_input("Quel est v tre nom ? ")
print "J'esp re que  a va bien aujourd'hui", name
#+ END_src
Editing it with C-c will open an UTF-8 buffer, so hopefully, this will be consistent with the utf-8 coding system declared on the second line. The regeneration of the examples/helloworld.py file is made with C-c C-v t (babel tangling ). Note that we don t add the #!/usr/bin/python first line in the source, as we want it to be added by the tangling process (which will also make the script executable). Once this is done, the script may be run :
$ python examples/helloworld.py 
Quel est v tre nom ? Fran ois
J'esp re que  a va bien aujourd'hui, cher Fran ois
Now, to add the colorized rendering in the lecture book, we do (here again, remove the spaces):
# +BEGIN_latex
\inputminted python examples/helloworld.py 
# +END_latex
And here it is, rendered by minted:
# -*- coding: utf-8 -*-
# Ceci est un exemple d'affichage d'une cha ne accentu e
name = raw_input("Quel est v tre nom ? ")
print "J'esp re que  a va bien aujourd'hui, cher", name
Here are links to : the source, the PDF result, and the generated Python program.

28 March 2013

Russ Allbery: wallet 1.0

Finally. The wallet is a system management tool for storing and retrieving secure data. I originally wrote it as part of our Kerberos v5 migration project to replace the old Kerberos v4 system we had for distributing srvtabs. It's still mostly used at Stanford for distributing keytabs, but it's extensible and supports storing arbitrary secure data. We also use it internally for storing database passwords, SSL private keys, and anything else that needs to be kept secure and retrieved later. It supports a rich ACL mechanism and namespace enforcement and is built on top of remctl for security and confidentiality. I did a bunch of development on it originally in 2006 and 2007, but then it solved our immediate problems and I didn't have much time to work on it. I used pre-1.0 versions because there was so much more on it that I wanted to do. But we've been running it in production for years, and while there's still tons more work that needs to be done, it really deserves a 1.0 version. Jon Robertson converted the database backend to DBIx::Class (which added PostreSQL support) and implemented schema upgrading, which were two of the major things that were keeping me from putting a 1.0 version on it. So it's time. Other new features since the previous release include a change to the default ACLs so that owners of objects can destroy them by default, a new ldap-attr ACL type that checks whether the caller has a particular attribute set in LDAP (which will let us use our entitlement system to control access), and support for storing and managing key rotation for WebAuth keyrings. There's also a new acl check command to determine whether an ACL exists and a new comment field and command to store an arbitrary comment about an object. Our local naming policy has also gotten much more complex, so I turned it into a module to provide an example to others of how to implement some reasonably detailed namespace and object autocreation controls. You can get the latest version from the wallet distribution page. I'm going to upgrade Debian packages to experimental or unstable (depending on the freeze status) as soon as I get a chance to write a bit more documentation and hopefully figure out debconf integration with the database setup support inside Debian for the server piece.

27 March 2013

Benjamin Mako Hill: The Institute for Cultural Diplomacy and Wikipedia

A month ago, Mark Donfried from the Institute for Cultural Diplomacy (ICD) an organization dedicated to promoting open dialogue sent me this letter threatening me with legal action because of contributions I ve made to Wikipedia. Yesterday, he sent me this followup threat. According to the letters, Donfried has threatened me with legal action because I participated in a discussion on Wikipedia that resulted in his organization s article being deleted. It is not anything I wrote in any Wikipedia article that made Donfried so upset although Donfried is also unhappy about at least one off-hand comment I made during the deletion discussion on a now-deleted Wikipedia process page. Donfried is unhappy that my actions, in small part, have resulted in his organization not having an article in Wikipedia. He is able to threaten me personally because unlike many people I edit Wikipedia using my real, full, name. Donfried s letter is the latest step in a saga that has been ongoing since last June. It has been a frustrating learning experience for me that has made me worried about Wikipedia, its processes, and its future. In Wikipedia, debates can be won by stamina. If you care more and argue longer, you will tend to get your way. The result, very often, is that individuals and organizations with a very strong interest in having Wikipedia say a particular thing tend to win out over other editors who just want the encyclopedia to be solid, neutral, and reliable. These less-committed editors simply have less at stake and their attention is more distributed. The ICD is a non-profit organization based in Berlin. According to its own website, a large part of the organization s activities are based around arranging conferences. Its goals peace, cultural interchange, human rights are admirable and close to my heart. Its advisors and affiliates are impressive. I had never heard of the ICD before their founder, Mark Donfried, emailed me in April 2012 asking me to give a keynote address at their conference on The 2012 International Symposium on Cultural Diplomacy & Human Rights. I replied, interested, but puzzled because my own research seems very far afield of both cultural diplomacy (which I had never heard of) and human rights. I replied saying:
What would you like me to talk about I ask because I don t consider myself an expert in (or even particularly knowledgeable about) cultural diplomacy. Did someone else refer you to me?
Donfried replied with a long message seemingly copy and pasted thanking me for considering attending and asking me for details of my talk. I replied again repeating text from my previous email and asking why he was interested in me. Donfried suggested a phone call to talk about details. But by this point, I had looked around the web for information about the ICD and had decided to decline the invitation. Among things I found was a blog post by my friend James Grimmelmann that suggests that, at least in his case, the ICD had a history of sending unsolicited email and an apparently inability to take folks off their email lists even after repeated requests. I also read the Wikipedia article about the ICD. Although the Wikipedia article was long and detailed, it sent off some internal Wikipedian-alarm-bells for me. The page read, to me, like an advertisement or something written by the organization being described; it simply did not read to me like an encyclopedia article written by a neutral third-party. I looked through the history of the article and found that the article had been created by a user called Icd_berlin who had made no other substantive edits to the encyclopedia. Upon further examination, I found that almost all other significant content contributions were from a series of anonymous editors with IP addresses associated with Berlin. I also found that a couple edits had removed criticism when it had been added to the article. The criticism was removed by an anonymous editor from Berlin. Criticisms on the article included links to a website called Inside the ICD which was a website that mostly consisted of comments by anonymous people claiming to be former interns of the ICD complaining about the working conditions at the organization. There were also many very positive descriptions of work at the ICD. A wide array of pseudonymous users on the site accused the negative commenters of being liars and detractors and the positive commenters of being ICD insiders. I also found that there had been evidence on Wikipedia also removed without discussion by an anonymous IP from Berlin of an effort launched by the youth wing of ver.di one of the largest trade unions in Germany to campaign for good internships at the ICD. Although details of the original campaign have been removed from ver.di s website, the campaigned ended after coming to an agreement with the ICD that made explicit a set of expectations and created an Intern Council. Although the article about ICD on Wikipedia had many citations, many were to the ICD s own website. Most of the rest were to articles that only tangentially mentioned the ICD. Many were about people with ICD connections but did not mention the ICD at all. As Wikipedia editor, I was worried that Wikipedia s policies on conflict of interest, advertising, neutrality, and notability were not being served by the article in its state. But as someone with no real experience or knowledge of the ICD, I wasn t sure what to do. I posted a request for help on Wikipedia asking for others to get involved and offer their opinions. It turns out, there were several editors who had tried to improve the article in the past and had been met by pro-ICD editors reverting their changes. Eventually, those editors lost patience or simply moved on to other topics. By raising the issue again, I kicked off a round of discussion about the article. At the termination of that discussion, the article was proposed for deletion under Wikipedia s Articles for Deletion policy. A new Wikipedia editor began working enthusiastically to keep the article by adding links and by arguing that the article should stay. The new user edited the Wikipedia article about me to accuse me of slander and defamation although they removed that claim after I tried to explain that I was only trying to help. I spent quite a bit of time trying to rewrite and improve the article during the deletion discussion and I went link by link through the many dozens of citations. During the deletion discussion, Mark Donfried contacted me over email and explained that his representatives had told him that I was working against the ICD in Wikipedia. He suggested that we meet. We had a tentative plan to meet in Berlin on an afternoon last July but, in the end, I was too busy trying to submit my thesis proposal and neither of us followed up to confirm a particular time within the time window we had set. I have still never met him. My feeling, toward the end of the deletion discussion on Wikipedia, was mostly exasperation. Somewhat reluctantly, I voted to delete the article saying:
Delete This AFD is a complete mess for all the reasons that the article itself is. Basically: there are a small number of people who seem to have a very strong interest in the Institute for Cultural Diplomacy having an article in Wikipedia and, from what I can tell, very little else. Hessin fahem, like all the major contributors to the page, joined Wikipedia in order to participate in this issue. This article has serious problems. I have posted a detailed list of my problems on the article talk page: primary sources, conflict of interest for nearly all substantive contributions and reading like an advert are the biggest issues. My efforts to list these problems were reverted without discussion by an anonymous editor from Berlin. I have seen no evidence that the Institute for Cultural Diplomacy satisfies WP:ORG but I agree that it is possible that it does. I strongly agree with Arxiloxos that articles should always be fixed, and not deleted, if they are fixable. But I also know that Wikipedia does not deserve this article, that I don t know to fix it, and that despite my efforts to address these issues (and I ll keep trying), the old patterns of editing have continued and the article is only getting worse. This ICD seems almost entirely based around a model that involves organizing conferences and then calling and emailing to recruit speakers and attendees. A large number of people will visit this Wikipedia article to find out more about the organization before deciding to pay for a conference or to join to do an internship. What Wikipedia shows to them reads like an advert, links almost exclusively to of pages on the organizations websites and seems very likely to have been written by the organization itself. We are doing an enormous disservice to our readers by keeping this page in its current form. If somebody wants to make a serious effort to improve the article, I will help and will happily reconsider my !vote. But after quite a bit of time trying to raise interest and to get this fixed, I m skeptical this can be addressed and my decision reflects this fact. mako 05:18, 12 June 2012 (UTC)
I concluded that although the organization might be notable according to Wikipedia s policies and although the Wikipedia article about it might be fixable, the pattern of editing gave me no faith that it could be fixed until something changed. When the article was deleted, things became quiet. Several months later a new article was created again, by an anonymous user with no other edit history. Although people tend to look closely at previously deleted new pages, this page was created under a different name: The Institute of Cultural Diplomacy and was not noticed. Deleted Wikipedia articles are only supposed to be recreated after they go through a process called deletion review. Because the article was recreated out of this process, I nominated it for what is called speedy deletion under a policy specifically dealing with recreated articles. It was deleted again. Once again, things were quiet. In January, it seems, the Inside the ICD website was threatened with a lawsuit by the ICD and the maintainers of the site took it down with the following message:
Apparently, the ICD is considering filing a lawsuit against this blog and it will now be taken down. We completely forgot about this blog. Let s hope no one is being sued. Farewell.
On February 25, the Wikipedia article on ICD was recreated once again out of process and by a user with almost no previous edit history. The next day, I received an email from Mark Donfried. In the message, Donfried said:
Please note that the ICD is completely in favor of fostering open dialogue and discussions, even critical ones, however some of your activities are raising serious questions about the motives behind your actions and some even seem to be motives of sabotage, since they resulted in ICD not having any Wikipedia page at all. We are deeply concerned regarding these actions of yours, which are causing us considerable damages. As the person who initiated these actions with Wikipedia and member of the board of Wikipedia [1], we would therefore request your answer regarding our questions below within the next 10 days (by March 6th). If we do not receive your response we will unfortunately have to consider taking further legal actions with these regards against you and other anonymous editors.
I responded to Donfried to say that I did not think it was prudent to speak with him while he was threatening me. Meanwhile, other Wikipedia editors nominated the ICD article for deletion once again and unanimously decided to delete it. And although I did not participate in the discussion, Donfried emailed again with more threats of legal action hours after the ICD article was deleted:
[A]s the case of the ICD and its presentation on the Wikipedia has seriously worsened in recent days, we see no alternative but to forward this case (including all relevant visible and anonymous contributors) to our legal representatives in both USA and Europe/Germany as well as to the authorities and other corresponded organizations in order to find a remedy to this case.
Donfried has made it very clear that his organization really wants a Wikipedia article and that they believe they are being damaged without one. But the fact that he wants one doesn t mean that Wikipedia s policies mean he should have one. Anonymous editors in Berlin and in unknown locations have made it clear that they really want a Wikipedia article about the ICD that does not include criticism. Not only do Wikipedia s policies and principles not guarantee them this, Wikipedia might be hurt as a project when this happens. The ICD claims to want to foster open dialogue and criticism. I think they sound like a pretty nice group working toward issues I care about personally. I wish them success. But there seems to be a disconnect between their goals and the actions of both their leader and proponents. Because I used my real name and was skeptical about the organization on discussion pages on Wikipedia, I was tracked down and threatened. Donfried insinuated that I was motivated to sabotage his organization and threatened legal action if I do not answer his questions. The timing of his first letter the day after the ICD page was recreated means that I was unwilling to act on my commitment to Wikipedia and its policies. I have no problem with the ICD and I deeply regret being dragged into this whole mess simply because I wanted to improve Wikipedia. That said, Donfried s threat has scared me off from attempts to improve the ICD articles. I suspect I will not edit ICD pages in Wikipedia in the future. The saddest part for me is that I recognize that what is in effect bullying is working. There are currently Wikipedia articles about the ICD in many languages. For several years, ICD has had an article on English Wikipedia. For almost all of that period, that article has consisted entirely of universally positive text, without criticism, and has been written almost entirely by anonymous editors who have only contributed to articles related to the ICD. In terms of the ICD and its article on Wikipedia, I still have hope. I encourage Donfried and his representatives to create accounts on Wikipedia with their full names just like I have. I encourage them to engage in open dialogue in public on the wiki. I encourage them go through deletion review, make any conflicts of interest they have unambiguously clear, and to work with demonstrably non-conflicted editors on Wikipedia to establish notability under Wikipedia s policies. The result still can be an awesome, neutral, article about their organization. I have offered both advice on how to do this and help in that process in the past. I have faith this can happen and I will be thrilled when it does. But the general case still worries me deeply. If I can be scared off by threats like these, anybody can. After all, I have friends at the Wikimedia Foundation, a position at Harvard Law School, and am close friends with many of the world s greatest lawyer-experts on both wikis and cyberlaw. And even I am intimidated into not improving the encyclopedia. I am concerned by what I believe is the more common case where those with skin in the game will fight harder and longer than a random Wikipedian. The fact that it s usually not me on the end of the threat gives me lots of reasons to worry about Wikipedia at a time when its importance and readership continues to grow as its editor-base remains stagnant.
[1] It s a minor mistake but worth pointing out that I am not on the board of Wikipedia ; I am on its advisory board which carries no power or responsbility within the organization. Sometimes, the foundation asks for my advice and I happily give it.

26 March 2013

Craig Small: procps-ng 3.3.7 Released

procps-ng version 3.3.7 was released today. It has some new and interesting features in the top program that Jim has been busy working on. There is a new filter feature which can exclude fields that match a value for example. The remainder of the changes are small bug fixes and getting the compile warnings count down with -Wall enabled. The library revision was updated but this did not involve an API or ABI change. procps-ng can be downloaded off the sourceforge page which has the current and previous releases stored there. Alternatively you can visit our gitorious page if git fetch is more your thing. Debian packages will be going into experimental until the freeze is over and we get things unblocked.

25 March 2013

Benjamin Mako Hill: MIT LaTeX Stationery

Color MIT LetterHead Example The MIT graphic identity website provides downloadable stationery templates for letterhead and envelopes. They provide both Microsoft Word and LaTeX templates. But although they provide both black and white and color templates for Word, they only provide the monochrome templates for LaTeX. When writing cover letters for the job market this year, I was not particularly interested in compromising on color and was completely unwilling to compromise on TeX. As a result, I ended up modifying each of the three templates to include color. In the process, I fixed a few bugs and documented one tricky issue. I ve published a git repository with my changes. It includes branches for each version of three of the old black and white templates as well as my my three new color templates. I hope others at MIT find it useful. I ve tried to keep the changes minimal. I ve emailed the folks at MIT Communication Production Services to see if they want to publish my modified versions. Until then, anyone interested can help themselves to the git repository. LaTeX user that you are, you probably prefer that anyway.

Joey Hess: difficulties in backing up live git repositories

But you can t just tar.gz up the bare repositories on the server and hope for the best. Maybe a given repository will be in a valid state; maybe it won t.
-- Jeff Mitchell in a followup to the recent KDE near git disaster This was a surprising statement to me. I seem to remember that one of (many) selling points for git talked about back in the day was that it avoided the problem that making a simple cp (or backup) of a repository could lead to an inconsistent result. A problem that subversion repositories had, and required annoying commands to work around. (svnadmin $something -- iirc the backend FSFS fixed or avoided most of this issue.) This prompted me to check how I handle it in ikiwiki-hosting. I must have anticipated a problem at some point, since ikisite backup takes care to lock the git repository in a way that prevents eg, incoming pushes while a backup is running. Probably, like the KDE developers, I was simply exercising reasonable caution. The following analysis has probably been written up before (train; limited network availability; can't check), but here are some scenarios to consider: I think that it's important git support naive backups of git repositories as well as possible, because that's probably how most backups of git repositories are made. We don't all have time to carefully tune our backup systems to do something special around our git repositories to ensure we get them in a consistent state like the KDE project did, and as their experience shows, even if we do it, we can easily introduce other, unanticipated problems. Can anyone else think of any other failure modes like these, or find holes in my slightly rushed analysis?
PS: git-annex is itself entirely crash-safe, to the best of my abilities, and also safe for naive backups. But inherits any problems with naive backups of git repositories.

22 March 2013

Richard Hartmann: Release Critical Bug report for Week 12

Let's see if we will have 82 open bugs next week ;) Current trend points towards week 16/17. The UDD bugs interface currently knows about the following release critical bugs: How do we compare to the Squeeze release cycle?
Week Squeeze Wheezy Diff
43 284 (213+71) 468 (332+136) +184 (+119/+65)
44 261 (201+60) 408 (265+143) +147 (+64/+83)
45 261 (205+56) 425 (291+134) +164 (+86/+78)
46 271 (200+71) 401 (258+143) +130 (+58/+72)
47 283 (209+74) 366 (221+145) +83 (+12/+71)
48 256 (177+79) 378 (230+148) +122 (+53/+69)
49 256 (180+76) 360 (216+155) +104 (+36/+79)
50 204 (148+56) 339 (195+144) +135 (+47/+90)
51 178 (124+54) 323 (190+133) +145 (+66/+79)
52 115 (78+37) 289 (190+99) +174 (+112/+62)
1 93 (60+33) 287 (171+116) +194 (+111/+83)
2 82 (46+36) 271 (162+109) +189 (+116/+73)
3 25 (15+10) 249 (165+84) +224 (+150/+74)
4 14 (8+6) 244 (176+68) +230 (+168/+62)
5 2 (0+2) 224 (132+92) +222 (+132/+90)
6 release! 212 (129+83) +212 (+129/+83)
7 release+1 194 (128+66) +194 (+128/+66)
8 release+2 206 (144+62) +206 (+144/+62)
9 release+3 174 (105+69) +174 (+105/+69)
10 release+4 120 (72+48) +120 (+72/+48)
11 release+5 115 (74+41) +115 (+74/+41)
12 release+6 93 (47+46) +93 (+47/+46)
13 release+7
14 release+8
15 release+9
16 release+10
17 release+11
18 release+12
Graphical overview of bug stats thanks to azhag:

20 March 2013

Masayuki Hatta: Recipe for Debian Haskell packaging

The following is what I usually do when I want to debianize a Haskell stuff from Hackage. This is a personal recipe, by no means official procedure or such. I may be wrong, badly at that.
  1. Check pkg-haskell repository whether your intended package already exists or not. Somebody might work on it already.
  2. Find the webpage of your intended package at HackageDB, then download Cabal source package . Untar it. The tarball filename should be changed to fit the Debian source package convention. For example, if the original name is something like foobar-1.0.tar.gz, then it should be haskell-foobar_1.0.orig.tar.gz.
  3. File ITP. reportbug is your friend.
  4. Run cabal-debian in untared dir:
     $ cabal-debian --debianize --quilt
    
    We do have the upstream tarball, so use quilt. See http://wiki.debian.org/UsingQuilt. It might give you warnings about changelog. debian/changelog doesn t exist yet, so ignore it.
  5. Now read http://wiki.debian.org/Haskell/CollabMaint/Processes and http://wiki.debian.org/Haskell/CollabMaint/PackageTemplate carefully. Then do things as they say. These two are short, but well-written instructions. Edit the generated debian/control appropriately. Do not forget to make your debian/copyright machine-friendly. See http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/.
  6. Test whether your package is flawless and really buildable. Do lintian. Use pbuilder.
     $ lintian haskell-foobar_1.0-1_amd64.changes
     # pbuilder --build haskell-foobar_1.0-1.dsc
    
  7. Read http://wiki.debian.org/Haskell/CollabMaint/DarcsBasic. Darcs is quite easy to use, and you don t have to be the master of Darcs to merely debianize things anyway. Basically, what I usually do is
     $ darcs record -a; darcs push --repo=debian
    
    Darcs is well-suited to Haskell-related development, but seems you can use Git, too.
  8. Change debian/changelog. UNRELEASED in the first line should be unstable or experimental . Do not forget to close ITP.
  9. Finally, do
     $ debuild -tc; debcommit -r; debrelease; darcs push --repo=debian
    
    That s all folks!

Raphael Geissert: A bashism a week: substrings

Sometimes obtaining a substring in a shell script is needed. The bashism of this week comes handy as it allows one to obtain a substring by indicating the offset and even the length of the substring. This is the $ varname:offset:length bashism, also known as substring expansion.

The portable "replacements" are simple if the offset (and the length) are static. For example, the following code would print the substring of "foo" consisting of only the last two characters:
var=foo
# Replace the bashism $ var:1 with:
echo $ var#?

The length can then be limited with additional pattern-matching removal expansions:
var="portable code"
# Replace the bashism $ var:3:5 with the following code

# Offset is 3, so we use three ? (interrogation) characters:
part=$ var#???

# Length is 5, so we use five ? characters:
echo $ part%$ part#?????

As it can be seen, it is not impossible to replace a substring expansion.

The portable code becomes slightly more complex if the offset and/or the length are dynamic. I leave that as an exercise for the readers.

Feel free to post your code as a comment (use the <pre> tags, please) or in another public way. My own response is already scheduled to be published next week at the same time as usual.

Note: substring expansions can also be replaced with a wide variety of external commands. This is a pure-POSIX shell scripting example.

19 March 2013

Gergely Nagy: Rebuttals, or the lack of thereof

It is customary during Debian Project Leader elections to write a rebuttal of the other candidate's platform. However, this is very hard to do this year, as there are a lot of overlaps between platforms, goals and ideas this time around. It is hard to rebute another platform, without also rebuting one's own. So instead of trying to highlight where I disagree (there's very little of that, if at all), I'd like to point out where I found each platform lacking.Mind you, these notes are primarily based off of the platforms, because I'm sadly still behind on my debian-vote@ reading list. Furthermore, due to the high level of similarity between platforms, I will not always single out one or the other, but mention what I found lacking, or troublesome in general instead. So unless stated otherwise, my thoughts below apply to both Moray's and Lucas' platforms.The thing I miss most from both platforms is the curious lack of plans about non-packaging project members (something which I strongly emphasize in my own platform). Similarly, even though Moray does mention encouraging more local events, I find those few paragraphs a little bit lacking; and Lucas does not even mention local teams in his platform, even though they are a very, very important part of the wider Debian community, and a terrific tool in our asset. They can further our reach by long miles! Considering their importance, little's said about either local teams or local events, even when the candidate does explicitly recognise their importance.On the other hand, Lucas writes about innovation within Debian (which I found lacking in Moray's platform) - but while he does bring up the topic, I sadly found that section of his platform vague, a mistake I made last year myself, and which I tried to avoid this time, by explicitly listing a few of my ideas on how I'd imagine inspiring & motivating people (which, in turn, would foster innovation within the project).From what I see - and I may be wrong here, and I am obviously biased - both platforms are reasonably explicit when it comes to easier topics, things that we know to work, and which we'd all continue doing in the very same spirit as our predecessor. But when it comes to experimenting, to new ideas, I found both platforms too vague. There are exceptions, of course, Moray's intern idea for example is sufficiently well explained, as far as it is possible within the limits of a platform. I found no similar thing in Lucas' platform, however, most of the ideas he explained were technical things, and those are the areas where we're fairly good at, where we already have a reasonably chance of improving - and thus, an easier topic. I was suprised by Lucas' platform, because even though he lists the need for experimentation important, the lack of truly experimental ideas was lacking - I found more in Moray's platform.All in all, I found the discussions on debian-vote@ much more useful so far, than the platforms, a lot of things were asked and answered there that the platforms did not touch.

17 March 2013

Andrew Cater: Kali Linux - Debian derivative for penetration testing/security

New Debian-based security distribution. The company behind BackTrack (Offensive Security) have just launched Kali Linux which will be tightly coupled to Debian and use Debian security updates, for example.

See also: http://www.kali.org/ and Raphael Hertzog's more informative post on Planet Debian.

Next.

Previous.