url_for is normally used to avoid hard coding of URL in Flask based webapps. I've been using it in SILPA port written using Flask by. But this was written long back and written as POC to show for Santhosh author of SILPA app which even managed to go into production :-). Recently I started organizing the code and started using flask.Blueprint which was previously written using flask.views.MethodView class and here I started facing problem with old templates. I'm just gonna explain what is the difference when using url_for with MethodView and Blueprint. Lets have some sample code for MethodView first, this code is from Flask documentation. Here we have a view which is derived from MethodView which has logic on how to handle the requests. We use add_url_rule to register rule to handle /users/ url end point and we pass the class as view function which is done by as_view class method and we can refer this method in our Jinja2 templates using name users. So a statement like in our templates will be converted to /users/ URL when page is rendered to client by Flask. Now when I replaced the MethodView in favor of Blueprint I started getting up werkzeug.routing.BuildError thrown on my face and I had no clue why!. Yeah I know I'm bad at reading documentation but even after reading documentation I was still thinking that should return me a proper URL and I was wondering why its failing. Finally after re-reading documentation for url_for it was becoming clear to me, url_for definition looks like below Here endpoint is actually a function which is supposed to be serving the URL and **values is the arguments for this function. The URL in question should be defined in python code using decorator. In my case following is the new function serving the web pages for SILPA. You can ignore function code but just note the decorators, here I'm registering the function serve_pages with page as argument for URL patterns / and /<page>, _BASE_URL here is mount point of application it can be just / or /mountpoint depending on that URL registered changes. Now I need to modify code for all url_for in my template to look like below The . in front of function is for referring current Blueprint, in my case the Flask will consider it as frontend.serve_pages as function name and generates appropriate URL at run time. So my fault was misunderstanding endpoint argument as URL endpoint but where as its actually function name supposed to serve the page. But when using MethodViews I can simply convert class to a function with my preferred name just like UserAPI.as_view('/') so url_for('/') just works.

Sustainable visual identity design is a design that recycles as many elements as possible with every new piece of communication. Not only is a brand strengthened every time the logo and any other identity forming visual element is used, a lot of work is also saved as well. Every logo variant is a weakening of the brand, not a strengthening. Of course design improvements can be made - always, all year 'round - preferably not right up to the next conference. I consider it a waste of time to create a sublogo for a minidebconf. The Debian logo is perfectly suitable as logo for a Debian conference. In the particular case of the Barcelona MiniDebConf 2014, I think the Debian Women logo is the proper logo to use, though, since it is a Debian Women project, contradictionary as it may sound. The same principle of recycling typography and grids. I'd like as much identity matter - and work - to be reused, as possible. There is no need to reinvent the wheel every time. A visual theme created with illustration and colour schemes is enough as identity indicator for smaller events like MiniDebConfs. In my opinion.

Logo and logotype The Debian Women logo. Event name set in the font Procipio, familiar to the Debian logo. The name is long and cannot be scaled down much. A version with the name set in two lines can be used in narrow space, like on a folder.

Theme The speech bubble is a part of the visual theme. Orange is a color I connect with Spain, sun and oranges. I suggest it used with red like the Debian logo red.

Poster The speech bubbles could be used in a varied form. I've stated a few important statements here. I'd like a shared list of statements. There must be other info on the poster as well. The white space at the bottom is for sponsor logos. The font I use in the speech bubbles in this draft is an Apple font called Flood. I'd like to find a free font with similar qualities.

Banner The statement in the speech bubble could easily be exchanged.

T-shirt There are many possibilities. I like the idea of printing just a speech bubble in the front and give people the opportunity to write their personal statements with textile pens. Nametags could be badges to be filled out on the spot with a pen.

I've been asked to make another design proposal for the Minidebconf. People have been brainstorming. I could have asked them to do that months ago. I didn't, because I was working with my own ideas. We need to know who and what we are aiming at with the graphic communication, as well as with the conference. Who is the design for? What should they do? Why? Who is the conference for? Who should be interested? Is the design for people in Barcelona only? Is it for women only? Is it for current developers? Or is it for local politicians? Male or female? Students? - only? Does the communication have other goals than getting people at the venue? Should the design reflect the organizers - or rather not? Does the geography have to play a role in the visual imagery? These are questions that should always be asked. I made the mistake to answer a lot of the questions myself, based on my second hand knowledge and guesses. A common mistake, I've made before and will probably make again. I'm used to working on my own, that's my poor excuse. To my mind, the gathering, the meeting in the flesh and the communication and sharing of knowledge, is far more important than local landmarks. I don't feel that strongly for a location oriented identity design. La Sagrada Familia doesn't have anything to do with a gathering for free software development, does it. How? Oh, that was the minidebconf in Barcelona
Oh, that was the minidebconf arranged by Debian Women
Oh, that was the minidebconf that made a point out of being inclusive
Oh, that was the minidebconf that The identity design might have more to do with support to the local group than with any of the other aspects of the conference. Interesting. This table of colors shows different ways a blue and a purple could go. I put the green colors there to have more options covered for the discussion today. You can see how the different hues interacts with the red of the logo. The colors can be referred to like this: B3, C6, A4 etc. Click on the image for full size version. Here is an idea based on all the purple colors. Fun, uplifting, feels like many voices. Too many colors for T-shirt print, maybe: Here is some ideas of how to implement a minimalistic solution that I recommend for minidebconfs: My suggestion is that the banner (and everything else displaying the event) has the logo of the organizers placed on it. The font used for headlines etc is semantically of another nature than the logo, though it could be the same. I prefer neutral fonts, that can be reused by the next minidebconf, and I suggest a sans serif font. A condensed font makes huge display easier to fit. Don't be confused by displayfonts, I aim to show you that type alone speaks loudly, and if this typography should be recycled, we should not choose one of them, but go with the "neutral" sans serif. I need suggestions of free and good sans serif fonts.

Since the MiniDebConf Jonas and I have been travelling in Spain, France and finally staying in Belgium for a week, getting some work done. It's been harder than imagined to work during travel. I haven't exercised either, and regained at least three of four kilos I spent much time and effort getting rid in the year preceding. I thrive in my home and find it hard to keep my own time and focus when I am deprived of my own space. It was challenging to give a talk, "Why aren't more designers using Debian or working for Debian", my first public talk. I've been working to recapture my points in writing, to make a stronger statement, but I seem to blur my own views with conflicting ones, and I'm loosing momentum every day. One of my reasons for speaking up was to do it even though I'm not at trained speaker and have "nothing" to contribute but my opinions from the angle of a user that happens to be a designer. Not claiming to be a superior designer, but one that would like to contribute if it was easier to figure out how. And since the community wants to encourage designers to contribute to the Debian project, I figured it to be a good idea to talk about how this has been challenging to me as a dedicated user and completely out of the question for any other designer I know - or knew before the minidebconf. No reseach, no scientific proofs, just my wiew from my "dumb user" and designer's perspective. I saw one single attendant rolling his eyes during my talk. I didn't care at that time, but I've given that look more consideration than the people approaching me after the talk, saying thank you for voicing their opinions and thoughts. I think that's absolutely astonishing and at the same time it's just typically me. It makes me angry, first with myself for not speaking to this man's perception of things, then with myself for not just letting go of that image. I'm really glad that so many seemed to listen with curiosity and interest. What if one more - or half of the auditorium - had rolled their eyes? I don't like to feel that vulnerable. The truth is, though, that I'm really not. I gave the talk against my fear of failure and public humiliation and I'm convinced that my thoughts and actions matter, just as anybody's does, if we dare to say what's on our minds and to take action. I believe it's in anybody's power to "make a difference" and even "change the world" - at least in a small way. I guess that's one of the underlying reasons to be a designer in the first place. That is quite a strong position to take. I've created the wikipage http://wiki.debian.org/Design - well knowing that design is a word with many meanings. Everything is design. Since the talk I've been in doubt about that page. About the project, my aim with it, what to do about it, how to move on with just a tiny babystep, and I realise that I'm simply afraid to be disturbing someone's peace, making people angry or roll their eyes at my fumbling attempts to figure out in public what can be done to make a thriving community of designers collaborating with coders to make better, more usable and attractive software in the free, wide world. I'm starting a design process, not presenting a perfect, finished solution. Now, having put these thoughts into words, perhaps, my mind will be somewhat appeaced and let me move on with my intended tasks of cultivating that acclaimed space in the Debian information jungle into a friendly and welcoming place with info that makes it easier to be a contributing designer in Debian.

Yves Rossy is the Jetman, he flys with a wing and four jet engines strapped to his body, he gave an interesting TED talk about flying along with some exciting videos [1]. Larry Brilliant gave an informative and inspiring TED talk about stopping pandemics [2]. I thought that Smallpox was the last disease to be eradicated but I was wrong. Michael Shermer gave an interesting TED talk about pattern recognition and self deception [3]. It s a pity that the kissing prank shown at the end only pranked women, they should be less sexist and prank men too. Raffaello D Andrea gave an interesting TED presentation about Athletic quadcopters [4]. It s very impressive and has the potential for several new human/machine sports. Lisa D wrote an insightful article about Prejudice Spillover discussing the way that people who aren t in minority groups only seem to care about injustice when a member of the majority is targetted by mistake [5]. Ron Garret wrote an insightful post about the Divine Right of Billionaires which debunks some stupid arguments by a billionaire [6]. Ron says that it s often instructive to examine incorrect arguments, especially when those arguments are advanced by smart people and demonstrates it in this post. Lisa D wrote an interesting post about her problems with financial aid bureaucracy [7]. She intended the post to be a personal one about her situation, but I think it illustrates problems with the various aid programs. If aid was available to her with less bureaucracy then she would be doing paid work, completing her studies, and heading towards post-graduate studies. Mark Shuttleworth wrote an insightful article about ACPI, security, and device tree [8]. It s the first time I ve seen a good argument for device tree. TED presented an interesting video-conference interview with Edward Snowden [9]. It s unusually long by TED standards but definitely worth watching. Tom Meagher (who s wife was raped and murdered two years ago) wrote an insightful article about rape culture [10]. Key Lay (the Victorian Chief Commissioner of Police) wrote a good article encouraging men to act to stop violence against women [11]. It s particularly noteworthy when a senior police officer speaks out about this given the difficulties women have had in reporting such crimes to police. Emily Baker wrote an insightful article about the lack of support for soldiers who survive war [12]. A lot of attention and money is spent remembering the soldiers who died in the field but little on those who live suffer afterwards, more soldiers die from suicide than enemy fire. Daniel Pocock wrote an informative article about the failings of SMS authentication for online banking [13]. While he has good points I think he s a little extreme. Stopping the least competent attackers is still a significant benefit as most potential attackers aren t that competent. Jess Zimmerman wrote an interesting article for Time about the Not All Men argument that is a current trend in derailing discussions about the treatment of women [14]. The Belle Jar has an insightful article Why Won t You Educate Me About Feminism about some ways that men pretend to care about the treatment of women [15]. Jon Evans wrote an article for Tech Crunch about the Honywell Bubble Count measure of diversity in people you follow on social media [16]. Currently on Twitter I follow 57 accounts of which 15 are companies and organisations, so I follow 42 people. I follow 13 women 31%, for a visible minority group other than my own it s 2/42 or 5%, for people who live in other countries I think it s 8/42 (although it s difficult to determine where some people live) which is 19%. So my Honywell number is 55. The Top Stocks forum has an interesting post by a Coal Seam Gas (CSG) worker [17]. It seems that CSG is even worse than I thought. Ashe Dryden wrote an informative post for Model View Culture about the backlash that members of minority groups (primarily women) receive when they speak out [18].

• [1] http://www.ted.com/talks/yves_rossy_fly_with_the_jetman
• [2] http://www.ted.com/talks/larry_brilliant_wants_to_stop_pandemics
• [3] http://www.ted.com/talks/michael_shermer_the_pattern_behind_self_deception
• [4] http://tinyurl.com/pwa7hwy
• [5] http://chaoticidealism.livejournal.com/131742.html
• [6] http://blog.rongarret.info/2014/02/the-divine-right-of-billionaires_22.html
• [7] http://chaoticidealism.livejournal.com/132730.html
• [8] http://www.markshuttleworth.com/archives/1332
• [9] http://www.ted.com/talks/edward_snowden_here_s_how_we_take_back_the_internet
• [10] http://www.abc.net.au/news/2014-04-18/meagher-the-danger-of-the-monster-myth/5399108
• [11] http://www.vicpolicenews.com.au/blogs/93-oursay/1302-ccp-ken-lay-on-family-violence.html
• [12] http://www.examiner.com.au/story/2248712/dont-forget-about-those-war-veterans-who-remain/
• [13] http://danielpocock.com/sms-logins-an-illusion-of-security
• [14] http://time.com/79357/not-all-men-a-brief-history-of-every-dudes-favorite-argument/
• [15] http://bellejar.ca/2014/02/22/why-wont-you-educate-me-about-feminism/
• [16] http://techcrunch.com/2014/04/26/whats-your-honeywell-bubble-count/
• [17] http://www.topstocks.com.au/stock_discussion_forum.php?action=show_thread&threadid=1106338
• [18] http://modelviewculture.com/pieces/dissent-unheard-of

Last week, in the spirit of always getting better, I asked Twitter how to practice saying "um" less while speaking. Here's what I learned. Most people suggested to practice intentionally inserting pauses instead of saying "um". Various strategies include voice acting classes, having a friend aid with a buzzer (presumably to help you notice), and to record practice sessions as well as the real deal. Nagle suggested that meditation helped him, which makes sense to me as I've experienced meditation helping me be more mindful in many different areas of my life, including mundane things like noticing that I've touched something that's probably not that clean and remembering not to touch my face until I wash them. Wilfully directing mindfulness toward noticing how I speak is something I'm really stoked about, though it's easy to get swept away in a default "performance mode" in front of a crowd. Hendrick pointed out that Cognitive Behavioural Therapy is the principle behind changing speech behaviour and is a great starting point for more research. This is going to be really useful next time I'm practicing for a talk! Thanks @zmagg, @hendricklee, @bcrypt, @ebroder, @mscain, and @nagle5000 for playing. (original Twitter thread)

Background I ve previously written about the claim that people use Autism as an excuse for bad behavior [1]. In summary it doesn t and such claims instead lead to people not being assessed for Autism. I ve also previously written about empathy and Autism in the context of discussions about conference sexual harassment [2]. The main point is that anyone who s going to blame empathy disorders for the widespread mistreatment of women in society and divert the subject from the actions of average men to men in minority groups isn t demonstrating empathy. Discussions of the actions of average men are so often derailed to cover Autism that the Geek Feminism Wiki has a page about the issue of blaming Autism [3]. The Latest Issue Last year Shanley Kane wrote an informative article for Medium titled What Can Men Do about the treatment of women in the IT industry [4]. It s a good article, I recommend reading it. As an aside @shanley s twitter feed is worth reading [5]. In response to Shanley s article Jeff Atwood wrote an article of the same title this year which covered lots of other things [6]. He writes about Autism but doesn t seem to realise that officially Asperger Syndrome is now Autism according to DSM-V (they decided that separate diagnosis of Autism, Asperger Syndrome, and PDD-NOS were too difficult and merged them). Asperger Syndrome is now a term that refers to historic issues (IE research that was published before DSM-V) and slang use. Gender and the Autism Spectrum Jeff claims that autism skews heavily towards males at a 4:1 ratio and cites the Epidemiology of Autism Wikipedia page as a reference. Firstly that page isn t a great reference, I fixed one major error (which was obviously wrong to anyone who knows anything about Autism and also contradicted the cited reference) in the first section while writing this post. The Wikipedia page cites a PDF about the Epidemiology of Autism that claims the 4.3:1 ratio of boys to girls [7]. However that PDF is a summary of other articles and the one which originated the 4.3:1 claim is behind a paywall. One thing that is worth noting in the PDF is that the section containing the 4.3:1 claim also references claims about correlations between race and Autism and studies contradicting such claims it notes the possibility of ascertainment bias . I think that anyone who reads that section should immediately consider the possibility of ascertainment bias in regard to the gender ratio. Most people who are diagnosed with Autism are diagnosed as children. An Autism diagnosis of a child is quite subjective, an important part is an IQ test (where the psychologist interprets the intent of the child in the many cases where answers aren t clear) to compare social skills with IQ. So whether a child is diagnosed is determined by the psychologist s impression of the child s IQ vs the impression of their social skills. Whether a child is even taken for assessment depends on whether they act in a way that s considered to be obviously different. Any child who is suspected of being on the Autism Spectrum will be compared other children who have been diagnosed (IE mostly boys) and this will probably increase the probability that a boy will be assessed. So an Aspie girl might not be assessed because she acts like other Aspie girls not like the Aspie boys her parents and teachers have seen. The way kids act is not solely determined by neuro-type. Our society expects and encourages boys to be louder than girls and take longer and more frequent turns to speak, this is so widespread that I don t think it s possible for parents to avoid it if their kids are exposed to the outside world. Because of this boys who would be diagnosed with Asperger Syndrome by DSM-IV tend to act in ways that are obviously different from other kids. While the combination of Autism and the the social expectations on girls tends to result in girls who are quiet, shy, and apologetic. The fact that girls are less obviously different and that their differences cause fewer difficulties for parents and teachers makes them less likely to be assessed. Note that the differences in behavior of boys and girls who have been diagnosed is noted by the professionals (and was discussed at a conference on AsperGirls that my wife attended) while the idea that this affects assessment rates is my theory. Jeff also cites the book The Essential Difference: Male And Female Brains And The Truth About Autism by Professor Simon Baron-Cohen (who s (in)famous for his Extreme Male Brain theory). The first thing to note about the Extreme Male Brain theory are that it depends almost entirely on the 4.3:1 ratio of males to females on the Autism Spectrum (which is dubious as I noted above). The only other evidence in support of it is subjective studies of children which suffer from the same cultural issues this is why double blind tests should be used whenever possible. The book Delusions of Gender by Cordelia Fine [8] debunks Simon Baron-Cohen s work among other things. The look inside feature of the Amazon page for Delusions of Gender allows you to read about Simon Baron-Cohen s work [9]. Now even if the Extreme Male Brain theory had any merit it would be a really bad idea to cite it (or a book based on it) if you want to make things better for women in the IT industry. Cordelia s book debunks the science and also shows how such claims about supposed essential difference are taken as exclusionary. The Problem with Jeff Atwood Jeff suggests in his post that men should listen to women. Then he and his followers have a huge flame-war with many women over twitter during which which he tweeted Trying to diversify my follows by following any female voices that engaged me in a civil, constructive way recently . If you only listen to women who agree with you then that doesn t really count as listening to women. When you have a stated policy of only listening to women who agree then it seems to be more about limiting what women may feel free to say around you. The Geek Feminism wiki page about the Tone Argument [10] says the following: One way in which the tone argument frequently manifests itself is as a call for civility. A way to gauge whether a request for civility is sincere or not is to ask whether the person asking for civility has more power along whatever axes are contextually relevant (see Intersectionality) than the person being called incivil , less power, or equal power. Often, people who have the privilege of being listened to and taken seriously level accusations of incivility as a silencing tactic, and label as incivil any speech or behavior that questions their privilege. For example, some men label any feminist thought or speech as hostile or impolite; there is no way for anybody to question male power or privilege without being called rude or aggressive. Likewise, some white people label any critical discussion of race, particularly when initiated by people of color, as incivil. Writing about one topic is also a really good idea. A blog post titled What Can Men Do should be about things that men can do. Not about Autism, speculation about supposed inherent differences between men and women which are based on bad research, gender diversity in various occupations, etc. Following up a post on What Can Men Do with discussion (in blog comments and twitter) about what women should do before they are allowed to join the conversation is ridiculous. Jeff s blog post says that men should listen to women, excluding women based on the tone argument is gross hypocrisy. Swearing Jeff makes a big deal of the fact that Shanley uses some profane language in her tweets. This combines a couple of different ways of silencing women. It s quite common for women to be held to a high standard of ladylike behavior, while men get a free pass on doing the same thing. One example of this is the Geek Feminism article about the results of Sarah Sharp s request for civility in the Linux kernel community [11]. That s not an isolated incident, to the best of my recollection in 20+ years my local Linux Users Group has had only one debate about profanity on mailing lists in that case a woman (who is no longer active in the group) was criticised for using lesser profanity than men used both before and after with no comment (as an experiment I used some gratuitous profanity a couple of weeks later and no-one commented). There is also a common difference in interpretation of expressions of emotion, when a woman seems angry then she invariably has men tell her to change her approach (even when there are obvious reasons for her anger) while when a man is angry the possibility that other people shouldn t make him angry will usually be considered. The issues related to the treatment of women have had a large affect on Shanley s life and her friend s lives. It s quite understandable that she is angry about this. Her use of profanity in tweets seems appropriate to the situation. Other Links Newsweek s Gentlemen in Technology article has a section about Jeff [12], it s interesting to note his history of deleting tweets and editing his post. I presume he will change his post in response to mine and not make any note of the differences. Jacob Kaplan-Moss wrote a good rebuttal to Jeff s post [13]. It s a good article and has some other relevant links that are worth reading.

• [1] http://etbe.coker.com.au/2012/04/23/autism-as-an-excuse/
• [2] http://etbe.coker.com.au/2010/12/08/empathy-autism-and-geeks/
• [3] http://geekfeminism.wikia.com/wiki/Autism_is_to_blame
• [4] https://medium.com/tech-culture-briefs/a1e93d985af0
• [5] https://twitter.com/shanley
• [6] http://blog.codinghorror.com/what-can-men-do/
• [7] http://idea.library.drexel.edu/bitstream/1860/2632/1/2006175339.pdf
• [8] http://en.wikipedia.org/wiki/Delusions_of_Gender
• [9] http://www.amazon.com/Delusions-Gender-Society-Neurosexism-Difference/dp/0393340244
• [10] http://geekfeminism.wikia.com/wiki/Tone_argument
• [11] http://geekfeminism.org/2013/09/05/tone-policing-a-tool-for-protecting-male-power/
• [12] http://www.newsweek.com/today-tabs-meet-gentlemen-technology-248696
• [13] http://jacobian.org/writing/what-can-men-do/

I hate the days when I wake up at odd hours for no good reason. This morning I woke up at 4:30am briefly, with no idea why. It makes me wonder if there was some sort of external noise, and that's what woke Zoe up yesterday morning. I got myself going at a reasonable hour this morning, and decided to walk to the Bulimba post office to collect a package that was waiting for me. It was a nice enough morning for a walk, and I'm glad I decided to do it. Along the way, I observed multiple grandmothers walking their grandchildren to school. I've been thinking a lot about next year lately. I feel very strongly about Zoe not having to go into after school care. It seems that one of the new societal norms is the grandparents have to step in with the school transportation and after school care. That's not going to be particularly practical in our situation though, so I'm currently leaning towards seeking some sort of flexible employment next year that will allow me to be able to be available after Zoe finishes school. I was fairly motivated to get stuck into my to do list after my walk. A long standing item was to cancel one of my remaining US credit cards, and the time difference worked out well to do that, so I finally took care of that. I then realised it was a good time for the US East Coast, so I did a Hangout with one of my best friends in New York. It was great to have a catch up. After that, I finished off my US tax return, and also sorted out some other stuff that needed finishing off. I had to go to the Valley to find a FedEx location, so I drove over there, and on the way back stopped off in Bulimba to get some Mother's Day cards. By that stage it was about time to pick up Zoe from Kindergarten, so I quickly grabbed some lunch and then drove over to pick her up. She didn't nap again today, so that made our departure from Kindergarten nice and easy. I'd booked both of us in for haircuts at 3pm, so we got home with enough time for Zoe to ride her scooter to the hairdresser. We had our haircuts and Zoe had a bit of a play in the play corner, and we headed back home. I'd bought a venus fly trap from Bunnings last week while we were picking up other bits and pieces. I wanted to show Zoe how they worked, so we watched a few YouTube videos. This one was particularly good. After that, we went down to the rubbish skip with our fly trap to see if we could catch any flies in it, but Zoe quickly lost interest. So we headed back upstairs, and triggered one with a toothpick. I probably could have done this on a Friday and called it a Science Friday activity with a botany bent. After that, we just watched some TV together on the couch until Sarah arrived to pick her up.

Distributing software efficiently accross your platform can be difficult. Every distribution comes with a package manager which is usually suited for this task. APT can be relied upon on when using Debian or a derivative. Unfortunately, the official repositories may not contain everything you need. When you require unpackaged software or more recent versions, it is possible to setup your own local repository. Most of what is presented here was setup for Dailymotion and was greatly inspired by the work done by Rapha l Pinson at Orange.

Setting up your repositories There are three kinds of repositories you may want to setup:

1. A distribution mirror. Such a mirror will save bandwidth, provide faster downloads and permanent access, even when someone searches Google on Google.
2. A local repository for your own packages with the ability to have a staging zone to test packages on some servers before putting them in production.
3. Mirrors for unofficial repositories, like Ubuntu PPA. To avoid unexpected changes, such a repository will also get a staging and a production zone.

Before going further, it is quite important to understand what a repository is. Let s illustrate with the following line from my /etc/apt/sources.list:

deb http://ftp.debian.org/debian/ unstable main contrib non-free

In this example, http://ftp.debian.org/debian/ is the repository and unstable is the distribution. A distribution is subdivided into components. We have three components: main, contrib and non-free. To setup repositories, we will use reprepro. This is not the only solution but it has a good balance between versatility and simplicity. reprepro can only handle one repository. So, the first choice is about how you will split your packages in repositories, distributions and components. Here is what matters:

• A repository cannot contain two identical packages (same name, same version, same architecture).
• Inside a component, you can only have one version of a package.
• Usually, a distribution is a subset of the versions while a component is a subset of the packages. For example, in Debian, with the distribution unstable, you choose to get the most recent versions while with the component main, you choose to get DFSG-free software only.

If you go for several repositories, you will have to handle several reprepro instances and won t be able to easily copy packages from one place to another. At Dailymotion, we put everything in the same repository but it would also be perfectly valid to have three repositories:

• one to mirror the distribution,
• one for your local packages, and
• one to mirror unofficial repositories.

Here is our target setup:

Initial setup First, create a system user to work with the repositories:

$ adduser --system --disabled-password --disabled-login \
>         --home /srv/packages \
>         --group reprepro

All operations should be done with this user only. If you want to setup several repositories, create a directory for each of them. Each repository has those subdirectories:

• conf/ contains the configuration files,
• gpg/ contains the GPG stuff to sign the repository¹,
• logs/ contains the logs,
• www/ contains the repository that should be exported by the web server.

Here is the content of conf/options:

outdir +b/www
logdir +b/logs
gnupghome +b/gpg

Then, you need to create the GPG key to sign the repository:

$ GNUPGHOME=gpg gpg --gen-key
Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 10y
Key expires at mer. 08 nov. 2023 22:30:58 CET
Is this correct? (y/N) y
Real name: Dailymotion Archive Automatic Signing Key
Email address: the-it-operations@dailymotion.com
Comment: 
[...]

By setting an empty password, you allow reprepro to run unattended. You will have to distribute the public key of your new repository to let APT check the archive signature. An easy way is to ship it in some package.

Local mirror of an official distribution Let s start by mirroring a distribution. We want a local mirror of Ubuntu Precise. For this, we need to do two things:

1. Setup a new distribution in conf/distributions.
2. Configure the update sources in conf/updates.

Let s add this block to conf/distributions:

# Ubuntu Precise
Origin: Ubuntu
Label: Ubuntu
Suite: precise
Version: 12.04
Codename: precise
Architectures: i386 amd64
Components: main restricted universe multiverse
UDebComponents: main restricted universe multiverse
Description: Ubuntu Precise 12.04 (with updates and security)
Contents: .gz .bz2
UDebIndices: Packages Release . .gz
Tracking: minimal
Update: - ubuntu-precise ubuntu-precise-updates ubuntu-precise-security
SignWith: yes

This defines the precise distribution in our repository. It contains four components: main, restricted, universe and multiverse (like the regular distribution in official repositories). The Update line starts with a dash. This means reprepro will mark everything as deleted before updating with the provided sources. Old packages will not be kept when they are removed from Ubuntu. In conf/updates, we define the sources:

# Ubuntu Precise
Name: ubuntu-precise
Method: http://fr.archive.ubuntu.com/ubuntu
Fallback: http://de.archive.ubuntu.com/ubuntu
Suite: precise
Components: main main multiverse restricted universe
UDebComponents: main restricted universe multiverse
Architectures: amd64 i386
VerifyRelease: 437D05B5
GetInRelease: no
# Ubuntu Precise Updates
Name: ubuntu-precise-updates
Method: http://fr.archive.ubuntu.com/ubuntu
Fallback: http://de.archive.ubuntu.com/ubuntu
Suite: precise-updates
Components: main restricted universe multiverse
UDebComponents: main restricted universe multiverse
Architectures: amd64 i386
VerifyRelease: 437D05B5
GetInRelease: no
# Ubuntu Precise Security
Name: ubuntu-precise-security
Method: http://fr.archive.ubuntu.com/ubuntu
Fallback: http://de.archive.ubuntu.com/ubuntu
Suite: precise-security
Components: main restricted universe multiverse
UDebComponents: main restricted universe multiverse
Architectures: amd64 i386
VerifyRelease: 437D05B5
GetInRelease: no

The VerifyRelease lines are GPG key fingerprint to use to check the remote repository. The key needs to be imported in the local keyring:

$ gpg --keyring /usr/share/keyrings/ubuntu-archive-keyring.gpg \
>     --export 437D05B5   GNUPGHOME=gpg gpg --import

Another important point is that we merge three distributions (precise, precise-updates and precise-security) into a single distribution (precise) in our local repository. This may cause some difficulties with tools expecting the three distributions to be available (like the Debian Installer²). Next, you can run reprepro and ask it to update your local mirror:

$ reprepro update

This will take some time on the first run. You can execute this command every night. reprepro is not the fastest mirror solution but it is easy to setup, flexible and reliable.

Repository for local packages Let s configure the repository to accept local packages. For each official distribution (like precise), we will configure two distributions:

• precise-staging contains packages that have not been fully tested and not ready to go to production.
• precise-prod contains production packages copied from precise-staging.

In our workflow, packages are introduced in precise-staging where they can be tested and will be copied to precise-prod when we want them to be available for production. You can adopt a more complex workflow if you need. The reprepro part is quite easy. We add the following blocks into conf/distributions:

# Dailymotion Precise packages (staging)
Origin: Dailymotion #  
Label: dm-staging   #  
Suite: precise-staging
Codename: precise-staging
Architectures: i386 amd64 source
Components: main role/dns role/database role/web #  
Description: Dailymotion Precise staging repository
Contents: .gz .bz2
Tracking: keep
SignWith: yes
NotAutomatic: yes #  
Log: packages.dm-precise-staging.log
 --type=dsc email-changes
# Dailymotion Precise packages (prod)
Origin: Dailymotion #  
Label: dm-prod      #  
Suite: precise-prod
Codename: precise-prod
Architectures: i386 amd64 source
Components: main role/dns role/database role/web #  
Description: Dailymotion Precise prod repository
Contents: .gz .bz2
Tracking: keep
SignWith: yes
Log: packages.dm-precise-prod.log

First notice we use several components (in ):

• main will contain packages that are not specific to a subset of the platform. If you put a package in main, it should work correctly on any host.
• role/* are components dedicated to a subset of the platform. For example, in role/dns, we ship a custom version of BIND.

The staging distribution has the NotAutomatic flag (in ) which disallows the package manager to install those packages except if the user explicitely requests it. Just below, when a new dsc file is uploaded, the hook email-changes will be executed. It should be in the conf/ directory. The Origin and Label lines (in ) are quite important to be able to define an explicit policy of which packages should be installed. Let s say we use the following /etc/apt/sources.list file:

# Ubuntu packages
deb http://packages.dm.gg/dailymotion precise main restricted universe multiverse
# Dailymotion packages
deb http://packages.dm.gg/dailymotion precise-prod    main role/dns
deb http://packages.dm.gg/dailymotion precise-staging main role/dns

All servers have the precise-staging distribution. We must ensure we won t install those packages by mistake. The NotAutomatic flag is one possible safe-guard. We also use a tailored /etc/apt/preferences:

Explanation: Dailymotion packages of a specific component should be more preferred
Package: *
Pin: release o=Dailymotion, l=dm-prod, c=role/*
Pin-Priority: 950
Explanation: Dailymotion packages should be preferred
Package: *
Pin: release o=Dailymotion, l=dm-prod
Pin-Priority: 900
Explanation: staging should never be preferred
Package: *
Pin: release o=Dailymotion, l=dm-staging
Pin-Priority: -100

By default, packages will have a priority of 500. By setting a priority of -100 to the staging distribution, we ensure the packages cannot be installed at all. This is stronger than NotAutomatic which sets the priority to 1. When a package exists in Ubuntu and in our local repository, we ensure that, if this is a production package, we will use ours by using a priority of 900 (or 950 if we match a specific role component). Have a look at the How APT Interprets Priorities section of apt_preferences(5) manual page for additional information. Keep in mind that version matters only when the priority is the same. To check if everything works as you expect, use apt-cache policy:

$ apt-cache policy php5-memcache
  Installed: 3.0.8-1~precise2~dm1
  Candidate: 3.0.8-1~precise2~dm1
  Version table:
 *** 3.0.8-1~precise2~dm1 0
        950 http://packages.dm.gg/dailymotion/ precise-prod/role/web amd64 Packages
        100 /var/lib/dpkg/status
     3.0.8-1~precise1~dm4 0
        900 http://packages.dm.gg/dailymotion/ precise-prod/main amd64 Packages
       -100 http://packages.dm.gg/dailymotion/ precise-staging/main amd64 Packages
     3.0.6-1 0
        500 http://packages.dm.gg/dailymotion/ precise/universe amd64 Packages

If we want to install a package from the staging distribution, we can use apt-get with the -t precise-staging option to raise the priority of this distribution to 990. Once you have tested your package, you can copy it from the staging distribution to the production distribution:

$ reprepro -C main copysrc precise-prod precise-staging wackadoodle

Local mirror of third-party repositories Sometimes, you want a software published on some third-party repository without to repackage it yourself. A common example is the repositories edited by hardware vendors. Like for an Ubuntu mirror, there are two steps: defining the distribution and defining the source. We chose to put such mirrors into the same distributions as our local packages but with a dedicated component for each mirror. This way, those third-party packages will share the same workflow as our local packages: they will appear in the staging distribution, we validate them and copy them to the production distribution. The first step is to add the components and an appropriate Update line to conf/distributions:

Origin: Dailymotion
Label: dm-staging
Suite: precise-staging
Components: main role/dns role/database role/web vendor/hp
Update: hp
# [...]
Origin: Dailymotion
Label: dm-prod
Suite: precise-prod
Components: main role/dns role/database role/web vendor/hp
# [...]

We added the vendor/hp component to both the staging and the production distributions. However, only the staging distribution gets an Update line (remember, packages will be copied manually into the production distribution). We declare the source in conf/updates:

# HP repository
Name: hp
Method: http://downloads.linux.hp.com/SDR/downloads/ManagementComponentPack/
Suite: precise/current
Components: non-free>vendor/hp
Architectures: i386 amd64
VerifyRelease: 2689B887
GetInRelease: no

Don t forget to add the GPG key to your local keyring. Notice an interesting feature of reprepro: we copy the remote non-free component to our local vendor/hp component. Then, you can synchronize the mirror with reprepro update. Once the packages have been tested, you will have to copy them in the production distribution.

Building Debian packages Our reprepro setup seems complete, but how do we put packages into the staging distribution? You have several options to build Debian packages for your local repository. It really depends on how much time you want to invest in this activity:

1. Build packages from source by adding a debian/ directory. This is the classic way of building Debian packages. You can start from scratch or use an existing package as a base. In the latest case, the package can be from the official archive but for a more recent distribution or a backport or from an unofficial repository.
2. Use a tool that will create a binary package from a directory, like fpm. Such a tool will try to guess a lot of things to minimize your work. It can even download everything for you.

There is no universal solution. If you don t have the time budget for building packages from source, have a look at fpm. I would advise you to use the first approach when possible because you will get those perks for free:

• You keep the sources in your repository. Whenever you need to rebuild something to fix an emergency bug, you won t have to hunt the sources which may be unavailable when you need them the most. Of course, this only works if you build packages that don t download stuff directly from the Internet.
• You also keep the recipe³ to build the package in your repository. If someone enables some option and rebuild the package, you won t accidently drop this option on the next build. Those changes can be documented in debian/changelog. Moreover, you can use a version control software for the whole debian/ directory.
• You can propose your package for inclusion into Debian. This will help many people once the package hits the archive.

Builders We chose pbuilder as a builder⁴. Its setup is quite straightforward. Here is our /etc/pbuilderrc:

DISTRIBUTION=$DIST
NAME="$DIST-$ARCH"
MIRRORSITE=http://packages.dm.gg/dailymotion
COMPONENTS=("main" "restricted" "universe" "multiverse")
OTHERMIRROR="deb http://packages.dm.gg/dailymotion $ DIST -staging main"
HOOKDIR=/etc/pbuilder/hooks.d
BASE=/var/cache/pbuilder/dailymotion
BASETGZ=$BASE/$NAME/base.tgz
BUILDRESULT=$BASE/$NAME/results/
APTCACHE=$BASE/$NAME/aptcache/
DEBBUILDOPTS="-sa"
KEYRING="/usr/share/keyrings/dailymotion-archive.keyring.gpg"
DEBOOTSTRAPOPTS=("--arch" "$ARCH" "--variant=buildd" "$ DEBOOTSTRAPOPTS[@] " "--keyring=$KEYRING")
APTKEYRINGS=("$KEYRING")
EXTRAPACKAGES=("dailymotion-archive-keyring")

pbuilder is expected to be invoked with DIST, ARCH and optionally ROLE environment variables. Building the initial bases can be done like this:

for ARCH in i386 amd64; do
  for DIST in precise; do
    export ARCH
    export DIST
    pbuilder --create
  done
done

We don t create a base for each role. Instead, we use a D hook to add the appropriate source:

#!/bin/bash
[ -z "$ROLE" ]    
  cat >> /etc/apt/sources.list <<EOF
deb http://packages.dm.gg/dailymotion $ DIST -staging role/$ ROLE 
EOF
 
apt-get update

We ensure packages from our staging distribution are preferred over other packages by adding an /etc/apt/preferences file in a E hook:

#!/bin/bash
cat > /etc/apt/preferences <<EOF
Explanation: Dailymotion packages are of higher priority
Package: *
Pin: release o=Dailymotion
Pin-Priority: 900
EOF

We also use a C hook to get a shell in case there is an error. This is convenient to debug a problem:

#!/bin/bash
apt-get install -y --force-yes vim less
cd /tmp/buildd/*/debian/..
/bin/bash < /dev/tty > /dev/tty 2> /dev/tty

A manual build can be run with:

$ ARCH=amd64 DIST=precise ROLE=web pbuilder \
>         --build somepackage.dsc

Version numbering To avoid to apply complex rules to chose a version number for a package, we chose to treat everything as a backport, even in-house software. We use the following scheme: X-Y~preciseZ+dmW.

• X is the upstream version⁵.
• Y is the Debian version. If there is no Debian version, use 0.
• Z is the Ubuntu backport version. Again, if such a version doesn t exist, use 0.
• W is our version of the package. We increment it when we make a change to the packaging. This is the only number we are allowed to control. All the others are set by an upstream entity, unless it doesn t exist and in this case, you use 0.

Let s suppose you need to backport wackadoodle. It is available in a more recent version of Ubuntu as 1.4-3. Your first backport will be 1.4-3~precise0+dm1. After a change to the packaging, the version will be 1.4-3~precise0+dm2. A new upstream version 1.5 is available and you need it. You will use 1.5-0~precise0+dm1. Later, this new upstream version will be available in some version of Ubuntu as 1.5-3ubuntu1. You will rebase your changes on this version and get 1.5-3ubuntu1~precise0+dm1. When using Debian instead of Ubuntu, a compatible convention could be : X-Y~bpo70+Z~dm+W.

Uploading To upload a package, a common setup is the following workflow:

1. Upload the source package to an incoming directory.
2. reprepro will notice the source package, check its correctness (signature, distribution) and put it in the archive.
3. The builder will notice a new package needs to be built and build it.
4. Once the package is built, the builder will upload the result to the incoming directory.
5. reprepro will notice again the new binary package and integrate it in the archive.

This workflow has the disadvantage to have many moving pieces and to leave the user in the dark while the compilation is in progress. As an alternative, a simple script can be used to execute each step synchronously. The user can follow on their terminal that everything works as expected. Once we have the .changes file, the build script just issues the appropriate command to include the result in the archive:

$ reprepro -C main include precise-staging \
>      wackadoodle_1.4-3~precise0+dm4_amd64.changes

Happy hacking!

---

1. The gpg/ directory could be shared by several repositories.
2. We teached Debian Installer to work with our setup with an appropriate preseed file.
3. fpm-cookery is a convenient tool to write recipes for fpm, similar to Homebrew or a BSD port tree. It could be used to achieve the same goal.
4. sbuild is an alternative to pbuilder and is the official builder for both Debian and Ubuntu. Historically, pbuilder was more focused on developers needs.
5. For a Git snapshot, we use something like 1.4-git20130905+1-ae42dc1 which is a snapshot made after version 1.4 (use 0.0 if no version has ever been released) at the given date. The following 1 is to be able to package different snapshots at the same date while the hash is here in case you need to retrieve the exact snapshot.

I ve seen many comments about swap space and SSD claiming that swap will inherently destroy SSD through using too many writes. The latest was in the comments of my post about swap space and SSD performance [1]. Note that I m not criticising the person who commented on my blog, everyone has heard lots of reports about possible problems that they avoid without analysing them in detail. The first thing to note is that the quality of flash memory varies a lot, the chips that are used in SSDs for workstation/server use are designed to last while those in USB-flash devices aren t. I ve documented my unsuccessful experiments with using USB-flash for the root filesystem of a gateway server [2] (and the flash device that wasn t used for swap died too). The real issue when determining whether swap will break your SSD is the amount of writes. While swapping can do a lot of writing quickly that usually doesn t happen unless something has gone wrong. The workstation that I m currently using has writes to the root fileystem outnumbering writes to swap by a factor of 130:1 (by volume of data written). On other days I ve seen it as low as 42:1, in either case it s writes to the root filesystem (which is BTRFS and includes /home etc) that will break the SSD if anything. For some other workstations I run I see ratios of 201:1 (that s with 8G of RAM), 57:1, and 23:1. In a quick search I couldn t find a single system I run where even 10% of disk writes were attributed to swap. This really isn t surprising given that adding RAM is a cheap way to improve the performance of most systems. If a SSD didn t do any write leveling (as is rumored to be the case with cheap USB flash devices) then swap use might still cause a problem because of the number of writes in a small area, but if that was the case then filesystem journals and other fixed data structures would be more likely to cause a problem and any swap based breakage would break swap not the root filesystem (although if swap was the first partition then it might also break the MBR). Of the workstations that are convenient to inspect the one with the most writes to the root filesystem and swap space (IE everything on /dev/sda) had 128G written per day for 1.2 days of uptime, but that involved running a filesystem balance and some torrent downloads (not the typical use). Among the two systems which had been running for more than 48 hours with typical use the most writes was 24G in a day. If a SSD can sustain 10,000 writes per block (which is smaller than quoted by most flash manufacturers nowadays) and has perfect wear leveling (which is unlikely) then the system with a 120G SSD and 128G written in a day could continue like that for almost 10,000 days or 27 years much longer than storage is expected to work or be useful. So for workstation use (where even 24G of writes per day probably counts as heavy use) a 120G SSD that can sustain 10,000 writes per block shouldn t be at great risk of wearing out. Conclusion I believe that swap is much less likely to break SSDs than regular file access on every system with a reasonable amount of RAM. For systems which don t have enough RAM you probably want the speed of SSD for swap space anyway in spite of the risks. If wear leveling works as designed and the 10,000+ writes per block claims are accurate then SSDs will massively outlast their useful life. Long before they wear out they should be too small, too slow, and probably not compatible. 26 years ago I had a 5.25 full height ST-506 disk in my desktop PC, it wouldn t physically fit in most systems I own now (unless I removed the DVD drive), there is no possibility of buying a controller (I don t own a system with an ISA bus), and that disk was too slow and small by today s standards. A 27yo SSD isn t going to be useful for anything, even for archive storage it s no good as no-one has tested long term storage and data could decay before then.

• [1] http://etbe.coker.com.au/2014/04/19/swap-space-ssd/
• [2] http://etbe.coker.com.au/2012/05/22/another-usb-flash-failure/

I'm moving. Well, not right here, right now. Rather less than two years. But I already know what my flat will look like and was able to influence that decision. And there will be more to influence, like what to do with common rooms in the builing, or what to put in the garden (voting for climbing facilities for my son of course!). It's this kind of co-housing project where you already know your neighbours beforehand and can find common grounds for decisions like that. The co-housing project I'm moving to is called Que[e]rbau. And it will be living up to its name. It is specificly aimed at people who live tolerance and acceptance, and also potentially live an sorta alternative lifestyle, defining their own identity; but not limited to those. There also will be conventional families living there who specificly don't want to raise their kids in a conservative environment. When I told about this plan to someone they asked me if I really want to do that. Their concerns were with respect to my son and if the house wouldn't become a target. I was puzzled at first, given that we have the Rainbow Parade, the Life Ball and most of all the Rosa Lila Villa since several years in Vienna and I'm not aware of any bigger disturbances it causes, rather the opposite. After thinking a while about it it sounded a bit for the wish of a Don't Ask Don't Tell environment. Recently there was this great documentary done by Vice on youtube about Young and Gay in Putin's Russia (watch all five parts of it, it's worth it). In the light of that I don't think hiding does improve the situation, rather the opposite. Not speaking about it doesn't improve acceptance. And actually, I was approached by at least one person during the Debian Women MiniDebconf about how brave I am considered. I'm not sure if it really is brave, I just don't want to lie to myself anymore, and I very rarely had troubles through that. The more open and natural you behave, the less confrontation area you leave left, and people notice that. Not totally unrelated to that, I created myself a new gpg key. It doesn't carry my official name anymore but just the name I prefer to be addressed with: Rhonda. It also carries a last name you might not have heard yet (it was adopted on the Discworld MUD several years ago, even before I wrote Mermaids; actually in connection with the person who partly triggered the poem), that's the reason I added a plain Rhonda UID to it for those who aren't aware of the last name. I will submit that key to keysigning parties from now on, and it of course is up to you if you feel comfortable with signing it.

/personal permanent link Comments: 0 Flattr this

On the last UDS we talked about migrating from upstart to systemd to boot Ubuntu, after Mark announced that Ubuntu will follow Debian in that regard. There s a lot of work to do, but it parallelizes well once developers can run systemd on their workstations or in VMs easily and the system boots up enough to still be able to work with it. So today I merged our systemd package with Debian again, dropped the systemd-services split (which wasn t accepted by Debian and will be unnecessary now), and put it into my systemd PPA. Quite surprisingly, this booted a fresh 14.04 VM pretty much right away (of course there s no Plymouth prettiness). The main two things which were missing were NetworkManager and lightdm, as these don t have an init.d script at all (NM) or it isn t enabled (lightdm). Thus the PPA also contains updated packages for these two which provide a proper systemd unit. With that, the desktop is pretty much fully working, except for some details like cron not running. I didn t go through /etc/init/*.conf with a small comb yet to check which upstart jobs need to be ported, that s now part of the TODO list. So, if you want to help with that, or just test and tell us what s wrong, take the plunge. In a 14.04 VM (or real machine if you feel adventurous), do

  sudo add-apt-repository ppa:pitti/systemd
  sudo apt-get update
  sudo apt-get dist-upgrade

This will replace systemd-services with systemd, update network-manager and lightdm, and a few libraries. Up to now, when you reboot you ll still get good old upstart. To actually boot with systemd, press Shift during boot to get the grub menu, edit the Ubuntu stanza, and append this to the linux line: init=/lib/systemd/systemd. For the record, if pressing shift doesn t work for you (too fast, VM, or similar), enable the grub menu with

  sudo sed -i '/GRUB_HIDDEN_TIMEOUT/ s/^/#/' /etc/default/grub
  sudo update-grub

Once you are satisfied that your system boots well enough, you can make this permanent by adding the init= option to /etc/default/grub (and possibly remove the comment sign from the GRUB_HIDDEN_TIMEOUT lines) and run sudo update-grub again. To go back to upstart, just edit the file again, remove the init=sudo update-grub again. I ll be on the Debian systemd/GNOME sprint next weekend, so I feel reasonably well prepared now. Update: As the comments pointed out, this bricked /etc/resolv.conf. I now uploaded a resolvconf package to the PPA which provides the missing unit (counterpart to the /etc/init/resolvconf.conf upstart job) and this now works fine. If you are in that situation, please boot with upstart, and do the following to clean up:

  sudo rm /etc/resolv.conf
  sudo ln -s ../run/resolvconf/resolv.conf /etc/resolv.conf

Then you can boot back to systemd. Update 2: If you want to help testing, please file bugs with a systemd-boot tag. See the list of known bugs when booting with systemd.

Here are two separate ideas about programming, Internet security, Internet architecture, and free software. Both of them are fundamental to everything those of us who work on free software are doing.

1. Writing secure and reliable code is a highly complex and demanding task. It's something that one has to learn, like any other skilled profession. It's not something we're very good at teaching via any mechanism other than apprenticeship and experimentation. The field is changing quickly; if you took ten years off from writing security-critical code, you would expect to have to learn multiple new tools (static analysis, testing techniques, security features), possibly new programming languages, and not infrequently new ways of thinking about threat models and vulnerabilities.
2. Nearly every computer user trusts tens of thousands of other programmers and millions of lines of code with their day-to-day computer security and reliability, without auditing that code themselves. Even if you have the skill required to audit the code (and very, very few people have the skill to audit all of the code that they use), you do not have the time. Therefore, our computer security is built on trust. We're trusting other programmers not to be malicious, which is obvious, but we're also trusting other programmers to be highly skilled, careful, cautious (but still fast and productive, since we quickly abandon software that isn't "actively developed"), and constantly adopting new techniques and treat models.

I think both of those principles are very widely understood and widely acknowledged. And, as we all know, both of those principles have failure modes, and those failures mean that our computers are nowhere near as secure as we would like them to be. There has been quite a lot of technical discussion about both of these principles in recent days and months, ranging from better code analysis and testing through the flaws in particular programming languages to serious questions about the trust model we use for verifying the code that we're running. I'm going to switch gears away from that discussion for a moment to talk about a social aspect. When a piece of code that one is using has a security vulnerability, it is not unusual to treat that as a trust failure. In other words, technical flaws are very quickly escalated to social flaws. This is less likely among practitioners, since we're all painfully aware of how many bugs we have in our own code, but it's still not unheard of, particularly if the authors of the code do not immediately and publicly show a socially-acceptable level of humility and contrition (in the middle of what is often a horrifically stressful experience). I think this reaction comes largely from fear. Anyone who has given it a moment's thought is painfully aware of just how much code they are running on trust, and just how many ways it could be compromised, accidentally or maliciously. And how frequently that code is compromised. The less control one has over a situation, the more terrifying it is, and the cold reality is that we have very little control over our computer security. There is a natural tendency, when afraid, to look for targets for that fear. Now, go back and think about the first principle. Anyone who has ever worked on or near free software is painfully aware that we have far more good ideas about how to improve computing and security than we have skilled resources to execute on those ideas. My own backlog of things that I've already thought about, know would be good ideas, and simply have to implement is probably longer than my remaining lifespan. I suspect that's the case for nearly all of us. In other words, we have a severe shortage of programmers who care and who have skill. We desperately need more skilled programmers who can write secure and reliable code. We may (and I think do) also need better tools, techniques, languages, protocols, and all the other machinery that we, as technical people, spend most of our time thinking about. But none of that changes the fact that we need more skilled people. In fact, it makes that need more acute: in addition to skilled people to write the code we use, we need skilled people to write the tools. Skilled people are not born. They're made. And in professions where training techniques are still in their infancy and where we don't have a good formal grasp on which techniques work, those skilled people are made primarily through apprenticeship, experimentation, and learning from failure. Worse, people who were skilled do not remain skilled without continually participating in that learning process. See the above point about a fast-changing field with evolving best practices. It's not enough to know how to write secure code to the best known practices today, or even enough to retrofit all of your existing code to current knowledge (which is often so large of an effort as to be practically impossible). You have to constantly, continually learn more, for which there is no reliable formal training. We have to try, fail, try again, and fail better. But failure that leads to a security vulnerability is treated as a loss of trust. We trusted that person to write secure code that we could use. They failed. Now we can't trust them. Based on the trust model of security, we should revoke their ability to try again and instead rely on people who have not failed, since that will make us more secure. Except now we just broke the learning process. And there's no such thing as a programmer who can stop learning. So what does that do to our resource pool? It's sadly ironic, but I believe the free software community writ large has a very serious collaboration problem: we do not tolerate each other's learning processes. This leads to a wide variety of social failures around hostile communities and the failures of meritocracy that other people have talked about at much greater length. But even if you set that all aside, it threatens our security. We need secure code, a lot of it, a lot more than we have right now. To get that code, we need people who can write it. We need to grow, encourage, and support those people and enable their learning processes. Code is written by people. If we rip people apart when they write bad, insecure code, we don't get better, secure code. We get fewer people writing security code. We get far fewer people writing security code in public, since some of the people who haven't been ripped apart will look at that experience and say, "No way am I going near that problem area. It's too scary. I don't want to end up like those programmers." Fewer people writing security code means fewer people learning how to write better security code. Fewer people capable of writing good, secure code is not a solution to any of our problems. If we do not tolerate, support, and encourage the learning process required to become a skilled programmer, or maintain one's skill in programming, we are destroying our future as a community. When you find code that is broken and badly written, you have found a problem that should be reported, analyzed, and corrected. You have also found a programmer who is about to have one of two very different types of experiences. Either they are about to learn how to become a better programmer, or they are about to be publicly shamed, humiliated, and treated as untrustworthy. Which branch they take is partly up to them, but it's also heavily influenced by how all of us react socially to the discovery of bad code. One of those branches leads to more good, secure code being written in the future. The other does not.

The Google Summer of Code 2014 programme reached another milestone recently: the accepted proposals were published, and over a thousand students were selected by nearly two hundred mentoring organisations, among them the syslog-ng project. We will be working with four students this year (twice we worked with last year), with more mentors, on a wider selection of projects. It was a tough decision to select the proposals, we received some very strong work this year.We would like to express our gratitude to both Debian, for giving us an extra slot (so we could accept four students instead of three), and Google and Carol Smith in particular, for allowing it, and putting up with our fumbling during the process.The accepted projects and students are:

• Tibor Benke: Integration with configuration management systems.

• L szl M sz ros: ZMQ transport, source and destination.

• Gy rgy Demarcsek: TLS support for the MongoDB destination.

• M ZES d m Istv n: AMQP source driver.

Good luck to all students, we're looking forward to working with you throughout the summer! Happy hacking!