Search Results: "zed"

20 October 2015

Russ Allbery: Review: The Oathbound

Review: The Oathbound, by Mercedes Lackey
Series: Vows and Honor #1
Publisher: DAW
Copyright: July 1988
ISBN: 0-88677-414-4
Format: Mass market
Pages: 302
This book warrants a bit of explanation. Before Arrows of the Queen, before Valdemar (at least in terms of publication dates), came Tarma and Kethry short stories. I don't know if they were always intended to be set in the same world as Valdemar; if not, they were quickly included. But they came from another part of the world and a slightly different sub-genre. While the first two Valdemar trilogies were largely coming-of-age fantasy, Tarma and Kethry are itinerant sword-and-sorcery adventures featuring two women with a soul bond: the conventionally attractive, aristocratic mage Kethry, and the celibate, goddess-sworn swordswoman Tarma. Their first story was published, appropriately, in Marion Zimmer Bradley's Swords and Sorceress III. This is the first book about Tarma and Kethry. It's a fix-up novel: shorter stories, bridged and re-edited, and glued together with some additional material. And it does not contain the first Tarma and Kethry story. As mentioned in my earlier Valdemar reviews, this is a re-read, but it's been something like twenty years since I previously read the whole Valdemar corpus (as it was at the time; I'll probably re-read everything I have on hand, but it's grown considerably, and I may not chase down the rest of it). One of the things I'd forgotten is how oddly, from a novel reader's perspective, the Tarma and Kethry stories were collected. Knowing what I know now about publishing, I assume Swords and Sorceress III was still in print at the time The Oathbound was published, or the rights weren't available for some other reason, so their first story had to be omitted. Whatever the reason, The Oathbound starts with a jarring gap that's no less irritating in this re-read than it was originally. Also as is becoming typical for this series, I remembered a lot more world-building and character development than is actually present in at least this first book. In this case, I strongly suspect most of that characterization is in Oathbreakers, which I remember as being more of a coherent single story and less of a fix-up of puzzle and adventure stories with scant time for character growth. I'll be able to test my memory shortly. What we do get is Kethry's reconciliation of her past, a brief look at the Shin'a'in and the depth of Tarma and Kethry's mutual oath (unfortunately told more than shown), the introduction of Warrl (again, a relationship that will grow a great deal more depth later), and then some typical sword and sorcery episodes: a locked room mystery, a caravan guard adventure about which I'll have more to say later, and two rather unpleasant encounters with a demon. The material is bridged enough that it has a vague novel-like shape, but the bones of the underlying short stories are pretty obvious. One can tell this isn't really a novel even without the tell of a narrative recap in later chapters of events that you'd just read earlier in the same book. What we also get is rather a lot of rape, and one episode of seriously unpleasant "justice." A drawback of early Lackey is that her villains are pure evil. My not entirely trustworthy memory tells me that this moderates over time, but early stories tend to feature villains completely devoid of redeeming qualities. In this book alone one gets to choose between the rapist pedophile, the rapist lord, the rapist bandit, and the rapist demon who had been doing extensive research in Jack Chalker novels. You'll notice a theme. Most of the rape happens off camera, but I was still thoroughly sick of it by the end of the book. This was already a cliched motivation tactic when these stories were written. Worse, as with the end of Arrow's Flight, the protagonists don't seem to be above a bit of "turnabout is fair play." When you're dealing with rape as a primary plot motivation, that goes about as badly as you might expect. The final episode here involves a confrontation that Tarma and Kethry brought entirely on themselves through some rather despicable actions, and from which they should have taken a lesson about why civilized societies have criminal justice systems. Unfortunately, despite an ethical priest who is mostly played for mild amusement, no one in the book seems to have drawn that rather obvious conclusion. This, too, I recall as getting better as the series goes along and Lackey matures as a writer, but that only helps marginally with the early books. Some time after the publication of The Oathbound and Oathbreakers, something (presumably the rights situation) changed. Oathblood was published in 1998 and includes not only the first Tarma and Kethry story but also several of the short stories that make up this book, in (I assume) something closer to their original form. That makes The Oathbound somewhat pointless and entirely skippable. I re-read it first because that's how I first approached the series many years ago, and (to be honest) because I'd forgotten how much was reprinted in Oathblood. I'd advise a new reader to skip it entirely, start with the short stories in Oathblood, and then read Oathbreakers before reading the final novella. You'd miss the demon stories, but that's probably for the best. I'm complaining a lot about this book, but that's partly from familiarity. If you can stomach the rape and one stunningly unethical protagonist decision, the stories that make it up are solid and enjoyable, and the dynamic between Tarma and Kethry is always a lot of fun (and gets even better when Warrl is added to the mix). I think my favorite was the locked room mystery. It's significantly spoiled by knowing the ending, and it has little deeper significance, but it's a classic sort unembellished, unapologetic sword-and-sorcery tale that's hard to come by in books. But since it too is reprinted (in a better form) in Oathblood, there's no point in reading it here. Followed by Oathbreakers. Rating: 6 out of 10

16 October 2015

Russ Allbery: Review: Ancillary Mercy

Review: Ancillary Mercy, by Ann Leckie
Series: Imperial Radch #3
Publisher: Orbit
Copyright: October 2015
ISBN: 0-316-24668-9
Format: Trade paperback
Pages: 330
This is the third book of the series that starts with Ancillary Justice. Please don't read it out of order. The three books form a rare thing in SF: a trilogy with a clear start and a clear ending. They're ideally read fairly close together so that you can remember the people involved. By the end of Ancillary Sword, Leckie had apparently resolved most of the plot in the Athoek system, although the Ghost Gate and broader imperial turmoil still lurked. I was expecting this novel to be another change of scene, and probably a broadening of the scope into the more sweeping events set off by earlier books. That's not what Ancillary Mercy is at all. Leckie caught me by surprise early, and the book kept not going in the direction I was expecting. And the result is brilliant. It's obviously difficult to talk about this in detail, since there's no way I'm going to spoil this or any of the previous books. This is some of the best SF I've ever read, and I want anyone who's not read the series yet to have the full experience. What I will say is that there was a lot of character growth left, including in some places that I thought would remain setting-mandated blind spots. Leckie tackles some very tricky themes, ones that are hard to do well because of how thoroughly the ground has already been explored, and still manages to find an original take and an even more original tone. And remains true to Breq's characterization throughout, including one critical scene that's masterful in its heart-breaking understatedness. Breq's flat affect is both very difficult and incredibly rewarding as a first-person protagonist. If I had to call out one thing that makes this series so special to me, it's the way that Leckie puts the reader in the middle of a restrained and very careful culture, centers the book on a character with a very unusual relationship to emotion, nearly completely avoids outward displays of feeling, treats emotions primarily as data to be weighed with other information, and yet provides just the right amount of subtle clues and external reflection to show just how central emotion is to everything that happens. This is an incredibly difficult balancing act, much harder than an openly emotional book (let alone the tough-guy cliches that military SF is normally full of), and it makes the occasional pay-offs so rewarding. I think some people might find Ancillary Mercy too understated, but I thought it was note-perfect throughout. And the ending was spectacular. For those who are less enamored of Breq and more interested in the plot, don't worry: we do get some resolution to most of the loose threads. There's clearly potential in this universe for more stories (and apparently more books coming!), but this trilogy comes to a definite end. If I had to quibble, I'd say that a few bits of the plot are too convenient: the resolution would have been impossible without one very critical mistake at the start of the second book, and there's at least one perfectly-timed accident. But Leckie's most critical plot resolution tool was properly hung on the wall in the last book and used perfectly. It's hard to manage a satisfying resolution to a twisty political conflict between very sharp antagonists; for me, at least, Leckie pulls it off. Looking back on the whole series, I find it striking how much more satisfying and believable Leckie's military culture is for her setting than those I normally read. I think we're too often given far-future military stories that are full of impulsive drama, high emotion, and physical feats. But in a world of starships and space stations, advanced computers, careful and systematized roles, and precise engineering, that's not realistic. While there is some physical violence required, problems in that world are more likely to look like operational computer problems than gladiatorial problems. That means the required tools are careful and fast analytical skills, quick iterative decision-making, and precise execution, not physical toughness and grit. In that world, adrenaline is at best useless and often your enemy. Leckie's world of tea sets and emotional control is partly driven by other world-building factors, but, particularly in this book, I was struck by how well-adapted it is to a military where calm decisions and efficient follow-through are far more important than adrenaline-driven courage. The emotion is absolutely there, but the catharsis doesn't come through bloody battles or feats of physicality. Again, so tricky to pull off, since it plays against the expectations we bring as readers to a war story, but I found it so much more satisfying when written well. I also love that this series is not about saving the world, at least in any direct sense. It is about justice, and ethics, and about standing up for what's right, but it's just as much about identity, about how to choose and define one's ethics, and how to make the most of the small number of decisions one gets to make. Everyone in this world is operating under sharp constraints: culture, ability, available alliances, physical limitations, and particularly the role into which they were cast. Huge things happen in this series, but not from the dramatic, order-overturning action of Chosen Ones. Rather, change is created by individuals who find small points of leverage and definition, and who make quiet, critical decisions to become better people than they were. I adore this series, and particularly this conclusion, beyond all rational measure. I'm sure that won't be universal, just as other readers weren't as enamored as I was of the direction Leckie went in Ancillary Sword. But this was a series conclusion that I was not expecting at all, that kept surprising me right up to the climax, and was exactly the conclusion that I wanted. If you liked the previous books, and particularly if you liked Ancillary Sword but thought Breq was a bit too much in control of the plot and a bit too able to make things go her way, I think you'll love Ancillary Mercy. This is simply the best SF series I've read in a long time. Rating: 10 out of 10

14 October 2015

Rhonda D'Vine: Post DebConf15

There are some things that I didn't mention in my sort-of quickly written entry about DebConf15. So first things first. When I received the mail about the room allocation I was at first confused. I was put into a room with other ladies, which I didn't expect. Granted, two of the other three names were people that knew me since a while, but it still felt like a mistake might have happened. But after a while I realized what has happened: It wasn't a mistake, it was intentional, I was finally recognized as woman for the room allocation too, which made me extremely happy. I was just just concerned about the third person who would be in our room who doesn't know me yet and whether it would make them feel uncomfortable. In the end, that was no trouble at all. I felt so empowered and more accepted than ever in this community. And when finally being on-site there another thing happened with me. I started to use the women's restroom. Up to now I usually had the feeling of "it's fine for me to use the male one, and I don't want other women to feel uncomfortable", but somehow, with a skirt on, it in the end made me feeling uncomfortable. Additionally, there were only three times in total when I used the male toilet (and one was on the boat for the daytrip), and every single time of it I felt extremely uncomfortable with it, like others might think I'm just faking it. It at least in my mind doesn't help with accepting me as female when I go to the male restroom. And it's not a Good choice! as a woman did put it during the conference dinner when there was a longer queue infront of the male restroom. It's not so much of a choice over here. But I give her the doubt of not knowing how important these little steps became to me over time. Totally unrelated to the restroom question but interestingly featuring it a fair bit I was made aware of the Assigned Male cartoon. I instantly fell in love with it, and in case you want to enlighten yourself a bit more about how some things you might say or do get received by trans people, be very much invited to read it. Sophie is currently on European tour with her book, unfortunately Vienna/Austria doesn't seem to be part of europe in that respect so I hope someone will be able to visit one of her stations to pick up a book for me ... And then there was also a small inofficial Nail Polish BoF going on at DebConf. I left it on my fingers for the next two weeks, totally in love with it. Unfortunately the nail polish I got for myself after DebConf had a rather big brush so I wasn't able to work on it, I failed miserably. ... which brings me to the empowerment that DebConf meant for me this year, and the time since. Given that I left the nail polish on I even took the comfort in being myself to go to work in my skirt on a more regular basis. Also, a very nice friend did visit me and we went lipstick shopping. I loved the color she chose, even though in the meantime it isn't visible enough for me and I guess I'll get another one rather sooner than later. Also, about what I mentioned in my last blog post was that my name change within the Debian project was granted. A quick update on that is that also now my GPG key got replaced. I guess it's finally time for me to write a gpg transition statement, even though I don't follow those myself. I still prefer meeting up with people face-to-face for signing their new keys. But given that it's called a transition statement makes it more appealing to me on that grounds. :) And I got invited to a local podcast show. Actually I know the person who does the podcast since several years already, he's also part of the local free software community who attends various events, and he does a podcast since several years now called Biertaucher (named after cooling the beer in a fountain). It is held in German language, so if you don't understand German you might want to skip these links.
In the first episode that I joined in I talked about DebConf. Afterwards we were sitting together and talking about that they would like to have more social topics too, not just technical things. So we took that chance and talked in Biertaucher #221 about Polyamory, which was a quite interesting experience. The host intentionally asked questions coming from a quite ignorant point of view, but it went nice. We were three poly people sharing our views and insight how it works for us.
Then there was Biertaucher #223 where it was just me and one of the hosts. We didn't had much to talk about from the past week, so we agreed to talk about Transgender in the end. Granted, it's mostly my personal story, but I guess I got some important topics addressed in a useful way. And, after getting my name changed in Debian, I thought about what it might take to get my name changed officially, too (as if it could get more official than using it throughout my work environment, both payed and voluntary, but ...). I covered that in the podcast, but mostly it is either quite expensive or requires me to change my gender in the register of births, which require a lot of other hassles that include psychiatry. Or, settle for a so-called "gender neutral" name as first name, which both doesn't sound very convincing somehow ... Only time can tell I guess. Guess that's enough for now, if I forgot something I might come back to it. :) One last note: I consider the Debian project a very welcoming one, and that can only work for a fair amount of people if the tone is right. So yes, I wholeheartly agree with the Code of Conduct. And I'm very disappointed to see that there are still people in the project that are advocating for a freedom of expression, so to say. Respectful communication with each other is a must in a bigger community to make it work, not something that might be a nice to have, and calling someone names and ridiculing them for stating that is absolutely not acceptable. I encourage those people to watch How to Throughly Offend and Insult People in Open Source presentation (or at least read the slides) that Gina Likins gave earlier this year. It might give them an idea why it's important to communicate respectful with each other, and that includes banning degrading terms like "SJW" from your vocabulary because it actually speaks a lot more about your own attitude than about the one of the person you use it for.

/personal permanent link Comments: 8 Flattr this

13 October 2015

Julien Danjou: Benchmarking Gnocchi for fun & profit

We got pretty good feedback on Gnocchi so far, even if we only had little. Recently, in order to have a better feeling of where we were at, we wanted to know how fast (or slow) Gnocchi was. The early benchmarks that some of the Mirantis engineers ran last year showed pretty good signs. But a year later, it was time to get real numbers and have a good understanding of Gnocchi capacity. Benchmark tools The first thing I realized when starting that process, is that we were lacking of tools to run benchmarks. Therefore I started to write some benchmark tools in python-gnocchiclient, which provides a command line tool to interrogate Gnocchi. I added a few basic commands to measure metric performance, such as: 
$ gnocchi benchmark metric create -w 48 -n 10000 -a low
+----------------------+------------------+
  Field                  Value             
+----------------------+------------------+
  client workers         48                
  create executed        10000             
  create failures        0                 
  create failures rate   0.00 %            
  create runtime         8.80 seconds      
  create speed           1136.96 create/s  
  delete executed        10000             
  delete failures        0                 
  delete failures rate   0.00 %            
  delete runtime         39.56 seconds     
  delete speed           252.75 delete/s   
+----------------------+------------------+

The command line tool supports the --verbose switch to have detailed progress report on the benchmark progression. So far it supports metric operations only, but that's the most interesting part of Gnocchi. Spinning up some hardware I got a couple of bare metal servers to test Gnocchi on. I dedicated the first one to Gnocchi, and used the second one as the benchmark client, plugged on the same network. Each server is made of 2 Intel Xeon E5-2609 v3 (12 cores in total) and 32 GB of RAM. That provides a lot of CPU to handle requests in parallel. Then I simply performed a basic RHEL 7 installation and ran devstack to spin up an installation of Gnocchi based on the master branch, disabling all of the others OpenStack components. I then tweaked the Apache httpd configuration to use the worker MPM and increased the maximum number of clients that can sent request simultaneously. I configured Gnocchi to use the PostsgreSQL indexer, as it's the recommended one, and the file storage driver, based on Carbonara (Gnocchi own storage engine). That means files were stored locally rather than in Ceph or Swift. Using the file driver is less scalable (you have to run on only one node or uses a technology like NFS to share the files), but it was good enough for this benchmark and to have some numbers and profiling the beast. The OpenStack Keystone authentication middleware was not enabled in this setup, as it would add some delay validating the authentication token. Metric CRUD operations Metric creation is pretty fast. I managed to attain 1500 metric/s created pretty easily. Deletion is now asynchronous, which means it's faster than in Gnocchi 1.2, but it's still slower than creation: 300 metric/s can be deleted. That does not sound like a huge issue since metric deletion is actually barely used in production. Retrieving metric information is also pretty fast and goes up to 800 metric/s. It'd be easy to achieve very higher throughput for this one, as it'd be easy to cache, but we didn't feel the need to implement it so far. Another important thing is that all of these numbers are constant and barely depends on the number of the metric already managed by Gnocchi.
Operation Details Rate
Create metric Created 100k metrics in 77 seconds 1300 metric/s
Delete metric Deleted 100k metrics in 190 seconds 524 metric/s
Show metric Show a metric 100k times in 149 seconds 670 metric/s
Sending and getting measures Pushing measures into metrics is one of the hottest topic. Starting with Gnocchi 1.1, the measures pushed are treated asynchronously, which makes it much faster to push new measures. Getting new numbers on that feature was pretty interesting. The number of metric per second you can push depends on the batch size, meaning the number of actual measurements you send per call. The naive approach is to push 1 measure per call, and in that case, Gnocchi is able to handle around 600 measures/s. With a batch containing 100 measures, the number of calls per second goes down to 450, but since you push 100 measures each time, that means 45k measures per second pushed into Gnocchi! I've pushed the test further, inspired by the recent blog post of InfluxDB claiming to achieve 300k points per second with their new engine. I ran the same benchmark on the hardware I had, which is roughly two times smaller than the one they used. I achieved to push Gnocchi to a little more than 120k measurement per second. If I had same hardware as they used, I could interpolate the results to achieve almost 250k measures/s pushed. Obviously, you can't strictly compare Gnocchi and InfluxDB since they are not doing exactly the same thing, but it still looks way better than what I expected. Using smaller batch sizes of 1k or 2k improve the throughput further to around 125k measures/s.
Operation Details Rate
Push metric 5k Push 5M measures with batch of 5k measures in 40 seconds 122k measures/s
Push metric 4k Push 5M measures with batch of 4k measures in 40 seconds 125k measures/s
Push metric 3k Push 5M measures with batch of 3k measures in 40 seconds 123k measures/s
Push metric 2k Push 5M measures with batch of 2k measures in 41 seconds 121k measures/s
Push metric 1k Push 5M measures with batch of 1k measures in 44 seconds 113k measures/s
Push metric 500 Push 5M measures with batch of 500 measures in 51 seconds 98k measures/s
Push metric 100 Push 5M measures with batch of 100 measures in 112 seconds 45k measures/s
Push metric 10 Push 5M measures with batch of 10 measures in 852 seconds 6k measures/s
Push metric 1 Push 500k measures with batch of 1 measure in 800 seconds 624 measures/s
Get measures Push 43k measures of 1 metric 260k measures/s
What about getting measures? Well, it's actually pretty fast too. Retrieving a metric with 1 month of data with 1 minute interval (that's 43k points) takes less than 2 second. Though it's actually slower than what I expected. The reason seems to be that the JSON is 2 MB big and encoding it takes a lot of time for Python. I'll investigate that. Another point I discovered, is that by default Gnocchi returns all the datapoints for each granularities available for the asked period, which might double the size of the returned data for nothing if you don't need it. It'll be easy to add an option to the API to only retrieve what you need though! Once benchmarked, that meant I was able to retrieve 6 metric/s per second, which translates to around 260k measures/s. Metricd speed New measures that are pushed into Gnocchi are processed asynchronously by the gnocchi-metricd daemon. When doing the benchmarks above, I ran into a very interesting issue: sending 10k measures on a metric would make gnocchi-metricd uses up to 2 GB RAM and 120 % CPU for more than 10 minutes. After further investigation, I found that the naive approach we used to resample datapoints in Carbonara using Pandas was causing that. I reported a bug on Pandas and the upstream author was kind enough to provide a nice workaround, that I sent as a pull request to Pandas documentation. I wrote a fix for Gnocchi based on that, and started using it. Computing the standard aggregation methods set (std, count, 95pct, min, max, sum, median, mean) for 10k batches of 1 measure (worst case scenario) for one metric with 10k measures now takes only 20 seconds and uses 100 MB of RAM 45 faster. That means that in normal operations, where only a few new measures are processed, the operation of updating a metric only takes a few milliseconds. Awesome! Comparison with Ceilometer For comparison sake, I've quickly run some read operations benchmark in Ceilometer. I've fed it with one month of samples for 100 instances polled every minute. That represents roughly 4.3M samples injected, and that took a while almost 1 hour whereas it would have taken less than a minute in Gnocchi. Then I tried to retrieve some statistics in the same way that we provide them in Gnocchi, which mean aggregating them over a period of 60 seconds over a month.
Operation Details Rate
Read metric SQL Read measures for 1 metric 2min 58s
Read metric MongoDB Read measures for 1 metric 28s
Read metric Gnocchi Read measures for 1 metric 2s
Obviously, Ceilometer is very slow. It has to look into 4M of samples to compute and return the result, which takes a lot of time. Whereas Gnocchi just has to fetch a file and pass it over. That also means that the more samples you have (so the more time you collect data and the more resources you have), slower Ceilometer will become. This is not a problem with Gnocchi, as I emphasized when I started designing it. Most Gnocchi operations are O(log R) where R is the number of metrics or resources, whereas most Ceilometer operations are O(log S) where S is the number of samples (measures). Since is R millions of time smaller than S, Gnocchi gets to be much faster. And what's even more interesting, is that Gnocchi is entirely scalable horizontally. Adding more Gnocchi servers (for the API and its background processing worker metricd) will multiply Gnocchi performances by the number of servers added. Improvements There are several things to improve in Gnocchi, such as splitting Carbonara archives to make them more efficient, especially from drivers such as Ceph and Swift. It's already on my plate, and I'm looking forwarding to working on that! And if you have any questions, feel free to shoot them in the comment section.

9 October 2015

Miriam Ruiz: Thick Skin (within Free/Open Source communities)

The definition of thick-skinned in different dictionaries ranges from not easily offended to largely unaffected by the needs and feelings of other people; insensitive , going through able to ignore personal criticism , ability to withstand criticism and show no signs of any criticism you may receive getting to you , an insensitive nature or impervious to criticism . It essentially describes an emotionally detached attitude regarding one s social environment, the capacity or ignoring or minimizing the effects of others criticism and the priorization of the protection of one s current state over the capacity of empathizing and taking into account what others may say that don t conform to one s current way of thinking. It is essentially setting up barriers against whatever others may do that might provoke any kind of crisis or change in you. There are a few underlying assumptions in the use of this term as a something good to have, when it comes to interactions with your own community: In the first place, it assumes that your own community is essentially hostile to you, and you will have to be constantly in guard against them. It assumes that it is better to set up barriers against the influence of others within your own comunity, because in fact your own peers are out there essentially to hurt you. Or, at least, they do not care a damn about you. In second place, it assumes that changes are wrong, that personal evolution is wrong, and that the more insensitive you are to your peer s opinions, the best, because they really have nothing to contribute to help you grow as a person. I m smart; you re dumb. I m big; you re little. I m right; you re wrong; And there s nothing you can do about it. (from the film Matilda). Matilda s dad is in fact the first reference that comes to my mind when we re talking about really thick skin. Scene of the film Matilda: I'm smart, You're dumb. I'm big, You're little. I'm right, You're wrong When the main recommendation when the level of aggressiveness within a community is that someone has to make their skin thicker, they are assuming that a bullying environment will help the results. This is nothing new. It s the same theoretical base that you can see in hazing and in other activities involving harassment, abuse or humiliation in college, when initiating a person into some groups. It s supposed to build character, to make someone closer to the alpha male stereotype and, in essence, make us better men (yes, I am using the word men on purpose, because insensitiveness is not usually seen as a positive trait in females). The assumption is that a community with a hard environment and individuals prepared for the war is more effective than a more civilized one. Luckily, that s not the point of view of most members of the Debian Community, and many other Free/Open Source projects. The Code of Conduct is very explicit when it says that a community in which people feel threatened is not a healthy community , and that is good. The Debian Project welcomes and encourages participation by everyone (Diversity Statement), including those with a thin skin, and I m happy about that. There are still a lot of things to improve, of course, but I have the feeling that -despite the occasional complains that having to be respectful to others take the fun away- we re moving in the right direction. The best tip I can give you on thickening your skin = don t. That is, don t thicken your skin. Having a thin skin means you re letting the world in, you re letting what s out there affect what s in you. It means you re connected. You re open. You re considerate and you ll consider it whatever it might be. Having a thin skin may be dangerous, sure, because you might take in so much that you pop, like that blueberry girl from Willy Wonka. But life is dangerous. A thick skin protects you from everything, but it also protects you from everything from the gentle touches of life, from the subtle emotions of others, the deep connections, the meaningful interactions. (Top Ten Tips on how to Thicken your Skin).

8 October 2015

Norbert Preining: Looking at the facts: Sarah Sharp s crusade

Much has been written around the internet about this geeky kernel maintainer Sarah Sharp who left kernel development. I have now spent two hours reading through lkml posts, and want to summarize a few mails from the long thread, since most of the usual news sites just rewrap the original blog of hers without adding any background. darth-cookie The whole thread evolved out call for stable kernel review by Greg Kroah-Hartman where he complained about too many patches that are not actually in rc1 before going into stable: 
<rant>
  I'm sitting on top of over 170 more patches that have been marked for
  the stable releases right now that are not included in this set of
  releases.  The fact that there are this many patches for stable stuff
  that are waiting to be merged through the main -rc1 merge window cycle
  is worrying to me.
  [...]
from where it developed into a typical Linus rant on people flagging crap for stable, followed by some jokes: 
On Fri, Jul 12, 2013 at 8:47 AM, Steven Rostedt <rostedt@goodmis.org> wrote:
> I tend to hold things off after -rc4 because you scare me more than Greg
> does ;-)
 
Have you guys *seen* Greg? The guy is a freakish giant. He *should*
scare you. He might squish you without ever even noticing.
 
              Linus
and Ingo Molnar giving advice to Greg KH: 
So Greg, if you want it all to change, create some _real_ threat: be frank 
with contributors and sometimes swear a bit. That will cut your mailqueue 
in half, promise!
with Greg KH taking a funny position in answering: 
Ok, I'll channel my "inner Linus" and take a cue from my kids and start
swearing more.
Up to now a pretty decent and normal thread with some jokes and poking, nobody minded, and reading through it I had a good time. The thread continues with a discussion on requirements what to submit to stable, and some side threads on particular commits. And then, out of the blue, Social Justice Warrior (SJW) Sarah Sharp pops in with a very important contribution: 
Seriously, guys?  Is this what we need in order to get improve -stable?
Linus Torvalds is advocating for physical intimidation and violence.
Ingo Molnar and Linus are advocating for verbal abuse.
 
Not *fucking* cool.  Violence, whether it be physical intimidation,
verbal threats or verbal abuse is not acceptable.  Keep it professional
on the mailing lists.
 
Let's discuss this at Kernel Summit where we can at least yell at each
other in person.  Yeah, just try yelling at me about this.  I'll roar
right back, louder, for all the people who lose their voice when they
get yelled at by top maintainers.  I won't be the nice girl anymore.
Onto which Linus answers in a great way: 
That's the spirit.
 
Greg has taught you well. You have controlled your fear. Now, release
your anger. Only your hatred can destroy me.
 
Come to the dark side, Sarah. We have cookies.
On goes Sarah, gearing up in her SJW mode and starting to rant: 
However, I am serious about this.  Linus, you're one of the worst
offenders when it comes to verbally abusing people and publicly tearing
their emotions apart.
 
http://marc.info/?l=linux-kernel&m=135628421403144&w=2
http://marc.info/?l=linux-acpi&m=136157944603147&w=2
 
I'm not going to put up with that shit any more.
Linus himself made clear what he thinks of her: 
Trust me, there's a really easy way for me to curse at people: if you
are a maintainer, and you make excuses for your bugs rather than
trying to fix them, I will curse at *YOU*.
 
Because then the problem really is you.
[...]
It is easy to verify what Linus said, by reading the above two links and the answers of the maintainers, both agreed that it was their failure and were sorry. (Mauro s answer, Rafael s answer) It is just the geeky SJW that was not even attacked (who would dare to attack a woman nowadays?). The overall reaction to her by the maintainers can be exemplified by Thomas Gleixner s post: 
Just for the record. I got grilled by Linus several times over the
last years and I can't remember a single instance where it was
unjustified.
What follows is a nearly endless discussion with Sarah meandering around, changing permanently her opinion what is acceptable. Linus tried to explain to her in simple words, without success, she continues to rant around. Here arguments are so weak I had nothing but good laugh: 
> Sarah, that's a pretty potent argument by Linus, that "acting 
> professionally" risks replacing a raw but honest culture with a
> polished but dishonest culture - which is harmful to developing
> good technology.
> 
> That's a valid concern. What's your reply to that argument?
 
I don't feel the need to comment, because I feel it's a straw man
argument.  I feel that way because I disagree with the definition of
professionalism that people have been pushing.
 
To me, being "professional" means treating each other with respect.  I
can show emotion, express displeasure, be direct, and still show respect
for my fellow developers.
 
For example, I find the following statement to be both direct and
respectful, because it's criticizing code, not the person:
 
"This code is SHIT!  It adds new warnings and it's marked for stable
when it's clearly *crap code* that's not a bug fix.  I'm going to revert
this merge, and I expect a fix from you IMMEDIATELY."
 
The following statement is not respectful, because it targets the
person:
 
"Seriously, Maintainer.  Why are you pushing this kind of *crap* code to
me again?  Why the hell did you mark it for stable when it's clearly
not a bug fix?  Did you even try to f*cking compile this?"
Fortunately, she was immediately corrected and Ingo Molnar wrote an excellent refutation (starting another funny thread) of all her emails, statements, accusations (all of the email is a good read): 
_That_ is why it might look to you as if the person was
attacked, because indeed the actions of the top level maintainer were
wrong and are criticised.
 
... and now you want to 'shut down' the discussion. With all due respect,
you started it, you have put out various heavy accusations here and elsewhere,
so you might as well take responsibility for it and let the discussion be
brought to a conclusion, wherever that may take us, compared to your initial view?
(He retracted that last statement, though I don t see a reason for it) Last but not least, let us return to her blog post, where she states herself that: 
FYI, comments will be moderated by someone other than me. As this is my blog, not a
government entity, I have the right to replace any comment I feel like with 
 fart fart fart fart .
and she made lots of use of it, I counted at least 10 instances. She seems to remove or fart fart fart any comment that is not in line with her opinion. Further evidence is provided by this post on lkml. Everyone is free to have his own opinion (sorry, his/her), and I am free to form my own opinion on Sarah Sharp by just simply reading the facts. I am more than happy that one more SJW has left Linux development, as the proliferation of cleaning of speech from any personality has taken too far a grip. Coming to my home-base in Debian, unfortunately there is no one in the position and the state of mind of Linus, so we are suffering the same stupidities imposed by social justice worriers and some brainless feminists (no, don t get me wrong, these are two independent attributes. I do NOT state that feminism is brainless) that Linus and the maintainer crew was able to fend of this time. I finish with my favorite post from that thread, by Steven Rosted (from whom I also stole the above image!): 
On Tue, 2013-07-16 at 18:37 -0700, Linus Torvalds wrote:
> Emotions aren't bad. Quite the reverse. 
 
Spock and Dr. Sheldon Cooper strongly disagree.
Post Scriptum (after a bike ride) The last point by Linus is what I criticize most on Debian nowdays, it has become a sterilized over-governed entity, where most fun is gone. Making fun is practically forbidden, since there is the slight chance that some minority somewhere on this planet might feel hurt, and by this we are breaking the CoC. Emotions are restricted to the Happy happy how nice we are and how good we are level of US and also Japanese self-reenforcement society. Post Post Scriptum I just read Sarah Sharp s post on What makes a good community? , and without giving a full account or review, I am just pi**ed by the usage of the word microaggressions I can only recommend everyone to read this article and this article to get a feeling how bs the idea of microaggressions has taken over academia and obviously not only academia. Post3 Scriptum I am happy to see Lars Wirzenius, Gunnar Wolf, and Mart n Ferrari opposing my view. I agree with them that my comments concerning Debian are not mainstream in Debian something that is not very surprising, though, and I think it is great that they have fun in Debian, like many other contributors. Post4 Scriptum Although nobody will read this, here is a great response from a female developer: 
[...] To Linus: You're a hero to many of us. Don't change. Please. You DO
NOT need to take time away from doing code to grow a pair of breasts
and judge people's emotional states: [...]
Nothing to add here!

7 October 2015

Emanuele Rocca: systemd is your friend

Today I want to talk a bit about some cool features of systemd, the default Debian init system since the release of Jessie. Ubuntu has also adopted systemd in 15.04, meaning that you are going to find it literally everywhere.
Logging The component responsible for logging in systemd is called journal. It collects and stores logs in a structured, indexed journal (hence the name). The journal can replace traditional syslog daemons such as rsyslog and syslog-ng, or work together with them. By default Debian keeps on using rsyslog, but if you don't need to ship logs to a centralized server (or do other fancy things) it is possible to stop using rsyslog right now and rely on systemd-journal instead. The obvious question is: why would anybody use a binary format for logs instead of a bunch of tried and true plain-text files? As it turns out, there are quite a lot of good reasons to do so. The killer features of systemd-journald for me are:
  • Index tons of logs while being able to search with good performance: O(log(n)) instead of O(n) which is what you get with text files
  • No need to worry about log rotation anymore, in any shape or form
The last point in particular is really critical in my opinion. Traditional log rotation implementations rely on cron jobs to check how much disk space is used by logs, compressing/removing old files. Log rotation is usually: 1) annoying to configure; 2) hard to get right; 3) prone to DoS attacks. With journald, there is pretty much nothing to configure. Log rotation is built into the daemon disk space allocation logic itself. This also allows to avoid vulnerability windows due to time-based rotation, which is what you get with logrotate and friends. Enough high-level discussions though, here is how to use the journal! Check if you already have the directory /var/log/journal, otherwise create it (as root). Then restart systemd-journald as follows: sudo systemctl restart systemd-journald You can get all messages produced since the last boot with journalctl -b. All messages produced today can get extracted using journalctl --since=today. Want to get all logs related to ssh? Try with journalctl _SYSTEMD_UNIT=ssh.service. There are many more filtering options available, you can read all about them with man journalctl. journald's configuration file is /etc/systemd/journald.conf. Two of the most interesting options are SystemMaxUse and SystemKeepFree, which can be used to change the amount of disk space dedicated to logging. They default to 10% and 15% of the /var filesystem respectively. Here is a little cheatsheet: 
journalctl -b                # Show all messages since last boot
journalctl -f                # Tail your logs
journalctl --since=yesterday # Show all messages produced since yesterday
journalctl -pcrit            # Filter messages by priority
journalctl /bin/su           # Filter messages by program
journalctl --disk-usage      # The amount of space in use for journaling
Further reading:
Containers A relatively little known component of systemd is systemd-nspawn. It is a small, straightforward container manager. If you don't already have a chroot somewhere, here is how to create a basic Debian Jessie chroot under /srv/chroots/jessie: 
$ debootstrap jessie /srv/chroots/jessie http://http.debian.net/debian/
With systemd-nspawn you can easily run a shell inside the chroot: 
$ sudo systemd-nspawn -D /srv/chroots/jessie
Spawning container jessie on /srv/chroots/jessie.
Press ^] three times within 1s to kill container.
/etc/localtime is not a symlink, not updating container timezone.
root@jessie:~#
Done. Everything works out of the box: no need for you to mount /dev, /run and friends, systemd-nspawn took care of that. Networking also works. If you want to actually boot the system, just add the -b switch to the previous command: 
$ sudo systemd-nspawn -b -D /srv/chroots/jessie
Spawning container jessie on /srv/chroots/jessie.
Press ^] three times within 1s to kill container.
/etc/localtime is not a symlink, not updating container timezone.
systemd 215 running in system mode. (+PAM +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP +GCRYPT +ACL +XZ -SECCOMP -APPARMOR)
Detected virtualization 'systemd-nspawn'.
Detected architecture 'x86-64'.
Welcome to Debian GNU/Linux jessie/sid!
Set hostname to <orion>.
[  OK  ] Reached target Remote File Systems (Pre).
[  OK  ] Reached target Encrypted Volumes.
[  OK  ] Reached target Paths.
[  OK  ] Reached target Swap.
[  OK  ] Created slice Root Slice.
[  OK  ] Created slice User and Session Slice.
[  OK  ] Listening on /dev/initctl Compatibility Named Pipe.
[  OK  ] Listening on Delayed Shutdown Socket.
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Listening on Journal Socket.
[  OK  ] Created slice System Slice.
[  OK  ] Created slice system-getty.slice.
[  OK  ] Listening on Syslog Socket.
         Mounting POSIX Message Queue File System...
         Mounting Huge Pages File System...
         Mounting FUSE Control File System...
         Starting Copy rules generated while the root was ro...
         Starting Journal Service...
[  OK  ] Started Journal Service.
[  OK  ] Reached target Slices.
         Starting Remount Root and Kernel File Systems...
[  OK  ] Mounted Huge Pages File System.
[  OK  ] Mounted POSIX Message Queue File System.
[  OK  ] Mounted FUSE Control File System.
[  OK  ] Started Copy rules generated while the root was ro.
[  OK  ] Started Remount Root and Kernel File Systems.
         Starting Load/Save Random Seed...
[  OK  ] Reached target Local File Systems (Pre).
[  OK  ] Reached target Local File Systems.
         Starting Create Volatile Files and Directories...
[  OK  ] Reached target Remote File Systems.
         Starting Trigger Flushing of Journal to Persistent Storage...
[  OK  ] Started Load/Save Random Seed.
         Starting LSB: Raise network interfaces....
[  OK  ] Started Create Volatile Files and Directories.
         Starting Update UTMP about System Boot/Shutdown...
[  OK  ] Started Trigger Flushing of Journal to Persistent Storage.
[  OK  ] Started Update UTMP about System Boot/Shutdown.
[  OK  ] Started LSB: Raise network interfaces..
[  OK  ] Reached target Network.
[  OK  ] Reached target Network is Online.
[  OK  ] Reached target System Initialization.
[  OK  ] Listening on D-Bus System Message Bus Socket.
[  OK  ] Reached target Sockets.
[  OK  ] Reached target Timers.
[  OK  ] Reached target Basic System.
         Starting /etc/rc.local Compatibility...
         Starting Login Service...
         Starting LSB: Regular background program processing daemon...
         Starting D-Bus System Message Bus...
[  OK  ] Started D-Bus System Message Bus.
         Starting System Logging Service...
[  OK  ] Started System Logging Service.
         Starting Permit User Sessions...
[  OK  ] Started /etc/rc.local Compatibility.
[  OK  ] Started LSB: Regular background program processing daemon.
         Starting Cleanup of Temporary Directories...
[  OK  ] Started Permit User Sessions.
         Starting Console Getty...
[  OK  ] Started Console Getty.
[  OK  ] Reached target Login Prompts.
[  OK  ] Started Login Service.
[  OK  ] Reached target Multi-User System.
[  OK  ] Reached target Graphical Interface.
         Starting Update UTMP about System Runlevel Changes...
[  OK  ] Started Cleanup of Temporary Directories.
[  OK  ] Started Update UTMP about System Runlevel Changes.
Debian GNU/Linux jessie/sid orion console
orion login:
That's it! Just one command to start a shell in your chroot or boot the container, again zero configuration needed. Finally, systemd provides a command called machinectl that allows you to introspect and control your container: 
$ sudo machinectl status jessie
jessie
           Since: Wed 2015-10-07 11:22:56 CEST; 55min ago
          Leader: 32468 (systemd)
         Service: nspawn; class container
            Root: /srv/chroots/jessie
         Address: fe80::8e70:5aff:fe81:2290
                  192.168.122.1
                  192.168.1.13
              OS: Debian GNU/Linux jessie/sid
            Unit: machine-jessie.scope
                   32468 /lib/systemd/systemd
                   system.slice
                     dbus.service
                       32534 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile -...
                     cron.service
                       32539 /usr/sbin/cron
                     systemd-journald.service
                       32487 /lib/systemd/systemd-journald
                     systemd-logind.service
                       32532 /lib/systemd/systemd-logind
                     console-getty.service
                       32544 /sbin/agetty --noclear --keep-baud console 115200 38400 9600 vt102
                     rsyslog.service
                       32540 /usr/sbin/rsyslogd -n
With machinectl you can also reboot, poweroff, terminate your containers and more. There are so many things to learn about systemd and containers! Here are some references. This stuff is pretty exciting. Now that all major distributions use systemd by default, we can expect to have access to tools like journalctl and systemd-nspawn everywhere!

6 October 2015

John Goerzen: Objects On Earth Are Closer Than They Appear

We all live beneath the great Big Dipper. So goes a line in a song I once heard the great Tony Brown sing. As I near the completion of my private pilot s training, I ve had more and more opportunities to literally see the wisdom in those words. Here s a story of one of them. Night
A shining beacon in space all alone in the night. Babylon 5
A night cross-country flight, my first, taking off from a country airport. The plane lifts into the dark sky. The bright white lights of the runway get smaller, and disappear as I pass the edge of the airport. Directly below me, it looks like a dark sky; pitch black except for little pinpoints of light at farmhouses and the occasional car. But seconds later, an expanse of light unfolds, from a city it takes nearly an hour to reach by car. Already it is in sight, and as I look off to other directions, other cities even farther away are visible, too. The ground shows a square grid, the streets of the city visible for miles. There are no highway signs in the sky. There are no wheels to keep my plane pointed straight. Even if I point the plane due south, if there is an east wind, I will actually be flying southwest. I use my eyes, enhanced by technology like a compass, GPS, and VHF radio beacons, to find my way. Before ever getting into the airplane, I have carefully planned my route, selecting both visual and technological waypoints along the way to provide many ways to ensure I am on course and make sure I don t get lost. Soon I see a flash repeating every few seconds in the distance an airport beacon. Then another, and another. Little pinpoints of light nestled in the square orange grid. Wichita has many airports, each with its beacon, and one of them will be my first visual checkpoint of the night. I make a few clicks in the cockpit, and soon the radio-controlled lights at one of the airports spring to life, illuminating my first checkpoint. More than a mile of white lights there to welcome any plane that lands, and to show a point on the path of any plane that passes. I continue my flight, sometimes turning on lights at airports, other times pointing my plane at lights from antenna towers (that are thousands of feet below me), sometimes keeping a tiny needle on my panel centered on a radio beacon. I land at a tiny, deserted airport, and then a few minutes later at a large commercial airport. On my way back home, I fly solely by reference to the ground directly over a freeway. I have other tools at my disposal, but don t need them; the steady stream of red and white lights beneath me are all I need. From my plane, there is just red and white. One after another, passing beneath me as I fly over them at 115 MPH. There is no citizen or undocumented immigrant, no rich or poor, no atheist or Christian or Muslim, no Democrat or Replubican, no American or Mexican, no adult or child, no rich or poor, no Porsche or Kia. Just red and white points of light, each one the same as the one before and the one after, stretching as far as I can see into the distance. All alike in the night. You only need to get a hundred feet off the ground before you realize how little state lines, national borders, and the machinery of politics and exclusion really mean. From the sky, the difference between a field of corn and a field of wheat is far more significant than the difference between Kansas and Missouri. This should be a comforting reminder to us. We are all unique, and beautiful in our uniqueness, but we are all human, each as valuable as the next. Up in the sky, even though my instructor was with me, during quiet times it is easy to feel all alone in the night. But I know it is not the case. Only a few thousand feet separate my plane from those cars. My plane, too, has red and white lights.
How often at night, when the heavens were bright,
With the light of the twinkling stars
Have I stood here amazed, and asked as I gazed,
If their glory exceed that of ours. John A. Lomax

Thorsten Alteholz: My Debian Activities in September 2015

FTP assistant Another month passed and another statistic arrives: This month I marked 341 packages for accept and rejected only 48 of them. Almost like last month I had to send 14 emails to maintainers. Squeeze LTS This was my fifteenth month that I did some work for the Squeeze LTS initiative, started by Raphael Hertzog at Freexian. This month I only got a workload of 14.5h. I finally uploaded a new version of php5. Unfortunately in one library a parameter to a function call introduced new values. As a result all running processes that used the old version of that library produced an error message until they got restarted. As complaints showed up on all channels, I rechecked my patches again and again but could not find an error. I wonder whether this happened once before. At least the php package does not have a mechanism to restart something
Altogether I uploaded those DLAs: I also started to work on an upload of freeimage and the next upload of php5. This month I also had another term of doing frontdesk work. So I answered questions on the IRC channel and looked for CVEs that are important for Squeeze LTS or could be ignored. Other stuff Some time ago someone mentioned pump.io and that it would be nice to have it in Debian. I found a Wiki page listing dependencies, with lots of stuff already done and just a few holes. It didn t look like much work todo until I realized that this page showed only the surface and the shoals are hidden below. Anyway, I started to work on it and up to now are uploaded and are still in NEW. Luckily most of them could be handled by npm2deb, so it was mainly routine piece of work. So, expect more to come I also polished some smaller packages and could even close some bugs:

5 October 2015

Julien Danjou: Gnocchi talk at OpenStack Paris Meetup #16

Last week, I've been invited to the OpenStack Paris meetup #16, whose subject was about metrics in OpenStack. Last time I spoke at this meetup was back in 2012, during the OpenStack Paris meetup #2. A very long time ago!
I talked for half an hour about Gnocchi, the OpenStack project I've been running for 18 months now. I started by explaining the story behind the project and why we needed to build it. Ceilometer has an interesting history and had a curious roadmap these last year, and I summarized that briefly. Then I talk about how Gnocchi works and what it offers to users and operators. The slides where full of JSON, but I imagine it offered a interesting view of what the API looks like and how easy it is to operate. This also allowed me to emphasize how many use cases are actually really covered and solved, contrary to what Ceilometer did so far. The talk has been well received and I got a few interesting questions at the end. The video of the talk (in French) and my slides are available on my talk page and below. I hope you'll enjoy it.

2 October 2015

Daniel Pocock: Want to be selected for Google Summer of Code 2016?

I've mentored a number of students in 2013, 2014 and 2015 for Debian and Ganglia and most of the companies I've worked with have run internships and graduate programs from time to time. GSoC 2015 has just finished and with all the excitement, many students are already asking what they can do to prepare and be selected for Outreachy or GSoC in 2016. My own observation is that the more time the organization has to get to know the student, the more confident they can be selecting that student. Furthermore, the more time that the student has spent getting to know the free software community, the more easily they can complete GSoC. Here I present a list of things that students can do to maximize their chance of selection and career opportunities at the same time. These tips are useful for people applying for GSoC itself and related programs such as GNOME's Outreachy or graduate placements in companies. Disclaimers There is no guarantee that Google will run the program again in 2016 or any future year until the Google announcement. There is no guarantee that any organization or mentor (including myself) will be involved until the official list of organizations is published by Google. Do not follow the advice of web sites that invite you to send pizza or anything else of value to prospective mentors. Following the steps in this page doesn't guarantee selection. That said, people who do follow these steps are much more likely to be considered and interviewed than somebody who hasn't done any of the things in this list. Understand what free software really is You may hear terms like free software and open source software used interchangeably. They don't mean exactly the same thing and many people use the term free software for the wrong things. Not all projects declaring themselves to be "free" or "open source" meet the definition of free software. Those that don't, usually as a result of deficiencies in their licenses, are fundamentally incompatible with the majority of software that does use genuinely free licenses. Google Summer of Code is about both writing and publishing your code and it is also about community. It is fundamental that you know the basics of licensing and how to choose a free license that empowers the community to collaborate on your code well after GSoC has finished. Please review the definition of free software early on and come back and review it from time to time. The The GNU Project / Free Software Foundation have excellent resources to help you understand what a free software license is and how it works to maximize community collaboration. Don't look for shortcuts There is no shortcut to GSoC selection and there is no shortcut to GSoC completion. The student stipend (USD $5,500 in 2014) is not paid to students unless they complete a minimum amount of valid code. This means that even if a student did find some shortcut to selection, it is unlikely they would be paid without completing meaningful work. If you are the right candidate for GSoC, you will not need a shortcut anyway. Are you the sort of person who can't leave a coding problem until you really feel it is fixed, even if you keep going all night? Have you ever woken up in the night with a dream about writing code still in your head? Do you become irritated by tedious or repetitive tasks and often think of ways to write code to eliminate such tasks? Does your family get cross with you because you take your laptop to Christmas dinner or some other significant occasion and start coding? If some of these statements summarize the way you think or feel you are probably a natural fit for GSoC. An opportunity money can't buy The GSoC stipend will not make you rich. It is intended to make sure you have enough money to survive through the summer and focus on your project. Professional developers make this much money in a week in leading business centers like New York, London and Singapore. When you get to that stage in 3-5 years, you will not even be thinking about exactly how much you made during internships. GSoC gives you an edge over other internships because it involves publicly promoting your work. Many companies still try to hide the potential of their best recruits for fear they will be poached or that they will be able to demand higher salaries. Everything you complete in GSoC is intended to be published and you get full credit for it. Imagine a young musician getting the opportunity to perform on the main stage at a rock festival. This is how the free software community works. It is a meritocracy and there is nobody to hold you back. Having a portfolio of free software that you have created or collaborated on and a wide network of professional contacts that you develop before, during and after GSoC will continue to pay you back for years to come. While other graduates are being screened through group interviews and testing days run by employers, people with a track record in a free software project often find they go straight to the final interview round. Register your domain name and make a permanent email address Free software is all about community and collaboration. Register your own domain name as this will become a focal point for your work and for people to get to know you as you become part of the community. This is sound advice for anybody working in IT, not just programmers. It gives the impression that you are confident and have a long term interest in a technology career. Choosing the provider: as a minimum, you want a provider that offers DNS management, static web site hosting, email forwarding and XMPP services all linked to your domain. You do not need to choose the provider that is linked to your internet connection at home and that is often not the best choice anyway. The XMPP foundation maintains a list of providers known to support XMPP. Create an email address within your domain name. The most basic domain hosting providers will let you forward the email address to a webmail or university email account of your choice. Configure your webmail to send replies using your personalized email address in the From header. Update your ~/.gitconfig file to use your personalized email address in your Git commits. Create a web site and blog Start writing a blog. Host it using your domain name. Some people blog every day, other people just blog once every two or three months. Create links from your web site to your other profiles, such as a Github profile page. This helps reinforce the pages/profiles that are genuinely related to you and avoid confusion with the pages of other developers. Many mentors are keen to see their students writing a weekly report on a blog during GSoC so starting a blog now gives you a head start. Mentors look at blogs during the selection process to try and gain insight into which topics a student is most suitable for. Create a profile on Github Github is one of the most widely used software development web sites. Github makes it quick and easy for you to publish your work and collaborate on the work of other people. Create an account today and get in the habbit of forking other projects, improving them, committing your changes and pushing the work back into your Github account. Github will quickly build a profile of your commits and this allows mentors to see and understand your interests and your strengths. In your Github profile, add a link to your web site/blog and make sure the email address you are using for Git commits (in the ~/.gitconfig file) is based on your personal domain. Start using PGP Pretty Good Privacy (PGP) is the industry standard in protecting your identity online. All serious free software projects use PGP to sign tags in Git, to sign official emails and to sign official release files. The most common way to start using PGP is with the GnuPG (GNU Privacy Guard) utility. It is installed by the package manager on most Linux systems. When you create your own PGP key, use the email address involving your domain name. This is the most permanent and stable solution. Print your key fingerprint using the gpg-key2ps command, it is in the signing-party package on most Linux systems. Keep copies of the fingerprint slips with you. This is what my own PGP fingerprint slip looks like. You can also print the key fingerprint on a business card for a more professional look. Using PGP, it is recommend that you sign any important messages you send but you do not have to encrypt the messages you send, especially if some of the people you send messages to (like family and friends) do not yet have the PGP software to decrypt them. If using the Thunderbird (Icedove) email client from Mozilla, you can easily send signed messages and validate the messages you receive using the Enigmail plugin. Get your PGP key signed Once you have a PGP key, you will need to find other developers to sign it. For people I mentor personally in GSoC, I'm keen to see that you try and find another Debian Developer in your area to sign your key as early as possible. Free software events Try and find all the free software events in your area in the months between now and the end of the next Google Summer of Code season. Aim to attend at least two of them before GSoC. Look closely at the schedules and find out about the individual speakers, the companies and the free software projects that are participating. For events that span more than one day, find out about the dinners, pub nights and other social parts of the event. Try and identify people who will attend the event who have been GSoC mentors or who intend to be. Contact them before the event, if you are keen to work on something in their domain they may be able to make time to discuss it with you in person. Take your PGP fingerprint slips. Even if you don't participate in a formal key-signing party at the event, you will still find some developers to sign your PGP key individually. You must take a photo ID document (such as your passport) for the other developer to check the name on your fingerprint but you do not give them a copy of the ID document. Events come in all shapes and sizes. FOSDEM is an example of one of the bigger events in Europe, linux.conf.au is a similarly large event in Australia. There are many, many more local events such as the Debian UK mini-DebConf in Cambridge, November 2015. Many events are either free or free for students but please check carefully if there is a requirement to register before attending. On your blog, discuss which events you are attending and which sessions interest you. Write a blog during or after the event too, including photos. Quantcast generously hosted the Ganglia community meeting in San Francisco, October 2013. We had a wild time in their offices with mini-scooters, burgers, beers and the Ganglia book. That's me on the pink mini-scooter and Bernard Li, one of the other Ganglia GSoC 2014 admins is on the right. Install Linux GSoC is fundamentally about free software. Linux is to free software what a tree is to the forest. Using Linux every day on your personal computer dramatically increases your ability to interact with the free software community and increases the number of potential GSoC projects that you can participate in. This is not to say that people using Mac OS or Windows are unwelcome. I have worked with some great developers who were not Linux users. Linux gives you an edge though and the best time to gain that edge is now, while you are a student and well before you apply for GSoC. If you must run Windows for some applications used in your course, it will run just fine in a virtual machine using Virtual Box, a free software solution for desktop virtualization. Use Linux as the primary operating system. Here are links to download ISO DVD (and CD) images for some of the main Linux distributions: If you are nervous about getting started with Linux, install it on a spare PC or in a virtual machine before you install it on your main PC or laptop. Linux is much less demanding on the hardware than Windows so you can easily run it on a machine that is 5-10 years old. Having just 4GB of RAM and 20GB of hard disk is usually more than enough for a basic graphical desktop environment although having better hardware makes it faster. Your experiences installing and running Linux, especially if it requires some special effort to make it work with some of your hardware, make interesting topics for your blog. Decide which technologies you know best Personally, I have mentored students working with C, C++, Java, Python and JavaScript/HTML5. In a GSoC program, you will typically do most of your work in just one of these languages. From the outset, decide which language you will focus on and do everything you can to improve your competence with that language. For example, if you have already used Java in most of your course, plan on using Java in GSoC and make sure you read Effective Java (2nd Edition) by Joshua Bloch. Decide which themes appeal to you Find a topic that has long-term appeal for you. Maybe the topic relates to your course or maybe you already know what type of company you would like to work in. Here is a list of some topics and some of the relevant software projects:
  • System administration, servers and networking: consider projects involving monitoring, automation, packaging. Ganglia is a great community to get involved with and you will encounter the Ganglia software in many large companies and academic/research networks. Contributing to a Linux distribution like Debian or Fedora packaging is another great way to get into system administration.
  • Desktop and user interface: consider projects involving window managers and desktop tools or adding to the user interface of just about any other software.
  • Big data and data science: this can apply to just about any other theme. For example, data science techniques are frequently used now to improve system administration.
  • Business and accounting: consider accounting, CRM and ERP software.
  • Finance and trading: consider projects like R, market data software like OpenMAMA and connectivity software (Apache Camel)
  • Real-time communication (RTC), VoIP, webcam and chat: look at the JSCommunicator or the Jitsi project
  • Web (JavaScript, HTML5): look at the JSCommunicator
Before the GSoC application process begins, you should aim to learn as much as possible about the theme you prefer and also gain practical experience using the software relating to that theme. For example, if you are attracted to the business and accounting theme, install the PostBooks suite and get to know it. Maybe you know somebody who runs a small business: help them to upgrade to PostBooks and use it to prepare some reports. Make something Make some small project, less than two week's work, to demonstrate your skills. It is important to make something that somebody will use for a practical purpose, this will help you gain experience communicating with other users through Github. For an example, see the servlet Juliana Louback created for fixing phone numbers in December 2013. It has since been used as part of the Lumicall web site and Juliana was selected for a GSoC 2014 project with Debian. There is no better way to demonstrate to a prospective mentor that you are ready for GSoC than by completing and publishing some small project like this yourself. If you don't have any immediate project ideas, many developers will also be able to give you tips on small projects like this that you can attempt, just come and ask us on one of the mailing lists. Ideally, the project will be something that you would use anyway even if you do not end up participating in GSoC. Such projects are the most motivating and rewarding and usually end up becoming an example of your best work. To continue the example of somebody with a preference for business and accounting software, a small project you might create is a plugin or extension for PostBooks. Getting to know prospective mentors Many web sites provide useful information about the developers who contribute to free software projects. Some of these developers may be willing to be a GSoC mentor. For example, look through some of the following: Getting on the mentor's shortlist Once you have identified projects that are interesting to you and developers who work on those projects, it is important to get yourself on the developer's shortlist. Basically, the shortlist is a list of all students who the developer believes can complete the project. If I feel that a student is unlikely to complete a project or if I don't have enough information to judge a student's probability of success, that student will not be on my shortlist. If I don't have any student on my shortlist, then a project will not go ahead at all. If there are multiple students on the shortlist, then I will be looking more closely at each of them to try and work out who is the best match. One way to get a developer's attention is to look at bug reports they have created. Github makes it easy to see complaints or bug reports they have made about their own projects or other projects they depend on. Another way to do this is to search through their code for strings like FIXME and TODO. Projects with standalone bug trackers like the Debian bug tracker also provide an easy way to search for bug reports that a specific person has created or commented on. Once you find some relevant bug reports, email the developer. Ask if anybody else is working on those issues. Try and start with an issue that is particularly easy and where the solution is interesting for you. This will help you learn to compile and test the program before you try to fix any more complicated bugs. It may even be something you can work on as part of your academic program. Find successful projects from the previous year Contact organizations and ask them which GSoC projects were most successful. In many organizations, you can find the past students' project plans and their final reports published on the web. Read through the plans submitted by the students who were chosen. Then read through the final reports by the same students and see how they compare to the original plans. Start building your project proposal now Don't wait for the application period to begin. Start writing a project proposal now. When writing a proposal, it is important to include several things:
  • Think big: what is the goal at the end of the project? Does your work help the greater good in some way, such as increasing the market share of Linux on the desktop?
  • Details: what are specific challenges? What tools will you use?
  • Time management: what will you do each week? Are there weeks where you will not work on GSoC due to vacation or other events? These things are permitted but they must be in your plan if you know them in advance. If an accident or death in the family cut a week out of your GSoC project, which work would you skip and would your project still be useful without that? Having two weeks of flexible time in your plan makes it more resilient against interruptions.
  • Communication: are you on mailing lists, IRC and XMPP chat? Will you make a weekly report on your blog?
  • Users: who will benefit from your work?
  • Testing: who will test and validate your work throughout the project? Ideally, this should involve more than just the mentor.
If your project plan is good enough, could you put it on Kickstarter or another crowdfunding site? This is a good test of whether or not a project is going to be supported by a GSoC mentor. Learn about packaging and distributing software Packaging is a vital part of the free software lifecycle. It is very easy to upload a project to Github but it takes more effort to have it become an official package in systems like Debian, Fedora and Ubuntu. Packaging and the communities around Linux distributions help you reach out to users of your software and get valuable feedback and new contributors. This boosts the impact of your work. To start with, you may want to help the maintainer of an existing package. Debian packaging teams are existing communities that work in a team and welcome new contributors. The Debian Mentors initiative is another great starting place. In the Fedora world, the place to start may be in one of the Special Interest Groups (SIGs). Think from the mentor's perspective After the application deadline, mentors have just 2 or 3 weeks to choose the students. This is actually not a lot of time to be certain if a particular student is capable of completing a project. If the student has a published history of free software activity, the mentor feels a lot more confident about choosing the student. Some mentors have more than one good student while other mentors receive no applications from capable students. In this situation, it is very common for mentors to send each other details of students who may be suitable. Once again, if a student has a good Github profile and a blog, it is much easier for mentors to try and match that student with another project. GSoC logo generic Conclusion Getting into the world of software engineering is much like joining any other profession or even joining a new hobby or sporting activity. If you run, you probably have various types of shoe and a running watch and you may even spend a couple of nights at the track each week. If you enjoy playing a musical instrument, you probably have a collection of sheet music, accessories for your instrument and you may even aspire to build a recording studio in your garage (or you probably know somebody else who already did that). The things listed on this page will not just help you walk the walk and talk the talk of a software developer, they will put you on a track to being one of the leaders. If you look over the profiles of other software developers on the Internet, you will find they are doing most of the things on this page already. Even if you are not selected for GSoC at all or decide not to apply, working through the steps on this page will help you clarify your own ideas about your career and help you make new friends in the software engineering community.

25 September 2015

Christian Perrier: Bugs #780000 - 790000

Thorsten Glaser reported Debian bug #780000 on Saturday March 7th 2015, against the gcc-4.9 package. Bug #770000 was reported as of November 18th so there have been 10,000 bugs in about 3.5 months, which was significantly slower than earlier. Salvatore Bonaccorso reported Debian bug #790000 on Friday June 26th 2015, against the pcre3 package. Thus, there have been 10,000 bugs in 3.5 months again. It seems that the bug report rate stabilized again. Sorry for missing bug #780000 annoucement. I'm doing this since....November 2007 for bug #450000 and it seems that this lack of attention is somehow significant wrt my involvment in Debian. Still, this involvment is still here and I'll try to "survive" in the project until we reach bug #1000000...:-) See you for bug #800000 annoucement and the result of the bets we placed on the date it would happen.

23 September 2015

Simon Josefsson: Cosmos A Simple Configuration Management System

Back in early 2012 I had been helping with system administration of a number of Debian/Ubuntu-based machines, and the odd Solaris machine, for a couple of years at $DAYJOB. We had a combination of hand-written scripts, documentation notes that we cut n paste d from during installation, and some locally maintained Debian packages for pulling in dependencies and providing some configuration files. As the number of people and machines involved grew, I realized that I wasn t happy with how these machines were being administrated. If one of these machines would disappear in flames, it would take time (and more importantly, non-trivial manual labor) to get its services up and running again. I wanted a system that could automate the complete configuration of any Unix-like machine. It should require minimal human interaction. I wanted the configuration files to be version controlled. I wanted good security properties. I did not want to rely on a centralized server that would be a single point of failure. It had to be portable and be easy to get to work on new (and very old) platforms. It should be easy to modify a configuration file and get it deployed. I wanted it to be easy to start to use on an existing server. I wanted it to allow for incremental adoption. Surely this must exist, I thought. During January 2012 I evaluated the existing configuration management systems around, like CFEngine, Chef, and Puppet. I don t recall my reasons for rejecting each individual project, but needless to say I did not find what I was looking for. The reasons for rejecting the projects I looked at ranged from centralization concerns (single-point-of-failure central servers), bad security (no OpenPGP signing integration), to the feeling that the projects were too complex and hence fragile. I m sure there were other reasons too. In February I started going back to my original needs and tried to see if I could abstract something from the knowledge that was in all these notes, script snippets and local dpkg packages. I realized that the essence of what I wanted was one shell script per machine, OpenPGP signed, in a Git repository. I could check out that Git repository on every new machine that I wanted to configure, verify the OpenPGP signature of the shell script, and invoke the script. The script would do everything needed to get the machine up into an operational stage again, including package installation and configuration file changes. Since I would usually want to modify configuration files on a system even after its initial installation (hey not everyone is perfect), it was natural to extend this idea to a cron job that did git pull , verified the OpenPGP signature, and ran the script. The script would then have to be a bit more clever and not redo everything every time. Since we had many machines, it was obvious that there would be huge code duplication between scripts. It felt natural to think of splitting up the shell script into a directory with many smaller shell scripts, and invoke each shell script in turn. Think of the /etc/init.d/ hierarchy and how it worked with System V initd. This would allow re-use of useful snippets across several machines. The next realization was that large parts of the shell script would be to create configuration files, such as /etc/network/interfaces. It would be easier to modify the content of those files if they were stored as files in a separate directory, an overlay stored in a sub-directory overlay/, and copied into the file system s hierarchy with rsync. The final realization was that it made some sense to run one set of scripts before rsync ing in the configuration files (to be able to install packages or set things up for the configuration files to make sense), and one set of scripts after the rsync (to perform tasks that require some package to be installed and configured). These set of scripts were called the pre-tasks and post-tasks respectively, and stored in sub-directories called pre-tasks.d/ and post-tasks.d/. I started putting what would become Cosmos together during February 2012. Incidentally, I had been using etckeeper on our machines, and I had been reading its source code, and it greatly inspired the internal design of Cosmos. The git history shows well how the ideas evolved even that Cosmos was initially called Eve but in retrospect I didn t like the religious connotations and there were a couple of rewrites on the way, but on the 28th of February I pushed out version 1.0. It was in total 778 lines of code, with at least 200 of those lines being the license boiler plate at the top of each file. Version 1.0 had a debian/ directory and I built the dpkg file and started to deploy on it some machines. There were a couple of small fixes in the next few days, but development stopped on March 5th 2012. We started to use Cosmos, and converted more and more machines to it, and I quickly also converted all of my home servers to use it. And even my laptops. It took until September 2014 to discover the first bug (the fix is a one-liner). Since then there haven t been any real changes to the source code. It is in daily use today. The README that comes with Cosmos gives a more hands-on approach on using it, which I hope will serve as a starting point if the above introduction sparked some interest. I hope to cover more about how to use Cosmos in a later blog post. Since Cosmos does so little on its own, to make sense of how to use it, you want to see a Git repository with machine models. If you want to see how the Git repository for my own machines looks you can see the sjd-cosmos repository. Don t miss its README at the bottom. In particular, its global/ sub-directory contains some of the foundation, such as OpenPGP key trust handling.

Daniel Pocock: The only way to ensure the VW scandal never happens again

The automotive industry has been in the spotlight after a massive scandal at Volkswagen, using code hidden in the engine management software to cheat emissions tests. What else is hidden in your new car's computer? Every technology we use in our lives is becoming more computerized, even light bulbs and toilet seats. In a large piece of equipment like a car, there are many opportunities for computerization. In most cases, consumers aren't even given a choice whether or not they want software in their car. It has long been known that such software is spying on the habits of the driver and this data is extracted from the car when it is serviced and uploaded to the car company. Car companies are building vast databases about the lifestyles, habits and employment activities of their customers. Computers aren't going away, so what can be done? Most people realize that computers aren't going to go away any time soon. That doesn't mean that people have to put up with these deceptions and intrusions on our lives. For years, many leading experts in the software engineering world have been promoting the benefits and principles of free software. What we mean by free is that users, regulators and other independent experts should have the freedom to see and modify the source code in the equipment that we depend on as part of modern life. In fact, experts generally agree that there is no means other than software freedom to counter the might of corporations like Volkswagen and their potential to misuse that power, as demonstrated in the emissions testing scandal. If Governments and regulators want to be taken seriously and protect society, isn't it time that they insisted that the car industry replaces all hidden code with free and open source software?

Russ Allbery: Review: Getting Back to Full Employment

Review: Getting Back to Full Employment, by Dean Baker & Jared Bernstein
Publisher: CEPR
Copyright: 2013
ISBN: 0-615-91836-0
Format: Kindle
Pages: 116
Getting Back to Full Employment is more of a policy paper than a book. The authors both work for economic think tanks; one of those, the Center for Economic and Policy research (co-founded by Baker), is the publisher of this book. It's also free on the Kindle, another sign that the purpose is more about educating and convincing the public than about selling a non-fiction analysis. I found out about this through Paul Krugman, and if you're a regular reader of Krugman's columns and blog, not much here will be a surprise. Baker and Bernstein are advocating what I would call conventional-liberal economic policy by US standards (which means that it's not really that liberal). The short version is that full employment is vital for improving the economic position of the average person, inflation is nowhere near as much of a risk as people claim, and the best economic action the US government can take at present is to aggressively pursue a full employment strategy without worrying excessively about inflation. If you're at all familiar with this debate, you probably already have an opinion about everything in that summary. I certainly do. If you follow economics at all, this debate has been raging ever since the 2008 financial crisis (and earlier versions of the debate were raging before then). Since I'm a regular Krugman reader, you probably know my opinion: the people who have been crying wolf about the dangers of inflation have been systematically wrong about everything for seven years solid, and it's amazing that anyone is still giving them the time of day. Baker and Bernstein make a similar argument at greater length, and with a more academic tone. The risk of any policy paper like this is that reactions will be almost entirely decided by one's pre-existing opinion about the political actions that each economic policy implies. If you (like me) have already decided that government intervention in the economy is useful to prevent concentration of wealth in the hands of a few and provide support for the most vulnerable, everything Baker and Bernstein say here is going to sound quite convincing. If you've already decided that government intervention in the economy almost always leads to tears, and that a large government with the ability to take economic action is far more dangerous than any of the effects of economic downturns and unemployment, it would surprise me if Baker and Bernstein will change your mind. In other words, I'm skeptical that policy papers like this accomplish much. That said, there are a lot of facts and data here, which at least help ground this ongoing argument in more specifics. The authors show you the math: the evidence for moderate inflation not leading to any serious impact for economies, the very strong correlations between low unemployment rates and rising median wages, the theoretical basis for lower bounds on unemployment rates and our extensive past history of assuming that lower bound is much higher than it actually is, and some pointed comparisons to countries like Germany that have taken many of the specific policy recommendations included here and have a far healthier labor market as a result. All the statistics, charts, and discussion of analysis techniques is fairly dry, but I think the data is compelling. Of course, I was already convinced by earlier reading. There are a few bits here that I hadn't seen before, or at least hadn't internalized. One is a subtle but very important statistical limitation: any uncertainty in measuring the inflation rate will lead to incorrect statistical findings that inflation slows growth. The explanation for this takes several pages and is somewhat technical: the short and somewhat inaccurate version is that growth is measured in inflation-adjusted GDP, so if you get inflation wrong, you magnify the correlation between GDP and inflation. An inaccurately-high measure of inflation rate simultaneously leads to underestimating GDP growth for that country; an inflation rate that's inaccurate on the low side leads to overestimating GDP growth. This is inherent in how this comparison is done, and it's hard to see how to avoid it. It's therefore worth taking any analysis of the growth impact from higher inflation with a grain of salt. (Please note here that we're talking about moderate inflation the typical rates between 1% and 8% that you see in developed economics.) Another bit that was new to me was the analysis of the connection between trade deficits and full employment. I typically dismiss any international trade analysis in macroeconomic policy for the US since international trade is a much smaller part of our economy than most people think it is, and because protectionism has been rightfully discarded as a policy approach. But Baker and Bernstein make an interesting argument here about the effects of a strong currency on increasing unemployment; even with the relatively small amount of trade, reducing the deficit would have a noticeable effect. (Increasing the volume of trade but not changing the balance would not.) The authors certainly don't recommend protectionism; they do recommend being willing to let the value of the dollar drop against other currencies as a good way to reduce the deficit, and to be aware that other central banks have intentional policies of propping up the dollar that we're largely ignoring. They also mention that this is a zero-sum game for the world economy as a whole, although I would have preferred a bit more emphasis on that. You can't get very far via beggar-thy-neighbor economic policy. But it's a good reminder that the obsession in some parts with not "devaluing the currency" is economic nonsense and directly hurts employment. The end of the book discusses a variety of sensible policy recommendations ranging from international currency policy to Germany's work-sharing program where the government subsidies reduction of worker hours the same way that we already subsidize unemployment. They all seemed reasonable to me, although, sadly, I have little faith that the dysfunctional US government will consider any of them seriously, particularly more direct approaches such as government taking on the role of employer of last resort. But they're fun to read about and imagine a US in which the government actually did sensible things like that. Getting Back to Full Employment is pretty dry, and it's hard to shake the feeling that writing things like this is basically pointless, so I'm not sure I can really recommend it. But if you like reading Krugman's columns and are interested in similar material from different people at a longer length, that's basically what this is. Rating: 6 out of 10

22 September 2015

Russ Allbery: INN 2.6.0

This is the first major release of INN in several years, and incorporates lots of updates to the build system, protocol support, and the test suite. Major highlights and things to watch out for: There are also tons of accumulated bug fixes, including (of course) all the fixes that were in INN 2.5.5. There are a lot of other changes, so I won't go into all the details here. If you're curious, take a look at the NEWS file. As always, thanks to Julien LIE for preparing this release and doing most of the maintenance work on INN! You can get the latest version from the official ISC download page or from my personal INN pages. The latter also has links to the full changelog and the other INN documentation.

21 September 2015

Norbert Preining: International Sad Hits Volume 1

I stumbled over this CD a few weeks ago, and immediately ordered it from some second-hand dealer in the US: International Sad Hits Volume 1: Altaic Group. Four artists from different countries (2x Japan, Korea, Turkey) and very different music style, but connected in one thing: That they don t fit into the happy-peppy culture of AKB48, JPOP, KPOP and the like, but singers and songwriters that probe the depths of sadness. With two of my favorite Japanese appearing in the list (Tomokawa Kazuki and Mikami Kan), there was no way I could not buy this CD. international-sad-hits-volume-1 The four artist combined in this excellent CD are: Fikret K z lok, a Turkish musician, singer and songwriter. Quoting from the pamphlet:
However, in 1983 K z lok returned with one of his most important and best albums: Zaman Zaman (Time to time). [ ] These albums presented an alternative to the horrible pop scene emerging in Turkey they criticized the political situation, the so-called intellectuals, and the pop stars.
The Korean artist is (Kim Doo Soo), who was the great surprise of the CD for me. The sadness and beauty that is transmitted through is music is special. The pamphlet of the CD states:
The strictness of his home atmosphere suffocated him, and in defiance against his father he dropped out, and walked along the roads. He said later that, Hatred towards the world, and the emptiness of life overwhelmed me. I lived my life with alcohol every day.
After some problems due to political crisis, and fierce reactions to his song Bohemian (included) made him disappear into the mountains for 10 years, only to return with even better music. The third artist is one of my big favorites, Tomokawa Kazuki (Official site, Wikipedia) from Japan. I have written often about Tomokawa, but found a very fitting description in the booklet:
Author Tatematsu Wahei has described Tomokawa as a man standing naked, his sensibility utterly exposed and tingling. It s an accurate response to a creativity that seems unmediated by embarrassment, voraciously feeding off the artist s personal concern.
The forth artist is again from Japan, Mikami Kan, a well known wild man from the Japanese music scene. After is debut at the Nakatsugawa All-Japan Folk Jamboree in 1971, he was something like a star for some years, but without a record deal his popularity decreased steadily.
During this period his songwriting gradually altered, becoming more dense, surreal and uncompromisingly personal.
Every song on this CD is a masterpiece by itself, but despite me being a great fan of Tomokawa, my favorite is Kim Doo Soo here, with songs that grip your heart and soul, stunningly beauty and sad at the same time.

19 September 2015

Norbert Preining: Movie: Werner Herzog Fata Morgana

Finally I found the time and peace to watch one of the strangest movies that I have ever seen, Werner Herzog s (Offical Site, Wikipedia) Fata Morgana (Wikipedia, IMDb). Footage prepared for a SF movie that never realized, converted into an impressionistic allegory and movie full of phantasms.
Werner_Herzog-Fata_Morgana Fata Morgana is a movie that escapes every description. Having read several critics voices, as well as Herzog s comments in the excellent interview/(auto)biography book Werner Herzog A guide for the perplexed, I was eager to see this old and special movie. Having laid my hand on a collection of Werner Herzog s old movies recently, I prepared some good Japanese sake, and was ready to enjoy the move. The initial sequence of airplanes landing on a very hot day, again and again, sets the stage for a series of dream-like sequences in this movie. Divided into three parts, the movie tells its story only by visual impressions and the reading of the Mayan creation myth Popol Vuh, at times intermixed with some Leonard Cohen songs. Part One Creation introduces the desert as the origin of the world, accompanied by the initial creation myth before humans were formed. Part 2 Paradise is when humans enter into the desert. Full of ramshackle houses and rotten buildings, the contrast of the spoken words and imaginary cannot be more stunning. The final part 3 The Golden Age is full of absurdities and strange appearances, not surprisingly also most humans appear here. Watching this movie is more a visual experience for me than about seeing a story line. While there are many ways to interpret the relation between the Popol Vuh and the movie sequences, for me it is more a visual experiment that tries in some sense to hypnotize the audience. For those who love strange movies, an absolute must. For all others probably a torture.

17 September 2015

Julien Danjou: My interview in le Journal du Hacker

A few days ago, the French equivalent of Hacker News, called "Le Journal du Hacker", interviewed me about my work on OpenStack, my job at Red Hat and my self-published book The Hacker's Guide to Python. I've spent some time translating it into English so you can read it if you don't understand French! I hope you'll enjoy it.
Hi Julien, and thanks for participating in this interview for the Journal du Hacker. For our readers who don't know you, can you introduce you briefly?
You're welcome! My name is Julien, I'm 31 years old, and I live in Paris. I now have been developing free software for around fifteen years. I had the pleasure to work (among other things) on Debian, Emacs and awesome these last years, and more recently on OpenStack. Since a few months now, I work at Red Hat, as a Principal Software Engineer on OpenStack. I am in charge of doing upstream development for that cloud-computing platform, mainly around the Ceilometer, Aodh and Gnocchi projects.
Being myself a system architect, I follow your work in OpenStack since a while. It's uncommon to have the point of view of someone as implied as you are. Can you give us a summary of the state of the project, and then detail your activities in this project?
The OpenStack project has grown and changed a lot since I started 4 years ago. It started as a few projects providing the basics, like Nova (compute), Swift (object storage), Cinder (volume), Keystone (identity) or even Neutron (network) who are basis for a cloud-computing platform, and finally became composed of a lot more projects. For a while, the inclusion of projects was the subject of a strict review from the technical committee. But since a few months, the rules have been relaxed, and we see a lot more projects connected to cloud-computing joining us. As far as I'm concerned, I've started with a few others people the Ceilometer project in 2012, devoted to handling metrics of OpenStack platforms. Our goal is to be able to collect all the metrics and record them to analyze them later. We also have a module providing the ability to trigger actions on threshold crossing (alarm). The project grew in a monolithic way, and in a linear way for the number of contributors, during the first two years. I was the PTL (Project Technical Leader) for a year. This leader position asks for a lot of time for bureaucratic things and people management, so I decided to leave my spot in order to be able to spend more time solving the technical challenges that Ceilometer offered. I've started the Gnocchi project in 2014. The first stable version (1.0.0) was released a few months ago. It's a timeseries database offering a REST API and a strong ability to scale. It was a necessary development to solve the problems tied to the large amount of metrics created by a cloud-computing platform, where tens of thousands of virtual machines have to be metered as often as possible. This project works as a standalone deployment or with the rest of OpenStack. More recently, I've started Aodh, the result of moving out the code and features of Ceilometer related to threshold action triggering (alarming). That's the logical suite to what we started with Gnocchi. It means Ceilometer is to be split into independent modules that can work together with or without OpenStack. It seems to me that the features provided by Ceilometer, Aodh and Gnocchi can also be interesting for operators running more classical infrastructures. That's why I've pushed the projects into that direction, and also to have a more service-oriented architecture (SOA)
I'd like to stop for a moment on Ceilometer. I think that this solution was very expected, especially by the cloud-computing providers using OpenStack for billing resources sold to their customers. I remember reading a blog post where you were talking about the high-speed construction of this brick, and features that were not supposed to be there. Nowadays, with Gnocchi and Aodh, what is the quality of the brick Ceilometer and the programs it relies on?
Indeed, one of the first use-case for Ceilometer was tied to the ability to get metrics to feed a billing tool. That's now a reached goal since we have billing tools for OpenStack using Ceilometer, such as CloudKitty. However, other use-cases appeared rapidly, such as the ability to trigger alarms. This feature was necessary, for example, to implement the auto-scaling feature that Heat needed. At the time, for technical and political reasons, it was not possible to implement this feature in a new project, and the functionality ended up in Ceilometer, since it was using the metrics collected and stored by Ceilometer itself. Though, like I said, this feature is now in its own project, Aodh. The alarm feature is used since a few cycles in production, and the Aodh project brings new features on the table. It allows to trigger threshold actions and is one of the few solutions able to work at high scale with several thousands of alarms. It's impossible to make Nagios run with millions of instances to fetch metrics and triggers alarms. Ceilometer and Aodh can do that easily on a few tens of nodes automatically. On the other side, Ceilometer has been for a long time painted as slow and complicated to use, because its metrics storage system was by default using MongoDB. Clearly, the data structure model picked was not optimal for what the users were doing with the data. That's why I started Gnocchi last year, which is perfectly designed for this use case. It allows linear access time to metrics (O(1) complexity) and fast access time to the resources data via an index. Today, with 3 projects having their own perimeter of features defined and which can work together Ceilometer, Aodh and Gnocchi finally erased the biggest problems and defects of the initial project.
To end with OpenStack, one last question. You're a Python developer for a long time and a fervent user of software testing and test-driven development. Several of your blogs posts point how important their usage are. Can you tell us more about the usage of tests in OpenStack, and the test prerequisites to contribute to OpenStack?
I don't know any project that is as tested on every layer as OpenStack is. At the start of the project, there was a vague test coverage, made of a few unit tests. For each release, a bunch of new features were provided, and you had to keep your fingers crossed to have them working. That's already almost unacceptable. But the big issue was that there was also a lot of regressions, et things that were working were not anymore. It was often corner cases that developers forgot about that stopped working. Then the project decided to change its policy and started to refuse all patches new features or bug fix that would not implement a minimal set of unit tests, proving the patch would work. Quickly, regressions were history, and the number of bugs largely reduced months after months. Then came the functional tests, with the Tempest project, which runs a test battery on a complete OpenStack deployment. OpenStack now possesses a complete test infrastructure, with operators hired full-time to maintain them. The developers have to write the test, and the operators maintain an architecture based on Gerrit, Zuul, and Jenkins, which runs the test battery of each project for each patch sent. Indeed, for each version of a patch sent, a full OpenStack is deployed into a virtual machine, and a battery of thousands of unit and functional tests is run to check that no regressions are possible. To contribute to OpenStack, you need to know how to write a unit test the policy on functional tests is laxer. The tools used are standard Python tools, unittest for the framework and tox to run a virtual environment (venv) and run them. It's also possible to use DevStack to deploy an OpenStack platform on a virtual machine and run functional tests. However, since the project infrastructure also do that when a patch is submitted, it's not mandatory to do that yourself locally.
The tools and tests you write for OpenStack are written in Python, a language which is very popular today. You seem to like it more than you have to, since you wrote a book about it, The Hacker's Guide to Python, that I really enjoyed. Can you explain what brought you to Python, the main strong points you attribute to this language (quickly) and how you went from developer to author?
I stumbled upon Python by chance, around 2005. I don't remember how I hear about it, but I bought a first book to discover it and started toying with that language. At that time, I didn't find any project to contribute to or to start. My first project with Python was rebuildd for Debian in 2007, a bit later. I like Python for its simplicity, its object orientation rather clean, its easiness to be deployed and its rich open source ecosystem. Once you get the basics, it's very easy to evolve and to use it for anything, because the ecosystem makes it easy to find libraries to solve any kind of problem. I became an author by chance, writing blog posts from time to time about Python. I finally realized that after a few years studying Python internals (CPython), I learned a lot of things. While writing a post about the differences between method types in Python which is still one of the most read post on my blog I realized that a lot of things that seemed obvious to me where not for other developers. I wrote that initial post after thousands of hours spent doing code reviews on OpenStack. I, therefore, decided to note all the developers pain points and to write a book about that. A compilation of what years of experience taught me and taught to the other developers I decided to interview in the book.
I've been very interested by the publication of your book, for the subject itself, but also the process you chose. You self-published the book, which seems very relevant nowadays. Is that a choice from the start? Did you look for an editor? Can you tell use more about that?
I've been lucky to find out about others self-published authors, such as Nathan Barry who even wrote a book on that subject, called Authority. That's what convinced me it was possible and gave me hints for that project. I've started to write in August 2013, and I ran the firs interviews with other developers at that time. I started to write the table of contents and then filled the pages with what I knew and what I wanted to share. I manage to finish the book around January 2014. The proof-reading took more time than I expected, so the book was only released in March 2014. I wrote a complete report on that on my blog, where I explain the full process in detail, from writing to launching. I did not look for editors though I've been proposed some. The idea of self-publishing really convince me, so I decided to go on my own, and I have no regret. It's true that you have to wear two hats at the same time and handle a lot more things, but with a minimal audience and some help from the Internet, anything's possible! I've been reached by two editors since then, a Chinese and Korean one. I gave them rights to translate and publish the books in their countries, so you can buy the Chinese and Korean version of the first edition of the book out there. Seeing how successful it was, I decided to launch a second edition in Mai 2015, and it's likely that a third edition will be released in 2016.
Nowadays, you work for Red Hat, a company that represents the success of using Free Software as a commercial business model. This company fascinates a lot in our community. What can you say about your employer from your point of view?
It only has been a year since I joined Red Hat (when they bought eNovance), so my experience is quite recent. Though, Red Hat is really a special company on every level. It's hard to see from the outside how open it is, and how it works. It's really close to and it really looks like an open source project. For more details, you should read The Open Organization, a book wrote by Jim Whitehurst (CEO of Red Hat), which he just published. It describes perfectly how Red Hat works. To summarize, meritocracy and the lack of organization in silos is what makes Red Hat a strong organization and puts them as one of the most innovative company. In the end, I'm lucky enough to be autonomous for the project I work on with my team around OpenStack, and I can spend 100% working upstream and enhance the Python ecosystem.

7 September 2015

Steve Kemp: The Jessie 8.2 point-release broke for me

I have about 18 personal hosts, all running the Jessie release of Debian GNU/Linux. To keep up with security updates I use unattended-upgrades. The intention is that every day, via cron, the system will look for updates and apply them. Although I mostly expect it to handle security updates I also have it configured such that point-releases will be applied by magic too. Unfortunately this weekend, with the 8.2 release, things broke in a significant way - The cron deamon was left in a broken state, such that all cronjobs failed to execute. I was amazed that nobody had reported a bug, as several people on twitter had the same experience as me, but today I read through a lot of bug-reports and discovered that #783683 is to blame: The solution: 
# dpkg --configure -a
# apt-get upgrade
I guess the good news is I spotted it promptly, with the benefit of hindsight the bug report does warn of this as being a concern, but I guess there wasn't a great solution. Anyway I hope others see this, or otherwise spot the problem themselves. In unrelated news the seaweedfs file-store I previously introduced is looking more and more attractive to me. I reported a documentation-related bug which was promptly handled, even though it turned out I was wrong, and I contributed CIDR support to whitelisting hosts which was merged in well. I've got a two-node "cluster" setup at the moment, and will be expanding that shortly. I've been doing a lot of little toy-projects in Go recently. This weekend I was mostly playing with the nats.io message-bus, and tying it together with sinatra.

Next.

Previous.