Search Results: "gordon"

31 December 2023

Chris Lamb: Favourites of 2023

This post should have marked the beginning of my yearly roundups of the favourite books and movies I read and watched in 2023. However, due to coming down with a nasty bout of flu recently and other sundry commitments, I wasn't able to undertake writing the necessary four or five blog posts In lieu of this, however, I will simply present my (unordered and unadorned) highlights for now. Do get in touch if this (or any of my previous posts) have spurred you into picking something up yourself

Books

Peter Watts: Blindsight (2006) Reymer Banham: Los Angeles: The Architecture of Four Ecologies (2006) Joanne McNeil: Lurking: How a Person Became a User (2020) J. L. Carr: A Month in the Country (1980) Hilary Mantel: A Memoir of My Former Self: A Life in Writing (2023) Adam Higginbotham: Midnight in Chernobyl (2019) Tony Judt: Postwar: A History of Europe Since 1945 (2005) Tony Judt: Reappraisals: Reflections on the Forgotten Twentieth Century (2008) Peter Apps: Show Me the Bodies: How We Let Grenfell Happen (2021) Joan Didion: Slouching Towards Bethlehem (1968)Erik Larson: The Devil in the White City (2003)

Films Recent releases

Unenjoyable experiences included Alejandro G mez Monteverde's Sound of Freedom (2023), Alex Garland's Men (2022) and Steven Spielberg's The Fabelmans (2022).
Older releases (Films released before 2022, and not including rewatches from previous years.) Distinctly unenjoyable watches included Ocean's Eleven (1960), El Topo (1970), L olo (1992), Hotel Mumbai (2018), Bulworth (1998) and and The Big Red One (1980).

26 July 2023

Shirish Agarwal: Manipur Violence, Drugs, Binging on Northshore, Alaska Daily, Doogie Kamealoha and EU Digital Resilence Act.

Manipur Videos Warning: The text might be mature and will have references to violence so if there are kids or you are sensitive, please excuse. Few days back, saw the videos and I cannot share the rage, shame and many conflicting emotions that were going through me. I almost didn t want to share but couldn t stop myself. The woman in the video were being palmed, fingered, nude, later reportedly raped and murdered. And there have been more than a few cases. The next day saw another video that showed beheaded heads, and Kukis being killed just next to their houses. I couldn t imagine what those people must be feeling as the CM has been making partisan statements against them. One of the husbands of the Kuki women who had been paraded, fondled is an Army Officer in the Indian Army. The Meiteis even tried to burn his home but the Army intervened and didn t let it get burnt. The CM s own statement as shared before tells his inability to bring the situation out of crisis. In fact, his statement was dumb stating that the Internet shutdown was because there were more than 100 such cases. And it s spreading to the nearby Northeast regions. Now Mizoram, the nearest neighbor is going through similar things where the Meitis are not dominant. The Mizos have told the Meitis to get out. To date, the PM has chosen not to visit Manipur. He just made a small 1 minute statement about it saying how the women have shamed India, an approximation of what he said.While it s actually not the women but the men who have shamed India. The Wire has been talking to both the Meitis, the Kukis, the Nagas. A Kuki women sort of bared all. She is right on many counts. The GOI while wanting to paint the Kukis in a negative light have forgotten what has been happening in its own state, especially its own youth as well as in other states while also ignoring the larger geopolitics and business around it. Taliban has been cracking as even they couldn t see young boys, women becoming drug users. I had read somewhere that 1 in 4 or 1 in 5 young person in Afghanistan is now in its grip. So no wonder,the Taliban is trying to eradicate and shutdown drug use among it s own youth. Circling back to Manipur, I was under the wrong impression that the Internet shutdown is now over. After those videos became viral as well as the others I mentioned, again the orders have been given and there is shutdown. It is not fully shut but now only Govt. offices have it. so nobody can share a video that goes against any State or Central Govt. narrative A real sad state of affairs Update: There is conditional reopening whatever that means When I saw the videos, the first thing is I felt was being powerless, powerless to do anything about it. The second was if I do not write about it, amplify it and don t let others know about it then what s the use of being able to blog

Mental Health, Binging on various Webseries Both the videos shocked me and I couldn t sleep that night or the night after. it. Even after doing work and all, they would come in unobtrusively in my nightmares While I felt a bit foolish, I felt it would be nice to binge on some webseries. Little I was to know that both Northshore and Alaska Daily would have stories similar to what is happening here While the story in Alaska Daily is fictional it resembles very closely to a real newspaper called Anchorage Daily news. Even there the Intuit women , one of the marginalized communities in Alaska. The only difference I can see between GOI and the Alaskan Government is that the Alaskan Government was much subtle in doing the same things. There are some differences though. First, the State is and was responsive to the local press and apart from one close call to one of its reporters, most reporters do not have to think about their own life in peril. Here, the press cannot look after either their livelihood or their life. It was a juvenile kid who actually shot the video, uploaded and made it viral. One needs to just remember the case details of Siddique Kappan. Just for sharing the news and the video he was arrested. Bail was denied to him time and time again citing that the Police were investigating . Only after 2 years and 3 months he got bail and that too because none of the charges that the Police had they were able to show any prima facie evidence. One of the better interviews though was of Vrinda Grover. For those who don t know her, her Wikipedia page does tell a bit about her although it is woefully incomplete. For example, most recently she had relentlessly pursued the unconstitutional Internet Shutdown that happened in Kashmir for 5 months. Just like in Manipur, the shutdown was there to bury crimes either committed or being facilitated by the State. For the issues of livelihood, one can take the cases of Bipin Yadav and Rashid Hussain. Both were fired by their employer Dainik Bhaskar because they questioned the BJP MP Smriti Irani what she has done for the state. The problems for Dainik Bhaskar or for any other mainstream media is most of them rely on Government advertisements. Private investment in India has fallen to record lows mostly due to the policies made by the Centre. If any entity or sector grows a bit then either Adani or Ambani will one way or the other take it. So, for most first and second generation entrepreneurs it doesn t make sense to grow and then finally sell it to one of these corporates at a loss GOI on Adani, Ambani side of any deal. The MSME sector that is and used to be the second highest employer hasn t been able to recover from the shocks of demonetization, GST and then the pandemic. Each resulting in more and more closures and shutdowns. Most of the joblessness has gone up tremendously in North India which the Government tries to deny. The most interesting points in all those above examples is within a month or less, whatever the media reports gets scrubbed. Even the firing of the journos that was covered by some of the mainstream media isn t there anymore. I have to use secondary sources instead of primary sources. One can think of the chilling effects on reportage due to the above. The sad fact is even with all the money in the world the PM is unable to come to the Parliament to face questions.
Why is PM not answering in Parliament,, even Rahul Gandhi is not there - Surya Pratap Singh, prev. IAS Officer.
The above poster/question is by Surya Pratap Singh, a retired IAS officer. He asks why the PM is unable to answer in either of the houses. As shared before, the Govt. wants very limited discussion. Even yesterday, the Lok Sabha TV just showed the BJP MP s making statements but silent or mic was off during whatever questions or statements made by the opposition. If this isn t mockery of Indian democracy then I don t know what is Even the media landscape has been altered substantially within the last few years. Both Adani and Ambani have distributed the media pie between themselves. One of the last bastions of the free press, NDTV was bought by Adani in a hostile takeover. Both Ambani and Adani are close to this Goverment. In fact, there is no sector in which one or the other is not present. Media houses like Newsclick, The Wire etc. that are a fraction of mainstream press are where most of the youth have been going to get their news as they are not partisan. Although even there, GOI has time and again interfered. The Wire has had too many 504 Gateway timeouts in the recent months and they had been forced to move most of their journalism from online to video, rather Youtube in order to escape both the censoring and the timeouts as shared above. In such a hostile environment, how both the organizations are somehow able to survive is a miracle. Most local reportage is also going to YouTube as that s the best way for them to not get into Govt. censors. Not an ideal situation, but that s the way it is. The difference between Indian and Israeli media can be seen through this
The above is a Screenshot shared by how the Israeli media has reacted to the Israeli Government s Knesset over the judicial overhaul . Here, the press itself erodes its own by giving into the Government day and night

Binging on Webseries Saw Northshore, Three Pines, Alaska Daily and Doogie Kamealoha M.D. which is based on Doogie Howser M.D. Of the four, enjoyed Doogie Kamealoha M.D. the most but then it might be because it s a copy of Doogie Howser, just updated to the new millenia and there are some good childhood memories associated with that series. The others are also good. I tried to not see European stuff as most of them are twisted and didn t want that space.

EU Digital Operational Resilience Act and impact on FOSS Few days ago, apparently the EU shared the above Act. One can read about it more here. This would have more impact on FOSS as most development of various FOSS distributions happens in EU. Fair bit of Debian s own development happens in Germany and France. While there have been calls to make things more clearer, especially for FOSS given that most developers do foss development either on side or as a hobby while their day job is and would be different. The part about consumer electronics and FOSS is a tricky one as updates can screw up your systems. Microsoft has had a huge history of devices not working after an update or upgrade. And this is not limited to Windows as they would like to believe. Even apple seems to be having its share of issues time and time again. One would have hoped that these companies that make billions of dollars from their hardware and software sales would be doing more testing and Q&A and be more aware about security issues. FOSS, on the other hand while being more responsive doesn t make as much money vis-a-vis the competitors. Let s take the most concrete example. The most successful mobile phone having FOSS is Purism. But it s phone, it has priced itself out of the market. A huge part of that is to do with both economies of scale and trying to get an infrastructure and skills in the States where none or minimally exists. Compared that to say Pinepro that is manufactured in Hong Kong and is priced 1/3rd of the same. For most people it is simply not affordable in these times. Add to that the complexity of these modern cellphones make it harder, not easier for most people to be vigilant and update the phone at all times. Maybe we need more dumphones such as Light and Punkt but then can those be remotely hacked or not, there doesn t seem to be any answers on that one. I haven t even seen anybody even ask those questions. They may have their own chicken and egg issues. For people like me who have lost hearing, while I can navigate smartphones for now but as I become old I don t see anything that would help me. For many an elderly population, both hearing and seeing are the first to fade. There doesn t seem to be any solutions targeted for them even though they are 5-10% of any population at the very least. Probably more so in Europe and the U.S. as well as Japan and China. All of them are clearly under-served markets but dunno a solution for them. At least to me that s an open question.

19 December 2022

Russ Allbery: Review: Artifact Space

Review: Artifact Space, by Miles Cameron
Series: Arcana Imperii #1
Publisher: Gollancz
Copyright: June 2021
ISBN: 1-4732-3262-7
Format: Kindle
Pages: 483
Artifact Space is a military (mostly) science fiction novel, the first of an expected trilogy. Christian Cameron is a prolific author of historical fiction under that name, thrillers under the name Gordon Kent, and historical fantasy under the name Miles Cameron. This is his first science fiction novel. Marca Nbaro is descended from one of the great spacefaring mercantile families, but it's not doing her much good. She is a ward of the Orphanage, the boarding school for orphaned children of the DHC, generous in theory and a hellhole in practice. Her dream to serve on one of the Greatships, the enormous interstellar vessels that form the backbone of the human trading network, has been blocked by the school authorities, a consequence of the low-grade war she's been fighting with them throughout her teenage years. But Marca is not a person to take no for an answer. Pawning her family crest gets her just enough money to hire a hacker to doctor her school records, adding the graduation she was denied and getting her aboard the Greatship Athens as a new Midshipper. I don't read a lot of military science fiction, but there is one type of story that I love that military SF is uniquely well-suited to tell. It's not the combat or the tactics or the often-trite politics. It's the experience of the military as a system, a collective human endeavor. One ideal of the military is that people come to it from all sorts of backgrounds, races, and social classes, and the military incorporates them all into a system built for a purpose. It doesn't matter who you are or what you did before: if you follow the rules, do your job, and become part of a collaboration larger than yourself, you have a place and people to watch your back whether or not they know you or like you. Obviously, like any ideal, many militaries don't live up to this, and there are many stories about those failures. But the story of that ideal, told well, is a genre I like a great deal and is hard to find elsewhere. This sort of military story shares some features with found family, and it's not a coincidence that I also like found family stories. But found family still assumes that these people love you, or at least like you. For some protagonists, that's a tricky barrier both to cross and to believe one has crossed. The (admittedly idealized) military doesn't assume anyone likes you. It doesn't expect that you or anyone around you have the right feelings. It just expects you to do your job and work with other people who are doing their job. The requirements are more concrete, and thus in a way easier to believe in. Artifact Space is one of those military science fiction stories. I was entirely unsurprised to see that the author is a former US Navy career officer. The Greatships here are, technically, more of a merchant marine than a full-blown military. (The author noted in an interview that he based them on the merchant ships of Venice.) The weapons are used primarily for defense; the purpose of the Greatships is trade, and every crew member has a storage allotment in the immense cargo area that they're encouraged to use. The setting is in the far future, after a partial collapse and reconstruction of human society, in which humans have spread through interstellar space, settled habitable planets, and built immense orbital cities. The Athens is trading between multiple human settlements, but its true destination is far into the deep black: Tradepoint, where it can trade with the mysterious alien Starfish for xenoglas, a material that humans have tried and failed to reproduce and on which much of human construction now depends. This is, to warn, one of those stories where the scrappy underdog of noble birth makes friends with everyone and is far more competent than anyone expects. The story shape is not going to surprise you, and you have to have considerable tolerance for it to enjoy this book. Marca is ridiculously, absurdly central to the plot for a new Middie. Sometimes this makes sense given her history; other times, she is in the middle of improbable accidents that felt forced by the author. Cameron doesn't entirely break normal career progression, but Marca is very special in a way that you only get to be as the protagonist of a novel. That said, Cameron does some things with that story shape that I liked. Marca's hard-won survival skills are not weirdly well-suited for her new life aboard ship. To the contrary, she has to unlearn a lot of bad habits and let go of a lot of anxiety. I particularly liked her relationship with her more-privileged cabin mate, which at first seemed to only be a contrast between Thea's privilege and Marca's background, but turned into both of them learning from each other. There's a great mix of supporting characters, with a wide variety of interactions with Marca and a solid sense that all of the characters have their own lives and their own concerns that don't revolve around her. There is, of course, a plot to go with this. I haven't talked about it much because I think the summaries of this book are a bit of a spoiler, but there are several layers of political intrigue, threats to the ship, an interesting AI, and a good hook in the alien xenoglas trade. Cameron does a deft job balancing the plot with Marca's training and her slow-developing sense of place in the ship (and fear about discovery of her background and hacking). The pacing is excellent, showing all the skill I'd expect from someone with a thriller background and over forty prior novels under his belt. Cameron portrays the tedious work of learning a role on a ship without boring the reader, which is a tricky balancing act. I also like the setting: a richly multicultural future that felt like it included people from all of Earth, not just the white western parts. That includes a normalized androgyne third gender, which is the sort of thing you rarely see in military SF. Faster-than-light travel involves typical physics hand-waving, but the shape of the hand-waving is one I've not seen before and is a great excuse for copying the well-known property of oceangoing navies that longer ships can go faster. (One tech grumble, though: while Cameron does eventually say that this is a known tactic and Marca didn't come up with anything novel, deploying spread sensors for greater resolution is sufficiently obvious it should be standard procedure, and shouldn't have warranted the character reactions it got.) I thoroughly enjoyed this. Artifact Space is the best military SF that I've read in quite a while, at least back to John G. Hemry's JAG in space novels and probably better than those. It's going to strike some readers, with justification, as cliched, but the cliches are handled so well that I had only minor grumbling at a few absurd coincidences. Marca is a great character who is easy to care about. The plot was tense and satisfying, and the feeling of military structure, tradition, jargon, and ship pride was handled well. I had a very hard time putting this down and was sad when it ended. If you're in the mood for that class of "learning how to be part of a collaborative structure" style of military SF, recommended. Artifact Space reaches a somewhat satisfying conclusion, but leaves major plot elements unresolved. Followed by Deep Black, which doesn't have a release date at the time of this writing. Rating: 9 out of 10

20 December 2020

Shirish Agarwal: Insane Logic and Farming in other countries

The people who are pro-Government and in this case pro-Corporate do not have any success stories that they can share. Hence, most of the time the arguments are that the other are bad. For e.g. quite a few people argue that we don t need farmers, we can just order from restaurant. They have completely disassociated the idea that even then you need farmers as unless the farmers put the seed in, till the soil and wait for the rains or have irrigation you won t get ripe vegetables which then has to be taken out, and somehow sold to the wholesaler from where it comes to the restaurant and then to your plate. Sadly, even the farm-to-fork infographics are so depressingly sad, you want to look away. If you see the infographic you see it is just not non-veg but also vegetarian food grains which go under lot of questionable practices. Even, with such scenarios that is done by corporations our people want to go ahead. I will share stories from other countries which tell how they are doing more. The Soldier-Farmer Another sad part of these protests have been soldiers who have been returning their medals. The ones who oppose have the gall to say they should return the cash rewards they got. So just like farmers, seems soldiers also do not need money to survive. They are supposed to live only on air and water. This is after the present Govt. has reduced their pensions after retirement and that too without any discussion
GOI pensions to ex-servicemen
Now I nor anybody else would have minded if these conditions were shared going forward rather than doing it retrospectively. People who usually go to the army are not in it for money but for the adventure and glory they bring. But they do also have a family and have a family responsibility. In most other countries, the soldier and his families are well-looked after. If you know that even after you die, the Government would look after your family, you will do your best. Unfortunately, many veterans in India themselves are asked to help by many war widows as the widows don t get family pensions. The proposal naturally has left many miffed. In fact many of the veterans who used to advise people to join the armed forces now advise young people to pursue civilian life and careers. This is when Indian Army has been ironically having shortage of officers from well over a decade and stresses felt by Army personnel also known for a long time. Even under this nationalistic Government, if it cannot take care of its soldiers, then forget about others.
India Defence Spending vis-a-vis other countries.
Now it is nobody s argument that India needs to improve its tooth-to-tail ratio but this is the wrong way to go about it. I would probably talk about that some other time as that totally needs its whole place. Even OROP, which was the mandate of this Government hasn t had been done in full as there are quite a few cases in the Supreme Court. Almost all the cases have been heard and only decisions have to be given which the SC for whatever reason doesn t want to give. They just keep changing the date of the hearing. Nowadays, in many suits/cases, the SC asks for fresh hearings even though all the old records are there. This is a newish phenomena which is being observed in SC. Why is it being done? Your guess is as good as mine. One thing for sure has changed, the SC which used to be citizen-focussed or enabler of human rights and used to be held as a beacon for judicial activism has changed but these are other topics which need their own space. Update 16/12/2020 The SC recommends setting up a committee to discuss farmer issues. And this is nothing new. This is called death by committee. When there is already so much literature on the subject, including the works done by Swaminathan Commission. There has been 6 reports which do look at farmer issues in a holistic manner. This is the Supreme Court giving an escape route to GOI. They also have abstained from having a whole session citing Covid. This is when the ruling Govt. is putting a massive 1000 crore on a new building on which the SC has put on hold. And even then the GOI went ahead and did a Bhoomi-Pujan (traditional ceremony when making a new construction from scratch.) Naturally due to the double whammy of both the pension reforms and now the laws to make corporate farming more aggressive has left a deep impact on the soldier-farmer that the state does not think or feel for him. Even the United States farm-aid eloquently describes how corporate farming has made independent farmers suffer. You read that, and it seems it is as the state of our farmers here in India. Even their average land-holding has dropped a bit. I have shared about the state of farmers in India, in two blog posts previously. And it is not just farm owners who have had it bad, even farm workers in U.S. The issue may look to be about the pandemic but goes far deeper. The Israeli Model The Israelis have always used collective farming and do have a large share in farming there. The old model called Kibbutz is what made Israelis self-sufficient in food and water and actually are world-leaders where they export their services to other nations on the same thing. France Just like many other countries, France also seems to have favored farmer co-operatives. Almost 75% of all farmers are in co-operatives. Italy The country world-famous for its wines and cheese are made by its co-operatives. In fact co-ops are the buzzword it seems in Italy, more so in Northern Italy. Asian economies Even Asian economies, especially East Asian economies by and large have been turning to co-operatives. Brazil Now Brazil is almost 40% more than India. In fact, in most of the indices, Brazil beats India handsomely. So one would be forgiven to think that Brazil must have corporate farming. But nothing could be further from the truth. The only downer is that they have high crime in some areas. Otherwise, they are in many ways better than India. In fact, I was surprised a few years ago to learn about Mercsour. I would have to admit though I learned much about Brazil when Debian was holding a debconf about a year back. Otherwise, I had known about the country for number of years but apart from its carnival and samba, hadn t known much about it. I did come to know that most of Latin America also loves spices as much as Indians do. They show that love by using hot sauces. I do one day wanna try one of their sauces to see what makes it tick. I do know they like to barbecue vegetables as much as barbecuing non-veg food. This is going a bit OT but then that s the foodie in me Conclusion I could have shared more countries which have chosen the co-operative way rather than corporate farming and that is simply because they know what is best for their people and what is best even politically. The new farm laws are neither grounded in farmer s welfare nor anything else. The Govt. has been trying to undermine the farmers for years together. In fact, Madhya Pradesh has openly said that they will not allow farmers from other states to sell in their state. Although, even before these laws there was nothing to restrict the farmer from selling his produce anywhere in the country. Angering the farmers is not good politics as was found sometime back but guessing some lessons need to be re-learned. One comment though, on social media I have seen many people especially youngsters having no real understanding of what inflation is all about. For e.g. if you ask them how come we are having a sort of record inflation in a technical recession (there has been a contraction, actually) and you see them putting themselves into bigger and bigger ditches. This does explain in part why the BJP wins in elections. If you do more rhetoric, which BJP is good as, rather than educating people than you are bound to win. You don t need plans, you don t need a vision, just rhetoric will do. What more evidence is needed when the economy is and was in a worse shape even before the pandemic and BJP won. I would probably write about that as that again needs lot of background and understanding as well as related terms.

14 July 2020

Markus Koschany: My Free Software Activities in June 2020

Welcome to gambaru.de. Here is my monthly report (+ the first week in July) that covers what I have been doing for Debian. If you re interested in Java, Games and LTS topics, this might be interesting for you. Debian Games Short news
Debian Java Misc Debian LTS This was my 52. month as a paid contributor and I have been paid to work 60 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following: Thanks for reading and see you next time.

26 June 2017

Jonathan Dowland: Coming in from the cold

I've been using a Mac day-to-day since around 2014, initially as a refreshing break from the disappointment I felt with GNOME3, but since then a few coincidences have kept me on the platform. Something happened earlier in the year that made me start to think about a move back to Linux on the desktop. My next work hardware refresh is due in March next year, which gives me about nine months to "un-plumb" myself from the Mac ecosystem. From the top of my head, here's the things I'm going to have to address: I think that's probably it: not a big list! Notably, I'm not locked into iTunes, which I avoid where possible; Apple's Photo app (formerly iPhoto) which is a bit of a disaster; nor Time Machine, which is excellent, but I have a backup system for other things in place that I can use.

8 January 2017

Bits from Debian: New Debian Developers and Maintainers (November and December 2016)

The following contributors got their Debian Developer accounts in the last two months: The following contributors were added as Debian Maintainers in the last two months: Congratulations!

7 September 2016

Antonio Terceiro: Debian CI updates for September 2016

debci 1.4 was released just a few days ago. Among general improvements, I would like to highlight: ci.debian.net has been upgraded to debci 1.4 just after that. At the same time I have also upgraded autodep8 and autopkgtest to their latest versions, available in jessie-backports. This means that it is now safe for Debian packages to assume the changes in autopkgtest 4.0 are available, in special the $AUTOPKGTEST_* environment variables. In other news, for several weeks there were had issues with tests not being scheduled when they should have. I was just assuming that the issue was due to the existing test scheduler, debci-batch, being broken. Today I was working on a new implementation that is going to be a lot faster, I started to hit a similar issue on my local tests, and finally realized what was wrong. The fact is that debci-batch stores the timestamp of the last time a package has been scheduled to run, and it there are no test result after that timestamp, it assumes the package is still in the queue to be tested, and does not schedule it again. It turns out that a few weeks ago, during maintainance work, I had cleared the queue, discarding all jobs that were there, but forgot to reset those timestamps, so when debci-batch came around again, it checked the timestamp of the last request and did not make new requests because there was no test result after that timestamp! I cleared all those timestamps, and the system should now go back to normal. That is it for now. I you want to contribute to the Debian CI project and want to get in touch, you can pop up on the #debci channel on the OFTC IRC network, or mail the autopkgtest-devel mailing list.

30 May 2014

Julien Danjou: OpenStack Design Summit Juno, from a Ceilometer point of view

Last week was the OpenStack Design Summit in Atlanta, GA where we, developers, discussed and designed the new OpenStack release (Juno) coming up. I've been there mainly to discuss Ceilometer upcoming developments. The summit has been great. It was my third OpenStack design summit, and the first one not being a PTL, meaning it was a largely more relaxed summit for me! On Monday, we started by a 2.5 hours meeting with Ceilometer core developers and contributors about the Gnocchi experimental project that I've started a few weeks ago. It was a great and productive afternoon, and allowed me to introduce and cover this topic extensively, something that would not have been possible in the allocated session we had later in the week. Ceilometer had his design sessions running mainly during Wednesday. We noted a lot of things and commented during the sessions in our Etherpads instances. Here is a short summary of the sessions I've attended. Scaling the central agent I was in charge of the first session, and introduced the work that was done so far in the scaling of the central agent. Six months ago, during the Havana summit, I proposed to scale the central agent by distributing the tasks among several node, using a library to handle the group membership aspect of it. That led to the creation of the tooz library that we worked on at eNovance during the last 6 months. Now that we have this foundation available, Cyril Roelandt started to replace the Ceilometer alarming job repartition code by Taskflow and Tooz. Starting with the central agent is simpler and will be a first proof of concept to be used by the central agent then. We plan to get this merged for Juno. For the central agent, the same work needs to be done, but since it's a bit more complicated, it will be done after the alarming evaluators are converted. Test strategy The next session discussed the test strategy and how we could improve Ceilometer unit and functional testing. There is a lot in this area to be done, and this is going to be one of the main focus of the team in the upcoming weeks. Having Tempest tests run was a goal for Havana, and even if we made a lot of progress, we're still no there yet. Complex queries and per-user/project data collection This session, led by Ildik V ncsa, was about adding finer-grained configuration into the pipeline configuration to allow per-user and per-project data retrieval. This was not really controversial, though how to implement this exactly is still to be discussed, but the idea was well received. The other part of the session was about adding more in the complex queries feature provided by the v2 API. Rethinking Ceilometer as a Time-Series-as-a-Service This was my main session, the reason we met on Monday for a few hours, and one of the most promising session I hope of the week. It appears that the way Ceilometer designed its API and storage backends a long time ago is now a problem to scale the data storage. Also, the events API we introduced in the last release partially overlaps some of the functionality provided by the samples API that causes us scaling troubles. Therefore, I've started to rethink the Ceilometer API by building it as a time series read/write service, letting the audit part of our previous sample API to the event subsystem. After a few researches and experiments, I've designed a new project called Gnocchi, which provides exactly that functionality in a hopefully scalable way. Gnocchi is split in two parts: a time series API and its driver, and a resource indexing API with its own driver. Having two distinct driver sets allows it to use different technologies to store each data type in the best storage engine possible. The canonical driver for time series handling is based on Pandas and Swift. The canonical resource indexer driver is based on SQLAlchemy. The idea and project was well received and looked pretty exciting to most people. Our hope is to design a version 3 of the Ceilometer API around Gnocchi at some point during the Juno cycle, and have it ready as some sort of preview for the final release. Revisiting the Ceilometer data model This session led by Alexei Kornienko, kind of echoed the previous session, as it clearly also tried to address the Ceilometer scalability issue, but in a different way. Anyway, the SQL driver limitations have been discussed and Mehdi Abaakouk implemented some of the suggestions during the week, so we should very soon see more performances in Ceilometer with the current default storage driver. Ceilometer devops session We organized this session to get feedbacks from the devops community about deploying Ceilometer. It was very interesting, and the list of things we could improve is long, and I think will help us to drive our future efforts. SNMP inspectors This session, led by Lianhao Lu, discussed various details of the future of SNMP support in Ceilometer. Alarm and logs improvements This mixed session, led by Nejc Saje and Gordon Chung, was about possible improvements on the alarm evaluation system provided by Ceilometer, and making logging in Ceilometer more effective. Both half-sessions were interesting and led to several ideas on how to improve both systems. Conclusion Considering the current QA problems with Ceilometer, Eoghan Glynn, the new Project Technical Leader for Ceilometer, clearly indicated that this will be the main focus of the release cycle. Personally, I will be focused on working on Gnocchi, and will likely be joined by others in the next weeks. Our idea is to develop a complete solution with a high velocity in the next weeks, and then works on its integration with Ceilometer itself.

7 April 2014

Russ Allbery: Review: Fantasy & Science Fiction, September/October 2011

Review: Fantasy & Science Fiction, September/October 2011
Editor: Gordon van Gelder
Issue: Volume 121, No. 3 & 4
ISSN: 1095-8258
Pages: 258
Another review of a magazine that I finished quite some time ago. Apologies for any inaccuracies or lack of depth in the reviews. There wasn't much in Charles de Lint's reviews in this issue that interested me, but Michelle West covers a great selection of books. Two of them (The Wise Man's Fear and The Quantum Thief) are already on my to-read list; the third, The Postmortal, sounded interesting and would go on my list to purchase if I didn't already have so many good books I've not read. Otherwise, this issue is short on non-fiction. The only other essay entry is a film review from Kathi Maio, which is the typical whining about all things film that F&SF publishes. "Rutger and Baby Do Jotenheim" by Esther M. Friesner: Baby is a former pole dancer with a toy poodle named Mister Snickers, which warns you right away that this story is going to involve a few over-the-top caricatures and more use of the word "piddle" than one might ideally want. Rutger is a mythology professor who tolerates her for the standard reasons in this sort of pairing. They're travelling across country to Baby's sister's wedding when their car breaks down in Minnesota, prompting an encounter with frost giants. As you might expect, this is a sort of fractured fairy tale, except based on Norse mythology instead of the more typical Grimm fare. The fun is in watching these two apparent incompetents (but with enough knowledge of mythology to clue in the reader) reproduce the confrontation between Thor and Utgard-Loki. The fight with old age is particularly entertaining. If you've read any of Friesner's other stories, you know what to expect: not much in the way of deeper meaning, but lots of fun playing with stereotypes and an optimistic, funny outcome. Good stuff. (7) "The Man Inside Black Betty" by Sarah Langan: This story comes with a mouthful of a subtitle: "Is Nicholas Wellington the World's Best Hope?" It's also a story that purports to be written by a fictional character, in this case one Saurub Ramesh (with Langan credited as having done "research"). It's told in the style of first-person journalism, relating the thoughts and impressions of Ramesh as he interviews Nicholas Wellington. The topic is Black Betty: a black hole above Long Island Sound. Wellington is a scientific genius and iconoclast with radical theories of black holes that contradict how the government has been attempting to deal with Black Betty, unsuccessfully. The structure here was well-handled, reminding me a lot of a Michael Lewis article during the financial collapse. Langan has a good feel for how journalism of this type mixes personalities, politics, and facts. But it's all setup and no story. We get some world building, and then it's over, with no resolution except pessimism. Meh. (4) "A Borrowed Heart" by Deborah J. Ross: Ross starts with the trappings of urban fantasy transplanted into a Victorian world: supernatural creatures about, a protagonist who is a high-class prostitute, and sex and a sucubus by the second page. It evolves from there into a family drama and an investigation, always giving the reader the impression that a vampire will jump out at any moment. But the ending caught me entirely by surprise and was far more effective due to its departure from the expected path. Well done. (7) "Bright Moment" by Daniel Marcus: The conflict between terraforming and appreciation for the universe as we find it is an old story pattern in science fiction, and Marcus doesn't add much here. I think the story would have been stronger if he'd found a way to write the same plot with a pure appeal to environmental beauty without the typical stakes-raising. But he does sprinkle the story with a few interesting bits, including a pod marriage and a futuristic version of extreme sports as a way of communing with nature. (6) "The Corpse Painter's Masterpiece" by M. Rickert: This is typical of my reaction to a Rickert story: shading a bit too much towards horror for me, a bit too cryptic, well-written but not really my thing. It's about a corpse painter who does the work of an informal mortician, improving the appearance of bodies for their funerals, and the sheriff who brings him all the dead bodies. It takes an odd macabre twist, and I have no idea what to make of the ending. (4) "Aisle 1047" by Jon Armstrong: Armstrong is best known for a couple of novels, Grey and Yarn, which entangle their stories in the future of marketing and commerce. One may be unsurprised, then, that this short story is on similar themes, with the intensity turned up to the parody point. Tiffan3 is a department-store saleswoman, spouting corporate slogans and advertising copy while trying to push customers towards particular products. The story follows the escalation into an all-out brand war, fought with the bubbly short-cut propaganda of a thirty-second commercial. For me, it fell awkwardly between two stools: it's a little too over-the-top and in love with its own bizarre alternate world to be effective satire, but the world is more depressing than funny and the advertising copy is grating. More of a curiosity than a successful story, I think. (5) "Anise" by Chris DeVito: Stories that undermine body integrity and focus on the fascinated horror of violation of physical boundaries aren't generally my thing, so take that into account in this review. Anise's husband died, but that's not as much of a problem as it used to be. Medical science can resurrect people via a sort of permanent, full-body life support system, making them more cyborg than human. "Anise" is about the social consequences of this technology in a world where a growing number of people have a much different relationship with their body than the typical living person today. It's a disturbing story that is deeply concerned with the physical: sex, blood, physical intimacy in various different forms, and a twisted type of psychological abuse. I think fans of horror will like this more than I did, although it's not precisely horror. It looks at the way one's perception of self and others can change by passing through a profound physical transformation. (5) "Spider Hill" by Donald Mead: I liked this story a lot better. It's about witchcraft and farm magic, about family secrets, and a sort of coming-of-age story (for a girl rather than a boy, for once). The main character is resourceful, determined, but also empathetic and aware of the impact of her actions, which made her more fun to read about. I doubt I'll remember this for too long, but when skimming through it again for a review, I had fond memories of it. (6) "Where Have All the Young Men Gone?" by Albert E. Cowdrey: Cowdrey in his paranormal investigation mode, which I like better than his horror mode. For once, the protagonist isn't even a lower-class or backwoods character. Instead, he's a military historian travelling in Austria who runs across a local ghost story. This is a fairly straightforward ghost investigation that follows a familiar path (albeit to an unusual final destination), but Cowdrey is a good story-teller and I liked the protagonist. (7) "Overtaken" by Karl Bunker: This is the sort of story that delivers its moral with the force of a hammer. It's not subtle. But if you're in the right mood for that, it's one of the better stories of its type. It's about a long-journey starship, crew in hibernation, that's overtaken by a far newer and faster mechanized ship from Earth that's attempting to re-establish contact with the old ships. The story is a conversation between the ship AIs. Save this one until you're in the mood for an old-fashioned defense of humanity. (8) "Time and Tide" by Alan Peter Ryan: Another pseudo-horror story, although I think it's better classified as a haunting. A wardrobe recalls a traumatic drowning in the childhood of the protagonist. As these things tend to do in stories like this, reality and memory start blurring and the wardrobe takes on a malevolent role. Not my sort of thing. (3) "What We Found" by Geoff Ryman: Any new Geoff Ryman story is something to celebrate. This is a haunting story on the boundaries between the scientific method and tribal superstition, deeply entangled with the question of how one recovers from national and familial trauma. How can we avoid passing the evils and madness of one generation down to the next? Much of the story is about family trauma, told with Ryman's exceptional grasp of character, but the science is entangled in an ingenious way that I won't spoil. As with Air, this is in no way science fiction. The science here would have fascinating and rather scary implications for our world, but clearly is not how science actually works. But as an insight into politics, and into healing, I found it a startlingly effective metaphor. I loved every bit of this. By far the best story of the issue. (9) Rating: 7 out of 10

9 November 2013

Russ Allbery: Review: Fantasy & Science Fiction, July/August 2011

Review: Fantasy & Science Fiction, July/August 2011
Editor: Gordon van Gelder
Issue: Volume 121, No. 1 & 2
ISSN: 1095-8258
Pages: 258
Nothing of particular interest in the book reviews in this issue, although I was entertained to see Charles de Lint review a collection of the Prince Valiant strip. I have memories of that strip being one of the most boring works of art created by mankind. De Lint, of course, quite likes it. Our tastes seem to be very disjoint, although I have to admit that I've not read it collected and it may be more coherent and more interesting in that format. The science column in this issue, by Paul Doherty and Pat Murphy, deserves special mention. It's on roshambo (rock-paper-scissors), human difficulties with randomness, and strategy in roshambo competitions. It's also the first essay I've read that clearly explains how there can be strategy to roshambo, and why that strategy is worth studying. Even though the primary topic of the column is randomness, it's worth reading from the perspective of strategy in competitive human vs. human games. "Bronsky's Dates with Death" by Peter David: Anyone familiar with PAD's body of work will immediately expect a humorous story with some deeper thoughtful bits, and that's exactly what this is. Bronsky, the title character, is a man almost incapable of saying exactly what's on his mind, and what's on his mind is his eventual death. Not that it bothers him that much; he's just thinking a lot about it. But his incessant discussion of it certainly bothers the people around him. This leads to a few entertaining exchanges with his family, and then to more entertaining exchanges with Death. Or Deaths, as there appear to be several different kinds. I found the exact metaphysics a bit confused, but the ending was still touching and a bit funny. (7) "The Way It Works Out and All" by Peter S. Beagle: This is a reprint of a special fund-raising story about Avram Davidson, so a lot of it was lost on me given that I know almost nothing about Davidson and have yet to get to any of his novels I own. But even without that background, it's a diverting story of hidden and parallel worlds and unexpected explorations. There isn't all that much in the way of a plot, but it's a nice bit of characterization set against a fun SF twist. (6) "Less Stately Mansions" by Rob Chilson: This is a story about conservatism in life, about a farmer staying on his farm and resisting change, and about nostalgia, but I liked it much better than I normally like stories with those themes. It's set against a future world in which climate change is making life increasingly untenable. Humans are migrating into space colonies of various types, but Jacob refuses. This frustrates some parts of the family who want a piece of the substantial cash-out he's being offered for his farm, which of course makes Jacob even more stubborn. It's more of an elegy than a story, but I think it captures a particular stubborn mood, and a conscious decision to go with what one knows even if it doesn't have a long future, quite well. (7) "The Ants of Flanders" by Robert Reed: This is the novella of the issue, and, as you might expect from the author, it's thoughtful, meaty, and satisfying. At the start of the book, the planet is visited by an extraterrestrial ship (or ships it's not entirely clear at first). One of the people near one landing is Bloch, a huge teenager who has an odd lack of natural fear. He stays near the center of the story as Reed slowly develops a cosmology and a galactic political background that makes it clear humans may be incidental to everything that's happening. I liked this. It's a touch depressing in spots, and Bloch is a strange protagonist, but the cosmology is not the normal SF background and sparks some thoughts about how a status quo would be maintained by powers that don't care much about individual lives. The interlude with the leopard is nicely done, even if its significance is inobvious at first. (7) "Hair" by Joan Aiken: This is one of those weird Gothic horror stories about creepy families and half-explained supernatural events that some people love and that do nothing for me. (3) "The Witch of Corinth" by Steven Saylor: This is straight historical fantasy, featuring a Roman and his Greek tutor (heroes, apparently, of a series of historical mysteries) visiting the ruins of Corinth and encountering some bloody and dangerous local conflicts. It's slow and atmospheric, carried along by good characterization and description of ruins. It's not that much of a mystery the characters don't figure things out as much as stick around until the answer becomes obvious but it kept me entertained throughout a sizable story. Numerous elements of the story appear to be fantasy and then get other explanations, but there is a fantasy twist to the ending. (6) "Sir Morgravain Speaks of Night Dragons and Other Things" by Richard Bowes: This odd story is set among the knights of King Arthur on Avalon, where they sleep (mostly), awaiting their call to aid Britain again. Most of the story is told as one-sided dialogue from Morgravain, interspersed with some italic narration. Again, not much of a plot; the story, as such, is figuring out what Morgravain is doing and the point of his interactions with the other knights. I thought it was slight and oddly pointless, but I may have just missed the point. (4) "Someone Like You" by Michael Alexander: A time travel story, but one that's less about time travel per se than about examining and speculating about the alternate paths childhood could take and whether those changes produce different people. It takes some time to figure out what's going on, during which Alexander fills in the protagonist's past and the murder mystery that drives the tale. The time travel mechanism is blatantly hand-waved, making this more of a fantasy than an SF story, which matches the emphasis on emotion and psychology. It's not a bad story, and I think I see where the author was going with the slowly-constructed central conflict, but I still found it hard to take the conflict that seriously. One of the problems with time travel is that undermining of causality also undermines finality of decisions and consequences in ways that can rob stories of their punch. (6) "The Ramshead Algorithm" by KJ Kabza: The first-person protagonist is a well-respected and experienced fixer in a world of chaos and dimensional connection, a world with physics and inhabitants very much unlike ours. But the story doesn't spend much time there; his connection to his home earth is threatened, and he returns to try to stabilize it, which leads to the reader discovering that his family considers him a worthless appendage on a wealthy business family whose sole purpose in life is to stay out of their way. And his portal is rooted in a hedge maze that his father intends to demolish. This is one of those stories that gets more interestingly complex the deeper one gets into it. Ramshead's family is badly screwed up along multiple axes, but not without some hope of redemption. He's desperate and ineffective in his home universe, but more confident and capable when it comes to dealing with dimensional portal problems (although he still seems very young and relies on tools given to him by others). And there's always more going on than it first appears, and not as few obvious villains as it first appears. Good stuff, although I would have liked to understand more about Ramshead's world. (7) Rating: 7 out of 10

2 November 2013

Russ Allbery: Review: Fantasy & Science Fiction, May/June 2011

Review: Fantasy & Science Fiction, May/June 2011
Editor: Gordon van Gelder
Issue: Volume 120, No. 5 & 6
ISSN: 1095-8258
Pages: 258
The editorial in this issue is about the investigation into the troubling death of long-time contributor F. Gwynplain MacIntyre (which was not his real name). It's disturbing, but to me it underscores one of the things that I love about the Internet: people for whom life isn't working very well can still find an outlet, make friendships, and control how they choose to present themselves to the world on-line. That's something quite valuable, and part of why the pushes for "real names" always gives me pause. Somewhat thematically related, this issue also features a non-fiction essay by Maria E. Alonzo about her investigation of Jesse Francis McComas, her great-uncle but better known to the SF community as one of the founding editors of F&SF and co-editor of the famous classic anthology Adventures in Time and Space. This is mostly a curiosity, but it's fun to read about the sense of triumph in tracking down lost family history. This issue also features a Chris Moriarty book review column, always a plus, as well as a few positive reviews of obscure superhero movies by Kathi Maio (plus the required grumbling about a more mainstream film). "The Final Verse" by Chet Williamson: This is more of a horror story than I would normally like, but I got pulled into the investigation of an old bluegrass song and the guesswork and footwork required to track down where it came from. Williamson does a good job with the tone and first-person narration, and the degree to which the protagonist cares about the song to the exclusion of the horrific happenings of the story blunts the horror. Not quite my thing, but I thought it was well-done and played well with the possible meanings of song lyrics. (6) "Stock Photos" by Robert Reed: This is well-written, like nearly all Reed stories, but it lacked enough clues for the reader for me. It's a very short story about a man who's out mowing his lawn when approached by two strangers who apparently want to take photographs of him for stock image collections. Then things get rather weird, but without any explanation, and the ending lost me completely. Frustrating. (It is partially explained by the later "The Road Ahead" story in this same issue.) (4) "The Black Mountain" by Albert E. Cowdrey: From one of F&SF's most reliable story-tellers to another, and this is a more typical story. Cowdrey offers an abandoned and very strange cathedral for an obscure religion, a conflict over a development project, and some rather creepy results, all told in Cowdrey's entertaining fashion. Some places you just don't mess with. (6) "Agent of Change" by Steven Popkes: Told Dos-Passos-style with news excerpts, web sites, and the transcript of an emergency committee, this story shows the discovery of Godzilla, or something akin to Godzilla, in the Pacific, where it's destroying whaling vessels. I do like this style of storytelling, and here it mixes well with humor and a bit of parody as Popkes shows how each different news outlet puts its own recognizable spin on the story. The story isn't particularly memorable, and it doesn't end so much as just stop, but it was fun. (7) "Fine Green Dust" by Don Webb: This story is dedicated to Neal Barrett, which will give SFF short story readers a warning of weirdness to come. In a near future where global warming as continued to make summers even more miserable, the protagonist happens across a naked woman painted green. The green turns out to be a sun block that claims to assist humans in metamorphosis into animals. Most of the story is the protagonist trying to decide what to think of that, interspersed with staring at his neighbor's naked daughter. It's mildly amusing if you don't think about it too much and don't mind the rather prominent male gaze. (5) "Rampion" by Alexandra Duncan: The novella of the story, this is set in Muslim Spain some time during the long fights between Muslims and Christians in the north. It's told as two parallel stories: one telling the protagonist's first meeting with his love, and the second following him as a blind man, some time later, deciding whether, and how, to re-engage with the world. The style feels like fantasy, but there's very little overt fantasy here, and the story could be read as historical adventure. It's good adventure, though; conventional in construction, but with some romance and some drama and a good ending. (7) "Signs of Life" by Carter Scholz: This is to science fiction what "Rampion" is to fantasy: not really SF in the classic sense, but fiction about the process of science. The protagonist works on gene sequencing and is mildly obsessed with a visualization of junk DNA in an attempt to find patterns in it. Like a lot of fiction about science, it's primarily concerned with office politics, grant funding, and an awful boss. There is a faint touch of the supernatural, but that strand of the story doesn't amount to much. There's a happy ending of sorts, but the story left me with a bad taste in my mouth, and I'd completely forgotten it by the time I sat down to write this review. (4) "Starship Dazzle" by Scott Bradfield: I've never seen much in Bradfield's ongoing series of stories about Dazzle, the talking dog. In this one, he's sent via rocket on a one-way trip into outer space and ends up making a bizarre sort of first contact. Like the other Dazzle stories, it's full of attempts at humor that don't really work for me, even though you'd think I'd be sympathetic to the mocking of our commercialization of everything. The ending is just silly, and not in a good way. (3) "The Old Terrologist's Tale" by S.L. Gilbow: I love the setup for this story. It's set in some sort of far future in which terraforming has become routine, and a group of people are telling each other stories over drinks. The first-person protagonist is a terrologist, someone who designs planets (and the technology is available to do this almost from scratch). The conversation is taking a turn towards the humiliating, with a politician belittling the work of terrologists, when an old terrologist who has been listening quietly starts telling a story about designing worlds, both mundane and dangerously beautiful. Gilbow does a great job here capturing blithe self-importance, the habit of belittling other people's technical work, and revenge via storytelling with a nasty barb. This was my favorite story of the issue. (7) "Altogether Elsewhere, Vast Herds of Reindeer" by Ken Liu: This is a rather odd but quite touching story about mothers, daughters, nature, connection, and uploading. It's set after a singularity, in a time when all humans are uploaded into computers and exploring higher dimensions, digital natives in a much deeper sense than is meant today. But Rene 's mother is an Ancient, from before the singularity and still three-dimensional, and she wants to spend some time with her daughter. That leads to a memorable moment of connection, without pulling Rene entirely out of her father's world. Well done. (7) "The Road Ahead" by Robert Reed: Two Reed stories in one issue! And this one is a sequel to "Stock Photos" from earlier, since apparently I wasn't the only one who found it hopelessly confusing. It provides some backstory and makes a bit more sense of the first story, and that also makes it a more interesting story in its own right. The stock photo concept wasn't entirely a lie, as I had thought it was after the first story. There is analysis, anticipation, and trends behind who the pair take pictures of. But this story explores some internal tension, some conflict between them and some knowledge that the woman has that the man doesn't. And in the process it makes everything creepier, but also more interesting, and provides a hint at a really dark way of viewing the news media. I would say that this salvages "Stock Photos," except that I don't think "Stock Photos" is necessary now that one can read this story. (7) "Music Makers" by Kate Wilhelm: This is another story about investigation of the history of music, mingled with the supernatural, but unlike the story that opened this issue, it's not horror. Rather, it's a gentle and sweet fantasy about the power of music and benevolent ghosts and a community coming together. It's a positive and happy note on which to end the issue. (6) Rating: 6 out of 10

10 September 2013

Julien Danjou: OpenStack Ceilometer Havana-3 milestone released

Last week, the third and last milestone of the Havana development branch of Ceilometer has been released and is now available for testing and download. This means the end of the OpenStack Havana development time is coming, and that the features are now frozen. New features
Eleven blueprints have been implemented as you can see on the release page. That's one more than during Havana-2, but it's less than was planned initially, though we had a pretty high score considering the size of our contributors team. I'm going to talk through some of them here, that are the most interesting for users. Bug fixes Fifty-six bugs were fixed, though most of them might not interest you so I won't elaborate too much on that. Go read the list if you are curious. Toward our final Havana release With the feature freeze in place, we're now focusing on fixing bugs and improving documentation. I'll try to make sure we'll get there without too much trouble for the 17th October 2013. Stay tuned!

22 July 2013

Daniel Pocock: Winning at any cost

It's not every day that a student messing around with keystroke loggers comes to fame through slashdot. Nonetheless, systematically rigging an election and getting sentenced to 12 months in a dorm with bars has helped raise 22 year old Matthew Weaver's profile well above that of the average script kiddie. Now let's stop and reflect on poor Weaver's future. You may be thinking that with an exchange program like this on his academic record he won't be so popular with employers. Given that he was busted by campus security rather than the FBI he won't even attract the interest of those companies who hire ex-hackers. So where could he go? How is it done in Australia? Not too long ago, when I was a student myself, one of our prominent universities was subjected to a very similar scam. Four members of the Tin Tin for NUS ticket at La Trobe University were implicated in stuffing the ballot the old fashioned way. The incidents even share the characteristics of chronic stupidity: just as Weaver had been caught voting for himself 259 times from the same IP address in a campus computer lab, team Tin Tin had tried to hand their bag of manipulated postal votes directly to the deputy returning officer rather than discretely posting them through the internal mail. According to an official report by the Deputy Returning Officer, Karsten Haley, all four candidates were charged with Dishonest Conduct and Interfering with Ballot Papers. Unfortunately, the report notes that
La Trobe University SRC Electoral Regulations do not empower the Returning Officer or Deputy to enforce charges or disciplinary procedures and the charges were never faced by the accused.
Given the seriousness of the matter, Haley did not give up his attempts to hold them to account. He escalated it to the Dean of the college and then to the University Secretary. He reports that "their disinterest was extraordinary" and that nobody would involve the police. Young Labor suspended Just over a year later, in 1997, the ALP's youth division for the state of Victoria, Young Labor, was suspended after attempts to rig the ballot to elect the Young Labor leadership team. The guilty parties were never publicly named. Nobody was formally suspended or expelled and this simply left them with more time on their hands to invest their energy in other elections. The suspension of Victorian Young Labor remained in effect for a number of years. The specific allegations about the Young Labor ballot suggest that those people particularly keen to win had printed fake student cards and given them to stooges who would impersonate other Young Labor members who had not attended to vote in person. Where are they now? It is no co-incidence that these students were (and still are) members of Labor Unity, a powerful faction within Australia's ruling Labor Party, the ALP. Most political organisations would presumably express concern about these allegations. The ALP does things differently. One of the students who withdrew his nomination in La Trobe, Mr Larocca, subsequently became Mayor in the City of Moreland, one of the ALP's strongholds. Even more remarkably from an outsider's viewpoint, another of these figures, Stephen Donnelly, is currently employed as the Assistant State Secretary of the ALP in Victoria. Communications like this newsletter reveal that he is one of the key figures in the party's pre-selection process. He has recently been appointed to direct the ALP's 2013 federal election campaign for the state of Victoria. Another co-incidence On the same weekend that Weaver was in the news for his antics, Donnelly's latest employer, the ALP's Victorian branch, was conducting pre-selection ballots to choose candidates for the upcoming federal election. So it's no surprise that Monday's newspaper headlines report fresh allegations of voting irregularities. Sadly, I've seen some of Labor Unity's bad behavior first hand. About 10 years ago I was living in South Melbourne, which is in the federal electoral district of Melbourne Ports. A young female friend of mine, a member of the local Elwood branch of the ALP, had spent election day handing out brochures for an ALP candidate in a marginal seat rather than assisting the controversial local ALP candidate, Michael Danby. A few days later I was witness to an incident where Danby aggressively confronted this young woman and demanded to know why he hadn't seen her handing out his own leaflets on polling day. He stood within centimeters of her and was literally looking down on her as he demanded some kind of apology to sooth his bruised ego. She looked terrified and barely responded. Within moments one of his handlers approached and physically moved Danby away from this young woman, I dare to think where things would have gone otherwise. Eye for talent Remarkably, at the same time, the infamous Stephen Donnelly had started shadowing Danby in his movements about the district. Fresh out of university, his talents had been recognised by Danby and he was employed in Danby's office, enabling him to continue honing his skills on a full-time basis with a tax-payer funded salary. What a remarkable contrast to the story of Weaver. Can anybody imagine a US congressman collecting Weaver from the prison gates and deploying him to an office on Capitol Hill? The biggest bankruptcy in student history Around the same time, Donnelly's Student Unity, the student arm of Labor Unity were successful in taking over the student union of my own campus, the University of Melbourne. Not long after I graduated I heard that they had been accused of skimming off $1 million from catering providers and a high-risk $46 million property transaction that put the organisation into liquidation. Unlike Mr Weaver, who's scheme at Cal State barely got off the ground, none of those involved in the Melbourne University incident has faced criminal proceedings. One ALP figure, Andrew Landeryou, spent several months in Costa Rica while wanted for questioning in the Supreme Court. His wife has just been endorsed for a seat in the Senate with support from various Labor Unity figures including Danby. The Gillard questions In 1996, around the same time that Donnelly & Co. were romping around student unions learning the tricks of the political trade, a lawyer quietly departed from the firm Slater and Gorden after an internal investigation into a property transaction linked to a union slush fund. Like Donnelly, this lawyer's next move was to take employment in the office of a Labor Party MP. More recently she was backed by Labor Unity to become Prime Minister. The union slush fund remains under investigation, frustrated by the disappearance of documents. The $60 million heist Recently I blogged about Gillard and Abbott, leaders of the two main political parties in Australia, agreeing to take $60 million of taxpayer money to fund their parties' campaigns in the upcoming federal election, giving themselves an obscenely unfair unadvantage over all other contestants. Where would that money end up? In the case of the ALP, does it appear likely that figures like the Victorian ALP's federal campaign director, Mr Donnelly, would be involved in the expenditure? National shame With this background, it becomes easier to understand the quality (or lack of it) in Australia's national leadership. When you consider that the generation responsible for the La Trobe incident, the Young Labor suspension and the MUSU bankruptcy are now growing into positions of greater responsibility in the ALP it leaves me feeling the quality of leadership is only going to get a lot worse before it starts getting better. For example, the recent incident where coloured people were fed to the sharks has nothing to do with the worldwide refugee crisis and everything to do with maintaining the dumbed-down level of political discourse that Labor Unity thugs and their followers can cope with. Real issues like climate change and energy policy, for example, appear to be beyond the pay grade of Australia's political class Ranjini - coloured, indefinite detention It is startling that up to her own recent demise, Gillard herself had repeatedly begged the public to stop asking questions about her own past and remember that Labor politicians are innocent until proven guilty - yet she had a pregnant coloured woman thrown into a concentration camp on unfounded fears about "national security". No evidence has ever been presented that poor Ranjini committed a crime, but the houses bought with money from trade unions, transactions handled through Gillard's own office, seem to be as solid as bricks and mortar. If only poor Matthew Weaver had been an Australian, how much further would his star have risen? Update: please sign the petition at change.org asking La Trobe university to re-examine the report and refer it formally to the police. If you are concerned about the plight of poor Ranjini and other people subject to Australia's domestic rendition program, please take a moment to see Letters for Ranjini

22 January 2013

Russ Allbery: Review: Fantasy & Science Fiction, March/April 2011

Review: Fantasy & Science Fiction, March/April 2011
Editor: Gordon van Gelder
Issue: Volume 120, No. 3 & 4
ISSN: 1095-8258
Pages: 258
Charles de Lint's book review column sticks with the sorts of things he normally reviews: urban and contemporary fantasy and young adult. Predictably, I didn't find that much of interest. But I was happy to see that not all the reviews were positive, and he talked some about how a few books didn't work. I do prefer seeing a mix of positive and negative (or at least critical) reviews. James Sallis's review column focuses entirely on Henry Kuttner and C.L. Moore (by way of reviewing a collection). I'm always happy to see this sort of review. But between that and de Lint's normal subject matter, this issue of F&SF was left without any current science fiction reviews, which was disappointing. Lucius Shepard's movie review column features stunning amounts of whining, even by Shepard's standards. The topic du jour is how indie films aren't indie enough, mixed with large amounts of cane-shaking and decrying of all popular art. I find it entertaining that the F&SF film review column regularly contains exactly the sort of analysis that one expects from literary gatekeepers who are reviewing science fiction and fantasy. Perhaps David Langford should consider adding an "As We See Others" feature to Ansible cataloging the things genre fiction fans say about popular movies. "Scatter My Ashes" by Albert E. Cowdrey: The protagonist of this story is an itinerant author who has been contracted to write a family history (for $100K, which I suspect is a bit of tongue-in-cheek wish fulfillment) and has promptly tumbled into bed with his employer. But he is somewhat serious about the writing as well, and is poking around in family archives and asking relatives about past details. There is a murder (maybe) in the family history, not to mention some supernatural connections. Those familiar with Cowdrey's writing will recognize the mix of historical drama, investigation, and the supernatural. Puzzles are, of course, untangled, not without a bit of physical danger. Experienced fantasy readers will probably guess at some of the explanation long before the protagonist does. Like most Cowdrey, it's reliably entertaining, but I found it a bit thin. (6) "A Pocketful of Faces" by Paul Di Filippo: Here's a bit of science fiction, and another mystery, this time following the basic model of a police procedural. The police in this case are enforcing laws around acceptable use of "faces" in a future world where one can clone someone's appearance from their DNA and then mount it on a programmable android. As you might expect from that setup, the possibilities are lurid, occasionally disgusting, and inclined to give the police nightmares. After some scene-setting, the story kicks in with the discovery of the face of a dead woman who, at least on the surface, no one should have any motive to clone. There were a few elements of the story that were a bit too disgusting for me, but the basic mystery plot was satisfying. I thought the ending was a let-down, however. Di Filippo tries to complicate the story and, I thought, went just a little too far, leaving motives and intent more confusing than illuminating. (6) "The Paper Menagerie" by Ken Liu: Back to fantasy, this time using a small bit of magic to illustrate the emotional conflicts and difficulties of allegiance for second-generation immigrants. Jack is the son of an American farther and a Chinese mother who was a mail-order bride. He's young at the start of the story and struggling with the embarassment and humiliation that he feels at his mother's history and the difficulties he has blending in with other kids, leading to the sort of breathtaking cruelty that comes so easily from teenagers who are too self-focused and haven't yet developed adult empathy. I found this very hard to read. The magic is beautiful, personal, and very badly damaged by the cruelty in ways that can never really be fixed. It's a sharp reminder of the importance of being open-hearted, but it's also a devastating reminder that the lesson is normally learned too late. Not the story to read if you're prone to worrying about how you might have hurt other people. (6) "The Evening and the Morning" by Sheila Finch: This long novella is about a third of the issue and is, for once, straight science fiction, a somewhat rare beast in F&SF these days. It's set in the far future, among humans who are members of the Guild of Xenolinguists and among aliens called the Venatixi, and it's about an expedition back to the long-abandoned planet of Earth. I had a lot of suspension of disbelief problems with the setup here. While Earth has mostly dropped out of memory, there's a startling lack of curiosity about its current condition among the humans. Finch plays some with transportation systems and leaves humanity largely dependent on other races to explain the failure to return to Earth, but I never quite bought it. It was necessary to set up the plot, which is an exploration story with touches of first contact set on an Earth that's become alien to the characters, but it seemed remarkably artificial to me. But, putting that aside, I did get pulled into the story. Its emotional focus is one of decline and senescence, a growing sense of futility, that's halted by exploration, mystery, and analysis. The question of what's happened on Earth is inherently interesting and engaging, and the slow movement of the story provides opportunities to build up to some eerie moments. The problem, continuing a theme for this issue, is the ending. Some of the reader's questions are answered, but most of the answers are old, well-worn paths in science fiction. The emotional arc of the story is decidedly unsatisfying, at least to me. I think I see what Finch was trying to do: there's an attempted undermining of the normal conclusion of this sort of investigation to make a broader point about how to stay engaged in the world. But it lacked triumph and catharsis for me, partly because the revelations that we get are too pedestrian for the build-up they received. It's still an interesting story, but I don't think it entirely worked. (6) "Night Gauntlet" by Walter C. DeBill, Jr., et al.: The full list of authors for this story (Walter C. DeBill, Jr., Richard Gavin, Robert M. Price, W.H. Pugmire, Jeffrey Thomas, and Don Webb) provides one with the first clue that it's gone off the rails. Collaborative storytelling, where each author tries to riff off the work of the previous author while spinning the story in a different direction, is something that I think works much better orally, particularly if you can watch facial expressions while the authors try to stump each other. In written form, it's a recipe for a poorly-told story. That's indeed what we get here. The setup is typical Cthulhu mythos stuff: a strange scientist obsessed with conspiracy theories goes insane, leaving behind an office with a map of linkages between apparently unrelated places. The characters in the story also start going insane for similar reasons, leading up to a typical confrontation with things man was not meant to know, or at least pay attention to. If you like that sort of thing, you may like this story better than I did, but I thought it was shallow and predictable. (3) "Happy Ending 2.0" by James Patrick Kelly: More fantasy, this time of the time travel variety. (I call it fantasy since there's no scientific explanation for the time travel and it plays a pure fantasy role in the story.) That's about as much as I can say without giving away the plot entirely (it's rather short). I can see what Kelly was going for, and I think he was largely successful, but I'm not sure how to react to it. The story felt like it reinforced some rather uncomfortable stereotypes about romantic relationships, and the so-called happy ending struck me as the sort of situation that was going to turn very nasty and very uncomfortable about five or ten pages past where Kelly ended the story. (5) "The Second Kalandar's Tale" by Francis Marion Soty: The main question I have about this story is one that I can't answer without doing more research than I feel like doing right now: how much of this is original to Soty and how much if it is straight from Burton's translation of One Thousand and One Nights. Burton is credited for the story, so I suspect quite a lot of this is from the original. Whether one would be better off just reading the original, or if Soty's retelling adds anything substantial, are good questions that I don't have the background to answer. Taken as a stand-alone story, it's not a bad one. It's a twisting magical adventure involving a djinn, a captive woman, some rather predictable fighting over the woman, and then a subsequent adventure involving physical transformation and a magical battle reminiscent of T.H. White. (Although I have quite likely reversed the order of inspiration if as much of this is straight from Burton as I suspect.) Gender roles, however, are kind of appalling, despite the presence of a stunningly powerful princess, due to the amount of self-sacrifice expected from every woman in the story. Personally, I don't think any of the men in the story are worth anywhere near the amount of loyalty and bravery that the women show. Still, it was reasonably entertaining throughout, in exactly the way that I would expect a One Thousand and One Nights tale to be. Whether there's any point in reading it instead of the original is a question I'll leave to others. (7) "Bodyguard" by Karl Bunker: This is probably the best science fiction of the issue. The first person protagonist is an explorer living with an alien race, partly in order to flee the post-singularity world of uploaded minds and apparent stagnation that Earth has become. It's a personal story that uses his analysis of alien mindsets (and his interaction with his assigned alien bodyguard) to flesh out his own desires, emotional background, and reactions to the world. There are some neat linguistic bits here that I quite enjoyed, although I wish they'd been developed at even more length. (The alien language is more realistic than it might sound; there are some human languages that construct sentences in a vaguely similar way.) It's a sad, elegiac story, but it grew on me. (7) "Botanical Exercises for Curious Girls" by Kali Wallace: One has to count this story as science fiction as well, although for me it had a fantasy tone because the scientific world seems to play by fantasy rules from the perspective of the protagonist. Unpacking that perspective is part of the enjoyment of the story. At the start, she seems to be a disabled girl who is being cared for by a strange succession of nurses who vary by the time of day, but as the story progresses, it becomes clear that something much stranger is going on. There are moments that capture a sense of wonder, reinforced by the persistantly curious and happy narrative voice, but both are undercut by a pervasive sense of danger and dread. This is a light story written about rather dark actions. My biggest complaint with the story is that it doesn't so much end as wander off into the sunset. It set up conflict within a claustrophobic frame, so I can understand the thematic intent of breaking free of that frame, but in the process I felt like the story walked away from all of the questions and structure that it created and ended in a place that felt less alive with potential than formless and oddly pointless. I think I wanted it to stay involved and engaged with the environment it had created. (6) "Ping" by Dixon Wragg: I probably should just skip this, since despite the table of contents billing and the full title introduction, it's not a story. It's a two-line joke. But it's such a bad two-line joke that I had to complain about it. I have no idea why F&SF bothered to reprint it. (1) "The Ifs of Time" by James Stoddard: This certainly fits with the Arabian Nights story in this issue. The timekeeper of a vast and rambling castle (think Gormenghast taken to the extreme) wanders into a story-telling session in a distant part of the castle. The reader gets to listen to four (fairly good) short stories about time, knowledge, and memory, told in four very different genres. All of this does relate to why the timekeeper is there, and the frame story is resolved by the end, but the embedded stories are the best part; each of them is interesting in a different way, and none of them outlast their welcome. This was probably the strongest story of this issue. (7) Rating: 6 out of 10

21 May 2012

Johannes Schauer: sisyphus wins ICRA 2012 VMAC

Sisyphus is a piece of software that I wrote as a member of a team from Jacobs University led by Prof. Dr. Andreas N chter. It managed to place our team first in this year's IEEE ICRA 2012 Virtual Manufacturing Automation Competition in all three rounds. The goal was, to stack a given set of boxes of different length, height and width on a pallet in a way that achieved optimal volume utilization, center of mass and interlock of the boxes. Besides the cartesian placement of a box on the pallet, the only other degree of freedom was a 90 rotation of the box around a vertical axis. Since the arrangement of boxes into a three dimensional container is NP hard (three dimensional orthogonal knapsack), I decided for a heuristic for an approximate solution. The premise is, that there are many boxes of equal height which was the case in the test cases that were available from the 2011 VMAC. Given this premise, my heuristic was, to arrange the boxes into layers of equal height and then stack these layers on top of each other. A set of boxes that would be left over or too little from the start to form its own full layer, would then be stacked on the top of the completed layers. There is a video of how this looked like. My code is now online on github and it even documented for everybody who is not me (or a potential future me of course). This blog post is about the "interesting" parts of sisyphus. You can read about the overall workings of it in the project's README. Python dict to XML and XML to Python dict The evaluation program for the challenge is reading XML files and the pallet size and the list of articles with their sizes are also given in XML format. So I had to have a way to easily read article information from XML and to easily dump my data into XML format. Luckily, all the XML involved was not making use of XML attributes at all, so the only children a node had, where other nodes. Thus, the whole XML file could be represented as an XML dictionary with keys being tagnames and the values being other dictionaries or lists or strings or integers. The code doing that uses xml.etree.ElementTree and turns out to be very simple: 
from xml.etree import ElementTree

def xmltodict(element):
    def xmltodict_handler(parent_element):
        result = dict()
        for element in parent_element:
            if len(element):
                obj = xmltodict_handler(element)
            else:
                obj = element.text
            if result.get(element.tag):
                if hasattr(result[element.tag], "append"):
                    result[element.tag].append(obj)
                else:
                    result[element.tag] = [result[element.tag], obj]
            else:
                result[element.tag] = obj
        return result
    return  element.tag: xmltodict_handler(element) 

def dicttoxml(element):
    def dicttoxml_handler(result, key, value):
        if isinstance(value, list):
            for e in value:
                dicttoxml_handler(result, key, e)
        elif isinstance(value, basestring):
            elem = ElementTree.Element(key)
            elem.text = value
            result.append(elem)
        elif isinstance(value, int) or isinstance(value, float):
            elem = ElementTree.Element(key)
            elem.text = str(value)
            result.append(elem)
        elif value is None:
            result.append(ElementTree.Element(key))
        else:
            res = ElementTree.Element(key)
            for k, v in value.items():
                dicttoxml_handler(res, k, v)
            result.append(res)
    result = ElementTree.Element(element.keys()[0])
    for key, value in element[element.keys()[0]].items():
        dicttoxml_handler(result, key, value)
    return result

def xmlfiletodict(filename):
    return xmltodict(ElementTree.parse(filename).getroot())

def dicttoxmlfile(element, filename):
    ElementTree.ElementTree(dicttoxml(element)).write(filename)

def xmlstringtodict(xmlstring):
    return xmltodict(ElementTree.fromstring(xmlstring))

def dicttoxmlstring(element):
    return ElementTree.tostring(dicttoxml(element))
Lets try this out: 
>>> from util import xmlstringtodict, dicttoxmlstring
>>> xmlstring = "<foo><bar>foobar</bar><baz><a>1</a><a>2</a></baz></foo>"
>>> xmldict = xmlstringtodict(xmlstring)
>>> print xmldict
 'foo':  'baz':  'a': ['1', '2'] , 'bar': 'foobar' 
>>> dicttoxmlstring(xmldict)
'<foo><baz><a>1</a><a>2</a></baz><bar>foobar</bar></foo>'
The dict container doesnt preserve order, but as XML doesnt require that, this is also not an issue. Arranging items in layers When it was decided, that I wanted to take the layered approach, it boiled down the 3D knapsack problem to a 2D knapsack problem. The problem statement now was: how to best fit small rectangles into a big rectangle? I decided for a simple and fast approach as it is explained in Jake Gordon's blog article. There is a demo of his code and should the site vanish from the net, the code is on github. This solution seemed to generate results that were "good enough" while simple to implement and fast to execute. If you look very hard, you can still see some design similarities between my arrange_spread.py and his packer.js code. Jake Gordon got his idea from Jim Scott who wrote an article of arranging randomly sized lightmaps into a bigger texture. There is also an ActiveState Code recipe from 2005 which looks very similar to the code by Jake Gordon. The posts of Jake Gordon and Jim Scott explain the solution well, so that I dont have to repeat it. Should the above resources go offline, I made backups of them here and here. There is also a backup of the ActiveState piece here. Spreading items out The algorithm above would cram all rectangles into a top-left position. As a result, there would mostly be space at the bottom and left edge of the available pallet surface. This is bad for two reasons:
  1. the mass is distributed unequally
  2. articles on the layer above at the bottom or left edge, are prone to overhang too much so that they tumble down
Instead all articles should be spread over the available pallet area, creating small gaps between them instead big spaces at the pallet borders. Since articles were of different size, it was not clear to me from the start what "equal distribution" would even mean because it was obvious that it was not as simple as making the space between all rectangles equal. The spacing had to be different between them to accommodate for differently sized boxes. The solution I came up with, made use of the tree structure, that was built by the algorithm that arranged the rectangles in the first place. The idea is, to spread articles vertically first, recursively starting with the deepest nodes and spreading them out in their parent rectangle. And then spreading them horizontally, spreading the upper nodes first, recursively resizing and spreading child nodes. The whole recursion idea created problems of its own. One of the nicest recursive beauty is the following function: 
def get_max_horiz_nodes(node):
    if node is None or not node['article']:
        return [], []
    elif node['down'] and node['down']['article']:
        rightbranch, sr = get_max_horiz_nodes(node['right'])
        rightbranch = [node] + rightbranch
        downbranch, sd = get_max_horiz_nodes(node['down'])
        ar = rightbranch[len(rightbranch)-1]['article']
        ad = downbranch[len(downbranch)-1]['article']
        if ar['x']+ar['width'] > ad['x']+ad['width']:
            return rightbranch, sr+[downbranch[0]]
        else:
            return downbranch, sd+[rightbranch[0]]
    else:
        rightbranch, short = get_max_horiz_nodes(node['right'])
        return [node] + rightbranch, short
get_max_horiz_nodes() traverses all branches of the tree that node has below itself and returns a tuple containing the list of nodes that form the branch that stretches out widest plus the list of nodes that are in the other branches (which are shorter than the widest). Another interesting problem was, how to decide on the gap between articles. This was interesting because the number resulting of the subtraction of the available length (or width) and the sum of the articles lengths (or widths), was mostly not divisible by the amount of gaps without leaving a rest. So there had to be an algorithm that gives me a list of integers, neither of them differing by more than one to any other, that when summed up, would give me the total amount of empty space. Or in other words: how to divide a number m into n integer pieces such that each of those integers doesnt differ more than 1 from any other. Surprisingly, generating this list doesnt contain any complex loop constructs: 
>>> m = 108 # total amount
>>> n = 7 # number of pieces
>>> d,e = divmod(m, n)
>>> pieces = (e)*[(d+1)]+(n-e)*[d]
>>> print pieces
[16, 16, 16, 15, 15, 15, 15]
>>> sum(pieces) == m
True
>>> len(pieces) == n
True
You can test out the algorithms that arrange rectangles and spread them out by cloning the git and then running: 
PYTHONPATH=. python legacy/arrange_spread.py
The results will be svg files test1.svg and test2.svg, the latter showing the spread-out result. Here is an example how the output looks like (without the red border which is drawn to mark the pallet border): arrange_spread2.py contains an adaption of arrange_spread.py for the actual problem. Permutations without known length When creating a layer out of articles of same height, then there are four strategies that I can choose from. It is four because there are two methods that I can either use or not. I can rotate the article by 90 per default or not and I can rotate the pallet or not. So every time that I build a new layer, there are those four options. Depending on which strategy I choose, there is a different amount of possible leftover articles that did not fit into any layer. The amount is different because each strategy is more or less efficient. To try out all combinations of possible layer arrangements, I have to walk through a tree where at each node I branch four times for each individual strategy. Individual neighboring nodes might be the same but this outcome is unlikely due to the path leading to those neighboring nodes being different. To simplify, lets name the four possible strategies for each layers 0, 1, 2 and 3. I now want an algorithm that enumerates through all possible permutations of those four numbers for "some" length. This is similar to counting. And the itertools module comes with the product() method that nearly does what I want. For example, should I know that my tree does not become deeper than 8 (read: no more than eight layers will be built), then I can just run: 
>>> for i in itertools.product([0,1,2,3], repeat=8):
...     print i
...
(0,0,0,0,0,0,0,0)
(0,0,0,0,0,0,0,1)
(0,0,0,0,0,0,0,2)
(0,0,0,0,0,0,0,3)
(0,0,0,0,0,0,1,0)
(0,0,0,0,0,0,1,1)
(0,0,0,0,0,0,1,2)
(0,0,0,0,0,0,1,3)
This would work if the number of layers created with each strategy was the same. But as each strategy behaves differently depending on the input, it cannot be known before actually trying out a sequence of strategies, how many layers it will yield. The strategy (0,0,0,0,0,0,0,0) might create 7 layers, resulting in (0,0,0,0,0,0,0,1), (0,0,0,0,0,0,0,2) and (0,0,0,0,0,0,0,3) yielding the same output as only the first 7 strategies count. This would create duplicates which I should not waste cpu cycles on later. It might also be that (0,0,0,0,0,0,1,0) turns out to be a combination of strategies that creates more than 8 layers in which case the whole thing fails. So what I need is a generator, that gives me a new strategy depending on how often it is asked for one. It should dynamically extend the tree of possible permutations to accommodate for any size. Since the tree will become humongous (4^11 = 4194304), already traversed nodes should automatically be cleaned so that only the nodes that make the current list of strategies stays in memory at any point in time. This sounded all complicated which made me even more surprised by how simple the solution was in the end. Here a version of the algorithm that could easily be ported to C: 
class Tree:
    def __init__(self, branch_factor):
        self.branch_factor = branch_factor
        self.root =  "value": None, "parent": None, "children": [] 
        self.current = self.root

    def next(self):
        if not self.current["children"]:
            self.current["children"] = [ "value":val, "parent":self.current, "children":[]  for val in range(self.branch_factor)]
        self.current = self.current["children"][0]
        return self.current["value"]

    def reset(self):
        if self.current["parent"]:
            self.current["parent"]["children"].pop(0)
        else:
            return False
        if self.current["parent"]["children"]:
            self.current = self.root
            return True
        else:
            self.current = self.current["parent"]
            return self.reset()

    def __str__(self):
        return str(self.root)
It would be used like this: 
>>> tree = Tree(4)
>>> print tree.next(), tree.next(), tree.next()
>>> while tree.reset():
...     print tree.next(), tree.next(), tree.next()
Which would be equivalent to calling itertools.product([1,2,3,4], 3). The special part is, that in each iteration of the loop I can call tree.next() an arbitrary amount of times, just how much it is needed. Whenever I cannot generate an additional layer anymore, I can call tree.reset() to start a new permutation. For my code I used a python specific version which is a generator: 
def product_varlength(branch_factor):
    root =  "value": None, "parent": None, "children": [] 
    current = root
    while True:
        if not current["children"]:
            current["children"] = [ "value":val, "parent":current, "children":[]  for val in range(branch_factor)]
        current = current["children"][0]
        if (yield current["value"]):
            while True:
                if current["parent"]:
                    current["parent"]["children"].pop(0)
                else:
                    return
                if current["parent"]["children"]:
                    current = root
                    break
                else:
                    current = current["parent"]
It is used like this: 
it = product_varlength(4)
print it.next(), it.send(False), it.send(False)
while True:
    print it.send(True), it.send(False), it.send(False)
Again, the expression in the loop can have any number of it.send(False). The first it.send(True) tells the generator to do a reset.

24 March 2012

Dirk Eddelbuettel: Initial release 0.1.0 of package RcppSMC

Hm, I realized that I announced this on Google+ (via Rcpp) as well as on Twitter, on the r-packages list, wrote a new and simple web page for it, but had not put it on my blog. So here is some catching up. Sequential Monte Carlo / Particle Filter is a (to quote the Wikipedia page I just linked to) sophisticated model estimation technique based on simulation. They are related to both Kalman Filters, and Markov Chain Monte Carlo methods. Adam Johansen has a rather nice set of C++ classes documentated in his 2009 paper in the Journal of Statistical Software (JSS). I started to play with these classes and realized that, once again, this would make perfect sense in an R extension built with the Rcpp package by Romain and myself (and in JSS too). So I put a first prototype onto R-Forge and emailed Adam who, to my pleasant surprise, was quite interested. And a couple of emails, and commits later, we are happy to present a very first release 0.1.0. I wrote a few words on a RcppSMC page on my website where you can find a few more details. But in short, we already have example functions demonstrating the backend classes by reproducing examples from
Johansen (2009)
and his example 5.1 via pfLineartBS() for a linear bootstrap example;
Doucet, Briers and Senecal (2006)
and their (optimal) block-sampling particle filter for a linear Gaussian model (serving as an illustration as the setup does of course have an analytical solution) via the function blockpfGaussianOpt()
Gordon, Salmond and Smith (1993)
and their ubiqitous nonlinear state space model via the function pfNonlinBS().
And to illustrate just why Rcpp is so cool for this, here is a little animation of a callback from the C++ code when doing the filtering on Adam's example 5.1. By passing a simple plotting function, written in R, to the C++ code, we can get a plot updated on every iteration. Here I cheated a little and used our old plot function with fixed ranges, the package now uses a more general function: Example of RcppSMC callback to R plot when estimation example 5.1 from Johansen (2009) The animation is of course due to ImageMagick glueing one hundred files into a single animated gif. More information about RcppSMC is on its page, and we intend to add more examples and extensions over time.

21 August 2011

Christian Perrier: 10 years being Debian Developer - part 1: before 1992

So, on July 31st 2011, it was exactly 10 years since I am a Debian developer. What happened in the meantime? What lead me to this? What turned this unskilled dude into a sometimes quite visible contributor of one of the major free software projects? If you're interested in that, please read on. Otherwise, well, this is just yet another "bubulle talks about self" post and you can skip it. Well, first of all, how did I end up being a DD? And first of firsts, how did I end up not being a random user of Windows, playing games on his desktop computer at work? I am not a computer scientist, an "informaticien" as we sometimes say in French (most often, what people who are not in computing stuff say...thinking that all folks working more or less closely to computers know everything about them). I graduated with a PhD in Materials science, in 1989 after conducting a research on "Influence of Yttrium Oxide dispersion on the strength of titanium alloys", at Onera, a French public research institution for aerospace and defence. I was then hired, still at Onera (where I'm still working, 25 years after starting my PhD work) to lead the Mechanical Testing Laboratory in the Materials Science department. So, my lab had many big machines designed to conduct creep and fatigue tests often at interestingly high temperatures such as 2000 C, on samples of various materials that are used in aircrafts structures, engines, etc. or in various space thingies (remember Herm s, the european shuttle?) or in various "things that fly but just one way and you shouldn't be there when they land". So, we had computers handling data acquisition for these tests. So I became involved with designing data acquisition setups, or even programming data acquisition programs (one of mine, written in Forth, ran all Onera creep tests until 2004 and successfully passed Y2K because I knew that 2000 was a leap year. It could even pass 2100 as I knew this is not a leap year..:-) One day, I had to buy a modem in order to communicate with out italian counterpart (CIRA) and exchange tests results. Then I had to learn how modems work in MS-Dos(yeah...). Then, I discovered "online" resources...indeed more that strange world that was then called "BBS" (Bulletin Board Systems), those mysterious things ran by some happy few who were using modems at their home place to communicate in "forums" and everything related to technology. PC-Board, Remote Access, Fidonet, etc. became familiar to me these days, back in 1988-1990. So familiar that I ended up running my own BBS at home, killing our phone bills and using very sophisticated techniques such as US Robotics "High Speed Transfer" modems that could be used at 19200bps asymetrical for very high speed transfers of kilobytes and kilobytes of useless MS-Dos "freeware" and "shareware". Indeed, my very first BBS didn't even have one of these: it was using a cheaper V.22bis modem operating at 2400bps. I was waiting for my order of a sophisticated HST modem to arrive from USA through obscure import channels meant to circumvent the French telephone company regulations that were forbidding the use of "unapproved" systems, in order to protect the famous French Telephone System from interferences brought by Bad American Material. Bubulle System was born. During those years, I discovered very interesting things: computers can run together once you draw a wire between them. That's called Local Area Network and you can even transfer data at 1Mbps between two computers, assuming you don't forget to put terminating resistors at the end of the line on these funny 10-baseT connectors at the back of your home-assembled PC that was using 2MB of RAM (bought for very cheap through the help of an American friend who was in touch with some Chinese folks who were selling 256kb RAM sticks for half the retail price....assuming you want to drive in a mysterious storing place close to Charles de Gaulle airport or Eurodisney). Hello Gordon. Yes, I know, we're still friends on Facebook and you're probably still using that weird programming language which you were, IIRC, the only person in the world to use. 2MB, that should be enough for barely anything, including running *multitasking* software where, miracle, I could run two tasks at the same time on my one and only PC at home. Miracle, I don't have to shutdown my BBS if I want to read forums on my friends BBS. Yay for Desqview/386! Multitasking for dummies Still, I have to build one of these "networks" at home. Elizabeth won't like that as it means one more box (home-made of course) in our living room and some more wires. And why the hell is it running 24/7? So that friends can visit the BBS, darling... And I can even communicate with them: I write a message, I get an answer the day after and so on. In one full week, we can have a great conversation that would have taken *minutes* to have in the real life. Isn't this the miracle of technlogy? And, yes, this is a good reason for having phone bills raising up to 500 Francs/month: people can "exchange" programs through my BBS, that horrible white box running in the living room. Often, these programs are written for free and some of them are even given with "source code", which allows people to *modify* them. And that even makes friends, you know? Imagine that some day we have to move from one house to another: then I can just call out "who wants to help bubulle moving?", and probably a dozen of (sometimes scary but always nice and polite...and sometimes even showered) geeks will pop up and happily carry boxes full of my vinyl LP collection all day long. An entire world of friends. "bubulle", you say, dear? What's that? That's my nickname. It was invented by one of these friends, a really strange guy named Ren Cougnenc who wrote this "free software" program anmed BBTH, which allows you to use modems to connect to BBS, and even to those many "Minitel" BBS we have in France, thanks to our wonderful world-leading technology using V.23 communication. Many people know me as "bubulle" because, you know, Perrier water has bubbles and Ren likes Gaston Lagaffe fish companion who he named "bubulle". Ren , I love you. You chose to leave this world back in 1998. We'll no longer have our "p'tits midis" in Antony where you were showing me the marvels of what's coming in next episode. All this was around 1988 and about 1992, doing all these mysterious things at home (between Jean-Baptiste and Sophie's diapers) while still working with data acquisition MS-Dos machines at work. How did this end up being a Debian Developer? You'll know in the next episode..:-)

7 February 2010

MJ Ray: The Phone Co-op 2010

Ed Mayo talking to the Phone Co-op meeting
Ed Mayo talking to the Phone Co-op meeting Yesterday I had the pleasure to be at the Phone Co-ops annual meeting, as those of you who follow me on identi.ca or elsewhere will already know. The proceedings included a speech from Cooperatives-UK Chief Exec Ed Mayo that included the co-op pub in Salford, spurning David Beckham s gift and this summer s Co-operatives Fortnight. As ever, the bits I liked best were the lively question-and-answer sessions, both on Ed s speech and the annual report. I took notes and reproduce them below, but I didn t capture every question, or every answer and none of this is verbatim. Questions to Ed about UK cooperation included:

Questions about the annual report asked about:

13 December 2009

Russell Coker: Links December 2009

Dan Gilbert gave an insightful TED talk about our mistaken expectations of happiness [1]. Don Marti has an insightful post about net neutrality and public property [2]. When net access requires access to public property then it should be sold in a neutral manner. Rachel Pike gave an interesting TED talk about the scientific research behind a climate headline [3]. The people who claim to be skeptical of the science should watch this. Mark Peters wrote an interesting article A Happy Writer Is a Lousy Writer about the correlation between emotional state and work quality [4]. Apparently watching a film about cancer will make people more careful and focussed on details. CERIAS has an interesting short article about Firefox security as well as some philosophy on why web browser security generally sucks [5]. Cory Doctorow writes in the Guardian about Peter Mandelson s new stupidity in trying to legislate against file sharing [6]. This is going to seriously damage the economy of every country that implements it. Charles Stross has been blogging a series of non-fiction essays about space colonisation, in The Myth of the Starship he describes how most ideas of space travel are bad and how the word ship is always going to be unsuitable [7]. Brent T. White is an associate professor of law at the University of Arizona who has written an interesting paper about mortgages [8]. He says that anyone who is underwater (IE owing more than the value of their house) should walk away. The credit damage from abandoning a bad mortgage apparently isn t that bad, and there is the possibility of negotiating with the bank to reduce the value of the loan to match the value of the house. Mako is working on a project to allow prisoners to blog [9]. It s basically a snail-mail to web gateway as the prisonsers are not allowed Internet access. PracticalEthicsnews.com has an article about the special status that homeopathy is given [10]. It also notes that homeopathic medicines include arsenic and mercury. Such quackery should be outlawed, a life sentence for homeopathy would be appropriate IMHO. Cory Doctorow wrote an interesting essay about why he is not selling one of his books in audio form (he s giving it away) [11]. He concludes by noting that he wants no license agreement except don t violate copyright law . The fact that he can t get anyone to sell an audio book under such terms is a good demonstration of how broken the marketplace is. Thulasiraj Ravilla gave an inspiring TED talk about the Aravind Eye Care System a program to bring the efficiency of McDonalds to eye surgery [12]. Hopefully that program can spawn similar programs for other branches of medicine and spread to more countries. In many ways they are providing better service (both in quality and speed) than people in first-world countries who pay a lot of money can expect to receive. Scott Kim gave an interesting TED talk about his work designing puzzles [13]. He is also a big fan of social networking, unfortunately (for people like me who don t like social networking) his web site ShuffleBrain.com relies on Facebook. Gordon Brown (UK Prime Minister) gave an inspiring TED talk about global ethic vs the national interest [14], with a particular focus on the global effort required to tackle the climate change problem. Now if only we could get Kevin Rudd to listen to that. Brough has written an interesting analysis of the AT&T network problems that are blamed on the iPhone [15]. His essential claim is that the problem is due to overly large buffers which don t cause TCP implementations to throttle the throughput. This seems similar to my observations of the Three network in Australia where ping times of 8 seconds or more will periodically occur. One particularly nasty corner case with this is when using a local DNS server I can have a DNS packet storm where basic requests time out while BIND uses a significant portion of available bandwidth (including ICMP messages from receiving ports that BIND has closed). To alleviate this I am now using the Google public DNS service [16] (the Three DNS servers never worked properly).

Next.