Search Results: "jfs"

7 February 2021

Chris Lamb: Favourite books of 2020

I won't reveal precisely how many books I read in 2020, but it was definitely an improvement on 74 in 2019, 53 in 2018 and 50 in 2017. But not only did I read more in a quantitative sense, the quality seemed higher as well. There were certainly fewer disappointments: given its cultural resonance, I was nonplussed by Nick Hornby's Fever Pitch and whilst Ian Fleming's The Man with the Golden Gun was a little thin (again, given the obvious influence of the Bond franchise) the booked lacked 'thinness' in a way that made it interesting to critique. The weakest novel I read this year was probably J. M. Berger's Optimal, but even this hybrid of Ready Player One late-period Black Mirror wasn't that cringeworthy, all things considered. Alas, graphic novels continue to not quite be my thing, I'm afraid. I perhaps experienced more disappointments in the non-fiction section. Paul Bloom's Against Empathy was frustrating, particularly in that it expended unnecessary energy battling its misleading title and accepted terminology, and it could so easily have been an 20-minute video essay instead). (Elsewhere in the social sciences, David and Goliath will likely be the last Malcolm Gladwell book I voluntarily read.) After so many positive citations, I was also more than a little underwhelmed by Shoshana Zuboff's The Age of Surveillance Capitalism, and after Ryan Holiday's many engaging reboots of Stoic philosophy, his Conspiracy (on Peter Thiel and Hulk Hogan taking on Gawker) was slightly wide of the mark for me. Anyway, here follows a selection of my favourites from 2020, in no particular order:

Fiction Wolf Hall & Bring Up the Bodies & The Mirror and the Light Hilary Mantel During the early weeks of 2020, I re-read the first two parts of Hilary Mantel's Thomas Cromwell trilogy in time for the March release of The Mirror and the Light. I had actually spent the last few years eagerly following any news of the final instalment, feigning outrage whenever Mantel appeared to be spending time on other projects. Wolf Hall turned out to be an even better book than I remembered, and when The Mirror and the Light finally landed at midnight on 5th March, I began in earnest the next morning. Note that date carefully; this was early 2020, and the book swiftly became something of a heavy-handed allegory about the world at the time. That is to say and without claiming that I am Monsieur Cromuel in any meaningful sense it was an uneasy experience to be reading about a man whose confident grasp on his world, friends and life was slipping beyond his control, and at least in Cromwell's case, was heading inexorably towards its denouement. The final instalment in Mantel's trilogy is not perfect, and despite my love of her writing I would concur with the judges who decided against awarding her a third Booker Prize. For instance, there is something of the longueur that readers dislike in the second novel, although this might not be entirely Mantel's fault after all, the rise of the "ugly" Anne of Cleves and laborious trade negotiations for an uninspiring mineral (this is no Herbertian 'spice') will never match the court intrigues of Anne Boleyn, Jane Seymour and that man for all seasons, Thomas More. Still, I am already looking forward to returning to the verbal sparring between King Henry and Cromwell when I read the entire trilogy once again, tentatively planned for 2022.

The Fault in Our Stars John Green I came across John Green's The Fault in Our Stars via a fantastic video by Lindsay Ellis discussing Roland Barthes famous 1967 essay on authorial intent. However, I might have eventually come across The Fault in Our Stars regardless, not because of Green's status as an internet celebrity of sorts but because I'm a complete sucker for this kind of emotionally-manipulative bildungsroman, likely due to reading Philip Pullman's His Dark Materials a few too many times in my teens. Although its title is taken from Shakespeare's Julius Caesar, The Fault in Our Stars is actually more Romeo & Juliet. Hazel, a 16-year-old cancer patient falls in love with Gus, an equally ill teen from her cancer support group. Hazel and Gus share the same acerbic (and distinctly unteenage) wit and a love of books, centred around Hazel's obsession of An Imperial Affliction, a novel by the meta-fictional author Peter Van Houten. Through a kind of American version of Jim'll Fix It, Gus and Hazel go and visit Van Houten in Amsterdam. I'm afraid it's even cheesier than I'm describing it. Yet just as there is a time and a place for Michelin stars and Haribo Starmix, there's surely a place for this kind of well-constructed but altogether maudlin literature. One test for emotionally manipulative works like this is how well it can mask its internal contradictions while Green's story focuses on the universalities of love, fate and the shortness of life (as do almost all of his works, it seems), The Fault in Our Stars manages to hide, for example, that this is an exceedingly favourable treatment of terminal illness that is only possible for the better off. The 2014 film adaptation does somewhat worse in peddling this fantasy (and has a much weaker treatment of the relationship between the teens' parents too, an underappreciated subtlety of the book). The novel, however, is pretty slick stuff, and it is difficult to fault it for what it is. For some comparison, I later read Green's Looking for Alaska and Paper Towns which, as I mention, tug at many of the same strings, but they don't come together nearly as well as The Fault in Our Stars. James Joyce claimed that "sentimentality is unearned emotion", and in this respect, The Fault in Our Stars really does earn it.

The Plague Albert Camus P. D. James' The Children of Men, George Orwell's Nineteen Eighty-Four, Arthur Koestler's Darkness at Noon ... dystopian fiction was already a theme of my reading in 2020, so given world events it was an inevitability that I would end up with Camus's novel about a plague that swept through the Algerian city of Oran. Is The Plague an allegory about the Nazi occupation of France during World War Two? Where are all the female characters? Where are the Arab ones? Since its original publication in 1947, there's been so much written about The Plague that it's hard to say anything new today. Nevertheless, I was taken aback by how well it captured so much of the nuance of 2020. Whilst we were saying just how 'unprecedented' these times were, it was eerie how a novel written in the 1940s could accurately how many of us were feeling well over seventy years on later: the attitudes of the people; the confident declarations from the institutions; the misaligned conversations that led to accidental misunderstandings. The disconnected lovers. The only thing that perhaps did not work for me in The Plague was the 'character' of the church. Although I could appreciate most of the allusion and metaphor, it was difficult for me to relate to the significance of Father Paneloux, particularly regarding his change of view on the doctrinal implications of the virus, and spoiler alert that he finally died of a "doubtful case" of the disease, beyond the idea that Paneloux's beliefs are in themselves "doubtful". Answers on a postcard, perhaps. The Plague even seemed to predict how we, at least speaking of the UK, would react when the waves of the virus waxed and waned as well:
The disease stiffened and carried off three or four patients who were expected to recover. These were the unfortunates of the plague, those whom it killed when hope was high
It somehow captured the nostalgic yearning for high-definition videos of cities and public transport; one character even visits the completely deserted railway station in Oman simply to read the timetables on the wall.

Tinker, Tailor, Soldier, Spy John le Carr There's absolutely none of the Mad Men glamour of James Bond in John le Carr 's icy world of Cold War spies:
Small, podgy, and at best middle-aged, Smiley was by appearance one of London's meek who do not inherit the earth. His legs were short, his gait anything but agile, his dress costly, ill-fitting, and extremely wet.
Almost a direct rebuttal to Ian Fleming's 007, Tinker, Tailor has broken-down cars, bad clothes, women with their own internal and external lives (!), pathetically primitive gadgets, and (contra Mad Men) hangovers that significantly longer than ten minutes. In fact, the main aspect that the mostly excellent 2011 film adaption doesn't really capture is the smoggy and run-down nature of 1970s London this is not your proto-Cool Britannia of Austin Powers or GTA:1969, the city is truly 'gritty' in the sense there is a thin film of dirt and grime on every surface imaginable. Another angle that the film cannot capture well is just how purposefully the novel does not mention the United States. Despite the US obviously being the dominant power, the British vacillate between pretending it doesn't exist or implying its irrelevance to the matter at hand. This is no mistake on Le Carr 's part, as careful readers are rewarded by finding this denial of US hegemony in metaphor throughout --pace Ian Fleming, there is no obvious Felix Leiter to loudly throw money at the problem or a Sheriff Pepper to serve as cartoon racist for the Brits to feel superior about. By contrast, I recall that a clever allusion to "dusty teabags" is subtly mirrored a few paragraphs later with a reference to the installation of a coffee machine in the office, likely symbolic of the omnipresent and unavoidable influence of America. (The officer class convince themselves that coffee is a European import.) Indeed, Le Carr communicates a feeling of being surrounded on all sides by the peeling wallpaper of Empire. Oftentimes, the writing style matches the graceless and inelegance of the world it depicts. The sentences are dense and you find your brain performing a fair amount of mid-flight sentence reconstruction, reparsing clauses, commas and conjunctions to interpret Le Carr 's intended meaning. In fact, in his eulogy-cum-analysis of Le Carr 's writing style, William Boyd, himself a ventrioquilist of Ian Fleming, named this intentional technique 'staccato'. Like the musical term, I suspect the effect of this literary staccato is as much about the impact it makes on a sentence as the imperceptible space it generates after it. Lastly, the large cast in this sprawling novel is completely believable, all the way from the Russian spymaster Karla to minor schoolboy Roach the latter possibly a stand-in for Le Carr himself. I got through the 500-odd pages in just a few days, somehow managing to hold the almost-absurdly complicated plot in my head. This is one of those classic books of the genre that made me wonder why I had not got around to it before.

The Nickel Boys Colson Whitehead According to the judges who awarded it the Pulitzer Prize for Fiction, The Nickel Boys is "a devastating exploration of abuse at a reform school in Jim Crow-era Florida" that serves as a "powerful tale of human perseverance, dignity and redemption". But whilst there is plenty of this perseverance and dignity on display, I found little redemption in this deeply cynical novel. It could almost be read as a follow-up book to Whitehead's popular The Underground Railroad, which itself won the Pulitzer Prize in 2017. Indeed, each book focuses on a young protagonist who might be euphemistically referred to as 'downtrodden'. But The Nickel Boys is not only far darker in tone, it feels much closer and more connected to us today. Perhaps this is unsurprising, given that it is based on the story of the Dozier School in northern Florida which operated for over a century before its long history of institutional abuse and racism was exposed a 2012 investigation. Nevertheless, if you liked the social commentary in The Underground Railroad, then there is much more of that in The Nickel Boys:
Perhaps his life might have veered elsewhere if the US government had opened the country to colored advancement like they opened the army. But it was one thing to allow someone to kill for you and another to let him live next door.
Sardonic aper us of this kind are pretty relentless throughout the book, but it never tips its hand too far into on nihilism, especially when some of the visual metaphors are often first-rate: "An American flag sighed on a pole" is one I can easily recall from memory. In general though, The Nickel Boys is not only more world-weary in tenor than his previous novel, the United States it describes seems almost too beaten down to have the energy conjure up the Swiftian magical realism that prevented The Underground Railroad from being overly lachrymose. Indeed, even we Whitehead transports us a present-day New York City, we can't indulge in another kind of fantasy, the one where America has solved its problems:
The Daily News review described the [Manhattan restaurant] as nouveau Southern, "down-home plates with a twist." What was the twist that it was soul food made by white people?
It might be overly reductionist to connect Whitehead's tonal downshift with the racial justice movements of the past few years, but whatever the reason, we've ended up with a hard-hitting, crushing and frankly excellent book.

True Grit & No Country for Old Men Charles Portis & Cormac McCarthy It's one of the most tedious cliches to claim the book is better than the film, but these two books are of such high quality that even the Coen Brothers at their best cannot transcend them. I'm grouping these books together here though, not because their respective adaptations will exemplify some of the best cinema of the 21st century, but because of their superb treatment of language. Take the use of dialogue. Cormac McCarthy famously does not use any punctuation "I believe in periods, in capitals, in the occasional comma, and that's it" but the conversations in No Country for Old Men together feel familiar and commonplace, despite being relayed through this unconventional technique. In lesser hands, McCarthy's written-out Texan drawl would be the novelistic equivalent of white rap or Jar Jar Binks, but not only is the effect entirely gripping, it helps you to believe you are physically present in the many intimate and domestic conversations that hold this book together. Perhaps the cinematic familiarity helps, as you can almost hear Tommy Lee Jones' voice as Sheriff Bell from the opening page to the last. Charles Portis' True Grit excels in its dialogue too, but in this book it is not so much in how it flows (although that is delightful in its own way) but in how forthright and sardonic Maddie Ross is:
"Earlier tonight I gave some thought to stealing a kiss from you, though you are very young, and sick and unattractive to boot, but now I am of a mind to give you five or six good licks with my belt." "One would be as unpleasant as the other."
Perhaps this should be unsurprising. Maddie, a fourteen-year-old girl from Yell County, Arkansas, can barely fire her father's heavy pistol, so she can only has words to wield as her weapon. Anyway, it's not just me who treasures this book. In her encomium that presages most modern editions, Donna Tartt of The Secret History fame traces the novels origins through Huckleberry Finn, praising its elegance and economy: "The plot of True Grit is uncomplicated and as pure in its way as one of the Canterbury Tales". I've read any Chaucer, but I am inclined to agree. Tartt also recalls that True Grit vanished almost entirely from the public eye after the release of John Wayne's flimsy cinematic vehicle in 1969 this earlier film was, Tartt believes, "good enough, but doesn't do the book justice". As it happens, reading a book with its big screen adaptation as a chaser has been a minor theme of my 2020, including P. D. James' The Children of Men, Kazuo Ishiguro's Never Let Me Go, Patricia Highsmith's Strangers on a Train, James Ellroy's The Black Dahlia, John Green's The Fault in Our Stars, John le Carr 's Tinker, Tailor Soldier, Spy and even a staged production of Charles Dicken's A Christmas Carol streamed from The Old Vic. For an autodidact with no academic background in literature or cinema, I've been finding this an effective and enjoyable means of getting closer to these fine books and films it is precisely where they deviate (or perhaps where they are deficient) that offers a means by which one can see how they were constructed. I've also found that adaptations can also tell you a lot about the culture in which they were made: take the 'straightwashing' in the film version of Strangers on a Train (1951) compared to the original novel, for example. It is certainly true that adaptions rarely (as Tartt put it) "do the book justice", but she might be also right to alight on a legal metaphor, for as the saying goes, to judge a movie in comparison to the book is to do both a disservice.

The Glass Hotel Emily St. John Mandel In The Glass Hotel, Mandel somehow pulls off the impossible; writing a loose roman- -clef on Bernie Madoff, a Ponzi scheme and the ephemeral nature of finance capital that is tranquil and shimmeringly beautiful. Indeed, don't get the wrong idea about the subject matter; this is no over over-caffeinated The Big Short, as The Glass Hotel is less about a Madoff or coked-up financebros but the fragile unreality of the late 2010s, a time which was, as we indeed discovered in 2020, one event away from almost shattering completely. Mandel's prose has that translucent, phantom quality to it where the chapters slip through your fingers when you try to grasp at them, and the plot is like a ghost ship that that slips silently, like the Mary Celeste, onto the Canadian water next to which the eponymous 'Glass Hotel' resides. Indeed, not unlike The Overlook Hotel, the novel so overflows with symbolism so that even the title needs to evoke the idea of impermanence permanently living in a hotel might serve as a house, but it won't provide a home. It's risky to generalise about such things post-2016, but the whole story sits in that the infinitesimally small distance between perception and reality, a self-constructed culture that is not so much 'post truth' but between them. There's something to consider in almost every character too. Take the stand-in for Bernie Madoff: no caricature of Wall Street out of a 1920s political cartoon or Brechtian satire, Jonathan Alkaitis has none of the oleaginous sleaze of a Dominic Strauss-Kahn, the cold sociopathy of a Marcus Halberstam nor the well-exercised sinuses of, say, Jordan Belford. Alkaitis is dare I say it? eminently likeable, and the book is all the better for it. Even the C-level characters have something to say: Enrico, trivially escaping from the regulators (who are pathetically late to the fraud without Mandel ever telling us explicitly), is daydreaming about the girlfriend he abandoned in New York: "He wished he'd realised he loved her before he left". What was in his previous life that prevented him from doing so? Perhaps he was never in love at all, or is love itself just as transient as the imaginary money in all those bank accounts? Maybe he fell in love just as he crossed safely into Mexico? When, precisely, do we fall in love anyway? I went on to read Mandel's Last Night in Montreal, an early work where you can feel her reaching for that other-worldly quality that she so masterfully achieves in The Glass Hotel. Her f ted Station Eleven is on my must-read list for 2021. "What is truth?" asked Pontius Pilate. Not even Mandel cannot give us the answer, but this will certainly do for now.

Running the Light Sam Tallent Although it trades in all of the clich s and stereotypes of the stand-up comedian (the triumvirate of drink, drugs and divorce), Sam Tallent's debut novel depicts an extremely convincing fictional account of a touring road comic. The comedian Doug Stanhope (who himself released a fairly decent No Encore for the Donkey memoir in 2020) hyped Sam's book relentlessly on his podcast during lockdown... and justifiably so. I ripped through Running the Light in a few short hours, the only disappointment being that I can't seem to find videos online of Sam that come anywhere close to match up to his writing style. If you liked the rollercoaster energy of Paul Beatty's The Sellout, the cynicism of George Carlin and the car-crash invertibility of final season Breaking Bad, check this great book out.

Non-fiction Inside Story Martin Amis This was my first introduction to Martin Amis's work after hearing that his "novelised autobiography" contained a fair amount about Christopher Hitchens, an author with whom I had a one of those rather clich d parasocial relationship with in the early days of YouTube. (Hey, it could have been much worse.) Amis calls his book a "novelised autobiography", and just as much has been made of its quasi-fictional nature as the many diversions into didactic writing advice that betwixt each chapter: "Not content with being a novel, this book also wants to tell you how to write novels", complained Tim Adams in The Guardian. I suspect that reviewers who grew up with Martin since his debut book in 1973 rolled their eyes at yet another demonstration of his manifest cleverness, but as my first exposure to Amis's gift of observation, I confess that I was thought it was actually kinda clever. Try, for example, "it remains a maddening truth that both sexual success and sexual failure are steeply self-perpetuating" or "a hospital gym is a contradiction like a young Conservative", etc. Then again, perhaps I was experiencing a form of nostalgia for a pre-Gamergate YouTube, when everything in the world was a lot simpler... or at least things could be solved by articulate gentlemen who honed their art of rhetoric at the Oxford Union. I went on to read Martin's first novel, The Rachel Papers (is it 'arrogance' if you are, indeed, that confident?), as well as his 1997 Night Train. I plan to read more of him in the future.

The Collected Essays, Journalism and Letters: Volume 1 & Volume 2 & Volume 3 & Volume 4 George Orwell These deceptively bulky four volumes contain all of George Orwell's essays, reviews and correspondence, from his teenage letters sent to local newspapers to notes to his literary executor on his deathbed in 1950. Reading this was part of a larger, multi-year project of mine to cover the entirety of his output. By including this here, however, I'm not recommending that you read everything that came out of Orwell's typewriter. The letters to friends and publishers will only be interesting to biographers or hardcore fans (although I would recommend Dorian Lynskey's The Ministry of Truth: A Biography of George Orwell's 1984 first). Furthermore, many of his book reviews will be of little interest today. Still, some insights can be gleaned; if there is any inconsistency in this huge corpus is that his best work is almost 'too' good and too impactful, making his merely-average writing appear like hackwork. There are some gems that don't make the usual essay collections too, and some of Orwell's most astute social commentary came out of series of articles he wrote for the left-leaning newspaper Tribune, related in many ways to the US Jacobin. You can also see some of his most famous ideas start to take shape years if not decades before they appear in his novels in these prototype blog posts. I also read Dennis Glover's novelised account of the writing of Nineteen-Eighty Four called The Last Man in Europe, and I plan to re-read some of Orwell's earlier novels during 2021 too, including A Clergyman's Daughter and his 'antebellum' Coming Up for Air that he wrote just before the Second World War; his most under-rated novel in my estimation. As it happens, and with the exception of the US and Spain, copyright in the works published in his lifetime ends on 1st January 2021. Make of that what you will.

Capitalist Realism & Chavs: The Demonisation of the Working Class Mark Fisher & Owen Jones These two books are not natural companions to one another and there is likely much that Jones and Fisher would vehemently disagree on, but I am pairing these books together here because they represent the best of the 'political' books I read in 2020. Mark Fisher was a dedicated leftist whose first book, Capitalist Realism, marked an important contribution to political philosophy in the UK. However, since his suicide in early 2017, the currency of his writing has markedly risen, and Fisher is now frequently referenced due to his belief that the prevalence of mental health conditions in modern life is a side-effect of various material conditions, rather than a natural or unalterable fact "like weather". (Of course, our 'weather' is being increasingly determined by a combination of politics, economics and petrochemistry than pure randomness.) Still, Fisher wrote on all manner of topics, from the 2012 London Olympics and "weird and eerie" electronic music that yearns for a lost future that will never arrive, possibly prefiguring or influencing the Fallout video game series. Saying that, I suspect Fisher will resonate better with a UK audience more than one across the Atlantic, not necessarily because he was minded to write about the parochial politics and culture of Britain, but because his writing often carries some exasperation at the suppression of class in favour of identity-oriented politics, a viewpoint not entirely prevalent in the United States outside of, say, Tour F. Reed or the late Michael Brooks. (Indeed, Fisher is likely best known in the US as the author of his controversial 2013 essay, Exiting the Vampire Castle, but that does not figure greatly in this book). Regardless, Capitalist Realism is an insightful, damning and deeply unoptimistic book, best enjoyed in the warm sunshine I found it an ironic compliment that I had quoted so many paragraphs that my Kindle's copy protection routines prevented me from clipping any further. Owen Jones needs no introduction to anyone who regularly reads a British newspaper, especially since 2015 where he unofficially served as a proxy and punching bag for expressing frustrations with the then-Labour leader, Jeremy Corbyn. However, as the subtitle of Jones' 2012 book suggests, Chavs attempts to reveal the "demonisation of the working class" in post-financial crisis Britain. Indeed, the timing of the book is central to Jones' analysis, specifically that the stereotype of the "chav" is used by government and the media as a convenient figleaf to avoid meaningful engagement with economic and social problems on an austerity ridden island. (I'm not quite sure what the US equivalent to 'chav' might be. Perhaps Florida Man without the implications of mental health.) Anyway, Jones certainly has a point. From Vicky Pollard to the attacks on Jade Goody, there is an ignorance and prejudice at the heart of the 'chav' backlash, and that would be bad enough even if it was not being co-opted or criminalised for ideological ends. Elsewhere in political science, I also caught Michael Brooks' Against the Web and David Graeber's Bullshit Jobs, although they are not quite methodical enough to recommend here. However, Graeber's award-winning Debt: The First 5000 Years will be read in 2021. Matt Taibbi's Hate Inc: Why Today's Media Makes Us Despise One Another is worth a brief mention here though, but its sprawling nature felt very much like I was reading a set of Substack articles loosely edited together. And, indeed, I was.

The Golden Thread: The Story of Writing Ewan Clayton A recommendation from a dear friend, Ewan Clayton's The Golden Thread is a journey through the long history of the writing from the Dawn of Man to present day. Whether you are a linguist, a graphic designer, a visual artist, a typographer, an archaeologist or 'just' a reader, there is probably something in here for you. I was already dipping my quill into calligraphy this year so I suspect I would have liked this book in any case, but highlights would definitely include the changing role of writing due to the influence of textual forms in the workplace as well as digression on ergonomic desks employed by monks and scribes in the Middle Ages. A lot of books by otherwise-sensible authors overstretch themselves when they write about computers or other technology from the Information Age, at best resulting in bizarre non-sequiturs and dangerously Panglossian viewpoints at worst. But Clayton surprised me by writing extremely cogently and accurate on the role of text in this new and unpredictable era. After finishing it I realised why for a number of years, Clayton was a consultant for the legendary Xerox PARC where he worked in a group focusing on documents and contemporary communications whilst his colleagues were busy inventing the graphical user interface, laser printing, text editors and the computer mouse.

New Dark Age & Radical Technologies: The Design of Everyday Life James Bridle & Adam Greenfield I struggled to describe these two books to friends, so I doubt I will suddenly do a better job here. Allow me to quote from Will Self's review of James Bridle's New Dark Age in the Guardian:
We're accustomed to worrying about AI systems being built that will either "go rogue" and attack us, or succeed us in a bizarre evolution of, um, evolution what we didn't reckon on is the sheer inscrutability of these manufactured minds. And minds is not a misnomer. How else should we think about the neural network Google has built so its translator can model the interrelation of all words in all languages, in a kind of three-dimensional "semantic space"?
New Dark Age also turns its attention to the weird, algorithmically-derived products offered for sale on Amazon as well as the disturbing and abusive videos that are automatically uploaded by bots to YouTube. It should, by rights, be a mess of disparate ideas and concerns, but Bridle has a flair for introducing topics which reveals he comes to computer science from another discipline altogether; indeed, on a four-part series he made for Radio 4, he's primarily referred to as "an artist". Whilst New Dark Age has rather abstract section topics, Adam Greenfield's Radical Technologies is a rather different book altogether. Each chapter dissects one of the so-called 'radical' technologies that condition the choices available to us, asking how do they work, what challenges do they present to us and who ultimately benefits from their adoption. Greenfield takes his scalpel to smartphones, machine learning, cryptocurrencies, artificial intelligence, etc., and I don't think it would be unfair to say that starts and ends with a cynical point of view. He is no reactionary Luddite, though, and this is both informed and extremely well-explained, and it also lacks the lazy, affected and Private Eye-like cynicism of, say, Attack of the 50 Foot Blockchain. The books aren't a natural pair, for Bridle's writing contains quite a bit of air in places, ironically mimics the very 'clouds' he inveighs against. Greenfield's book, by contrast, as little air and much lower pH value. Still, it was more than refreshing to read two technology books that do not limit themselves to platitudinal booleans, be those dangerously naive (e.g. Kevin Kelly's The Inevitable) or relentlessly nihilistic (Shoshana Zuboff's The Age of Surveillance Capitalism). Sure, they are both anti-technology screeds, but they tend to make arguments about systems of power rather than specific companies and avoid being too anti-'Big Tech' through a narrower, Silicon Valley obsessed lens for that (dipping into some other 2020 reading of mine) I might suggest Wendy Liu's Abolish Silicon Valley or Scott Galloway's The Four. Still, both books are superlatively written. In fact, Adam Greenfield has some of the best non-fiction writing around, both in terms of how he can explain complicated concepts (particularly the smart contract mechanism of the Ethereum cryptocurrency) as well as in the extremely finely-crafted sentences I often felt that the writing style almost had no need to be that poetic, and I particularly enjoyed his fictional scenarios at the end of the book.

The Algebra of Happiness & Indistractable: How to Control Your Attention and Choose Your Life Scott Galloway & Nir Eyal A cocktail of insight, informality and abrasiveness makes NYU Professor Scott Galloway uncannily appealing to guys around my age. Although Galloway definitely has his own wisdom and experience, similar to Joe Rogan I suspect that a crucial part of Galloway's appeal is that you feel you are learning right alongside him. Thankfully, 'Prof G' is far less err problematic than Rogan (Galloway is more of a well-meaning, spirited centrist), although he, too, has some pretty awful takes at time. This is a shame, because removed from the whirlwind of social media he can be really quite considered, such as in this long-form interview with Stephanie Ruhle. In fact, it is this kind of sentiment that he captured in his 2019 Algebra of Happiness. When I look over my highlighted sections, it's clear that it's rather schmaltzy out of context ("Things you hate become just inconveniences in the presence of people you love..."), but his one-two punch of cynicism and saccharine ("Ask somebody who purchased a home in 2007 if their 'American Dream' came true...") is weirdly effective, especially when he uses his own family experiences as part of his story:
A better proxy for your life isn't your first home, but your last. Where you draw your last breath is more meaningful, as it's a reflection of your success and, more important, the number of people who care about your well-being. Your first house signals the meaningful your future and possibility. Your last home signals the profound the people who love you. Where you die, and who is around you at the end, is a strong signal of your success or failure in life.
Nir Eyal's Indistractable, however, is a totally different kind of 'self-help' book. The important background story is that Eyal was the author of the widely-read Hooked which turned into a secular Bible of so-called 'addictive design'. (If you've ever been cornered by a techbro wielding a Wikipedia-thin knowledge of B. F. Skinner's behaviourist psychology and how it can get you to click 'Like' more often, it ultimately came from Hooked.) However, Eyal's latest effort is actually an extended mea culpa for his previous sin and he offers both high and low-level palliative advice on how to avoid falling for the tricks he so studiously espoused before. I suppose we should be thankful to capitalism for selling both cause and cure. Speaking of markets, there appears to be a growing appetite for books in this 'anti-distraction' category, and whilst I cannot claim to have done an exhausting study of this nascent field, Indistractable argues its points well without relying on accurate-but-dry "studies show..." or, worse, Gladwellian gotchas. My main criticism, however, would be that Eyal doesn't acknowledge the limits of a self-help approach to this problem; it seems that many of the issues he outlines are an inescapable part of the alienation in modern Western society, and the only way one can really avoid distraction is to move up the income ladder or move out to a 500-acre ranch.

17 April 2017

Russell Coker: More KVM Modules Configuration

Last year I blogged about blacklisting a video driver so that KVM virtual machines didn t go into graphics mode [1]. Now I ve been working on some other things to make virtual machines run better. I use the same initramfs for the physical hardware as for the virtual machines. So I need to remove modules that are needed for booting the physical hardware from the VMs as well as other modules that get dragged in by systemd and other things. One significant saving from this is that I use BTRFS for the physical machine and the BTRFS driver takes 1M of RAM! The first thing I did to reduce the number of modules was to edit /etc/initramfs-tools/initramfs.conf and change MODULES=most to MODULES=dep . This significantly reduced the number of modules loaded and also stopped the initramfs from probing for a non-existant floppy drive which added about 20 seconds to the boot. Note that this will result in your initramfs not supporting different hardware. So if you plan to take a hard drive out of your desktop PC and install it in another PC this could be bad for you, but for servers it s OK as that sort of upgrade is uncommon for servers and only done with some planning (such as creating an initramfs just for the migration). I put the following rmmod commands in /etc/rc.local to remove modules that are automatically loaded:
rmmod btrfs
rmmod evdev
rmmod lrw
rmmod glue_helper
rmmod ablk_helper
rmmod aes_x86_64
rmmod ecb
rmmod xor
rmmod raid6_pq
rmmod cryptd
rmmod gf128mul
rmmod ata_generic
rmmod ata_piix
rmmod i2c_piix4
rmmod libata
rmmod scsi_mod In /etc/modprobe.d/blacklist.conf I have the following lines to stop drivers being loaded. The first line is to stop the video mode being set and the rest are just to save space. One thing that inspired me to do this is that the parallel port driver gave a kernel error when it loaded and tried to access non-existant hardware.
blacklist bochs_drm
blacklist joydev
blacklist ppdev
blacklist sg
blacklist psmouse
blacklist pcspkr
blacklist sr_mod
blacklist acpi_cpufreq
blacklist cdrom
blacklist tpm
blacklist tpm_tis
blacklist floppy
blacklist parport_pc
blacklist serio_raw
blacklist button On the physical machine I have the following in /etc/modprobe.d/blacklist.conf. Most of this is to prevent loading of filesystem drivers when making an initramfs. I do this because I know there s never going to be any need for CDs, parallel devices, graphics, or strange block devices in a server room. I wouldn t do any of this for a desktop workstation or laptop.
blacklist ppdev
blacklist parport_pc
blacklist cdrom
blacklist sr_mod
blacklist nouveau blacklist ufs
blacklist qnx4
blacklist hfsplus
blacklist hfs
blacklist minix
blacklist ntfs
blacklist jfs
blacklist xfs

30 September 2016

Chris Lamb: Free software activities in September 2016

Here is my monthly update covering what I have been doing in the free software world (previous month):
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most Linux distributions provide binary (or "compiled") packages to end users. The motivation behind the Reproducible Builds effort is to allow verification that no flaws have been introduced either maliciously and accidentally during this compilation process by promising identical binary packages are always generated from a given source. My work in the Reproducible Builds project was also covered in our weekly reports #71, #72, #71 & #74. I made the following improvements to our tools:


diffoscope is our "diff on steroids" that will not only recursively unpack archives but will transform binary formats into human-readable forms in order to compare them.

  • Added a global Progress object to track the status of the comparison process allowing for graphical and machine-readable status indicators. I also blogged about this feature in more detail.
  • Moved the global Config object to a more Pythonic "singleton" pattern and ensured that constraints are checked on every change.


disorderfs is our FUSE filesystem that deliberately introduces nondeterminism into the results of system calls such as readdir(3).

  • Display the "disordered" behaviour we intend to show on startup. (#837689)
  • Support relative paths in command-line parameters (previously only absolute paths were permitted).


strip-nondeterminism is our tool to remove specific information from a completed build.

  • Fix an issue where temporary files were being left on the filesystem and add a test to avoid similar issues in future. (#836670)
  • Print an error if the file to normalise does not exist. (#800159)
  • Testsuite improvements:
    • Set the timezone in tests to avoid a FTBFS and add a File::StripNondeterminism::init method to the API to to set tzset everywhere. (#837382)
    • "Smoke test" the strip-nondeterminism(1) and dh_strip_nondeterminism(1) scripts to prevent syntax regressions.
    • Add a testcase for .jar file ordering and normalisation.
    • Check the stripping process before comparing file attributes to make it less confusing on failure.
    • Move to a lookup table for descriptions of stat(1) indices and use that for nicer failure messages.
    • Don't uselessly test whether the inode number has changed.
  • Run perlcritic across the codebase and adopt some of its prescriptions including explicitly using oct(..) for integers with leading zeroes, avoiding mixing high and low-precedence booleans, ensuring subroutines end with a return statement, etc.

I also submitted 4 patches to fix specific reproducibility issues in golang-google-grpc, nostalgy, python-xlib & torque.


Patches contributed

Debian LTS

This month I have been paid to work 12.75 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 608-1 for mailman fixing a CSRF vulnerability.
  • Issued DLA 611-1 for jsch correcting a path traversal vulnerability.
  • Issued DLA 620-1 for libphp-adodb patching a SQL injection vulnerability.
  • Issued DLA 631-1 for unadf correcting a buffer underflow issue.
  • Issued DLA 634-1 for dropbear fixing a buffer overflow when parsing ASN.1 keys.
  • Issued DLA 635-1 for dwarfutils working around an out-of-bounds read issue.
  • Issued DLA 638-1 for the SELinux policycoreutils, patching a sandbox escape issue.
  • Enhanced Brian May's find-work --unassigned switch to take an optional "except this user" argument.
  • Marked matrixssl and inspircd as being unsupported in the current LTS version.

  • python-django 1:1.10.1-1 New upstream release and ensure that django-admin startproject foo creates files with the correct shebang under Python 3.
  • gunicorn:
    • 19.6.0-5 Don't call chown(2) if it would be a no-op to avoid failure under snap.
    • 19.6.0-6 Remove now-obsolete conffiles and logrotate scripts; they should have been removed in 19.6.0-3.
  • redis:
    • 3.2.3-2 Call ulimit -n 65536 by default from SysVinit scripts to normalise the behaviour with systemd. I also bumped the Debian package epoch as the "2:" prefix made it look like we are shipping version 2.x. I additionaly backported this upload to Debian Jessie.
    • 3.2.4-1 New upstream release, add missing -ldl for dladdr(3) & add missing dependency on lsb-base.
  • python-redis (2.10.5-2) Bump python-hiredis to Suggests to sync with Ubuntu and move to a machine-readable debian/copyright. I also backported this upload to Debian Jessie.
  • adminer (4.2.5-3) Move mysql-server dependencies to default-mysql-server. I also backported this upload to Debian Jessie.
  • gpsmanshp (1.2.3-5) on behalf of the QA team:
    • Move to "minimal" debhelper style, making the build reproducible. (#777446 & #792991)
    • Reorder linker command options to build with --as-needed (#729726) and add hardening flags.
    • Move to machine-readable copyright file, add missing #DEBHELPER# tokens to postinst and prerm scripts, tidy descriptions & other debian/control fields and other smaller changes.

I sponsored the upload of 5 packages from other developers:

I also NMU'd:

FTP Team

As a Debian FTP assistant I ACCEPTed 147 packages: alljoyn-services-1604, android-platform-external-doclava, android-platform-system-tools-aidl, aufs, bcolz, binwalk, bmusb, bruteforce-salted-openssl, cappuccino, captagent, chrome-gnome-shell, ciphersaber, cmark, colorfultabs, cppformat, dnsrecon, dogtag-pki, dxtool, e2guardian, flask-compress, fonts-mononoki, fwknop-gui, gajim-httpupload, glbinding, glewmx, gnome-2048, golang-github-googleapis-proto-client-go, google-android-installers, gsl, haskell-hmatrix-gsl, haskell-relational-query, haskell-relational-schemas, haskell-secret-sharing, hindsight, i8c, ip4r, java-string-similarity, khal, khronos-opencl-headers, liblivemedia, libshell-config-generate-perl, libshell-guess-perl, libstaroffice, libxml2, libzonemaster-perl, linux, linux-grsec-base, linux-signed, lua-sandbox, lua-torch-trepl, mbrola-br2, mbrola-br4, mbrola-de1, mbrola-de2, mbrola-de3, mbrola-ir1, mbrola-lt1, mbrola-lt2, mbrola-mx1, mimeo, mimerender, mongo-tools, mozilla-gnome-keyring, munin, node-grunt-cli, node-js-yaml, nova, open-build-service, openzwave, orafce, osmalchemy, pgespresso, pgextwlist, pgfincore, pgmemcache, pgpool2, pgsql-asn1oid, postbooks-schema, postgis, postgresql-debversion, postgresql-multicorn, postgresql-mysql-fdw, postgresql-unit, powerline-taskwarrior, prefix, pycares, pydl, pynliner, pytango, pytest-cookies, python-adal, python-applicationinsights, python-async-timeout, python-azure, python-azure-storage, python-blosc, python-can, python-canmatrix, python-chartkick, python-confluent-kafka, python-jellyfish, python-k8sclient, python-msrestazure, python-nss, python-pytest-benchmark, python-tenacity, python-tmdbsimple, python-typing, python-unidiff, python-xstatic-angular-schema-form, python-xstatic-tv4, quilt, r-bioc-phyloseq, r-cran-filehash, r-cran-png, r-cran-testit, r-cran-tikzdevice, rainbow-mode, repmgr, restart-emacs, restbed, ruby-azure-sdk, ruby-babel-source, ruby-babel-transpiler, ruby-diaspora-prosody-config, ruby-haikunator, ruby-license-finder, ruby-ms-rest, ruby-ms-rest-azure, ruby-rails-assets-autosize, ruby-rails-assets-blueimp-gallery, ruby-rails-assets-bootstrap, ruby-rails-assets-bootstrap-markdown, ruby-rails-assets-emojione, ruby-sprockets-es6, ruby-timeliness, rustc, skytools3, slony1-2, snmp-mibs-downloader, syslog-ng, test-kitchen, uctodata, usbguard, vagrant-azure, vagrant-mutate & vim.

26 September 2014

Holger Levsen: 20140925-reproducible-builds

Reproducible builds? I never did any - manually :) I've never done a reproducible build attempt of any package, manually, ever. But what I've done now is setting up reproducible builds on which will build hundreds or thousands of packages, hopefully reproducibly, regularily in the future. Thanks to Lunar's and many other peoples work, this was actually rather easy. If you want to do this manually, it should take you just a few minutes to setup a suitable build environment. So three days ago when I wasn't exactly bored I decided that it was a good moment to implement some reproducible build jobs on jenkins.d.n, and so I gave it a try and two hours later the basic implementation was working, and then it was an evening and morning of fine tuning until I was mostly satisfied. Since then there has been some polishing, but the basic setup is done and has been working since. What's the result? One job, reproducible_setup will just create a suitable environment for pbuilding reproducible packages as documented so well on the Debian wiki. And as that job runs 3.5 minutes only (to debootstrap from scratch), it's run daily. And then there are currently 16 other jobs, which test reproducible builds in different areas: d-i, core, some six major desktops and some selected desktop applications, some security + privacy related packages, some build chains we have in Debian, libreoffice and Most of these jobs run several hours, but luckily not days. And they discover packages which still fail to build reproducibly, which already has caused some bugs to be filed, eg. #762732 "libdebian-installer: please do not write timestamps in Doxygen generated documentation". So this is the output from testing the reproducibilty of all debian-installer packages: 72 packages were successfully built reproducibly, while 6 packages failed to do so. I was quite impressed by these numbers as AFAIK noone tried to build d-i reproducibly before.
72 packages successfully built reproducibly: userdevfs user-setup usb-discover udpkg tzsetup rootskel rootskel-gtk rescue preseed pkgsel partman-xfs partman-target partman-partitioning partman-nbd partman-multipath partman-md partman-lvm partman-jfs partman-iscsi partman-ext3 partman-efi partman-crypto partman-btrfs partman-basicmethods partman-basicfilesystems partman-base partman-auto partman-auto-raid partman-auto-lvm partman-auto-crypto partconf os-prober oldsys-preseed nobootloader network-console netcfg net-retriever mountmedia mklibs media-retriever mdcfg main-menu lvmcfg lowmem localechooser live-installer lilo-installer kickseed kernel-wedge kbd-chooser iso-scan installation-report installation-locale hw-detect grub-installer finish-install efi-reader dh-di debian-installer-utils debian-installer-netboot-images debian-installer-launcher clock-setup choose-mirror cdrom-retriever cdrom-detect cdrom-checker cdebconf-terminal cdebconf-entropy bterm-unifont base-installer apt-setup anna 
6 packages failed to built reproducibly: win32-loader libdebian-installer debootstrap console-setup cdebconf busybox 
What's also impressive: all packages for the newly introduced Cinnamon Desktop build reproducibly from the start! The jenkins setup is configured via just three small files: That's it and that's enough to keep several cores busy for days. :-) But as each job only takes a few hours each is scheduled twice a month and more jobs and packages shall be added in future (with some heuristics to schedule known good packages less often...) I guess it's an appropriate opportunity to say "many thanks to Profitbricks", who have been donating the powerful virtual machine is running on since October 2012. I also want to say "many many thanks to Helmut" (Grohne) who has recently joined me in maintaining this jenkins setup. And then I'd like to thank "the KGB trio" (Gregor, Tincho and Dam!) for providing those KGB bots on IRC, which are very helpful for providing notifications on IRC channels and last but not least thanks to everybody who contributed so that reproducible builds got this far! Keep up the jolly good work! And If you happen to know failing packages not included in job-cfg/reproducible.yaml I'd like to hear about those, so they'll get regularily tested and appear on the radar, until finally bugs are filed, fixed and migrated to stable. So one day all binary packages in Debian stable will be build reproducibly. An important step on this road is probably to have this defined as an release goal for Jessie+1. And then for jessie+1 hopefully the first 10k packages will build reproducibly? Or whooping 23k maybe? ;-) And maybe release jessie+2 with 100%?!? We will see! Even Jessie already has quite some packages (someone needs to count them...) which build reproducibly with just modified dpkg(-dev) and debhelper packages alone... So let's fix all the bugs! That said, an easier start for most of you is probably the list of useful things you (yes, you!) can do! :-) Oh, and last but surely not least in my book: many thanks too to the nice people hosting me so friendly in the last days! Keep on rockin'!

1 September 2013

Raphaël Hertzog: My Free Software Activities in August 2013

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (47.50 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Package Tracking System There are only 2-3 weeks left in the summer of code project dedicated to rewrite the package tracking system. We have come a long way during August check it out yourself in The rewrite doesn t have all the features of the old PTS yet, but I opted to keep some of the easy and less interesting features for others to re-implement. Instead I asked Marko to work in the coming weeks on new features that will bring more value, like the possibility to have user accounts with the possibility to easily review and tweak all your subscriptions on the web, and like the possibility to subscribe to groups of packages (i.e. those managed by a team). Our main problem right now is that exim has a pretty poor default behavior of forking hundreds of processes if you get hundreds of mails (in a batch) to an address that delivers via a pipe (postfix is saner, it serializes the deliveries on pipes). The new PTS is much more modular and its memory footprint is bigger (about 3 times more for the process that delivers mails, 30Mb instead of 10Mb), and in such a situation we managed to run out of memory for now we worked around the situation with an exim setting that queues mails once the load gets too high but it s a poor workaround IMO. We could obviously implement our own queue and a daemon but I d like to avoid this. So who knows how to tell exim to behave? :-) On the positive side, Marko has gotten some feedback from people who like the new PTS and are using it daily already. And several persons have expressed their interest to work on the new codebase already. On my side, I created a package so that it s easy to deploy for derivatives. In this process, I revamped the way we manage the Django settings (for development and for production). The package is not finished yet, but it s mostly usable already. But I still want to do some cleanup/refactoring in the models before others start deploying it. We must also enable South to make it possible to upgrade easily afterwards. DebConf 13 in Vaumarcus From August 10th to 17th, I was attending DebConf 13. It matched the only week of vacation that my wife had this summer so we went there with the whole family (that is with a 3 years old son, and 6 months old one). Thus I could not immerse myself in Debconf and missed all the nice things that happen outside of the talk rooms. I picked 3-4 interesting talks per day and I spent the rest with my family. On the positive side, I was pleased that my wife could meet (or at least see) some other Debian people. She knows quite a few (of you) by name because I have been telling her Debian stories for years now Debian France Debian France sold quite some merchandise during Debconf but I didn t take care of that. It was supervised by Sylvestre Ledru but fortunately he got the help of multiple persons, both to bring everything there, to sell it, and to bring back the rest. The good news of the month is that the upstream author of galette published a new version with all the features that we ordered him a few months ago. We send now automatic reminders to members who must renew their subscription, we have automatic update of our accounting books (in a ledger file in a git repository) when we people donate or pay their subscription via the paypal form on our website. I was so pleased to finally have this that I took some hours to finalize the packaging of galette, so that it could be uploaded to Debian. It s now waiting in the NEW queue. I also spent multiple hours to write the python script that is executed by galette and that updates the accounting files. Misc Debian stuff Debian Packaging. I did two uploads of logidee-tools to fix bugs #718671 and #718836. I created a package for Dolibarr a PHP-based CRM and ERP software (it doesn t do accounting however), it s sitting in the NEW queue for almost a month already. I forwarded #719000 to the upstream Publican developers. I filed #720393 to request a new upstream version of libphp-mailer. git-multimail. After its deployment on Alioth last month, Niels Thykier reported me a case where it lead to bounces, I filed this as a new upstream ticket and in fact I fixed it myself a few days after. I got the fixed version installed on Alioth. dpkg. I investigated why the the automatic builds of dpkg were no longer happening and asked Michael Prokop if he could install a newer version of gettext in the build chroot. He told me that he would need a backport for that so I asked Santiago Vila if he was willing to provide it and he kindly accepted. A few days after, the package was in backports and I m now again running the latest dpkg out of git thanks to the nice service provided by Michael. Misc discussions. The thread about user planets drifted into a discussion of how to avoid promotional posts on such planets and in that context someone again brought up the Debian Machine Usage Policy as a way to shut down any kind of (self-)promotional content on planet if there s money involved. This always irritates me and this time I opted to ask James Troup about the origin of that clause in the DMUP. So who is willing to work with DSA to fix the DMUP so that people stop abusing it in contexts where it doesn t make sense? I also participated in some discussions concerning dgit. I like the ideas behind the tool, but I m saddened by the behavior of Ian Jackson. I helped him to fill his gap of knowledge about new sources formats but he keeps on bashing about the 3.0 (quilt) source format both in the manual page and in the output of the program. He believes that dgit is no longer an experiment but the truth is that it s still a poorly commented Perl script doing lots of hackish things. Kali Linux Between Debconf and all, I haven t done much for Kali except a couple of fixes. There s a nice story of how I tracked a bug in live-installer on the Kali blog. That fix has been committed to Debian. I also improved live-build to include xfsprogs/jfsutils on the ISO image when you include the debian-installer (so that you don t end up in problems when you pick JFS or XFS as file systems for your installation). Thanks See you next month for a new summary of my activities.

One comment Liked this article? Click here. My blog is Flattr-enabled.

3 July 2011

Benjamin Drung: Symptoms of a dying graphic card

All started in December last year. I saw a red and a green pixel on my monitor. Moving the window containing the wrong-colors pixels corrected them. The wrong-colored pixels reappear from time to time in increased quantity. Then a second symptom appeared: The screen went black and came back one second later. This was triggered by moving a window and scrolling. I wasn t sure what the reason was. Either it was a bug somewhere in the X stack or some hardware was dying. dmesg showed multiple problems with the radeon driver:
[283808.667454] radeon 0000:01:00.0: ffff88021f815c00 unpin not necessary
[283808.667820] radeon 0000:01:00.0: GPU softreset
[283808.667823] radeon 0000:01:00.0: R_008010_GRBM_STATUS=0xE57024A4
[283808.667825] radeon 0000:01:00.0: R_008014_GRBM_STATUS2=0 00330302
[283808.667826] radeon 0000:01:00.0: R_000E50_SRBM_STATUS=0x200000C0
[283808.667832] radeon 0000:01:00.0: R_008020_GRBM_SOFT_RESET=0x00007FEE
[283808.682844] radeon 0000:01:00.0: R_008020_GRBM_SOFT_RESET=0 00000001
[283808.698840] radeon 0000:01:00.0: R_008010_GRBM_STATUS=0 00003028
[283808.698843] radeon 0000:01:00.0: R_008014_GRBM_STATUS2=0 00000002
[283808.698845] radeon 0000:01:00.0: R_000E50_SRBM_STATUS=0x200000C0
[283808.699845] radeon 0000:01:00.0: GPU reset succeed
[283808.717570] [drm] Clocks initialized !
[283808.765829] [drm] ring test succeeded in 0 usecs
[283808.765838] [drm] ib test succeeded in 1 usecs
[283808.765840] [drm] Enabling audio support
[283812.521265] radeon 0000:01:00.0: GPU lockup CP stall for more than 1000msec
[283812.521269] [ cut here ]
[283812.521294] WARNING: at /build/buildd/linux-2.6.35/drivers/gpu/drm/radeon/radeon_fence.c:235 radeon_fence_wait+0 365/0x3d0 [radeon]()
[283812.521297] Hardware name:
[283812.521299] GPU lockup (waiting for 0x00AE670A last fence id 0x00AE6705)
[283812.521301] Modules linked in: btrfs zlib_deflate crc32c libcrc32c ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs nls_utf8 udf ip6table_filter ip6_tables binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_intel kvm parport_pc ppdev snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_emu10k1 snd_ac97_codec ac97_bus snd_pcm snd_page_alloc snd_util_mem snd_hwdep snd_seq_midi radeon snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device ttm snd pl2303 drm_kms_helper coretemp usbserial joydev psmouse soundcore drm serio_raw intel_agp i2c_algo_bit lp parport hid_cherry usbhid hid firewire_ohci firewire_core usb_storage crc_itu_t e1000e ahci libahci pata_marvell
[283812.521360] Pid: 2194, comm: compiz Tainted: G W 2.6.35-24-generic #42-Ubuntu
[283812.521362] Call Trace:
[283812.521370] [<ffffffff8106089f>] warn_slowpath_common+0x7f/0xc0
[283812.521374] [<ffffffff81060996>] warn_slowpath_fmt+0 46/0 50
[283812.521390] [<ffffffffa01bd775>] radeon_fence_wait+0 365/0x3d0 [radeon]
[283812.521394] [<ffffffff8107f730>] ? autoremove_wake_function+0 0/0 40
[283812.521410] [<ffffffffa01bdf71>] radeon_sync_obj_wait+0 11/0 20 [radeon]
[283812.521418] [<ffffffffa01751a3>] ttm_bo_wait+0 103/0x1c0 [ttm]
[283812.521435] [<ffffffffa01d4e1a>] radeon_gem_wait_idle_ioctl+0x9a/0 150 [radeon]
[283812.521447] [<ffffffffa010f433>] drm_ioctl+0 463/0 520 [drm]
[283812.521465] [<ffffffffa01d4d80>] ? radeon_gem_wait_idle_ioctl+0 0/0 150 [radeon]
[283812.521470] [<ffffffff81162f0d>] vfs_ioctl+0x3d/0xd0
[283812.521473] [<ffffffff811637e1>] do_vfs_ioctl+0 81/0 340
[283812.521477] [<ffffffff811535f1>] ? vfs_read+0 181/0x1a0
[283812.521480] [<ffffffff81163b21>] sys_ioctl+0 81/0xa0
[283812.521484] [<ffffffff8100a0f2>] system_call_fastpath+0 16/0x1b
[283812.521487] [ end trace 6d5e03bab743abfa ]
[283812.521493] [drm] Disabling audio support
[283812.525575] [drm:radeon_ib_schedule] *ERROR* radeon: couldn t schedule IB(10).
[283812.525579] [drm:radeon_cs_ioctl] *ERROR* Faild to schedule IB !
[283812.527021] [drm:radeon_ib_schedule] *ERROR* radeon: couldn t schedule IB(11).
[283812.527024] [drm:radeon_cs_ioctl] *ERROR* Faild to schedule IB !
[283812.527921] [drm:radeon_ib_schedule] *ERROR* radeon: couldn t schedule IB(12).
[283812.527923] [drm:radeon_cs_ioctl] *ERROR* Faild to schedule IB !
The time between going black reduces every time. The system wasn t usable any more in the end. After testing that the screen was going blank on other systems like a live CD of Ubuntu and a not noteworthy proprietary system, it was clear that some hardware component was dying. My first assumption was verified after replacing it: My graphic card, a Radeon HD 4670, died. This was three weeks ago and around 25 month after I bought the graphics card. So I didn t have to worry if replacing the fan voided the guarantee, because the guarantee lasts only 24 month. Conclusion:
  1. Not every error is a software bug.
  2. The hardware component with the highest failure rate is the graphics card, followed by the motherboard. Four graphics cards died last year (in four different systems owned by four different people, in three different households).
  3. Not every error is a software bug.

18 November 2010

Erich Schubert: Myself on Google StreetView

I found myself on Google StreetView (which just launched today for some major cities in Germany)
I'm quite well recognizable if you know me: my (old, this is 2008) bicycle has this quite unique shape with the up-bent middle bar. At that time, the saddle was damaged, so it has tape on it. And I was wearing this black flat cap all the time back then. The T-Shirt probably says on it, along with an Onion ... and I can fix the date: it must have been the July 10 2008
If you wonder how I found myself: I used this great search engine called "brain memory". When I picked up my thesis from binding (I was heading to hand in my final thesis, that somewhat makes the day special, doesn't it?) I saw the Google car. And obviously, I also remember where I had my thesis bound. So all I needed was to type in the address and see if they used the footage.
On a side note, it's interesting to see how many people let themselves be scared into getting their houses blurred by the politicians. Google StreetView is to German politics what terrorists are to the US: a campaign.

18 March 2009

Biella Coleman: FLOSS Manuals in Cambdrige, MA March 21-22nd

I recently got to spend some time hanging with part of the FLOSS manual crew and they basically are as busy as ever holding sprints across the US and Europe to write documentation and manuals for Free Software. They are holding a mini sprint in Cambridge, MA and along with the FSF will be writing a manual on the command line in about two days. If the idea of writing documentation makes you feel warm and happy inside, then this is the even for you. They are always looking for volunteers and I assure you, the experience will be well worth your time, effort, and participation.

3 November 2008

Enrico Zini: rc-buggy packages sorted by popularity

rc-buggy packages sorted by popularity After doing a round of tag reviews so that we release lenny with the latest tag contributions, I still had a bit of spare time. Since people are urging to fix the last bugs I thought I could contribute a list of RC-buggy packages sorted by popularity:
import urllib2
import sys, re #, gzip
PKGRE = re.compile(r'<strong>Package: </strong><a href="[^"]+" name="([^"]+)">')
POPCONRE = re.compile(r'^(\d+)\s+(\S+)')
PKGMAPRE = re.compile(r'^([^(]+)\(([^)]+)')
COMMASPLIT = re.compile(r'\s*,\s*')
# Allow to download other package views
if len(sys.argv) > 1:
    url = sys.argv[1]
    url = ""
def read_packages(url):
    "Download the list of rc-buggy packages"
    fd = urllib2.urlopen(url)
    for line in fd:
        mo =
        if mo:
# FIXME: this is a deprecated data source, but at the moment it is just what
#        is needed
def read_bintosrc(url = ""):
    "Download the binary to source package map"
    fd = urllib2.urlopen(url)
    for line in fd:
        mo =
        if mo:
            src =
            bin = COMMASPLIT.split(
            for p in bin:   
                yield p, src
def read_popcon(url = ""):
    "Download popcon information"
    fd = urllib2.urlopen(url)
    # It doesn't work on urllib2 objects because it wants tell()
    #fd = gzip.GzipFile(fileobj=fd, filename=url, mode="rb")
    for line in fd:
        mo = POPCONRE.match(line)
        if mo:
            yield, int(
print "Reading mapping of binary packages to source packages..."
bin2src = dict(read_bintosrc())
print "Reading popcon scores..."
# We actually want the score of source packages, and for it we use the score of
# its binary package with the highest score
scores = dict()
for pkg, score in read_popcon():
    pkg = bin2src.get(pkg, pkg)
    if pkg not in scores:
        scores[pkg] = score
#scores = dict(read_popcon(""))
print "Reading list of rc-buggy packages..."
# The bug page has a mix of source and binary packages: convert all to source
# packages
packages = sorted(read_packages(url), key=lambda x:scores.get(bin2src.get(x,x), 0))
for p in packages:
    print scores.get(bin2src.get(p,p), 0), p
This can be run on as ~enrico/popzimmer (0 is the rank of packages that for some reason cannot be mapped to any rank):
$ ~enrico/popzimmer 
Reading mapping of binary packages to source packages...
Reading popcon scores...
Reading list of rc-buggy packages...
0 roxen-fonts-iso8859-2
2 dpkg
11 pam
61 glibc
61 libc6
123 initramfs-tools
149 bind9
150 grub
180 portmap
213 nfs-common
279 libgl1-mesa-dri
298 libsnmp-base
298 snmpd
309 avahi
397 mysql-dfsg-5.0
408 xserver-xorg-input-wacom
409 xserver-xorg-video-vesa
428 docbook-xml
567 xine-lib
600 evolution-data-server
669 eog
671 xserver-xorg-video-intel
691 epiphany-browser
691 epiphany-gecko
695 kdelibs4c2a
698 uswsusp
754 guile-1.6
959 gpm
964 cups
990 kdebase
1002 ghostscript
1069 giflib
1076 xulrunner-1.9
1177 libapache2-mod-perl2
1217 vlc-nox
1229 kernel-package
1311 kdeutils-dev
1520 libparted1.8-10
1767 wireshark
1785 selinux-policy-default
1908 dia
1943 htop
1983 boost
2140 cantlr
2145 transmission
2159 emacs22
2312 swfdec-mozilla
2449 eclipse
2449 libswt3.2-gtk-java
2728 lilo
2898 libnss-ldap
3013 java-access-bridge
3051 blender
3099 libtemplate-perl
3499 fglrx-atieventsd
3602 virtualbox-ose
3619 jfsutils
3806 anjuta
3962 kbuild
4076 libgnustep-gui0.14
4222 libwebkit-1.0-1
4332 rosegarden
4412 matplotlib
4412 python-matplotlib
4950 mail-notification-evolution
5195 nut-cgi
5273 sbcl
5538 ruby1.9
5679 kmymoney2
6076 websvn
6147 miro
6264 gnat-4.1
6431 gallery2
6478 gnome-chess
6512 joystick
6862 istanbul
6937 wordpress
7476 luatex
7635 csound
7635 python-csoundac
8257 libpam-mount
8471 wmcpuload
8631 nbsmtp
8642 oxine
8774 axiom
8949 libengine-pkcs11-openssl
9052 clamav-getfiles
9221 open-iscsi
9262 ptex-bin
9701 egroupware-core
10012 debian-cd
10127 libjogl-java
10453 libjson-xs-perl
10698 gcjwebplugin
10736 request-tracker3.6
11294 python-extclass
11518 rkward
11600 alevt
12929 otrs2
13205 bindgraph
13956 icecc-monitor
14223 netmaze
14505 cdebconf
14526 mumble
14687 fnonlinear
14724 recite
14789 jbidwatcher
15766 libnss-ldapd
15812 glpi
15905 lustre-source
15982 win32-loader
16232 ampache
17168 libghc6-missingh-dev
17311 acl2
17830 apertium
18462 emacspeak
18469 glassfish
19257 isight-firmware-tools
19719 libnagios-object-perl
20144 libwx-perl
20465 dns2tcp
21614 python-kinterbasdb-dbg
21690 hf
21902 libcobra-java
22365 tomcat6
22573 libphp-snoopy
23501 coco-java
23926 boxbackup-server
24009 libghc6-wash-dev
26323 libdesktop-notify-perl
26439 opendb
28426 adolc
28795 xorp
29986 libgdata-java
31262 libjboss-serialization-java
31472 pixelpost
31658 cournol
32939 mediamate
34365 gforge-plugin-scmcvs
34659 libgearman-client-async-perl
42233 libhamcrest-java
48076 mahara
55303 mxallowd
85881 roxen-fonts-iso8859-1

So, if you don't know from what package to start fixing bugs, "Begin at the beginning,", the King said, very gravely, "and go on till you come to the end: then stop."

7 October 2008

Andrew Pollock: [debian]

$ host -t txt descriptive text "Javier Pen~a <>" descriptive text "PGP D6 91 BB 89 88 7A F7  A0 BA E4 27 FD 7A 6A FE DA" descriptive text "PGP 86AA DB04 D2F5 CC55  3A4D 940F B1A9 DD82 DC81 4B09"

30 April 2008

Adrian von Bidder: Filesystems in Linux

With Hans Reiser convicted for murder, some seem to feel that reiserfs is more or less dead. Jason Perlow writes a very strange article on ZDNet to which I'm replying to it mainly because he alludes that Debian so far has failed to react. First, default installations of Debian create ext3 and not reiserfs filesystems (Please correct me if I'm wrong. I've just recently installed a fresh etch, but I didn't specifically look at the fs.) And even if it were reiserfs (v3), I don't see why a reaction would be called for now. The stability of reiserfs has come up every once, before the whole murderer story begun, and that the interaction between the reiserfs developers (including, of course, Hans Reiser) and the kernel team were always difficult has also been known for a long time. This is the kind of reason where I think it's appropriate for Debian to take steps (i.e. switching to a different filesystem), not a single event, where it is not even clear yet how the reiserfs (v3 and v4) efforts will move on. On to the technical stuff: Perlow tries, but doesn't really arrive at understanding the issues he's writing about. Reiser 4 is discounted without a single remark on its technical merits (I can't comment either as I have not looked at it so far.) Why he discounts ext4 is not clear to me (because it is not ripe for production use yet — but that's even more true of ZFS and this Linux-NTFS thingy he rambles about further down...) He discounts JFS2 because it hasn't got a new release for several years (is that bad in a filesystem?) but then touts ZFS as a great idea with minor licensing problems, without speaking of patents which is where the real problems lie (not to mention the fact that the Linux ZFS port probably is much less tested than ext4 or JFS.) And in a final jump into fantasy-land he mentions that NTFS might just be ideal for Linux, and Microsoft is said to have started cooperating nicely with the Free Software world, so all licensing and patent issues are certainly going away Real Soon Now™. At least for Novell, these issues shouldn't be a problem, I guess. Not mentioned by Jason are btrfs (which has a quite tightly coupled network filesystem brother, crfs and is in a very early state of development), and hammer, which comes from the BSD world and currently lacks a Linux port. Both efforts are probably more likely to replace ext3 or reiser on Linux than both ZFS and NTFS: no patent issues, no license issues, and the development is actually done by a community and not a single company. Update: Julien Blanche ha a much more succinct response to Jason Perlow. More intersting to read than mine, too.

18 November 2007

Theodore Ts'o: Does perfect code exist? (Abstractions, Part 1)

Bryan Cantrill recently wrote a blog entry, where among other things, he philosophized on the concept of “perfect code”. He compares software to math, arguing that Euclid’s greatest common denominator algorithm shows no sign of wearing out, and that when code achieves perfection (or gets close to perfection), “it sediments into the information infrastructure” and the abstractions defined by that code becomes “the bedrock that future generations may build upon”. Later, in the comments of his blogs, when pressed to give some examples of such perfection, he cites a clever algorithm coded by his mentor to divide a high resolution timestamp by a billion extremely efficiently, and Solaris’s “cyclic subsystem”, a timer dispatch function. Watching his talk at Google where his introduction and sidebar book review on Scott Rosenberg’s “On Dreaming in Code”, it’s clear that he very passionately believes that it is possible to write perfect code, and that one should strive for that at all times. Perhaps that’s because he mostly writes code for operating systems, where the requirements change slowly, and for one OS in particular, Solaris, which tries far harder than most software projects to keep published interfaces stable for as long as possible. In contrast, the OS I’ve spent a lot of time hacking around, Linux, quite proudly states that at least inside the kernel, interfaces can not and should not be stable. Greg Kroah-Hart’s “Stable API Nonsense” is perhaps one of the strongly and most passionate expositions of that philosophy. I can see both sides of the argument, and in their place, both have something to offer. To Bryan’s first point, it is absolutely true that interfaces can become “bedrock” upon which entire ecosystems are built. Perhaps one of the most enduring and impactful example would be the Unix programming interface, which has since become enshrined by POSIX.2 and successor standards. I would argue, though, that it is the interface that is important, and not the code which initially implemented it. If the interface is powerful enough, and if it appears at the right time, and the initial implementation is good enough (not perfect!), then it can establish itself by virtue of the software which uses it becoming large enough that it assumes an important all out of scale with its original intention. Of course, sometimes such an interface is not perfect. There is an apocryphal story that when S. Feldman at AT&T labs first wrote the ‘make’ utility, that he did so rather quickly, and then put it available for his fellow lab members to use, and then went home to sleep. In some versions of the story he had stayed up late and/or pulled an all-nighter to write it, so he slept a long time. When he came back to work, he had come up with a number of ways to improve the syntax of the Makefile. Unfortunately, (so goes the story) that too many teams were already using the ‘make’ utility, so he didn’t feel he could change the Makefile syntax. I have no evidence that this ever took place, and I suspect it is an urban myth that was invented to explain why Makefiles have a rather ugly and unfortunate syntax that many would call defects, including the use of syntactically significant tab characters which are indistinguishable from other forms of leading whitespace. Another example which is the bane of filesystem designers everywhere are the Unix readdir(2), telldir(2), and seekdir(2) interfaces. These interfaces fundamentally assume that directories are stored in linear linked lists, and filesystems that wish to use more sophisticated data structures, such as b-trees, have to go to extraordinary lengths in order support these interfaces. Very few programs use telldir(2) and seekdir(2), but some filesystems such as JFS maintain two b-trees instead of one just to cater to telldir/seekdir. And yet, it is absolutely true that interfaces can be the bedrock for an entire industry. Certainly no matter what its warts, the Unix/Posix interface has proven the test of time, and it has been responsible for the success of many a company and many billions of dollars of market capitalization. But is this the same as perfect code? No, but if billions of dollars of user applications are going to be depending on that code, it’s best if code which implement such an interface be high quality, and should attempt to achieve perfection. But what does it mean for code to be perfect? For a particular environment, if the requirements can be articulated clearly, I can accept that code can reach perfection, in that it becomes as fast as possible (for the given computer platform), and it handles all exception cases, etc., etc. Unfortunately, in the real world, the environment and the requirements inevitably change over time. For example, Bryan’s cyclic subsystem, which he proudly touts as being if not perfect, almost so, and which executes at least 100 times a second on every Solaris system in the world. I haven’t looked at the cyclic system in any detail, since I don’t want to get myself contaminated (the CDDL and GPLv2 licenses are intentionally incompatible, and given that companies — including Sun — have sued over IPR issues, well, one can’t be too careful), but waking up the CPU from its low-power state 100 times a second isn’t a good thing at all if you are worried about energy conservation in data centers — or in laptops. For example, on my laptop, it is possible to keep wakeups down to no more than 30-35 times a second and it would be possible to do more but for an abstraction limitation. Suppose for example an process wants to be sleep and then receive a wakeup 1 milliseconds later, and so requests this via usleep(). At the same time, another application wants to sleep until some file descriptor activity takes place, or after 1.2 milliseconds takes place. 0.3 milliseconds later, a third process requests a sleep, this time for 0.8 milliseconds. Now, it could be that in all of the above cases, the applications don’t actually need exact timing; if they all get their wakeups plus or minus some fraction of a millisecond, they would be quite cool with that. Unfortunately, the standard timer interfaces have no way of expressing this, and so the OS can’t combine the three wakeups at T+1.0, T+1.1, and T+1.2 milliseconds into one wakeup at T+1.1ms. So this is where Greg K-H’s “Stable API Nonsense” comes into play. We may not be able to solve this problem at the userspace level, but we darn well can solve this problem inside the kernel. Inside the kernel, we can change the timer abstraction to allow device drivers and kernel routines to provide a timer delta plus a notion of how much accuracy is required for a particular timer request. Doing so might change a timer structure that previously external device drivers had depended upon — but too bad, that’s why stable ABI/API’s are not supported for internal kernel interfaces. Is this that the interface could have been extended? Well, perhaps, and perhaps not; if an interface is well designed, it is possible it can be extended in an API and/or ABI compatible way. There usually is a performance cost to doing so, and sometimes it may make sense to pay that that cost, and sometimes it may not. I’ll talk more about that in a future essay. Yet note what happened to the timer implementation. We have a pretty sophisticated timer implementation inside Linux, that uses heap data structures and buckets of timers for efficiency. So while I might not call it perfect, it is pretty good. But, oops! Thanks to this new requirement of energy efficiency, it will likely need to get changed to support variable levels of accuracy and the ability to fire multiple timers that are (more or less) coming due in a single CPU wakeup cycle. Does that make it no longer perfect, or no longer pretty good? Well, it just had a new requirement impact the code, and if criteria for perfection is for the abstraction defined by the code to be “bedrock” and never-changing, and no need to make any changes in said code over multiple years, I would argue that little to no code, even OS code, can ever achieve perfection by that definition — unless that code is no longer being used. (This is the first of a multi-part series of essays on abstractions that I have planned. The next essay which I plan to write will be entitled “Layer surfing, or jumping between layers”, and will be coming soon to a blog near you….)

17 July 2007

Miriam Ruiz: Some more pending packages

I said I was going to upload the packages I don’t have time to finish for the moment, but that are in good shape, meaning they compile without problem in latest SID, and are just lacking man pages, icons and so, here’s the next round of them:

5 April 2007

Evan Prodromou: 14 Germinal CCXV

Snow, beautiful snow! We've got tons of snow coming out of the sky today. It's big and thick, what they call fourrure de li vre (rabbit fur) in Quebec. It's hard to believe we are having snow this late in the season (halfway into Germinal!), but it's kind of nice to have some snow. Especially since the United Nations report saying that Canada will lose its snow cover by 2100. Snow, please snow. tags:

JFS So, I'm one of those idiots who uses the IBM Journaled File System 2 (JFS2) as his root partition. Why? Because I've tried all the different journalling filesystems for Linux (reiserfs, ext3, xfs, jfs) and JFS is the one that works for me. It just seems to be the low-profile, clean, nicely-working FS. Unfortunately, I've been tracking Feisty Fawn for the last couple of weeks, and somewhere in there the JFS utility package, jfsutils, got uninstalled. Who knows why? But when I rebooted my machine today to try out a new kernel, my root partition came up in read-only mode -- because the system couldn't check the partition, because fsck.jfs couldn't be found. The fix, of course, would be to boot a live CD, chroot to the hard disk, and apt-get install the jfsutils. I typically use Finnix for this and other rescue/recovery projects, but in this situation Finnix failed me. My main computer is a honkin' amd64, and the x86 Finnix disk didn't want to let me chroot and run any amd64 binaries (like the stuff I needed to apt-get install jfsutils). So I ended up downloading and burning a Feisty live CD, dicking around with boot parameters for a while (noapic nolapic nosplash nonothing...), and finally got to a nice, friendly Gnome terminal. I mounted the hard drive using the Nautilus tools, and then tried to chroot to it -- D'OH! Permission denied. Took me a while to figure out that Nautilus mounts drives noexec by default. Once I got that working, I got jfsutils installed and was able to boot my machine again. Yay. Moral of the story: figure out a way to make packages that are crucial to you crucial to your package manager, too. Oh, and Finnix needs an amd64 build. tags:

YULblog YULblog meetup tonight at the Quincaillerie. YULblog is our local Montreal monthly blogging get-together. It's pretty fun, and has the highest female-to-male ratio of any technology event in Montreal. Which is great. I have a ton of work to do tonight (since I spent the afternoon dicking around with live CDs), but I'm going out anyways. YULblog comes but once a month, after all. tags:

N hours later Good time, that! I walked down rue Rachel to the Quincaillerie this evening around 9PM through a storm that was like being in a snowfight with Borribles. Thud thud thud! Big chunks of snow. But the Q. was pretty hoppin' for a snowy night in April, and I got to see Robin, Steve, Niko, Patrick and others. It was a fun time; the snow made it all the more special. tags:

14 August 2006

Andrew Pollock: [tech] Found one small problem with using ATA over Ethernet

And I can't believe I'm the only person who's struck this, but so far Google hasn't turned up any useful leads. The problem is pretty obvious, when the system boots (well Debian Etch at least), it runs at priority 35 in /etc/rcS.d. That tries to mount all the local filesystems (it does the determination for what is locate or remote by the filesystem type in /etc/fstab, there's a hard-coded list of filesystem types to explicitly not try and mount). Then the network comes up at priority 39. The problem's rather obvious. I did a reboot test tonight to see if everything would load and come up correctly on reboot, and well, it didn't even boot up properly, because /srv couldn't be fscked, let alone mounted, and that's my AoE volume. My /etc/fstab contains:
/dev/etherd/e0.0        /srv    jfs     defaults        0       0
The problem is, as far as the init scripts are concerned, based on filesystem-type, this is a local filesystem. The init scripts don't have the concept of a remote block device. I tried moving all the networking initialisation scripts before the script, but think there's an inherent delay between the network coming up and the aoe driver figuring out what shelves and slots are actually available and creating the relevant entries in /dev/etherd The thing is, this seems to be a rather essential thing that I'm trying to do, so I can't believe I'm the first person to run into this. I'll sleep on it and see if the lazyweb comes up with anything...

13 August 2006

Andrew Pollock: [tech] More benchmarking

When I did my Red Hat Certified Engineer training, the second time around (when I updated from Red Hat 7.1 to Enterprise Linux 3), the trainer mentioned this Ext3 vs Reiserfs benchmark that Gurulabs performed. Probably because it had such a memorable name, I've remembered it and referred people to it, and referred to it myself, a few times over the years since. So it seemed only fair to try out their benchmarking methods (i.e. use NetApp's postmark with a few different settings) on the uber-disk as well, while I had it idle. So I did. Firstly, I ran it locally on minotaur, the piddling little 866Mhz Pentium III, with 320Mb of RAM (but with 1.5Tb of disk attached directly via USB), and then I ran the exact same tests from teevee, the box destined to become my MythTV server (3.0Ghz Pentium 4, 1Gb of RAM), with the disks mounted with ATA over Ethernet. This was with a JFS filesystem on a single logical volume striped across four physical volumes. I ran the tests three times, just like the Gurulabs benchmark, first time around, it was with: The idea being that this will generate around 150Mb of data, which should all fit in the RAM (of either system) The second time around was with: The idea being that this, generating 1400Mb of data, will exceed the RAM on both systems (but not uniformally, obviously). The final run was a total thrash with: This one causes about 19Gb of data to be written, and absolutely hammers the living daylights out of the disks (or the network, and the disks, in the case of ATAoE). So, now for the results:
Test case 1, with locally attached disks
        22 seconds total
        21 seconds of transactions (2380 per second)
        27170 created (1235 per second)
                Creation alone: 2000 files (2000 per second)
                Mixed with transactions: 25170 files (1198 per second)
        24986 read (1189 per second)
        24697 appended (1176 per second)
        27170 deleted (1235 per second)
                Deletion alone: 2340 files (2340 per second)
                Mixed with transactions: 24830 files (1182 per second)
        149.53 megabytes read (6.80 megabytes per second)
        161.85 megabytes written (7.36 megabytes per second)
Test case 1, with ATAoE attached disks
        27 seconds total
        25 seconds of transactions (2000 per second)
        27170 created (1006 per second)
                Creation alone: 2000 files (2000 per second)
                Mixed with transactions: 25170 files (1006 per second)
        24986 read (999 per second)
        24697 appended (987 per second)
        27170 deleted (1006 per second)
                Deletion alone: 2340 files (2340 per second)
                Mixed with transactions: 24830 files (993 per second)
        149.53 megabytes read (5.54 megabytes per second)
        161.85 megabytes written (5.99 megabytes per second)
A degradation in performance, sure, but not a huge one. Graph of the results
Test case 2, with locally attached disks
        218 seconds total
        209 seconds of transactions (239 per second)
        27144 created (124 per second)
                Creation alone: 2000 files (400 per second)
                Mixed with transactions: 25144 files (120 per second)
        24830 read (118 per second)
        25094 appended (120 per second)
        27144 deleted (124 per second)
                Deletion alone: 2288 files (572 per second)
                Mixed with transactions: 24856 files (118 per second)
        1405.24 megabytes read (6.45 megabytes per second)
        1535.91 megabytes written (7.05 megabytes per second)
Test case 2, with ATAoE attached disks
        165 seconds total
        158 seconds of transactions (316 per second)
        27144 created (164 per second)
                Creation alone: 2000 files (1000 per second)
                Mixed with transactions: 25144 files (159 per second)
        24830 read (157 per second)
        25094 appended (158 per second)
        27144 deleted (164 per second)
                Deletion alone: 2288 files (457 per second)
                Mixed with transactions: 24856 files (157 per second)
        1405.24 megabytes read (8.52 megabytes per second)
        1535.91 megabytes written (9.31 megabytes per second)
Whoa, who'd have thought this? Better performance over the wire? I'm blaming the fact that minotaur is a bit of a weakling compared to teevee, and teevee had a better caching advantage with more RAM. Mind you, you'd expect to see the same for the first test as well, unless size is a factor. Inconclusive results I guess. Graph of the results
Test case 3, with locally attached disks
        7356 seconds total
        7310 seconds of transactions (27 per second)
        104099 created (14 per second)
                Creation alone: 4000 files (117 per second)
                Mixed with transactions: 100099 files (13 per second)
        100246 read (13 per second)
        99560 appended (13 per second)
        104099 deleted (14 per second)
                Deletion alone: 4198 files (349 per second)
                Mixed with transactions: 99901 files (13 per second)
        18995.06 megabytes read (2.58 megabytes per second)
        19738.85 megabytes written (2.68 megabytes per second)
Test case 3, with ATAoE attached disks
        12320 seconds total
        12111 seconds of transactions (16 per second)
        104099 created (8 per second)
                Creation alone: 4000 files (32 per second)
                Mixed with transactions: 100099 files (8 per second)
        100246 read (8 per second)
        99560 appended (8 per second)
        104099 deleted (8 per second)
                Deletion alone: 4198 files (49 per second)
                Mixed with transactions: 99901 files (8 per second)
        18995.06 megabytes read (1.54 megabytes per second)
        19738.85 megabytes written (1.60 megabytes per second)
So presumably the network latency made its presence felt with data volumes of this magnitude. Graph of the results

10 August 2006

Martin F. Krafft: Through with XFS

I am through with XFS, once and for all. Well, at least for laptops. I still think it's a good filesystem when you can ensure that the power never goes, and your hardware is reliable, but it's just not adequate for laptops or even desktops. I ran into some serious problems a while ago, but managed to recover. Two nights ago, however, three XFS filesystems on my laptop decided to blow up and left my system thoroughly broken. I guess as the hibernate maintainer, I should really start doing my tests somewhere else than my main system... It all started out with a dist-upgrade and this output:
dpkg: error processing /var/cache/apt/archives/dpkg_1.13.22_i386.deb
(--unpack): unable to make backup link of
'./usr/share/man/man1/dpkg-deb.1.gz' before installing new version: Unknown
error 990
Looking at /usr/share/man/man1, I started to anticipate the apocalypse:
# ls -l /usr/share/man/man1
total 7956
?????????? ? ? ?    ? ? ? 7zr.1.gz
?????????? ? ? ?    ? ? ? 822-date.1.gz
?????????? ? ? ?    ? ? ?
?????????? ? ? ?    ? ? ? Defoma::Common.1.gz
So I look at the log, and amidst kernel oops notices, there's this lovely cookie:
Filesystem "hda6": Corruption of in-memory data detected.  Shutting down
filesystem: hda6
Filesystem hda6 is /usr, so at that time I figured "it could have been worse", booted to single user, and remade the filesystem with the intention to simply reinstall all packages... when I found /var/lib/dpkg/info to be in similar condition. The rest of /var seemed fine, but I resolved then that there was no hope in reviving this system. Fortunately I brought an external drive that had just enough free space to hold my /home and some other stuff, but since USB is really slow when it comes to shifting large amounts of data, I decided to do something productive in the mean time and to answer some outstanding mails. It wasn't difficult to get SSH back up, so I started to work on a remote machine and used the time efficiently. Some time later, though, I got confused in the mist of screen sessions and was browsing my home directory on the laptop, thinking I was elsewhere (my home directories are mostly synchronised), when I noticed a directory in similar condition as the above. Oh shit. Imagine my pain and fear as I first thought my remote machine was also dying, imagine the sigh when I found out I was on the local filesystem, and imagine the shock when I realised that /home was also affected by the XFS breakage... A glance around /var confirmed that the XFS breakage was actually spreading and had now affected three filesystems on this machine. Fortunately, by that time, I had copied everything to the external drive, and decided to put my laptop and myself to sleep. I woke the next morning to the task of reinstalling the thing and decided to be optimistic about it. After all, a reinstall would mean I could finally try partman-crypto and encrypt my laptop's data to protect against leaking sensitive stuff in the case of loss or theft of my laptop. The installation was not as painless as I had hoped, but that was mainly because I ran into a known problem with the graphical installer and partman-crypto, which does not allow to set up volumes with random encryption keys (e.g. swap; see the forthcoming announcement for the beta3 release of the installer), and a bunch of smaller bugs. I had to restart the installation with the traditional frontend to get what I wanted, but other than that, I was very impressed with what our installer development team has accomplished! And a special round of gratitude to Frans Pop for not losing his patience while helping me on several occasions throughout the process. Now, 24 hours after the incident, I am back to normal with a fresh laptop and no data lost (except for one directory which I pulled from a mirrored remote machine; it had no local modifications (so why did XFS screw it up anyway?)). The fonts are all jaggy, so there's something I have to figure out. All things considered, I am sad to have lost 24 hours, but I can also relax more now, without fear of further XFS breakage or loss of private data. Update: Oh, and despite this, I did choose ext3 for all my laptop's filesystems. JFS was really cumbersome and slow last time I tried it, and I surely would not touch RazorFS after experiencing serious data loss on numerous occasions. Update: Two responses so far. Full ack for Julien (except for him laughing at me), Ingo's post warrants a reply though. First, ext3 is also journaled, and if you're about to say "yeah, but it's a hack on top of ext2, well... ext2 is damn mature, and journaling isn't really rocket science, so that "hack" isn't going to be too complicated. In fact, I like the idea of journaling being an option rather than a built-in feature. Second, of course you're supposed to keep backups. But since you keep backups, my top requirement of a filesystem is not "how to get the data back", but "how to ensure it does not break. If it breaks, I can reinstall and restore from backup, but that's a certain amount of time lost. If it doesn't break, well, that's like stealing a little something back from death then, isn't it? Third, I do follow the linux-xfs mailing list, but so what? I did not have write cache enabled, and I was running the kernel at the time of the mishap. Lastly, you point to "excellent tools" to recover the filesystem. I am not sure how excellent xfs_repair really is when it reports "bad magic number 0x0 on dir inode 4696727" during the run, claims to have fixed it, I mount the filesystem, unmount it, run xfs_repair again and get the same message. No filesystem is perfect, and as we know from Biella's problems (among many others, ext3 is no exception. But we did get her data off! So then it's really an open field again, crap filesystem against crap filesystem. I guess at this point it helps to know that ext3 actually follows VFS semantics, while on XFS, a completed sync() syscall does not actually mean it has written the data to disk (see e.g. #317479). And then there are bugs like #239111... ext3 it is for now. If that let's me down, I'll try JFS. If that fails and noone has actually implemented a proper filesystem, I might have a go myself. Haha. Update: Alceste Scalas adds:
Ingo is right when he says that every filesystem has bugs --- but bugs apart, the design of Ext3 (i.e. its physical-block journaling) makes it a far more reliable choice for desktop and laptop PCs, expecially for people without an UPS. An Ext3 filesystem could only crash because of a bug or an hardware failure, while an XFS filesystem can be trashed even without bugs or hardware failures, due to the unavoidable consequences of a power loss on PC-class hardware.
He also alerted me to this mailing list post, which compares data=ordered journaling of ext3 (which almost noone does for performance reasons) with XFS and RazorFS. Update: You may also be interested in this post. Update: Otavio Salvador points me to this FAQ entry, which SGI must have added very lately. It explains how to deal with the directory corruption that was part of my problem. I guess I would have liked to know earlier, but I consider the outcome with dm-crypt + ext3 a win anyhow.

Andrew Pollock: [tech] Performance comparison between LVM and a straight disk

The beauty of having your hardware arrive in installments is you can do all sorts of fooling around with lots of storage that you wouldn't otherwise do because you'd be falling over yourself to get it all up and running yesterday. So as I impatiently wait for my TV tuner card to arrive, I've been doing some "benchmarking" to see how various different configurations perform. One of the things I wanted to find out was how much overhead did LVM give you. So I took one of the four disks in my non-redundant array of Bloody Huge Storage, and ran bonnie++ on it as a straight JFS volume on /dev/sda and as a logical volume in a volume group that used all of the logical extents provided by making /dev/sda into a physical volume. Again, I ran bonnie++ as:
bonnie++ -d /mnt -s 2G -n 1024 -u nobody:nogroup
So here's the results of running it on a JFS filesystem straight on /dev/sda:
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
minotaur         2G 13712  95 18561  21  9152  11 13874  94 20446  18 142.2   1
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
               1024  5999  37  7125  19  1604  12   616  12   168   0    61   1
And here's the results of running it on a JFS filesystem on a logical volume that used all of the free logical extents provided by a volume group consisting of a physical volume made from /dev/sda
Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
minotaur         2G 13790  96 18566  21  9132  11 14202  96 20628  18 146.3   1
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
               1024  5989  37  7056  20  1605  12   611  12   167   0    61   1
Yup, that's right. It was actually slightly faster in a few cases. I can't explain that one. But it certainly goes to show that there is no noticeable performance impact of using LVM. Nice.

5 August 2006

Andrew Pollock: [tech] I can only presume I am an idiot

So it seems I was a bit quick to declare the USB controller a dud last night. Overnight, Simon Richter attested on IRC that he also had a USB controller bearing the same PCI ID, and his worked fine. (Although his had the option for extra power from the floppy drive power cable, which I find interesting. Mine does not). So tonight I worked through the various combinations to get to the bottom of the problem. I put a Sun QFE card in that PCI slot to make sure the riser and the slot were working correctly. It worked fine. Then I put the USB controller back in and tried a USB thumb drive, that worked. So I immediately narrowed in on the hard drive itself, and the IDE-USB adapter. I tried it on my laptop. Nothing. Aha. So I tried the hard drive directly with IDE, and it worked. So I assumed that the IDE-USB adapter must have been faulty, and tried a different one. It worked. I cycled through the other three IDE-USB adapters, and they all worked. Then I tried again with the one from last night, and it too worked! I can only presume that in my haste, or with my fuzzy head (I've got a cold or the 'flu or something), I managed to connect the IDE-USB adapter to the hard drive incorrectly. Don't ask me how, considering it's a keyed interface, but it's not the first time I've managed to screw up plugging in an IDE hard drive. Anyway, with that hurdle behind me now, I made four physical volumes out of the disks (no partitions at all), and then made a volume group, and a single logical volume that is striped across all four physical volumes (yes, I know if I have a disk failure I can kiss the lot goodbye, but I'm in performance testing mode at the moment). I then exported that logical volume with vblade and mounted it on what will be my MythTV box.
apollock@teevee:~$ df -Th
Filesystem    Type    Size  Used Avail Use% Mounted on
              ext3    259M  156M   90M  64% /
tmpfs        tmpfs    502M     0  502M   0% /dev/shm
/dev/sda1     ext3    228M   25M  192M  12% /boot
              ext3     63G  295M   59G   1% /home
              ext3    368M  8.1M  340M   3% /tmp
              ext3    4.7G  1.7G  2.8G  38% /usr
              ext3    2.9G  897M  1.8G  33% /var
tmpfs        tmpfs     10M  124K  9.9M   2% /dev
               jfs    1.5T  2.2G  1.5T   1% /mnt
I've currently got bonnie++ hammering away at the "disk" to see what sort of performance I'll get, and whether it'll be fast enough to record TV. Now I just need the TV tuner card to turn up, and I can start fiddling in earnest.

25 July 2006

Julien Danjou: DeDuBu contest #1

Bug Welcome to this 1st issue of DeDuBu contest, the monthly championship of the dumbest bug reported to the Debian BTS. The challengers How the vote has been done Five Debian related people voted for these bugs: Pierre Habouzit, Pierre Machard, Julien Louis, Mohammed Adn ne Trojette and Julien Cristau. Full ranking Bugs Challengers The winners About DeDuBu