If for some reasons you can t use dedicated DBaaS for your PostgreSQL (like AWS RDS) then you need to run your database server on a cloud instance. In these kind of setup, when you scale up or down your instance size, you need to adjust PostgreSQL parameters according to the changing RAM size. There are several parameters in PostgreSQL that highly depend on RAM size. An example is shared_buffers for which a rule of thumb says that is should be set to 0.25*RAM. In DBaaS, when you scale the DB instance up or down, parameters are adjusted for you by the cloud provider, e.g. AWS RDS uses parameter groups for that reason, where particular parameters are defined depending on the size of the RAM of the RDS instance. So what can you when you do not have RDS or any other DBaaS? You can always keep several configuration files on your instance, each for a different memory size, you can rewrite you config every time you change the size of the instance or you can use Ansible role for that. Our Ansible role will be very simple, we will have two tasks. One will change the PostgreSQL config, the second one will just restart the database server: Now we need the template, where are the calculations take place. RAM size will be taken from the Ansible s fact called ansible_memtotal_mb. Since it returns RAM size in MBs, we will stick to MBs. We will define the following parameters, you can adjust them to your needs:

• shared_buffers, as 0.25*RAM size,
• work_mem, as shared_buffers/max_connections,
• maintenance_work_mem, as RAM GBs times 64MB,
• effective_cache_size, as 0.75*RAM size.

For max_connections we will define a default role variable of 100 but we will allow to specify it at a runtime. The relevant parts of the postgresql.conf.j2 are below: You can now run the role every time you change the instance size, and the config will be changed accordingly to the RAM size. You can extend the role and maybe add other constraints and change max_connections to you specific needs. An example playbook could look like: And run it: The complete role can be found in my github repo.

Hi, I am in two minds of what to write about Doha. My job has been vastly simplified by a friend when he shared with me https://www.youtube.com/watch?v=LdrAd-44LW0 . That video is more relevant and more closer to the truth than whatever I can share. As can be seen it is funny but more sad the way Qatarians are trying to figure out how things will be and as can be seen it seems to heading towards a real estate bubble . They would have to let go of the Sharia if they are thinking of wealthy westerners coming to stay put. I am just sad to know that many of my country-men are stuck there and although I hope the best for them, I dread it may turn out the way it has turned out for many people of Indians, and especially from Kerala in Saudi Arabia. I would touch about the Kerala situation probably in another blog post as this time is exclusively for legal aspects which were discussed in Debconf. A bit of backgrounder here, one part of my family is lawyers which means I have somewhat notion of law as practiced in our land. As probably everybody knows, India was ruled by the British for around 150 odd years. One of the things that they gave while leaving was/is the IPC (Indian Penal Code) and is practiced with the common law concept. The concept means precedence of any judgement goes quite some way in framing rulings and law of the land as time goes on besides the lobbying and the politics which happens in any democracy. Free software would not have been there without the GPL The General Public License. And the license is as much a legal document as it s something that the developers can work without becoming deranged, as it is one of the more simpler licenses to work with. My own understanding of the legal, ethical and moral issues around me were framed by two-three different TV shows, books (fiction and non-fiction alike) apart from what little news I heard in family. One was M*A*S*H* (with Alan Alda and his frailness, anarchism, humanism, civil rights), the Practise and Boston Legal which does lay bare the many grey areas that lawyers have to deal with ( The Practice also influenced a lot of civil rights understanding and First amendment, but as it is a TV show, how much of it is actually practiced for lawyers and how much moral dilemma they are can only be guessed at.) . In books it is artists like John Grisham, Michael Connelly as well as Perry Mason Agatha Christie. In non-fiction look at the treasures under bombayhighcourt e-books corner and series of Hamlyn Lectures. I would have to warn that all of the above are major time-sinks but rewarding in their own way. Also haven t read all of them as time and interests are constrained but do know they are good for understanding bit of our history. I do crave for a meetup kind of scenario when non-lawyers can read and discuss about facets of law . All that understanding was vastly amplified by Groklaw.net which made non-lawyers at the very least be able to decipher and understand what is going on in the free software world. After PJ (Pamela Jones) closed it in 2013 due to total surveillance by the Free World (i.e. the United States of America, NSA) we have been thirsty. We do get occasionally somewhat mildly interesting articles in lwn.net or arstechnica.net but nowhere the sheer brilliance of groklaw. So, it was a sheer stroke of luck that I met Mr. Bradley M. Kuhn who works with Karen Sandler on Software Conservancy. While I wanted to be there for his presentation, it was just one of those days which doesn t go as planned. However, as we met socially and over e-mail there were two basic questions which I asked him which also imbibes why we need to fight for software freedom in the court of law. Below is a re-wording of what he shared . Q1. why do people think that GPL still needs to be challenged in the court of law while there are gpl-violations which has been more or less successfully defended in the court of law ? Bradley Kuhn the GPL violations is basically a violation of one or more clauses of the GPL license and not the GPL license as a whole and my effort during my lifetime would be to make/have such precedents that the GPL is held as a valid license in the court of law. Q2. Let s say IF GPL is held to be valid in the court of law, would FSF benefit monetarily, at least to my mind it might be so, as more people and comapnies could be convinced to use strong copyleft licenses such as GPLv3 or AGPLv3 . Bradley Kuhn It may or may not. It is possible that even after winning, that people and especially companies may go for weak copyleft licenses if it suits them. The only benefit would probably would be to those people who are already using GPLv3 as the law could be used to protect them as well. Although we would want and welcome companies who would use strong copyleft license such as the GPL, the future is in future and hence uncertain. Both possibilities co-exist. While Bradley didn t say it, I would add further here it probably would mean also moving from being a more offensive mode (which GPL-violations is based upon where a violation occurs and somebody either from the victim s side or a by-stander notices the violation, brings it to the notice of the victim and the GPL-volations team.) to perhaps it being defended by the DMCA people themselves, once GPL is held as a valid license in the eyes of law. Although should you use the DMCA or not is a matter of choice, personal belief system as well as your legal recourses. I have to share that the FSF and the GPL-violations team are probably very discerning when they take up the fight as most of the work done by them is pro-bono (i.e. they don t make a single penny/paisa from the work done therein.) and hence in view of scarce resources, it makes sense to go only for the biggest violators in the hopes that you can either make them agree to compensate and agree to the terms of license of any software/hardware combination or sue them and take a bigger share of the reward/compensation awarded by the Court to help the defendant and maybe some of the proceeds donated by the defendant and people like you and me to make sure that Conservancy and the GPL-violations team is still around to help the next time something similar happens. Now, as far as his presentation is concerned, whose video can be seen at http://meetings-archive.debian.net/pub/debian-meetings/2016/debconf16/The_Supreme_Court_of_DFSGFree.webm , I thought it was tame. While he talked about gaming the system in some sense, he was sharing that the system debian-legal works (most-of-the-time). The list actually works because many far more brilliant people than me take time to understand the intricacies of various licenses and how they should be interpreted through the excellently written Debian Free Software Guidelines and whether the license under discussion contravenes the DFSG or is part of it. I do agree with his point though that the ftp-master/s and the team may not be the right person to judge the license in adherence to the DFSG, or her/is not giving a reason for rejecting a package to not entering into the package archive. I actually asked the same question on debian-legal and while I had guessed, it seems there is enough review of the licenses per-se as answer from Paul Wise shows. Charles Pessley also shared an idea he has documented which probably didn t get much traction as involves more work on DD s without any benefit to show for it. All in all I hope it sheds some light on why there is need to be more aware of law in software freedom. Two Organizations which work on software freedom from legal standpoint are SFLC (Delhi) headed by the charming Mr. Eben Moglen and ALF (Bangalore). I do hope more people, especially developers take a bit more interest in some of the resources mentioned above.
Filed under: Miscellenous Tagged: #Alternative Law Forum, #bombayhighcourt e-library, #Common Law, #Debconf16, #Fiction, #Hewlyn lectures, #India, #Jurispudence, #legal fiction, #real estate bubble, #SFLC.in, #Software Freedom, #timesink, Doha, Law

I have been delighted with git for several years now. It's a very powerful tool and I use it every day.
I try to use git in all possible tasks: bind servers, configurations, firewalls, and other personal stuff.

However, there has been always a thing in my git-TODO: a blog managed with git.

After a bit of searching, I found an interesting technology: jekyllrb hosted at github pages. Jekyll looked easy to manage and easy to learn for a newbie like me.
There are some very good looking blogs out there using this combination, for example: https://rsms.me/

But I was lazy to migrate this 'ral-arturo' blog from blogger to jekyll, so I decided to create a new blog from scratch.

This time, the new blog in written in Spanish and is about adventures, nature, travels and outdoor sports.
Perhaps you noticed this article about the Mulhacen mountain (BTW, we did it! :-))
The new blog is called alfabravo.org,

I like the workflow with git & jekyll & github:

• clone the repository
• write a new post in markdown
• read it and correct it locally with 'jekyll serve'
• commit and push to github
• done!

Who knows, perhaps this 'ral-arturo' blog ends being migrated to the new system as well.

Here is my monthly update covering a large part of what I have been doing in the free software world (previously):

• Presented "Reproducible Builds status update" at DebConf16, the annual Debian GNU/Linux developer conference.
• Attended a Core Infrastricture Initiative summit meeting. The CII was set up in the wake of the Heartbleed SSL vulnerability to support software projects that are critical to the functioning of the internet.

• Ensured that the Webconverger web kiosk operating system builds reproducibly. I may rework some of the patches to libisoburn and libisofs before sending them upstream. This work was sponsored by Webconverger.
• Proposed a pull request for Regex Replace (a Chrome extension to automatically replace text on webpages) to ensure that the rules were correctly HTML encoded on the options page. (#3)
• Proposed a change to ronn, a documentation generator that "is the opposite of roff", to make the output reproducible. (#98)
• Fixed an issue in django-enumfield, a custom Django web development field for type-safe named constants, to make the Enum.get interface more consistent. (#36)
• Proposed a change to txt2tags to make the output use SOURCE_DATE_EPOCH and non-timezone timestamps. (#204).

---

The last day of TUG 2016, or at least the last day of talks, brought four one-hour talks from special guests, and several others, where many talks told us personal stories and various histories.
A great finish of a great conference. Jennifer Claudio The case for justified text Due to a strange timezone bug in my calendar program, I completely overslept a morning meeting and breakfast, as well as the first talk, so unfortunately I don t have anything to report about this surely intersting talk comparing justification in various word processors and TeX. Leyla Akhmadeeva, Rinat Gizatullin, Boris Veytsman Are justification and hyphenation good or bad for the reader? Still half dizzy and without coffee, I couldn t really follow this talk, and only woke up till the end when there was a lot of interesting discussion about speed reading and its non-existence (because it is simply skimming over text), and improvements on reading comprehension. Charles Bigelow Looking for legibility Another special guest, Charles Bigelow, presented a huge pool of research and work into readability, and how attitude and usage of fonts change over time. A very involving and well laid out talk, full of interesting background images and personal opinions and thoughts. Charles also touched onto topics of readability on modern devices like e-readers and mobiles. He compared the recent developments in font design for mobile devices with their work on Lucida 20+ years ago, and concluded that both arrived at the same solutions. A very educating and amusing talk packed full with information on readability. I surely will revisit the recording in a study session. David Walden Some notes on the history of digital typography David touches on many topics of the history of digital typography which he has experienced himself over the years: First the development of newspaper production and printing, then the evolvement of editors from simple text editors over word processors to full-fledged DTP programs. Finally he touches on various algorithmic problems that appear in the publishing business. Tim Inkster The beginning of my career Tim, our fanatastic guide through his print shop the Procupine s Quill on the second excursion day, talked about his private ups and downs in the printing business, all filled with an infinite flow of funny stories and surprising anecdotes. Without slides and anything but his voice and his stories, he kept us hanging on his lips without a break. I recommend watching the recording of his talk because one cannot convey the funny comments and great stories he shared with us in this simple blog. Joe Clark Type and tiles on the TTC Joe unveils the history of rise and fall of the underground types and
tiles in Toronto. It is surprising to me that a small metro network as in Toronto can have such a long history of changes of design, layout, presentation. Some of the photos completely stymed me how can anyone put up signs like that? I was thinking. To quote Joe (hopefully I remember correctly):

You see what happens without adult supervision.

Abdelouahad Bayar Towards an operational (La)TeX package supporting optical scaling of dynamic mathematical symbols A technical talk about a trial in providing optical scaling of mathematical symbols. As far as I understand it tries to improve on the TeX way of doing extensible math symbols by glueing things together. It seems to be highly involved and technically interesting project, but I couldn t completely grasp the aim of it. Michael Cohen, Blanca Mancilla, John Plaice Zebrackets: A score of years and delimiters John introduced us to Zebrackets, stripped parentheses and brackets, to help us keep track of pairing of those beasts. But as we know, Zebras are very elusive animals, and so we saw lots of stripped brackets around. The idea of better markup of matching parentheses is definitely worth developing. Charles Bigelow Probably approximately not quite correct: Revise, repeat The second talk of Charles, this time on the history of the Lucida fonts, from the early beginnings drawn on graph paper to recent developments using FontLab producing OpenType fonts. A truly unique crash course through the development of one of the very big families of fonts, and one of the first outside Computer Modern that had also support for proper math typesetting in TeX.

Aggressively legible!

One of the key phrases that popped up again and again was aggressively legible, mostly in negative connotations as far to fat symbols or far to big Arabic letters. But for me this font family is still close to my heart. I purchased it back than from Y&Y for my PhD thesis, and since then have upgraded to the TUG version including the OpenType fonts, and I use them for most of my presentation. Maybe I like the aggressive legibility! Chuck slided in lots of nice comments about Kris Holmes, the development practice in their cooperation, stories of business contacts, and many more, making this talk a very lively and amusing, and at the same time very educating talk.

---

This concluded the TUG conference talks, and we thanked Pavneet for his excellent organization. But since we still have up to two days of excursions, many people dispersed quickly, just to meet again for a optional Type and Tile Tour 3-5 subway stops with discussion of typesetting This guided tour through the underground of Toronto, guided by Joe Clark who spoke in the morning about this topic, was attended by far too many participants. I think there were around 25 when we left. I thought that this will not work out properly, and so decided to leave the group and wander around alone. The last program point for today was dinner with a blues music concert in the near by Jazz Bistro:
Excellent life music in a bit schick/sophisticated atmosphere was a good finish for this excellent day. With Herb from MacTeX and his wife we killed two bottles of red wine, before slowly tingling back to the hotel. A great finishing day of talks.

The first day of the real conference started with an excellent overview of what one can do with TeX, spanning from traditional scientific journal styles to generating router configuration for cruising ships.
All this was crowned with an invited talk my Kevin Larson from Microsoft s typography department on how to support reading comprehension. Pavneet Aurora Opening: Passport to the TeX canvas Pavneet, our never-sleeping host and master of organization, opened the conference with a very philosophical introduction, touching upon a wide range of topics ranging from Microsoft, Twitter to the beauty of books, pages, and type. I think at some point he even mentioned TeX, but I can t remember for sure. His words put up a very nice and all-inclusive stage, a community that is open to all kind of influences with any disregard or prejudice. Let us hope that is reflects reality. Thanks Pavneet. Geoffrey Poore Advances in PythonTeX Our first regular talk was by Geoffrey reporting on recent advances in PythonTeX, a package that allows including python code in your TeX document. Starting with an introduction to PythonTeX, Geoggrey reports about an improved verbatim environment, fvextra, which patches fancyvrb, and improved interaction between tikz and PythonTeX. As I am a heavy user of listings for my teaching on algebraic specification languages, I will surely take a look at this package and see how it compares to listings. Stefan Kottwitz TeX in industry I: Programming Cisco network switches using TeX Next was Stefan from Lufthansa Industry Solutions, who reported first about his working environment, Cruise Ships with a very demanding IT infrastructure he has to design and implement. Then he introduced us to his way of generating IP configurations for all the devices using TeX. The reason he chose this method is that it allows him to generate at the same time proper documentation. It was surprising for me to hear that by using TeX he could far more efficiently and quicker produce well designed and easily accessible documentation, which helped both the company as well as made the clients happy! Stefan Kottwitz TeX in industry II: Designing converged network solutions After a coffee break, Stefan continued his exploration into industrial usage of TeX, this time about using tikz to generate graphics representing the network topology on the ships. Boris Veytsman Making ACM LaTeX styles Next up was Boris which brought us back to traditional realms of TeX when he guided us into the abyss of ACM LaTeX styles he tried to maintain for some time, until he plunged into a complete rewrite of the styles. Frank Mittelbach Alice goes floating global optimized pagination including picture placements The last talk before lunch (probably a strategic placement, otherwise Frank would continue for hours and hours) was Frank on global optimization of page breaks. Frank showed us what can and can not be done with current LaTeX, and how to play around with global optimization of pagination, using Alice in Wonderland as running example. We can only hope that his package is soon available in an easily consumable version to play around. Thai lunch Pavneet has organized three different lunch-styles for the three days of the conference, today s was Thai with spring rools, fried noodles, one kind of very interesting orange noodles, and chicken something. Michael Doob baseball rules summary After lunch Michael gave us an accessible explanation of the most arcane rules a game can have the baseball rules by using pseudo code. I think the total number of loc needed to explain the overall rules would fill more pages than the New York phonebook, so I am deeply impressed by all those who can understand these rules. Some of us even wandered off in the late afternoon to see a match with life explanations of Michael. Amartyo Banerjee, S.K. Venkatesan A Telegram bot for printing LaTeX files Next up was Amartyo who showed a Telegram (as in messenger application) bot running on a Raspberry Pi, that receives (La)TeX files and sends back compiled PDF files. While it is not ready for consumption (If you sneeze the bot will crash!), it looks like a promising application. Furthermore, it is nice to see how open APIs (like Telegram) can spur development of useful tools, while closed APIs (including threatening users, like WhatApp) hinders it. Norbert Preining Security improvements in the TeX Live Manager and installer Next up was my own talk about beefing up the security of TeX Live by providing integrity and authenticity checks via GnuPG, a feature that has been introduced with the recent release of TeX Live 2016. The following discussion gave me several good idea on how to further improve security and usability. Arthur Reutenauer -The TeX Live M sub-project (and open discussion) Arthur presented the TeX Live M (where the M stands supposedly for Mojca, who couldn t attend unfortunately) project: Their aim is to provide a curated and quality verified sub-part of TeX Live that is sufficiently complete for many applications, and easier for distributors and packagers. We had a lively discussion after Arthur s short presentation, mostly about why TeX Live does not have a on-the-fly installation like MikTeX. I insisted that this is already possible, using the tex-on-the-fly package which uses the mktextex infrastructure, but also caution against using it by default due to delays induced by repeatably reading the TeX Live database. I think this is a worth-while project for someone interested in learning the internals of TeX Live, but I am not sure whether I want to invest time into this feature. Another discussion point was about a testing infrastructure, which I am currently working on. This is in fact high on my list, to have some automatic minimal functionality testing a LaTeX package should at least load! Kevin Larson Reading between the lines: Improving comprehension for students Having a guest from Microsoft is rather rare in our quite Unix-centered environment, so big thanks to Pavneet again for setting up this contact, and big thanks to Kevin for coming. Kevin gave us a profound introduction to reading disabilities and how to improve reading comprehension. Starting with an excursion into what makes a font readable and how Microsoft develops optimally readable fonts, he than turned to reading disabilities like dyslexia, and how markup of text can increase students comprehension rate. He also toppled my long-term believe that dyslexia is connected to the similar shape of letters which are somehow visually malprocessed this was the scientific status from the 1920ies till the 70ies, but since then all researchers have abandoned this interpretation and dyslexia is now linked to problems linking shape to phonems. Kevin did an excellent job with a slightly difficult audience some people picking about grammer differences between British and US English and permanently derailing the discussion, and even more the high percentage of typographically somehow specially tasted participants. After the talk I had a lengthy discussion with Kevin about if/how this research can be carried over to non-Roman writing systems, in particular Kanji/Hanzi based writing systems, where dyslexia shows itself probably in different context. Kevin also mentioned that they want to add interword space to Chinese to help learners of Chinese (children, foreigners) to better parse, and studies showed that this helps a lot in comprehension. On a meta-level, this talk bracketed with the morning introduction by Pavneet, describing an open environment with stimulus back and forth in all directions. I am very happy that Kevin took the pain to come in his tight schedule, and I hope that the future will bring better cooperation at the end we are all working somehow on the same front only the the tools differ.

---

After the closing of the session, one part of our group went off to the baseball match, while another group dived into a Japanese-style Izakaya where we managed to kill huge amounts of sake and quite an amount of food. The photo shows me after the first bottle of sake, while just seeping on an intermediate small amount of genshu (kind of strong undiluted sake) before continuing to the next bottle. An interesting and stimulating first day of TUG, and I am sure that everyone was looking forward to day 2.

Review: The Run of His Life, by Jeffrey Toobin

Publisher:	Random House
Copyright:	1996, 1997
Printing:	2015
ISBN:	0-307-82916-2
Format:	Kindle
Pages:	498

The O.J. Simpson trial needs little introduction to anyone who lived through it in the United States, but a brief summary for those who didn't. O.J. Simpson is a Hall of Fame football player and one of the best running backs to ever play the game. He's also black, which is very relevant much of what later happened. After he retired from professional play, he became a television football commentator and a spokesperson for various companies (particularly Hertz, a car rental business). In 1994, he was arrested for the murder of two people: his ex-wife, Nicole Brown Simpson, and Ron Goldman (a friend of Nicole's). The arrest happened after a bizarre low-speed police chase across Los Angeles in a white Bronco that was broadcast live on network television. The media turned the resulting criminal trial into a reality TV show, with live cable television broadcasts of all of the court proceedings. After nearly a full year of trial (with the jury sequestered for nine months more on that later), a mostly black jury returned a verdict of not guilty after a mere four hours of deliberation. Following the criminal trial, in an extremely unusual legal proceeding, Simpson was found civilly liable for Ron Goldman's death in a lawsuit brought by his family. Bizarre events surrounding the case continued long afterwards. A book titled If I Did It (with "if" in very tiny letters on the cover) was published, ghost-written but allegedly with Simpson's input and cooperation, and was widely considered a confession. Another legal judgment let the Goldman family get all the profits from that book's publication. In an unrelated (but also bizarre) incident in Las Vegas, Simpson was later arrested for kidnapping and armed robbery and is currently in prison until at least 2017. It is almost impossible to have lived through the O.J. Simpson trial in the United States and not have formed some opinion on it. I was in college and without a TV set at the time, and even I watched some of the live trial coverage. Reactions to the trial were extremely racially polarized, as you might have guessed. A lot of black people believed at the time that Simpson was innocent (probably fewer now, given subsequent events). A lot of white people thought he was obviously guilty and was let off by a black jury for racial reasons. My personal opinion, prior to reading this book, was a common "third way" among white liberals: Simpson almost certainly committed the murders, but the racist Los Angeles police department decided to frame him for a crime that he did commit by trying to make the evidence stronger. That's a legitimate reason in the US justice system for finding someone innocent: the state has an obligation to follow correct procedure and treat the defendant fairly in order to get a conviction. I have a strong bias towards trusting juries; frequently, it seems that the media second-guesses the outcome of a case that makes perfect sense as soon as you see all the information the jury had (or didn't have). Toobin's book changed my mind. Perhaps because I wasn't watching all of the coverage, I was greatly underestimating the level of incompetence and bad decision-making by everyone involved: the prosecution, the defense, the police, the jury, and the judge. This court case was a disaster from start to finish; no one involved comes away looking good. Simpson was clearly guilty given the evidence presented, but the case was so badly mishandled that it gave the jury room to reach the wrong verdict. (It's telling that, in the far better managed subsequent civil case, the jury had no trouble reaching a guilty verdict.) The Run of His Life is a very detailed examination of the entire Simpson case, from the night of the murder through the end of the trial and (in an epilogue) the civil case. Toobin was himself involved in the media firestorm, breaking some early news of the defense's decision to focus on race in The New Yorker and then involved throughout the trial as a legal analyst, and he makes it clear when he becomes part of the story. But despite that, this book felt objective to me. There are tons of direct quotes, lots of clear description of the evidence, underlying interviews with many of the people involved to source statements in the book, and a comprehensive approach to the facts. I think Toobin is a bit baffled by the black reaction to the case, and that felt like a gap in the comprehensiveness and the one place where he might be accused of falling back on stereotypes and easy judgments. But other than hole, Toobin applies his criticism even-handedly and devastatingly to all parties. I won't go into all the details of how Toobin changed my mind. It was a cumulative effect across the whole book, and if you're curious, I do recommend reading it. A lot was the detailed discussion of the forensic evidence, which was undermined for the jury at trial but looks very solid outside the hothouse of the case. But there is one critical piece that I would hope would be handled differently today, twenty years later, than it was by the prosecutors in that case: Simpson's history of domestic violence against Nicole. With what we now know about patterns of domestic abuse, the escalation to murder looks entirely unsurprising. And that history of domestic abuse was exceedingly well-documented: multiple external witnesses, police reports, and one actual prior conviction for spousal abuse (for which Simpson did "community service" that was basically a joke). The prosecution did a very poor job of establishing this history and the jury discounted it. That was a huge mistake by both parties. I'll mention one other critical collection of facts that Toobin explains well and that contradicted my previous impression of the case: the relationship between Simpson and the police. Today, in the era of Black Lives Matter, the routine abuse of black Americans by the police is more widely known. At the time of the murders, it was less recognized among white Americans, although black Americans certainly knew about it. But even in 1994, the Los Angeles police department was notorious as one of the most corrupt and racist big-city police departments in the United States. This is the police department that beat Rodney King. Mark Fuhrman, one of the police officers involved in the case (although not that significantly, despite his role at the trial), was clearly racist and had no business being a police officer. It was therefore entirely believable that these people would have decided to frame a black man for a murder he actually committed. What Toobin argues, quite persuasively and with quite a lot of evidence, is that this analysis may make sense given the racial tensions in Los Angeles but ignores another critical characteristic of Los Angeles politics, namely a deference to celebrity. Prior to this trial, O.J. Simpson largely followed the path of many black athletes who become broadly popular in white America: underplaying race and focusing on his personal celebrity and connections. (Toobin records a quote from Simpson earlier in his life that perfectly captures this approach: "I'm not black, I'm O.J.") Simpson spent more time with white businessmen than the black inhabitants of central Los Angeles. And, more to the point, the police treated him as a celebrity, not as a black man. Toobin takes some time to chronicle the remarkable history of deference and familiarity that the police showed Simpson. He regularly invited police officers to his house for parties. The police had a long history of largely ignoring or downplaying his abuse of his wife, including not arresting him in situations that clearly seemed to call for that, showing a remarkable amount of deference to his side of the story, not pursuing clear violations of the court judgment after his one conviction for spousal abuse, and never showing much inclination to believe or protect Nicole. Even on the night of the murder, they started following a standard playbook for giving a celebrity advance warning of investigations that might involve them before the news media found out about them. It seems clear, given the evidence that Toobin collected, that the racist Los Angeles police didn't focus that animus at Simpson, a wealthy celebrity living in Brentwood. He wasn't a black man in their eyes; he was a rich Hall of Fame football player and a friend. This obviously raises the question of how the jury could return an innocent verdict. Toobin provides plenty of material to analyze that question from multiple angles in his detailed account of the case, but I can tell you my conclusion: Judge Lance Ito did a horrifically incompetent job of managing the case. He let the lawyers wander all over the case, interacted bizarrely with the media coverage (and was part of letting the media turn it into a daytime drama), was not crisp or clear about his standards of evidence and admissibility, and, perhaps worst of all, let the case meander on at incredible length. With a fully sequestered jury allowed only brief conjugal visits and no media contact (not even bookstore shopping!). Quite a lot of anger was focused on the jury after the acquittal, and I do think they reached the wrong conclusion and had all the information they would have needed to reach the correct one. But Toobin touches on something that I think would be very hard to comprehend without having lived through it. The jury and alternate pool essentially lived in prison for nine months, with guards and very strict rules about contact with the outside world, in a country where compensation for jury duty is almost nonexistent. There were a lot of other factors behind their decision, including racial tensions and the sheer pressure from judging a celebrity case about which everyone has an opinion, but I think it's nearly impossible to underestimate the psychological tension and stress from being locked up with random other people under armed guard for three quarters of a year. It's hard for jury members to do an exhaustive and careful deliberation in a typical trial that takes a week and doesn't involve sequestration. People want to get back to their lives and families. I can only imagine the state I would be in after nine months of this, or how poor psychological shape I would be in to make a careful and considered decision. Similarly, for those who condemned the jury for profiting via books and media appearances after the trial, the current compensation for jurors is $15 per day (not hour). I believe at the time it was around $5 per day. There are a few employers who will pay full salary for the entire jury service, but only a few; many cap the length at a few weeks, and some employers treat all jury duty as unpaid leave. The only legal requirement for employers in the United States is that employees that serve on a jury have their job held for them to return to, but compensation is pathetic, not tied to minimum wage, and employers do not have to supplement it. I'm much less inclined to blame the jurors than the system that badly mistreated them. As you can probably tell from the length of this review, I found The Run of His Life fascinating. If I had followed the whole saga more closely at the time, this may have been old material, but I think my vague impressions and patchwork of assumptions were more typical than not among people who were around for the trial but didn't invest a lot of effort into following it. If you are like me, and you have any interest in the case or the details of how the US criminal justice system works, this is a fascinating case study, and Toobin does a great job with it. Recommended. Rating: 8 out of 10

Version 0.0.5 of RcppCCTZ arrived on CRAN a couple of days ago. It reflects an upstream fixed made a few weeks ago. CRAN tests revealed that g++-6 was tripping over one missing #define; this was added upstream and I subsequently synchronized with upstream. At the same time the set of examples was extended (see below). Somehow useR! 2016 got in the way and while working on the then-incomplete examples during the traveling I forgot to release this until CRAN reminded me that their tests still failed. I promptly prepared the 0.0.5 release but somehow failed to update NEWS files etc. They are correct in the repo but not in the shipped package. Oh well. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. In fact, it is two libraries. One for dealing with civil time: human-readable dates and times, and one for converting between between absolute and civil times via time zones. It requires only a proper C++11 compiler and the standard IANA time zone data base which standard Unix, Linux, OS X, ... computers tend to have in /usr/share/zoneinfo. RcppCCTZ connects this library to R by relying on Rcpp. Two good examples are now included, and shown here. The first one tabulates the time difference between New York and London (at a weekly level for compactness): Because the two continents happen to spring forward and fall backwards between regular and daylight savings times, there are, respectively, two and one week periods where the difference is one hour less than usual. A second example shifts the time to a different time zone: Note that because we return a POSIXct object, it is printed by R with the default (local) TZ attribute (for "America/Chicago" in my case). A more direct example asks what time it is in my time zone when it is midnight in Tokyo: More changes will come in 0.0.6 as soon as I find time to translate the nice time_tool (command-line) example into an R function. Changes in this version are summarized here:

Changes in version 0.0.5 (2016-07-09)

• New utility example functions toTz() and tzDiff
• Synchronized with small upstream change for additional #ifdef for compiler differentiation

We also have a diff to the previous version thanks to CRANberries. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

DebConf 16 in Capetown/South Africa was fantastic for many reasons. My Capetown/South Africa/Culture/Flight related lessons:

• Avoid flying on Sundays (especially in/from Austria where plenty of hotlines are closed on Sundays or at least not open when you need them)
• Actually turn back your seat on the flight when trying to sleep and not forget that this option exists *cough*
• While UCT claims to take energy saving quite serious (e.g. turn off the lights mentioned at many places around the campus), several toilets flush all their water, even when trying to do just small business and also two big lights in front of a main building seem to be shining all day long for no apparent reason
• There doesn t seem to be a standard for the side of hot vs. cold water-taps
• Soap pieces and towels on several toilets
• For pedestrians there s just a very short time of green at the traffic lights (~2-3 seconds), then red blinking lights show that you can continue walking across the street (but *should* not start walking) until it s fully red again (but not many people seem to care about the rules anyway :))
• Warning lights of cars are used for saying thanks (compared to hand waving in e.g. Austria)
• The 40km/h speed limit signs on the roads seem to be showing the recommended minimum speed :-)
• There are many speed bumps on the roads
• Geese quacking past 11:00 p.m. close to a sleeping room are something I m also not used to :-)
• Announced downtimes for the Internet connection are something I m not used to
• WLAN in the dorms of UCT as well as in any other place I went to at UCT worked excellent (measured ~22-26 Mbs downstream in my room, around 26Mbs in the hacklab) (kudos!)
• WLAN is available even on top of the Table Mountain (WLAN working and being free without any registration)
• Number26 credit card is great to withdraw money from ATMs without any extra fees from common credit card companies (except for the fee the ATM itself charges but displays ahead on-site anyway)
• Splitwise is a nice way to share expenses on the road, especially with its mobile app and the money beaming using the Number26 mobile app

My technical lessons from DebConf16:

• ran into way too many yak-shaving situations, some of them might warrant separate blog posts
• finally got my hands on gbp-pq (manage quilt patches on patch queue branches in git): very nice to be able to work with plain git and then get patches for your changes, also having upstream patches (like cherry-picks) inside debian/patches/ and the debian specific changes inside debian/patches/debian/ is a lovely idea, this can be easily achieved via Gbp-Pq: Topic debian with gbp s pq and is used e.g. in pkg-systemd, thanks to Michael Biebl for the hint and helping hand
• David Bremner s gitpkg/git-debcherry is something to also be aware of (thanks for the reminder, gregoa)
• autorevision: extracts revision metadata from your VCS repository (thanks to pabs)
• blhc: build log hardening check
• Guido s gbp skills exchange session reminded me once again that I should use gbp import-dsc download $URL_TO_DSC more often
• sources.debian.net features specific copyright + patches sections (thanks, Matthieu Caneill)
• dpkg-mergechangelogs(1) for 3-way merge of debian/changelog files (thanks, buxy)
• meta-git from pkg-perl is always worth a closer look
• ifupdown2 (its current version is also available in jessie-backports!) has some nice features, like ifquery running $interface to get the life configuration of a network interface, json support ( ifquery format=json ) and makotemplates support to generate configuration for plenty of interfaces

BTW, thanks to the video team the recordings from the sessions are available online.

I write free software and I have some users. My primary support channels are over email and IRC, which means I do not have direct access to the system where my software runs. When one of my users has a problem, we go through one or more cycles of them reporting what they see and me asking them for more information, or asking them to try this thing or that thing and report results. This can be quite frustrating. I want, nay, need to improve this. I've been thinking about this for a while, and talking with friends about it, and here's my current ideas. First idea: have a script that gathers as much information as possible, which the user can run. For example, log files, full configuration, full environment, etc. The user would then mail the output to me. The information will need to be anonymised suitably so that no actual secrets are leaked. This would be similar to Debian's package specific reportbug scripts. Second idea: make it less likely that the user needs help solving their issue, with better error messages. This would require error messages to have sufficient explanation that a user can solve their problem. That doesn't necessarily mean a lot of text, but also code that analyses the situation when the error happens to include things that are relevant for the problem resolving process, and giving error messages that are as specific as possible. Example: don't just fail saying "write error", but make the code find out why writing caused an error. Third idea: in addition to better error messages, might provide diagnostics tools as well. A friend suggested having a script that sets up a known good set of operations and verifies they work. This would establish a known-working baseline, or smoke test, so that we can rule things like "software isn't completely installed". Do you have ideas? Mail me (liw@liw.fi) or tell me on identi.ca (@liw) or Twitter (@larswirzenius).

For some time now, I ve been running OpenWRT on an RT-N66U device. I initially set that because I had previously been using my Debian-based file/VM server as a firewall, and this had some downsides: every time I wanted to reboot that, Internet for the whole house was down; shorewall took a fair bit of care and feeding; etc. I ve been having indications that all is not well with OpenWRT or the N66U in the last few days, and some long-term annoyances prompted me to search out a different solution. I figured I could buy an embedded x86 device, slap Debian on it, and be set. The device I wound up purchasing happened to have pfsense preinstalled, so I thought I d give it a try. As expected, with hardware like that to work with, it was a lot more capable than OpenWRT and had more features. However, I encountered a number of surprising issues. The biggest annoyance was that the system wouldn t allow me to set up a static DHCP entry with the same IP for multiple MAC addresses. This is a very simple configuration in the underlying DHCP server, and OpenWRT permitted it without issue. It is quite useful so my laptop has the same IP whether connected by wifi or Ethernet, and I have used it for years with no issue. Googling it a bit turned up some rather arrogant pfsense people saying that this is broken and poor design, and that your wired and wireless networks should be on different VLANs anyhow. They also said just give it the same hostname for the different IPs but it rejects this too. Sigh. I discovered, however, that downloading the pfsense backup XML file, editing the IP within, and re-uploading it gets me what I want with no ill effects! So then I went to set up DNS. I tried to enable the DNS Forwarder , but it wouldn t let me do that while the DNS Resolver was still active. Digging in just a bit, it appears that the DNS Forwarder and DNS Resolver both provide forwarding and resolution features; they just have different underlying implementations. This is not clear at all in the interface. Next stop: traffic shaping. Since I use VOIP for work, this is vitally important for me. I dove in, and found a list of XML filenames for wizards: one for Dedicated Links and another for Multiple Lan/Wan . Hmmm. Some Googling again turned up that everyone suggests using the Multiple Lan/Wan wizard. Fine. I set it up, and notice that when I start an upload, my download performance absolutely tanks. Some investigation shows that outbound ACKs aren t being handled properly. The wizard had created a qACK queue, but neglected to create a packet match rule for it, so ACKs were not being dealt with appropriately. Fixed that with a rule of my own design, and now downloads are working better again. I also needed to boost the bandwidth allocated to qACK (setting it to 25% seemed to do the trick). Then there was the firewall rules. The interface section is first-match-wins, whereas the floating section is last-match-wins. This is rather non-obvious. Getting past all the interface glitches, however, the system looks powerful, solid, and well-engineered under the hood, and fairly easy to manage.

So, I've been sick. Quite sick, as for the past ~2 weeks I wasn't able to bike, run, work or do much beside watch movies, look at photos and play some light games (ARPGs rule in this case, all you need to do is keep the left mouse button pressed). It was supposed to be only a light viral infection, but it took longer to clear out than I expected, probably due to it happening right after my dental procedure (and possibly me wanting to restart exercise too soon, to fast). Not fun, it felt like the thing that refills your energy/mana bar in games broke. I simply didn't feel restored, despite sleeping a lot; 2-3 naps per day sound good as long as they are restorative, if they're not, sleeping is just a chore. The funny thing is that recovery happened so slow, that when I finally had energy it took me by surprise. It was like oh, wait, I can actually stand and walk without feeling dizzy! Wohoo! As such, yesterday was a glorious Saturday I was therefore able to walk a bit outside the house this weekend and feel like having a normal cold, not like being under a cursed: -4 vitality spell. I expect the final symptoms to clear out soon, and that I can very slowly start doing some light exercise again. Not tomorrow, though In the meantime, I'm sharing a picture from earlier this year that I found while looking through my stash. Was walking in the forest in Pontresina on a beatiful sunny day, when a sudden gust of wind caused a lot of the snow on the trees to fly around and make it look a bit magical (photo is unprocessed beside conversion from raw to jpeg, this is how it was straight out of the camera): Why a winter photo? Because that's exactly how cold I felt the previous weekend: 30 C outside, but I was going to the doctor in jeans and hoodie and cap, shivering

This series has focused on new projects. I could have said more about significant work that didn't involve starting new projects. A big example was when I added dh to debhelper, which led to changes in a large percentage of debian/rules files. And I've contributed to many other free software projects. I guess I've focused on new projects becuase it's easier to remember things I've started myself. And because a new project is more wide open, with more scope for interesting ideas, especially when it's free software being created just because I want to, with no expectations of success. But starting lots of your own projects also makes you responsible for maintaining a lot of stuff. Ten years ago I had dozens of projects that I'd started and was still maintaining. Over time I started pulling away from Debian, with active projects increasingly not connected with it. By the end, I'd left and stopped working on the old projects. Nearly everything from my first decade in free software was passed on to new maintainers. It's good to get out from under old projects and concentrate on new things. I saved propellor for last in this series, because I think it may point toward the future of my software. While git-annex was the project that taught me Haskell, propellor's design is much more deeply influenced by the Haskell viewpoint. Absorbing that viewpoint has itself been a big undertaking for me this decade. It's like I was coasting along feeling at the top of my game and wham got hit by the type theory bus. And now I see that I was stuck in a rut before, and begin to get a feeling of many new possibilities. That's a good feeling to have, twenty years in.

A while back, I found myself in need of two webservers that would terminate TLS (with different rules). I wanted to run some custom code I d written (which uses TLS peer authentication), and also nginx on port 443. The best way I figured out how to do this was to write a tool to sit on port 443, and parse TLS Client Hello packets, and dispatch to the correct backend depending on the SNI name. SNI, or Server Name Indication allows the client to announce (yes over cleartext!) what server it s looking for, similar to the HTTP Host header. Sometimes, like in the case above, the Host header won t work, since you ve already done a TLS handshake by the time you figure out who they re looking for. I also spun the Client Hello parser out into its own importable package, just in case someone else finds themselves in this same boat. The code s up on github.com/paultag/sniff!

My experiences with Amazon reviewing have been somewhat unusual. A review of a smart switch I wrote received enough attention that the vendor pulled the product from Amazon. At the time of writing, I'm ranked as around the 2750th best reviewer on Amazon despite having a total of 18 reviews. But the world of Amazon reviews is even stranger than that, and the past couple of weeks have given me some insight into it.

Amazon's success is fairly phenomenal. It's estimated that there's over 50 million people in the US paying $100 a year to get free shipping on Amazon purchases, and combined with Amazon's surprisingly customer friendly service there's a lot of people with a very strong preference for choosing Amazon rather than any other retailer. If you're not on Amazon, you're hurting your sales.

And if you're an established brand, this works pretty well. Some people will search for your product directly and buy it, leaving reviews. Well reviewed products appear higher up in search results, so people searching for an item type rather than a brand will still see your product appear early in the search results, in turn driving sales. Some proportion of those customers will leave reviews, which helps keep your product high up in the results. As long as your products aren't utterly dreadful, you'll probably maintain that position.

But if you're a brand nobody's ever heard of, things are more difficult. People are unlikely to search for your product directly, so you're relying on turning up in the results for more generic terms. But if you're selling a more generic kind of item (say, a Bluetooth smart bulb) then there's probably a number of other brands nobody's ever heard of selling almost identical objects. If there's no reason for anybody to choose your product then you're probably not going to get any reviews and you're not going to move up the search rankings. Even if your product is better than the competition, a small number of sales means a tiny number of reviews. By the time that number's large enough to matter, you're probably onto a new product cycle.

In summary: if nobody's ever heard of you, you need reviews but you're probably not getting any.

The old way of doing this was to send review samples to journalists, but nobody's going to run a comprehensive review of 3000 different USB cables and even if they did almost nobody would read it before making a decision on Amazon. You need Amazon reviews, but you're not getting any. The obvious solution is to send review samples to people who will leave Amazon reviews. This is where things start getting more dubious.

Amazon run a program called Vine which is intended to solve this problem. Send samples to Amazon and they'll distribute them to a subset of trusted reviewers. These reviewers write a review as normal, and Amazon tag the review with a "Vine Voice" badge which indicates to readers that the reviewer received the product for free. But participation in Vine is apparently expensive, and so there's a proliferation of sites like Snagshout or AMZ Review Trader that use a different model. There's no requirement that you be an existing trusted reviewer and the product probably isn't free. You sign up, choose a product, receive a discount code and buy it from Amazon. You then have a couple of weeks to leave a review, and if you fail to do so you'll lose access to the service. This is completely acceptable under Amazon's rules, which state "If you receive a free or discounted product in exchange for your review, you must clearly and conspicuously disclose that fact". So far, so reasonable.

In reality it's worse than that, with several opportunities to game the system. AMZ Review Trader makes it clear to sellers that they can choose reviewers based on past reviews, giving customers an incentive to leave good reviews in order to keep receiving discounted products. Some customers take full advantage of this, leaving a giant number of 5 star reviews for products they clearly haven't tested and then (presumably) reselling them. What's surprising is that this kind of cynicism works both ways. Some sellers provide two listings for the same product, the second being significantly more expensive than the first. They then offer an attractive discount for the more expensive listing in return for a review, taking it down to approximately the same price as the original item. Once the reviews are in, they can remove the first listing and drop the price of the second to the original price point.

The end result is a bunch of reviews that are nominally honest but are tied to perverse incentives. In effect, the overall star rating tells you almost nothing - you still need to actually read the reviews to gain any insight into whether the customer actually used the product. And when you do write an honest review that the seller doesn't like, they may engage in heavy handed tactics in an attempt to make the review go away.

It's hard to avoid the conclusion that Amazon's review model is broken, but it's not obvious how to fix it. When search ranking is tied to reviews, companies have a strong incentive to do whatever it takes to obtain positive reviews. What we're left with for now is having to laboriously click through a number of products to see whether their rankings come from thoughtful and detailed reviews or are just a mass of 5 star one liners.

comments

Background: I ve been relearning Rust (more about that in a separate post, some time later), and in doing so, I chose to implement the low-level parts of git (I ll touch the why in that separate post I just promised). Disclaimer: It s friday. This is not entirely(?) a serious post. So, I was looking at Documentation/technical/index-format.txt, and saw:

32-bit number of index entries.

What? The index/staging area can t handle more than ~4.3 billion files? There I was, writing Rust code to write out the index.

try!(out.write_u32::<NetworkOrder>(self.entries.len()));

(For people familiar with the byteorder crate and wondering what NetworkOrder is, I have a use byteorder::BigEndian as NetworkOrder) And the Rust compiler rightfully barfed:

error: mismatched types:
 expected  u32 ,
    found  usize  [E0308]

And there I was, wondering: mmmm should I just add as u32 and silently truncate or hey what does git do? And it turns out, git uses an unsigned int to track the number of entries in the first place, so there is no truncation happening. Then I thought but what happens when cache_nr reaches the max? Well, it turns out there s only one obvious place where the field is incremented. What? Holy coffin nails, Batman! No overflow check? Wait a second, look 3 lines above that:

ALLOC_GROW(istate->cache, istate->cache_nr + 1, istate->cache_alloc);

Yeah, obviously, if you re incrementing cache_nr, you already have that many entries in memory. So, how big would that array be?

        struct cache_entry **cache;

So it s an array of pointers, assuming 64-bits pointers, that s ~34.3 GB. But, all those cache_nr entries are in memory too. How big is a cache entry?

struct cache_entry  
        struct hashmap_entry ent;
        struct stat_data ce_stat_data;
        unsigned int ce_mode;
        unsigned int ce_flags;
        unsigned int ce_namelen;
        unsigned int index;     /* for link extension */
        unsigned char sha1[20];
        char name[FLEX_ARRAY]; /* more */
 ;

So, 4 ints, 20 bytes, and as many bytes as necessary to hold a path. And two inline structs. How big are they?

struct hashmap_entry  
        struct hashmap_entry *next;
        unsigned int hash;
 ;
struct stat_data  
        struct cache_time sd_ctime;
        struct cache_time sd_mtime;
        unsigned int sd_dev;
        unsigned int sd_ino;
        unsigned int sd_uid;
        unsigned int sd_gid;
        unsigned int sd_size;
 ;

Woohoo, nested structs.

struct cache_time  
        uint32_t sec;
        uint32_t nsec;
 ;

So all in all, we re looking at 1 + 2 + 2 + 5 + 4 32-bit integers, 1 64-bits pointer, 2 32-bits padding, 20 bytes of sha1, for a total of 92 bytes, not counting the variable size for file paths. The average path length in mozilla-central, which only has slightly over 140 thousands of them, is 59 (including the terminal NUL character). Let s conservatively assume our crazy repository would have the same average, making the average cache entry 151 bytes. But memory allocators usually allocate more than requested. In this particular case, with the default allocator on GNU/Linux, it s 156 (weirdly enough, it s 152 on my machine). 156 times 4.3 billion 670 GB. Plus the 34.3 from the array of pointers: 704.3 GB. Of RAM. Not counting the memory allocator overhead of handling that. Or all the other things git might have in memory as well (which apparently involves a hashmap, too, but I won t look at that, I promise). I think one would have run out of memory before hitting that integer overflow. Interestingly, looking at Documentation/technical/index-format.txt again, the on-disk format appears smaller, with 62 bytes per file instead of 92, so the corresponding index file would be smaller. (And in version 4, paths are prefix-compressed, so paths would be smaller too). But having an index that large supposes those files are checked out. So let s say I have an empty ext4 file system as large as possible (which I m told is 2^60 bytes (1.15 billion gigabytes)). Creating a small empty ext4 tells me at least 10 inodes are allocated by default. I seem to remember there s at least one reserved for the journal, there s the top-level directory, and there s lost+found ; there apparently are more. Obviously, on that very large file system, We d have a git repository. git init with an empty template creates 9 files and directories, so that s 19 more inodes taken. But git init doesn t create an index, and doesn t have any objects. We d thus have at least one file for our hundreds of gigabyte index, and at least 2 who-knows-how-big files for the objects (a pack and its index). How many inodes does that leave us with? The Linux kernel source tells us the number of inodes in an ext4 file system is stored in a 32-bits integer. So all in all, if we had an empty very large file system, we d only be able to store, at best, 2^32 22 files And we wouldn t even be able to get cache_nr to overflow. while following the rules. Because the index can keep files that have been removed, it is actually possible to fill the index without filling the file system. After hours (days? months? years? decades?*) of running

seq 0 4294967296   while read i; do touch $i; git update-index --add $i; rm $i; done

One should be able to reach the integer overflow. But that d still require hundreds of gigabytes of disk space and even more RAM.

• At the rate it was possible to add files to the index when I tried (yeah, I tried), for a few minutes, and assuming a constant rate, the estimate is close to 2 years. But the time spent reading and writing the index file increases linearly with its size, so the longer it d run, the longer it d take.

Ok, it s actually much faster to do it hundreds of thousand files at a time, with something like:

seq 0 100000 4294967296   while read i; do j=$(seq $i $(($i + 99999))); touch $j; git update-index --add $j; rm $j; done

At the rate the first million files were added, still assuming a constant rate, it would take about a month on my machine. Considering reading/writing a list of a million files is a thousand times faster than reading a list of a billion files, assuming linear increase, we re still talking about decades, and plentiful RAM. Fun fact: after leaving it run for 5 times as much as it had run for the first million files, it hasn t even done half more One could generate the necessary hundreds-of-gigabytes index manually, that wouldn t be too hard, and assuming it could be done at about 1 GB/s on a good machine with a good SSD, we d be able to craft a close-to-explosion index within a few minutes. But we d still lack the RAM to load it. So, here is the open question: should I report that integer overflow? Wow, that was some serious procrastination. Edit: Epilogue: Actually, oops, there is a separate integer overflow on the reading side that can trigger a buffer overflow, that doesn t actually require a large index, just a crafted header, demonstrating that yes, not all integer overflows are equal.

With more than 10 million users, the Scratch online community is the largest online community where kids learn to program. Since it was created, a central goal of the community has been to promote remixing the reworking and recombination of existing creative artifacts. As the video above shows, remixing programming projects in the current web-based version of Scratch is as easy is as clicking on the see inside button in a project web-page, and then clicking on the remix button in the web-based code editor. Today, close to 30% of projects on Scratch are remixes. Remixing plays such a central role in Scratch because its designers believed that remixing can play an important role in learning. After all, Scratch was designed first and foremost as a learning community with its roots in the Constructionist framework developed at MIT by Seymour Papert and his colleagues. The design of the Scratch online community was inspired by Papert s vision of a learning community similar to Brazilian Samba schools (Henry Jenkins writes about his experience of Samba schools in the context of Papert s vision here), and a comment Marvin Minsky made in 1984:

Adults worry a lot these days. Especially, they worry about how to make other people learn more about computers. They want to make us all computer-literate. Literacy means both reading and writing, but most books and courses about computers only tell you about writing programs. Worse, they only tell about commands and instructions and programming-language grammar rules. They hardly ever give examples. But real languages are more than words and grammar rules. There s also literature what people use the language for. No one ever learns a language from being told its grammar rules. We always start with stories about things that interest us.

In a new paper titled Remixing as a pathway to Computational Thinking that was recently published at the ACM Conference on Computer Supported Collaborative Work and Social Computing (CSCW) conference, we used a series of quantitative measures of online behavior to try to uncover evidence that might support the theory that remixing in Scratch is positively associated with learning. Of course, because Scratch is an informal environment with no set path for users, no lesson plan, and no quizzes, measuring learning is an open problem. In our study, we built on two different approaches to measure learning in Scratch. The first approach considers the number of distinct types of programming blocks available in Scratch that a user has used over her lifetime in Scratch (there are 120 in total) something that can be thought of as a block repertoire or vocabulary. This measure has been used to model informal learning in Scratch in an earlier study. Using this approach, we hypothesized that users who remix more will have a faster rate of growth for their code vocabulary. Controlling for a number of factors (e.g. age of user, the general level of activity) we found evidence of a small, but positive relationship between the number of remixes a user has shared and her block vocabulary as measured by the unique blocks she used in her non-remix projects. Intriguingly, we also found a strong association between the number of downloads by a user and her vocabulary growth. One interpretation is that this learning might also be associated with less active forms of appropriation, like the process of reading source code described by Minksy. The second approach we used considered specific concepts in programming, such as loops, or event-handling. To measure this, we utilized a mapping of Scratch blocks to key programming concepts found in this paper by Karen Brennan and Mitchel Resnick. For example, in the image below are all the Scratch blocks mapped to the concept of loop . We looked at six concepts in total (conditionals, data, events, loops, operators, and parallelism). In each case, we hypothesized that if someone has had never used a given concept before, they would be more likely to use that concept after encountering it while remixing an existing project. Using this second approach, we found that users who had never used a concept were more likely to do so if they had been exposed to the concept through remixing. Although some concepts were more widely used than others, we found a positive relationship between concept use and exposure through remixing for each of the six concepts. We found that this relationship was true even if we ignored obvious examples of cutting and pasting of blocks of code. In all of these models, we found what we believe is evidence of learning through remixing. Of course, there are many limitations in this work. What we found are all positive correlations we do not know if these relationships are causal. Moreover, our measures do not really tell us whether someone has understood the usage of a given block or programming concept.However, even with these limitations, we are excited by the results of our work, and we plan to build on what we have. Our next steps include developing and utilizing better measures of learning, as well as looking at other methods of appropriation like viewing the source code of a project.

This blog post and the paper it describes are collaborative work with Sayamindu Dasgupta, Andr s Monroy-Hern ndez, and William Hale. The paper is released as open access so anyone can read the entire paper here. This blog post was also posted on Sayamindu Dasgupta s blog and on Medium by the MIT Media Lab.

I tend to save up my vacation time to use in summer for family activities, and today was one of those days. Yesterday, Jacob and Oliver enjoyed planning what they were going to do with me. They ruled out all sorts of things nearby, but they decided they would like to fly to Ponca City, explore the oil museum there, then eat at Enrique s before flying home. Of course, it is not particularly hard to convince me to fly somewhere. So off we went today for some great father-son time. The weather on the way was just gorgeous. We cruised along at about a mile above ground, which gave us pleasantly cool air through the vents and a smooth ride. Out in the distance, a few clouds were trying to form. Whether I m flying or driving, a pilot is always happy to pass a small airport. Here was the Winfield, KS airport (KWLD): This is a beautiful time of year in Kansas. The freshly-cut wheat fields are still a vibrant yellow. Other crops make a bright green, and colors just pop from the sky. A camera can t do it justice. They enjoyed the museum, and then Oliver wanted to find something else to do before we returned to the airport for dinner. A little exploring yielded the beautiful and shady Garfield Park, complete with numerous old stone bridges. Of course, the hit of any visit to Enrique s is their ice cream tacos (sopapillas with ice cream). Here is Oliver polishing off his. They had both requested sightseeing from the sky on our way back, but both fell asleep so we opted to pass on that this time. Oliver slept through the landing, and I had to wake him up when it was time to go. I always take it as a compliment when a 6-year-old sleeps through a landing! Most small airports have a bowl of candy setting out somewhere. Jacob and Oliver have become adept at finding them, and I will usually let them talk me into a piece of candy at one of them. Today, after we got back, they were intent at exploring the small gift shop back home, and each bought a little toy helicopter for $1.25. They may have been too tired to enjoy it though. They ve been in bed for awhile now, and I m still smiling about the day. Time goes fast when you re having fun, and all three of us were. It is fun to see them inheriting my sense of excitement at adventure, and enjoying the world around them as they go. The lady at the museum asked how we had heard about them, and noticed I drove up in an airport car (most small airports have an old car you can borrow for a couple hours for free if you re a pilot). I told the story briefly, and she said, So you flew out to this small town just to spend some time here? Yep. Wow, that s really neat. I don t think we ve ever had a visitor like you before. Then she turned to the boys and said, You boys are some of the luckiest kids in the world. And I can t help but feel like the luckiest dad in the world.

A while back, I saw a Kickstarter for one of the most well designed and pretty sleep trackers on the market. I fell in love with it, and it has stuck with me since. A few months ago, I finally got my hands on one and started to track my data. Naturally, I now want to store this new data with the rest of the data I have on myself in my own databases. I went in search of an API, but I found that the Sense API hasn't been published yet, and is being worked on by the team. Here's hoping it'll land soon! After some subdomain guessing, I hit on api.hello.is. So, naturally, I went to take a quick look at their Android app and network traffic, lo and behold, there was a pretty nicely designed API. This API is clearly an internal API, and as such, it's something that should not be considered stable. However, I'm OK with a fragile API, so I've published a quick and dirty API wrapper for the Sense API to my GitHub.. I've published it because I've found it useful, but I can't promise the world, (since I'm not a member of the Sense team at Hello!), so here are a few ground rules of this wrapper:

• I make no claims to the stability or completeness.
• I have no documentation or assurances.
• I will not provide the client secret and ID. You'll have to find them on your own.
• This may stop working without any notice, and there may even be really nasty bugs that result in your alarm going off at 4 AM.
• Send PRs! This is a side-project for me.

This module is currently Python 3 only. If someone really needs Python 2 support, I'm open to minimally invasive patches to the codebase using six to support Python 2.7. Working with the API: First, let's go ahead and log in using python -m sense. Now, let's see if we can pull up information on my Sense: Neat! Pretty cool. Look, you can even see my WiFi AP! Let's try some more and pull some trends out. I plan to keep maintaining it as long as it's needed, so I welcome co-maintainers, and I'd love to see what people build with it! So far, I'm using it to dump my room data into InfluxDB, pulling information on my room into Grafana. Hopefully more to come! Happy hacking!

Debian packages can run arbitrary code via maintainer scripts (sometimes shortened into maintscripts ) during installation/removal etc. While they certainly have their use cases, their failure modes causes exciting bugs like fails to install or the dreaded fails to remove . They also have other undesirable effects such as:

• Bugs in/Updates to auto-generated snippets require a rebuild of all packages (not to mention the obvious code-duplication in all packages).
• In case of circular dependencies[1] all having postinst scripts, dpkg will have to guess which package to configure first.
• They require forking a shell at least once for each maintscript.
• They complicate the implementations of e.g. detached chroot creation.

Accordingly, I think we should aim for a more declarative packaging style. To help facilitate this, I have implemented 3 tracking tags in Lintian.

• no-ctrl-scripts in 2.5.44
• ctrl-script in 2.5.44
• debhelper-autoscript-in-maintainer-scripts in 2.5.45

With these, we were able to learn that 73.5% of all packages do not have any of these scripts. But I can now also produce a list of helpers that insert the most maintainer script snippets. The current top 15 is:

1. dhpython with 3775 instances
   • This is an umbrella for all helpers using dh-python s python module, see #827774.
2. dh_installmenu with 1861 instances
   • This will presumably drop as menu files should be replaced with desktop files.
3. dh_makeshlibs with 1396 remaining instances
   • This should be in rapid decline since we decided to replace its scripts with a dpkg trigger. Unfortunately, it can take quite a while until all of them have been rebuilt with a new version of debhelper
4. dh_installinit with 1224 instances
5. dh_python2 with 1168 instances
6. dh_installdebconf with 772 instances
7. dh_installdeb with 754 instances
   • These are the dpkg-maintscript-helper snippets for rm_conffile , mv_conffile etc. Hopefully in the near future, dpkg will support these directly.
8. dh_systemd_enable with 447 instances
9. dh_installemacsen with 179 instances
10. dh_icons with 165 instances
11. dh_installtex with 137 instances
12. dh_apache2 with 117 instances
13. dh_installudev with 98 instances
14. dh_installxfonts with 87 instances
15. dh_systemd_start with 79 instances

With this list, it seems to me that some obvious focus areas would be:

• Replacing the python scripts (I presume it is the byte-code handling, but I have not looked at this at all)
• Migrating away from menu files
• Support enabling + starting/stopping/restarting a service declaratively.
  • This might have a hidden requirement on declaratively creating service users if we want these packages to become truly maintscript-less .

Eventually we will also have to dig through all the manual maintainer scripts. But I think we got plenty to start with. [1] For some, circular dependencies in itself is an issue. I can certainly appreciate them as being suboptimal, but most of the issues we have are probably caused by insufficient tooling rather than a theoretical issue (that is, if we remove all postinst scripts).
Filed under: Debhelper, Debian, Lintian