Dirk Eddelbuettel: #4: Simpler shoulders()
Welcome to the fourth post in the repulsively random R ramblings series, or R4 for short.
My twitter feed was buzzing about a nice (and as yet unpublished, ie not-on-CRAN) package thankr by Dirk Schumacher which compiles a list of packages (ordered by maintainer count) for your current session (or installation or ...) with a view towards saying thank you to those whose packages we rely upon. Very nice indeed.
I had a quick look and run it twice ... and had a reaction of ewwww, really? as running it twice gave different results as on the second instance a boatload of tibblyverse packages appeared. Because apparently kids these day can only slice data that has been tidied or something.
So I had another quick look ... and put together an alternative version using just base R (as there was only one subfunction that needed reworking):
source(file="https://raw.githubusercontent.com/dirkschumacher/thankr/master/R/shoulders.R")
format_pkg_df <- function(df) # non-tibblyverse variant
tb <- table(df[,2])
od <- order(tb, decreasing=TRUE)
ndf <- data.frame(maint=names(tb)[od], npkgs=as.integer(tb[od]))
colpkgs <- function(m, df) paste(df[ df$maintainer == m, "pkg_name"], collapse=",")
ndf[, "pkg"] <- sapply(ndf$maint, colpkgs, df)
ndf
R> shoulders() ## by Dirk Schumacher, with small modifications
maint npkgs pkg
1 R Core Team <R-core@r-project.org> 9 compiler,graphics,tools,utils,grDevices,stats,datasets,methods,base
2 Dirk Eddelbuettel <edd@debian.org> 4 RcppTOML,Rcpp,RApiDatetime,anytime
3 Matt Dowle <mattjdowle@gmail.com> 1 data.table
R> 
shoulders() function is now free of side effects, and thankr is now a clean micropackage free of external depends from any verse, be it tiddly or grumpy. I look forward to seeing it on CRAN soon!
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
Debian Stretch feels like an excellent release by the Debian project.
The final 
In preparation for watching the new Planet Earth II (
This is now 10 years old but still top of the technique, even compared to the new 
White sharks are dangerous, we all know, but did you know that they can actually fly? Well, not actually fly, but they can jump as far as completely leaving the sea:
The first episode finishes in the Okavango Delta with elephants bathing in the sea. Here is a young club dancing under water.
Deeply impressed, deeply moved. In light of anti-environmental anti-climate-change anti-intelligence comedian at the White House, more of us should actually see these great pieces of journalism, let it influence our thinking, and hopefully vote differently when it is our turn again.
Next to the mSATA HDD sits the additional board, which houses the Power, USB, Audio In, and the SD Card reader.
Opening that up, I got the small board. I barely use the SD Card reader, and given the annoyances I had to suffer because of it, there was no more mercy in killing that device.
So, next was to unsolder the SD Card reader completely.
Once done, and fitted back into the machine, everything has been working awesomely great in the last 2 weeks. This entire fix costed me 0. So sometimes, fixing a bug is all that matters.
In the Hindi language, a nice phrase for such a scenario remnids me of the great 
The only thing that had to stop is leaving our users in the dark on what is happening.
Sorry for the long post, but there are some subjects which are worth writing more than 140 characters about 
Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you re interested in Android, Java, Games and LTS topics, this might be interesting for you.
Debian Android
And on the master branch as of 
An interesting thing to note here is that the glibc allocator runaway memory use was, this time, more pronounced on 0.4.0 than on master. It was the opposite originally, but as I mentioned in the past ASLR is making it not happen exactly the same way each time.
While I m here, one thing I failed to mention in the previous posts is that all these measurements were done by cloning a local mercurial clone of 
So where does that put us?
A few observations to make here:
(Bucket size is 16 bytes)
What is not completely obvious from the logarithmic scale is that, in fact, 98.4% of the allocations are less than 512 bytes with the current master (35c18e7), and they were 95.5% with be75326. Interestingly, though, in absolute numbers, there are less allocations smaller than 512 bytes in current master than in be75326 (1,194,268,071 vs 1,214,784,494). This suggests the extra allocations that happen during some phases are larger than that.
There are clearly less allocations across the board (apart from very few exceptions), and close to an order of magnitude less allocations larger than 1MiB. In fact, widening the bucket size to 32KiB shows an order of magnitude difference (or close) for most buckets:
An interesting thing to note is how some sizes are largely overrepresented in the data with buckets of 16 bytes, like 768, 1104, 2048, 4128, with other smaller bumps for e.g. 2144, 2464, 2832, 3232, 3696, 4208, 4786, 5424, 6144, 6992, 7920 While some of those are powers of 2, most aren t, and some of them may actually represent objects sized with a power of 2, but that have an extra PyObject overhead.
While looking at allocation stats, I got to wonder what the lifetimes of those allocations looked like. So I scanned the allocator logs and measured the distance between when an allocation is made and when it is 
What comes out of this data:
As I have previously blogged, I recently managed to resurrect my old Drupal site that ran in the Amazon AWS cloud, and get it working again on a new host. I have just written up a summary of how I battled through the process, which can be found 
Home office minister Amber Rudd recently suggested British firms should 
All this racism, it's just not cricket. Or is it? One of the most remarkable cricketers to play for England in recent times, 
In fact, after Brexit, people like Venables and Thompson will remain in Britain while 
What does "maximum control" over other human beings mean to you?
The new British PM has said she wants "
Echoes of the Third Reich
Most people associate acts of mass murder with the Germans who lived in the time of Adolf Hitler. These are the stories told over and and over again in movies, books and the press.
Look more closely, however, and it appears that the vast majority of Germans were not in immediate contact with the gas chambers. Even Gobels' secretary writes that 
There is no middle ground between "soft" and "hard" brexit
An important point for British citizens and foreigners in the UK to consider today is that there is no compromise between a "soft" Brexit and a "hard" Brexit. It is one or the other. Anything less (for example, a deal that is "better" for British companies and worse for EU citizens) would imply that the British are a superior species and it is impossible to imagine the EU putting their stamp on such a deal. Anybody from the EU who is trying to make a life in the UK now is playing a game of Russian Roulette - sure, everything might be fine if it morphs into "soft" Brexit, but if Theresa May has her way, at some point in your life, maybe 20 years down the track, you could be rounded up by the gestapo and thrown behind bars for a parking violation. 
Yesterday