Search Results: "edd"

17 August 2017

Shirish Agarwal: Composers are not given due recognition

Update Some youtube links are not viewable or even seen on planet.debian.org. Seems p.d.o. tries its best to remove external links, sorry for the breakage. Beware some youtube-links would be shared in this entry, sorry couldn t find a better/easier media platform to work with. If anyone knows any other platform or wants to suggest, feel free to either mail me or let me know in comments. I want to start today s sharing with a picture of Ganesha I saw today. It is and was public art hence sharing it without an issue. Sketch of Ganesha/Ganapati This is starting of festivities time in India and Ganesha or Ganpati is looked up as a good omen in India. The festival of Ganesh Chaturthi would be starting on the 25th of August and is a sight to behold. Just like Rio has its carnival, Ganesh Chaturthi is also a carnival. We also have parades where people come with Pandals (or temporary structures) The mythology says he has a sweet tooth (hence lot of distribution of sweets, especially modak) and anything which might be troubling people, he creates solutions for them. Here is one video of how people celebrate his immersion in India. This is from my home-town few years ago, every year the madness and the celebrations are becoming more and more. People from far off come to see how we celebrate and see how different people make their Pandals. While some are with music, others are with social messages. Usually people start going to see these structures after dusk and return home way after midnight or early morning. I hope to do this endeavour after many years. One is drunk from hearing all sorts of different kids of music, decoration, messages, a feast and a strain to all the senses. Ganesha immersion celebrations https://www.youtube.com/watch?v=hjWfpGUryho If one is interested one can find more info. at https://en.wikipedia.org/wiki/Ganesh_Chaturthi After quite a bit of time, I wrote an article about various foss internships which I knew besides GSOC over the years. I finally penned them down at https://itsfoss.com/best-open-source-internships/ Interestingly, I was amazed to see that all FOSS U.S. projects (outside of GSOC) are for students who are either living or studying in U.S. and have a student work visa (which from private discussions I came to know is lot harder to get nowadays than before). Except for the National Science Foundation (NSF) which probably has U.S. defence relations and hence they might be sensitive, I fail to understand other institutes preferences for only getting people from the U.S. and hence having a lesser talent pool of people. This also affects the growth of the projects themselves. Just think how limited Debian would have been if it had decided to only have people from only any one community develop it. Dunno if this is due to the present President Trump or these policies had been there before. It would be nice and interesting if people in the know can share. What has also been interesting to watch is Mr. Trump blaming low-cost manufacturing centres like India and China when as far as I recall, lot of manufacturing, specifically auto-mobiles manufacturing was shifted out of the U.S. to Ireland and other places years before which are relatively high-cost places (at least compared to India). I *believe* the change was as early as in 1980 s itself where India was insulated and had a limited market for everything (similar to Russian communism as shown in popular media but not so bad.) Interestingly, it took almost a month for the perl 2.56 to make the transition smoothly. It took quite a bit of time for all the components to work together and be installable. Also saw this few days back http://fortune.com/2017/04/12/auto-industry-decline/ While Tesla is expensive even by American standards the idea of lesser parts, lesser complexity and hence lower costs to use, maintain is good. I do hope that he and his team or any of the competitors do overcome the significant challenges. Any significant improvement in battery technology is bound to have huge impact in almost everything that is used in 21st century. Two recent articles tell me the future may become present very quickly. https://www.purdue.edu/newsroom/releases/2017/Q2/instantly-rechargeable-battery-could-change-the-future-of-electric-and-hybrid-automobiles.html Toyota could finally start mass producing electric cars thanks to China I do hope to see EV being prevalent before the next decade is over otherwise we don t have any hope due to climate change. As for my health, I am much better than before. Just to share some stats, before my illness for lack of better word, I was 120 kgs. , when I was kept in the hospital for about 2-2.5 weeks I came down to 95 kgs. and now back upto 108 kgs. Do go for exercising every other day and trying to get back the strength, stamina and increasing a bit of both. Doctors have given me another 4-5 months after which a brain scan will reveal if there are any remaining blood clots in the brain or not. Lastly, while it has become somewhat of a sensitive issue to love Muslims or to talk about their work in any field in the current political climate, there are 4-5 music pieces I listen whenever I can, especially before going to bed. While almost all the pieces have been sung and written by Muslims, sadly I don t know who the composers of these beautiful songs are. While it is much easier to get the names of the singer and the lyricist, one of the more important roles in my view is the composer or/and music arranger. Without them, the songs would not have the same haunting quality that the songs have. While I have been lucky to find the names of the composer/music arranger for the pieces below but this is not the case if and when the songs comes on television. I do remember in old times at least on Radio they used to mention about who has given the music as well, dunno in modern times. I am sharing the songs, and hopefully will also share the translations if I find on the web, please see the lyrics. The numbering is for convenience only and am torn in these 4-5 songs which is the best. Just to share these are all sufi love songs except the last one which I am sharing. 1. Lyrical song https://www.youtube.com/watch?v=ehqN6oTpmb8 Translation with video of song http://www.bollynook.com/en/lyrics/6443/aaj-din-chadheya/ While there probably are stories with each song, I was lucky to find the story about this one. The lyrics of the song are actually a love lost Punjabi poet who writes in the memory of his beloved to which he could not marry and he pens those when standing in line for his liquor. The story goes on that he marries a girl later in life who bears a resemblance to his beloved whom he couldn t forget till his dying day. 2. Lyrical song https://www.youtube.com/watch?v=uTC_2c83qn0 The same song has been sung by different people and I love them all the more for it. Another video https://www.youtube.com/watch?v=3G7Qg4LJ7WE Another video https://www.youtube.com/watch?v=kOsvNuR3m5Y Translation http://www.bollynook.com/en/lyrics/10703/o-re-piya/ 3. Lyrical song https://www.youtube.com/watch?v=qG7Kms_YA5Q The translation http://www.filmyquotes.com/songs/885 The translation of the song is a bit crude but then translations are supposed to be crude  Anyways, the above song is what would be called a perfect Sufi song. I hope people enjoy the longing and the silence which follows this piece. Another classic one 4. Lyrical song https://www.youtube.com/watch?v=Ube5XhN_lpM English translation http://www.ardhamy.com/song/aye-dil-e-naadan While the song is from the movie Razia Sultana and was a flop as the movie was about Race and controversial then as it probably would be now. As seen in the other songs of the same genre, it has strands of longing, loneliness as seen of the above. 5. Lyrical song https://www.youtube.com/watch?v=tv242qOnHJA This one is not a sufi song but I love all the women and the girls and the way they enhanced the song. I dunno how much they must have practised as it s a very fast and peppy song and doesn t give time to the singer to breathe except for that one section which has a bit of Carnatic music. At the very end I would like to share http://www.globalrhythm.net/ I have found some interesting sounds on the site. Hope the site enriches you as well. FWIW I have no links with the site except as somebody who likes to diversify his music listening. Lastly, for a long period of time, I had been hearing the criticism, especially for FOSS games that they don t have AAA quality assets. Recently I came across a game called Starship Theory (sadly its only for MS-Windows) Game video https://www.youtube.com/watch?v=imaL2pjNURg You look at the game and see the number of videos the guy has made. What FOSS game developers can learn from this, you don t need high-end 2.5/3d models, clipart will do but need depth in gameplay which can make FOSS games be popular and also earn a pretty bundle. I do hope some FOSS game upstream developers take note and use that game s inspiration to bring more depth. That doesn t mean games like 0ad are not liked by people but it takes huge amount of time and resources. 0ad video https://www.youtube.com/watch?v=DHx5XBtypcQ Hope you have a good time with all the ideas, anecdotes and videos I shared above.
Filed under: Miscellenous Tagged: #FOSS Internships, #Ganesh Chaturthi, #Ganeshji, #planet-debian, #Sufi Bollywood Music, FOSS, FOSS games, politics

16 August 2017

Simon McVittie: DebConf 17: Flatpak and Debian

The indoor garden at Coll ge de Maisonneuve, the DebConf 17 venue
Decorative photo of the indoor garden
I'm currently at DebConf 17 in Montr al, back at DebConf for the first time in 10 years (last time was DebConf 7 in Edinburgh). It's great to put names to faces and meet more of my co-developers in person! On Monday I gave a talk entitled A Debian maintainer's guide to Flatpak , aiming to introduce Debian developers to Flatpak, and show how Flatpak and Debian (and Debian derivatives like SteamOS) can help each other. It seems to have been quite well received, with people generally positive about the idea of using Flatpak to deliver backports and faster-moving leaf packages (games!) onto the stable base platform that Debian is so good at providing. A video of the talk is available from the Debian Meetings Archive. I've also put up my slides in the DebConf git-annex repository, with some small edits to link to more source code: A Debian maintainer's guide to Flatpak. Source code for the slides is also available from Collabora's git server. The next step is to take my proof-of-concept for building Flatpak runtimes and apps from Debian and SteamOS packages, flatdeb, get it a bit more production-ready, and perhaps start publishing some sample runtimes from a cron job on a Debian or Collabora server. (By the way, if you downloaded that source right after my talk, please update - I've now pushed some late changes that were necessary to fix the 3D drivers for my OpenArena demo.) I don't think Debian will be going quite as far as Endless any time soon: as Cosimo outlined in the talk right before mine, they deploy their Debian derivative as an immutable base OS with libOSTree, with all the user-installable modules above that coming from Flatpak. That model is certainly an interesting thing to think about for Debian derivatives, though: at Collabora we work on a lot of appliance-like embedded Debian derivatives, with a lot of flexibility during development but very limited state on deployed systems, and Endless' approach seems a perfect fit for those situations. [Edited 2017-08-16 to fix the link for the slides, and add links for the video]

15 August 2017

Dirk Eddelbuettel: #9: Compacting your Shared Libraries

Welcome to the nineth post in the recognisably rancid R randomness series, or R4 for short. Following on the heels of last week's post, we aim to look into the shared libraries created by R. We love the R build process. It is robust, cross-platform, reliable and rather predicatable. It. Just. Works. One minor issue, though, which has come up once or twice in the past is the (in)ability to fully control all compilation options. R will always recall CFLAGS, CXXFLAGS, ... etc as used when it was compiled. Which often entails the -g flag for debugging which can seriously inflate the size of the generated object code. And once stored in $ RHOME /etc/Makeconf we cannot on the fly override these values. But there is always a way. Sometimes even two. The first is local and can be used via the (personal) ~/.R/Makevars file (about which I will have to say more in another post). But something I have been using quite a bite lately uses the flags for the shared library linker. Given that we can have different code flavours and compilation choices---between C, Fortran and the different C++ standards---one can end up with a few lines. I currently use this which uses -Wl, to pass an the -S (or --strip-debug) option to the linker (and also reiterates the desire for a shared library, presumably superfluous):
SHLIB_CXXLDFLAGS = -Wl,-S -shared
SHLIB_CXX11LDFLAGS = -Wl,-S -shared
SHLIB_CXX14LDFLAGS = -Wl,-S -shared
SHLIB_FCLDFLAGS = -Wl,-S -shared
SHLIB_LDFLAGS = -Wl,-S -shared
Let's consider an example: my most recently uploaded package RProtoBuf. Built under a standard 64-bit Linux setup (Ubuntu 17.04, g++ 6.3) and not using the above, we end up with library containing 12 megabytes (!!) of object code:
edd@brad:~/git/rprotobuf(feature/fewer_warnings)$ ls -lh src/RProtoBuf.so
-rwxr-xr-x 1 edd edd 12M Aug 14 20:22 src/RProtoBuf.so
edd@brad:~/git/rprotobuf(feature/fewer_warnings)$ 
However, if we use the flags shown above in .R/Makevars, we end up with much less:
edd@brad:~/git/rprotobuf(feature/fewer_warnings)$ ls -lh src/RProtoBuf.so 
-rwxr-xr-x 1 edd edd 626K Aug 14 20:29 src/RProtoBuf.so
edd@brad:~/git/rprotobuf(feature/fewer_warnings)$ 
So we reduced the size from 12mb to 0.6mb, an 18-fold decrease. And the file tool still shows the file as 'not stripped' as it still contains the symbols. Only debugging information was removed. What reduction in size can one expect, generally speaking? I have seen substantial reductions for C++ code, particularly when using tenmplated code. More old-fashioned C code will be less affected. It seems a little difficult to tell---but this method is my new build default as I continually find rather substantial reductions in size (as I tend to work mostly with C++-based packages). The second option only occured to me this evening, and complements the first which is after all only applicable locally via the ~/.R/Makevars file. What if we wanted it affect each installation of a package? The following addition to its src/Makevars should do:
strippedLib: $(SHLIB)
        if test -e "/usr/bin/strip"; then /usr/bin/strip --strip-debug $(SHLIB); fi
.phony: strippedLib
We declare a new Makefile target strippedLib. But making it dependent on $(SHLIB), we ensure the standard target of this Makefile is built. And by making the target .phony we ensure it will always be executed. And it simply tests for the strip tool, and invokes it on the library after it has been built. Needless to say we get the same reduction is size. And this scheme may even pass muster with CRAN, but I have not yet tried. Lastly, and acknowledgement. Everything in this post has benefited from discussion with my former colleague Dan Dillon who went as far as setting up tooling in his r-stripper repository. What we have here may be simpler, but it would not have happened with what Dan had put together earlier.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

14 August 2017

Michal Čihař: New projects on Hosted Weblate

Hosted Weblate provides also free hosting for free software projects. The hosting requests queue was over one month long, so it's time to process it and include new project. This time, the newly hosted projects include: If you want to support this effort, please donate to Weblate, especially recurring donations are welcome to make this service alive. You can do them on Liberapay or Bountysource.

Filed under: Debian English SUSE Weblate

13 August 2017

Dirk Eddelbuettel: RProtoBuf 0.4.10

RProtoBuf provides R bindings for the Google Protocol Buffers ("ProtoBuf") data encoding and serialization library used and released by Google, and deployed fairly widely in numerous projects as a language and operating-system agnostic protocol. A new releases RProtoBuf 0.4.10 just appeared on CRAN. It is a maintenance releases replacing one leftover errorenous use of package= in .Call with the correct PACKAGE= (as requsted by CRAN). It also integrates a small robustification in the deserializer when encountering invalide objects; this was both reported and fixed by Jeffrey Shen.

Changes in RProtoBuf version 0.4.10 (2017-08-13)
  • More careful operation in deserializer checking for a valid class attribute (Jeffrey Shen in #29 fixing #28)
  • At the request of CRAN, correct one .Call() argument to PACKAGE=; update routine registration accordingly

CRANberries also provides a diff to the previous release. The RProtoBuf page has an older package vignette, a 'quick' overview vignette, a unit test summary vignette, and the pre-print for the JSS paper. Questions, comments etc should go to the GitHub issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

11 August 2017

Dirk Eddelbuettel: #8: Customizing Spell Checks for R CMD check

Welcome to the eight post in the ramblingly random R rants series, or R4 for short. We took a short break over the last few weeks due to some conferencing followed by some vacationing and general chill. But we're back now, and this post gets us back to initial spirit of (hopefully) quick and useful posts. Perusing yesterday's batch of CRANberries posts, I noticed a peculiar new directory shown the in the diffstat output we use to compare two subsequent source tarballs. It was entitled .aspell/, in the top-level directory, and in two new packages by R Core member Kurt Hornik himself. The context is, of course, the not infrequently-expressed desire to customize the spell checking done on CRAN incoming packages, see e.g. this r-package-devel thread. And now we can as I verified with (the upcoming next release of) RcppArmadillo, along with a recent-enough (i.e. last few days) version of r-devel. Just copying what Kurt did, i.e. adding a file .aspell/defaults.R, and in it pointing to rds file (named as the package) containing a character vector with words added to the spell checker's universe is all it takes. For my package, see here for the peculiars. Or see here:
edd@bud:~/git/rcpparmadillo/.aspell(master)$ cat defaults.R 
Rd_files <- vignettes <- R_files <- description <-
    list(encoding = "UTF-8",
         language = "en",
         dictionaries = c("en_stats", "RcppArmadillo"))
edd@bud:~/git/rcpparmadillo/.aspell(master)$ r -p -e 'readRDS("RcppArmadillo.rds")'
[1] "MPL"            "Sanderson"      "Templated"
[4] "decompositions" "onwards"        "templated"
edd@bud:~/git/rcpparmadillo/.aspell(master)$     
And now R(-devel) CMD check --as-cran ... is silent about spelling. Yay! But take this with a grain of salt as this does not yet seem to be "announced" as e.g. yesterday's change in the CRAN Policy did not mention it. So things may well change -- but hey, it worked for me. And this all is about aspell, here is something topical about a spell to close the post:

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

6 August 2017

Daniel Silverstone: STM32 and RTFM

I have been working with STM32 chips on-and-off for at least eight, possibly closer to nine years. About as long as ST have been touting them around. I love the STM32, and have done much with them in C. But, as my previous two posts may have hinted, I would like to start working with Rust instead of C. To that end, I have been looking with great joy at the work which Jorge Aparicio has been doing around Cortex-M3 and Rust. I've had many comments in person at Debconf, and also several people mention on Twitter, that they're glad more people are looking at this. But before I can get too much deeper into trying to write my USB stack, I need to sort a few things from what Jorge has done as demonstration work.

Okay, this is fast, but we need Ludicrous speed All of Jorge's examples seem to leave the system clocks in a fairly default state, excepting turning on the clocks to the peripherals needed during the initialisation phase. Sadly, if we're going to be running the USB at all, we need the clocks to run a tad faster. Since my goal is to run something moderately CPU intensive on the end of the USB too, it makes sense to try and get our STM32 running at maximum clock speed. For the one I have, that's 72MHz rather than the 8MHz it starts out with. Nine times more cycles to do computing in makes a lot of sense. As I said above, I've been doing STM32 in C a lot for many years; and fortunately I have built systems with the exact chip that's on the blue-pill before. As such, if I rummage, I can find some old C code which does what we need...
    /* Enable HSE */
    RCC_HSEConfig(RCC_HSE_ON);
    /* Wait till HSE is ready */
    HSEStartUpStatus = RCC_WaitForHSEStartUp();
    if (HSEStartUpStatus == SUCCESS)
     
      /* Enable Prefetch Buffer */
      FLASH_PrefetchBufferCmd(FLASH_PrefetchBuffer_Enable);
      /* Flash 2 wait state */
      FLASH_SetLatency(FLASH_Latency_2);
      /* HCLK = SYSCLK */
      RCC_HCLKConfig(RCC_SYSCLK_Div1);
      /* PCLK2 = HCLK */
      RCC_PCLK2Config(RCC_HCLK_Div1);
      /* PCLK1 = HCLK/2 */
      RCC_PCLK1Config(RCC_HCLK_Div2);
      /* ADCCLK = PCLK2/6 */
      RCC_ADCCLKConfig(RCC_PCLK2_Div6);
      /* PLLCLK = 8MHz * 9 = 72 MHz */
      RCC_PLLConfig(RCC_PLLSource_HSE_Div1, RCC_PLLMul_9);
      /* Enable PLL */
      RCC_PLLCmd(ENABLE);
      /* Wait till PLL is ready */
      while (RCC_GetFlagStatus(RCC_FLAG_PLLRDY) == RESET)
       
      /* Select PLL as system clock source */
      RCC_SYSCLKConfig(RCC_SYSCLKSource_PLLCLK);
      /* Wait till PLL is used as system clock source */
      while (RCC_GetSYSCLKSource() != 0x08)
       
     
This code, rather conveniently, uses an 8MHz external crystal so we can almost direct-port it to the blue-pill Rust code and see how we go. If you're used to the CMSIS libraries for STM32, then you won't completely recognise the above since it uses the pre-CMSIS core libraries to do its thing. Library code from 2008 and it's still good on today's STM32s providing they're in the right family :-) A direct conversion to Rust, using Jorge's beautifully easy to work with crates made from svd2rust results in:
    fn make_go_faster(rcc: &RCC, flash: &FLASH)  
        rcc.cr.modify( _, w  w.hseon().enabled());
        while !rcc.cr.read().hserdy().is_ready()  
        flash.acr.modify( _, w  w.prftbe().enabled());
        flash.acr.modify( _, w  w.latency().two());
        rcc.cfgr.modify( _, w  w
                        .hpre().div1()
                        .ppre2().div1()
                        .ppre1().div2()
                        // .adcpre().bits(8)
                        .pllsrc().external()
                        .pllxtpre().div1()
                        .pllmul().mul9()
        );
        rcc.cr.modify( _, w  w.pllon().enabled());
        while rcc.cr.read().pllrdy().is_unlocked()  
        rcc.cfgr.modify( _,w  w.sw().pll());
        while !rcc.cfgr.read().sws().is_pll()  
     
Now, I've not put the comments in which were in the C code, because I'm being very lazy right now, but if you follow the two together you should be able to work it through. I don't have timeouts for the waits, and you'll notice a single comment there (I cannot set up the ADC prescaler because for some reason the SVD is missing any useful information and so the generated crate only carries an unsafe function (bits()) and I'm trying to steer clear of unsafe for now. Still, I don't need the ADC immediately, so I'm okay with this. By using this function in the beginning of the init() function of the blinky example, I can easily demonstrate the clock is going faster since the LED blinks more quickly. This function demonstrates just how simple it is to take bit-manipulation from the C code and turn it into (admittedly bad looking) Rust with relative ease and without any of the actual bit-twiddling. I love it.

Mess with time, and you get unexpected consequences Sadly, when you mess with the clock tree on a microcontroller, you throw a lot of things out of whack. Not least, by adjusting the clock frequency up we end up adjusting the AHB, APB1, and APB2 clock frequencies. This has direct consequences for peripherals floating around on those busses. Fortunately Jorge thought of this and while the blue-pill crate hard-wires those frequencies to 8MHz, they are, at least, configurable in code in some sense. If we apply the make_go_faster() function to the serial loopback example, it simply fails to work because now the bus which the USART1 peripheral is connected to (APB2) is going at a different speed from the expected power-on default of 8MHz. If you remember from the function, we did .hpre().div1() which set HCLK to 72MHz, then .ppre1().div2() which sets the APB1 bus clock to be HCLK divided by 2, and .ppre2().div1() which sets APB2 bus clock to be HCLK. This means that we'd need to alter src/lib.rs to reflect these changes in the clock frequences and in theory loopback would start working once more. It'd be awkward to try and demonstrate all that to you since I only have a phone camera to hand, but if you own a blue-pill then you can clone Jorge's repo and have a go yourself and see that I'm not bluffing you. With all this done, it'll be time to see if we can bring the USB peripheral in the STM32 online, and that will be the topic of my next post in this discovery series.

4 August 2017

Dirk Eddelbuettel: R for System Adminstration

Just getting back from the most fun meetup I have been to in quite some time: episode 23 (by their count) of Open Source Open Mic hosted by Matt Godbolt and Joe Walnes here in Chicago. Nothing but a sequence of lightning talks. Plus beer and pizza. Sounds awesome? It was! We had fantastic talks across at least half a dozen languages, covering both new-ish (Pony) and interesting ones such (Rust, Go, ...) plus of course some Javascript and some Python, no Java (yay!) and a few batshit crazy things like a self-hosting database in its own (shell) code, a terminal gif viewer (!!), and more. And it gave me an opportunity to quickly (one evening and morning commute) jam out a presentation about what is in the title: R for system administration. And I am only half-joking. I had used R a couple of years ago when I needed to select, subset, modify, ... a large number of image files given some timestamp and filename patterns. And given how well R works in a vectorised manner with both regular expressions and timestamps, as well as on top of essentially all standard POSIX-style operating system / file-system functions, I picked up that thread again on the problem of ... cleaning up the file storage underlying CRANberries which by now has well over fifty-seven thousand (!!) tarballs of CRAN packages based on now ten years of CRANberries. So I showed how to prune this in essentially half a dozen lines of R (and data.table code), plus some motivation---all just right for a lightning talk. Seemingly the talk went well enough as quite a few folks gave a thumbs up and compliments over beers afterwards. But see for yourself as the slides are now uploaded to my standard talks page. My thanks to Matt and Joe for organizing the meetup. I think I will be back.

1 August 2017

Paul Wise: FLOSS Activities July 2017

Changes

Issues

Review

Administration
  • Debian: fsck/reboot a buildd, reboot a segfaulting buildd, report/fix broken hoster contact, ping hoster about down machines, forcibly reset backup machine, merged cache patch for network-test.d.o, do some samhain dances, fix two stunnel services, update an IP address in LDAP, fix /etc/aliases on one host, reboot 1 non-responsive VM
  • Debian mentors: security updates, reboot
  • Debian wiki: whitelist several email addresses
  • Debian build log scanner: deploy my changes
  • Debian PTS: deploy my changes
  • Openmoko: security updates & reboots

Communication
  • Ping Advogato users on Planet Debian about updating/removing their feeds since it shut down
  • Invite deepin to the Debian derivatives census
  • Welcome Deepin to the Debian derivatives census
  • Inquire about the status of GreenboneOS, HandyLinux

Sponsors All work was done on a volunteer basis.

Reproducible builds folks: Reproducible Builds: Weekly report #118

Here's what happened in the Reproducible Builds effort between Sunday July 23 and Saturday July 29 2017: Toolchain development and fixes Packages reviewed and fixed, and bugs filed Reviews of unreproducible packages 4 package reviews have been added, 2 have been updated and 24 have been removed in this week, adding to our knowledge about identified issues. Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by: diffoscope development Misc. This week's edition was written by Chris Lamb, Mattia Rizzolo & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Russell Coker: QEMU for ARM Processes

I m currently doing some embedded work on ARM systems. Having a virtual ARM environment is of course helpful. For the i586 class embedded systems that I run it s very easy to setup a virtual environment, I just have a chroot run from systemd-nspawn with the --personality=x86 option. I run it on my laptop for my own development and on a server my client owns so that they can deal with the hit by a bus scenario. I also occasionally run KVM virtual machines to test the boot image of i586 embedded systems (they use GRUB etc and are just like any other 32bit Intel system). ARM systems have a different boot setup, there is a uBoot loader that is fairly tightly coupled with the kernel. ARM systems also tend to have more unusual hardware choices. While the i586 embedded systems I support turned out to work well with standard Debian kernels (even though the reference OS for the hardware has a custom kernel) the ARM systems need a special kernel. I spent a reasonable amount of time playing with QEMU and was unable to make it boot from a uBoot ARM image. The Google searches I performed didn t turn up anything that helped me. If anyone has good references for getting QEMU to work for an ARM system image on an AMD64 platform then please let me know in the comments. While I am currently surviving without that facility it would be a handy thing to have if it was relatively easy to do (my client isn t going to pay me to spend a week working on this and I m not inclined to devote that much of my hobby time to it). QEMU for Process Emulation I ve given up on emulating an entire system and now I m using a chroot environment with systemd-nspawn. The package qemu-user-static has staticly linked programs for emulating various CPUs on a per-process basis. You can run this as /usr/bin/qemu-arm-static ./staticly-linked-arm-program . The Debian package qemu-user-static uses the binfmt_misc support in the kernel to automatically run /usr/bin/qemu-arm-static when an ARM binary is executed. So if you have copied the image of an ARM system to /chroot/arm you can run the following commands like the following to enter the chroot: cp /usr/bin/qemu-arm-static /chroot/arm/usr/bin/qemu-arm-static
chroot /chroot/arm bin/bash Then you can create a full virtual environment with /usr/bin/systemd-nspawn -D /chroot/arm if you have systemd-container installed. Selecting the CPU Type There is a huge range of ARM CPUs with different capabilities. How this compares to the range of x86 and AMD64 CPUs depends on how you are counting (the i5 system I m using now has 76 CPU capability flags). The default CPU type for qemu-arm-static is armv7l and I need to emulate a system with a armv5tejl. Setting the environment variable QEMU_CPU=pxa250 gives me armv5tel emulation. The ARM Architecture Wikipedia page [2] says that in armv5tejl the T stands for Thumb instructions (which I don t think Debian uses), the E stands for DSP enhancements (which probably isn t relevant for me as I m only doing integer maths), the J stands for supporting special Java instructions (which I definitely don t need) and I m still trying to work out what L means (comments appreciated). So it seems clear that the armv5tel emulation provided by QEMU_CPU=pxa250 will do everything I need for building and testing ARM embedded software. The issue is how to enable it. For a user shell I can just put export QEMU_CPU=pxa250 in .login or something, but I want to emulate an entire system (cron jobs, ssh logins, etc). I ve filed Debian bug #870329 requesting a configuration file for this [1]. If I put such a configuration file in the chroot everything would work as desired. To get things working in the meantime I wrote the below wrapper for /usr/bin/qemu-arm-static that calls /usr/bin/qemu-arm-static.orig (the renamed version of the original program). It s ugly (I would use a config file if I needed to support more than one type of CPU) but it works. #include <stdio.h>
#include <unistd.h>
#include <stdlib.h> int main(int argc, char **argv)

if(setenv("QEMU_CPU", "pxa250", 1))

printf("Can't set $QEMU_CPU\n");
return 1;

execv("/usr/bin/qemu-arm-static.orig", argv);
printf("Can't execute \"%s\" because of qemu failure\n", argv[0]);
return 1;

31 July 2017

Chris Lamb: Free software activities in July 2017

Here is my monthly update covering what I have been doing in the free software world during July 2017 (previous month): I also blogged about my recent lintian hacking and installation-birthday package.
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. (I have generously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area.) This month I:
  • Assisted Mattia with a draft of an extensive status update to the debian-devel-announce mailing list. There were interesting follow-up discussions on Hacker News and Reddit.
  • Submitted the following patches to fix reproducibility-related toolchain issues within Debian:
  • I also submitted 5 patches to fix specific reproducibility issues in autopep8, castle-game-engine, grep, libcdio & tinymux.
  • Categorised a large number of packages and issues in the Reproducible Builds "notes" repository.
  • Worked on publishing our weekly reports. (#114 #115, #116 & #117)

I also made the following changes to our tooling:
diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • comparators.xml:
    • Fix EPUB "missing file" tests; they ship a META-INF/container.xml file. [ ]
    • Misc style fixups. [ ]
  • APK files can also be identified as "DOS/MBR boot sector". (#868486)
  • comparators.sqlite: Simplify file detection by rewriting manual recognizes call with a Sqlite3Database.RE_FILE_TYPE definition. [ ]
  • comparators.directory:
    • Revert the removal of a try-except. (#868534)
    • Tidy module. [ ]

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.

  • Add missing File::Temp imports in the JAR and PNG handlers. This appears to have been exposed by lazily-loading handlers in #867982. (#868077)

buildinfo.debian.net

buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.

  • Avoid a race condition between check-and-creation of Buildinfo instances. [ ]


Debian My activities as the current Debian Project Leader are covered in my "Bits from the DPL emails to the debian-devel-announce mailing list.
Patches contributed
  • obs-studio: Remove annoying "click wrapper" on first startup. (#867756)
  • vim: Syntax highlighting for debian/copyright files. (#869965)
  • moin: Incorrect timezone offset applied due to "84600" typo. (#868463)
  • ssss: Add a simple autopkgtest. (#869645)
  • dch: Please bump $latest_bpo_dist to current stable release. (#867662)
  • python-kaitaistruct: Remove Markdown and homepage references from package long descriptions. (#869265)
  • album-data: Correct invalid Vcs-Git URI. (#869822)
  • pytest-sourceorder: Update Homepage field. (#869125)
I also made a very large number of contributions to the Lintian static analysis tool. To avoid duplication here, I have outlined them in a separate post.

Debian LTS

This month I have been paid to work 18 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 1014-1 for libclamunrar, a library to add unrar support to the Clam anti-virus software to fix an arbitrary code execution vulnerability.
  • Issued DLA 1015-1 for the libgcrypt11 crypto library to fix a "sliding windows" information leak.
  • Issued DLA 1016-1 for radare2 (a reverse-engineering framework) to prevent a remote denial-of-service attack.
  • Issued DLA 1017-1 to fix a heap-based buffer over-read in the mpg123 audio library.
  • Issued DLA 1018-1 for the sqlite3 database engine to prevent a vulnerability that could be exploited via a specially-crafted database file.
  • Issued DLA 1019-1 to patch a cross-site scripting (XSS) exploit in phpldapadmin, a web-based interface for administering LDAP servers.
  • Issued DLA 1024-1 to prevent an information leak in nginx via a specially-crafted HTTP range.
  • Issued DLA 1028-1 for apache2 to prevent the leakage of potentially confidential information via providing Authorization Digest headers.
  • Issued DLA 1033-1 for the memcached in-memory object caching server to prevent a remote denial-of-service attack.

Uploads
  • redis:
    • 4:4.0.0-1 Upload new major upstream release to unstable.
    • 4:4.0.0-2 Make /usr/bin/redis-server in the primary package a symlink to /usr/bin/redis-check-rdb in the redis-tools package to prevent duplicate debug symbols that result in a package file collision. (#868551)
    • 4:4.0.0-3 Add -latomic to LDFLAGS to avoid a FTBFS on the mips & mipsel architectures.
    • 4:4.0.1-1 New upstream version. Install 00-RELEASENOTES as the upstream changelog.
    • 4:4.0.1-2 Skip non-deterministic tests that rely on timing. (#857855)
  • python-django:
    • 1:1.11.3-1 New upstream bugfix release. Check DEB_BUILD_PROFILES consistently, not DEB_BUILD_OPTIONS.
  • bfs:
    • 1.0.2-2 & 1.0.2-3 Use help2man to generate a manpage.
    • 1.0.2-4 Set hardening=+all for bindnow, etc.
    • 1.0.2-5 & 1.0.2-6 Don't use upstream's release target as it overrides our CFLAGS & install RELEASES.md as the upstream changelog.
    • 1.1-1 New upstream release.
  • libfiu:
    • 0.95-4 Apply patch from Steve Langasek to fix autopkgtests. (#869709)
  • python-daiquiri:
    • 1.0.1-1 Initial upload. (ITP)
    • 1.1.0-1 New upstream release.
    • 1.1.0-2 Tidy package long description.
    • 1.2.1-1 New upstream release.

I also reviewed and sponsored the uploads of gtts-token 1.1.1-1 and nlopt 2.4.2+dfsg-3.

Debian bugs filed
  • ITP: python-daiquiri Python library to easily setup basic logging functionality. (#867322)
  • twittering-mode: Correct incorrect time formatting due to "84600" typo. (#868479)

29 July 2017

Dirk Eddelbuettel: Updated overbought/oversold plot function

A good six years ago I blogged about plotOBOS() which charts a moving average (from one of several available variants) along with shaded standard deviation bands. That post has a bit more background on the why/how and motivation, but as a teaser here is the resulting chart of the SP500 index (with ticker ^GSCP): Example chart of overbought/oversold levels from plotOBOS() function The code uses a few standard finance packages for R (with most of them maintained by Joshua Ulrich given that Jeff Ryan, who co-wrote chunks of these, is effectively retired from public life). Among these, xts had a recent release reflecting changes which occurred during the four (!!) years since the previous release, and covering at least two GSoC projects. With that came subtle API changes: something we all generally try to avoid but which is at times the only way forward. In this case, the shading code I used (via polygon() from base R) no longer cooperated with the beefed-up functionality of plot.xts(). Luckily, Ross Bennett incorporated that same functionality into a new function addPolygon --- which even credits this same post of mine. With that, the updated code becomes
## plotOBOS -- displaying overbough/oversold as eg in Bespoke's plots
##
## Copyright (C) 2010 - 2017  Dirk Eddelbuettel
##
## This is free software: you can redistribute it and/or modify it
## under the terms of the GNU General Public License as published by
## the Free Software Foundation, either version 2 of the License, or
## (at your option) any later version.
suppressMessages(library(quantmod))     # for getSymbols(), brings in xts too
suppressMessages(library(TTR))          # for various moving averages
plotOBOS <- function(symbol, n=50, type=c("sma", "ema", "zlema"),
                     years=1, blue=TRUE, current=TRUE, title=symbol,
                     ticks=TRUE, axes=TRUE)  
    today <- Sys.Date()
    if (class(symbol) == "character")  
        X <- getSymbols(symbol, from=format(today-365*years-2*n), auto.assign=FALSE)
        x <- X[,6]                          # use Adjusted
      else if (inherits(symbol, "zoo"))  
        x <- X <- as.xts(symbol)
        current <- FALSE                # don't expand the supplied data
     
    n <- min(nrow(x)/3, 50)             # as we may not have 50 days
    sub <- ""
    if (current)  
        xx <- getQuote(symbol)
        xt <- xts(xx$Last, order.by=as.Date(xx$ Trade Time ))
        colnames(xt) <- paste(symbol, "Adjusted", sep=".")
        x <- rbind(x, xt)
        sub <- paste("Last price: ", xx$Last, " at ",
                     format(as.POSIXct(xx$ Trade Time ), "%H:%M"), sep="")
     
    type <- match.arg(type)
    xd <- switch(type,                  # compute xd as the central location via selected MA smoother
                 sma = SMA(x,n),
                 ema = EMA(x,n),
                 zlema = ZLEMA(x,n))
    xv <- runSD(x, n)                   # compute xv as the rolling volatility
    strt <- paste(format(today-365*years), "::", sep="")
    x  <- x[strt]                       # subset plotting range using xts' nice functionality
    xd <- xd[strt]
    xv <- xv[strt]
    xyd <- xy.coords(.index(xd),xd[,1]) # xy coordinates for direct plot commands
    xyv <- xy.coords(.index(xv),xv[,1])
    n <- length(xyd$x)
    xx <- xyd$x[c(1,1:n,n:1)]           # for polygon(): from first point to last and back
    if (blue)  
        blues5 <- c("#EFF3FF", "#BDD7E7", "#6BAED6", "#3182BD", "#08519C") # cf brewer.pal(5, "Blues")
        fairlylight <<- rgb(189/255, 215/255, 231/255, alpha=0.625) # aka blues5[2]
        verylight <<- rgb(239/255, 243/255, 255/255, alpha=0.625) # aka blues5[1]
        dark <<- rgb(8/255, 81/255, 156/255, alpha=0.625) # aka blues5[5]
        ## buglet in xts 0.10-0 requires the <<- here
      else  
        fairlylight <<- rgb(204/255, 204/255, 204/255, alpha=0.5)  # two suitable grays, alpha-blending at 50%
        verylight <<- rgb(242/255, 242/255, 242/255, alpha=0.5)
        dark <<- 'black'
     
    plot(x, ylim=range(range(x, xd+2*xv, xd-2*xv, na.rm=TRUE)), main=title, sub=sub, 
         major.ticks=ticks, minor.ticks=ticks, axes=axes) # basic xts plot setup
    addPolygon(xts(cbind(xyd$y+xyv$y, xyd$y+2*xyv$y), order.by=index(x)), on=1, col=fairlylight)  # upper
    addPolygon(xts(cbind(xyd$y-xyv$y, xyd$y+1*xyv$y), order.by=index(x)), on=1, col=verylight)    # center
    addPolygon(xts(cbind(xyd$y-xyv$y, xyd$y-2*xyv$y), order.by=index(x)), on=1, col=fairlylight)  # lower
    lines(xd, lwd=2, col=fairlylight)   # central smooted location
    lines(x, lwd=3, col=dark)           # actual price, thicker
 
and the main change are the three calls to addPolygon. To illustrate, we call plotOBOS("SPY", years=2) with an updated plot of the ETF representing the SP500 over the last two years: Updated example chart of overbought/oversold levels from plotOBOS() function Comments and further enhancements welcome!

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

25 July 2017

Gunnar Wolf: Getting ready for DebConf17 in Montreal!


(image shamelessly copied from Noodles' Emptiness) This year I will only make it to DebConf, not to DebCamp. But, still, I am very very happy and excited as the travel date looms nearer! I have ordered some of the delicacies for the Cheese and Wine party, signed up for the public bicycle system of Montreal, and done a fair share of work with the Content Team; finally today we sent out the announcement for the schedule of talks. Of course, there are several issues yet to fix, and a lot of things to do before traveling... But, no doubt about this: It will be an intense week! Oh, one more thing while we are at it: The schedule as it was published today does not really look like we have organized stuff into tracks But we have! This will be soon fixed, adding some color-coding to make tracks clearer on the schedule. This year, I pushed for the Content Team to recover the notion of tracks as an organizative measure, and as something that delivers value to DebConf as a whole. Several months ago, I created a Wiki page for the DebConf tracks, asking interested people to sign up for them. We currently have the following tracks registered:
Blends
Andreas Tille
Debian Science
Michael Banck
Cloud and containers
Luca Filipozzi
Embedded
Pending
Systems administration, automation and orchestation
Pending
Security
Gunnar Wolf
We have two tracks still needing a track coordinator. Do note that most of the tasks mentioned by the Wiki have already been carried out; what a track coordinator will now do is to serve as some sort of moderator, maybe a recurring talkmeister, ensuring continuity and probably providing for some commentary, giving some unity to its sessions. So, the responsibilities for a track coordinator right now are quite similar to what is expected for video team volunteers but to a set of contiguous sessions. If you are interested in being the track coordinator/moderator for Embedded or for Systems administration, automation and orchestation or even to share the job with any of the other, registered, coordinators, please speak up! Mail content@debconf.org and update the table in the Wiki page. See you very soon in Montreal!

20 July 2017

Benjamin Mako Hill: Testing Our Theories About Eternal September

Graph of subscribers and moderators over time in /r/NoSleep. The image is taken from our 2016 CHI paper.
Last year at CHI 2016, my research group published a qualitative study examining the effects of a large influx of newcomers to the /r/nosleep online community in Reddit. Our study began with the observation that most research on sustained waves of newcomers focuses on the destructive effect of newcomers and frequently invokes Usenet s infamous Eternal September. Our qualitative study argued that the /r/nosleep community managed its surge of newcomers gracefully through strategic preparation by moderators, technological systems to reign in on norm violations, and a shared sense of protecting the community s immersive environment among participants. We are thrilled that, less a year after the publication of our study, Zhiyuan Jerry Lin and a group of researchers at Stanford have published a quantitative test of our study s findings! Lin analyzed 45 million comments and upvote patterns from 10 Reddit communities that a massive inundation of newcomers like the one we studied on /r/nosleep. Lin s group found that these communities retained their quality despite a slight dip in its initial growth period. Our team discussed doing a quantitative study like Lin s at some length and our paper ends with a lament that our findings merely reflected, propositions for testing in future work. Lin s study provides exactly such a test! Lin et al. s results suggest that our qualitative findings generalize and that sustained influx of newcomers need not doom a community to a descent into an Eternal September. Through strong moderation and the use of a voting system, the subreddits analyzed by Lin appear to retain their identities despite the surge of new users. There are always limits to research projects work quantitative and qualitative. We think the Lin s paper compliments ours beautifully, we are excited that Lin built on our work, and we re thrilled that our propositions seem to have held up! This blog post was written with Charlie Kiene. Our paper about /r/nosleep, written with Charlie Kiene and Andr s Monroy-Hern ndez, was published in the Proceedings of CHI 2016 and is released as open access. Lin s paper was published in the Proceedings of ICWSM 2017 and is also available online.

19 July 2017

Dirk Eddelbuettel: RcppAPT 0.0.4

A new version of RcppAPT -- our interface from R to the C++ library behind the awesome apt, apt-get, apt-cache, ... commands and their cache powering Debian, Ubuntu and the like -- arrived on CRAN yesterday. We added a few more functions in order to compute on the package graph. A concrete example is shown in this vignette which determines the (minimal) set of remaining Debian packages requiring a rebuild under R 3.4.* to update their .C() and .Fortran() registration code. It has been used for the binNMU request #868558. As we also added a NEWS file, its (complete) content covering all releases follows below.

Changes in version 0.0.4 (2017-07-16)
  • New function getDepends
  • New function reverseDepends
  • Added package registration code
  • Added usage examples in scripts directory
  • Added vignette, also in docs as rendered copy

Changes in version 0.0.3 (2016-12-07)
  • Added dumpPackages, showSrc

Changes in version 0.0.2 (2016-04-04)
  • Added reverseDepends, dumpPackages, showSrc

Changes in version 0.0.1 (2015-02-20)
  • Initial version with getPackages and hasPackages

A bit more information about the package is available here as well as as the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

18 July 2017

Reproducible builds folks: Reproducible Builds: week 116 in Stretch cycle

Here's what happened in the Reproducible Builds effort between Sunday July 9 and Saturday July 15 2017: Packages reviewed and fixed, and bugs filed Reviews of unreproducible packages 13 package reviews have been added, 12 have been updated and 19 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been added: 3 issue types have been updated: Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by: diffoscope development Version 84 was uploaded to unstable by Mattia Rizzolo. It included contributions already reported from the previous weeks, as well as new ones: After the release, development continued in git with contributions from: strip-nondeterminism development Versions 0.036-1, 0.037-1 and 0.038-1 were uploaded to unstable by Chris Lamb. They included contributions from: reprotest development Development continued in git with contributions from: buildinfo.debian.net development tests.reproducible-builds.org Misc. This week's edition was written by Bernhard M. Wiedemann, Chris Lamb, Mattia Rizzolo, Vagrant Cascadian & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

15 July 2017

Joey Hess: Functional Reactive Propellor

I wrote this code, and it made me super happy!
data Variety = Installer   Target
    deriving (Eq)
seed :: UserInput -> Versioned Variety Host
seed userinput ver = host "foo"
    & ver (   (== Installer) --> hostname "installer"
          < > (== Target)    --> hostname (inputHostname userinput)
          )
    & osDebian Unstable X86_64
    & Apt.stdSourcesList
    & Apt.installed ["linux-image-amd64"]
    & Grub.installed PC
    & XFCE.installed
    & ver (   (== Installer) --> desktopUser defaultUser
          < > (== Target)    --> desktopUser (inputUsername userinput)
          )
    & ver (   (== Installer) --> autostartInstaller )
This is doing so much in so little space and with so little fuss! It's completely defining two different versions of a Host. One version is the Installer, which in turn installs the Target. The code above provides all the information that propellor needs to convert a copy of the Installer into the Target, which it can do very efficiently. For example, it knows that the default user account should be deleted, and a new user account created based on the user's input of their name. The germ of this idea comes from a short presentation I made about propellor in Portland several years ago. I was describing RevertableProperty, and Joachim Breitner pointed out that to use it, the user essentially has to keep track of the evolution of their Host in their head. It would be better for propellor to know what past versions looked like, so it can know when a RevertableProperty needs to be reverted. I didn't see a way to address the objection for years. I was hung up on the problem that propellor's properties can't be compared for equality, because functions can't be compared for equality (generally). And on the problem that it would be hard for propellor to pull old versions of a Host out of git. But then I ran into the situation where I needed these two closely related hosts to be defined in a single file, and it all fell into place. The basic idea is that propellor first reverts all the revertible properties for other versions. Then it ensures the property for the current version. Another use for it would be if you wanted to be able to roll back changes to a Host. For example:
foos :: Versioned Int Host
foos ver = host "foo"
    & hostname "foo.example.com"
    & ver (   (== 1) --> Apache.modEnabled "mpm_worker"
          < > (>= 2) --> Apache.modEnabled "mpm_event"
          )
    & ver ( (>= 3)   --> Apt.unattendedUpgrades )
foo :: Host
foo = foos  version  (4 :: Int)
Versioned properties can also be defined:
foobar :: Versioned Int -> RevertableProperty DebianLike DebianLike
foobar ver =
    ver (   (== 1) --> (Apt.installed "foo" <!> Apt.removed "foo")
        < > (== 2) --> (Apt.installed "bar" <!> Apt.removed "bar")
        )
Notice that I've embedded a small DSL for versioning into the propellor config file syntax. While implementing versioning took all day, that part was super easy; Haskell config files win again! API documentation for this feature PS: Not really FRP, probably. But time-varying in a FRP-like way.
Development of this was sponsored by Jake Vosloo on Patreon.

Dirk Eddelbuettel: Rcpp 0.12.12: Rounding some corners

The twelveth update in the 0.12.* series of Rcpp landed on CRAN this morning, following two days of testing at CRAN preceded by five full reverse-depends checks we did (and which are always logged in this GitHub repo). The Debian package has been built and uploaded; Windows and macOS binaries should follow at CRAN as usual. This 0.12.12 release follows the 0.12.0 release from late July, the 0.12.1 release in September, the 0.12.2 release in November, the 0.12.3 release in January, the 0.12.4 release in March, the 0.12.5 release in May, the 0.12.6 release in July, the 0.12.7 release in September, the 0.12.8 release in November, the 0.12.9 release in January, the 0.12.10.release in March, and the 0.12.11.release in May making it the sixteenth release at the steady and predictable bi-montly release frequency. Rcpp has become the most popular way of enhancing GNU R with C or C++ code. As of today, 1097 packages (and hence 71 more since the last release in May) on CRAN depend on Rcpp for making analytical code go faster and further, along with another 91 in BioConductor. This releases contain a fairly large number of fairly small and focused pull requests most of which either correct some corner cases or improve other aspects. JJ tirelessly improved the package registration added in the previous release and following R 3.4.0. Kirill tidied up a number of small issues allowing us to run compilation in even more verbose modes---usually a good thing. Jeroen, Elias Pipping and Yo Gong all contributed as well, and we thank everybody for their contributions. All changes are listed below in some detail.

Changes in Rcpp version 0.12.12 (2017-07-13)
  • Changes in Rcpp API:
    • The tinyformat.h header now ends in a newline (#701).
    • Fixed rare protection error that occurred when fetching stack traces during the construction of an Rcpp exception (Kirill M ller in #706).
    • Compilation is now also possibly on Haiku-OS (Yo Gong in #708 addressing #707).
    • Dimension attributes are explicitly cast to int (Kirill M ller in #715).
    • Unused arguments are no longer declared (Kirill M ller in #716).
    • Visibility of exported functions is now supported via the R macro atttribute_visible (Jeroen Ooms in #720).
    • The no_init() constructor accepts R_xlen_t (Kirill M ller in #730).
    • Loop unrolling used R_xlen_t (Kirill M ller in #731).
    • Two unused-variables warnings are now avoided (Jeff Pollock in #732).
  • Changes in Rcpp Attributes:
    • Execute tools::package_native_routine_registration_skeleton within package rather than current working directory (JJ in #697).
    • The R portion no longer uses dir.exists to no require R 3.2.0 or newer (Elias Pipping in #698).
    • Fix native registration for exports with name attribute (JJ in #703 addressing #702).
    • Automatically register init functions for Rcpp Modules (JJ in #705 addressing #704).
    • Add Shield around parameters in Rcpp::interfaces (JJ in #713 addressing #712).
    • Replace dot (".") with underscore ("_") in package names when generating native routine registrations (JJ in #722 addressing #721).
    • Generate C++ native routines with underscore ("_") prefix to avoid exporting when standard exportPattern is used in NAMESPACE (JJ in #725 addressing #723).

Thanks to CRANberries, you can also look at a diff to the previous release. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

5 July 2017

Shirish Agarwal: Debian 9 release party at Co-hive

Dear all, This would be a biggish one so please have a chai/coffee or something stronger as it would take a while. I would start with attempt at some alcohol humor. While some people know that I have had a series of convulsive epileptic seizure I had shared bits about it in another post as well. Recovery is going through allopathic medicines as well as physiotherapy which I go to every alternate day. One of the exercises that I do in physiotherapy sessions is walk cross-legged on a line. While doing it today, it occurred to me that this is the same test that a Police inspector would do if they caught you drinking or are suspected of drunk driving. While some in the police force have now also have breath analyzer machines to determine alcohol content in the breath and body (and ways to deceive it are also there) the above exercise is still an integral part of examination. Now few of my friends who do drink have and had made expertise of walking on a line, while I due to this neurological disorder still have issues of walking on a line. So while I don t think of a drinking party in the near future (6 months at least), if I ever do get caught with a friend who is drunk (by association I would also be a suspect) by a policeman who doesn t have a breath analyzer machine, I could be in a lot of trouble. In addition if I tell him I have a neurological disorder I am bound to land up in a cell as he will think I m trying to make a fool of him. If you are able to picturize the situation, I m sure you will get a couple of laughs. Now coming to the release party, I was a bit apprehensive. It s been quite a while I had faced an audience and just coming out of illness didn t know how well or ill-prepared I would be for the session. I had forsaken/given up exercising two days earlier before the event as I wanted to have loose body, loose limbs all over. I also took a mild sedative (1mg) the day before just so I will have a fit night sleep and be able to focus all my energies on the big day. (I don t recommend sedatives unless the doctor prescribes) and I did have a doctor prescription so was able to have a nice sleep. I didn t do any Debian study as I hoped my somewhat long experience with both Ubuntu and Debian should help me. On the d-day, I had asked dhanesh (the organizer of the event) to accompany me from home to venue and back as I was unsure of the journey as it was around 9-10 kms. from my place and while I had been to the venue about couple of years back, I had just a mild remembrance of the place. Anyways, Dhanesh compiled with my request and together we reached the venue before the appointed 1500 hrs. As it was a Sunday I was unsure as how many people would turn up as people usually like to cozy up on a Sunday. Around 1530 hrs everybody showed up The whole group It included couple of co-organizers with most people being newbies so while I had thought of showing how to contribute via reporting bugs or putting up patches, had to set that aside and explain how things work in free software and open-source world. We didn t get into the debate of free vs open-source or free/open-source/open-core as that would have been counter-productive and probably confusing for newbies. We did however share the debian tree structure debian-tree-structure-discussion I was stumped by /var and /proc . I hadn t taken my lappy as it is expensive (a lenovo thinkpad I love very dearly) and I was unsure if I would be able to take care of it (weight wise). Dhanesh had told me that he had Debian on his lappy + zsh both of which are my favourites. While back at home I realized /var has been relegated to having apache/server logs and stuff like that, I do recall (vaguely) a thread on debian-devel about removing /var although that discussion went nowhere. One of the bugs that we hit early on is that nobody had enough space on their hdd to have Debian comfortably. It took a while to get an external hdd and push some of the content from somebody s lappy to the external drive to have space for the installation. I did share the /. /home, optional swap while Dhanesh helped by sharing about having a separate /boot partition as well which I had forgotten. I can t even begin to remember the number of times having a separate /boot partition has helped me in all of my systems. That done, we did try to install/show Debian 9 without network but were hit with #866629 so wasn t able to complete the installation. We had got the latest 9.0.1 as I had seen Steve s message about issues with the live images but even then we were hit with the above bug. As shared in the bug history, it might be a good idea to have the last couple of RC s (Release Candidate releases) as pre-release parties so people have a chance to report bugs and get them fixed. It was also nice to see Praveen raising the seriousness of the bug shared above. The next day I also filed #866971 as I had mistaken the release to be a regular release and not the live instance. I have pushed my rationale and hope the right thing happens. As installation takes a bit of time, we used the time to share about Google s Summer of Code and absence of Debian from GSoC this year. I should have directed them to an itfoss article I wrote sometime ago and also shared that Debian is also looking to having a similar apprenticeship within Debian itself. There were questions about why Debian would like to take the administrative overhead, my response was that it probably had to do with Debian wanting more control over the process. While Debian has had some great luck getting all number of seats that it asks for in GSoC, the ball is always in Google s court. Having that uncertainty off would be beneficial to Debian both in short-term as well as long-term. One interesting stat that was shared with me was that something akin to 89 students from India had been selected this year to GSoC even with the lower stipend and that is a far cry from the 10-15 students who are able to complete GSoC every year. Let s see what happens this year. One of the interesting fact/gossip I shared with them is that Google uses a modified/forked Debian internally which it probably would never release ever. There were quite a few queries about GSoC which resulted into how contributions are made and how git had become the master of all VCS (Version Control Systems). While I do have pet bugs about git (the biggest one being for places/countries having not big bandwidth git fails many a times while cloning). I *think* the bug has been reported enough times but haven t seen any improvements yet. There is a need of a solution like wget and wget -c so git just works even under the most trying circumstances (bandwidth wise) We also shared what a repo is and Dhanesh helpfully did a git shortlog to show people how commits are commented. It was from his official work so he couldn t show anything apart from the shortlog. I also shared how non-technical people can help with regard to documentation, artwork but didn t get into any concrete examples although wiki.debian.org would have been a good start. I also don t know if it s a fact or not but it seems/seemed that moinmoin (the current wiki solution used by debian) seems to have got sectional edit feature which I used sometime back. If moinmoin has got this feature then it is on par with mediawiki, although do know that mediawiki has lot more features going for it. Dhanesh did manage to install a Debian 8.0.7 (while 8.8.0 was the last release) which might have been better. The debian-installer (d-i) looks the same even though I know there are improvements with respect to UEFI and many updated components. There are and were many bugs which I wanted to share but didn t know if it was the right forum or not, for e.g. #597176 which probably needs improvements in other libraries along with the JPEG 2000 support #604859 all of which I m subscribed to. We also had a talk about code documentation and code readability, python (as almost everything in Debian is based on python) I had actually wanted to show metrics.debian.net but had seen it was down the day before and again checked to see it is down now as well, hence reported it, will hopefully come on Debian BTS some time soonish. The idea was to share that Debian does have/uses other programming languages as well and is only limited by people interested in a specific language and be willing to package and maintain packages in that specific programming language. As Dhanesh was part of Firefox OS and a Campus Ambassador we did discuss what went wrong in Firefox OS deployment and Firefox as a whole, specifically between the community and Mozilla the Corporation. In this there were lots that was lot that I wasn t able to share as had become tired otherwise would have shared that ncurses debian-installer interface although bit ugly to look at is still good as it has speech for visually differently abled people as well as people with poor sight. There is and was lots to share about Debian but was conscious that I might over-burden the audience and my stamina was also stretched. We also shared about automated builds bud didn t show either travis.debian.net or jeankins.debian.net We also had a small discussion about Artificial Intelligence, Robotics and how the outlook for people coming in Computer Science looks today. One thing I forgot to share we did do the cake-cutting together Dhanesh-Shirish cutting cake together Dhanesh is on the left while I m on the right. We did have a cute 3-4 year old boy as our guest as well bud didn t get good pictures of him. Lastly, the Debian cake itself Debian 9 cake We also talked about the kernel and subsystem maintainers and how Linus is the overlord. Look forward to comments. I am hoping Dhanesh might recollect some points that I might have missed/forgotten. Update 07/07/17 All programming languages stats. can be seen here
Filed under: Miscellenous Tagged: #alcohol humor, #co-hive, #crystal-gazing, #Debian bugs, #Debian cake, #Debian-release-party-9-pune, #live-installer, #moinmoin, #planet-debian, debian-installer, Pune

Next.

Previous.