Search Results: "nate"

23 October 2014

Dirk Eddelbuettel: Introducing Rocker: Docker for R

You only know two things about Docker. First, it uses Linux
containers. Second, the Internet won't shut up about it. -- attributed to Solomon Hykes, Docker CEO

So what is Docker? Docker is a relatively new open source application and service, which is seeing interest across a number of areas. It uses recent Linux kernel features (containers, namespaces) to shield processes. While its use (superficially) resembles that of virtual machines, it is much more lightweight as it operates at the level of a single process (rather than an emulation of an entire OS layer). This also allows it to start almost instantly, require very little resources and hence permits an order of magnitude more deployments per host than a virtual machine. Docker offers a standard interface to creation, distribution and deployment. The shipping container analogy is apt: just how shipping containers (via their standard size and "interface") allow global trade to prosper, Docker is aiming for nothing less for deployment. A Dockerfile provides a concise, extensible, and executable description of the computational environment. Docker software then builds a Docker image from the Dockerfile. Docker images are analogous to virtual machine images, but smaller and built in discrete, extensible and reuseable layers. Images can be distributed and run on any machine that has Docker software installed---including Windows, OS X and of course Linux. Running instances are called Docker containers. A single machine can run hundreds of such containers, including multiple containers running the same image. There are many good tutorials and introductory materials on Docker on the web. The official online tutorial is a good place to start; this post can not go into more detail in order to remain short and introductory.

So what is Rocker? rocker logo At its core, Rocker is a project for running R using Docker containers. We provide a collection of Dockerfiles and pre-built Docker images that can be used and extended for many purposes. Rocker is the the name of our GitHub repository contained with the Rocker-Org GitHub organization. Rocker is also the name the account under which the automated builds at Docker provide containers ready for download.

Current Rocker Status

Core Rocker Containers The Rocker project develops the following containers in the core Rocker repository
  • r-base provides a base R container to build from
  • r-devel provides the basic R container, as well as a complete R-devel build based on current SVN sources of R
  • rstudio provides the base R container as well an RStudio Server instance
We have settled on these three core images after earlier work in repositories such as docker-debian-r and docker-ubuntu-r.

Rocker Use Case Containers Within the Rocker-org organization on GitHub, we are also working on
  • Hadleyverse which extends the rstudio container with a number of Hadley packages
  • rOpenSci which extends hadleyverse with a number of rOpenSci packages
  • r-devel-san provides an R-devel build for "Sanitizer" run-time diagnostics via a properly instrumented version of R-devel via a recent compiler build
  • rocker-versioned aims to provided containers with 'versioned' previous R releases and matching packages
Other repositories will probably be added as new needs and opportunities are identified.

Deprecation The Rocker effort supersedes and replaces earlier work by Dirk (in the docker-debian-r and docker-ubuntu-r GitHub repositories) and Carl. Please use the Rocker GitHub repo and Rocker Containers from Docker.com going forward.

Next Steps We intend to follow-up with more posts detailing usage of both the source Dockerfiles and binary containers on different platforms. Rocker containers are fully functional. We invite you to take them for a spin. Bug reports, comments, and suggestions are welcome; we suggest you use the GitHub issue tracker.

Acknowledgments We are very appreciative of all comments received by early adopters and testers. We also would like to thank RStudio for allowing us the redistribution of their RStudio Server binary. Published concurrently at rOpenSci blog and Dirk's blog.

Authors Dirk Eddelbuettel and Carl Boettiger

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Erich Schubert: Clustering 23 mio Tweet locations

To test scalability of ELKI, I've clustered 23 million Tweet locations from the Twitter Statuses Sample API obtained over 8.5 months (due to licensing restrictions by Twitter, I cannot make this data available to you, sorry.
23 million points is a challenge for advanced algorithms. It's quite feasible by k-means; in particular if you choose a small k and limit the number of iterations. But k-means does not make a whole lot of sense on this data set - it is a forced quantization algorithm, but does not discover actual hotspots.
Density-based clustering such as DBSCAN and OPTICS are much more appropriate. DBSCAN is a bit tricky to parameterize - you need to find the right combination of radius and density for the whole world. Given that Twitter adoption and usage is quite different it is very likely that you won't find a single parameter that is appropriate everywhere.
OPTICS is much nicer here. We only need to specify a minimum object count - I chose 1000, as this is a fairly large data set. For performance reasons (and this is where ELKI really shines) I chose a bulk-loaded R*-tree index for acceleration. To benefit from the index, the epsilon radius of OPTICS was set to 5000m. Also, ELKI allows using geodetic distance, so I can specify this value in meters and do not get much artifacts from coordinate projection.
To extract clusters from OPTICS, I used the Xi method, with xi set to 0.01 - a rather low value, also due to the fact of having a large data set.
The results are pretty neat - here is a screenshot (using KDE Marble and OpenStreetMap data, since Google Earth segfaults for me right now):
Screenshot of Clusters in central Europe
Some observations: unsurprisingly, many cities turn up as clusters. Also regional differences are apparent as seen in the screenshot: plenty of Twitter clusters in England, and low acceptance rate in Germany (Germans do seem to have objections about using Twitter; maybe they still prefer texting, which was quite big in Germany - France and Spain uses Twitter a lot more than Germany).
Spam - some of the high usage in Turkey and Indonesia may be due to spammers using a lot of bots there. There also is a spam cluster in the ocean south of Lagos - some spammer uses random coordinates [0;1]; there are 36000 tweets there, so this is a valid cluster...
A benefit of OPTICS and DBSCAN is that they do not cluster every object - low density areas are considered as noise. Also, they support clusters of different shape (which may be lost in this visualiation, which uses convex hulls!) and different size. OPTICS can also produce a hierarchical result.
Note that for these experiments, the actual Tweet text was not used. This has a rough correspondence to Twitter popularity "heatmaps", except that the clustering algorithms will actually provide a formalized data representation of activity hotspots, not only a visualization.
You can also explore the clustering result in your browser - the Google Drive visualization functionality seems to work much better than Google Earth.
If you go to Istanbul or Los Angeles, you will see some artifacts - odd shaped clusters with a clearly visible spike. This is caused by the Xi extraction of clusters, which is far from perfect. At the end of a valley in the OPTICS plot, it is hard to decide whether a point should be included or not. These errors are usually the last element in such a valley, and should be removed via postprocessing. But our OpticsXi implementation is meant to be as close as possible to the published method, so we do not intend to "fix" this.
Certain areas - such as Washington, DC, New York City, and the silicon valley - do not show up as clusters. The reason is probably again the Xi extraction - these region do not exhibit the steep density increase expected by Xi, but are too blurred in their surroundings to be a cluster.
Hierarchical results can be found e.g. in Brasilia and Los Angeles.
Compare the OPTICS results above to k-means results (below) - see why I consider k-means results to be a meaningless quantization?
k-means clusters
Sure, k-means is fast (30 iterations; not converged yet. Took 138 minutes on a single core, with k=1000. The parallel k-means implementation in ELKI took 38 minutes on a single node, Hadoop/Mahout on 8 nodes took 131 minutes, as slow as a single CPU core!). But you can see how sensitive it is to misplaced coordinates (outliers, but mostly spam), how many "clusters" are somewhere in the ocean, and that there is no resolution on the cities? The UK is covered by 4 clusters, with little meaning; and three of these clusters stretch all the way into Bretagne - k-means clusters clearly aren't of high quality here.
If you want to reproduce these results, you need to get the upcoming ELKI version (0.6.5~201410xx - the output of cluster convex hulls was just recently added to the default codebase), and of course data. The settings I used are:
-dbc.in coords.tsv.gz
-db.index tree.spatial.rstarvariants.rstar.RStarTreeFactory
-pagefile.pagesize 500
-spatial.bulkstrategy SortTileRecursiveBulkSplit
-time
-algorithm clustering.optics.OPTICSXi
-opticsxi.xi 0.01
-algorithm.distancefunction geo.LngLatDistanceFunction
-optics.epsilon 5000.0 -optics.minpts 1000
-resulthandler KMLOutputHandler -out /tmp/out.kmz
and the total runtime for 23 million points on a single core was about 29 hours. The indexes helped a lot: less than 10000 distances were computed per point, instead of 23 million - the expected speedup over a non-indexed approach is 2400.
Don't try this with R or Matlab. Your average R clustering algorithm will try to build a full distance matrix, and you probably don't have an exabyte of memory to store this matrix. Maybe start with a smaller data set first, then see how long you can afford to increase the data size.

19 October 2014

Dirk Eddelbuettel: littler 0.2.1

max-heap image A new maintenance release of littler is available now. The main change are a few updates and extensions to the examples provided along with littler. Several of those continue to make use of the wonderful docopt package by Edwin de Jonge. Carl Boettiger and I are making good use of these littler examples, particularly to install directly from CRAN or GitHub, in our Rocker builds of R for Docker (about which we should have a bit more to blog soon too). Full details for the littler release are provided as usual at the ChangeLog page. The code is available via the GitHub repo, from tarballs off my littler page and the local directory here. A fresh package has gone to the incoming queue at Debian; Michael Rutter will probably have new Ubuntu binaries at CRAN in a few days too. Comments and suggestions are welcome via the mailing list or issue tracker at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Benjamin Mako Hill: Another Round of Community Data Science Workshops in Seattle

Pictures from the CDSW sessions in Spring 2014Pictures from the CDSW sessions in Spring 2014
I am helping coordinate three and a half day-long workshops in November for anyone interested in learning how to use programming and data science tools to ask and answer questions about online communities like Wikipedia, free and open source software, Twitter, civic media, etc. This will be a new and improved version of the workshops run successfully earlier this year. The workshops are for people with no previous programming experience and will be free of charge and open to anyone. Our goal is that, after the three workshops, participants will be able to use data to produce numbers, hypothesis tests, tables, and graphical visualizations to answer questions like: If you are interested in participating, fill out our registration form here before October 30th. We were heavily oversubscribed last time so registering may help. If you already know how to program in Python, it would be really awesome if you would volunteer as a mentor! Being a mentor will involve working with participants and talking them through the challenges they encounter in programming. No special preparation is required. If you re interested, send me an email.

18 October 2014

Erich Schubert: Beware of trolls - do not feed

A particularly annoying troll has been on his hate crusade against systemd for months now.
Unfortunately, he's particularly active on Debian mailing lists (but apparently also on Ubuntu and the Linux Kernel mailing list) and uses a tons of fake users he keeps on setting up. Our listmasters have a hard time blocking all his hate, sorry.
Obviously, this is also the same troll that has been attacking Lennart Poettering.
There is evidence that this troll used to go by the name "MikeeUSA", and has quite a reputation with anti-feminist hate for over 10 years now.
Please, do not feed this troll.
Here are some names he uses on YouTube: Gregory Smith, Matthew Bradshaw, Steve Stone.
Blacklisting is the best measure we have, unfortunately.
Even if you don't like the road systemd is taking or Lennart Poetting personall - the behaviour of that troll is unacceptable to say the least; and indicates some major psychological problems... also, I wouldn't be surprised if he is also involved in #GamerGate.
See this example (LKML) if you have any doubts. We seriously must not tolerate such poisonous people.
If you don't like systemd, the acceptable way of fighting it is to write good alternative software (and you should be able to continue using SysV init or openRC, unless there is a bug, in Debian - in this case, provide a bug fix). End of story.

Rhonda D'Vine: Trans Gender Moves

Yesterday I managed to get the last ticket from the waitinglist for the premiere of Trans Gender Moves. It is a play about the lives of three people: A transman, a transwoman and an intersexual person. They tell stories from their life, their process of finding their own identity over time. With in parts amusing anecdotes and ones that gets you thinking I can just wholeheartly encourage you to watch it if you have the chance to. It will still be shown the next few days, potentially extending depending on the requests for tickets, from what I've been told by one of the actors. The most funny moment for me though was when I was talking with one of the actors about that it really touched me that I was told that one of them will be moving into into the same building I will be moving into in two year's time. Unfortunately that will be delayed a bit because they found me thinks field hamster or the likes in the ground and have to wait until spring for them to move. :/

/personal permanent link Comments: 5 Flattr this

17 October 2014

Erich Schubert: Google Earth on Linux

Google Earth for Linux appears to be largely abandoned by Google, unfortunately. The packages available for download cannot be installed on a modern amd64 Debian or Ubuntu system due to dependency issues.
In fact, the adm64 version is a 32 bit build, too. The packages are really low quality, the dependencies are outdated, locales support is busted etc.
So here are hacky instructions how to install nevertheless. But beware, these instructions are a really bad hack.
  1. These instructions are appropriate for version 7.1.2.2041-r0. Do not use them for any other version. Things will have changed.
  2. Make sure your system has i386 architecture enabled. Follow the instructions in section "Configuring architectures" on the Debian MultiArch Wiki page to do so
  3. Install lsb-core, and try to install the i386 versions of these packages, too!
  4. Download the i386 version of the Google Earth package
  5. Install the package by forcing dependencies, via
    sudo dpkg --force-depends -i google-earth-stable_current_i386.deb
    
  6. As of now, your package manager will complain, and suggest to remove the package again. To make it happy, we have to hack the installed packages list. This is ugly, and you should make a backup. You can totally bust your system this way... Fortunately, the change we're doing is rather simple. As admin, edit the file /var/lib/dpkg/status. Locate the section Package: google-earth-stable. In this section, delete the line starting with Depends:. Don't add in extra newlines or change anything else!
  7. Now the package manager should believe the dependencies of Google Earth are fulfilled, and no longer suggest removal. But essentially this means you have to take care of them yourself!
Some notes on using Google Earth:

Petter Reinholdtsen: Debian Jessie, PXE and automatic firmware installation

When PXE installing laptops with Debian, I often run into the problem that the WiFi card require some firmware to work properly. And it has been a pain to fix this using preseeding in Debian. Normally something more is needed. But thanks to my isenkram package and its recent tasksel extension, it has now become easy to do this using simple preseeding. The isenkram-cli package provide tasksel tasks which will install firmware for the hardware found in the machine (actually, requested by the kernel modules for the hardware). (It can also install user space programs supporting the hardware detected, but that is not the focus of this story.) To get this working in the default installation, two preeseding values are needed. First, the isenkram-cli package must be installed into the target chroot (aka the hard drive) before tasksel is executed in the pkgsel step of the debian-installer system. This is done by preseeding the base-installer/includes debconf value to include the isenkram-cli package. The package name is next passed to debootstrap for installation. With the isenkram-cli package in place, tasksel will automatically use the isenkram tasks to detect hardware specific packages for the machine being installed and install them, because isenkram-cli contain tasksel tasks. Second, one need to enable the non-free APT repository, because most firmware unfortunately is non-free. This is done by preseeding the apt-mirror-setup step. This is unfortunate, but for a lot of hardware it is the only option in Debian. The end result is two lines needed in your preseeding file to get firmware installed automatically by the installer:
base-installer base-installer/includes string isenkram-cli
apt-mirror-setup apt-setup/non-free boolean true
The current version of isenkram-cli in testing/jessie will install both firmware and user space packages when using this method. It also do not work well, so use version 0.15 or later. Installing both firmware and user space packages might give you a bit more than you want, so I decided to split the tasksel task in two, one for firmware and one for user space programs. The firmware task is enabled by default, while the one for user space programs is not. This split is implemented in the package currently in unstable. If you decide to give this a go, please let me know (via email) how this recipe work for you. :) So, I bet you are wondering, how can this work. First and foremost, it work because tasksel is modular, and driven by whatever files it find in /usr/lib/tasksel/ and /usr/share/tasksel/. So the isenkram-cli package place two files for tasksel to find. First there is the task description file (/usr/share/tasksel/descs/isenkram.desc):
Task: isenkram-packages
Section: hardware
Description: Hardware specific packages (autodetected by isenkram)
 Based on the detected hardware various hardware specific packages are
 proposed.
Test-new-install: show show
Relevance: 8
Packages: for-current-hardware
Task: isenkram-firmware
Section: hardware
Description: Hardware specific firmware packages (autodetected by isenkram)
 Based on the detected hardware various hardware specific firmware
 packages are proposed.
Test-new-install: mark show
Relevance: 8
Packages: for-current-hardware-firmware
The key parts are Test-new-install which indicate how the task should be handled and the Packages line referencing to a script in /usr/lib/tasksel/packages/. The scripts use other scripts to get a list of packages to install. The for-current-hardware-firmware script look like this to list relevant firmware for the machine:
#!/bin/sh
#
PATH=/usr/sbin:$PATH
export PATH
isenkram-autoinstall-firmware -l
With those two pieces in place, the firmware is installed by tasksel during the normal d-i run. :) If you want to test what tasksel will install when isenkram-cli is installed, run DEBIAN_PRIORITY=critical tasksel --test --new-install to get the list of packages that tasksel would install. Debian Edu will be pilots in testing this feature, as isenkram is used there now to install firmware, replacing the earlier scripts.

16 October 2014

Bits from Debian: Help empower the Debian Outreach Program for Women

Debian is thrilled to participate in the 9th round of the GNOME FOSS Outreach Program. While OPW is similar to Google Summer of Code it has a winter session in addition to a summer session and is open to non-students. Back at DebConf 14 several of us decided to volunteer because we want to increase diversity in Debian. Shortly thereafter the DPL announced Debian's participation in OPW 2014. We have reached out to several corporate sponsors and are thrilled that so far Intel has agreed to fund an intern slot (in addition to the slot offered by the DPL)! While that makes two funded slots we have a third sponsor that has offered a challenge match: for each dollar donated by an individual to Debian the sponsor will donate another dollar for Debian OPW. This is where we need your help! If we can raise $3,125 by October 22 that means we can mentor a third intern ($6,250). Please spread the word and donate today if you can at: http://debian.ch/opw2014/ If you'd like to participate as intern, the application deadline is the same (October 22nd). You can find out more on the Debian Wiki.

13 October 2014

Steve McIntyre: Successful Summer of Code in Linaro

It's past time I wrote about how Linaro's students fared in this year's Google Summer of Code. You might remember me posting earlier in the year when we welcomed our students. We started with 3 student projects at the beginning of the summer. One of the students unfortunately didn't work out, but the other two were hugely successful. Gaurav Minocha was a graduate student at the University of British Columbia, Vancouver, Canada. He worked on Linux Flattened Device Tree Self-checking, mentored by Grant Likely from Linaro's Office of the CTO. Gaurav achieved all of his project's goals, and he was invited to Linaro's recent Linaro Connect USAConnect conference in California to meet people and and talk about his project. He and Grant presented a session on their work; it was filmed, and video is online. Grant said he was very happy with Gaurav's "strong, solid performance" during the project. Varad Gautam was a student at Birla Institute of Technology and Science, Pilani, India. He succeeded in porting UEFI to the BeagleBone Black. Leif Lindholm from the Linaro Enterprise Group was his mentor for the summer. At the end of the summer, Varad delivered a UEFI port ready for booting Linux and his code was included in Linaro's September UEFI release. Leif said that he was "very pleased with Varad's self sufficiency and ability to pick up an entirely new software project very quickly". We were hoping to invite Gaurad to Connect in California also, but travel document delays got in the way. With luck we'll see him at the next Connect in Hong Kong in February 2015. Well done, guys! It was great to work with these young developers for the summer, and we wish them lots more success in their future endeavours. Google have also just confirmed that they will be running the Summer of Code program again in 2015. I'm hoping that Linaro will be accepted again next year as a mentoring organisation. I'll post more about that early next year.

Dirk Eddelbuettel: Seinfeld streak at GitHub

Early last year, I referred to a Seinfeld Streak in a blog post referring to almost two months of updates to the Rcpp Gallery. This is sometimes called Jerry Seinfeld's secret to productivity: Just keep at it. Don't break the streak. I now have different streak:
github activity october 2013 to october 2014
Now we'll see how far this one will go.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

12 October 2014

Iustin Pop: Day trip on the Olympic Peninsula

Day trip on the Olympic Peninsula TL;DR: drove many kilometres on very nice roads, took lots of pictures, saw sunshine and fog and clouds, an angry ocean and a calm one, a quiet lake and lots and lots of trees: a very well spent day. Pictures at http://photos.k1024.org/Daytrips/Olympic-Peninsula-2014/. Sometimes I travel to the US on business, and as such I've been a few times in the Seattle area. Until this summer, when I had my last trip there, I was content to spend any extra days (weekend or such) just visiting Seattle itself, or shopping (I can spend hours in the REI store!), or working on my laptop in the hotel. This summer though, I thought - I should do something a bit different. Not too much, but still - no sense in wasting both days of the weekend. So I thought maybe driving to Mount Rainier, or something like that. On the Wednesday of my first week in Kirkland, as I was preparing my drive to the mountain, I made the mistake of scrolling the map westwards, and I saw for the first time the Olympic Peninsula; furthermore, I was zoomed in enough that I saw there was a small road right up to the north-west corner. Intrigued, I zoomed further and learned about Cape Flattery ( the northwestern-most point of the contiguous United States! ), so after spending a bit time reading about it, I was determined to go there. Easier said than done - from Kirkland, it's a 4h 40m drive (according to Google Maps), so it would be a full day on the road. I was thinking of maybe spending the night somewhere on the peninsula then, in order to actually explore the area a bit, but from Wednesday to Saturday it was a too short notice - all hotels that seemed OK-ish were fully booked. I spent some time trying to find something, even not directly on my way, but I failed to find any room. What I did manage to do though, is to learn a bit about the area, and to realise that there's a nice loop around the whole peninsula - the 104 from Kirkland up to where it meets the 101N on the eastern side, then take the 101 all the way to Port Angeles, Lake Crescent, near Lake Pleasant, then south toward Forks, crossing the Hoh river, down to Ruby Beach, down along the coast, crossing the Queets River, east toward Lake Quinault, south toward Aberdeen, then east towards Olympia and back out of the wilderness, into the highway network and back to Kirkland. This looked like an awesome road trip, but it is as long as it sounds - around 8 hours (continuous) drive, though skipping Cape Flattery. Well, I said to myself, something to keep in mind for a future trip to this area, with a night in between. I was still planning to go just to Cape Flattery and back, without realising at that point that this trip was actually longer (as you drive on smaller, lower-speed roads). Preparing my route, I read about the queues at the Edmonds-Kingston ferry, so I was planning to wake up early on the weekend, go to Cape Flattery, and go right back (maybe stop by Lake Crescent). Saturday comes, I - of course - sleep longer than my trip schedule said, and start the day in a somewhat cloudy weather, driving north from my hotel on Simonds Road, which was quite nicer than the usual East-West or North-South roads in this area. The weather was becoming nicer, however as I was nearing the ferry terminal and the traffic was getting denser, I started suspecting that I'll spend a quite a bit of time waiting to board the ferry. And unfortunately so it was (photo altered to hide some personal information): Waiting for the ferry. The weather at least was nice, so I tried to enjoy it and simply observe the crowd - people were looking forward to a weekend relaxing, so nobody seemed annoyed by the wait. After almost half an hour, time to get on the ferry - my first time on a ferry in US, yay! But it was quite the same as in Europe, just that the ship was much larger. Once I secured the car, I went up deck, and was very surprised to be treated with some excellent views: Harbour view Looking towards the sun   and away from it The crossing was not very short, but it seemed so, because of the view, the sun, the water and the wind. Soon we were nearing the other shore; also, see how well panorama software deals with waves :P! Near the other shore And I was finally on the "real" part of the trip. The road was quite interesting. Taking the 104 North, crossing the "Hood Canal Floating Bridge" (my, what a boring name), then finally joining the 101 North. The environment was quite varied, from bare plains and hills, to wooded areas, to quite dense forests, then into inhabited areas - quite a long stretch of human presence, from the Sequim Bay to Port Angeles. Port Angeles surprised me: it had nice views of the ocean, and an interesting port (a few big ships), but it was much smaller than I expected. The 101 crosses it, and in less than 10 minutes or so it was already over. I expected something nicer, based on the name, but Anyway, onwards! Soon I was at a crossroads and had to decide: I could either follow the 101, crossing the Elwha River and then to Lake Crescent, then go north on the 113/112, or go right off 101 onto 112, and follow it until close to my goal. I took the 112, because on the map it looked "nicer", and closer to the shore. Well, the road itself was nice, but quite narrow and twisty here and there, and there was some annoying traffic, so I didn't enjoy this segment very much. At least it had the very interesting property (to me) that whenever I got closer to the ocean, the sun suddenly disappeared, and I was finding myself in the fog: Foggy road So my plan to drive nicely along the coast failed. At one point, there was even heavy smoke (not fog!), and I wondered for a moment how safe was to drive out there in the wilderness (there were other cars though, so I was not alone). Only quite a bit later, close to Neah Bay, did I finally see the ocean: I saw a small parking spot, stopped, and crossing a small line of trees I found myself in a small cove? bay? In any case, I had the impression I stepped out of the daily life in the city and out into the far far wilderness: Dead trees on the beach Trees growing on a rock Small panorama of the cove There was a couple, sitting on chairs, just enjoying the view. I felt very much intruding, behaving like I did as a tourist: running in, taking pictures, etc., so I tried at least to be quiet . I then quickly moved on, since I still had some road ahead of me. Soon I entered Neah Bay, and was surprised to see once more blue, and even more blue. I'm a sucker for blue, whether sky blue or sea blue , so I took a few more pictures (watch out for the evil fog in the second one): View towards Neah Bay port Sea view from Neah Bay Well, the town had some event, and there were lots of people, so I just drove on, now on the last stretch towards the cape. The road here was also very interesting, yet another environment - I was driving on Cape Flattery Road, which cuts across the tip of the peninsula (quite narrow here) along the Waatch River and through its flooding plains (at least this is how it looked to me). Then it finally starts going up through the dense forest, until it reaches the parking lot, and from there, one goes on foot towards the cape. It's a very easy and nice walk (not a hike), and the sun was shining very nicely through the trees: Sunny forest Sun shinning down Wooden path But as I reached the peak of the walk, and started descending towards the coast, I was surprised, yet again, by fog: Ugly fog again! I realised that probably this means the cape is fully in fog, so I won't have any chance to enjoy the view. Boy, was I wrong! There are three viewpoints on the cape, and at each one I was just "wow" and "aah" at the view. Even thought it was not a sunny summer view, and there was no blue in sight, the combination between the fog (which was hiding the horizon and even the closer islands), the angry ocean which was throwing wave after wave at the shore, making a loud noise, and the fact that even this seemingly inhospitable area was just teeming with life, was both unexpected and awesome. I took here waay to many pictures, here are just a couple inlined: First view at the cape Birds 'enjoying' the weather Foggy shore I spent around half an hour here, just enjoying the rawness of nature. It was so amazing to see life encroaching on each bit of land, even though it was not what I would consider a nice place. Ah, how we see everything through our own eyes! The walk back was through fog again, and at one point it switched over back to sunny. Driving back on the same road was quite different, knowing what lies at its end. On this side, the road had some parking spots, so I managed to stop and take a picture - even though this area was much less wild, it still has that outdoors flavour, at least for me: Waatch River Back in Neah Bay, I stopped to eat. I had a place in mind from TripAdvisor, and indeed - I was able to get a custom order pizza at "Linda's Woodfired Kitchen". Quite good, and I ate without hurry, looking at the people walking outside, as they were coming back from the fair or event that was taking place. While eating, a somewhat disturbing thought was going through my mind. It was still early, around two to half past two, so if I went straight back to Kirkland I would be early at the hotel. But it was also early enough that I could - in theory at least - still do the "big round-trip". I was still rummaging the thought as I left On the drive back I passed once more near Sekiu, Washington, which is a very small place but the map tells me it even has an airport! Fun, and the view was quite nice (a bit of blue before the sea is swallowed by the fog): Sekiu view After passing Sekiu and Clallam Bay, the 112 curves inland and goes on a bit until you are at the crossroads: to the left the 112 continues, back the same way I came; to the right, it's the 113, going south until it meets the 101. I looked left - remembering the not-so-nice road back, I looked south - where a very appealing, early afternoon sun was beckoning - so I said, let's take the long way home! It's just a short stretch on the 113, and then you're on the 101. The 101 is a very nice road, wide enough, and it goes through very very nice areas. Here, west to south-west of the Olympic Mountains, it's a very different atmosphere from the 112/101 that I drove on in the morning; much warmer colours, a bit different tree types (I think), and more flat. I soon passed through Forks, which is one of the places I looked at when searching for hotels. I did so without any knowledge of the town itself (its wikipedia page is quite drab), so imagine my surprise when a month later I learned from a colleague that this is actually a very important place for vampire-book fans. Oh my, and I didn't even stop! This town also had some event, so I just drove on, enjoying the (mostly empty) road. My next planned waypoint was Ruby Beach, and I was looking forward to relaxing a bit under the warm sun - the drive was excellent, weather perfect, so I was watching the distance countdown on my Garmin. At two miles out, the "Near waypoint Ruby Beach" message appeared, and two seconds later the sun went out. What the I was hoping this is something temporary, but as I slowly drove the remaining mile I couldn't believe my eyes that I was, yet again, finding myself in the fog I park the car, thinking that asking for a refund would at least allow me to feel better - but it was I who planned the trip! So I resigned myself, thinking that possibly this beach is another special location that is always in the fog. However, getting near the beach it was clear that it was not so - some people were still in their bathing suits, just getting dressed, so it seems I was just unlucky with regards to timing. However, I the beach itself was nice, even in the fog (I later saw online sunny pictures, and it is quite beautiful), the the lush trees reach almost to the shore, and the way the rocks are sitting on the beach: A lonely dinghy Driftwood  and human construction People on the beach Since the weather was not that nice, I took a few more pictures, then headed back and started driving again. I was soo happy that the weather didn't clear at the 2 mile mark (it was not just Ruby Beach!), but alas - it cleared as soon as the 101 turns left and leaves the shore, as it crosses the Queets river. Driving towards my next planned stop was again a nice drive in the afternoon sun, so I think it simply was not a sunny day on the Pacific shore. Maybe seas and oceans have something to do with fog and clouds ! In Switzerland, I'm very happy when I see fog, since it's a somewhat rare event (and seeing mountains disappearing in the fog is nice, since it gives the impression of a wider space). After this day, I was a bit fed up with fog for a while Along the 101 one reaches Lake Quinault, which seemed pretty nice on the map, and driving a bit along the lake - a local symbol, the "World's largest spruce tree". I don't know what a spruce tree is, but I like trees, so I was planning to go there, weather allowing. And the weather did cooperate, except that the tree was not so imposing as I thought! In any case, I was glad to stretch my legs a bit: Path to largest spruce tree Largest spruce tree, far view Largest spruce tree, closer view Very short path back to the road However, the most interesting thing here in Quinault was not this tree, but rather - the quiet little town and the view on the lake, in the late afternoon sun: Quinault Quinault Lake view The entire town was very very quiet, and the sun shining down on the lake gave an even stronger sense of tranquillity. No wind, not many noises that tell of human presence, just a few, and an overall sense of peace. It was quite the opposite of the Cape Flattery and a very nice way to end the trip. Well, almost end - I still had a bit of driving ahead. Starting from Quinault, driving back and entering the 101, driving down to Aberdeen: Afternoon ride then turning east towards Olympia, and back onto the highways. As to Aberdeen and Olympia, I just drove through, so I couldn't make any impression of them. The old harbour and the rusted things in Aberdeen were a bit interesting, but the day was late so I didn't stop. And since the day shouldn't end without any surprises, during the last profile change between walking and driving in Quinault, my GPS decided to reset its active maps list and I ended up with all maps activated. This usually is not a problem, at least if you follow a pre-calculated route, but I did trigger recalculation as I restarted my driving, so the Montana was trying to decide on which map to route me - between the Garmin North America map and the Open StreeMap one, the result was that it never understood which road I was on. It always said "Drive to I5", even though I was on I5. Anyway, thanks to road signs, and no thanks to "just this evening ramp closures", I was able to arrive safely at my hotel. Overall, a very successful, if long trip: around 725 kilometres, 10h:30m moving, 13h:30m total: Track picture There were many individual good parts, but the overall think about this road trip was that I was able to experience lots of different environments of the peninsula on the same day, and that overall it's a very very nice area. The downside was that I was in a rush, without being able to actually stop and enjoy the locations I visited. And there's still so much to see! A two nights trip sound just about right, with some long hikes in the rain forest, and afternoons spent on a lake somewhere. Another not so optimal part was that I only had my "travel" camera (a Nikon 1 series camera, with a small sensor), which was a bit overwhelmed here and there by the situation. It was fortunate that the light was more or less good, but looking back at the pictures, how I wish that I had my "serious" DSLR So, that means I have two reasons to go back! Not too soon though, since Mount Rainier is also a good location to visit . If the pictures didn't bore you yet, the entire gallery is on my smugmug site. In any case, thanks for reading!

11 October 2014

Dirk Eddelbuettel: RPushbullet 0.1.0 with a lot more awesome

A new release 0.1.0 of the RPushbullet package (interfacing the neat Pushbullet service) landed on CRAN today. It brings a number of goodies relative to the first release 0.0.2 of a few months ago: There is a whole boat load of more wickedness in the Pushbullet API so if anybody feels compelled to add it, fire off pull requests at GitHub. More details about the package are at the RPushbullet webpage and the RPushbullet GitHub repo. Courtesy of CRANberries, there is also a diffstat report for this release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

10 October 2014

Matthias Klumpp: Listaller + Glick: Some new ideas

As you might know, due to invasive changes in PackageKit, I am currently rewriting the 3rd-party application installer Listaller. Since I am not the only one looking at the 3rd-party app-installation issue (there is a larger effort going on at GNOME, based on Lennarts ideas), it makes sense to redesign some concepts of Listaller. Currently, dependencies and applications are installed into directories in /opt, and Listaller contains some logic to make applications find dependencies, and to talk to the package manager to install missing things. This has some drawbacks, like the need to install an application before using it, the need for applications to be relocatable, and application-installations being non-atomic. Glick2 There is/was another 3rd-party app installer approach on the GNOME side, by Alexander Larsson, called Glick2. Glick uses application bundles (do you remember Klik from back in the days?) mounted via FUSE. This allows some neat features, like atomic installations and software upgrades, no need for relocatable apps and no need to install the application. However, it also has disadvantages. Quoting the introduction document for Glick2: Bundling isn t perfect, there are some well known disadvantages. Increased disk footprint is one, although current storage space size makes this not such a big issues. Another problem is with security (or bugfix) updates in bundled libraries. With bundled libraries its much harder to upgrade a single library, as you need to find and upgrade each app that uses it. Better tooling and upgrader support can lessen the impact of this, but not completely eliminate it. This is what Listaller does better, since it was designed to do a large effort to avoid duplication of code. Also, currently Glick doesn t have support for updates and software-repositories, which Listaller had. Combining Listaller and Glick ideas So, why not combine the ideas of Listaller and Glick? In order to have Glick share resources, the system needs to know which shared resources are available. This is not possible if there is one huge Glick bundle containing all of the application s dependencies. So I modularized Glick bundles to contain just one software component, which is e.g. GTK+ or Qt, GStreamer or could even be a larger framework (e.g. GNOME 3.14 Platform ). These components are identified using AppStream XML metadata, which allows them to be installed from the distributor s software repositories as well, if that is wanted. If you now want to deploy your application, you first create a Glick bundle for it. Then, in a second step, you bundle your application bundle with it s dependencies in one larger tarball, which can also be GPG signed and can contain additional metadata. The resulting metabundle will look like this: glick-libundle This doesn t look like we share resources yet, right? The dependencies are still bundled with the application requiring them. The trick lies in the installation step: While the application above can be executed right away without installing it, there will also be an option to install it. For the user, this will mean that the application shows up in GNOME-Shell s overview or KDEs Plasma launcher, gets properly registered with mimetypes and is if installed for all users available system-wide. Technically, this will mean that the application s main bundle is extracted and moved to a special location on the file system, so are the dependency-bundles. If bundles already exist, they will not be installed again, and the new application will simply use the existing software. Since the bundles contain information about their dependencies, the system is able to determine which software is needed and which can simply be deleted from the installation directories. If the application is started now, the bundles are combined and mounted, so the application can see the libraries it depends on. Additionally, this concept allows secure updates of applications and shared resources. The bundle metadata contains an URL which points to a bundle repository. If new versions are released, the system s auto-updater can automatically pick these up and install them this means e.g. the Qt bundle will receive security updates, even if the developer who shipped it with his/her app didn t think of updating it. Conclusion So far, no productive code exists for this I just have a proof-of-concept here. But I pretty much like the idea, and I am thinking about going further in that direction, since it allows deploying applications on the Linux desktop as well as deploying software on servers in a way which plays nice with the native package manager, and which does not duplicate much code (less risk of having not-updated libraries with security flaws around). However, there might be issues I haven t thought about yet. Also, it makes sense to look at GNOME to see how the whole 3rd-party app deployment issue develops. In case I go further with Listaller-NEXT, it is highly likely that it will make use of the ideas sketched above (comments and feedback are more than welcome!).

8 October 2014

Jonathan Dowland: Ansible

I've just recently built the large bulk of VMs that we use for first semester teaching. This year that was 112. We use the same general approach for these as our others: get a generic base image up and running, with just enough configuration complete so a puppet client starts up; get it talking to our master; let puppet take it from there. There are pragmatic balances between how much we do in the kickstart versus how much we do in puppet, but also when we build a new VM from scratch versus when we clone an existing image, and how specialisation we do in the clone image. Unfortunately this year we ended up in a situation where our clone image wouldn't talk to our puppet master out of the box, due to some changes we'd made to our master set up since the clone image was prepared. We didn't really have enough time to re-clone the entire set of VMs from a fixed base image, and instead needed to fix them whilst up. However we couldn't rely on puppet to do that, since they wouldn't talk to the puppet master. We needed to manually reset the puppet client state per VM and then re-establish a trust relationship with the correct master (which is not the default master hostname in our environment anymore). Luckily, we deploy a local account with a known passphrase via the kickstart, which also has sudo access, as an interim measure before puppet strips it back out again and sets up proper LDAP and Kerberos authentication. So we can at least get into the boxes. However logging into 112 VMs by hand is not a particularly pleasant task. In the past I might have tried to achieve this using something like clusterssh but this year I decided to give ansible a try instead. Ansible started life, I believe, as a tool that would let you run arbitrary commands on remote hosts, including navigating ssh and sudo as required, without needing any agent software on the remote end. It has since seemed to grow into an enterprise product in its own right, seemingly in competition with puppet, chef, cfengine et al. Looking at the Ansible website now I'd be rather put off by just how "enterprisey" it has become - much as I am by the puppet website, if I'm honest - but if you persevere past the webinars, testimonials, etc. etc., you can find yourself to the documentation, and running an arbitrary command is as simple as The hosts file format is simple
[somehosts]
host1
host2
...
[otherhosts]
host3
The command line can be a little bit more complex, especially if you need to use one username for ssh, another for sudo, and you don't want to use ssh key auth:
ansible -i ./hostsfile somehosts -k -u someuser \
    --sudo -K -a 'puppet agent --onetime --no-daemonize --verbose 
"all" would work where I've used somehosts in the example above. So there you go: using one configuration management system to bootstrap another. I'm sure I've reserved myself a special place in hell for this.

7 October 2014

Joachim Breitner: New website layout

After 10 years I finally got around to re-decorating my website. One reason was ICFP, where just too many people told me that I don t look like on my old website any more (which is very true). Another reason was that I was visting my brother, who is very good at web design (check out his portfolio), who could help me a bit. I wanted something practical and maybe a bit staid, so I drew inspiration from typical Latex typography, and also from Edward Z. Yang s blog: A serif font (Utopia) for the main body, justified and hyphenated text. Large section headers in a knobbly bold sans-serif font (Latin Modern Sans, which reasonably resembles Computer Modern). To intensify that impression, I put the main text on a white box that lies like a paper on the background. As a special gimmic the per-page navigation (or, in the case of the blog, the list of categories) is marked up like a figure in a paper. Of course this would be very dire without a suitable background. I really like the procedural art by Jared Tarbell, espcially substrate and interAggregate. Both have been turned into screensavers shipped with xscreensaver, so I hacked the substrate code to generate a seamless tile and took a screenshot of the result. I could not make up my mind yet how dense it has to be to look good, so I for every page I randomly pick one of six variants randomly for now. I simplified the navigation a bit. The old News section has been removed recently already. The Links section is gone I guess link lists on homepages are so 90s. The section Contact and About me are merged and awaiting some cleanup. The link to the satire news Heisse News is demoted to a mention on the Contents section. This hopefully helps to make the site navigatable on mobile devices (the old homepage was unusable). CSS media queries adjust the layout slightly on narrow screens, and separately for print devices. Being the nostaltic I am, I still keep the old design, as well as the two designs before that, around and commented their history.

Andrea Veri: The GNOME Infrastructure is now powered by FreeIPA!

As preannounced here the GNOME Infrastructure switched to a new Account Management System which is reachable at https://account.gnome.org. All the details will follow. Introduction It s been a while since someone actually touched the underlaying authentication infrastructure that powers the GNOME machines. The very first setup was originally configured by Jonathan Blandford (jrb) who configured an OpenLDAP istance with several customized schemas. (pServer fields in the old CVS days, pubAuthorizedKeys and GNOME modules related fields in recent times) While OpenLDAP-server was living on the GNOME machine called clipboard (aka ldap.gnome.org) the clients were configured to synchronize users, groups, passwords through the nslcd daemon. After several years Jeff Schroeder joined the Sysadmin Team and during one cold evening (date is Tue, February 1st 2011) spent some time configuring SSSD to replace the nslcd daemon which was missing one of the most important SSSD features: caching. What surely convinced Jeff to adopt SSSD (a very new but promising sofware at that time as the first release happened right before 2010 s Christmas) and as the commit log also states ( New sssd module for ldap information caching ) was SSSD s caching feature. It was enough for a certain user to log in once and the /var/lib/sss/db directory was populated with its login information preventing the LDAP daemon in charge of picking up login details (from the LDAP server) to query the LDAP server itself every single time a request was made against it. This feature has definitely helped in many occasions especially when the LDAP server was down for a particular reason and sysadmins needed to access a specific machine or service: without SSSD this wasn t ever going to work and sysadmins were probably going to be locked out from the machines they were used to manage. (except if you still had /etc/passwd , /etc/group and /etc/shadow entries as fallback) Things were working just fine except for a few downsides that appeared later on:
  1. the web interface (view) on our LDAP user database was managed by Mango, an outdated tool which many wanted to rewrite in Django that slowly became a huge dinosaur nobody ever wanted to look into again
  2. the Foundation membership information were managed through a MySQL database, so two databases, two sets of users unrelated to each other
  3. users were not able to modify their own account information on their own but even a single e-mail change required them to mail the GNOME Accounts Team which was then going to authenticate their request and finally update the account.
Today s infrastructure changes are here to finally say the issues outlined at (1, 2, 3) are now fixed. What has changed? The GNOME Infrastructure is now powered by Red Hat s FreeIPA which bundles several FOSS softwares into one big bundle all surrounded by an easy and intuitive web UI that will help users update their account information on their own without the need of the Accounts Team or any other administrative entity. Users will also find two custom fields on their Overview page, these being Foundation Member since and Last Renewed on date . As you may have understood already we finally managed to migrate the Foundation membership database into LDAP itself to store the information we want once and for all. As a side note it might be possible that some users that were Foundation members in the past won t find any detail stored on the Foundation fields outlined above. That is actually expected as we were able to migrate all the current and old Foundation members that had an LDAP account registered at the time of the migration. If that s your case and you still would like the information to be stored on the new setup please get in contact with the Membership Committee at stating so. Where can I get my first login credentials? Let s make a little distinction between users that previously had access to Mango (usually maintainers) and users that didn t. If you were used to access Mango before you should be able to login on the new Account Management System by entering your GNOME username and the password you were used to use for loggin in into Mango. (after loggin in the very first time you will be prompted to update your password, please choose a strong password as this account will be unique across all the GNOME Infrastructure) If you never had access to Mango, you lost your password or the first time you read the word Mango on this post you thought why is he talking about a fruit now? you should be able to reset it by using the following command:
ssh -l yourgnomeuserid account.gnome.org
The command will start an SSH connection between you and account.gnome.org, once authenticated (with the SSH key you previously had registered on our Infrastructure) you will trigger a command that will directly send your brand new password on the e-mail registered for your account. From my tests seems GMail sees the e-mail as a phishing attempt probably because the body contains the word password twice. That said if the e-mail won t appear on your INBOX, please double-check your Spam folder. Now that Mango is gone how can I request a new account? With Mango we used to have a form that automatically e-mailed the maintainer of the selected GNOME module which was then going to approve / reject the request. From there and in the case of a positive vote from the maintainer the Accounts Team was going to create the account itself. With the recent introduction of a commit robot directly on l10n.gnome.org the number of account requests reduced its numbers. In addition to that users will now be able to perform pretty much all the needed maintenance on their accounts themselves. That said and while we will probably work on building a form in the future we feel that requesting accounts can definitely be achieved directly by mailing the Accounts Team itself which will mail the maintainer of the respective module and create the account. As just said the number of account creations has become very low and the queue is currently clear. The documentation has been updated to reflect these changes at: https://wiki.gnome.org/AccountsTeam
https://wiki.gnome.org/AccountsTeam/NewAccounts The migration of all the user data and ACLs has been massive and I ve been spending a lot of time reviewing the existing HBAC rules trying to spot possible errors or misconfigurations. If you happen to not being able to access a certain service as you were used to in the past, please get in contact with the Sysadmin Team. All the possible ways to contact us are available at https://wiki.gnome.org/Sysadmin/Contact. What is missing still? Now that the Foundation membership information has been moved to LDAP I ll be looking at porting some of the existing membership scripts to it. What I managed to port already are welcome e-mails for new or existing members. (renewals) Next step will be generating a membership page from LDAP (to populate http://www.gnome.org/foundation/membership) and all the your-membership-is-going-to-lapse e-mails that were being sent till today. Other news /home/users mount on master.gnome.org You will notice that loggin in into master.gnome.org will result in your home directory being empty, don t worry, you did not lose any of your files but master.gnome.org is now currently hosting your home directories itself. As you may have been aware of adding files to the public_html directory on master resulted in them appearing on your people.gnome.org/~userid space. That was unfortunately expected as both master and webapps2 (the machine serving people.gnome.org s webspaces) were mounting the same GlusterFS share. We wanted to prevent that behaviour to happen as we wanted to know who has access to what resource and where. From today master s home directories will be there just as a temporary spot for your tarballs, just scp and use ftpadmin against them, that should be all you need from master. If you are interested in receiving or keeping using your people.gnome.org s webspace please mail stating so. Other news a shiny and new error 500 page has been deployed Thanks to Magdalen Berns (magpie) a new error 500 web page has been deployed on all the Apache istances we host. The page contains an iframe of status.gnome.org and will appear every single time the web server behind the service you are trying to reach will be unreachable for maintenance or other purposes. While I hope you won t see the page that often you can still enjoy it at https://static.gnome.org/error-500/500.html. Make sure to whitelist status.gnome.org on your browser as it currently loads it without https. (as the service is currently hosted on OpenShift which provides us with a *.rhcloud.com wildcard certificate, which differs from the CN the browser would expect it to be)

Andrea Veri: The GNOME Infrastructure is now powered by FreeIPA!

As preannounced here the GNOME Infrastructure switched to a new Account Management System which is reachable at https://account.gnome.org. All the details will follow. Introduction It s been a while since someone actually touched the underlying authentication infrastructure that powers the GNOME machines. The very first setup was originally configured by Jonathan Blandford (jrb) who configured an OpenLDAP istance with several customized schemas. (pServer fields in the old CVS days, pubAuthorizedKeys and GNOME modules related fields in recent times) While OpenLDAP-server was living on the GNOME machine called clipboard (aka ldap.gnome.org) the clients were configured to synchronize users, groups, passwords through the nslcd daemon. After several years Jeff Schroeder joined the Sysadmin Team and during one cold evening (date is Tue, February 1st 2011) spent some time configuring SSSD to replace the nslcd daemon which was missing one of the most important SSSD features: caching. What surely convinced Jeff to adopt SSSD (a very new but promising sofware at that time as the first release happened right before 2010 s Christmas) and as the commit log also states ( New sssd module for ldap information caching ) was SSSD s caching feature. It was enough for a certain user to log in once and the /var/lib/sss/db directory was populated with its login information preventing the LDAP daemon in charge of picking up login details (from the LDAP server) to query the LDAP server itself every single time a request was made against it. This feature has definitely helped in many occasions especially when the LDAP server was down for a particular reason and sysadmins needed to access a specific machine or service: without SSSD this wasn t ever going to work and sysadmins were probably going to be locked out from the machines they were used to manage. (except if you still had /etc/passwd , /etc/group and /etc/shadow entries as fallback) Things were working just fine except for a few downsides that appeared later on:
  1. the web interface (view) on our LDAP user database was managed by Mango, an outdated tool which many wanted to rewrite in Django that slowly became a huge dinosaur nobody ever wanted to look into again
  2. the Foundation membership information were managed through a MySQL database, so two databases, two sets of users unrelated to each other
  3. users were not able to modify their own account information on their own but even a single e-mail change required them to mail the GNOME Accounts Team which was then going to authenticate their request and finally update the account.
Today s infrastructure changes are here to finally say the issues outlined at (1, 2, 3) are now fixed. What has changed? The GNOME Infrastructure is now powered by Red Hat s FreeIPA which bundles several FOSS softwares into one big bundle all surrounded by an easy and intuitive web UI that will help users update their account information on their own without the need of the Accounts Team or any other administrative entity. Users will also find two custom fields on their Overview page, these being Foundation Member since and Last Renewed on date . As you may have understood already we finally managed to migrate the Foundation membership database into LDAP itself to store the information we want once and for all. As a side note it might be possible that some users that were Foundation members in the past won t find any detail stored on the Foundation fields outlined above. That is actually expected as we were able to migrate all the current and old Foundation members that had an LDAP account registered at the time of the migration. If that s your case and you still would like the information to be stored on the new setup please get in contact with the Membership Committee at stating so. Where can I get my first login credentials? Let s make a little distinction between users that previously had access to Mango (usually maintainers) and users that didn t. If you were used to access Mango before you should be able to login on the new Account Management System by entering your GNOME username and the password you were used to use for loggin in into Mango. (after loggin in the very first time you will be prompted to update your password, please choose a strong password as this account will be unique across all the GNOME Infrastructure) If you never had access to Mango, you lost your password or the first time you read the word Mango on this post you thought why is he talking about a fruit now? you should be able to reset it by using the following command:
ssh -l yourgnomeuserid account.gnome.org
The command will start an SSH connection between you and account.gnome.org, once authenticated (with the SSH key you previously had registered on our Infrastructure) you will trigger a command that will directly send your brand new password on the e-mail registered for your account. From my tests seems GMail sees the e-mail as a phishing attempt probably because the body contains the word password twice. That said if the e-mail won t appear on your INBOX, please double-check your Spam folder. Now that Mango is gone how can I request a new account? With Mango we used to have a form that automatically e-mailed the maintainer of the selected GNOME module which was then going to approve / reject the request. From there and in the case of a positive vote from the maintainer the Accounts Team was going to create the account itself. With the recent introduction of a commit robot directly on l10n.gnome.org the number of account requests reduced its numbers. In addition to that users will now be able to perform pretty much all the needed maintenance on their accounts themselves. That said and while we will probably work on building a form in the future we feel that requesting accounts can definitely be achieved directly by mailing the Accounts Team itself which will mail the maintainer of the respective module and create the account. As just said the number of account creations has become very low and the queue is currently clear. The documentation has been updated to reflect these changes at: https://wiki.gnome.org/AccountsTeam
https://wiki.gnome.org/AccountsTeam/NewAccounts I was used to have access to a specific service but I don t anymore, what should I do? The migration of all the user data and ACLs has been massive and I ve been spending a lot of time reviewing the existing HBAC rules trying to spot possible errors or misconfigurations. If you happen to not being able to access a certain service as you were used to in the past, please get in contact with the Sysadmin Team. All the possible ways to contact us are available at https://wiki.gnome.org/Sysadmin/Contact. What is missing still? Now that the Foundation membership information has been moved to LDAP I ll be looking at porting some of the existing membership scripts to it. What I managed to port already are welcome e-mails for new or existing members. (renewals) Next step will be generating a membership page from LDAP (to populate http://www.gnome.org/foundation/membership) and all the your-membership-is-going-to-lapse e-mails that were being sent till today. Other news /home/users mount on master.gnome.org You will notice that loggin in into master.gnome.org will result in your home directory being empty, don t worry, you did not lose any of your files but master.gnome.org is now currently hosting your home directories itself. As you may have been aware of adding files to the public_html directory on master resulted in them appearing on your people.gnome.org/~userid space. That was unfortunately expected as both master and webapps2 (the machine serving people.gnome.org s webspaces) were mounting the same GlusterFS share. We wanted to prevent that behaviour to happen as we wanted to know who has access to what resource and where. From today master s home directories will be there just as a temporary spot for your tarballs, just scp and use ftpadmin against them, that should be all you need from master. If you are interested in receiving or keeping using your people.gnome.org s webspace please mail <accounts AT gnome DOT org> stating so. Other news a shiny and new error 500 page has been deployed Thanks to Magdalen Berns (magpie) a new error 500 web page has been deployed on all the Apache istances we host. The page contains an iframe of status.gnome.org and will appear every single time the web server behind the service you are trying to reach will be unreachable for maintenance or other purposes. While I hope you won t see the page that often you can still enjoy it at https://static.gnome.org/error-500/500.html. Make sure to whitelist status.gnome.org on your browser as it currently loads it without https. (as the service is currently hosted on OpenShift which provides us with a *.rhcloud.com wildcard certificate, which differs from the CN the browser would expect it to be) Updates UPDATE on status.gnome.org s SSL certificate: the certificate has been provisioned and it should result in the 500 s page to be displayed correctly with no warnings from your browser. UPDATE from Adam Young on Kerberos ports being closed on many DC s firewalls:
The next version of upstream MIT Kerberos will have support for fetching a ticket via ports 443 and marshalling the request over HTTPS. We ll need to run a proxy on the server side, but we should be able to make it work: Read up here
http://adam.younglogic.com/2014/06/kerberos-firewalls

6 October 2014

Julian Andres Klode: Acer Chromebook 13 (FHD): Initial impressions

Today, I received my Acer Chromebook 13, in the glorious FullHD variant with 4GB RAM. For those of you who don t know it, the Acer Chromebook 13 is a 13.3 inch chromebook powered by a Tegra K1 cpu. Chromebook This version cannot be ordered currently, only pre-orders were shipped yesterday (at least here in Germany). I cannot even review it on Amazon (despite having it bought there), as they have not enabled reviews for it yet. The device feels solidly built, and looks good. It comes in all-white matte plastic and is slightly reminiscent of the old white MacBooks. The keyboard is horrible, there s no well defined pressure point. It feels like your typing on a pillow. The display is OK, an IPS would be a lot nicer to work with, though. Oh, and it could be brighter. I do not think that using it outside on a sunny day would be a good idea. The speakers are loud and clear compared to my ThinkPad X230. The performance of the device is about acceptable (unfortunately, I do not have any comparison in this device class). Even when typing this blog post in the visual wordpress editor, I notice some sluggishness. Opening the app launcher or loading the new tab page while music is playing makes the music stop for or skip a few ms (20-50ms if I had to guess). Running a benchmark in parallel or browsing does not usually cause this stuttering, though. There are still some bugs in Chrome OS: Loading the Play Books library the first time resulted in some rendering issues. The Browser process always consumes at least 10% CPU, even when idling, with no page open; this might cause some of the sluggishness I mentioned above. Also watching Flash videos used more CPU than I expected given that it is hardware accelerated. Finally, Netflix did not work out of the box, despite the Chromebook shipping with a special Netflix plugin. I always get some unexpected issue-type page. Setting the user agent to Chrome 38 from Windows, thus forcing the use of the EME video player instead of the Netflix plugin, makes it work. I reported these software issues to Google via Alt+Shift+I. The issues appeared on the current version of the stable channel, 37.0.2062.120. What s next? I don t know.
Filed under: Chromebook

Julian Andres Klode: Acer Chromebook 13 (FHD): Initial impressions

Today, I received my Acer Chromebook 13, in the glorious FullHD variant with 4GB RAM. For those of you who don t know it, the Acer Chromebook 13 is a 13.3 inch chromebook powered by a Tegra K1 cpu. Chromebook This version cannot be ordered currently, only pre-orders were shipped yesterday (at least here in Germany). I cannot even review it on Amazon (despite having it bought there), as they have not enabled reviews for it yet. The device feels solidly built, and looks good. It comes in all-white matte plastic and is slightly reminiscent of the old white MacBooks. The keyboard is horrible, there s no well defined pressure point. It feels like your typing on a pillow. The display is OK, an IPS would be a lot nicer to work with, though. Oh, and it could be brighter. I do not think that using it outside on a sunny day would be a good idea. The speakers are loud and clear compared to my ThinkPad X230. The performance of the device is about acceptable (unfortunately, I do not have any comparison in this device class). Even when typing this blog post in the visual wordpress editor, I notice some sluggishness. Opening the app launcher or loading the new tab page while music is playing makes the music stop for or skip a few ms (20-50ms if I had to guess). Running a benchmark in parallel or browsing does not usually cause this stuttering, though. There are still some bugs in Chrome OS: Loading the Play Books library the first time resulted in some rendering issues. The Browser process always consumes at least 10% CPU, even when idling, with no page open; this might cause some of the sluggishness I mentioned above. Also watching Flash videos used more CPU than I expected given that it is hardware accelerated. Finally, Netflix did not work out of the box, despite the Chromebook shipping with a special Netflix plugin. I always get some unexpected issue-type page. Setting the user agent to Chrome 38 from Windows, thus forcing the use of the EME video player instead of the Netflix plugin, makes it work. I reported these software issues to Google via Alt+Shift+I. The issues appeared on the current version of the stable channel, 37.0.2062.120. What s next? I don t know.
Filed under: Chromebook

Next.

Previous.