Search Results: "ema"

6 November 2014

Steve Kemp: Planning how to configure my next desktop

I recently setup a bunch of IPv6-only accessible hosts, which I mentioned in my previous blog post. In the end I got them talking to the IPv4/legacy world via the installation of an OpenVPN server - they connect over IPv6 get a private 10.0.0.0/24 IP address, and that is masqueraded via the OpenVPN-gateway. But the other thing I've been planning recently is how to configure my next desktop system. I generally do all development, surfing, etc, on one desktop system. I use virtual desktops to organize things, and I have a simple scripting utility to juggle windows around into the correct virtual-desktop as they're launched. Planning a replacement desktop means installing a fresh desktop, then getting all the software working again. These days I'd probably use docker images to do development within, along with a few virtual machines (such as the pbuilder host I used to release all my Debian packages). But there are still niggles. I'd like to keep the base system lean, with few packages, but you can't run xine remotely, similarly I need mpd/sonata for listening to music, emacs for local stuff, etc, etc. In short there is always the tendency to install yet-another package, service, or application on the desktop, which makes migration a pain. I'm not sure I could easily avoid that, but it is worth thinking about. I guess I could configure a puppet/slaughter/cfengine host and use that to install the desktop - but I've always done desktops "manually" and servers "magically" so it's a bit of a change in thinking.

Matthias Klumpp: The state of AppStream/GNOME-Software in Debian Jessie

or Why do I not see any update notifications on my brand-new Debian Jessie installation?? This is a short explanation of the status quo, and also explains the no update notifications issue in a slightly more detailed way, since I am already getting bug reports for that. As you might know, GNOME provides GNOME-Software for installation of applications via PackageKit. In order to work properly, GNOME-Software needs AppStream metadata, which is not yet available in Debian. There was a GSoC student working on the necessary code for that, but the code is not yet ready and doesn t produce good results yet. Therefore, I postponed AppStream integration to Jessie+1, with an option to include some metadata for GNOME and KDE to use via a normal .deb package. Then, GNOME was updated to 3.14. GNOME 3.14 moved lots of stuff into GNOME-Software, including the support for update-notifications (which have been in g-s-d before). GNOME-Software is also the only thing which can edit the application groups in GNOME-Shell, at least currently. So obviously, there was no a much stronger motivation to support GNOME-Software in Jessie. The appstream-glib library, which GNOME-Software uses exclusively to read AppStream metadata, didn t support the DEP-11 metadata format which Debian uses in place of the original AppSTream XML for a while, but does so in it s current development branch. So that component had to be packaged first. Later, GNOME-Software was uploaded to the archive as well, but still lacked the required metadata. That data was provided by me as a .deb package later, locally generated using the current code by my SoC student (the data isn t great, but better than nothing). So far with the good news. But there are multiple issues at time. First of all, the appstream-data package didn t pass NEW so far, due to it s complex copyright situation (nothing we can t resolve, since app-install-data, which appstream-data would replace, is in Debian as well). Also, GNOME-Software is exclusively using offline-updates (more information also on [1] and [2]) at time. This isn t always working at the moment, since I haven t had the time to test it properly and I didn t expect it to be used in Debian Jessie as well[3]. Furthermore, the offline-updates feature requires systemd (which isn t an issue in itself, I am quite fine with that, but people not using it will get unexpected results, unless someone does the work to implement offline-updates with sysvinit). Since we are in freeze at time, and obviously this stuff is not ready yet, GNOME is currently without update notifications and without a way to change the shell application groups. So, how can we fix this? One way would of course be to patch notification support back into g-s-d, if the new layout there allows doing that. But that would not give us the other features GNOME-Software provides, like application-group-editing. Implementing that differently and patching it to make it work would be more or at least the same amount of work like making GNOME-Software run properly. I therefore prefer getting GNOME-Software to run, at least with basic functionality. That would likely mean hiding things like the offline-update functionality, and using online-updates with GNOME-PackageKit instead. Obviously, this approach has it s own issues, like doing most of the work post-freeze, which kind of defeats the purpose of the freeze and would need some close coordination with the release-team. So, this is the status quo at time. It is kind of unfortunate that GNOME moved crucial functionality into a new component which requires additional integration work by the distributors so quickly, but that s something which isn t worth to talk about. We need a way forward to bring update-notifications back, and there is currently work going on to do that. For all Debian users: Please be patient while we resolve the situation, and sorry for the inconvenience. For all developers: If you would like to help, please contact me or Laurent Bigonville, there are some tasks which could use some help.
As a small remark: If you are using KDE, you are lucky Apper provides the notification support like it always did, and thanks to improvements in aptcc and PackageKit, it even is a bit faster now. For the Xfce and <other_desktop> people, you need to check if your desktop provides integration with PackageKit for update-checking. At least Xfce doesn t, but after GNOME-PackageKit removed support for it (which was moved to gnome-settings-daemon and now to GNOME-Software) nobody stepped up to implement it yet (so if you want to do it it s not super-complicated, but knowledge of C and GTK+ is needed). - [3]: It looks like dpkg tries to ask a debconf question for some reason, or an external tool like apt-listchanges is interfering with the process, which must run completely unsupervised. There is some debugging needed to resolve these Debian-specific issues.

5 November 2014

Rapha&#235;l Hertzog: My Free Software Activities in October 2014

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Packaging work With the Jessie freeze approaching, I took care of packaging some new upstream releases that I wanted to get in. I started with zim 0.62, I had skipped 0.61 due to some annoying regressions. Since I had two bugs to forward, I took the opportunity to reach out to the upstream author to see if he had some important fixes to get into Jessie. This resulted in me pushing another update with 3 commits cherry picked from the upstream VCS. I also sponsored a wheezy-backports of the new version. I pushed two new bugfixes releases of Publican (4.2.3 and 4.2.6) but I had to include a work-around for a bug that I reported earlier on docbook-xml (#763598: the XML catalog doesn t allow libxml2/xmllint to identify the local copy of some entities files) and that is unlikely to be fixed in time for Jessie. Last but not least, I pushed the first point release of Django 1.7, aka version 1.7.1 to unstable and asked release managers to ensure it migrates to testing before the real freeze. This is important because the closer we are to upstream, the easier it is to apply security patches during the lifetime of Jessie (which will hopefully be 5 years, thanks to Debian LTS!). I also released a backport of python-django 1.7 to wheezy-backports. I sponsored galette 0.7.8+dfsg-1 fixing an RC bug so that it can get back to testing (it got removed from testing due to the bug). Debian LTS See my dedicated report for the paid work I did on that area. Apart from that, I took some time to get in touch with all the Debian consultants and see if they knew some companies to reach out. There are a few new sponsors in the pipe thanks to this, but given the large set of people that it represents, I was expecting more. I used this opportunity to report all bogus entries (i.e bouncing email, broken URL) to the maintainer of the said webpage. Distro Tracker Only 30 commits this month, with almost no external contribution, I m a bit saddened by this situation because it s not very difficult to contribute to this project and we have plenty of easy bugs to get you started. That said I m still happy with the work done. Most of the changes have been made for Kali but will be useful for all derivatives: it s now possible to add external repositories in the tracker and not display them in the list of available versions, and not generate automatic news about those repositories. There s a new derivative application which is only in its infancy but can already provide a useful comparison of a derivative with its parent. See it in action on the Kali Package Tracker: http://pkg.kali.org/derivative/ Thanks to Offensive Security which is sponsoring this work! Since I have pushed Django 1.7 to wheezy-backports, all distro tracker instances that I manage are now running that version of Django and I opted to make that version mandatory. This made it possible to add initial Django migrations and rely on this new feature for future database schema upgrade (I have voluntarily avoided schema change up to now to avoid problems migrating from South to Django migrations). Thanks See you next month for a new summary of my activities.

One comment Liked this article? Click here. My blog is Flattr-enabled.

Carl Chenet: Send the same short message on Twitter, Pump.io, Diaspora* and a lot more

Follow me on Identi.ca or Twitter or Diaspora*diaspora-banner This is a feedback about installing a self hosted instance of Friendica on a Debian server (Jessie). If you re not interested in why I use Friendica, just go to Prerequisite for Friendica section below. Frustration about social networks Being a huge user of short messages, I was quite frustated to spend so much time on my Twitter account. To be quite honest, there is no much I like about this social network, except the huge population of people being potentially interested in what I write. twitter-logo I also have been using Identi.ca (now powered by Pump.io) for a while. But I tried for a while to manage both networks Pump.io and Twitter by hand and it was quite painful. And something was telling me another social network was going to appear from nowhere one of these days and I ll be just horrible to try to keep it up this way. identica-logo So I was looking for a scalable solution not asking too much personal investment. Subscribing to Diaspora* some days ago on the Framasphere pod, tintouli told me to try Friendica. Hmmm, what s Friendica ? f-intro Friendica is a content manager you can plug on almost anything: social networks (Facebook, Twitter, Pump.io, Diaspora*, ), but also WordPress, XMPP, emails I m in fact just discovering the power of this tool but to plug it on my different social network accounts was quite a good use case for me. And I guess if you re still reading, for you too. I tried to use some shared public servers but I was not quite happy with the result, one connector was still missing or public servers were really unstable. So I m at last self hosting my Friendica. Here is how. Prerequisite for Friendica You need to install the following packages: # apt-get install apache2 libapache2-mod-php5 php5 php5-curl php5-gd php5-mysql mysql-server git Having a already self-modified /etc/php5/apache2/php.ini, I encountered a small issue with libCurl and had to manually add the following line in the php.ini: extension=curl.so Setting up MySQL Connect to MySQL and create an empty database with a dedicated user: # mysql -u root -pV3rYS3cr3t -e create database friendica; GRANT ALL PRIVILEGES ON friendica.* TO friendica@localhost IDENTIFIED BY R3AlLyH4rdT0Gu3ss mysql Setting up Apache My server hosts several services, so I use a subdomain friendica.mydomain.com. If you use a subdomain, of course check you do have declared this subdomain in your DNS zone. apache-logo I use SSL encryption with a wildcard certificate for all my subdomains. My Friendica data are stored in /var/www/friendica. Here is my virtual host configuration for Friendica stored in the file /etc/apache2/sites-available/friendicassl.conf : <VirtualHost *:443>
ServerName friendica.mydomain.com
DocumentRoot /var/www/friendica/
DirectoryIndex index.php index.html
ErrorLog /var/log/apache2/friendica-error-ssl.log
TransferLog /var/log/apache2/friendica-access-ssl.log SSLEngine on
SSLCertificateFile /etc/ssl/certs/mydomain/mydomain.com.crt
SSLCertificateKeyFile /etc/ssl/private/mydomain.com.key
SSLVerifyClient None <Directory /var/www/friendica/>
AllowOverride All
Options FollowSymLinks
Order allow,deny
Allow from all
</Directory>
</VirtualHost> After writing the configuration file, just launch the following commands and it should be good for the Apache configuration: # a2ensite friendicassl && /etc/init.d/apache2 reload friendica-trust Setting up Friendica Get the master zip file of Friendica, copy it on your server and decompress it. Something like : # cd /var/www/ && wget https://github.com/friendica/friendica/archive/master.zip && unzip master.zip && mv friendica-master friendica You need to give www-data (Apache user) the rights to write in /var/www/friendica/view/smarty3/ : # chown -R www-data:www-data /var/www/friendica/view/smarty3 && chmod -R ug+w var/www/friendica/view/smarty3 Ok, I guess we re all set, lets launch the installation process! Using your web browser, connect to friendica.mydomain.com. First step you ll see the installation window which checks the prerequisite before installing. Complete if something is missing.
friendica-installation

First window of the Friendica installation process

Second step asks the host/user/password of the database, complete and the installation process starts. Hopefully all goes just fine. Next you ll have to create a /var/www/friendica/.htconfig.php with the content that the last page of the installation process provides. Just copy/paste, check the rights of this file and now you can connect again to see the register page of friendica at the url https://friendlica.mydomain.com/register . Pretty cool! Register a user That s a fairly easy step. You just need to check before that your server is able to send emails, because the password is going to be sent to you by email. If it is ok, you should now identify on the welcome page of friendica and access your account. That s a huge step to broadcast your short messages everywhere, but we have some last steps before being able to send your short messages on all the social networks we need. A small break, create an app for twitter on apps.twitter.com To send your short messages to Twitter, you need to create an app on apps.twitter.com. Just check you re logged in Twitter and connect to apps.twitter.com. Create an app called with a unique name (apparently), then go to the Keys and Access tokens page, note the consumer key and the consumer secret. You ll later need the name of the app, the consumer key and the consumer secret. friendica-decentralized-network Install and configure the addons Friendica uses an addon system in order to plug on the different third-parties it needs. We are going to configure Twitter, Pump.io and the Diaspora* plug. Let s go back to our server and launches some commands: # cd /tmp && git clone https://github.com/friendica/friendica-addons.git && cd friendica-addons # tar xvf twitter.tgz -C /var/www/friendica/addon # tar xvf pumpio.tgz -C /var/www/friendica/addon # cp -a diaspora /var/www/friendica/addon You need to modify your /var/www/friendica/.htconfig.php file and add the following content at the end: // names of your addons, separated by a comma $a->config['system']['addon'] = pumpio, twitter, diaspora ; // your Twitter consumer key
$a->config['twitter']['consumerkey'] = P4Jl2Pe4j7Lj91eIn0AR8vIl2 ; // your Twitter consumer secret
$a->config['twitter']['consumersecret'] = 1DnVkllPik9Ua8jW4fncxwtXZJbs9iFfI5epFzmeI8VxM9pqP1 ; // you Twitter app name $a->config['twitter']['application_name'] = whatever-twitter ; banner_friendica Connect again to Friendica. Go to settings => social networks, you will see the options for Twitter, Pump.io and Diaspora*. Complete the requested information for each of them. Important options you should not forget to check are: Pump.io Twitter Diaspora* Done? Now it s time to send your first broadcasted short message. Yay! Send a short message to your different social networks Connect to Friendica, click on the network page, write your short message in the Share box. Click on the lock lock, you ll see the following setup: friendica-post-default It means your short messages will be broadcasted to the three networks. Or more, it s up to you! That s my setup, feel free to modify. Now close the lock window and send your message. For me it takes some time to appear on Twitter and Diaspora* and it immediatly appears on Identi.ca. Last words friendica-stand Friendica offers to take back the control of your data, by broadcasting content on different media from a single source. While self hosting, you keep your data whatever happends and are not subject to companies losing your data like recently Twitpic. Moreover the philosophy behind Friendica pushed me to dig and test the solution What about you? How do you proceed to broadcast your short messages? Does Friendica offer a good solution in your opinion? Are you interested in the philosophy behind this project? Feel free to share your thoughs in the comments. LAST MINUTE: hey, this article is on Hacker News, don t hesitate to vote for it if you liked it!

4 November 2014

Junichi Uekawa: emacs-gdb interfaces.

emacs-gdb interfaces. emacs and gdb interfaces have evolved. I think -fullname option is old interface used by gud-gdb. I think there's also --annotate=3 which was used in emacs23. emacs24 M-x gdb uses -i=mi. Some things in M-x gdb annoy me but I'm not finding a good docuemntation on this matter. Hmm..

Tiago Bortoletto Vaz: A few words on the recent Brazilian elections

The Brazilian presidential election was exceedingly intense this year. Among many inferences that we can make by following the news and investigating data from the voting results I'd like to share this one, which in my opinion reflects quite well the vote preferences in the country. First, let me introduce you "Bel gua", a small town located in the Northeast region of Brazil. It has 6,524 habitants, 3 buses and 2 hospitals. According to the Brazilian Institute of Geography and Statistics (IBGE), the income per person in Bel gua is $146 BRL (or U$59) per month. Believe it or not, it used to be much less. Actually, the city reported in 2013 the highest economic jump in the country, rising more than a thousand positions in the ranking of GDP per capita (from position 4,991 to 3,849). This recent growth was consequence of the social welfare program of the Brazilian government, which also boosted artisanal and manioc flour production. This federal assistance is called "Bolsa fam lia", which benefits 1.814 families in Bel gua.
"Bolsa Fam lia currently gives families with per-capita monthly income below $140 BRL (poverty line, ~$56 USD) a monthly stipend of $32 BRL (~$13 USD) per vaccinated child (< 16 years old) attending school (up to 5), and $38 BRL (~$15 USD) per youth (16 or 17 years old) attending school (up to 2). Furthermore, to families whose per-capita monthly income below $70 BRL (extreme poverty line, ~$28 USD), the program gives the Basic Benefit $70 BRL per month." (from Wikipedia)
Contrary to what many of my middle-class friends believe, and as you can calculate yourself, this little amount of money does not offer anybody a luxury life. It does not make anybody stopping working, nor stopping looking for paid job (but yes, it makes people to start saying NO to forced labor, which is amazing, right?).
/images/belagua.jpg

Bel gua, where Dilma got 93.93% of votes (photo by Clarissa Carramilo / from G1)

Also, Bel gua has no much physicians around because doctors in Brazil usually wouldn't live in a such city. But now Bel gua population can be treated by foreign doctors imported by the recently launched program "Mais M dicos" (More Physicians for Brazil), which hosts two Cuban doctors 15km away. Finally, Bel gua people have light, due to the "Luz para Todos" ("Light for All") program. It's not surprising that Bel gua has re-elected the party which has motivated these changes. For 2014 presidential election, Bel gua people gave 3.558 votes (93.93%) to Dilma Rousseff (candidate of the current government, from a left-ish party), against 230 (6,07%) to A cio Neves (from the right coalition), being the city with the largest amount of votes for Dilma, proportionally, followed by "Serrano do Maranh o" (93,75%), which is located in the same region. On the other hand, the city which gave, proportionally, the largest amount of votes for A cio Neves has a population of 5,564,635 habitants, where most of citizens are not Brazilians, not yet. Miami, located in US, was the city where Brazilian residents would elect A cio by 7,225 votes (91,79%), against 646 (8,21%) for Dilma, followed by Atlanta/US (89,47%) and Houston/US (89,22%).
/images/miami.jpg

Miami, where 91,79% of Brazilians voted A cio (photo by Marc Averette / from Wikipedia)

It's so clear that we do what people call "selfish vote". In general, we don't care about which party has better proposals for the society as whole. Rich people will go against any serious social equality proposal, which will necessarily be followed by higher taxes on their fortunes. As middle class citizens, we care about dollar rates, because we want to get cheaper iStuff from Miami. We're also very upset by the fact that new apartments are being built without that small room in the back, which has been used to accommodate a subservient housemaid who, until last year was not even legally considered a worker. Those people from Bel gua, who used to live in extreme poverty for decades, serving as slaves, they mostly care about having something to eat. Now they eat, so they can think better, they can work, they can sell what they produce in their little yard. And like middle-class and rich citizens, they will give their vote in exchange of what they think is better for them. The big difference here is, if we ask Bel gua people why they voted for Dilma, with no embarrassment they will make it very clear, that's because her government has provided them lots of benefits. Asking the same question for most Brazilians in Miami, Atlant, Houston or S o Paulo, you'll get not only a bunch of allegedly moral/altruistic reasons, but they will also try to delegitimize in many ways the votes from those marginalized citizens. You'll never get the real reasons from them. They will even try to convince you that whoever receives federal assistance should automatically lose right to vote. Such a statement may seem ridiculous, however it has been very present recently. Actually, such hate speech is happening right now. While I'm writing this post about 2500 people are protesting in S o Paulo streets, asking for an immediate military coup because they don't agree with the elections result. These people keep pushing the limits of ridiculousness. Dilma won with 51.64% of valid votes, a very tight result. The country is clearly divided, mostly by hate, unfortunately.

3 November 2014

Chris Lamb: Generating gradiented fade images using ImageMagick

Whilst gradienting images is certainly possible with CSS, current browser support means that it can still make sense to do it yourself, especially if front-end performance is a concern. However, to avoid manual work in Gimp or Photoshop, you can use ImageMagick to generate them for you:
$ wget --quiet -Obackground.jpg http://i.imgur.com/WCjlJ.jpg
$ convert background.jpg \
      -alpha set -channel A \
      -sparse-color Barycentric '%w,%[fx:h-300] opaque %w,%h transparent' \
      -background '#ffcc32' -flatten \
      background-gradiented.jpg
300 here refers to the height or "speed" of the gradient and the target colour is specified by with -background. Before: https://chris-lamb.co.uk/wp-content/2014/gradient_before.png After: https://chris-lamb.co.uk/wp-content/2014/gradient_after.png

31 October 2014

Russell Coker: Samsung Galaxy Note 3

In June last year I bought a Samsung Galaxy Note 2 [1]. Generally I was very happy with that phone, one problem I had is that less than a year after purchasing it the Ingress menus burned into the screen [2]. 2 weeks ago I bought a new Galaxy Note 3. One of the reasons for getting it is the higher resolution screen, I never realised the benefits of a 1920*1080 screen on a phone until my wife got a Nexus 5 [3]. I had been idly considering a Galaxy Note 4, but $1000 is a lot of money to pay for a phone and I m not sure that a 2560*1440 screen will offer much benefit in that size. Also the Note 3 and Note 4 both have 3G of RAM, as some applications use more RAM when you have a higher resolution screen the Note 4 will effectively have less usable RAM than the Note 3. My first laptop cost me $3,800 in 1998, that s probably more than $6,000 in today s money. The benefits that I receive now from an Android phone are in many ways greater than I received from that laptop and that laptop was definitely good value for money for me. If the cheapest Android phone cost $6,000 then I d pay that, but given that the Note 3 is only $550 (including postage) there s no reason for me to buy something more expensive. Another reason for getting a new phone is the limited storage space in the Note 2. 16G of internal storage is a limit when you have some big games installed. Also the recent Android update which prevented apps from writing to the SD card meant that it was no longer convenient to put TV shows on my SD card. 32G of internal storage in the Note 3 allows me to fit everything I want (including the music video collection I downloaded with youtube-dl). The Note 2 has 16G of internal storage and an 8G SD card (that I couldn t fully use due to Android limitations) while the Note 3 has 32G (the 64G version wasn t on sale at any of the cheap online stores). Also the Note 3 supports an SD card which will be good for my music video collection at some future time, this is a significant benefit over the Nexus 5. In the past I ve written about Android service life and concluded that storage is the main issue [4]. So it is a bit unfortunate that I couldn t get a phone with 64G of storage at a reasonable price. But the upside is that getting a cheaper phone allows me to buy another one sooner and give the old phone to a relative who has less demanding requirements. In the past I wrote about the warranty support for my wife s Nexus 5 [5]. I should have followed up on that before, 3 days after that post we received a replacement phone. One good thing that Google does is to reserve money on a credit card to buy the new phone and then send you the new phone before you send the old one back. So if the customer doesn t end up sending the broken phone they just get billed for the new phone, that avoids excessive delays in getting a replacement phone. So overall the process of Google warranty support is really good, if 2 products are equal in other ways then it would be best to buy from Google to get that level of support. I considered getting a Nexus 5 as the hardware is reasonably good (not the greatest but quite good enough) and the price is also reasonably good. But one thing I really hate is the way they do the buttons. Having the home button appear on the main part of the display is really annoying. I much prefer the Samsung approach of having a hardware button for home and touch-screen buttons outside the viewable area for settings and back. Also the stylus on the Note devices is convenient on occasion. The Note 3 has a fake-leather back. The concept of making fake leather is tacky, I believe that it s much better to make honest plastic that doesn t pretend to be something that it isn t. However the texture of the back improves the grip. Also the fake stitches around the edge help with the grip too. It s tacky but utilitarian. The Note 3 is slightly smaller and lighter than the Note 2. This is a good technical achievement, but I d rather they just gave it a bigger battery. Update USB 3 One thing I initially forgot to mention is that the Note 3 has USB 3. This means that it has a larger socket which is less convenient when you try and plug it in at night. USB 3 seems unlikely to provide any benefit for me as I ve never had any of my other phones transfer data at rates more than about 5MB/s. If the Note 3 happens to have storage that can handle speeds greater than the 32MB/s a typical USB 2 storage device can handle then I m still not going to gain much benefit. USB 2 speeds would allow me to transfer the entire contents of a Note 3 in less than 20 minutes (if I needed to copy the entire storage contents). I can t imagine myself having a real-world benefit from that. The larger socket means more fumbling when charging my phone at night and it also means that the Note 3 cable can t be used in any other phone I own. In a year or two my wife will have a phone with USB 3 support and that cable can be used for charging 2 phones. But at the moment the USB 3 cable isn t useful as I don t need to have a phone charger that can only charge one phone. Conclusion The Note 3 basically does everything I expected of it. It s just like the Note 2 but a bit faster and with more storage. I m happy with it.

30 October 2014

Matthew Garrett: Hacker News metrics (first rough approach)

I'm not a huge fan of Hacker News[1]. My impression continues to be that it ends up promoting stories that align with the Silicon Valley narrative of meritocracy, technology will fix everything, regulation is the cancer killing agile startups, and discouraging stories that suggest that the world of technology is, broadly speaking, awful and we should all be ashamed of ourselves.

But as a good data-driven person[2], wouldn't it be nice to have numbers rather than just handwaving? In the absence of a good public dataset, I scraped Hacker Slide to get just over two months of data in the form of hourly snapshots of stories, their age, their score and their position. I then applied a trivial test:
  1. If the story is younger than any other story
  2. and the story has a higher score than that other story
  3. and the story has a worse ranking than that other story
  4. and at least one of these two stories is on the front page
then the story is considered to have been penalised.

(note: "penalised" can have several meanings. It may be due to explicit flagging, or it may be due to an automated system deciding that the story is controversial or appears to be supported by a voting ring. There may be other reasons. I haven't attempted to separate them, because for my purposes it doesn't matter. The algorithm is discussed here.)

Now, ideally I'd classify my dataset based on manual analysis and classification of stories, but I'm lazy (see [2]) and so just tried some keyword analysis:
KeywordPenalisedUnpenalised
Women134
Harass20
Female51
Intel23
x8634
ARM34
Airplane12
Startup4626

A few things to note:
  1. Lots of stories are penalised. Of the front page stories in my dataset, I count 3240 stories that have some kind of penalty applied, against 2848 that don't. The default seems to be that some kind of detection will kick in.
  2. Stories containing keywords that suggest they refer to issues around social justice appear more likely to be penalised than stories that refer to technical matters
  3. There are other topics that are also disproportionately likely to be penalised. That's interesting, but not really relevant - I'm not necessarily arguing that social issues are penalised out of an active desire to make them go away, merely that the existing ranking system tends to result in it happening anyway.

This clearly isn't an especially rigorous analysis, and in future I hope to do a better job. But for now the evidence appears consistent with my innate prejudice - the Hacker News ranking algorithm tends to penalise stories that address social issues. An interesting next step would be to attempt to infer whether the reasons for the penalties are similar between different categories of penalised stories[3], but I'm not sure how practical that is with the publicly available data.

(Raw data is here, penalised stories are here, unpenalised stories are here)


[1] Moving to San Francisco has resulted in it making more sense, but really that just makes me even more depressed.
[2] Ha ha like fuck my PhD's in biology
[3] Perhaps stories about startups tend to get penalised because of voter ring detection from people trying to promote their startup, while stories about social issues tend to get penalised because of controversy detection?

comment count unavailable comments

Keith Packard: Glamor cleanup

Glamor Cleanup Before I start really digging in to reworking the Render support in Glamor, I wanted to take a stab at cleaning up some cruft which has accumulated in Glamor over the years. Here's what I've done so far. Get rid of the Intel fallback paths I think it's my fault, and I'm sorry. The original Intel Glamor code has Glamor implement accelerated operations using GL, and when those fail, the Intel driver would fall back to its existing code, either UXA acceleration or software. Note that it wasn't Glamor doing these fallbacks, instead the Intel driver had a complete wrapper around every rendering API, calling special Glamor entry points which would return FALSE if GL couldn't accelerate the specified operation. The thinking was that when GL couldn't do something, it would be far faster to take advantage of the existing UXA paths than to have Glamor fall back to pulling the bits out of GL, drawing to temporary images with software, and pushing the bits back to GL. And, that may well be true, but what we've managed to prove is that there really aren't any interesting rendering paths which GL can't do directly. For core X, the only fallbacks we have today are for operations using a weird planemask, and some CopyPlane operations. For Render, essentially everything can be accelerated with the GPU. At this point, the old Intel Glamor implementation is a lot of ugly code in Glamor without any use. I posted patches to the Intel driver several months ago which fix the Glamor bits there, but they haven't seen any review yet and so they haven't been merged, although I've been running them since 1.16 was released... Getting rid of this support let me eliminate all of the _nf functions exported from Glamor, along with the GLAMOR_USE_SCREEN and GLAMOR_USE_PICTURE_SCREEN parameters, along with the GLAMOR_SEPARATE_TEXTURE pixmap type. Force all pixmaps to have exact allocations Glamor has a cache of recently used textures that it uses to avoid allocating and de-allocating GL textures rapidly. For pixmaps small enough to fit in a single texture, Glamor would use a cache texture that was larger than the pixmap. I disabled this when I rewrote the Glamor rendering code for core X; that code used texture repeat modes for tiles and stipples; if the texture wasn't the same size as the pixmap, then texturing would fail. On the Render side, Glamor would actually reallocate pixmaps used as repeating texture sources. I could have fixed up the core rendering code to use this, but I decided instead to just simplify things and eliminate the ability to use larger textures for pixmaps everywhere. Remove redundant pixmap and screen private pointers Every Glamor pixmap private structure had a pointer back to the pixmap it was allocated for, along with a pointer to the the Glamor screen private structure for the related screen. There's no particularly good reason for this, other than making it possible to pass just the Glamor pixmap private around a lot of places. So, I removed those pointers and fixed up the functions to take the necessary extra or replaced parameters. Similarly, every Glamor fbo had a pointer back to the Glamor screen private too; I removed that and now pass the Glamor screen private parameter as needed. Reducing pixmap private complexity Glamor had three separate kinds of pixmap private structures, one for 'normal' pixmaps (those allocated by them selves in a single FBO), one for 'large' pixmaps, where the pixmap was tiled across many FBOs, and a third for 'atlas' pixmaps, which presumably would be a single FBO holding multiple pixmaps. The 'atlas' form was never actually implemented, so it was pretty easy to get rid of that. For large vs normal pixmaps, the solution was to move the extra data needed by large pixmaps into the same structure as that used by normal pixmaps and simply initialize those elements correctly in all cases. Now, most code can ignore the difference and simply walk the array of FBOs as necessary. The other thing I did was to shrink the number of possible pixmap types from 8 down to three. Glamor now exposes just these possible pixmap types: Future Work

29 October 2014

Gunnar Wolf: Guests in the classroom: @chemaserralde talks about real time scheduling

Last Wednesday I had the pleasure and honor to have a great guest again at my class: Jos Mar a Serralde, talking about real time scheduling. I like inviting different people to present interesting topics to my students a couple of times each semester, and I was very happy to have Chema come again. Chema is a professional musician (formally, a pianist, although he has far more skills than what a title would confer to him Skills that go way beyond just music), and he had to learn the details on scheduling due to errors that appear when recording and performing. The audio could use some cleaning, and my main camera (the only one that lasted for the whole duration) was by a long shot not professional grade, but the video works and is IMO quite interesting and well explained. So, here is the full video (also available at The Internet archive), all two hours and 500MB of it for you to learn and enjoy!

Rhonda D'Vine: Feminist Year

If someone would have told me that I would visit three feminist events this year I would have slowly nodded at them and responded with "yeah, sure..." not believing it. But sometimes things take their own turns. It all started with the Debian Women Mini-Debconf in Barcelona. The organizers did ask me how they have to word the call for papers so that I would feel invited to give a speech, which felt very welcoming and nice. So we settled for "people who identify themselves as female". Due to private circumstances I didn't prepare well for my talk, but I hope it was still worth it. The next interesting part though happened later when there were lightning talks. Someone on IRC asked why there are male people in the lightning talks, which was explicitly allowed for them only. This also felt very very nice, to be honest, that my talk wasn't questioned. Those are amongst the reasons why I wrote My place is here, my home is Debconf. Second event I went to was the FemCamp Wien. It was my first event that was a barcamp, I didn't know what to expect organization wise. Topic-wise it was set about Queer Feminism. And it was the first event that I went to which had a policy. Granted, there was an extremely silly written part in it, which naturally ended up in a shit storm on twitter (which people from both sides did manage very badly, which disappointed me). Denying that there is sexism against cis-males is just a bad idea, but the background of it was that this wasn't the topic of this event. The background of the policy was that usually barcamps but events in general aren't considered that save of a place for certain people, and that this barcamp wanted to make it clear that people usually shying away from such events in the fear of harassment can feel at home there.
And what can I say, this absolutely was the right thing to do. I never felt any more welcomed and included in any event, including Debian events sorry to say that so frankly. Making it clear through the policy that everyone is on the same boat with addressing each other respectfully totally managed to do exactly that. The first session of the event about dominant talk patterns and how to work around or against them also made sure that the rest of the event was giving shy people a chance to speak up and feel comfortable, too. And the range of the sessions that were held was simply great. This was the event that I came up with the pattern that I have to define the quality of an event on the sessions that I'm unable to attend. The thing that hurt me most in the afterthought was that I couldn't attend the session about minorities within minorities. :/ Last but not least I attended AdaCamp Berlin. This was a small unconference/barcamp dedicated to increase women's participation in open technology and culture named after Ada Lovelace who is considered the first programmer. It was a small event with only 50 slots for people who identify as women. So I was totally hyper when I received the mail that was accepted. It was another event with a policy, and at first reading it looked strange. But given that there are people who are allergic to ingredients of scents, it made sense to raise awareness of that topic. And given that women are facing a fair amount of harassment in the IT and at events, it also makes sense to remind people to behave. After all it was a general policy for all AdaCamps, not for this specific one with only women.
I enjoyed the event. Totally. And that's not only because I was able to meet up with a dear friend who I haven't talked to in years, literally. I enjoyed the environment, and the sessions that were going on. And quite similar to the FemCamp, it started off with a session that helped a lot for the rest of the event. This time it was about the Impostor Syndrome which is extremely common for women in IT. And what can I say, I found myself in one of the slides, given that I just tweeted the day before that I doubted to belong there. Frankly spoken, it even crossed my mind that I was only accepted so that at least one trans person is there. Which is pretty much what the impostor syndrome is all about, isn't it. But when I was there, it did feel right. And we had great sessions that I truly enjoyed. And I have to thank one lady once again for her great definition on feminism that she brought up during one session, which is roughly that feminism for her isn't about gender but equality of all people regardless their sexes or gender definition. It's about dropping this whole binary thinking. I couldn't agree more. All in all, I totally enjoyed these events, and hope that I'll be able to attend more next year. From what I grasped all three of them think of doing it again, the FemCamp Vienna already has the date announced at the end of this year's event, so I am looking forward to meet most of these fine ladies again, if faith permits. And keep in mind, there will always be critics and haters out there, but given that thy wouldn't think of attending such an event anyway in the first place, don't get wound up about it. They just try to talk you down. P.S.: Ah, almost forgot about one thing to mention, which also helps a lot to reduce some barrier for people to attend: The catering during the day and for lunch both at FemCamp and AdaCamp (there was no organized catering at the Debian Women Mini-Debconf) did take off the need for people to ask about whether there could be food without meat and dairy products by offering mostly Vegan food in the first place, even without having to query the participants. Often enough people otherwise choose to go out of the event or bring their own food instead of asking for it, so this is an extremely welcoming move, too. Way to go!

/personal permanent link Comments: 2 Flattr this

27 October 2014

Junichi Uekawa: Running git grep under emacs compilation mode.

Running git grep under emacs compilation mode. It's driving me nuts because there's 0xfeff(BOM) at the beginning which seems to break file name matching.

26 October 2014

Joey Hess: a programmable alarm clock using systemd

I've taught my laptop to wake up at 7:30 in the morning. When it does, it will run whatever's in my ~/bin/goodmorning script. Then, if the lid is still closed, it will go back to sleep again. So, it's a programmable alarm clock that doesn't need the laptop to be left turned on to work. But it doesn't have to make noise and wake me up (I rarely want to be woken up by an alarm; the sun coming in the window is a much nicer method). It can handle other tasks like downloading my email, before I wake up. When I'm at home and on dialup, this tends to take an hour in the morning, so it's nice to let it happen before I get up. This took some time to figure out, but it's surprisingly simple. Besides ~/bin/goodmorning, which can be any program/script, I needed just two files to configure systemd to do this. /etc/systemd/system/goodmorning.timer
[Unit]
Description=good morning
[Timer]
Unit=goodmorning.service
OnCalendar=*-*-* 7:30
WakeSystem=true
Persistent=false
[Install]
WantedBy=multi-user.target
/etc/systemd/system/goodmorning.service
[Unit]
Description=good morning
RefuseManualStart=true
RefuseManualStop=true
ConditionACPower=true
[Service]
Type=oneshot
ExecStart=/bin/systemd-inhibit --what=handle-lid-switch --why=goodmorning /bin/su joey -c "/usr/bin/timeout 45m /home/joey/bin/goodmorning"
installation After installing those files, run (as root): systemctl enable goodmorning.timer; systemctl start goodmorning.timer Then, you'll also need to edit /etc/systemd/logind.conf, and set LidSwitchIgnoreInhibited=no -- this overrides the default, which is not to let systemd-inhibit block sleep on lid close. almost too easy I don't think this would be anywhere near as easy to do without systemd, logind, etc. Especially the handling of waking the system at the right time, and the behavior around lid sleep inhibiting. The WakeSystem=true relies on some hardware support for waking from sleep; my laptop supported it with no trouble but I don't know how broadly available that is. Also, notice the ConditionACPower=true, which I added once I realized I don't want the job to run if I forgot to leave the laptop plugged in overnight. Technically, it will still wake up when on battery power, but then it should go right back to sleep. Quite a lot of nice peices of systemd all working together here! xfce workaround If using xfce, xfce4-power-manager takes over handling of lid close from systemd, and currently prevents the system from going back to sleep if the lid is still closed when goodmorning finishes. Happily, there is an easy workaround; this configures xfce to not override the lid switch behavior: xfconf-query -c xfce4-power-manager -n -p /xfce4-power-manager/logind-handle-lid-switch -t bool -s true Other desktop environments may have similar issues. why not a per-user unit? It would perhaps be better to use the per-user systemd, not the system wide one. Then I could change the time the alarm runs without using root. What's prevented me from doing this is that systemd-inhibit uses policykit, and policykit prevents it from being used in this situation. It's a lot easier to run it as root and use su, than it is to reconfigure policykit.

Colin Watson: Moving on, but not too far

The Ubuntu Code of Conduct says:
Step down considerately: When somebody leaves or disengages from the project, we ask that they do so in a way that minimises disruption to the project. They should tell people they are leaving and take the proper steps to ensure that others can pick up where they left off.
I've been working on Ubuntu for over ten years now, almost right from the very start; I'm Canonical's employee #17 due to working out a notice period in my previous job, but I was one of the founding group of developers. I occasionally tell the story that Mark originally hired me mainly to work on what later became Launchpad Bugs due to my experience maintaining the Debian bug tracking system, but then not long afterwards Jeff Waugh got in touch and said "hey Colin, would you mind just sorting out some installable CD images for us?". This is where you imagine one of those movie time-lapse clocks ... At some point it became fairly clear that I was working on Ubuntu, and the bug system work fell to other people. Then, when Matt Zimmerman could no longer manage the entire Ubuntu team in Canonical by himself, Scott James Remnant and I stepped up to help him out. I did that for a couple of years, starting the Foundations team in the process. As the team grew I found that my interests really lay in hands-on development rather than in management, so I switched over to being the technical lead for Foundations, and have made my home there ever since. Over the years this has given me the opportunity to do all sorts of things, particularly working on our installers and on the GRUB boot loader, leading the development work on many of our archive maintenance tools, instituting the +1 maintenance effort and proposed-migration, and developing the Click package manager, and I've had the great pleasure of working with many exceptionally talented people. However. In recent months I've been feeling a general sense of malaise and what I've come to recognise with hindsight as the symptoms of approaching burnout. I've been working long hours for a long time, and while I can draw on a lot of experience by now, it's been getting harder to summon the enthusiasm and creativity to go with that. I have a wonderful wife, amazing children, and lovely friends, and I want to be able to spend a bit more time with them. After ten years doing the same kinds of things, I've accreted history with and responsibility for a lot of projects. One of the things I always loved about Foundations was that it's a broad church, covering a wide range of software and with a correspondingly wide range of opportunities; but, over time, this has made it difficult for me to focus on things that are important because there are so many areas where I might be called upon to help. I thought about simply stepping down from the technical lead position and remaining in the same team, but I decided that that wouldn't make enough of a difference to what matters to me. I need a clean break and an opportunity to reset my habits before I burn out for real. One of the things that has consistently held my interest through all of this has been making sure that the infrastructure for Ubuntu keeps running reliably and that other developers can work efficiently. As part of this, I've been able to do a lot of work over the years on Launchpad where it was a good fit with my remit: this has included significant performance improvements to archive publishing, moving most archive administration operations from excessively-privileged command-line operations to the webservice, making build cancellation reliable across the board, and moving live filesystem building from an unscalable ad-hoc collection of machines into the Launchpad build farm. The Launchpad development team has generally welcomed help with open arms, and in fact I joined the ~launchpad team last year. So, the logical next step for me is to make this informal involvement permanent. As such, at the end of this year I will be moving from Ubuntu Foundations to the Launchpad engineering team. This doesn't mean me leaving Ubuntu. Within Canonical, Launchpad development is currently organised under the Continuous Integration team, which is part of Ubuntu Engineering. I'll still be around in more or less the usual places and available for people to ask me questions. But I will in general be trying to reduce my involvement in Ubuntu proper to things that are closely related to the operation of Launchpad, and a small number of low-effort things that I'm interested enough in to find free time for them. I still need to sort out a lot of details, but it'll very likely involve me handing over project leadership of Click, drastically reducing my involvement in the installer, and looking for at least some help with boot loader work, among others. I don't expect my Debian involvement to change, and I may well find myself more motivated there now that it won't be so closely linked with my day job, although it's possible that I will pare some things back that I was mostly doing on Ubuntu's behalf. If you ask me for help with something over the next few months, expect me to be more likely to direct you to other people or suggest ways you can help yourself out, so that I can start disentangling myself from my current web of projects. Please contact me sooner or later if you're interested in helping out with any of the things I'm visible in right now, and we can see what makes sense. I'm looking forward to this!

25 October 2014

Miriam Ruiz: Video game players and Gamers are different things

Even though the Wikipedia defines gamer as someone who partakes in interactive gaming, such as (predominantly) video games or board games , this doesn t really gets close to that term means socially at the moment. Going back to Wikipedia, we find that the video game subculture is a form of new media subculture that has been influenced by video games , so it might be quite accurate to define gamers as members of that subculture. You will find that most of the uses of the term gamer in the social networks and in the blogosphere refer to that. Please notice that, even though it is quite likely that most of the gamers play video games, the other way round does not need to be true and, in fact, it isn t. Not everyone who plays video games belongs to the video game subculture, shares their point of view, their values and aesthetics, or even know about it. Kind of like what happens with the word hacker . Not everyone who hacks around with a computer belongs to the hacker subculture. Mostly everyone who has access to the technology plays video games now. From babies and kids to grandparents. And people play them in every possible technological system around, not only on video game consoles or personal computers, but alse on mobile phones, tablets, web browsers. And many of those people who use different kind of technologies to play video games are not gamers. Not in the sense of belonging to the video game subculture. It is important to acknowledge that: that the video game subculture does not have the monopoly over video games or the video game developing industry anymore. As you can imagine, all this rand doesn t come from nowhere. During the last months, we have been witnessing a fight between some conservative core members of the video game subculture and people who want to bring some fresh air into the sociocultural elements of that subculture. Namely, that women shouldn t be discriminated inside it. As every time that a women raises her voice to complain about anything in the Internet, they have been subjected to insults, attacks, rape and death threats, etc. I m talking about something called #GamerGate, and even though I m not going to get into it, I will provide some URLs in case someone might be interested. Please acknowledge that not all the points of view might be represented in this list (in fact, they are not, as I won t be promoting in my blog things that I severely disagree with), so search the web for more information if you want to get that. I ve never been a gamer myself, meaning part of the subculture I mentioned. At some point I was probably closer tho the core values they had then than I am now. In any case, video games have already consolidated themselves as an important part of current culture, entertainment, education and socialization, and are definitely here to stay. That will probably mean that the percentage of gamers (members of the video game subculture) will become smaller. as the number of non-gamer video game players keeps raising.

24 October 2014

Daniel Pocock: Positive results from Outreach Program for Women

In 2013, Debian participated in both rounds of the GNOME Outreach Program for Women (OPW). The first round was run in conjunction with GSoC and the second round was a standalone program. The publicity around these programs and the strength of the Google and Debian brands attracted a range of female candidates, many of whom were shortlisted by mentors after passing their coding tests and satisfying us that they had the capability to complete a project successfully. As there are only a limited number of places for GSoC and limited funding for OPW, only a subset of these capable candidates were actually selected. The second round of OPW, for example, was only able to select two women. Google to the rescue Many of the women applying for the second round of OPW in 2013 were also students eligible for GSoC 2014. Debian was lucky to have over twenty places funded for GSoC 2014 and those women who had started preparing project plans for OPW and getting to know the Debian community were in a strong position to be considered for GSoC. Chandrika Parimoo, who applied to Debian for the first round of OPW in 2013, was selected by the Ganglia project for one of five GSoC slots. Chandrika made contributions to PyNag and the ganglia-nagios-bridge. Juliana Louback, who applied to Debian during the second round of OPW in 2013, was selected for one of Debian's GSoC 2014 slots working on the Debian WebRTC portal. The portal is built using JSCommunicator, a generic HTML5 softphone designed to be integrated in other web sites, portal frameworks and CMS systems. Juliana has been particularly enthusiastic with her work and after completing the core requirements of her project, I suggested she explore just what is involved in embedding JSCommunicator into another open source application. By co-incidence, the xTuple development team had decided to dedicate the month of August to open source engagement, running a program called haxTuple. Juliana had originally applied to OPW with an interest in financial software and so this appeared to be a great opportunity for her to broaden her experience and engagement with the open source community. Despite having no prior experience with ERP/CRM software, Juliana set about developing a plugin/extension for the new xTuple web frontend. She has published the extension in Github and written a detailed blog about her experience with the xTuple extension API. Participation in DebConf14 Juliana attended DebConf14 in Portland and gave a presentation of her work on the Debian RTC portal. Many more people were able to try the portal for the first time thanks to her participation in DebConf. The video of the GSoC students at DebConf14 is available here. Continuing with open source beyond GSoC Although GSoC finished in August, xTuple invited Juliana and I to attend their annual xTupleCon in Norfolk, Virginia. Google went the extra mile and helped Juliana to get there and she gave a live demonstration of the xTuple extension she had created. This effort has simultaneously raised the profile of Debian, open source and open standards (SIP and WebRTC) in front of a wider audience of professional developers and business users. Juliana describes her work at xTupleCon, Norfolk, 15 October 2014 It started with OPW The key point to emphasize is that Juliana's work in GSoC was actually made possible by Debian's decision to participate in and promote Outreach Program for Women in 2013. I've previously attended DebConf myself to help more developers become familiar with free and open RTC technology. I wasn't able to get there this year but thanks to the way GSoC and OPW are expanding our community, Juliana was there to help out.

23 October 2014

Gunnar Wolf: Listadmin *YES*

Petter posted yesterday about Listadmin, the quick way to moderate mailman lists. Petter: THANKS. I am a fan of automatization. But, yes, I had never thouguht of doing this. Why? Don't know. But this is way easier than using the Web interface for Mailman:
$ listadmin 
fetching data for conoc_des@my.example.org ... nothing in queue
fetching data for des_polit_pub@my.example.org ... nothing in queue
fetching data for econ_apl@my.example.org ... nothing in queue
fetching data for educ_ciencia_tec@my.example.org ... nothing in queue
fetching data for est_hacend_sec_pub@my.example.org ... 
[1/1] ============== est_hacend_sec_pub@my.example.org ======
From:     sender@example.org                                            
Subject:  Invitaci n al Taller Insumo Producto                          
Reason:   El cuerpo del mensaje es demasiado grande: 777499    Spam? 0  
Approve/Reject/Discard/Skip/view Body/Full/jump #/Undo/Help/Quit ? a
Submit changes? [yes] 
fetching data for fiscal_fin@my.example.org ... nothing in queue
fetching data for historia@my.example.org ... nothing in queue
fetching data for industrial@my.example.org ... nothing in queue
fetching data for medio_amb@my.example.org ... nothing in queue
fetching data for mundial@my.example.org ... nothing in queue
fetching data for pol_des@my.example.org ... nothing in queue
fetching data for sec_ener@my.example.org ... nothing in queue
fetching data for sec_prim@my.example.org ... nothing in queue
fetching data for trab_tec@my.example.org ... nothing in queue
fetching data for urb_reg@my.example.org ... nothing in queue
fetching data for global@my.example.org ... nothing in queue
I don't know how in many years of managing several mailing lists I never thought about this! I'm echoing this, as I know several of my readers run mailman as well, and might not be following Planet Debian.

Dirk Eddelbuettel: Introducing Rocker: Docker for R

You only know two things about Docker. First, it uses Linux
containers. Second, the Internet won't shut up about it. -- attributed to Solomon Hykes, Docker CEO

So what is Docker? Docker is a relatively new open source application and service, which is seeing interest across a number of areas. It uses recent Linux kernel features (containers, namespaces) to shield processes. While its use (superficially) resembles that of virtual machines, it is much more lightweight as it operates at the level of a single process (rather than an emulation of an entire OS layer). This also allows it to start almost instantly, require very little resources and hence permits an order of magnitude more deployments per host than a virtual machine. Docker offers a standard interface to creation, distribution and deployment. The shipping container analogy is apt: just how shipping containers (via their standard size and "interface") allow global trade to prosper, Docker is aiming for nothing less for deployment. A Dockerfile provides a concise, extensible, and executable description of the computational environment. Docker software then builds a Docker image from the Dockerfile. Docker images are analogous to virtual machine images, but smaller and built in discrete, extensible and reuseable layers. Images can be distributed and run on any machine that has Docker software installed---including Windows, OS X and of course Linux. Running instances are called Docker containers. A single machine can run hundreds of such containers, including multiple containers running the same image. There are many good tutorials and introductory materials on Docker on the web. The official online tutorial is a good place to start; this post can not go into more detail in order to remain short and introductory.

So what is Rocker? rocker logo At its core, Rocker is a project for running R using Docker containers. We provide a collection of Dockerfiles and pre-built Docker images that can be used and extended for many purposes. Rocker is the the name of our GitHub repository contained with the Rocker-Org GitHub organization. Rocker is also the name the account under which the automated builds at Docker provide containers ready for download.

Current Rocker Status

Core Rocker Containers The Rocker project develops the following containers in the core Rocker repository
  • r-base provides a base R container to build from
  • r-devel provides the basic R container, as well as a complete R-devel build based on current SVN sources of R
  • rstudio provides the base R container as well an RStudio Server instance
We have settled on these three core images after earlier work in repositories such as docker-debian-r and docker-ubuntu-r.

Rocker Use Case Containers Within the Rocker-org organization on GitHub, we are also working on
  • Hadleyverse which extends the rstudio container with a number of Hadley packages
  • rOpenSci which extends hadleyverse with a number of rOpenSci packages
  • r-devel-san provides an R-devel build for "Sanitizer" run-time diagnostics via a properly instrumented version of R-devel via a recent compiler build
  • rocker-versioned aims to provided containers with 'versioned' previous R releases and matching packages
Other repositories will probably be added as new needs and opportunities are identified.

Deprecation The Rocker effort supersedes and replaces earlier work by Dirk (in the docker-debian-r and docker-ubuntu-r GitHub repositories) and Carl. Please use the Rocker GitHub repo and Rocker Containers from Docker.com going forward.

Next Steps We intend to follow-up with more posts detailing usage of both the source Dockerfiles and binary containers on different platforms. Rocker containers are fully functional. We invite you to take them for a spin. Bug reports, comments, and suggestions are welcome; we suggest you use the GitHub issue tracker.

Acknowledgments We are very appreciative of all comments received by early adopters and testers. We also would like to thank RStudio for allowing us the redistribution of their RStudio Server binary. Published concurrently at rOpenSci blog and Dirk's blog.

Authors Dirk Eddelbuettel and Carl Boettiger

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

22 October 2014

Petter Reinholdtsen: listadmin, the quick way to moderate mailman lists - nice free software

If you ever had to moderate a mailman list, like the ones on alioth.debian.org, you know the web interface is fairly slow to operate. First you visit one web page, enter the moderation password and get a new page shown with a list of all the messages to moderate and various options for each email address. This take a while for every list you moderate, and you need to do it regularly to do a good job as a list moderator. But there is a quick alternative, the listadmin program. It allow you to check lists for new messages to moderate in a fraction of a second. Here is a test run on two lists I recently took over:
% time listadmin xiph
fetching data for pkg-xiph-commits@lists.alioth.debian.org ... nothing in queue
fetching data for pkg-xiph-maint@lists.alioth.debian.org ... nothing in queue
real    0m1.709s
user    0m0.232s
sys     0m0.012s
%
In 1.7 seconds I had checked two mailing lists and confirmed that there are no message in the moderation queue. Every morning I currently moderate 68 mailman lists, and it normally take around two minutes. When I took over the two pkg-xiph lists above a few days ago, there were 400 emails waiting in the moderator queue. It took me less than 15 minutes to process them all using the listadmin program. If you install the listadmin package from Debian and create a file ~/.listadmin.ini with content like this, the moderation task is a breeze:
username username@example.org
spamlevel 23
default discard
discard_if_reason "Posting restricted to members only. Remove us from your mail list."
password secret
adminurl https:// domain /mailman/admindb/ list 
mailman-list@lists.example.com
password hidden
other-list@otherserver.example.org
There are other options to set as well. Check the manual page to learn the details. If you are forced to moderate lists on a mailman installation where the SSL certificate is self signed or not properly signed by a generally accepted signing authority, you can set a environment variable when calling listadmin to disable SSL verification:
PERL_LWP_SSL_VERIFY_HOSTNAME=0 listadmin
If you want to moderate a subset of the lists you take care of, you can provide an argument to the listadmin script like I do in the initial screen dump (the xiph argument). Using an argument, only lists matching the argument string will be processed. This make it quick to accept messages if you notice the moderation request in your email. Without the listadmin program, I would never be the moderator of 68 mailing lists, as I simply do not have time to spend on that if the process was any slower. The listadmin program have saved me hours of time I could spend elsewhere over the years. It truly is nice free software. As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b. Update 2014-10-27: Added missing 'username' statement in configuration example. Also, I've been told that the PERL_LWP_SSL_VERIFY_HOSTNAME=0 setting do not work for everyone. Not sure why.

Next.

Previous.