Search Results: "tar"

23 February 2014

Sylvain Le Gall: Release of OASIS 0.4.2

I am happy to announce the release of OASIS v0.4.2. Logo OASIS small OASIS is a tool to help OCaml developers to integrate configure, build and install systems in their projects. It should help to create standard entry points in the source code build system, allowing external tools to analyse projects easily. This tool is freely inspired by Cabal which is the same kind of tool for Haskell. You can find the new release here and the changelog here. More information about OASIS in general on the OASIS website. Here is a quick summary of the important changes: BSD3 and BSD4 are still valid but marked as deprecated. When one of the modules of a library has the name of the library, ocamlbuild tends to just transform this module into a .cmxs. Now, the use of a .mldylib fix that problem and the .cmxs really contains all modules of the library. I have published a quick intermediate version 0.4.1, a few days after the previous release. This was a bug fix related to threads. I also decided to skip the release of last month. I was in the US at this time and didn't have time to work enough on OASIS (christmas vacation and a travel to the US). This month I am back on track. This new version doesn't feature a lot of visible changes. I mostly work on the command line interface code, in order to be able to create external plugins. A first external plugin is almost ready, but need some more polishing before release. This first plugin project is a port of the script that I have used for a long time and that was present in the source code of oasis (oasis-dist.ml). It will be a project on its own and have a different release cycle. The point of this plugin is to create .tar.gz out of an OASIS enabled project. I also must admit that I am very happy to see contributors sending me pull-request through GitHub. It helps me a lot and I also realize that the learning curve to enter OASIS code is steep. This last point is something I will try to improve.

22 February 2014

Petter Reinholdtsen: New home and release 1.0 for netgroup and innetgr (aka ng-utils)

Many years ago, I wrote a GPL licensed version of the netgroup and innetgr tools, because I needed them in Skolelinux. I called the project ng-utils, and it has served me well. I placed the project under the Hungry Programmer umbrella, and it was maintained in our CVS repository. But many years ago, the CVS repository was dropped (lost, not migrated to new hardware, not sure), and the project have lacked a proper home since then. Last summer, I had a look at the package and made a new release fixing a irritating crash bug, but was unable to store the changes in a proper source control system. I applied for a project on Alioth, but did not have time to follow up on it. Until today. :) After many hours of cleaning and migration, the ng-utils project now have a new home, and a git repository with the highlight of the history of the project. I published all release tarballs and imported them into the git repository. As the project is really stable and not expected to gain new features any time soon, I decided to make a new release and call it 1.0. Visit the new project home on https://alioth.debian.org/projects/ng-utils/ if you want to check it out. The new version is also uploaded into Debian Unstable.

Dirk Eddelbuettel: RcppBDT 0.2.2

A new maintenance release of the RcppBDT package is now on CRAN. There is no new code in this; it mainly accomodates a request by CRAN to standardize the \code Imports: and \code Depends: relationships in the \code DESCRIPTION file. We also updated the build process now that we can use Rcpp 0.11.0. Having said that, there is a lot of new code in the git repository but I guess I'll talk about that when I can make a new upload. Yesterday, this balked because win-builder felt the need for a long long int in 64-bit mode which we can provide once C++11 will be permitted under R 3.1.0, due out in April. Oddly, no such need arose in 64-bit Linux. Oh well. The complete NEWS entry is below:
Changes in version 0.2.2 (2014-02-21)
  • Updates to current CRAN Policy standards for Depends: and Imports: in DESCRIPTION
  • Also take advantage of new Rcpp 0.11.0 build options
  • New code remains in GitHub repo master branch until R 3.1.0 is release so that we can use C++11 in order to get the long long int support without which the package does not build on Windows (whereas Linux is fine)
Courtesy of CRANberries, there is also a diffstat report for 0.2.2 relative to 0.2.1. As always, feedback is welcome and the rcpp-devel mailing list off the R-Forge page for Rcpp is the best place to start a discussion.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Vasudev Kamath: First experience of driving in Bangalore

So finally I did it, I managed to ride the 2-wheeler in Bangalore roads!. Officially I've got a license to drive 2 and 4 wheelers back in 2011 but I was never confident enough to ride the vehicle on road and in traffic. Partially this is because I don't own a vehicle and I've never practiced driving, and partially because of road and traffic conditions in India I was bit afraid to do so. So today morning my roommate and me planned to go to temple in the early morning. We got up at 530 in the morning and we were ready to drive to Mahalaxmi Layout at 615. Since during this time traffic is very less I asked my roommate shall I drive and he agreed!. So I drove the bike from Malleshwaram to Mahalaxmi Layout and it was fun. I did had hick up in uphill driving as my mind was not co-ordinating well with breaks and gear. I did have hickups in first gear too but I guess this will be alright once I get hold of things. I'm still not having enough confidence to drive during the peak hours because people are not at all following traffic rules. They just change the lane without any hints or just come to middle of road from side roads without looking at other side to see if there is a vehicle is coming from other side of the road (yes they don't have courtesy to wait). And finally they can shout at persons who is actually following rules. In general very less people have traffic discipline in India. But I will adapt to this once I start driving so I'm now all set to get my 2 wheeler soon and ride on!

Steve Kemp: Changing my stack ..

For the past few years I've hosted all my websites in a "special" way: The webserver I chose initially was thttpd, which gained points because it was small, auditable, and simple to launch. Something like this was my recipe:
#!/bin/sh
exec thttpd -D -C /srv/steve.org.uk/thttpd.conf
Unfortunately thttpd suffers from a few omissions, most notably it doesn't support either "Keep-Alive", or "Compression" (i.e. gzip/deflate), so it would always be slower than I wanted. On the plus side it was simple to use, supported CGI scripts, and served me well once I'd patched it to support X-Forwarded-For for IPv6 connections. Recently I setup a server optimization site and was a little disappointed that the site itself scored poorly on Google's page-speed test. So I removed thttpd for that site, and replacing it with nginx. The end result was that the site scored 98/100 on Google's page-speed test. Progress. Unfortunately I couldn't do that globally because nginx doesn't support old-school plain CGI scripts. So last night I removed both nginx and thttpd, and now every site on my box is hosted using lighttpd. There weren't too many differences in the setup, though I had to add some rules to add caching for *.css, etc, and some of my code needed updating. Beyond that today I've setup a dedicated docker host - which allows me to easily spin up containers. Currently I've got graphite monitoring for my random hosts, and a wordpress guest for plugin development/testing. Now to go back to reading Off to be the wizard .. - not as good as Rick Cook's wizardry series (which got less good as time went on, but started off strongly), but still entertaining.

21 February 2014

Jakub Wilk: For those who care about snowclones

Instances of the for those who care about X snowclone on Debian mailing lists:

Guido G nther: Truncating git history

When starting to work on a new project I start from an empty git repository right away so I can try out different ideas, revert easily, can diff against old versions (to check if I missed something) and have a commit history to record fixmes and todos. However when making the repo public these things are not of much interest anymore so I truncate the history. To be on the save side I want to keep that history locally though. Assuming the repo is on master, I do:
# 1.) Move the old master out of the way
git branch -m branch master oldmaster
# 2.) Get the current tree at HEAD
tree=$(git rev-parse HEAD^ tree )
# 3.) Create a new commit without ancestry
commit=$(git commit-tree -m "Initial commit" $tree)
# 4.) Make this the new master
git branch master $commit
# 5.) Switch to the new branch
git checkout master
Done. One can now add a remote and push the master branch. The old history is still there locally on the completely detached commit history ending at oldmaster:
$ git log --pretty=short --graph --decorate master oldmster
* commit d5cf3371eaefbaa8efac10c0fb9e7597da17b423 (HEAD, master)
  Author: Guido G nther <agx@sigxcpu.org>
      Initial commit
* commit 64103ff72bde13d7ec4cf0489ad2a80f3ac249d3 (oldmster)
  Author: Guido G nther <agx@sigxcpu.org>
 
      Another uninteresting commit message
 
* commit bd7332a79380bb217eca09cbd7f6ff0e5174deb8
  Author: Guido G nther <agx@sigxcpu.org>
 
      Uninteresting commit message
 
... <more old history>
Update: Uli Heller pointed out that this is the same as using git checkout's --orphan option.

Andrew Pollock: [life] Day 24, Rhythmic gymnastics, BJJ, lunch with the great grandparents, Science Friday

Wowsers, today was a pretty full, but laid back day. Zoe has this small wrist-sized ring with a bunch of different coloured long ribbons on it that she got from a birthday party a while ago. She plays with it every now and then, and uses it to torment Smudge. This morning she was dancing around with it, and it made me think of rhythmic gymnastics, so using my trusty Chromecast, I dug up some Youtube coverage from the 2012 Olympics. Zoe loved the sparkly costumes and the routines. The first video alternated between ball and hoop routines. Zoe went out and rummaged around her craft drawers and came back with a styrofoam ball and was using her crown from yesterday as a hoop. It kept her occupied for about an hour. I watched the first three competitors with her and she continued to watch it while I had a shower. I see a hula hoop in her future. We biked over to her Brazilian Jiu Jitsu class and had another really good class. I think Zoe's really enjoying the mix of gymnastics-style activities and game play. She also got to tackle me today, which she thoroughly enjoyed. The instructor suggested we might like to try out one of the 4-7 year old proper martial arts classes, so we'll try that in the coming weeks. We biked home the scenic way again and picked up a few additional supplies for lunch. When we got home, Zoe went off and played with some of her toys on her own. It's great that she's getting more independent and self-directed like that. I'd invited Zoe's great-grandparents (on her mother's side) over for lunch today. I think it's wonderful that Zoe has great-grandparents who are still alive and able to be in her life. We had a nice lunch and they wanted to see the kinetic sand I'd blogged about, so we got that out after lunch for a play. Then Zoe wanted to watch some more rhythmic gymnastics with them, so we did that for a bit. After they left, I swapped over to rhythmic gymnastics with the ribbon and clubs. The ribbon is my personal favourite one to watch, and I think Zoe enjoyed it too, because she immediately started waving her ribbons all over the place. I trialled skipping her nap today, and instead called watching the gymnastics "quiet time". After quiet time, we did our Science Friday activity, which was messing around with conductive play dough. Zoe had about as much fun just messing around with the play dough as she did running electricity through it, and enjoyed mushing the non-conductive white play dough in with the conductive blue play dough. We did a few experiments, notably one with three LEDs in series, and then shorting the play dough at either end of the series to see what it did to the lights. She enjoyed wearing her lab coat and safety glasses. I'd wanted to do bottle rockets today, because we'd built a cardboard box rocket earlier in the week, and that seemed like a good theme to stick with, but I was ill-prepared, and it was too hot anyway. I want to do Diet Coke and Mentos and reuse the Coke bottles for the bottle rockets, so that'll have to be in a few weeks' time. I prepped dinner on and off while Zoe played with the play dough, so it was all ready to pop in the oven nice and early. After I cleaned up the mess from the play dough, we played with her coloured peg board game for a bit, and then she wanted to see how the sofa bed folded out, so we did that I used it as an opportunity to give it a thorough vacuuming. After that she watched a bit more TV while dinner was in the oven. After dinner, we walked down to the Hawthorne Garage for a babyccino. Zoe had to pat every dog that crossed our path. Zoe didn't seem too worse for wear for having dropped her nap (which was to be expected, it's only a recent phenomenon for her to nap at home), but she was showing signs of possibly coming down with a cold (being a bit sniffy, without having obvious signs of a runny nose) so I put her to bed about half an hour early. She went down pretty easily, which was a pleasant change. I did miss the 2 hour opportunity to get things done during the day though. C'est la vie. The thing I liked most today is Zoe's independently going and playing with toys that she's had for ages, but not really shown much interest in. She's had this Melissa and Doug board with interchangeable sheets with animals on them and a bunch of different colour octagonal plastic pegs that you're supposed to match the coloured pegs to whatever colours are on the sheet you put on the board. We did that this afternoon, and this morning she played a lace up game where you thread shoelaces through some thick cardboard outlines of animals. I think she got both for Christmas in 2012.

20 February 2014

Francois Marier: Hardening ssh Servers

Basic configuration There are a few basic things that most admins will already know (and that tiger will warn you about if you forget):
  • only allow version 2 of the protocol
  • disable root logins
  • disable password authentication
This is what /etc/ssh/sshd_config should contain:
Protocol 2
PasswordAuthentication no
PermitRootLogin no

Whitelist approach to giving users ssh access To ensure that only a few users have ssh access to the server and that newly created users don't have it enabled by default, create a new group:
addgroup sshuser
and then add the relevant users to it:
adduser francois sshuser
Finally, add this to /etc/ssh/sshd_config:
AllowGroups sshuser

Deterring brute-force (or dictionary) attacks One way to ban attackers who try to brute-force your ssh server is to install the fail2ban package. It keeps an eye on the ssh log file (/var/log/auth.log) and temporarily blocks IP addresses after a number of failed login attempts. Another approach is to hide the ssh service using Single-Packet Authentication. I have fwknop installed on some of my servers and use small wrapper scripts to connect to them.

Using restricted shells For those users who only need an ssh account on the server in order to transfer files (using scp or rsync), it's a good idea to set their shell (via chsh) to a restricted one like rssh. Should they attempt to log into the server, these users will be greeted with the following error message:
This account is restricted by rssh.
Allowed commands: rsync 
If you believe this is in error, please contact your system administrator.
Connection to server.example.com closed.

Restricting authorized keys to certain IP addresses In addition to listing all of the public keys that are allowed to log into a user account, the ~/.ssh/authorized_keys file also allows (as the man page points out) a user to impose a number of restrictions. Perhaps the most useful option is from which allows a user to restrict the IP addresses which can login using a specific key. Here's what one of my authorized_keys looks like:
from="192.0.2.2" ssh-rsa AAAAB3Nz...zvCn bot@example
You may also want to include the following options to each entry: no-X11-forwarding, no-user-rc, no-pty, no-agent-forwarding and no-port-forwarding.

Increasing the amount of logging The first thing I'd recommend is to increase the level of verbosity in /etc/ssh/sshd_config:
LogLevel VERBOSE
which will, amongst other things, log the fingerprints of keys used to login:
sshd: Connection from 192.0.2.2 port 39671
sshd: Found matching RSA key: de:ad:be:ef:ca:fe
sshd: Postponed publickey for francois from 192.0.2.2 port 39671 ssh2 [preauth]
sshd: Accepted publickey for francois from 192.0.2.2 port 39671 ssh2 
Secondly, if you run logcheck and would like to whitelist the "Accepted publickey" messages on your server, you'll have to start by deleting the first line of /etc/logcheck/ignore.d.server/sshd. Then you can add an entry for all of the usernames and IP addresses that you expect to see. Finally, it is also possible to log all commands issued by a specific user over ssh by enabling the pam_tty_audit module in /etc/pam.d/sshd:
session required pam_tty_audit.so enable=francois
However this module is not included in wheezy and has only recently been re-added to Debian.

Identitying stolen keys One thing I'd love to have is a way to identify a stolen public key. Given the IP restrictions described above, if a public key is stolen and used from a different IP, I will see something like this in /var/log/auth.log:
sshd: Connection from 198.51.100.10 port 39492
sshd: Authentication tried for francois with correct key but not from a permitted host (host=198.51.100.10, ip=198.51.100.10).
sshd: Failed publickey for francois from 198.51.100.10 port 39492 ssh2
sshd: Connection closed by 198.51.100.10 [preauth]
So I can get the IP address of the attacker (likely to be a random VPS or a Tor exit node), but unfortunately, the key fingerprints don't appear for failed connections like they do for successful ones. So I don't know which key to revoke. Is there any way to identify which key was used in a failed login attempt or is the solution to only ever have a single public key in each authorized_keys file and create a separate user account for each user?

Andrew Pollock: [life] Day 23, Playgroup and swimming and some more taxes

Today was a pretty good day. I've found that I quite enjoyed it. Zoe woke up a bit early, around 5:30am. She decided that despite the ridiculous heat, she wanted porridge for breakfast, with cherries in it, so we pitted the cherries and threw it all into the Thermomix to cook while she got dressed. After breakfast and the obligatory bit of TV, we biked over to Playgroup. Zoe's got a few toys that she's taken a liking to, and immediately sought them out. She also seems to really enjoy just running the length of the hall after Playgroup is all done and everything has been packed away and most of the kids are gone. Today the craft activity was making paper crowns, and she wanted to that immediately upon arrival. After we got home, while I was preparing lunch, she blinged it up a bit more with some stuff from home. She took a 2 hour nap after lunch, and I did some more work on my taxes, and then, because it was still stupidly hot, we biked over to Colmslie pool for a swim. Zoe's swimming progress continues in leaps and bounds. She's had one lesson at her new level, and she's started with a kickboard and is doing strokes, so I borrowed a kickboard from the pool, and she did really well with it. We also just fooled around with the kickboard a bit. I took it to the bottom of the pool and launched it out of the water with its own buoyancy, and Zoe tried standing on it under water like a skateboard. The sinkies were also popular again. We grabbed a few things for dinner from the Hawthorne Garage on the way home, and I managed to get Zoe to bed a little bit early. I thought I was going to have a smooth bedtime, but unfortunately, as is a common occurrence at the moment, after lights out, she started crying for her mother. It took about 15 minutes to get her from near-hysterical to calm and drowsy. I'm starting to think I might need to either shorten or eliminate the afternoon naps. One of her complaints recently has been that she's had trouble sleeping at bedtime. That said, she still seems to get to sleep relatively quickly once she gets into the right frame of mind.

19 February 2014

Mario Lang: Enhancing Hub to support GitHub social features

In my article Contributing on GitHub I recently explained that I can not use certain social features of GitHub on their website directly. I also explained that I make great use of Hub to circumvent some accessibility issues I have with GitHub. Well, it turns out that adding star, unstar, follow and unfollow to Hub was actually a lot easier then I thought. It is long ago (probably 10 years) that I touched Ruby code the last time. However, since I do learn a lot from examples, I was actually able to add what I needed just by looking at surrounding code. Well, I had to lookup how you add an element to the end of an array (push) but that was basically it. You can have a look at the final outcome in Hub pull request #490. The code resides in the social branch of my hub fork. If you already have an OAuth token for hub, you need to login to GitHub and edit your token to add the user:follow scope if you want to use the follow and unfollow commands. Interestingly, this type of operation is accessible enough to still work with Lynx! I hope this gets merged, I don't particularily like to maintain my own forks of quite obvious functionality. Anyway, problem solved, mostly. All I am missing now is the ability to delete a repository. A dangerous feature, so probably controversial. Anyway, something for another day.

Andrew Pollock: [life] Day 22, Kindergarten, cleaning and Officeworks outing

Today was my "clean the house" day. I also had a chiropractic adjustment first thing, and a massage before lunch. I biked to Kindergarten to pick up Zoe, and found her holding court with a bunch of her classmates, giving a show and tell session about the Cinderella snow globe and echidna plush toy she'd taken to Kindergarten. The kids were taking turns to raise their hand and ask questions. I was out of earshot, so I couldn't really make out what was being said, but it all looked very cute from where I was standing. She didn't have a nap today. Sarah's going down to Tasmania until early next month, hiking, and so I've got Zoe for the next 12 nights. This is the longest she will have been away from her mother by quite a significant amount. On top of this, she's been doing rather poorly at bedtime with me the last few nights I've had her. I wanted to try and minimise the emotional impact for both of us as much as possible, so I wanted to make an Advent-style calendar to count down the number of sleeps until Mummy got back. After we got home from Kindergarten, we drove over to Officeworks to pick up some bits and pieces. It ended up being quite a long outing because it took a while for me to figure out how to convince the self-service photo printing kiosk to actually print photos. Zoe parked herself in the kid's colouring area, and she was quite happy there while I went looking for a few things, but after a while wandered off to find me and we spent some time looking for each other. I was pretty impressed with how well she went looking for me. She didn't freak out, and was just wandering around the store looking for me. We had a talk about staying put. Then we called grandma from the carpark to wish her a happy birthday before we headed home. That ended up being quite a lengthy outing, and I started dinner as soon as we got home. Bedtime actually went quite smoothly tonight for a change. No idea if the poster or the lack of a nap helped or not.

18 February 2014

Chris Lamb: Estimating Training Stress Score (TSS) for running on Strava

In cycling, a ride's Training Stress Score is a function of that ride's duration, average power and the intensity of the ride relative to the rider's capability. This Slowtwitch article provides a good overview on how intensity and TSS is calculated on a bike. However, having TSS values for other sports allows a multisport athelete to take into consideration the physiological cost of activities in different sports. This is achieved by ensuring, say, 50 TSS on the bike "counts" the same as a 50 TSS run. This can be used to simply determine the length, intensity and scheduling of the athletes next workout (to ensure adequate recovery) regardless of the combination of sports, or to identify the athlete's long-term tolerance to and targets for training load using metrics such as Chronic Training Load. To make this possible when using Strava, I wrote a Chrome extension that estimates the TSS score of a run from its Grade Adjusted Pace distribution:
https://chris-lamb.co.uk/wp-content/2014/strava-rtss.png

The "TSS (estimated)" value is calculated by the extension.

Source code is available. If you found this Strava extension useful, you might like my extensions to quickly switch between metric and imperial units or to change the default comparison filter.

Christoph Egger

Writing this because there seems to be no correct documentation on the relevant google websites and it turns out to be non-trivial. Our goal here is to unsubscribe from a ordinary google group. Mails from the google group contain the quoted footer:
-- 
You received this message because you are subscribed to the Google
Groups "FOO" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to FOO+unsubscribe@googlegroups.com.
Visit this group at http://groups.google.com/group/FOO
For more options, visit https://groups.google.com/groups/opt_out.
Seems easy enough, so let's send a Mail to this FOO+unsubscribe address. Back comes a E-Mail:
From: FOO <FOO+unsubconfirm@googlegroups.com>
Subject: Unsubscribe request for FOO [ EJzZjpgFhDHd9seTdRA0 ]
To: Christoph Egger <christoph@example.com>
Date: Tue, 18 Feb 2014 18:55:24 +0000 (38 minutes, 53 seconds ago)
 [Leave This Group]
Visit Go 
[Start] your own group, [visit] the help center, or [report]
abuse.
So click on the [Leave This Group] link and be done? Unfortunately not. Looking at the link you notice it's called http://groups.google.com/group/FOO/subscribe -- no token and "subscribe"? I actually want to unsubscribe! And indeed, clicking gets an Interface that offers to "Enter the email address to subscribe:" + Captcha. And whatever it does, it -- of course -- doesn't unsubscribe. (My guess is, it would actually work if you had a real google account associated with that email address and were logged in to that account but there's no way of verifying this as already the first condition is false in this case) Now if you disable HTML completely for the email, a totally different content emerges:
Hello christoph@example.com,
We have received your request to unsubscribe from FOO. In order for us to complete the request, please reply to this email or visit the following confirmation URL:
http://groups.google.com/group/FOO/subscribe
If you have questions related to this or any other Google Group, visit the Help Center at http://groups.google.com/support/.
Thanks,
Google Groups
Still the non-functional link, however it also mentions a different solution: "please reply to this email" which was not present in the HTML mail at all. And it works.

Thomas Koch: Waiting for a STATE folder in the XDG basedir spec

The XDG Basedirectory specification proposes default homedir folders for the categories DATA (~/.local/share), CONFIG (~/.config) and CACHE (~/.cache). One category however is missing: STATE. This category has been requested several times but nothing happened.
Examples for state data are:
The missing STATE category is especially annoying if you're managing your dotfiles with a VCS (e.g. via VCSH) and you care to keep your homedir tidy.
If you're as annoyed as me about the missing STATE category, please voice your opinion on the XDG mailing list.
Of course it's a very long way until applications really use such a STATE directory. But without a common standard it will never happen.

Mario Lang: Conntributing on GitHub

I really like to read code written by different people. I've always been the type of learner that can cover a lot of ground by looking at examples. A few of the programming tricks I acquired over time have definitely been learnt by reading programs written by other people. There are of course cases of disagrement with coding style (lets put that very mildly), so not every project is fun to read. But I tend to find projects I like enough to actually look at their source code. Now, if you get into this habit, there is also the occasional spotting of a bug. Many are really cosmetic, like typos in comments or documentation. Others are maybe more serious, who knows what can happen if you read a few lines of code written by someone you have never met in person. This is one of the original arguments of open source and free software. It basically goes like: "People will read your code and report problems." I know that happens, but I also know that there are cases of people finding relatively minor things during a small code review, which they don't report, or which get lost in some developers INBOX. I remember waiting months to see memory corruption fixes for a particular project I submitted by email to actually appear in the development repository. In some cases you really need to track your patches and prod someone to remind them about your contribution if need be. True, some of these issues are probably minor enough to not really hurt if they get lost, but I have a feel that a lot of minor things also sort of add up. This is where GitHub comes into play. We have had project hosting services since at least a decade now, but none of them (to my experience) have ever made it so easy for all involved parties to submit and merge changes into projects. Pull requests are just amazing. Well, at least for me, they are since a few months. I have a sort of mixed hate-love relationship with GitHub when it comes to a topic that is very important to me, Accessibility. The last project hosting site I really liked for their simple web interface (read, still usable with Lynx) was Google Code. That was in the days when Google still seemed to care about such things. These days, their newer services are next to unusable to me (with the tools I prefer). SourceForge was always a catastrophy. Many things were cumbersome to do, because the site is so loaded with links to all sorts of things I never need. However, at the time I was using it, all the things I really needed did work. GitHub is a different beast, in a different era of the internet. While I can read project pages and a few basic things perfectly fine, some parts of the GitHub site are simply unusable for no particular reason. For instance, I can edit (some of my) preferences and even save them. But I can not star a project, nor can I follow other developers. Invoking these links gives me a 404, which seems to indicate that some JavaScript (read, client side code execution) is required which my browser does not perform. However, I really wonder why. Because something like "I want to star this project" will eventually have to end up as a request on the server anyway, so it seems technically unnecessary to rely on Javascript for such an operation to succeed. Luckily, these days, hip projects do have an API. An GitHub is no exception. I want to emphasis that an API alone is not an accessibility fix, since you rarely have the time to implement a clinet for a random API, just to get access to a service you want to use. Sometimes you do, and an API is a great enabler in such situations, but a reference client with a rich feature-set is much better. I have to admit that I haven't evaluated any of the possible alternatives, but one client I found for GitHub's API did really boost my productivity in the last year. I am talking about Hub. It is a simple wrapper around Git that adds a few and enhances some other git commands. Most importantly, hub fork will fork the current checkout on GitHub, and hub pull-request allows you to quickly (and I really mean quickly!) open a pull request for some commits you've just done to your fork of some project. This is very helpful, and very accessible. While it does not solve all my accessibility problems with GitHub, it at least enables me to take part in the GitHub workflow. I can easily fork projects and submit pull reuqests for changes I did. And that is what I did, at least a bit, in roughly the past year. I am not sure exactly, but I think it was spring when someone made me aware of Hub. One approach would be to check my GitHub activity graph, but I probably don't have to mention that it is not accessible. Here is a list of some of my contributions to open source projects on GitHub since I discovered a good client for the GitHub API: To summarize: I love GitHub for what it does to the free software community. I think the simplicity (and transparency) of contributions is to the benefit of free and open source software. However, I also hate GitHub sometimes when I need to ask a friend to do relatively trivial changes for me. What I actually want, is a command-line client like Hub that makes most or even all functionality provided on the website available. True, some things like graphs are not immediately easy to adapt. But I really shouldn't need to use a modern full-fledged web browser to tell GitHub that I am interested in the activities of a certain user. Or delete a fork. Yes, thats right. I can use Hub to fork a project, but there is no way to delete it again. Sort of strange for casual small-scale contributions. I guess the workflow there is: fork, commit, pull-request, wait for merge, delete. I don't want to have all sorts of forks on GitHub that I actually don't actively use/maintain. So once a pull request was merged, and I don't have any other things I am working on right now, I actually want to get rid of the fork again. It is easy enough to do a fork should I ever need it again.

Mario Lang: Contributing on GitHub

I really like to read code written by different people. I've always been the type of learner that can cover a lot of ground by looking at examples. A few of the programming tricks I acquired over time have definitely been learnt by reading programs written by other people. There are of course cases of disagrement with coding style (lets put that very mildly), so not every project is fun to read. But I tend to find projects I like enough to actually look at their source code. Now, if you get into this habit, there is also the occasional spotting of a bug. Many are really cosmetic, like typos in comments or documentation. Others are maybe more serious, who knows what can happen if you read a few lines of code written by someone you have never met in person. This is one of the original arguments of open source and free software. It basically goes like: "People will read your code and report problems." I know that happens, but I also know that there are cases of people finding relatively minor things during a small code review, which they don't report, or which get lost in some developers INBOX. I remember waiting months to see memory corruption fixes for a particular project I submitted by email to actually appear in the development repository. In some cases you really need to track your patches and prod someone to remind them about your contribution if need be. True, some of these issues are probably minor enough to not really hurt if they get lost, but I have a feel that a lot of minor things also sort of add up. This is where GitHub comes into play. We have had project hosting services since at least a decade now, but none of them (to my experience) have ever made it so easy for all involved parties to submit and merge changes into projects. Pull requests are just amazing. Well, at least for me, they are since a few months. I have a sort of mixed hate-love relationship with GitHub when it comes to a topic that is very important to me, Accessibility. The last project hosting site I really liked for their simple web interface (read, still usable with Lynx) was Google Code. That was in the days when Google still seemed to care about such things. These days, their newer services are next to unusable to me (with the tools I prefer). SourceForge was always a catastrophy. Many things were cumbersome to do, because the site is so loaded with links to all sorts of things I never need. However, at the time I was using it, all the things I really needed did work. GitHub is a different beast, in a different era of the internet. While I can read project pages and a few basic things perfectly fine, some parts of the GitHub site are simply unusable for no particular reason. For instance, I can edit (some of my) preferences and even save them. But I can not star a project, nor can I follow other developers. Invoking these links gives me a 404, which seems to indicate that some JavaScript (read, client side code execution) is required which my browser does not perform. However, I really wonder why. Because something like "I want to star this project" will eventually have to end up as a request on the server anyway, so it seems technically unnecessary to rely on Javascript for such an operation to succeed. Luckily, these days, hip projects do have an API. An GitHub is no exception. I want to emphasis that an API alone is not an accessibility fix, since you rarely have the time to implement a clinet for a random API, just to get access to a service you want to use. Sometimes you do, and an API is a great enabler in such situations, but a reference client with a rich feature-set is much better. I have to admit that I haven't evaluated any of the possible alternatives, but one client I found for GitHub's API did really boost my productivity in the last year. I am talking about Hub. It is a simple wrapper around Git that adds a few and enhances some other git commands. Most importantly, hub fork will fork the current checkout on GitHub, and hub pull-request allows you to quickly (and I really mean quickly!) open a pull request for some commits you've just done to your fork of some project. This is very helpful, and very accessible. While it does not solve all my accessibility problems with GitHub, it at least enables me to take part in the GitHub workflow. I can easily fork projects and submit pull reuqests for changes I did. And that is what I did, at least a bit, in roughly the past year. I am not sure exactly, but I think it was spring when someone made me aware of Hub. One approach would be to check my GitHub activity graph, but I probably don't have to mention that it is not accessible. Here is a list of some of my contributions to open source projects on GitHub since I discovered a good client for the GitHub API: To summarize: I love GitHub for what it does to the free software community. I think the simplicity (and transparency) of contributions is to the benefit of free and open source software. However, I also hate GitHub sometimes when I need to ask a friend to do relatively trivial changes for me. What I actually want, is a command-line client like Hub that makes most or even all functionality provided on the website available. True, some things like graphs are not immediately easy to adapt. But I really shouldn't need to use a modern full-fledged web browser to tell GitHub that I am interested in the activities of a certain user. Or delete a fork. Yes, thats right. I can use Hub to fork a project, but there is no way to delete it again. Sort of strange for casual small-scale contributions. I guess the workflow there is: fork, commit, pull-request, wait for merge, delete. I don't want to have all sorts of forks on GitHub that I actually don't actively use/maintain. So once a pull request was merged, and I don't have any other things I am working on right now, I actually want to get rid of the fork again. It is easy enough to do a fork should I ever need it again.

17 February 2014

Rog rio Brito: Some new Youtube-dl functionality

I wrote in a previous post that Youtube changed their way of delivering videos, with the use of Dynamic Adaptive Streaming over HTTP. On top of that, they started serving both the audio and the video in separate streams, which meant trouble for downloader tools like youtube-dl. As I mentioned in that previous post:
What does this mean in practical terms for users of youtube-dl? Well, if you wanted to download videos in resolutions like the 480p (format 35) that I mentioned, then you will probably have to change your way of doing things, until a more automated solution is in place. You will have to download both the audio and the video and, then, "combine" them (that is, multiplex them) to create one "normal" video file with both the audio and the video.
And latter, I wrote:
Otherwise, to download 480p videos (which I do for lectures and so on with other projects of mine, like edx-dl) I have to call youtube-dl twice: once for format 135 and another for format 140, since the old (?) format 35 files are much smaller than the lower resolution 360p files (due to the former being encoded in High profile vs. the latter being encoded in Constrained Baseline profile).
(...) Well, now, we don't have this problem anymore: The new release of youtube-dl brings us many goodies, including that it is possible to automatically combine/merge/multiplex audio and video formats that Youtube now offers separately (See the previous comments about separate audio and video). Now, if you want a 480p video in H.264 format, High profile, with 128kbps AAC audio (this used to be Youtube's format 35), you can specify format -f 135+140 on the command-line and it will download both the audio, the video and multiplex it with ffmpeg (or avconv, depending on what you have installed). Besides being convenient, this automatic downloading and merging makes it unnecessary to write scripts to, say, retrieve all the videos in a playlist that contains a lot of lectures (see one example here, taught by Benedict Gross). Version 2014.02.17 of youtube-dl will be soon in your favorite mirror of the Debian archives.

Martin Zobel-Helas: Have YOU registered Services at Debian Services Census?

Enrico and me have just started a census on Debian Services floating around, and we want YOU to participate!

The Debian Services Census is an attempt to gather detailed information about software services for the Debian community. It doesn't matter where these services live or who provides them. For now we are only interested in their existence.

Services can be anything from the mail forwarding service for @debian.org addresses that lives on master.debian.org operated by DSA, web applications, email bots, command line scripts you can run on debian.org machines, automated package checkers that report bugs to people, Debian-related services that people run on their own infrastructure like screenshots.debian.net and son on. Have a look at https://wiki.debian.org/Services for more examples.

If you think it's a service, then we think it's a service too and we want to know about it. Feel free to contribute data whether you are the maintainer of that service or not.

Here is how to let us know about the service:
  • please visit https://wiki.debian.org/Services and verify if the service is already listed there
  • if the service you had in mind is *not* listed, please create an entry for it using the "Add a new service" button.
If you are the maintainer of a service, we also encourage you to subscribe to the debian-services-admin@lists.debian.org mailing list, which is a low-traffic list and should be the contact point for inter-service communication and coordination.

Update: Up to now, 51 services have been registered (most of them within 24 hours after the d-d-a mail, WOW), but we are sure that there are a lot of more services, that still need to be registered. Help us to get a complete list!

Keith Packard: MicroPeak Approved for NAR Contests

MicroPeak Approved for NAR Contests The NAR Contest Board has approved MicroPeak for use in contests requiring a barometric altimeter starting on the 1st of April, 2014. You can read the announcement message on the contestRoc Yahoo message board here: Contest Board Approves New Altimeter The message was sent out on the 30th of January, but there is a 90 day waiting period after the announcement has been made before you can use MicroPeak in a contest, so the first date approved for contest flights is April 1. After that date, you should see MicroPeak appear in Appendix G of the pink book, which lists the altimeters approved for contest use Thanks much to the NAR contest board and all of the fliers who helped get MicroPeak ready for this!

Next.

Previous.