Search Results: "js"

11 January 2012

Craig Small: MediaServer with Rygel

Rygel XVI

Image by Cayusa via Flickr

Like a lot of people, I have one of those set-top TV boxes that can record TV shows at set times. I made sure that I could get at the files (using a FTP server in this case) and that the files were some sort of common standard (MPEG 4 TS). I also have a bunch of mp3 music files. That s fine when I m on the desktop because the files are local. I wanted to make these available to anyone in the household. DLNA seemed to be a reasonably OK way of doing this, the problem was, how to get it working in Linux? A lot of the problem is that it is hard to find a DLNA only server. Sure MythTV could do it, but it needs a tv tuner or a lot of fiddling around. XMBC can also do it, but it needs to be running a GUI. I even tried mediatomb but could not get the thing to compile as the library calls to mozjs were all using deprecated functions. I just wanted a daemon that served stuff, nothing more; no fancy ui, no need for X just file serving goodness. Rygel is almost that. You could say it is a user server much like a torrent client/server. The nice thing is you can fiddle around with rygel so it becomes close to a real server. This is how I did it. First, I made a rygel user with a home directory, but disabled login. I don t like programs running root if they don t need it and rygel doesn t need it. The home directory needs to be writeable to the rygel user too otherwise the program doesn t work too well. I use /var/local/rygel as its home. For the configuration, copy /etc/rygel.conf to ~rygel/.config/rygel.conf This is the configuration file for rygel. I disabled all of the modules except MediaExport. Make sure you disable Tracker otherwise MediaExport will not work. Tracker is only useful for real users who are logged in and have dbus etc going which this user is certainly not. I made a simple rygeld file in /usr/local/sbin which basically starts the program, forks and grabs the PID to write to a pidfile. This mean it was easier to track the program in the init scripts.
#!/bin/sh
#
# Rygel daemon handling
RYGEL='/usr/bin/rygel'
RYGEL_ARGS=''
su -s /bin/sh -c "nohup $RYGEL $RYGEL_ARGS > /var/local/rygel/rygel.log 2>&1 &"
rygel
EXIT_CODE=$?
if [ $EXIT_CODE != 0 ] ; then
        return $EXIT_CODE
fi
PGROUP= ps --no-headers -o pgrp $$ 
PID= pgrep -g $PGROUP -f $RYGEL 
echo $PID > /var/run/rygel.pid
exit 0
In case you were wondering, the pgrp finds the program group so the pgrep finds the right rygel process that has the same program group as the starting shell. The init script is a standard init script except the exec flag checks for /usr/bin/rygel but the start line starts /usr/local/sbin/rygeld This is because we want to kill the real rygel process but start it with the script. This setup works rather well. You do get some messages in the logfile about dbus not working but it is harmless. I tried disabling the mpris and external plugins but no matter what flag or configuration file option I tried, they would always try to start and fail with no dbus. Rygel is a a reasonably light-weight way of serving media to your home network. It idles 200 MB virtual with 16MB resident and when idle uses almost no CPU.

9 January 2012

Steve McIntyre: Armhf buildds and status in Debian

Current status Back in September, I wrote about the machines that I set up to help bootstrap the new armhf port in Debian. Basing on Konstantinos' huge efforts in bringing up the new "architecture" in debian-ports, we started importing armhf into the main Debian archive on the 24th of November. Since then, those builders have been churning away night and day to build the huge collection of software that makes up the Debian archive. The current state can be seen on the armhf buildd status page, and there's a nice graph showing how quickly we've managed to run from 0 to over 90% of the archive here. (Click on the image for a larger version, or visit https://buildd.debian.org/stats/ for other versions. We overtook hurd-i386 quickly and are now ahead of the kfreebsd-* architectures. armhf bootstrap graph We've recently brought 3 more similar build machines online (hildegard, howells and hummel), again sponsored by the nice folks at Linaro but now hosted at the York NeuroImaging Centre at the University of York. This gives us both more build horsepower to keep up with building more different bits of Debian (experimental, updates etc.) and more redundancy in case of problems. We now have the vast majority of the archive built, and now a number of us are concentrating on fixing the remaining issues: language bootstraps and bugs. Also, on the 7th of January we were just added into testing, the next step on our path for inclusion as a Debian release architecture. Setting up the machines A lot of people have been asking me about the physical setup I showed in my last blog about these machines, so here's more details for those who are interested. Mount the 6 boards into the mini-rack, connect up the Molex power connectors to each board, attach ethernet cables and turn it all on! Each board comes with a micro-SD card containing uboot and an Ubuntu installation. I've configured uboot to boot off the hard drive directly, but leaving configuration available to use the Ubuntu on the micro-SD as a simple rescue system should the need arise. The Quickstart boards are not ideal physically for two reasons: the lack of SATA power, plus you need to push a power button on each board to boot it - they don't boot automatically the moment power is applied. However, they're quite inexpensive little machines and have done a great job of building the Debian archive so far! The ideal machines for us would also include more RAM at this point. CPU on these is adequate, but the larger C++ packages (yay webkit!) use a huge amount of memory at link time. Linking in swap is not the best thing, performance-wise... :-( UPDATE 2012-01-12: Ian tells me that the newer Quickstart-R boards apparently have a different power controller; these now boot up straight away without needing you to push a button. That sounds useful.

6 January 2012

Steve Kemp: Tonight I've mostly been using Sinatra

This evening I've mostly been using Sinatra to build a little file storage service which uses a REST API. That means I can upload a file:
skx@birthday:~/hg/sinatra$ curl -X PUT -F file=@/etc/fstab http://localhost:4567/
 "id":"dbd1bdc11b5a1a8e80588a135648b4c2edffb49a","path":"/" 
Download that same file:
skx@birthday:~/hg/sinatra$ curl -X GET -F id=dbd1bdc11b5a1a8e80588a135648b4c2edffb49a  \
   http://localhost:4567/
# /etc/fstab: static file system information.
..
/dev/cdrom        /media/cdrom0   udf,iso9660 user,noauto     0       0
Get an index of files:
skx@birthday:~/hg/sinatra$ curl http://localhost:4567/
[ "id":"dbd1bdc11b5a1a8e80588a135648b4c2edffb49a","type":"file" ]
And finally we can delete a file:
skx@birthday:~/hg/sinatra$ curl -X DELETE -F "id=dbd1bdc11b5a1a8e80588a135648b4c2edffb49a" \
  http://localhost:4567/
Removed
We can also upload to different paths so we can replicate a file-system if we wanted to. (I added in "type" to hold either "file" or "directory", though I guess if we were to code up a FUSE client we'd want to store things like ctime, UID, GID, etc. THe list operation will show both files and sub-directories) The code was trivial once I got the hang of Sinatra, and I'm pretty pleased with it so far. I don't yet need to use it for anything, but I'm thinking of unifying the way that I store images on a couple of sites - and fetching them via JSON and Javascript might be an option this was an experiment in that direction. (Though I'd probably want to hook in rsync so we replicated the eventual upload location for safety.) In other news I've been all organized and upgraded the kernel on my guest:
steve@steve:~$ uptime
 22:00:28 up  4:18,  1 user,  load average: 0.14, 0.05, 0.05
steve@steve:~$ uname -r
3.2.0-kvm-hosting.org-i386-20120106
So for once I'm up to date with a cutting edge kernel. Happy times. ObQuote: "How you expect to run with the wolves come night when you spend all day sparring with the puppies? " - The Wire (Omar)

19 December 2011

John Sullivan: Where Shall I Wander

Google Maps is now mapping the indoors. I saw an ad for this while passing through MSP yesterday (given how much time I spend on the Internet, it's strange and a little embarrassing to learn about new things on the Internet from airport billboards), since one of their initial targets is the infamous Mall of America.
Detailed floor plans automatically appear when you re viewing the map and zoomed in on a building where indoor map data is available. The familiar blue dot icon indicates your location within several meters, and when you move up or down a level in a building with multiple floors, the interface will automatically update to display which floor you re on. All this is achieved by using an approach similar to that of My Location for outdoor spaces, but fine tuned for indoors.
Thoughts about this: Title from John Ashbery

3 December 2011

Steve Kemp: So I removed some more software from my host

Today I was idly performing some maintainence upon one of my hosts, and it crossed my mind to look beneath /etc in there I found:
/etc/python
/etc/python2.4
/etc/python2.5
/etc/python2.6
That made me look more closely at the contents of /etc - the following command output was surprising:
steve@steve:~$ ls /etc   wc -l
187
Is that average? Heavy? Light? I have no idea, but I purged a hell of a lot of software today. Now I have only python v2.6 although for some reason I still have:
python
python-apt
python-apt-common
python-central
python-minimal
python-support
python2.6
python2.6-minimal
I suspect I could drop the pything2.6-minimal package, but for the moment I'm done. I have to make pretty people look exceptional with my magical camera. Anyway as part of this cleanup I ran a quick sanity-check on which processes are running and I think, short of kernel processes, I'm as minimal as I can be. I understand the purpose and reason for every running service:
UID        PID  CMD
root         1  init [2]
pdnsd    14091  /usr/sbin/pdnsd --daemon -p /var/run/pdnsd.pid
root     14199  /usr/sbin/monit -c /etc/monit/monitrc -s /var/lib/monit/monit.state
root     14206  /usr/sbin/syslog-ng -p /var/run/syslog-ng.pid
root     14234  /usr/sbin/cron
102      14595  /usr/sbin/exim4 -bd -q30m
redis    14627  /usr/bin/redis-server /etc/redis/redis.conf
root     14637  /usr/sbin/sshd
These are basic services; I use monit to ensure those essential daemons keep running. The only oddity there is probably the local DNS cache, but it is useful if you run any kind of DNS blacklist-using service, for example.
root     14794  /sbin/getty -L ttyS0 9600 vt100
I need a serial console login for emergencies.
root     14796  runsv node-reverse-proxy
root     14797  /bin/sh ./run
root     14799  /opt/node/bin/node node-reverse-proxy.js --config ./rewrites.js
These three processes combine to run my reverse proxy which routes incoming HTTP requests to a number of local thttpd instances.
qpsmtpd  27309    /usr/bin/perl -Tw /usr/bin/qpsmtpd-prefork --port 25 --user qpsmtpd --pid-file /var/run/qpsmtpd/qpsmtpd.pid --detach
..
The perl SMTP daemon which runs my incoming mail, passing it to exim4 which listens upon 127.0.0.1:2525. You can read about my setup in the out-of-date writeup Chris & I put together.
 /usr/bin/memcached -m 64 -p 11211 -u root -l 127.0.0.1
Memory cache for transient items.
s-blog    thttpd -C /etc/thttpd/sites.enabled/blog.steve.org.uk
1030      thttpd -C /etc/thttpd/sites.enabled/edinburgh-portraits.com
s-hg      thttpd -C /etc/thttpd/sites.enabled/hg.steve.org.uk
s-ipv4    thttpd -C /etc/thttpd/sites.enabled/ipv4.steve.org.uk
s-ipv6    thttpd -C /etc/thttpd/sites.enabled/ipv6.steve.org.uk
s-kvm     thttpd -C /etc/thttpd/sites.enabled/kvm-hosting.org
...
One thttpd instance is launched for each distinct HTTP site my server runs. Each site runs under its own UID, with its own chrooted directory tree. This is important for security. Each local instance listens upon 127.0.0.1 - and the reverse proxy previously mentioned rewrites connections to the appropriate one.
1016     28812     /usr/bin/perl -I./lib/ -I./ /usr/local/bin/blogspam
My anti-spam filter for blog comments.
Here is my christmas challenge. Can you identify each service upon your host? Do you know why you're running what you're running? Me? I had no idea I had a dbus deamon running. Now I've purged it. Ha! ObQuote - "I owe everything to George Bailey. Help him, dear Father." - It's a wonderful life.

28 November 2011

Paul Tagliamonte: Another post from the "Useless Script" department

A while back I wrote some nodejs that mish-moshed email and web 2.0 long-poll action into a very ugly, but fairly up-to-date showing of debian changes. That is over on my server here. The code s fairly simple, so I also wrote a python consumer here. Since it s so simple, I ve pasted it below, too. Feel free to use (but please not abuse) my server :)
#!/usr/bin/env python
import time
import urllib
import json
service = "http://pault.ag:8080/tail"
lastts  = 0
def loop():
        global lastts
        while True:
                f = urllib.urlopen("%s?since=%s" % (service, str(lastts)))
                msgs = json.loads(f.read())
                for x in msgs['messages']:
                        print x['text']
                        if lastts < x['timestamp']:
                                lastts = x['timestamp']
loop()

22 November 2011

Jonas Smedegaard: Asia 2011

In september I visited Brussels, Belgium. EPFSUG had kindly invited me to give a talk about FreedomBox in the European Parliament, and together we extended to also visit other organizations - both grassroots and more formal. I returned home exhausted but also fuelled with renewed energy and passion, and decided to engage in more such activities. Also, an old promise of mine to visit friends in the Aceh region of Indonesia (whom I met in Taiwan a few years back) rumbled in the back of my head. After a bit of juggling with travel routes and sponsoring options, my course was set for a 2 month journey with Debian Pure Blends as main theme:
  1. England: London
  2. India: Khammam, Hyderabad, Mangaluru, Bengaluru
  3. Vietnam: Ho Chi Minh City, Can Tho
  4. Thailand: Bangkok
  5. Malaysia: Putrajaya
  6. Indonesia: Surabaya, Yogyakarta, Bogor, Banda Aceh, Takengon
  7. England: Cambridge
Debian will be sponsoring East Asia part, and Debian enthusiasts in India seek sponsorship for India part.

Jonas Smedegaard: Asia 2011 - India

Arrival in a new country is always exciting. This was my first time ever to visit India, and although I have heard especially cultural bits and pieces, I was as usual nowhere near "well" prepared. How to fill out the registration forms (surprisingly needed in addition to the visa already gathered ahead of departure) when your only known address in India is on the laptop you completely drained the battery of during the flight? Luckily they tolerated the "address during stay" to be left blank. I got out in the heat of Hyderabad in the late afternoon, got a cab, and had my host instruct the cab driver - over the phone via roaming to Denmark - where to drop me off. After a long ride with cows and beautiful dressed pedestrians casually crossing the high speed road and a short pitstop at an ATM, I finally met Pavithran. Until then we only knew each other from casual online chat. (I should later learn that my first impression was quite unusual - not cows or clothes or chat, but roads capable of driving at high speed!) Pavithran checked me into a small hotel and we visited his home. It was in the middle of being rebuild, so impossible to stay at as had been our original planed. After a few hours of looking at the neighbourhood and talking about possible events during the week, we decided to cancel the hotel and instead go visit his parents in Khammam, some 5 hours away by bus This text is part of my Asia 2011 scriblings.

14 November 2011

Vincent Bernat: A quick look at Typekit

Back in the dark ages of the web, before the introduction of CSS 3, browsers could only display fonts available on the client system. Thanks to the introduction of @font-face and its now widespread support, this is over. Now, Web designers have access to a large selection of fonts and most users will be able to appreciate them. Typekit provides a subscription-based library of hosted, high-quality fonts to use on websites. I expose here my own, mostly uneducated1 on the typographic side, look on this service.

Fonts embedding 101 There are several ways to embed fonts for the Web. Let s have a look at the most common ones.

Self-hosting and @font-face A first possibility is to host fonts yourself and provide a CSS file with the appropriate @font-face declarations. There are five important things to know:
  1. There is no common format. You will need to host different versions of the same font to support most browsers.
  2. Of course, Internet Explorer comes with its own quirks that you need to work around.
  3. Most browsers enforce a same origin policy. You need to serve your fonts from the same domain as the requesting page. You can circumvent such a limitation by enabling cross-origin resource sharing: add an Access-Control-Allow-Origin header when serving fonts.
  4. Fonts are unlikely to change. To leverage caching, configure an explicit expiration date far in the future. Something like 30 days.
  5. There is no MIME types registered for fonts. Don t bother setting one, even if Google Chrome likes to complain about them in the console.
One great resource is Font Squirrel which provides kits with fonts and the appropriate CSS declarations. You choose your font family (or upload one of your own), the formats that you want to use (check them all) and you will get a ZIP archive with all fonts and a CSS file. You may want to tweak it a bit to enable automatic use of the bold variant when needed and to use a local copy if available. For example, if you choose Crimson, an oldstyle typeface, you may want to declare the bold version with font-weight: bold and add local names:
@font-face  
    font-family: 'Crimson';
    src: url('Crimson-Bold-webfont.eot');
    src: local('Crimson Bold'), local('Crimson Text Bold'),
         url('Crimson-Bold-webfont.eot?#iefix') format('embedded-opentype'),
         url('Crimson-Bold-webfont.woff') format('woff'),
         url('Crimson-Bold-webfont.ttf') format('truetype'),
         url('Crimson-Bold-webfont.svg#CrimsonBold') format('svg');
    font-weight: bold;
    font-style: normal;
 

Google Web Fonts For open-source fonts, you can also use Google Web Fonts instead of hosting the files yourself. Build a collection of fonts and choose the variants that you want to use for each of them. You will then be provided with a code snippet to paste in the <head> element of your page. For example, for Crimson:
<link href='http://fonts.googleapis.com/css?family=Crimson+Text:400,700,400italic'
      rel='stylesheet' type='text/css'>
Why choose this solution over self-hosting?
  1. You don t use your own bandwith to serve fonts. This may be important if you pay for it.
  2. Google uses its own CDN to serve fonts. If you don t use a CDN yourself, this can be a major performance improvement.
  3. Caching is more efficient since many sites will use the same font.
  4. It s easier to do it right.
Unless you are concerned about Google tracking your users2, I think this is a better solution than self-hosting. Quick delivery of fonts is pretty important since content rendering can be delayed by them.

Typekit: the good parts So, why choose Typekit? First, there is the very large choice of fonts. The first paid plan comes with more than 500 fonts from about 80 foundries. There are about a hundred serif fonts and a couple hundred sans serif ones. Typekit provides a convenient interface to browse fonts. You can either search fonts by name or by characteristics. In the last case, you can filter on properties like the classification (serif, sans serif, decorative, etc.), weight, width, contrast, x-height, recommended use or language support. Once a font is selected and added to your kit, you can easily choose what variants you want. The kit total size is displayed. If you happen to meet any problem or have some questions, Typekit support is very responsive. I got clear and crisp answers in less than a few hours in the week-end.

and the bad ones Unfortunately, Typekit has also some shortcomings. One of the most commonly cited is that Javascript is mandatory. No Javascript, no fonts. If you want to display the appropriate fonts for users without Javascript, you should just not use Typekit.

Not free, nor free While Typekit offers some open-source fonts, most of them are not. Moreover, only a few of them are available in the free plan. While I would prefer to use open-source fonts, I don t consider this a strong requirement: if you are dissatisfied for some reason, switching back to open-source fonts (or another service) is fast and easy. I like serif fonts a lot and they get less attention from the FLOSS world: it is difficult to find one whose readability on screen is excellent. Droid Serif is a nice font but is a bit too black for me. Maybe Cardo would be a good match. You may disagree and in this case, Google Web Fonts or self-hosting seem the perfect match.

Difficulty to choose and test fonts While browsing fonts is easy and smooth, choosing and testing them is another matter. For example, Typekit is missing some interface to compare font rendering for whole paragraphs at a given size. It also misses a way to check how two fonts render together, one for headings and one for paragraphs. Here, Google Web Fonts is more versatile. Things become really difficult when you want to test your selected fonts for real. Except if you have invested into one of the expensive plans, you can only include a limited number of fonts in your kit. Each time you want to try another set of fonts, you need to modify your kit and wait for it to be distributed across Typekit network. You may have to wait five or ten minutes between each change! Another inconvenient thing is that once you publish your site, any change to the kit will affect the published version. Therefore, you need to use a second kit for development and play back and forth with those two kits. What would be convenient here is a special kit with all fonts from the library with restricted access to some IP to avoid abuse. This is something done by Fontdeck, a competitor.

Poor subset support Because you want to keep the font size small (about 30 kB for each variant), Typekit usually proposes two subsets: the default one contains only latin characters and the complete one with all glyphs. If some glyph is absent from the default subset, you need to use the complete one which is roughly two to four times the size of the default one. This is a bit huge for only one character. What would be nice is to have subsets for each language. In my case, I would take a French subset because I need the glyph which appears in French in words like c ur (heart or core) or il (eye). I don t want to select the complete subset just for this character. When a glyph is absent, the next font in the CSS stack is used. For example, if the selected font is FF Tisa Web Pro and the next font in the stack is Droid Serif, here is how the word c ur may be rendered. On the first line, FF Tisa Web Pro contains the appropriate glyph. On the second line, the glyph is absent and taken from Droid Serif. Since x-height and contrast are a bit different, the replaced glyph seems out of tone. At small sizes, the difference accounts for more than 10%. Various rendering of word "c ur" I have also added a third line where the ligature is built from o and e and the following style is applied to e :
.lf-ligature  
    margin-left: -0.172em;
    zoom: 1; /* Here to trigger hasLayout on IE7 */
 
As you can see, the result seems good. I have a piece of Javascript turning a ligature into the appropriate markup. It can be extended to handle more ligatures and could be adapted to handle other composed characters (like the ones in Anton n Dvo k ).

Defective caching Because a document can only be rendered after the fonts have been loaded3, it is pretty important to deliver fonts as fast as possible. Typekit uses Edgecast as a CDN and DynECT for DNS hosting. Two files are downloaded. Both of them are hosted on use.typekit.com. The first one is a script that will act as a loader. Based on the characteristics of your browser, it will choose how to download the requested fonts for best rendering. The second file is generally a big CSS file containing the fonts. Here are the headers returned for the first file:
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=300
Content-Encoding: gzip
Content-Type: text/javascript
Date: Tue, 08 Nov 2011 18:34:16 GMT
ETag: "388467610+gzip"
Expires: Tue, 08 Nov 2011 18:39:16 GMT
Last-Modified: Sat, 05 Nov 2011 09:31:57 GMT
Server: ECS (cdg/D624)
Vary: Accept-Encoding
X-Cache: HIT
Content-Length: 7440
First, notice that the content is served compressed. Good point. The next headers to look at are Cache-Control, Date, ETag and Expires: the server requests the content to be cached for 5 minutes. When the browser notices that its copy is out-of-date, it will issue a new request including a If-None-Match: 388467610+gzip header. Unless the kit has been modified, the server will answer with 304 Not Modified and no content. The copy of the browser will be valid for another 5 minutes. This means that after 5 minutes, at least 3 round-trips are needed to check if our fonts are up-to-date. Worse, sometimes, Typekit servers may be just slow. They can wait one second before answering, just to send a 304. Since the first file is a script embedded in <head>, the whole page is blocked until you get the answer. Here is an illustrative example with Chromium 14 where a page using Typekit is loaded while all static assets are in the navigator cache and things have gone wrong (which is pretty rare): Screenshot of Chromium Inspector Since fonts are needed, the rendering has been paused for 3.5 seconds until Typekit was fully loaded. If we look more closely, we may also notice we got a status 200 from the server instead of 304. Here is an excerpt of the browser request:
GET /tyt0atd.js HTTP/1.1
Host: use.typekit.com
Connection: keep-alive
Accept-Encoding: gzip,deflate,sdch
If-None-Match: "2200251225+gzip"
If-Modified-Since: Thu, 10 Nov 2011 11:38:11 GMT
And here is an excerpt of the answer headers:
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=300
Content-Type: text/javascript
Date: Fri, 11 Nov 2011 17:10:03 GMT
ETag: "2200251225"
Expires: Fri, 11 Nov 2011 17:15:03 GMT
Last-Modified: Thu, 10 Nov 2011 11:38:11 GMT
Server: EOS (lax001/54E5)
Content-Length: 25755
What happened? For some unknown reason, the server denied the access to the compressed version of the resource: there is no Content-Encoding: gzip while the browser did send Accept-Encoding: gzip. The tag provided by the browser (2200251225+gzip) came obviously from the compressed resource and therefore does not match. RFC 2616 requests If-Modified-Since header to be ignored if the tag does not match. Consequently, the server answers with 200 OK and resend the whole file. The same problem happened with the CSS file. Let me stress again how important the problem is: we had to download again the two files but we also downloaded the non-compressed version. Now, let s suppose that we did not run into this problem and the server answered with 304 Not Modified. The result is still not satisfactory: we would have waited 1.3 seconds to get both answers from the server. Keep in mind that most of the time, Typekit is pretty fast. Here is how I think this should have been done: each time you modify your font kit, you get a new URL to use (containing the version of the font kit for example). This resource and the associated fonts (whose URL is also versioned) are requested to be cached for several days. As a sidenote, this also enables to modify a kit for your development environment without modifying the kit for your production site. I suspect that Typekit did not go this way to track kit usage more accurately and enforce the appropriate pricing. Having multiple versions of the same kit would also allow us to use more fonts that we are paying for. UPDATED: Typekit support knows the problem and is working on it:
You ve identified a small bug in our CDN provider that we re working with them to fix. Hopefully I can give an explanation of what s happening. The CDN edge servers have a cache of recently served kits on them. Any request that is served from that cache will be correctly Gzipped and handle ETags. Unfortunately, if the content isn t already in cache and needs to be fetched from the master origin server then the request won t be gzipped correctly which has the side effect of breaking ETags. [ ] We are working to on getting this bug fixed, but any change to a globally distributed CDN will take some time. We re also working on having our content cached longer than 5 minutes whilst still allowing people to quickly update their site.

Conclusion I have mixed feelings about Typekit. While the difficulty to test fonts and the lack of language-related subsets are annoying, I have been able to work around them. However, the caching problem is more annoying. I am still considering going back to Google Web Fonts just because of this.

  1. While I have a high interest in typography, have my own copy of The Elements of Typographic Style by Robert Bringhurst and have throughly read The Elements of Typographic Style Applied to the Web , I am still fairly novice on this matter.
  2. Google serves fonts from a cookie-less domain. Technically, they could still do some kind of tracking but I believe they don t.
  3. In fact, browsers have different strategies on how to render before fonts availability. Paul Irish wrote a comprehensive article on FOUT. One common strategy is to render a blank space until downloaded fonts are available. If this takes too much time, fallback fonts may be used.

12 November 2011

Kai Wasserb ch: New Greasemonkey script available for WotC's Magic section

No matter whether you're a professional or hobby player, if you play Magic: The Gathering, you certainly visit Wizards of the Coast's homepage from time to time. And while other parts of the WotC's page have been made unusable without ECMAScript enabled, the MTG part can be used without it mostly. The one part, that didn't work (and I needed) was card links. They have some ECMAScript to fetch the image from the Gatherer and overlay that on the current page, if you hover over a card link. If you click, it's again some script, that opens a Gatherer window. I just want all card links to be plain links pointing to the respective entries in the rather lightweight magiccards.info card search, which displays all relevant data in a compact layout. Today I finally got around to throw the ten or so lines of ECMAScript together, Greasemonkey needs to make that happen. If you like to give it a try, you can download it from my Greasemonkey script collection (direct link for the impatient: wotc_with_noscript_and_card_links.user.js). As an additional note: I'd like to point out, that there are other scripts available from userscripts.org which can do the same. But all those scripts did a lot else (like using jQuery to fetch the image on mouse over from magiccards.info).

10 November 2011

Jonas Smedegaard: Asia 2011 - London

First event of my trip was somewhat a gimmick: Give a 30 minutes talk on FreedomBox to an audience of 1 person, in the transit area of Heathrow airport, London. A guy from India now living in London helped preparing my visit to India, an was eager to participate - so much that he was willing to drive out to the airport to meet me. The meeting failed, unfortunately: My flight got delayed, and my one-man "audience" got caught in other duties. :-( Fun to try nevertheless. Also looking back, that crazy attempt to squeeze in a meeting during a 1 hour stop-over was an indicator of the general pace of the India trip This text is part of my Asia 2011 scriblings.

4 November 2011

Kai Wasserb ch: Greasemonkey scripts on dev.carbon-project.org

If you value your privacy, you might want to use a search engine like Ixquick which tries to protect your privacy. But if you value your privacy you probably have NoScript installed and have it hide <noscript> elements, as many pages hide tracking methods for users with deactivated JS in such tags. That makes every result page (but the first) inaccessible, because Ixquick has its non-JS navigation in noscript tags too. Now you could either activate JavaScript for Ixquick or work around this problem. Thanks to Greasemonkey and a little script I wrote, working around this issue is really easy. At this point, there is only this one script on dev.carbon-project.org, but in case I write another one, it'll end up there too. ;-)

3 November 2011

Philipp Kern: PAV on Linux on System z

There are various presentations that state the goodness of PAV on Linux. Most revolve around using multipath-tools to assemble a volume if you don't have HyperPAV. But it turns out that the DASD device driver does multipathing for them internally in current kernels (which includes the squeeze kernel).

So all you need to do is setting those alias devices online. When you do that the kernel will log that it detected a new device, but you'll find that it won't create any dasd* device nodes for them, nor will it list partitions. lsdasd will only show you "alias" without mentioning the base volume, but you can fetch that information easily from the uid sysfs entry.

1 November 2011

Philipp Kern: Useful Firefox extensions

Many people around me switched to Chrome or Chromium. I also used it for a bit, but I was a bit disappointed about the extensions available. To show why, here's a list of the extensions I've currently installed:
If Firefox on Android were quicker to start and faster overall, I might even use it there. But as-is it's not very useful. Sadly this also means that I can't use Firefox Sync on my phone and as I don't use Chrome on my desktop I also can't use Chrome to Phone. So I usually go and build a QR code on my laptop and read that with Android's Barcode Scanner.

Of course I'm actually using Iceweasel and I'm very grateful for Mike Hommey's efforts to track the release channel on mozilla.debian.net.

28 October 2011

Kai Wasserb ch: Google recommends using OpenStreetMaps/OpenLayers

Maybe you've already heard, that Google is starting to charge a fee for using the Maps API (Google may decide you're an non-profit organization and waive the fees). That is, in my opinion, Google telling you to look for better alternatives. And thanks to the awesome effort of the OpenStreetMaps community, there is such an alternative. Thus today's addition to the mini-tips series will tell you, how you get an OpenStreetMaps-powered map on your website.
  1. First you need to determine the coordinates, where you want to center, the map. You can do this with OpenRouteService.org. Just search for the address, right-click on the marker highlighting the result, and set it as e.g. your start point, that'll give you the coordinates in the start field on the left. Alternatively you can just move your pointer over a place on the map and have a look at the lower right corner, where the current coordinates for the pointer position are displayed.
  2. Then you need to add a few things to the website, where the card is to be shown:
    • Include the OpenLayers script in the <head> of your webpage:
      <script src="http://www.openlayers.org/api/OpenLayers.js" type="text/javascript" />
      
    • Add a short ECMAScript snippet right after that, which will initialize your map:
      function osmInit()  
        document.getElementById("ol_map").style.display = 'block';
        var lonLat = new OpenLayers.LonLat(13.376096, 52.518598) // Center of the map
        .transform(
          new OpenLayers.Projection("EPSG:4326"),   // transform from WGS 1984
          new OpenLayers.Projection("EPSG:900913")  // to Spherical Mercator Projection
        );
        var map = new OpenLayers.Map("ol_map");
        var osmlayer = new OpenLayers.Layer.OSM();
        map.addLayer(osmlayer);
        map.setCenter(lonLat,
          16 // Zoom level
        );
       
      
      The first line is not needed, but I find it nice to hide the map on default, in case the user has disabled ECMAScript they haven't a large empty area somewhere on the website. The first line changes then the display value from "none" to "block". The rest is, I hope, already explained by the code itself.
    • Now only two more changes are needed, first you need to add onload="osmInit();" to your <body> tag. And afterwards an element (say a <p>) with the ID ol_map somewhere in the website <body>. You should set a width and height for that tag (style is your friend), otherwise the map might be bigger than you like.
  3. Done. Enjoy your free (not just as in free beer) map!
You can, of course, customize your map further by adding additional layers and changing the available options (e.g. the zoom level). If you would want to mark a location on your map, you'd need to add something like
var markers = new OpenLayers.Layer.Markers("POI1");
map.addLayer(markers);
markers.addMarker(new OpenLayers.Marker(lonLat));
to your ECMAScript snipped after you added the OpenStreetMaps layer. These three lines would mark the location given by the variable lonLat. For further information, you can check out the OpenStreetMaps wiki and the OpenLayers documentation. In case you wonder, where my coordinates from this example point, you can check. Oh, and don't forget to consider a donation to OpenStreetMaps, so the project can pay the bills for its servers.

21 October 2011

Russell Coker: Dedicated vs Virtual Servers

A common question about hosting is whether to use a dedicated server or a virtual server. Dedicated Servers If you use a dedicated server then you will face the risk of problems which interrupt the boot process. It seems that all the affordable dedicated server offerings lack any good remote management, so when the server doesn t boot you either have to raise a trouble ticket with the company running the Data-Center (DC) or use some sort of hack. Hetzner is a dedicated server company that I have had good experiences with [1], when a server in their DC fails to boot you can use their web based interface (at no extra charge or delay) to boot a Linux recovery environment which can then be used to fix whatever the problem may be. They also charge extra for hands-on support which could be used if the Linux recovery environment revealed no errors but the system just didn t work. This isn t nearly as good as using something like IPMI which permits remote console access to see error messages and more direct control of rebooting. The up-side of a dedicated server is performance. Some people think that avoiding virtualisation improves performance, but in practice most virtual servers use virtualisation technologies that have little overhead. A bigger performance issue than the virtualisation overhead is the fact that most companies running DCs have a range of hardware in their DC and your system (whether a virtual server or a dedicated server) will be on a random system from their DC. I have observed hosting companies to give different speed CPUs and for dedicated servers different amounts of RAM for the same price. I expect that the disk IO performance also varies a lot but I have no evidence. As long as the hosting company provides everything that they offered before you sign the contract you can t complain. It s worth noting that CPU performance is either poorly specified or absent in most offers and disk IO performance is almost never specified. One advantage of dedicated servers in this regard is that you get to know the details of the hardware and can therefore refuse certain low spec hardware. The real performance benefit of a dedicated server is that disk IO performance won t be hurt by other users of the same system. Disk IO is the real issue as CPU and RAM are easy to share fairly but disk performance is difficult to share and is also a significant bottleneck on many servers. Dedicated servers also have a higher minimum price due to the fact that a real server is being used which involves hardware purchase and rack space. Hetzner s offers which start at E29 per month are about as cheap as it s possible to get. But it appears that the E29 offer is for an old server new hardware starts at E49 per month which is still quite cheap. But no dedicated server compares to the virtual servers which can be rented for prices less than $10 per month. Virtual Servers A virtual server will typically have an effective management interface. You should expect to get web based access to the system console as well as ssh console access. If console access is not sufficient to recover the system then there is an option to boot from a recovery device. This allows you to avoid many situations that could potentially result in down-time and when things go wrong it allows you to recover faster. Linode is an example of a company that provides virtual servers and provides a great management interface [2]. It would take a lot of work with performance monitoring and graphing tools to give the performance overview that comes for free with the Linode interface. Disk IO performance can suck badly on virtual servers and it can happen suddenly and semi-randomly. If someone else who is using a virtual server on the same hardware is the target of a DoS attack then your performance can disappear. Performance for CPU is generally fairly reliable though. So a CPU bound server would be a better fit on the typical virtual server options than a disk IO bound server. Virtual servers are a lot cheaper at the low end so if you don t need the hardware capabilities of a minimal dedicated server (with 1G of RAM for Hetzner and a minimum of 8G of RAM for some other providers) then you can save a lot of money by getting a virtual server. Finally the options for running a virtual machine under a virtual machine aren t good, AFAIK the only options that would work on a commercial VPS offering are QEMU (an x86 CPU instruction emulator), Hercules (a S/370 S/390, and Z series IBM mainframe emulator), and similar CPU emulators. Please let me know if there are any other good options for running a virtual machine on a VPS. Now while these emulators are apparently good for debugging OS development they aren t something that are generally useful for running a virtual machine. I knew someone who ran his important servers under Hercules so that x86 exploits couldn t be used for attacking them, but apart from that CPU emulation isn t generally useful for servers. Summary If you want to have entire control of the hardware or if you want to run your own virtual machines that suit your needs (EG one with lots of RAM and another with lots of disk space) then a dedicated server is required. If you want to have minimal expense or the greatest ease of sysadmin use then a virtual server is a better option. But the cheapest option for virtual hosting is to rent a server from Hetzner, run Xen on it, and then rent out DomUs to other people. Apart from the inevitable pain that you experience if anything goes wrong with the Dom0 this is a great option. As an aside, if anyone knows of a reliable company that offers some benefits over Hetzner then please let me know. What I would Like to See There is no technical reason why a company like Linode couldn t make an offer which was a single DomU on a server taking up all available RAM, CPU, and disk space. Such an offer would be really compelling if it wasn t excessively expensive. That would give Linode ease of management and also a guarantee that no-one else could disrupt your system by doing a lot of disk IO. This would be really easy for Linode (or any virtual server provider) to implement. There is also no technical reason why a company like Linode couldn t allow their customers to rent all the capacity of a physical system and then subdivide it among DomUs as they wish. I have a few clients who would be better suited by Linode DomUs that are configured for their needs rather than stock Linode offerings (which never seem to offer the exact amounts of RAM, disk, or CPU that are required). Also if I had a Linode physical server that only had DomUs for my clients then I could make sure that none of them had excessive disk IO that affected the others. This would require many extra features in the Linode management web pages, so it seems unlikely that they will do it. Please let me know if there is someone doing this, it s obvious enough that someone must be doing it. Update: A Rimuhosting employee pointed out that they offer virtual servers on dedicated hardware which meets this criteria [3]. Rimuhosting allows you to use their VPS management system for running all the resources in a single server (so no-one else can slow your VM down) and also allow custom partitioning of a server into as many VMs as you desire.

18 October 2011

John Goerzen: Greece part 3: Water

See also parts 1 and 2. That s a photo of Vlicha Beach, near our hotel. But before I talk about the Greek beaches, I need to explain something about living in Kansas. Kansas is in the middle of the United States. The nearest ocean is the Gulf of Mexico, which is 700 miles (1100km) away. That s roughly the same as the distance between Berlin and Minsk, or New York and Chicago. And I believe it s farther away from a saltwater body than all (or almost all) of Europe. So we re not just going to the beach every weekend or something. Vlicha Beach was all those incredible things you ever hear about beaches. The water was so clear that I could easily see my feat while wading in it. That, and the fish swimming around them. It was peaceful, relaxing, and picturesque. Between the deep blue of the sea and the same in the sky, I suppose it is no wonder that blue is often associated with Greece. We weren t exactly the only ones there. Though I think we were the only ones there with Kindles, which seemed to be much more popular in the USA than in Europe. We got several interested people carrying paper books asking what they were. Towards dusk, the mist would become more pronounced and the mountains off in the distance started to fade: The evening before we were set to fly home, we spent some time sitting on our balcony watching dusk set in. One last gaze out over the beautiful Aegean, the misty mountains, and the boats in the distance.

16 October 2011

Tim Retout: BITE server

This week, Google released an extension called BITE which lets you file bug reports from within Chrome (or Chromium). If you are testing web applications, it lets you attach screenshots and/or automated tests to reproduce the bug you've found. There's just one small catch: they haven't released a server to go with the client. Oops. Apparently the internal systems are too tightly integrated to make that possible. I have hacked up enough stubs of a BITE server in Perl to get the client to "log in" and show off some features. (Warning: it doesn't actually do anything useful yet.) So far, I have learnt:

5 October 2011

Tollef Fog Heen: The SugarCRM rest interface

We use SugarCRM at work and I've complained about its not-very-RESTy REST interface. John Mertic a (the?) SugarCRM Community Manager asked me about what problems I'd had (apart from its lack of RESTfulness) and I said I'd write a blog post about it. In our case, the REST interface is used to integrate Sugar and RT so we get a link in both interfaces to jump from opportunities to the corresponding RT ticket (and back again). This should be a fairly trivial exercise or so you would think. The problems, as I see it are: My first gripe is the complete lack of REST in the URLs. Everything is just sent to https://sugar/service/v2/rest.php. Usually a POST, but sometimes a GET. It's not documented what to use where. The POST parameters we send when logging in are:
method=>"login"
input_type=>"JSON"
response_type=>"JSON"
rest_data=>json($params)
$params is a hash as follows:
user_auth =>  
            user_name => $USERNAME,
            password => $PW,
            version => "1.2",
 ,
application => "foo",
Nothing seems to actually care about the value of application, nor about the user_auth.version value. The password is the md5 of the actual password, hex encoded. I'm not sure why it is, as this adds absolutely no security, but it is. This is also not properly documented. This gives us a JSON object back with a somewhat haphazard selection of attributes (reformatted here for readability):
 
     "id":"<hex session id>,
     "module_name":"Users",
     "name_value_list":  
             "user_id":  
                     "name":"user_id",
                     "value":"1"
              ,
             "user_name":  
                     "name":"user_name",
                     "value":"<username>"
              ,
             "user_language":  
                     "name":"user_language",
                     "value":"en_us"
              ,
             "user_currency_id":  
                     "name":"user_currency_id",
                 "value":"-99"
              ,
             "user_currency_name":  
                     "name":"user_currency_name",
                     "value":"Euro"
              
      
 
What is the module_name? No real idea. In general, when you get back an id and a module_name field, it tells you that the id exists is an object that exists in the context of the given module. Not here, since the session id is not a user. The worst here is the name_value_list concept which is used all over the REST interface. First, it's not a list, it's a hash. Secondly, I have no idea what would be wrong by just using keys directly in the top level object, so the object would have looked somewhat like:
 
     "id":"<hex session id>,
     "user_id": 1,
     "user_name": "<username>,
     "user_language":"en_us",
     "user_currency_id": "-99",
     "user_currency_name": "Euro"
 
Some people might argue that since you can have custom field names this can cause clashes. Except, it can't, since they're all suffixed with _c. So we're now logged in and can fetch all opportunities. This we do by posting:
method=>"get_entry_list",
input_type=>"JSON",
response_type=>"JSON",
rest_data=>to_json([
            $sid,
            $module,
            $where,
            "",
            $next,
            $fields,
            $links,
            1000
])
Why this is a list rather than a hash? Again, I don't know. A hash would make more sense to me. The resulting JSON looks like:
 
    "result_count" : 16,
    "relationship_list" : [],
    "entry_list" : [
        
          "name_value_list" :  
             "rt_status_c" :  
                "value" : "resolved",
                "name" : "rt_status_c"
              ,
             [ ]
           ,
          "module_name" : "Opportunities",
          "id" : "<entry_uuid>"
        ,
       [ ]
    ],
    "next_offset" : 16
 
Now, entry_list actually is a list here, which is good and all, but there's still the annoying name_value_list concept. Last, we want to update the record in Sugar, to do this we do:
method=>"set_entry",
input_type=>"JSON",
response_type=>"JSON",
rest_data=>to_json([
    $sid,
    "Opportunities",
    $fields
])
$fields is not a name_value_list, but instead is:
 
    "rt_status_c" : "resolved",
    "id" : "<status text>"
 
Why this works and my attempts at using a proper name_value_list didn't work? I have no idea. I think that pretty much sums it up. I'm sure there are other problems in there (such as the over 100 lines of support code for the about 20 lines of actual code that does useful work), though.

3 October 2011

Francois Marier: Three Firefox extensions to enhance SSL security

There has been a lot of talk recently questioning the trust authorities that underpin the SSL/TLS world. After a few high-profile incidents, it is clear that there is something wrong with this structure.

While some people have suggested that DNSSEC might solve this problem, here are three Firefox add-ons that can be used today to enhance the security of HTTPS:

Unlike the Convergence approach which completely takes over certificate handling, all three of the above add-ons can be used together.

Next.

Previous.