Search Results: "ana"

15 October 2008

John Goerzen: The Best Photo Printing Services

When I first got a digital SLR camera a few years ago, I spent some time evaluating photo printing options. I sent a set of test prints to several different companies, and wound up deciding to use Shutterfly for my photo printing.

It's been some time since then, so I figured it was about time to re-evaluate options.

There are three main things to consider when obtaining prints of photos: cost of prints, quality of prints at delivery, and longevity of prints. Many people look only at cost, and those that look past cost rarely look past quality at longevity. Longevity (or permanence) is, in my opinion, the most important factor to consider. But let's look at all of them.

Longevity

All color photo print technologies fade or otherwise degrade over time. There are several main things that can cause this: exposure to light, exposure to gases (especially ozone), and exposure to humidity. The impact of these different items varies by print technology, paper, and process.

Longevity: Traditional Photo Paper

Most services such as Shutterfly, Snapfish, Ofoto, as well as retailers such as Walmart, Target, and Walgreens print on traditional photo paper.

Historically, traditional photo paper has had terrible fading characteristics. Typically, old photos exposed to light -- for instance, hanging on a wall underneath a light bulb or in the sun -- will exhibit severe fade towards yellow or magenta. Modern papers are much better, but you will still not want to leave them directly in the sun or under lights.

Over at TPR, you can find several papers analyzing this. Perhaps the most interesting are test of digital prints and image permanence: comparing the technologies.

To attempt to summarize, what you see is that of the two common consumer-grade photo papers, Kodak Royal and Fuji Crystal Archive, the Fuji paper has better permanence. However, the pro-quality Kodak Professional Supra Endura is better than either of them by a significant margin. The photo papers didn't really have a problem with humidity, and not much of a problem with ozone.

Longevity: Inkjet

So what about inkjets and laser processes? The main thing I learned from my research is that there is a tremendous amount of variance here. When looking solely at top-quality inkjet papers and inks, they ranged everywhere from having little bits of them flake off after a few months to doing better than any of the photo papers. This variance occurred even with similar papers from the same manufacturer.

As far as light exposure is concerned, every inkjet paper tested by TPR did better than Kodak Royal. Some fared about the same as Fuji Crystal Archive. And some even beat out Kodak Supra Endura.

But look at the ozone exposure test and you see a far different picture. All of the photo papers did really well, though suprisingly Supra Endura was the worst of them (though the scale is so small here that the difference is pretty minimal).

Not one of the inkjet papers came even close to that, and some faded so fast that they couldn't even make it halfway through the test.

As far as humidity is concerned, it barely impacted the photo paper prints at all. Inkjet impacts were still mostly small, but also were mostly more impacted than the photo paper.

Longevity: Laser

And finally, how about laser prints. There was huge variety here. An HP printer they tested faded so fast it couldn't complete the light exposure test, while a Konica-Minolta fared better than Kodak Supra Endura in one test. In another light exposure test, they performed about the same as the photo papers. Ozone had little effect on the laser papers, except for the Dell test, which faded so fast they had to abort the test early. Humidity also had little effect on the laser prints.

Longevity: Other Examples

Brett Wilson did some interesting experiments with photos hung in front of a window as well. Wilhelm Imaging also has done some research in this area, though they are done using fluorescent lights only, which is prejudicial in favor of inkjet papers and against photo papers.

For the ultimate in longevity, true black and white photo paper is well-regarded.

TPR also has a report covering different lacquer and other finishes often applied to prints to improve their longevity.

Longevity: Conclusions

With the exact right combination of inkjet paper and ink, you can get a print that will last exceptionally long. Some inkjet paper and ink combinations have really poor longevity characteristics, however, as do the laser papers.

Photo papers, especially Kodak Supra Endura, but even Fuji Crystal Archive, will last quite awhile as well, and are probably the best storage if kept in dark conditions such as an album.

Print Quality

When looking at print quality, I'm concerned with correct color balance, saturation, cropping, consistency, sharpness, contract, and detail in both dark and light areas -- the typical things you'd look for.

Printing inkjet prints at home leaves all this up to your printer, inks, papers, and you, so we can't really compare meaningfully.

Back when I looked at it, Shutterfly and Ofoto did the best. Walmart, Walgreens, dotPhoto, and many of the other online sites were really too poor to even bother with. Ofoto had a bit of a yellow tint to it, while Shutterfly did pretty well.

Most of the reviews of the major sites I've seen lately rank Shutterfly or Snapfish at the top in terms of quality. Many also rank Ofoto/Kodak near the top.

Walmart and Walgreens produce consistently bad results, and I only use them if all that matters is speed. If you ever read a review where somebody says that they get good results from one of those places, find a review where somebody has a better idea what they're talking about.

Besides your mass-market sites like Shutterfly and Ofoto, there are also some slightly more expensive sites. The two best-known are mpix.com and adorama.com. Mpix is a division of Miller's Professional Imaging, a pro lab. Mpix is basically a website where you can submit photos in the way you would to one of these other sites, and they can do their thing. Their standard rate includes human color correction on every print, but they offer a cheaper rate if you don't need that. Their standard paper is Kodak Pro Supra Endura (the one that ranked highest in longevity tests), and the options go up from there. They also offer printing on true black and white paper, as well as wrap-around canvas and other such options.

I've sent some work their way and have been exceptionally happy with the result. Everything from the color of the prints down to the packaging shows extreme care and attention to detail. Among the reviews that cover mpix -- many don't -- it almost always comes out at the top of the pack, easily beating out Shutterfly and Snapfish. It can be more pricy (though sometimes actually cheaper, since shipping is flat-rate with them), though not significantly.

Adorama is also sometimes mentioned. Most people seem to prefer mpix over adorama, but some prefer adorama. It's a bit cheaper when you don't need color correction for some prints, but sometimes -- right now, for instance -- closes for weeks at a time to observe Jewish holidays. It didn't seem to be enough for me to bother.

Cost

Perhaps the easiest to figure out is cost, since websites typically list it right there for you.

High-quality inkjet prints are by far the most expensive option. I looked at 4x6 prints. Using the high-quality paper/ink combinations that did well in the TPR tests, and when buying supplies in bulk to get the best possible discount, it's still 25 cents or more for a single 4x6 print. That's with doing all the color work yourself, and not even figuring in the cost of the inevitable experimenting with getting the software set up right and the wasted prints when ink gets low. It also doesn't figure in the cost of the printer.

At mpix, if you do your own color correction, it's 19 cents per print, plus shipping. Prints are on Kodak Professional Supra Endura Shipping is a flat rate of $3 for 50 or fewer 4x6 or smaller prints, or a flat rate of $5.95 for USPS Priority Mail for anything else. They offer FedEx at an additional cost, which maxes out at a flat rate of $12.75 for overnight shipping. Some items such as framed canvas prints are more expensive.

Other options at an additional cost include color correction by a human, metallic paper, Ilford B&W paper, etc. There are also some more typical pro-lab options: red-eye removal by a human, custom retouching, etc.

At Shutterfly, a 4x6 print goes for 15 cents. You can get them as low as 10 cents if you buy a pre-paid plan of 600 prints for $60. Shipping varies based on the number of prints, starting at $1.79 for up to 10 prints, ranging up to $21 for 500 prints, with 3 cents per print after that. Priority and express mail is available at an additional charge. Shutterfly's standard paper is Fuji Crystal Archive.

At Snapfish, a 4x6 print goes for 9 cents. You can get them as low as 8 cents if you buy a pre-paid plan of 250 prints. Shipping at Snapfish follows a complicated formula, but is generally approximately the same as Shutterfly. Though if you print more than 595 prints, watch out for the 10 cents per print overage fee. They don't state what paper they use. Recent reports seem to suggest that it's Fuji Crystal Archive, but some reports claim it's Kodak Royal.

My Conclusions

I'm planning to switch most of my prints to mpix -- their higher quality and cheaper shipping offset their higher per-print cost for me. Also, I plan to try out Snapfish and see what sort of quality I get from them for things where quality isn't really that critical.

MJ Ray: Poverty, Credit Unions and Community Banking

Today is Blog Action Day about poverty. As a member of several cooperatives, I have a pretty strong hope that the responsible lending of credit unions can help people escape from poverty. The basic process of a credit union is that you save for a while, then when you’ve shown that you will put aside that amount of money regularly, you can borrow some multiple of it and repay at the same rate you were saving. In addition, you get a voice in controlling the credit union itself, although it is limited by financial services regulations and so on. At yesterday’s meeting of the Cooperatives-SW board in Taunton, some members expressed concern about a “social enterprise” called South-West Pound backed by a company called South West Community Banking Partnership. Although it’s called that, apparently it’s not a bank and also not a credit union: its members don’t manage it. It’s working with credit unions, but I’m very unclear on how it’s regulated. From the few details on their website, it looks to me like it could even be running a Farepak-style scheme which happens to hold deposits in credit unions instead of banks. Why is concealing member-controlled unions allowable as a “social enterprise”? Why would anyone go to a “community banking partnership” instead of direct to a credit union? (Coo, I’m getting a bit angry again, both at the lack of information and the hide-them-to-promote-them idea.) Tomorrow (Thu 16 Oct) is International Credit Union Day. If you’ve not already done so, please go open an account and help your local community to fund itself, then see if you can get involved in publicising them to communities who suffer financial exclusion.

MJ Ray: Software in the Public Interest October 2008

The monthly IRC board meeting of Software in the Public Interest will take place later today, as announced by SPI’s secretary last week. While the announcement is back on time (yay!), the agenda isn’t (aww!). I’d be quite interested to learn how SPI is going to try to reduce the risk to its reserves, given the current slow decline of its primary bank which is not one of the first US banks getting bailed out. I think the best way for not-for-profits to avoid risking donations at the moment is to avoid having them in their bank accounts, in line with the Better Business Bureau standard that
“the charity’s unrestricted net assets available for use should not be more than three times the size of the past year’s expenses or three times the size of the current year’s budget, whichever is higher.”
Back in June 2005, SPI’s board of the time (Ian Jackson, John Goerzen, Jimmy Kaplowitz, David Graham, Bruce Perens, Benj. Mako Hill, Branden Robinson) decided to “remain noncompliant” with that standard and I fear that chicken could be coming home to roost now. I hope we don’t lose anything, but AIUI we’ve got nearly $150,000 in play. Update: Unlike its UK analogue, the Federal Deposit Insurance Corporation covers corporation accounts up to $250,000, so SPI is only risking temporary unavailability, not yet a risk of loss. Thanks to bd_ for pointing me to that.

14 October 2008

Vincent Fourmond: Another bzr2git example

Today, I tried to switch one of my personal repositories for debian-related scripts from bzr to git, just like I've done for pmount. The trick was that this time, I wanted to only move a part of the repository, not the full repository. I had only been working before with CVS and SVN who have the idea of modules, which is not the case for bzr or git. So I made the mistake of using a single bzr for several projects, which I realize now is not a very good idea... In any case, I managed to drop some parts and here is how I did it; the trick was precisely to use SVN as an intermediate storage medium, using the following configurations files. First, bzr2svn.conf
[DEFAULT]
verbose = True
patch-name-format = ""
[project]
source = bzr:source
target = svn:target
start-revision = INITIAL
state-file = tailor.state
[bzr:source]
repository = /home/vincent/debian-devel/bzr/scripts
[svn:target]
repository = file:///tmp/testtai
module = biniou
and svn2git.conf:
[DEFAULT]
verbose = True
patch-name-format = ""
[project]
source = svn:source
target = git:target
start-revision = INITIAL
root-directory = /home/vincent/tmp/debian-mr-copyright-mode
state-file = tailor.state
[svn:source]
repository = file:///tmp/testtai
module = biniou/debian-mr-copyright-mode
[git:target]
git-command=/usr/bin/git
Then, all that is left to do is to run tailor on the resulting configuration files:
~ tailor -D -c bzr2svn.conf
~ tailor -D -c svn2git.conf

Francesco P. Lovergine: OSM and N810

I finally managed to use my Nokia N810 to do some GPS logging for OSM. The Nokia internal GPS is quite slow, so I used an external antenna and the well known Maemo Mapper tracking capability to log my first 60 kms in my area. I'm not too much happy with the result, because it seems the Nokia tablet is quite slow in tracking points: other trackers are someway configurable on those regards, but Mapper seems not taking in account any user preference about data sampling. Too bad, even if it is able to use OSM maps as default background, which is a great plus for a mapper. Definitively it needs a look to sources to understand what happens under the cover.

13 October 2008

Russell Coker: Some RAID Issues

I just read an interesting paper titled An Analysis of Data Corruption in the Storage Stack [1]. It contains an analysis of the data from 1,530,000 disks running at NetApp customer sites. The amount of corruption is worrying, as is the amount of effort that is needed to detect them. NetApp devices have regular “RAID scrubbing” which involves reading all data on all disks at some quiet time and making sure that the checksums match. They also store checksums of all written data. For “Enterprise” disks each sector stores 520 bytes, which means that a 4K data block is comprised of 8 sectors and has 64 bytes of storage for a checksum. For “Nearline” disks 9 sectors of 512 bytes are used to store a 4K data block and it’s checksum. These 64byte checksum includes the identity of the block in question, the NetApp WAFL filesystem writes a block in a different location every time, this allows the storage of snapshots of old versions and also means that when reading file data if the location that is read has data from a different file (or a different version of the same file) then it is known to be corrupt (sometimes writes don’t make it to disk). Page 3 of the document describes this. Page 13 has an analysis of error location and the fact that some disks are more likely to have errors at certain locations. They suggest configuring RAID stripes to be staggered so that you don’t have an entire stripe covering the bad spots on all disks in the array. One thing that was not directly stated in the article is the connection between the different layers. On a Unix system with software RAID you have a RAID device and a filesystem layer on top of that, and (in Linux at least) there is no way for a filesystem driver to say “you gave me a bad version of that block, please give me a different one”. Block checksum errors at the filesystem level are going to be often caused by corruption that leaves the rest of the RAID array intact, this means that the RAID stripe will have a mismatching checksum. But the RAID driver won’t know which disk has the error. If a filesystem did checksums on metadata (or data) blocks and the chunk size of the RAID was greater than the filesystem block size then when the filesystem detected an error a different version of the block could be generated from the parity. NetApp produced an interesting guest-post on the StorageMojo blog [2]. One point that they make is that Nearline disks try harder to re-read corrupt data from the disk. This means that a bad sector error will result in longer timeouts, but hopefully the data will be returned eventually. This is good if you only have a single disk, but if you have a RAID array it’s often better to just return an error and allow the data to be retrieved quickly from another disk. NetApp also claim that “Given the realities of today’s drives (plus all the trends indicating what we can expect from electro-mechanical storage devices in the near future) - protecting online data only via RAID 5 today verges on professional malpractice“, it’s a strong claim but they provide evidence to support it. Another relevant issue is the size of the RAID device. Here is a post that describes the issue of the Unrecoverable Error Rate (UER) and how it can impact large RAID-5 arrays [3]. The implication is that the larger the array (in GB/TB) the greater the need for RAID-6. It has been regarded for a long time that a larger number of disks in the array drove a greater need for RAID-6, but the idea that larger disks in a RAID array gives a greater need for RAID-6 is a new idea (to me at least). Now I am strongly advising all my clients to use RAID-6. Currently the only servers that I run which don’t have RAID-6 are legacy servers (some of which can be upgraded to RAID-6 - HP hardware RAID is really good in this regard) and small servers with two disks in a RAID-1 array.

Vincent Fourmond: Sanitizing bad UTF-8

I've hit a problem recently where statsvn does not manage to generate statistics on one of my repositories. After investigation, it turned out that there are some raw latin1 sequences in the XML log file (from a conversion from latin to UTF-8 long time ago), and that makes logfile parsing fail. To sanitize this, I use the following filter:
iconv -c -f UTF-8 -t UTF-8
This way, obnoxious sequences are dropped and processing can go on. (or, at least, could go on if it was not failing just a little later with a Null pointer exception...).

Jose Carlos Garcia Sogo: Compiz

Today my 5 years old laptop gave me some filesystem problem after running apt-get update, which resulted in some corrupted packages, among them libc6, with the result of my system becoming non-functional. So I needed a recovery CD to be able to mount read-only root partition. I had around an Ubuntu 8.04 Hardy Heron LTS, so I used it to run its included LiveCD and recover from there. At the same time I was curious about it, as I hadn’t had chance to test it before. One of the things that surprised me most was that Compiz was enabled by default. After a second thinking that this was going to be a problem, making the system unusable (remember, a 5 years old laptop, and I had checked myself compiz some time ago with a catastrophic result), I started to meve windoes, jump from one virtual desktop to other, and minimiza and maximize windows without a problem. The work that had been made amazed me! So, some fsck and apt-get commands after, I managed to reboot on my Debian system, and I installed compiz. The result: my system goes faster using compiz, with a bunch of effects enabled than running plain metacity window manager. I have a also installed some compiz fusion plugins, and they are running without a glitch and with an incredible speed. Obviously, I have not chosen full quality, but it is enough to show effects in an efficient way. I have improved my user experience and system speed at the same time… Once again Linux shows that it can get the best of old hardware. Good work, guys!

12 October 2008

Russell Coker: EC2 Security

One thing that concerns me about using any online service is the security. When that service is a virtual server running in another country the risks are greater than average. I’m currently investigating the Amazon EC2 service for some clients, and naturally I’m concerned about the security. Firstly they appear to have implemented a good range of Xen based security mechanisms, their documentation is worth reading by anyone who plans to run a Xen server for multiple users [1]. I think it would be a good thing if other providers would follow their example in documenting the ways that they protect their customers. Next they seem to have done a good job at securing the access to the service. You use public key encryption for all requests to the service and they generate the keypair. While later in this article I identify some areas that could be improved, I want to make it known that overall I think that EC2 is a good service and it seems generally better than average in every way. But it’s a high profile service which deserves a good deal of scrutiny and I’ve found some things that need to be improved. The first problem is when it comes to downloading anything of importance (kernel modules for use in a machine image, utility programs for managing AMIs, etc). All downloads are done via http (not https) and the files are not signed in any way. This is an obvious risk that anyone who controls a router could compromise EC2 instances by causing people to download hostile versions of the tools. The solution to this is to use https for the downloads AND to use GPG to sign the files, https is the most user-friendly way of authenticating the files (although it could be argued that anyone who lacks the skill needed to use GPG will never run a secure server anyway) and GPG allows end to end encryption and would allow me to verify files that a client is using if the signature was downloaded at the same time. More likely problems start when it comes to the machine images that they provide. They have images of Fedora Core 4, Fedora Core 6, and Fedora 8 available. Fedora releases are maintained until one month after the release of two subsequent versions [6], so Fedora 8 will end support one month after the release of Fedora 10 (which will be quite soon) and Fedora Core 6 and Fedora Core 4 have been out of support for a long time. I expect that someone who wanted to 0wn some servers that are well connected could get a list of exploits that work on FC4 or FC6 and try them out on machines running on EC2. While it is theoretically possible for Amazon staff to patch the FC4 images for all security holes that are discovered, it would be a lot of work, and it wouldn’t apply to all the repositories of FC4 software. So making FC4 usable as a secure base for an online service really isn’t a viable option. Amazon’s page on “Tips for Securing Your EC2 Instance” [3] mostly covers setting up ssh, I wonder whether anyone who needs advice on setting up ssh can ever hope to run a secure server on the net. It does have some useful information on managing the EC2 firewall that will be of general interest. One of the services that Amazon offers is to have “shared images” where any Amazon customer can share an image with the world. Amazon has a document about AMI security issues [4], but it seems to only be useful against clueless security mistakes by the person who creates an image not malice. If a hostile party creates a machine image you can expect that you won’t discover the problem by looking for open ports and checking for strange processes. The Amazon web page says “you should treat shared AMIs as you would any foreign code that you might consider deploying in your own data center and perform the appropriate due diligence“, the difference of course is that most foreign code that you might consider deploying comes from companies and is shipped in shrink-wrap packaging. I don’t count the high quality free software available in a typical Linux distribution in the same category as this “foreign code”. While some companies have accidentally shipped viruses on installation media in the past it has been quite rare. But I expect hostile AMIs on EC2 to be considerably more common. Amazon recommends that people know the source of the AMIs that they use. Of course there is a simple way of encouraging this, Amazon could refrain from providing a global directory of AMIs without descriptions (the output of “ec2dim -x all“) and instead force customers to subscribe to channels containing AMIs that have not been approved by Amazon staff (the images that purport to be from Oracle and Red Hat could easily have their sources verified and be listed as trusted images if they are what they appear to be). There seems to be no way of properly tracking the identity of the person who created a machine image within the Amazon service. The ec2dim command only gives an ID number for the creator (and there seems to be no API tool to get information on a user based on their ID). The web interface gives a name and an Amazon account name. The next issue is that of the kernel. Amazon notes that they include “vmsplice root exploit patch” in the 2.6.18 kernel image that they supply [2], however there have been a number of other Linux kernel security problems found since then and plenty of security issues for 2.6.18 were patched before the vmsplice bug was discovered - were they patched as well? The file date stamp on the kernel image and modules files of 20th Feb 2008 indicates that there are a few kernel security issues which are not patched in the Amazon kernel. To fix this the obvious solution is to use a modern distribution image. Of course without knowing what other patches they include (they mention a patch for better network performance) this is going to be difficult. It seems that we need some distribution packages of kernels designed for EC2, they would incorporate the Amazon patches and the Amazon configuration as well as all the latest security updates. I’ve started looking at the Amazon EC2 kernel image to see what I should incorporate from it to make a Debian kernel image. It would be good if we could get such a package included in an update to Debian/Lenny. Also Red Hat is partnering with Amazon to offer RHEL on EC2 [5], I’m sure that they provide good kernels as part of that service - but as the costs for RHEL on EC2 more than double the cost of the cheapest EC2 instance I expect that only the customers that need the the larger instances all the time will use it. The source for the RHEL kernels will of course be good for CentOS (binaries produced from such sources may be in CentOS already, I haven’t checked). This is not an exhaustive list of security issues related to EC2, I may end up writing a series of posts about this.

Y Giridhar Appaji Nag: te_IN translation for Debian Installer

My parents found it strange that I spend considerable time in the night doing "some Linux thing", so the other day I explained to them what Free software is and we talked about copyright and licensing. I was very happy that they were able to appreciate it. I also gave a Debian Installer demo. While there are a bunch of Indian languages supported in the Debian Installer, (Telugu) isn't one of them. They noticed it and motivated me to work on te_IN translation for d-i and promised to help me. So the three of us have been discussing translation strings. I have been committing changes slowly and intend to complete this activity in a few months. Lack of interest in translation meant that I never gave much thought to it but I see that translating software is rather difficult and it is an activity that would benefit a lot from two or three people doing it together. I found a glossary at swecha.org but there are a bunch of problems I am using these simple guidelines for translation: A few resources that I have been using: I will have to find a good quality comprehensive Telugu general knowledge book or an official A.P. govt. document to translate ISO 3166 codes etc. The Manorama yearbook would've been great, unfortunately Telugu is not one of the few languages it is published in. I will also bring my copy of Sankaranarayan's dictionary from my next month Hyderabad visit. If you have suggestions on any of these (pointers to guidelines, resources that are DFSG free and I could copy from etc.) that would make doing this easy for me, please do let me know. Naidu does good one-off Telugu string translations in Launchpad (so does [info]praveenkumarg, btw), and Naidu has promised to do some of the work as well as review. If all goes well, I will try and create a debian-10n-telugu sometime in the future.

Rob Taylor: Boston Awesome

I’m at the Boston GNOME summit right now. Yesterday a lot of work got done on gobject-introspection and AT-SPI D-Bus, which is completely awesome. I had a great discussion with Dan Williams about where we should go with gnome-phone-manager, NM’s new ModemManager and OpenMoko’s gpsd. My interest here is having a consistent place for getting GSM/CDMA cell location information. A lot of very interesting ideas have been flying around about new user experience ideas. It seems a lot of people have been thinking similar things but didn’t really know that others were also working/thinking on the same ideas. We’ve just started the future user experience hackfest, I hope out of this we can forge some common future direction and common projects. I’ll report more after the hackfest!

10 October 2008

Julien Danjou: Space invaders for awesome

I was sure it will be possible. With current awesome git version it's possible to run a space invaders game. Space invaders With only 380 lines of Lua. Is there any window manager capable of doing this ? ;-) And don't say it's useless! Thanks to Gregor "farhaven" Best for wasting his time with that.

9 October 2008

Dirk Eddelbuettel: More running data visualization

A few years into this running hobby, I realized that my times were getting better. But I had no feel for by how much, or whether that was a constant rate of improvement etc pp. Long story short, I started to plot some of the data. What seemed natural was to record the date, the distance in miles as well as in a qualitative variable, and finally the average pace. Additionally, I played with groupings into just three categories 'short', 'mid' and 'long'. This leads to a natural 'one-factor' model of pace as a function of race date grouped by race distance. And given how easy it is to do conditional plots in R, I quickly arrived at something that already resembled the following chart: (pace by date given group lattice chart) At first, some of the groups had too few data points to actually reliably construct regression lines, let alone non-parametric smoothers. But over time more and more data points were added as I kept running races. Including for example the somewhat disappointing result from last year's Chicago marathon in record heat that resulted in the outlier in the last panel. It actually made the smooth fit turn upwards! Luckily, the subsequent times in New York last fall, London in April, and of course in Berlin last month helped to dampen the effect of the one outlier, resulting in a more normal straight line for marathon performance that is comparable to the other four race lengths. All in all I am now quite happy with the chart. The combination of the non-parametric loess smoother and the robust linear regression (using lrm from the MASS package for R) shows that most groups exhibit very little non-linearity as both regression curves are very close to each other. The curvature in the '10m' group is probably mostly a small-sample effect. And I am obviously happy with the fact that three of the five panels show their respective last race as a PR :) The R script containing the data and code is available here but requires some familiarity with the lattice package for R (as the lattice book would provide).

Felipe Augusto van de Wiel: 10 Oct 2008

Random news

/back
Yes, I was away... long and sad story involving health issues. :-( I'm trying to get back, hopefully everything will be fine. Let me publicly apologize to some people that was expecting some work from me and I postponed it, not because of slackness but because I was unable to do it. I'm catching up with a lot of tasks and trying to solve long standing issues, feel free to ping me to get some update on a specific matter. I am really sorry!


Slogan
Do you know about the poll to choose a slogan for the release of Debian 5.0, also known as "lenny"?

No!? Shame on you! :-)

People behind Debian Art started a discussion to get opinions! After some input they prepared a poll, maybe they should try to use a doodle. Right now, The Universal Operating System is quite popular, but a lot of people did some nice suggestions, and some of them would like to see a slogan some way related to the Toy Story (tm) character Lenny:



The discussion is distributed across three lists: Debian Desktop, Debian Publicity and Debian Curiosa, I would also point to the comments made by Gustavo Noronha da Silva (here) and Andre Felipe Machado (here).

If you didn't say anything so far, please do!

8 October 2008

Sven Mueller: How to solve a credit crisis

Anand Kumria wrote:
If IBM were to go bankrupt, would the government step in? Unlikely. Investors would lose (money), staff — another word for investors — would lose (jobs), but customers would win (their computers would keep working). Some customers would win more than others (especially those who had the equipment on lease); if no one is collecting, why pay?
I’m wondering here where Anand got the idea that once a company went bankrupt, that you don’t need to pay to that company anymore. When a company goes bankrupt, at least in Germany the following happens: A trustee/liquidator is selected. This liquidator is then collecting the information who owes money to the company and who still needs to get (how much) money from it. The liquidator also has to check the option of selling company assets (which might include the contracts of customers that still have to pay) to fulfill the debts of the bankrupt company. After he turned all assets into money, the money is distributed among those who still have to get money from the bankrupt company. Anyway, regarding his main argument that the (average) customer of a company (bank in this case) should never have to pay for the failed speculations of that company, I somehow have to agree with him. Someone putting money into a regular bank account or papers with fixed interest rate should never lose his money. But there are also customers buying bank shares with a chance of higher revenue than with fixed interest rate papers. These should suffer from failure of the bank management, as they more or less explicitly wanted to be tied to the success (or failure) of the bank. However, this is mostly irrelevant, since the failure of so many “investment banks” has side affects that might cost the average inhabitant of the affected countries even more than the discussed rescue plans. One of these effects is that the banks are now much more conservative regarding lease and mortgage plans, effectively leaving many home owners with no option to fulfill previous obligations (remaining debt after a previous mortgage expired can’t be refinanced by a new mortgage), causing them to have to sell their homes to pay the first mortgage. This is in some way stupid because this causes people who were perfectly paying their mortgage rates to loose their house, while the bank which would be giving them a new mortgage could get a new and good customer, improving their income. On the other hand, if the other side effects of the current crisis cause those “good” mortgage customers to loose their jobs, they might turn into bad customers who are unable to pay their rates. All in all, this is a spiral that could cause the whole economy to break down (a small example: The bank is not giving out mortgages, so no one will build new houses so the builders loose their jobs so they don’t pay their mortgage rates anymore,…. - over simplified, but still shows what I mean). Unless the spiral is terminated in time, before too drastic things happen. All in all, I do understand why the politicians try to rescue those banks (or at least the customers of those banks), though I think that in an economy with slightly higher regulation, there wouldn’t be the need for such a rescue plan. I know there are some german banks affected by the crisis as well (among them Hypo Real Estate and others), but the average private customer of such banks shouldn’t loose money due to the regulations we have in place. In general, there should be some security fund which makes sure that private customers never loose money put into regular bank accounts or fixed interest papers, vice-versa, banks should calculate mortgages so that they can be pretty sure their customers are actually able to pay off their rates - it doesn’t make sense if someone starts off having to pay 500$/month for their mortgage and has to pay over 1000$ a few years (as in 2-3 years) later, because the bank raised the interest that much. I have no problem with people loosing money from shares of banks or other companies directly or indirectly through investment funds.

MJ Ray: Nestle Boycott 20th Anniversary

It’s the 20th anniversary of the Nestle Boycott this week. The theme is “Change Nestl , Change the World”. This is one of few boycotts that I wholeheartedly support and participate in. While reading a report from the NI cooperative, I learnt of a new outrage by Nestle. Over in Canada, Wellington Water Watchers document a Nestle plant that is (as I understand it), taking drinking water for free, bottling it as “Aqua Fina”, trucking it to cities and selling it. (Thanks to whoever it was gave me the WWW link - I didn’t find the email, so please claim it in a comment if you want it.) You may remember that when Coca-Cola tried to do the same by selling contaminated Kent tap water as “Dasani”, it was widely ridiculed as Peckham Spring. Finally, “Dasani” was withdrawn from sale. It’s disappointing that Nestle’s Peckham Spring seems to be on sale in Canada, at a lower “manufacturing” cost than Coke’s UK one. Do we really want multinationals selling us our tap water in an inefficient way? No. Boycott Nestle.

Ian Wienand: Why symbol visibility is good

ELF has two related concepts for managing symbols in your programs. The first concept is the symbol binding. Global binding means the symbol is visible outside the file being built; local binding is the opposite and keeps the symbol local only (static) and weak is like global, but suggests that the symbol can be overridden.
$ cat syms.c
static int local(void)    
int global(void)    
int  __attribute__((weak)) weak(void)    
$ gcc -o syms -c syms.c
$ readelf --syms ./syms
Symbol table '.symtab' contains 10 entries:
   Num:    Value  Size Type    Bind   Vis      Ndx Name
...
     5: 00000000     8 FUNC    LOCAL  DEFAULT    1 local
     8: 00000008     8 FUNC    GLOBAL DEFAULT    1 global
     9: 00000010     8 FUNC    WEAK   DEFAULT    1 weak
...
This is all well and good, but starts breaking down when you want to load many different modules and keep strict API's (such as, say, dynamic libraries!). Consider that for two files to share a common function, the function must end up with a global visibility.
$ cat file1.c
void common_but_not_part_of_api(void)    
$ cat file2.c
extern void common_but_not_part_of_api(void);
void api_function(void)  
     common_but_not_part_of_api();
 
$ gcc -shared -fPIC  -o library file1.c file2.c
$ readelf --syms ./library
Symbol table '.symtab' contains 60 entries:
   Num:    Value  Size Type    Bind   Vis      Ndx Name
...
    53: 00000424    29 FUNC    GLOBAL DEFAULT   11 api_function
    55: 0000041c     5 FUNC    GLOBAL DEFAULT   11 common_but_not_part_of_api
...
In the example above, both the function we want exported (api_function) and the function we don't wish exported (common_but_not_part_of_api) end up with exactly the same attributes. Binding attributes are useful for the linker putting together object files; but aren't a complete solution. To combat this, ELF provides for visibility attributes. Symbols can be default, protected, hidden or internal. Using these attributes, we can flag extra information for the dynamic loader so it can know which symbols are for public consumption, and which are for internal use only. The most logical way to use this is to make all symbols by default hidden with -fvisibility=hidden and then "punch holes in the wall" for those symbols you want visible.
$ cat file1.c
void common_but_not_part_of_api(void)    
$ cat file2.c
extern void common_but_not_part_of_api(void);
void  __attribute__((visibility("default"))) api_function(void)  
      common_but_not_part_of_api();
 
$ gcc -fvisibility=hidden -shared -fPIC  -o library file1.c file2.c
$ readelf --syms ./library
Symbol table '.symtab' contains 60 entries:
   Num:    Value  Size Type    Bind   Vis      Ndx Name
    48: 000003cc     5 FUNC    LOCAL  HIDDEN   11 common_but_not_part_of_api
    54: 000003d4    29 FUNC    GLOBAL DEFAULT  11 api_function
Now the dynamic loader has enough information to distinguish between the two, and can stop any external access to common_but_not_part_of_api easily. This extra information also has potential for performance improvements. Any time a symbol may be overridden, the compiler must generate a program lookup table (PLT) entry for the function so that the dynamic loader can re-direct the function call. The PLT is a trampoline which gets the correct address of the function being called (from the global offset table, GOT) and bounces the function call to the right place. An example should illustrate: Bouncing via the PLT In the first example, there was not enough information to tell if the function would ever be able to be overridden, hence a PLT entry had to be created and the function called through it (disassemble it to see the details!). With correct symbol visibility attributes, there is enough information to know that common_but_not_part_of_api is never to be overridden, hence the PLT (and the associated costs of trampolining) can be avoided. The internal attribute is even stricter; it says that this function will never be called from outside this module (for example, we might pass the address of common_but_not_part_of_api to anyone). This can lead to even better code, because on many architectures transitioning to another module might involve flipping global pointer registers or other similarly expensive operations. So that's how symbol binding and visibility attributes can work together to get you the best performance possible from your program!

7 October 2008

Kai Hendry: Marking up mail

Marking up email on the Web is not a pretty picture in 2008. What I look like on a telecon Notable mail archiving services: Random thoughts: Any other relevant RFCs, tools or tips I might have missed?

Alexander Reichle-Schmehl: What you can do for Lenny (Update #2)

You probably noticed by now, that Debian GNU/Linux 5.0 aka Lenny hasn't been released in September. Well, that's a shame, but very easy to explain: Too many release critical bugs. Well, it's pretty hard to estimate, how fast RC bugs will be fixed, and apparently our release team was a bit too optimistic :( The big question is: What can you do, to help release Lenny at least in this quarter? That's pretty easy: Fix rc-bugs, take care, that the fixed packages are migrated to Lenny, do upgrade tests, document problems in the release-notes. Pretty simple, isn't it? For users Even as a simple user (aren't we all just users?) you may help getting Lenny released. Some things you can do: See? Even as a simple user without deeper technical knowledge you can help us getting Lenny in shape to be released. If you have technical knowledge: Very good! You might want to read the next section, too, and see what applies to you, there :) For maintainers It basically boils down to two things: If your packages have RC bugs open in Lenny fix them and take care, that the fix will propagate to Lenny. If your packages don't have RC bugs open, fix someone else's RC bug. Surely you don't think, the release team will fix the remaining rc bugs, do you? And surely you understand, that your shiny rc bug free packages are kind of useless, if they aren't released? To search for bugs to be fixed, take a look at the unofficial rc bugs thingy. The URL lists RC bugs open in both Sid and Lenny. Obviously they should be fixed ASAP. If no one takes care about these packages, they might be removed from Lenny (if possible). Again: Try to reproduce the bug, try fix it, upload an NMU (or send your patch to the bug report and search for an sponsor). You'll notice, that some of these bugs already have a patch. In that case, your job would be to test the patch, report that to the bug report and offer to sponsor an NMU. Another interesting list is the list of rc bugs open only in Lenny. These bugs have been fixed, but the fix hasn't propagated to Lenny, yet. Normally, the release team will grant freeze exceptions for these packages if possible. However, if the changes to the fixed version are quite grave or the package in Sid depends on a newer package than in Lenny that's not possible. In these cases look out for packages marked as need tpu upload or similar. Oh, and if you could refrain from upload new upstream versions of packages to Sid, you would make all our lives easier. Some reasons: And of course you should spend your time fixing rc bugs anyway ;) Update: Rhonda pointed out, that running rc-alert -d TU -o and will limit the output to bugs open in both Sid and Lenny, which is the more interesting list of bugs.

6 October 2008

Matthew Garrett

Things:
Things I want to do:

Next.

Previous.