Search Results: "cas"

2 October 2007

Jonathan McDowell: Moving on

On Wednesday 17th October the majority of my belongings will leave Norwich, to arrive in Castlerock at some point the next week (exact day to be confirmed). I will then start work for 3par as a software developer in Belfast on Monday 29th. It's been 6 years since we moved to Norwich and I'll be sad to leave it, and all the friends I've made here, behind. I'd have stayed a bit longer but 3par were keen for me to get started. I imagine I'll be in the Fat Cat on Saturday 13th if people are around. And I expect visited in NI once we get settled in!

Neil Williams: 2038 overflows

Adrian (also sent by email):
Re: 2038 issues on mobile platforms.

It is possible to handle 64bit time on 32bit systems. I got disillusioned with the libglib2.0-0 support (GDate) and I already had a library capable of 64bit mathematics (libqof1) so I wrote QofDate and QofTime that can handle true 64bit date and time calculations, formatting and scanning on 32bit or 64bit installations. I estimate that this gives us enough capacity to handle time for a couple of dozen times the age of the universe to nanosecond precision. The maths can also go backwards as far as it can go forwards so it could handle the date of the Big Bang with nanosecond precision too - if anybody actually works that out.
:-)
I'll be using this library in Emdebian to provide support for 64bit maths for gpe-expenses and the pre-alpha gpe-cash (stripped out gnucash for embedded devices) so at least your financial records will be safe.
http://cashutil.sourceforge.net/gpecash.php
http://gpe-expenses.sourceforge.net/

"QofTime is a lot like the unix 'struct timespec' except that it uses a
64-bit signed int to store the seconds. This should adequately cover
dates in the distant future as well as the distant past, as long as
these are not more than a couple of dozen times the age of the
universe. Values of this type can range from -9,223,372,036,854,775,808
to 9,223,372,036,854,775,807."


http://qof.sourceforge.net/doxy/group__QOFTIME.html#gb93e930907d0e2eba07b1fa7496e7822

http://qof.sourceforge.net/doxy/group__QOFTIME.html

QofTime is defined as 64bit on all systems and will not overflow until
the year 292,471,208,679 (not counting leap years). i.e. approx.
9223372036854775808 / (60*60*24*365). This also means that some values
of QofTime cannot be converted to a time_t on systems where time_t is
defined as 32bit.

http://qof.sourceforge.net/doxy/group__Date.html

A QofDate is theoretically able to go forward to the year
292,471,206,707 AD and back to the year 292,471,206,708 BC. Whether
such dates actually exist is outside the scope of this documentation.

http://packages.debian.org/sid/libqof-doc
http://packages.debian.org/sid/libqof1

1 October 2007

Joey Hess: pristine-tar followup

Madduck wondered why a package using revision control needed pristine tarballs at all. Firstly, the example he gave of tagging an upstream release and creating a tarball of it to build the package against is exactly what pristine-tar does. Except it preserves permissions and timestamps and other tar cruft. Sometimes timestamp or permissions issues do cause issues with package builds, so being able to build against the original files from the pristine tarball is potentially useful to catch and fix such problems. As to other uses of pristine tarballs... It's hard to list the advantages to using pristine tarballs, since Debian (rpm too) has had them forever, and lots of people may depend on them in ways that we don't anticipate. For example, I've seen FreeBSD point at ftp.debian.org to get tarballs for their packages, since they don't archive tarballs on their own, and sometimes the tarball isn't available upstream (or ftp,debian.org is the upstream). That's a wacky example, but I've seen it more often than I've seen pristine tarballs used for the often-cited md5sum comparison case. Rather than try to enumerate all the possible reasons for pristine tarballs, and then try to whittle the list down to reasons that actually make sense, and cost-benefit analise those, it seemed better to just try to write a solution to the problem. (Or at least, to most of it -- a hopefully small subset of the set of reasons for pristine tarballs are not satisfied by pristine-tar.) I've also been theoretically interested in this problem of generating pristine tarballs from version controlled source ever since Scott Remnant mentioned at a meeting (140 mb video of meeting here) that he had a tool to do it -- and AFAIK never shared the tool! So the challange factor was another reason to write it. However, pristine-tar has already saved me hundreds of megabytes of disk space, sped up my release process (avoiding bug #225483) and simplified the way I work, so it's already a net win for me, even if it turns out we don't need pristine tarballs for anything at all!

30 September 2007

Marc 'Zugschlus' Haber: Booting from USB stick on older hardware

I spent the better part of the weekend with trying to get an older Athlon XP mainboard to boot grub from an USB stick. Unfortunately, without success. Since a few months, my notebook boots Windows XP when it is booted naively. Only if the correct USB stick is plugged in on boot, the Debian system that I work with comes up. The USB stick has grub, a kernel and an initrd which is used to boot and unlock the crypto file systems (crypted LVs) on the disk and can be removed once the system has booted. I wanted to use the same scheme for a “newly built” new desktop from old parts. Unfortunately, this was much harder than I expected. Since a few years, I use the USB-ZIP format for USB sticks that need to boot. It has shown that this format has the best chance to successfully boot. For more information about this format and about how to build it, see /usr/share/syslinux/README.usbkey on any recent Debian system with syslinux installed. My grml USB stick that I always have on me is formatted like this, and the “new” desktop boots it just fine. The boot stick for my notebook is an USB-ZIP as well, only that its primary partition is not FAT, but ext3. Unfortunately, the BIOS of the “new” desktop seems to refuse booting grub. Neither my notebook’s stick nor the new stick even come to the “loading stage x” output of grub, let alone to the menu. syslinux works fine, and both sticks boot just fine on my notebook and on a qemu on the “new” desktop. Since the grub command line has saved me multiple times in the past, I didn’t really like the idea of having to use syslinux, and decided to boot the “new” desktop conventionally from hard disk without the use of an USB boot stick. Until I got this far, the complete saturday and the better part of sunday was wasted. Darn. This reminds me of a mainboard from 2002 (incidentally from the same vendor), which checked whether the first hard disk had an active primary partition on it before it even tried to execute the MBR code. If no primary partition was found (which was common case with the partitioning scheme that I used back then which only had logical drives - this was before I discovered LVM), the BIOS gave the error message “Not found active primary partition” (sic!) and refused to boot. Stupid programmers trying to outsmart the sysadmins of their products. But at least, the error message was reasonably clear back then.

Christian Perrier: RWC: Fiji and Scotland in 1/4th

Yesterday, New Zealand crushed Romania (85-8), just like they crushed all other teams they faced up to now. Of course, my only hope is that they stop crushing teams...as France is very likely to be the next one. Indeed, the only question about New Zealand is about their defensive skills as all teams they faced scored one try against them. Particularly, their wing players seem to be pretty "weak" in defense ("weak" in terms of New Zealand...:-)). I haven't seen anything of Australia-Canada (37-6). Apparently, the Australians were pretty quiet (6 tries, still!). Very unfortunately, I haven't seen THE match between Fiji and Wales (38-34). The Fiji team created an "half surprise" by qualifying for 1/4th finals. All reports I read talk about a wonderful match, with great indecision, a full commitment by both teams, and 9 tries. Sigh. I screwed programming my VCR and we were strongly disappointed this morning. Finally, Scotland managed to qualify after beating Italy 18-16. Up to the very last minute, Italy hoped to qualify but Chris Paterson scoring 6 penalties out of 6 attempted converted all occasions the Scots had to score points. I'm slightly disappointed for the Italians who played great games all along this World Cup and would have deserved to qualify and meet Argentina, France or Ireland. Today, the final 2 teams for the 1/4th finals will be determined. Argentina has great chances to finish first (they only need one bonus point) and meet Scotland. France needs to win against Georgia. Scoring the bonus point would secure the qualification for them. Ireland has little chances to qualify as they need to beat Argentina without Argentina to score a bonus point, therefore with more than 7 points difference. The most probable ticket is Argentina finishing first and France 2nd (thus meeting New Zealand).

Christian Perrier: Smith reviews

This project that sounded like a joke when a French person announced on April 1st that he plans to review English in some texts of Debian apckages, is now well going on. Contrary to what the status page shows, this is far from being a single-person project. I receive invaluable help from a few people, with a very special mention to Justin B Rye and MJ Ray. Both of them (and a few other people, including the package maintainers) carefully review my rewrites for the considerable amount of Frenglish I introduce. One of the funniest part is reading both of them sarcastically commenting on the "dialect" we're using as we decided to go for US-style spelling and writing style (by far the most used in Debian) while MJ and Justin are definitely citizens of the UK (and very clever in GB English). But both "play the game" ('jouer le jeu' as we way in French) very constructively. I think that the improvement for packages at the end of the process (a very long one as it includes translation updates for debconf templates) is noticeable and nearly always much appreciated by package maintainers. We are indeed only limited by time. I have decided to begin with packages that use debconf (a few hundreds) and I cannot start more than 1 package per day (the initial review is very time-consuming: about 20 minutes for one package and up to 1 hour for some of them). So, of course, new contributors will be welcomed (even those speaking dialects!).

29 September 2007

Jeff Bailey: And the next stop is...

One of the things through this move has been that we've been trying to make sure that whatever place we live in is close enough to "stuff" that Angie doesn't go nuts. The bitch about corporate housing is that there's no point in trying to meet your neighbours - everyone is transient. While I never came home to the walls smeared with f ces but I think she was contemplating it.

So, we're in new corporate housing now in Mountain View. This time we're a block off of Castro and El Camino Real, several neighbours have said hi, there's other families around and a starbucks around the corner. I'm expecting she'll be much happier here.

And I found a store that sells lychee-fruit cups. I haven't had those since I left Vancouver! The salt-water taffy store's products aren't vegan, sadly.

Martin F. Krafft: Uninformed slagger

Scott, I would have expected a little more from you than your previous post on why you chose bazaar! Update: my initial post was offensive towards Scott, so I edited it a little, following a good discussion with Scott via Email. Even though his post didn't make it explicit, his post was meant to be a reaction to someone else praising Git above everything else. Or, in his words:
You're responding with a defence of git because I posted a defence of bzr in response to championing of git.
I decided to leave most of the original post untouched, nevertheless, but please read it in this context. I guess the sensible conclusion is (as always): to each their own. Inline updates are enclosed in [brackets].
Git is less of a version control system than it is a filesystem and a means to communicate among developers, and when used in that sense, it's extremely powerful and intuitive to use. And it is also a version control system, like Bazaar. It's blazing speed isn't the only killer feature. By no means, however, is it "heavily optimised for the 'I only apply patches' development model, at the expense of ordinary development models" [anymore]. It also supports centralised workflows just as well as it supports distributed approaches, and you can switch between them [this is not really true when compared to how Bazaar mimiques CVS/SVN behaviour; Git cannot do that]. Here are the git commands needed to do the same things Scott showed off in your post:
$ cd myproject
$ git init
$ editor .gitignore
$ git add .    # git rm ...
Let's branch before we commit. checkout -b is a shortcut to create a branch (git branch myproject-foo) and immediately switch to it (git checkout myproject-foo):
$ git checkout -b myproject-foo
$ git commit
Git keeps all branches within the same repository and lets you switch between them. Now, that is a killer feature: whether I'm versioning code or stuff like webpages, I often have external pointers to my code. For instance, I may have libfoo as a sibling of foo-ng and the latter refers to libfoo in the Makefile. If I wanted to try a new approach in libfoo, I'd have to change foo-ng's Makefile to point to libfoo-newbranch instead, or juggle directories around. That sucks. With git, I just checkout a different branch in the same space:
$ git checkout master
And even better: if you don't like it, you could just as well copy the repository to create a separate (remote) branch from which you can later pull or cherry-pick changes to the main branch:
$ git remote add foo ../myproject-foo
$ git fetch foo
$ git merge foo/master
Now back to your steps, and I'll stick to the git way of branching. Note how git does not differentiate between pulling and merging in the sense that you do. If there are no local modifications, git just fast-forwards the branch with the commits on the master:
$ git merge myproject-foo
Even better, say you don't want the last two commits:
$ git merge myproject-foo~2
Unless git can fast-forward, it also treats a merge as a single commit and the ancestry information allows you to inspect its component commits just like you would. gitk is another killer feature, which I believe bazaar copied from git, just like the bazaar folks adopted the concept of rebasing, which basically lets you rewrite history quite freely, another killer feature (which can be quite badly abused if you don't know what you're doing). And then there is git bisect and git-svn and git filter-branch and git describe and git stash and gitweb and, and, and So if I compare the git commands I used to the way you used bazaar, I fail to see much of a difference. Do you? It is true that git comes with an astounding number of additional commands, most of which you never have to use, but which are available to the user [but its interface is a huge improvement over GNU arch, nevertheless, and if only because the commands are named more logically and revisions and branches do not have to adhere to one of the strangest syntaxes ever]. I don't deny that their sheer abundance is overwhelming and confusing at first (a general rule for Git for beginners seems to be just to ignore ever command with a hyphen in it). But once you get out of the newbie stage (it took me about two weeks, the duration of Debconf7 to become an acquainted git user; in fact, David Nusinow placed me among the ranks of real git wizards Pierre and Keith before the conference was over) arghs, I hate long remarks in parenthesis; once you leave the newbie stage, you'll appreciate all these commands, which make it really easy to script complex tools with git, making any plugin architecture pretty obsolete. Finally, I found myself trusting git way more than any other version control system I've tried simply because it's so transparent and you are allowed to edit stuff under .git, which is mostly plan text files (and objects, which you [could but] don't have to touch). It's transparent, and it stays true to the Unix way of letting small tools do their jobs instead of the monolithic approach. NP: Enchant: Tug of War Update: Carl Worth has a similar beef with people thinking that Git is too complicated and has ported a chapter from a book on Mercurial to Git to show how that's not the case. It's a good read. Update: Scott replied (quoted with permission), and I include my replies inline:
"I can tell you why you chose bazaar: because you had to, because you are being paid to use it."
Sorry, that just isn't true. I'm paid to lead the Ubuntu Desktop Team; this has nothing to do with version control, except that the various upstreams and packages use different systems.
I am sorry about this accusation. It was low.
You appear to miss my point about Git, or maybe I didn't make it strong enough. It's not that I don't understand it; it's that I had to take time and effort to understand it. And I've noticed that every reply to my post misses out the fact that with Git, you have to take action to ensure your changes are actually incorporated into the commit; either with git add or commit -a. This exactly the kind of anti-social behaviour that I have issues with.
Well, Git people call this a feature and it is very handy, but I see your point. There is no way to configure git-commit to always include -a, so you'll have to work around it:
git config alias.ci 'commit -a'
This creates git ci to do exactly what you want. You cannot overwrite commit that way though. But it could be trivial to add a configuration option to git-commit, if you think that the way that Git exposes the index is an adoption-stopper:
$ file -L =git-commit
/usr/local/bin/git-commit: POSIX shell script text executable
Your post also introduces yet more over-complication by demonstrating about making a branch (in an empty branch) before you commit.
Your post does the same ("A common operation is realising that the commit you re about to make should really go on a new branch for now"), which is why I did it. You don't have to do that at all.
(Side-bar: bzr's design also permits multiple branches within the same repository.)
So I read the manpage and found bzr init-repository, but no way to add a branch to an existing clone, and also no way to switch between branches within a clone. In your original post, you claimed: " Bazaar s command set works the way you do." Well, the way I work is by "switching" between or "checking out" a new branch in-place, but I could not trivially find the command to do this. This is probably because I have not invested enough time into understanding Bazaar, but isn't that what your claim is all about: that "[you] had to take time and effort to understand [Git]" and that Bazaar just works for you?
My post was why I choose bazaar, and why I have rejected other revision control systems that I have attempted to use.
So why did you post it? Surely because you wanted to express your preference over Git. In doing so, you made a couple of claims about Git which are simply untrue, which is why I replied. And the reason I replied is simply because other people who try to make a choice between Git and Bazaar may find your post and take away from it the information that Bazaar is superior to Git, which it is not.
I'm glad that you prefer a different system; but please don't insult me by suggesting that I'm being paid to use the one I prefer, or that I haven't bothered to learn one I'm "slagging off".
But you did slag Git in that you basically present its shortcomings side-by-side to Bazaar's greatness, which is a common PR/marketing thing to do. So just like I failed to create an in-place branch above because I did not bother to learn Bazaar, you did not bother to learn how to do with Git what you showed off with Bazaar, or else I'll have to assume you would not have written up the comparison as you did.
I have to use git as much as I use bzr, and I simply prefer bzr.
That was not the message I took from your original post.

Steve Kemp: It eats the pain

Randomisation is a good thing for media players, but it must be done properly. What do I mean by this? I mean randomness which only works going forwards. As an example consider the case where you have several films/videos in a directory and you run this:
xine --loop=shuffle *.mpeg *.avi
This appears to do what you want, showing each film in a random order but it is subtly doing the wrong thing. If you're watching film "two", having just finished film "five" you'd expect to be able to return to film "five" by pressing "previous"/Page-Up - however that doesn't happen. Xine decides that "moving backwards" in the playlist means selecting another film at random, rather than the previously viewed film which was selected at random - the two are different - and the current behaviour is both counter-intuitive and plain wrong IMHO. I forget the name of the media player that I used briefly which also behaved like this, but it was enough to make me abandon it almost immediately. So, think of the children. If you have a random mode, make sure that going to previous items actually does return to previously played items and not random ones.

MJ Ray: World Masters Coverage 2

In addition to the BBC and Eurosport coverage posted before, I’ve spotted the World Championships free-to-air on ZDF (some point between 12:47 and 18:00 CET tomorrow) and SWR (highlights today, but they seem to be the host broadcaster…) and SLO2 too. RAI3 also shows it in the programme guide, but didn’t seem to cover all of the time trial…

Joey Hess: foo feed

I've added a foo feed to my blog. Stuff posted there won't show up in my main blog feed, so it probably won't go across your eyeballs. I've not been blogging a lot lately, because it often seems not worth wasting bandwidth for random, half-thought-out, or unimportant stuff, and often not worth the effort to try to synthesise something interesting out of it all. I'll probably use the foo feed for such things, as well as for replies to blog posts that don't add a lot of substance. Just grist for the google mill. Maybe that'll mean it becomes the interesting feed that everyone reads.. (PS, am I the only DD on Planet Debian who occasionally gets the heebie-jeebies about my every post scrolling across uncountable computer screens as part of that screensaver? Hi, you. Yes, you, the one reading this in a large-font crawl across your screen right now, who has no idea who I am, or even that there's a person typing this.)

28 September 2007

Clint Adams: A thrusting hit will never find

Once upon a time I proposed a DebConf talk about how to write zsh completion functions, but it was rejected. Accordingly, I didn't waste any time preparing materials for it, so I never have anything to throw at people when they ask for some kind of introduction. Here we have a fictitious program called arismom. Usage information from the fictitious manpage and the fictitious --help output is as follows:
Usage: arismom [OPTION]... [FILE]...
Do it Jersey style.
  -a, --all                       do all those things
  -b                              bubble
      --bounce                    bonuce
      --CoC=STYLE                 adhere to STYLE code of conduct
  -d, --debian=PACKAGE            dedicate actions to Debian PACKAGE
  -e, --ensqualm=USER             ensqualm USER first
  -t, --tempdir=DIRECTORY         spew temporary files into DIRECTORY
So let's cut to the quick. Create a file called _arismom somewhere in your function search path. This is described by the array $fpath. You can view its contents by typing print -l $fpath, and you can add a a directory to the beginning with fpath=(~/.zsh/scratch $fpath) or to the end with fpath+=(~/.zsh/scritch). For the purposes of this blog entry, we'll pretend you have a ~/.michaelbolton/squatch directory in your $fpath and that you are now editing ~/.michaelbolton/squatch/_arismom. The first line of the file, at the very tippy top, should read
#compdef arismom
This ensures that when the completion system boots up and finds your file, it will associate your function with the command arismom and complete options and arguments for it accordingly. Speaking of arguments, skip a line for aesthetic equilibrium, and invoke the _arguments utility function.
_arguments \
_arguments is sort of the cdbs of the zsh completion fleet. By the end of this blog entry, you'll have no idea how it works, and if you want to do anything particularly complex with it, you might encounter some resistance. For those of you unfamiliar with Z-Shell syntax or shell syntax in general, the trailing backslash means I'm a-gonna feed you a ton of information about the command-line interface to arismom. So tell it about the first option already.
  '(-a --all)' -a,--all '[do all those things]' \
To oversimplify, this declares that -a and --all are options which produce the identical behavior of do all those things . Specifically, the part in parentheses says to not complete either -a or --all when either -a or --all is already on the command-line. The part in braces is merely brace expansion; for that reason it is outside of the single quotes. If you're unfamiliar with brace expansion, try print '(alice)' bob,carnie '[wilson]' to see how it expands. Finally, the phrase in brackets is an explanation of the option, which may or may not be displayed depending on your configuration. Next, do a short option that has no long option equivalent.
  '-b[bubble]' \
A long option with no short option equivalent looks similar. Don't feel limited by upstream's inadequate descriptions, misspellings, or poor grammar.
  '--bounce[amplify bounce level according to X-la algorithm]' \
Some options take arguments. Now we use colons.
  '--CoC=[adhere to CoC]:CoC style:(mjg59 buxy ubuntu)' \
The first column is the same optspec by which you've been so excited thus far, and the part between the colons is the message or description of that which will be matched. The part after the last colon is the action; in this case you are specifying a list of possibilities within single parentheses. In most cases, you'll want to be more dynamic than a pre-defined list, and there are many helper functions all ready to serve you.
  '(-d --debian)' -d,--debian= '[dedicate actions to Debian package]:package:_deb_packages avail' \
_deb_packages is a function that completes Debian packages; it can take avail, installed, or uninstalled to restrict which set of packages it offers. In this case we want it to complete any package available from your sources.
  '(-e --ensqualm)' -e,--ensqualm= '[ensqualm user first]:user to ensqualm:_users' \
Here the _users function will complete usernames.
  '(-t --tempdir)' -t,--tempdir= '[spew]:temp dir:_files -/' \
Normally the _files function will complete files, but you can tell it that you only want directories with the -/ option. Finally, we want to cover all the remaining arguments (which according to the fictitious usage information is a list of files). In this case, you happen to believe that files with the .nj extension are to be completed.
  '*:NJ files:_files -g "*.nj"'
The -g option specifies a glob pattern to match files. Now the entire file should look like this.
#compdef arismom
_arguments \
  '(-a --all)' -a,--all '[do all those things]' \
  '-b[bubble]' \
  '--bounce[amplify bounce level according to X-la algorithm]' \
  '--CoC=[adhere to CoC]:CoC style:(mjg59 buxy ubuntu)' \
  '(-d --debian)' -d,--debian= '[dedicate actions to Debian package]:package:_deb_packages avail' \
  '(-e --ensqualm)' -e,--ensqualm= '[ensqualm user first]:user to ensqualm:_users' \
  '(-t --tempdir)' -t,--tempdir= '[spew]:temp dir:_files -/' \
  '*:NJ files:_files -g "*.nj"'
There you have it. Restart zsh and try tab-completing various things after arismom. P.S. I expect bug reports containing functions for dpatch-edit-patch, pkill, and pgrep by tomorrow morning. P.P.S. Why is Scott James Remnant still on Planet?

Ingo Juergensmann: Swap sizes

On planet.d.o I find Russell Cokers blog one of the most interesting blogs at all. Today he blogged about swap sizes in the past and nowadays.
I agree with him that having 16 GB swap space on a 8 GB system is next to useless, but I disagree with this:

A Linux machine with 16M of RAM and 32M of swap MIGHT work acceptably for some applications (although when I was running Linux machines with 16M of RAM I found that if swap use exceeded about 16M then the machine became so slow that a reboot was often needed).


I think this is not always true. Instead it all depends on your swap setup, even more bigger machines (2-8 GB RAM). As some might know, I'm running some m68ks as autobuilders. All of these Amigas are quite low on RAM compared to today standards: 64 MB and 128 MB. On the other hand, some packages are require lots of memory during the build, sometimes even 700-800 MB (rare cases), which means lots and lots of swapping.

The trick is (of course) to distribute the swap space to several disks instead of just one big swap area. This gives a great performance boost while swapping, especially when swapping is not linear. Using 2 SCSI host adapters and 7 SCSI disks in one of the m68k made that machine very responsive even when using 300-400 MB of swap.

So, distributing the swap space to several disks (with same swap priority) gives a performance boost. But keep in mind that it might crash your machine, when one of the disk dies. Alas, this happens of course as well, when you use just a single swap partition. ;)

Russell Coker: Swap Space

There is a wide-spread myth that swap space should be twice the size of RAM. This might have provided some benefit when 16M of RAM was a lot and disks had average access times of 20ms. Now disks can have average access times less than 10ms but RAM has increased to 1G for small machines and 8G or more for large machines. Multiplying the seek performance of disks by a factor of two to five while increasing the amount of data stored by a factor of close to 1000 is obviously not going to work well for performance. A Linux machine with 16M of RAM and 32M of swap MIGHT work acceptably for some applications (although when I was running Linux machines with 16M of RAM I found that if swap use exceeded about 16M then the machine became so slow that a reboot was often needed). But a Linux machine with 8G of RAM and 16G of swap is almost certain to be unusable long before the swap space is exhausted. Therefore giving the machine less swap space and having processes be killed (or malloc() calls fail - depending on the configuration and some other factors) is probably going to be a better situation. There are factors that can alleviate the problems such as RAID controllers that implement write-back caching in hardware, but this only has a small impact on the performance requirements of paging. The 512M of cache RAM that you might find on a RAID controller won’t make that much impact on the IO requirements of 8G or 16G of swap. I often make the swap space on a Linux machine equal the size of RAM (when RAM is less than 1G) and be half the size of RAM for RAM sizes from 2G to 4G. For machines with more than 4G of RAM I will probably stick to a maximum of 2G of swap. I am not convinced that any mass storage system that I have used can handle the load from more than 2G of swap space in active use. The reason for the myths about swap space size are due to some old versions of Unix that used to allocate a page of disk space for every page of virtual memory. Therefore having swap space less than or equal to the size of RAM was impossible and having swap space less than twice the size of RAM was probably a waste of effort (see this reference [1]). However Linux has never worked this way, in Linux the virtual memory size is the size of RAM plus the size of the swap space. So while the “double the size of RAM” rule of thumb gave virtual memory twice the size of physical RAM on some older versions of Unix it gave three times the size of RAM on Linux! Also swap spaces smaller than RAM have always worked well on Linux (I once ran a Linux machine with 8M of RAM and used a floppy disk as a swap device). As far as I recall some time ago (I can’t remember how long) the Linux kernel would by default permit overcommitting of memory. For example if a program tried to malloc() 1G of memory on a machine that had 64M of RAM and 128M of swap then the system call would succeed. However if the program actually tried to use that memory then it would end up getting killed. The current policy is that /proc/sys/vm/overcommit_memory determines what happens when memory is overcommitted, the default value 0 means that the kernel will estimate how much RAM and swap is available and reject memory allocation requests that exceed that value. A value of 1 means that all memory allocation requests will succeed (you could have dozens of processes each malloc 2G of RAM on a machine with 128M of RAM and 128M of swap). A value of 2 means that a different policy will be followed, incidentally my test results don’t match the documentation for value 2. Now if you run a machine with /proc/sys/vm/overcommit_memory set to 0 then you have an incentive to use a moderately large amount of swap, safe in the knowledge that many applications will allocate memory that they don’t use, so the fact that the machine would deliver unacceptably low performance if all the swap was used might not be a problem. In this case the ideal size for swap might be the amount that is usable (based on the storage speed) plus a percentage of the RAM size to cater for programs that allocate memory and never use it. By “moderately large” I mean something significantly less than twice the size of RAM for all machines less than 7 years old. If you run a machine with /proc/sys/vm/overcommit_memory set to 1 then the requirements for swap space should decrease, but the potential for the kernel to run out of memory and kill some processes is increased (not that it’s impossible to have this happen when /proc/sys/vm/overcommit_memory is set to 0). The debian-administration.org site has an article about a package to create a swap file at boot [2] with the aim of making it always be twice the size of RAM. I believe that this is a bad idea, the amount of swap which can be used with decent performance is a small fraction of the storage size on modern systems and often less than the size of RAM. Increasing the amount of RAM will not increase the swap performance, so increasing the swap space is not going to do any good.

Lucas Nussbaum: ZFS as LVM killer really?

From the ZFS FAQ:
Can devices be removed from a ZFS pool? You can remove a device from a mirrored ZFS configuration by using the zpool detach command. Removal of a top-level vdev, such as an entire RAID-Z group or a disk in an unmirrored configuration, is not currently supported. This feature is planned for a future release.
buzz, buzz, buzz…

27 September 2007

Russell Coker: Eating Corpses

Davyd Madeley writes about vegetarianism for the environment [1] which is listed in Wikipedia as Environmental Vegetarianism [2]. He links to an article on the Huffington Post [3]. The Huffington Post article in turn links to an article on GoVeg.com about global warming [4]. Mass-produced meat is not only bad for the environment but there are also health issues related to meat consumption (due to bad practices in mass farming, combining the meat of thousands of animals into mince thus increasing the spread of bad meat, and the fact that most people in first-world countries consume significantly more meat than anyone did at any time in history). One thing that doesn’t get mentioned in these posts is the fact that farming is not required to produce meat. In fact the meat that is most healthy (due to lack of carcinogenic chemicals and free-range feeding) and has the strongest flavour (which may be a good or bad thing depending on whether you actually like the flavour of meat) is from wild animals. If you don’t like the taste of meat (which seems to be the case when people don’t like game meat) then why eat it at all? In Australia large numbers of kangaroos are killed because they eat grass more efficiently than cattle (they have evolved over tens of thousands of years to survive in Australian conditions unlike cattle). There are also a number of foreign animals that have run wild in Australia and are considered vermin, this includes rabbit, pig, buffalo, deer and camel (all of which are tasty). Even among native animals there are often times when a cull is needed. If some good seasons allow the population to increase then when there is a bad season the population has to reduce and it’s often better for them to be culled (thus providing plenty of food for the surviving animals) than for all of them to starve. There is a game meat wholesaler I’ve visited a few times that sells buffalo, rabbit, pig, camel, crocodile, possum, emu, kangaroo, and some other animals. All of the meat is from wild animals (apart from rabbit and pig none of those animals can be domesticated). I’m sure that every region has such a wholesaler that will sell to interested individuals if you know where to look (it seems impossible to buy any game meat other than kangaroo retail in Australia). Finally one thing that offends me is people who eat meat but are not prepared to kill the animal. If you aren’t prepared to kill it then you shouldn’t pay someone else to do so on your behalf! Claiming that “the animal was going to be killed anyway” is a pitiful excuse that is only suitable for children. It’s acceptable for children to eat meat without thinking about where it came from. But adults should be able to deal with the fact that eating meat means killing animals - or become vegetarian if they can’t cope with it. The book 3001 The Final Odyssey pioneered the term “corpse food” for eating meat. I believe that the term is accurate and should be used. If you can’t stomach eating corpses then there are many good vegetarian options available. There are many vegetarians in the Linux community. As these issues are getting discussed a lot recently maybe it would be good to have the vegetarians choose some good vegetarian restaurants to have Linux meetings on occasion. Davyd got a bit of negative feedback on his post, maybe if he invited a bunch of his local Linux people to have dinner at a vegetarian restaurant and they enjoyed the food then the reaction to such ideas would be more positive.

Dirk Eddelbuettel: Dear Samsung,

Your ML-3051N is a lovely little 'network printer. Hopefully, it will serve us well in the next few years as a replacement for the thirteen year old HP Laserjet ML we bought (for about five times the price: a native postscript printer was rather expensive in 1994) a long time ago. And you even mention that it can talk to Linux! But still, I have a few things to talk to you about:
  1. as you mention Linux, it's a tad odd that you enumerate various flavours (RH 8 to 9; FC 1 to 3; Mandrake 9 to 10.2; SuSE 8.2 to 9.2) but happen to exclude the two variants that run around here: Debian and its cousin Ubuntu. Besides, the versions of those other flavours are a tad old, no?
  2. as you ship Linux software on a cdrom, it is even odder that the install shell script fails (note: spaces between #! and /bin/sh may not be a good idea). Fails not once, but twice: you say /bin/sh, but you meant /bin/bash as you use features of the latter. Oh well, I guess we all made that mistake in our youth. Irregardless, the software never installed. Which is a pity as this would be about the first 'household appliance' I bought with native Linux support. So close, and yet so far.
  3. the control panel is a cumbersome way to set a static IP and to turn the odd protocol off.
  4. did I really have to install all this windoze stuff on my wife's laptop just to learn that the printer does in fact have a nice web interface? You could have mentioned that with Linux info, no? It is in fact a rather nice and complete web interface.
  5. as you supply networking support, do you really have to enable every possible protocol under the sun? I mean slp, snmp, multicast dns, dynamic dns, raw tcp, ipp, ethertalk, netware ...? I still see my cable modem go down 'seemingly randomly' rigth after I print even though I now disabled just about everything but raw tcp to port 9100 as well as ipp to 613. Still, dropping the cable connection just because the printer wakes up is odd, isn't it?
  6. Cups, foomatic and all the other printing goodies do not yet know the ML-3050 family, so we are making do with the ML-2151NPS settings for postscript. Works fine, so I won't bother copying the ppd file from the cdrom to the handful of computers around here.
Kind regards, Dirk Updated to fix a markup error. And the shebanh space is standard, I am told.

26 September 2007

Miriam Ruiz: Piklab in Debian

I’ve been so busy lately that I forgot to write about it: Piklab is now in Debian. I don’t know if any of you has done and development with PIC microcontrollers, but it’s not that there was a lot of free programs for doing so. Even now, it might be problematic depending on which hardware you have for programming them. In any case, Piklab is an integrated development environment for applications based on Microchip PIC and dsPIC microcontrollers similar to the MPLAB environment, which makes it easier to develop. It’s a real pity that we cannot consider the newer versions of SDCC as DFSG-free, due some non-commercial clauses, because it is a real nice companion compiler for Piklab.

Scott James Remnant: Why I choose Bazaar (a history of revision control)

Like any sensible software developer, I have a close relationship with revision control systems. In my previous job, I was an SCM Engineer (see Software configuration management) which meant I had an even closer relationship than most, since we were running the CVS servers and actively using them to track changes and deployments. We all know, deep down, that revision control systems shouldn’t exist. This kind of thing should be inherent in the design of the operating system, through standard file and filesystem formats. The OLPC interface is making some headway towards that, but for the rest of us, it means using a revision control tool throughout the development process. Unfortunately, even though the tool is expected to be the most-used command on your system, very few of them are particularly easy to use. Thus there’s a large learning curve, and people become religious about their choice since they have invested significant time in using it. Just to spice the mix up, not only will people religiously defend their choice of revision control system, but they’ll do so while actively hating it. In the beginning there was CVS and we all thought that it was pretty good. It was based on the simpler RCS and shared a file-format with it, but introduced control of directory trees and remote operation. Actually, in reality, CVS wasn’t that good. Its command set could be a little strange and inconsistent (e.g. it’s not possible to diff between two dates on a branch); the support for branching assumed that all branches would be merged into the mainline, and only once; and nobody ever really knew how to create a new project in a repository (tip. cvs import is wrong). But we all used it anyway, and we muddled through. It did have some good features; it was simple, fast and pretty reliable–when it did break, you could usually fix the repository yourself. And most importantly of all, we understood how to drive it. And so it was for many years, until Subversion (SVN) came along. Subversion intended to be “a better CVS”, perhaps this goal should have made us suspicious at the time since CVS was already being a pretty good CVS by itself; unfortunately we hated CVS so much we flocked to the new system in hope. In hindsight, Subversion didn’t really improve on CVS much at all. In fact, arguably, the only real improvement was the addition of atomic commits (in CVS, each commit is per-file, so it’s manual labour to work out which change was made to two files at the same time). (Its support for branching, tagging, copying, renaming, etc. were no better than CVS’s when done in the repository by hand.) The cost of this single new feature was a much more complicated interface (with two separate commands), a backend that tended to break down weekly and a lethargic slowness to its operation. Most people I know now justify their use of Subversion instead of CVS by “Subversion is maintained, CVS isn’t” which is a somewhat self-fulfilling justification. While the mass conversion to Subversion, and ensuing disappointment and frustration, was going on; something new appeared on the horizon: Arch. Arch was different, it broke one of the core assumptions of revision control, that of the repository as a cathedral. In CVS, and Subversion like it, if somebody wants to modify your code (even if on a branch) you need to give them access to your own repository. In some cases (especially with CVS), vast access control and permission structures would be in place to ensure proper behaviour. With Arch, you don’t; all you need to give to anyone is read access. Anybody can make their own branch by copying yours and committing to their own copy. This model also necessitated fixing a long standing problem that CVS had; Arch has repeatable (smart) merging. If you merge from a branch, you can merge again later, and again, and again. Arch made this possible through each commit (changeset) having a globally unique identifier; made from the branch’s own globally unique identifier and the changeset number in the branch. Unfortunately while this was a massive step in a new direction, Arch had an absolutely terrible user interface. Its command list was terrifying with over 100 commands, many of which had multiple word names (tla set-tree-version). It exposed too many of its own innards, and expected you to learn them. It also forced baroque file naming semantics on its users and strange policy (though shalt not commit without first running “make clean”). Efforts were made to improve Arch’s user interface through projects such as baz, but they were always to be doomed from the start. We’ve since seen an explosion of new revision control systems; Monotone, Darcs, Git and Bazaar. What’s especially interesting is the commonality between these systems. They are all “distributed” like Arch, though they also all discard the strange “unique branch identifier” convention and instead simply assign a unique identifier to each file or commit. This means that they all support personal branches, and by necessity all support repeatable (smart) merging. So how do they differ, what are their killer features and killer problems? Monotone is all about repository integrity, ensuring that every commit is both authorised and intact. It pays for this with a severe lack of speed. Darcs is based around a “theory of patches”, a branch is not made up of its history but by the collection of patches in it. Unfortunately this often breaks down, and darcs frequently gets stuck calculating even trial and commonplace branch models. Git is very strange to me; its killer feature appears to be the speed at which it can handle very large trees, but the interface is as insane as Arch’s was. It is heavily optimised for the “I only apply patches” development model, at the expense of ordinary development models (it shares an issue with Arch where calculating annotations on an individual file is an expensive operation). What about Bazaar? Its killer feature is that it is designed to work the way you do. The command set is relatively small, and each of them works in the most obvious manner. It also supports plugins so that you can always implement your own workflow. Of all the revision control systems, it’s the only one (that I’m aware of) that supports both distributed and centralised workflows (and lets you go distributed when you need to, e.g. when you’re on a plane). Here’s a few examples of how Bazaar’s command set works the way you do. To start managing some code in bzr:
$ cd myproject
$ bzr init
To add the files, copy in your usual .bzrignore file and just add everything:
$ cp ~/bzrignore .bzrignore
$ bzr add
added foo.c
added bar.c
Check the output for mistakenly added files, adjust .bzrignore and remove the file with bzr rm. A common operation is realising that the commit you’re about to make should really go on a new branch for now:
$ cd ..
$ cp -a myproject myproject-foo
$ cd myproject-foo
$ bzr commit
A copy of a Bazaar branch is a different branch, you can commit to it separately. There’s a bzr branch command for it too (which deals with issues such as bound branches, checkouts, etc.) but it’s nice to demonstrate that Bazaar does what you’d expect even when you don’t use its own commands. Pulling changes from another branch (where you haven’t made any modifications yet) is easy:
$ bzr pull ../myproject
As is merging (when your branches have diverged):
$ bzr merge ../myproject
One particularly nice feature is that after a merge, you see the merge as a single commit and it can be treated as such; but it also has the set of merged commits indented under it–you can examine these as individual commits as well! What’s the downside of Bazaar? Well, it’s not the fastest system (but by no means the slowest), for small to medium sized projects this is never an issue but may be for extremely large projects–fortunately the developers are improving its performance all the time! But that doesn’t matter; it is, honestly, the first revision control system that I don’t hate.

Lucas Nussbaum: Compiz interest

From time to time, I try Compiz, to see if how it has evolved. The last time was yesterday (I also switched to the xserver-xorg-driver-ati from experimental). But as usual, after using it for a few minutes, I can’t help switching back to metacity. I don’t think that Compiz’s visual effects bring anything on the usability POV, and I just find them annoying after the initial “WOW”. Of course, it’s nice to show off, but to do actual work? Are there people really using it all the time?

Next.

Previous.