Yes, just when you thought the spamming of Planet Debian with "Lenny released" blog posts had finally stopped, here comes another one :-)
Let me join the crowd by saying a great "thank you!" to all the people who made this release possible, especially so the release team who organized everything, as well as the thousands of contributors (in one form or another) who helped shape the new release!
Personally, I'm eager to try out the new Linux 2.6.28 kernel package in unstable now (which have been uploaded today or so, but haven't yet reached my mirror), since they contain mainline wireless drivers for my One A110 netbook, among many other things.
Also, in the next few days I'll probably re-install my NSLU2 ARM box using the latest Lenny installer, following the HOWTOs by Martin Michlmayr (I'll probably write about the experience later). This re-install is long overdue, as I'm currently running the box from an 1GB thumb drive, which works ok, but I'm slowly running out of space. So I'll re-install on a 4 GB (or bigger) thumb drive.
I ve been using git this last few days and I am still working on a workflow for my projects. Unfortunately, as others have noticed, git violates POLS is so many ways, it ends up being hard to get.
Creating a remote repository seems to be the first thing to bite a developer switching to git (mainly if coming from a centralized SCM). I have not decided which is the best way of doing it, but I ve been using git-daemon via inetd and a path in my remote hosting holding all my repositories for public pulling, and ssh for pushing. Here is how to create it:
local$ ssh spectra@remotehost
remotehost$ mkdir /var/git/myproject.git
remotehost$ cd /var/git/myproject.git
remotehost$ git --bare init
remotehost$ logout
local$ cd myproject
local$ git remote add remotehost spectra@remotehost:/var/git/myproject.git
local$ git push remotehost master
...
otherlocal$ git clone git://remotelocal/myproject.git
otherlocal$ cd myproject
otherlocal$ git remote add remotehost spectra@remotehost:/var/git/myproject.git
(hack hack hack)
otherlocal$ git push remotehost master
Now everybody can pull your repository at git://remotehost/myproject.git and you can push and pull to it via ssh. Note that you have to setup git-daemon, which is pretty straight forward. I am using it as an inetd daemon, but you can use it as a standalone one. Debian has a package which does just that.
Now, some people think logging in a remote server just to create an empty repository is too much. Well repositories are just .git directories. It happens that you can push for the first time by rsyncing your .git with a remote host:
local$ cd myproject
local$ rsync -a .git/ spectra@remotehost:/var/git/myproject.git
local$ git remote add remotehost spectra@remotehost:/var/git/myproject.git
(hack hack hack)
local$ git push remotehost master
(Apparently you can push using rsync every time, but it s regarded as wiser to keep your probably crappy local repository commits separated from the public repository otherwise commit messages like Please, don t use this code are likely to pop up everywhere :) ). Now, I don t know if this have side effects, but it works :-)
Another thing to notice is that git is more directed at pulling than pushing. This may be because of its designer: the way Linus works is by pulling changes from others repositories and not by letting others push em into his one. And this is another violation of POLS for most of the people, who is used to commit their changes into some remote repository. Rather than that, people using git would expose their own repositories in order to have it pulled by others.
I also agree with most of the git critics wrt git commands There s a lot of examples and I am not going deeper in this but I think it was a bad choice calling checkout what git does when told to checkout , for instance. Yes, I know different tools, different ways of seeing it but everyone was already used to what centralized SCMs call their operations, and I think it would only help git adopting same names for the same operations, and inventing new ones for those proper of decentralized operations. Anyway, once you get it (and I have not completely got it yet), it seems all flow as expected. If this adaptation fails, there s still Easy Git to the rescue!
One last thing that I think contribute to the git sucks effect: git-svn. This is a great tool, but it was built from git s point of view Given it s intended as a glue for Subversion newcomers, it would benefit more from being built from svn s point of view. This was mentioned by a colleague developer in my company, when he just couldn t understand why one have to git svn rebase instead of a simple update. So git-svn also suffers of the bad command naming habit git do. Of course, that given that you came from this environment (I am sure git and git-svn makes perfect sense for Linus & cia :-) ). I have not tried yet, but yap seems to be targeted on providing an alternative to git-svn.
I start to move some servers when lenny was freeze, and did not have any complain, the update was really smooth; but I still let some servers to be migrated just at the end of the release cycle, and sure, they fail I actually have some fun migrating freeradius and cacti, and note that I actually will have more fun migrating asterisk and openser servers, but for the sake of this release, is the fun that every BOFH should expect, and also, cheers and thanks to everyone who work on this release Torvalds said he rejects GNU GPL Version 3 because he approves of tivoization, because he does not agree that users deserve the freedom to change their own copies of software.By choosing the GPL for his own code Linus Torvalds gave users the freedom to change their own copies of Linux. This certainly seems at odds with what is quoted. There are many different points of view about how the cause of mukta software and its users can be strengthened. I think we can agree that it is certainly not strengthened by shooting each other in the foot. The GPLv3 bolsters-up one aspect of this cause: use the legal system to make if difficult (if not impossible) to take free software and put it in proprietary bottles. Linus Torvalds and all his "show me the code" buddies are addressing another aspect; to ensure that there are enough people accomplished in the task of reading, improving and writing high quality free software. The West has traditionally had more success with the law. When free programs was written in the 70's and 80's by people who did not care to preserve its freedom with a free license, this code was gobbled up and re-gurgitated as proprietary software. Once free licenses became commonplace in the 90's, the West has had a resurgence of high quality public code and coders. Unfortunately, some of the latter tend to look down on "mere users". Over here, in India, we have little faith in the legal system. (For reasons, see this report and a more recent one). We know that freedom is hard fought even when the law is technically on your side. At the same time, the brahminical approach of knowledge being safe-guarded by the self-proclaimed elite was what rang the death knell for Indian knowledge systems in the past. So it is important that the breed of skeptical people in India who tinker with their computers and software (and everything else) grows.
We have far too many cattle who will follow any flute-playing good-looking young person who is willing to fight their demons for them.
All of this being said, it s time to revisit this question is all of this needed for a smart , better by design next-generation SSD such as Intel s? Aligning your file system on an erase block boundary is critical on first generation SSD s, but the Intel X25-M is supposed to have smarter algorithms that allow it to reduce the effect of write-amplification. The details are a little bit vague, but presumably there is a mapping table which maps sectors (at some internal sector size we don t know for sure whether it s 512 bytes or some larger size) to individual erase blocks. If the file system sends a series of 4k writes for file system blocks 10, 12, 13, 32, 33, 34, 35, 64, 65, 66, 67, 96, 97, 98, 99, followed by a barrier operation, a traditional SSD might do read/modify/write on four 128k erase blocks one covering the blocks 0-31, another for the blocks 32-63, and so on. However, the Intel SSD will simply write a single 128k block that indicates where the latest versions of blocks 10, 12, 13, 32, 33, 34, 35, 64, 65, 66, 67, 96, 97, 98, and 99 can be found.
This technique tends to work very well. However, over time, the table will get terribly fragmented, and depending on whether the internal block sector size is 512 or 4k (or something in between), there could be a situation where all but one or two of the internal sectors on the disks have been mapped away to other erase blocks, leading to fragmentation of the erase blocks. This is not just a theoretical problem; there are reports from the field that this happens relatively easy. For example, see Allyn Malventano s Long-term performance analysis of Intel Mainstream SSDs and Marc Prieur s report from BeHardware.com which includes an official response from Intel regarding this phenomenon. Laurent Gilson posted on the Linux-Thinkpad mailing list that when he tried using the X25-M to store commit journals for a database, that after writing 170% of the capacity of an Intel SSD, the small writes caused the write performance to go through the floor. More troubling, Allyn Malventano indicated that if the drive is abused for too long with a mixture of small and large writes, it can get into a state where the performance degredation is permanent, and even a series of large writes apparently does not restore the drive s function only an ATA SECURITY ERASE command to completely reset the mapping table seems to help.
So, what can be done to prevent this? Allyn s review speculates that aligning writes to erase write boundaries can help I m not 100% sure this is true, but without detailed knowledge of what is going on under the covers in Intel s SSD, we won t know for sure. It certainly can t hurt, though, and there is a distinct possibility that the internal sector size is larger than 512 bytes, which means the default partitioning scheme of 255 heads/63 sectors is probably not a good idea. (Even Vista has moved to a 240/63 scheme, which gives you 8k alignment of partitions; I prefer 224/56 partitioning, since the days when BIOS s used C/H/S I/O are long gone.)
The Ext3 and Ext4 file system tend to defer meta-data writes by pinning them until a transaction commit; this definitely helps, and ext4 allows you to configure an erase block boundary, which should also be helpful. Enabling laptop mode will discourage writing to the disk except in large blocks, which probably helps significantly as well. And avoiding fsync() in applications will also be helpful, since a cache flush operation will force the SSD to write to an erase block even if it isn t completely filled. Beyond that, clearly some experimentation will be needed. My current thinking is to use a standard file system aging workload, and then performing an I/O benchmark to see if there has been any performance degradation. I can then vary various file system tuning parameters and algorithms, confirm whether or not a heavy fsync workload makes the performance worse.
In the long term, hopefully Intel will release a firmware update which adds support for ATA TRIM/DISCARD commands, which will allow the file system to inform the SSD that various blocks have been deleted and no longer need to be preserved by the SSD. I suspect this will be a big help, if the SSD knows that certain sectors no longer need to be preserved, it can avoid copying them when trying to defragment the SSD. Given how expensive the X25-M SSD s are, I hope that there will be a firmware update to support this, and that Intel won t leave its early adopters high and dry by only offering this functionality in newer models of the SSD. If they were to do that, it would leave many of these early adopters, especially your humble writer (who paid for his SSD out of his own pocket), to be quite grumpy indeed. Hopefully, though, it won t come to that.
Update: I ve since penned a follow-up post Should Filesystems Be Optimized for SSD s?
Related posts (automatically generated):
All of this being said, it s time to revisit this question is all of this needed for a smart , better by design next-generation SSD such as Intel s? Aligning your file system on an erase block boundary is critical on first generation SSD s, but the Intel X25-M is supposed to have smarter algorithms that allow it to reduce the effect of write-amplification. The details are a little bit vague, but presumably there is a mapping table which maps sectors (at some internal sector size we don t know for sure whether it s 512 bytes or some larger size) to individual erase blocks. If the file system sends a series of 4k writes for file system blocks 10, 12, 13, 32, 33, 34, 35, 64, 65, 66, 67, 96, 97, 98, 99, followed by a barrier operation, a traditional SSD might do read/modify/write on four 128k erase blocks one covering the blocks 0-31, another for the blocks 32-63, and so on. However, the Intel SSD will simply write a single 128k block that indicates where the latest versions of blocks 10, 12, 13, 32, 33, 34, 35, 64, 65, 66, 67, 96, 97, 98, and 99 can be found.
This technique tends to work very well. However, over time, the table will get terribly fragmented, and depending on whether the internal block sector size is 512 or 4k (or something in between), there could be a situation where all but one or two of the internal sectors on the disks have been mapped away to other erase blocks, leading to fragmentation of the erase blocks. This is not just a theoretical problem; there are reports from the field that this happens relatively easy. For example, see Allyn Malventano s Long-term performance analysis of Intel Mainstream SSDs and Marc Prieur s report from BeHardware.com which includes an official response from Intel regarding this phenomenon. Laurent Gilson posted on the Linux-Thinkpad mailing list that when he tried using the X25-M to store commit journals for a database, that after writing 170% of the capacity of an Intel SSD, the small writes caused the write performance to go through the floor. More troubling, Allyn Malventano indicated that if the drive is abused for too long with a mixture of small and large writes, it can get into a state where the performance degredation is permanent, and even a series of large writes apparently does not restore the drive s function only an ATA SECURITY ERASE command to completely reset the mapping table seems to help.
So, what can be done to prevent this? Allyn s review speculates that aligning writes to erase write boundaries can help I m not 100% sure this is true, but without detailed knowledge of what is going on under the covers in Intel s SSD, we won t know for sure. It certainly can t hurt, though, and there is a distinct possibility that the internal sector size is larger than 512 bytes, which means the default partitioning scheme of 255 heads/63 sectors is probably not a good idea. (Even Vista has moved to a 240/63 scheme, which gives you 8k alignment of partitions; I prefer 224/56 partitioning, since the days when BIOS s used C/H/S I/O are long gone.)
The Ext3 and Ext4 file system tend to defer meta-data writes by pinning them until a transaction commit; this definitely helps, and ext4 allows you to configure an erase block boundary, which should also be helpful. Enabling laptop mode will discourage writing to the disk except in large blocks, which probably helps significantly as well. And avoiding fsync() in applications will also be helpful, since a cache flush operation will force the SSD to write to an erase block even if it isn t completely filled. Beyond that, clearly some experimentation will be needed. My current thinking is to use a standard file system aging workload, and then performing an I/O benchmark to see if there has been any performance degradation. I can then vary various file system tuning parameters and algorithms, confirm whether or not a heavy fsync workload makes the performance worse.
In the long term, hopefully Intel will release a firmware update which adds support for ATA TRIM/DISCARD commands, which will allow the file system to inform the SSD that various blocks have been deleted and no longer need to be preserved by the SSD. I suspect this will be a big help, if the SSD knows that certain sectors no longer need to be preserved, it can avoid copying them when trying to defragment the SSD. Given how expensive the X25-M SSD s are, I hope that there will be a firmware update to support this, and that Intel won t leave its early adopters high and dry by only offering this functionality in newer models of the SSD. If they were to do that, it would leave many of these early adopters, especially your humble writer (who paid for his SSD out of his own pocket), to be quite grumpy indeed. Hopefully, though, it won t come to that.
Update: I ve since penned a follow-up post Should Filesystems Be Optimized for SSD s?
Related posts (automatically generated):
We just had an interesting problem with a project based upon Drupal. From time to time one would get
an error 500 (internal server error) for no apparent reason. Reloading the page would usually fix the issue. Attemps to search Apache logs which pages threw that error and searching for something in common turned out nothing. Via google we soon found out, that adding AddType x-mapp-php5 .php to our .htaccess file would fix the problem. Don't know why; googling in reverse that that does didn't help; most pages you get tell you that you can fix several php and/or Drupal related problems with that. So can anyone give me an hint, what happens, what the problem was and why on earth enabling php again solves the problem? PS: For testing purpouse we did something like the following: for x in $(seq 1 15000) ; do if $(wget -q $URL_TO_CHECK -O/dev/null) ; then echo "$x: klappt" ; else echo "$x: 500er" ; fi ; done >> test-log and then grep -c 500er. Before we added the line, we had nine internal server errors. With that line enabled, we didn't had a single one in over 50000 access. So that line indeed has some effect, but why?
Got myself a new laptop
this week. Not that the other one broke down or some such, but it
was getting old; and its single-core 1.3Ghz PowerPC was
starting to be somewhat slow in comparison to what's available on the
market today. Additionally, while it was still functional, there
were some issues with wear-out; for instance, sometimes the
battery would just fall out while I was walking around with it, or the
network cable connection would not fit entirely, and similar minor
problems
The new one is quite a bargain; 2G of RAM, 2Ghz Core2Duo, one of
those newfangled 'n' wireless network interfaces, plenty of diskspace,
and other interesting things; and all that for only 600ish. You'd say
it's a netbook, but it isn't although I guess the low price of
those netbook systems is pulling down the bottom line of these
machines.
Anyway. After using it for slightly more than a day, I'm quite happy
with it. I did have to upgrade a few packages in order to make all the
hardware work properly; for instance, kernel 2.6.26 does not yet support
the wireless (so I'm running 2.6.28 now), and the X.org in unstable can
only get at the screen in vesa mode, which means I'm running the 7.4
X.org packages from experimental.
But overall, most of it (that I care about) works flawlessly.
With a single exception: I have no audio. The weird thing is, the
audio chipset is recognized, and I can play around with alsamixer; but
no matter, there is no sound. Which is kindof annoying. It doesn't
appear that I'm the only one with this problem (witness this
debugging page on the alsa wiki), so I'm sure I'll get there.
Eventually.
Oh wel.
While fetching the laptop, I was told that the NSLU2 which I had on
backorder had arrived, so I took that with me, too. Add to that the 160G
disk that I'd fetched earlier, and the USB NIC that I also got, and the
Thecus N2100 that's still running at my parents' place will soon be
moved to my home again and its 500G RAID1 with it.
Whee.
If you're interested in free software --- and free network services in
particular --- and should try to join me in Boston for the weekend of
March 21st and 22nd.
The FSF is organizing its annual members meeting again. This year the
model is very different. For a start, the audience isn't limited to FSF
members and the conference is not just about FSF projects and work.
Instead, the meeting has been rebranded LibrePlanet and has been
broken up into a two-day event that is going to talk about and then try
to tackle some of the biggest problems facing the world of free and open
source software. Saturday March 21st will feature a series of talks
about major issues facing free software. Sunday March 22nd will be
focused on an unconference attempt to tackle and explore several of the
key themes or tracks: network services, high priority projects, and the
nascent LibrePlanet activism network.
I'll be focused on the track around free network services which I'm
helping organize in part through Autonomo.us. For more information
on that angle, please take a look at my blog post over at
Autonomo.us. We're going to have a great group of people at the track
and I'm excited by the idea that that we'll be able to make some real
progress on the issues.
I encourage anyone who thinks they might be able to make it to consider
doing so. There are details including travel, location, hotel
information and much more on the event web page and wiki (login is
required to RSVP). Please spread the word!
Some guy from New Zealand blogged about a vulnerability
in .desktop files as used by Gnome and KDE, and claims you can write a
virus with them for Linux. He actually meant a Trojan rather than a
virus, but still.
I'm not saying the vulnerability does not exist. It does, and the
trojan he describes should work in theory.
However, in practice, I believe that the real reason why there aren't
any Linux viruses is not the fact that Linux is somehow 'safer', but
really the fact that Linux isn't a monoculture like Windows is.
If you want to write a Windows virus, you have to deal with, at most,
four to five different versions of windows that are currently still in
active use with most users. Target that, and your virus will live like
there's no tomorrow.
If you want to write a Linux virus, the range of target system is
much, much larger. In his very blog entry that describes the
vulnerability, 'foobar' describes already two special cases he has to
consider: the fact that KDE and Gnome write .desktop entries that get
executed on startup to different locations, and the fact that,
while most distributions ship either curl or wget, none of them has
standardized on either of those, requiring the virus writer to account
for that, too.
While his blog entry stops at that, there's way, way more he
has to deal with. If our supposed trojan writer wants his trojan to
proliferate, he will have to deal with distributions that come with or
without SE Linux; distributions that ship with a /tmp mounted noexec;
distributions that ship with python 2.3, 2.4, 2.5, or 3.0; users who
use Thunderbird, Evolution, Kmail, mutt, pine, or another mail client;
and possibly much, much more (this only describes what's required as
described in the referenced blog post). All these differences would make
writing some malware that would exploit the described vulnerability
tedious, at best, and rather impractical in most cases.
Again, I'm not saying the vulnerability does not exist; it does. But
I think this lack of monoculture under Linux, not the (perceived)
strength of the platform, is what helps users defend against malware on
Linux.
memisc package for R:
One of the aims of this package is to make life easier for useRs who deal with survey data sets. It provides an infrastructure for the management of survey data including value labels, definable missing values, recoding of variables, production of code books, and import of (subsets of) SPSS and Stata files. Further, it provides functionality to produce tables and data frames of arbitrary descriptive statistics and (almost) publication-ready tables of regression model estimates. Also some convenience tools for graphics, programming, and simulation are provided. [emphasis added]How did I miss this package before? It makes analyzing NES data heck, any data with value labels and missing values in R an almost sane thing to do.
Hey hey, look what I had in my mailbox this morning:
The website for Tourism Queensland's extremely
popular promotion to find someone to be paid $150,000 to be an island
caretaker for 6 months is hosted by a Canberra company on a server in
Canberra.
So much for supporting local business.
(Small world, one of the Executive Directors was briefly my manager at
CyberTrust, but that's the Canberra IT scene for you)
In other news, my HP 2510p's screen was replaced last month after the hinge snapped. A chunk of plastic off the hinge cover snapped off two days ago. I'm somewhat puzzled by this, since I can't see any plausible way force could be applied to it - it's as if it came away slightly and then got crushed when I tried to close the lid. On top of the motherboard having been replaced 4 times now (twice due to faulty power connectors, one due to the fan being replaced and the motherboard being swapped at the same time, once because the machine started refusing to boot at LCA) and it still being slightly tempremental when booting, I'm not overly impressed - especially when I've only had it 18 months. Nobody else I know with one seems to have had the same level of difficulty, though dreadful thermal issues (especially when using the dock) seem to be common.
Next.