A new version of kernel-package in Incoming at the
time of writing adds support for creating a package which contains
the Linux kernel debug image. This means the debugging information
for the modules in the kernel image package, and the uncompressed
vmlinux image. This builds on suggestions and code from Troy Heber,
Theodore Y. Ts o, and Dann Frazier.
As support for kexec/kdump support becomes more real, it will be
very useful to be able to build kernels that have debugging
information available, but not necessarily to install the debugging
information on every single client system.
The .ko object files are stripped of the debugging
information before they are placed in the standard
linux-image installation package. However, before that, a
copy of the vmlinux and unstripped .ko files are saved
in a debuginfo package, and everything except for the
debugging information is removed from them using objcopy
--keep-only-debug. This means that if someone ends up with a
crash dump, they can send it to a support engineer and only the
support engineer needs to install the debuginfo package and use the
crash utility to debug the crash dump. It s also useful
for developers, since the debuginfo information can be stored
somewhere outside of /lib for storing its debug
information, for example. This is useful for keeping the size of
the root partition small, for those who keep /
separate from /usr.
The locations used are compatible also with SystemTap,
which provides free software infrastructure to simplify the
gathering of information about the running Linux system. This
assists diagnosis of a performance or functional problem.
SystemTap eliminates the need for the developer to go
through the tedious and disruptive instrument, recompile, install,
and reboot sequence that may be otherwise required to collect
data.
Also, I had to clear out some FUD about kernel-package from the
ircbot dpkg on the OFTC Debian IRC channel, since
someone had implied that kernel-package was some how
obsolete. As can be seen, it is being actively developed, and
features are being added apace.
ProxyCommands.
A while ago, I got fed up to have to add new entries for each host I wanted to join with a jump server, especially when I only need these entries sporadicly, and decided to write a generic configuration. I ended up with this setup:
Host *%*
ProxyCommand ssh $(echo %h cut -d%% -f1) nc -w1 $(echo %h cut -d%% -f2) %p
The trick here is that you can use subshell expansions in a ProxyCommand. So, when I ssh to host1%host2 , the first subshell expansion returns host1 and the second host2 , this setup ends up being the equivalent of :
Host host1%host2
ProxyCommand ssh host1 nc -w1 host2 %p
which is quite similar to the setup from my previous post.
Later on, I came up with an even more powerful implementation:
Host *%*
ProxyCommand ssh $(echo %h awk -F%% ' OFS="%%"; NF--; print $0 ') nc $(echo %h awk -F%% ' print $NF ') %p
Here, the first awk splits at the % characters and returns all fields except the last one, and the second awk returns only the last field. As a consequence, sshing to host1%host2%host3%host4 will have the first subshell expansion return host1%host2%host3 and the second host4 . The setup will then be equivalent to:
Host host1%host2%host3%host4
ProxyCommand ssh host1%host2%host3 nc -w1 host4 %p
The ssh in the ProxyCommand will, in turn, trigger the rule again so that the result is that host4 will be contacted from host3, which is contacted from host2 that we contacted from host1.
In the meanwhile, I decided % was not that nice a separator, and switched to using /, which also allows for a nicer setup with the same recursive effect:
Host */*
ProxyCommand ssh $(dirname %h) nc -w1 $(basename %h) %p
Finally, since some remote hosts don t have nc installed, I usually copy it in my $HOME on these servers and changed my setup to:
Host */*
ProxyCommand ssh $(dirname %h) PATH=.:\$PATH nc -w1 $(basename %h) %p
The main drawback of this method is that the more jump hosts you use, the more your ssh traffic is encapsulated (recursively) in other ssh traffic. The advantage, though, is that you don t need to forward an agent onto untrusted servers to use ssh key authentication on any of the jump or final servers, nor to forward X11 or tunnel multiple times.
Ok, so at work, pretty much the whole company uses Windows, of some sort. The web team, however, are pretty adamant that they use the best tools available for them, and we use Ubuntu because of this (because it s the easiest option that we all have the same distro, and as long as it s Linux, we don t mind)
Anyway, at the moment we re working through the process of moving all our internal business applications from Visual Basic.NET (urgh!) to PHP + a web based app. The Product Owner for this is one of the bosses, who currently uses Windows Vista.
He has, however, seen us all using Ubuntu and it s got him interested in using it. He s said that he wants to try using Ubuntu, and see whether he can get his work done on that, and slowly transition to it, as obviously, he needs to be able to use Windows for the .NET stuff (which sadly, doesn t work well under WINE).
So, we set him up a Virtual Machine, and he got to grips with what he was doing, great. Except for the fact that he had to allocate memory to it, switching back and forth between windows and Ubuntu was a pain. He eventually gave up on this idea, and got a spare machine, a spare monitor, and set it up next to his machine.
Still, however, it was a bit of a pain, he d have to switch position on his desk, start using a different keyboard and mouse, and i was all a bit of hassle.
Recently, we ve gotten a big screen (which isn t actually that big!) for the Web Team to monitor the servers, our order download service, etc etc. We were showing our manager some of the stuff we could do with it, and he asked the question So, how do you control it - to which my response was to move my mouse off of the edge of my screen, and onto the big screen. So is that a third monitor? he asked. No I replied, it s another machine.
Enter Synergy
So this is the point where I explained to our manager exactly what synergy is. Trying to explain it however, is a bit of a hassle, so I ll try my best here. Synergy is an application that allows you to control another machine s keyboard and mouse from your own PC. It s a bit of a mix between a KVM switch, and Remote Desktop, but it doesn t require extra hardware, and you dont have to relay the video across the network to show it on your screen too.
At this point, our boss got a glint in his eye
Does it work in Windows? he asked. At this point, I didn t realise that he had a seperate machine for Ubuntu. I told him it did anyway, but I wasn t too sure whether it worked with Vista
Lets give it a go anyway
So we went over to his machine, and I installed quicksynergy on his ubuntu box, while he downloaded the Windows installer for Synergy. 5 minutes later, he was grinning like a maniac as he moved his mouse from one machine to another.
Now he s happily using Ubuntu without any hassle, it s just a third screen to him - with different stuff on it.
I think the moral here is, there are different ways to get people to be able to use Linux, and Synergy is one of those great tools that enable people to use Linux without having all the hassle that comes with switching over.
I think also, a quote from m manager sums it up.
This is one of the best things I ve seen all year. I thought it d take forever to setup, but it was so quick. I can be so much more productive nowI hope that this will be one of the turning points for my manager, and we ll have another convert by the end of the year
Thanks to all the hard work by the fglrx packaging team, DRI, MESA, Xorg, Radeon,... etc people!
I've been using fglrx with my laptop since I've had it:
01:00.0 VGA compatible controller: ATI Technologies Inc Radeon Mobility X1400And to the people behind the Debian packaging..
* xserver-xorg-video-radeon from testing.
* libdrm-intel1 and libdrm2 from unstable
* libgl1-mesa-dri and libgl1-mesa-glx from experimental
vader:~# apt-get --purge remove fglrx-driver fglrx-kernel-src fglrx-kernel-2.6.26-1-686 fglrx-sourceReading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
fglrx-driver* (9-2-2)
fglrx-kernel-2.6.26-1-686* (9-2-2+2.6.26-13)
fglrx-kernel-src* (9-2-2)
fglrx-source* (9-2-2)
0 upgraded, 0 newly installed, 4 to remove and 17 not upgraded.
After this operation, 38.8MB disk space will be freed.
Do you want to continue [Y/n]?
I probably lost the ability to suspend and resume but I don't usually do it with that laptop.
Now let's hope my lapop at work will be adapted easily too :)
UPDATE: Seems you lose OpenGL 2.0 if you use the free driver. If you are doing OpenGL stuff like me then you are in trouble ;- read more
A couple of months ago, I switched over to ThePhoneCoop s LLU package. The new service is good so far. The ADSL is blazingly fast (I wonder if it will remain so good) and the main drawbacks so far are a lack of reverse DNS (currently escalating that one up through tech support) and a slightly noisy dialling tone on the phone. I was initially switched to a dynamic IP address (which was a change I wasn t told about in advance), but got a static on request so we could restrict the VPNs a bit more.
As usual, LUGgers were a source of help when it looked like going a bit wrong. Thanks to oddbloke for positive feedback about Buffalo routers (they work fine with OpenWRT, but not tried ADSL). Big thanks to Noodles for pointing out that the Buffalo router I linked wasn t an ADSL router, despite being listed as such, that the only ADSL chipset to have full open source support is the AR7 (but the wifi is dodgy) and that the Broadcom chipsets have 2.6 support but maybe with a binary blob.
I m still not sure if a new router is necessary. The infernal Belkin seems to be working, but it s disconnecting and rebooting a lot more often. I seem to be on the TalkTalk/CPW network - anyone know of problems with Belkin routers? I ve not seen anything yet.
Also, who else has unbundled and how was it for you?
Many people in the western world are completely unaware that in the Moldavian Republic there is an ongoing protest against the current Communist Party which is at power after the parliamentary elections held on Sunday. The protestants accuse the Communists of election fraud and there are numerous reports of dead people being recorded as voters, people which, by religion, chose not to vote, and other several people which couldn't have voted.
I just wanted to make an USB stick bootable and wondered why mkdiskimage -4 /dev/sda 0 32 64 complained about the disk
having too many cylinders. After a few moments, it ocurred to me that since libata, the system hard disk has become sda
and that the stick was sdb or sdc. One ctrl-C later, fdisk confirmed both counts: That I accidentally started
mkdiskimaging my main system hard disk and that the partition table was already gone.
A few hours later, the notebook is back in business without too much data loss. Lucky me.
My notebook has an alibi installation of Windows XP in its first primary partition sized 4 GB. Since I
aborted the mkdiskimage process early enough, I hoped that only the windows installation was hosed. The Linux system is
entirely in LVM (with crypted LVs), and LVM hadn t noticed yet that its PV was gone. I reckoned that I still had a
chance to rescue my data if I didn t reboot.
So I hastily commandeered an USB hard disk (thanks, $CUSTOMER, for quickly supplying one), fdisked, pvcreated and
vgextended it and started the pvmove. While this was happening, I theoretically was able to continue working, which I
didn t because I was afraid of losing. About half an hour into the copy process, the USB disk chose to deregister
itself and the pvmove aborted. Rebooting the disk made it reappear as sde (it was sdd before), and miraculously, a new
pvmove call just worked. However, a good fraction of actions on the shell resulted in a lot of input/output errors on
the screen. I guessed that this was the first sign of my data finally dying, but I let the pvmove continue.
After the pvmove was finished (which was rather fast since the move of the crypted LV means that the
crypted data is moved and not de- and recrypted), I had to recreate a /dev/sda2 partition with type 8e before the LVM
tools allowed me to vgreduce the VG.
I then zeroed out the first 10 GB of the internal disk and spent a few hours reinstalling and patching XP from scratch
before I finally went through the pvcreate, vgextend, pvmove routine in the other direction. Then the big moment
arrived: fsck of all LVs.
I guess that the reason for the input/output errors was that /usr was completely hosed (no single file outside
lost+found), but /home and all other file systems holding data where the way back to the (a week old) backup would have
been quite painful were ok. I pulled /, /usr and /var from the backup to be consistent again, updated to current sid
(avoiding KDE 4 for the time being) and could start working again.
The only real loss was a few hours of work that were stored in /usr/local which died along with /usr, so the result of
my stupidity was not very catastrophic but still a good lesson. And I have learned that LVM is a robust and resilient
beast. Good work. Now I d like to know what happened to my /usr - I always had the impression that pvmove should
not break data even when the target disk suddenly goes away...
/proc/pid/mounts should work on any kernel supporting namespaces (since 2.4.19):
$ md5sum /proc/[0-9]*/mounts awk -F [ /]* ns[$1] = ns[$1] ? ns[$1] , $3 : $3 END for (n in ns) print n, ns[n]
/proc/[0-9]* is used instead of /proc/* to avoid listing self, too. The awk script displays a summary : a md5 sum followed by the PID of all processes which /proc/self/mounts file has this md5 sum./proc/pid/mounts files may be different between namespaces, even when the same mount points are used because the ordering may differ. This method is obviously unreliable, except when all namespaces have different mount points. During my tests, it has been reliable to detect whether a process uses the same namespace as the init process, but different processes with different namespaces were reliably listed together as having the same namespace.
The next method, based on the contents of /proc/pid/mountinfo will work on any kernel implementing this file (since 2.6.26):
$ md5sum /proc/[0-9]*/mountinfo awk -F [ /]* ns[$1] = ns[$1] ? ns[$1] , $3 : $3 END for (n in ns) print n, ns[n]
The script is essentially the same, only pointing at mountinfo instead of mounts./proc/pid/mountinfo does contain different information even when no mount point was changed within a namespace. Unfortunately, chrooted processes can have a different content while not being in a separate namespace. I ve not investigated further, but you may be able to spot chrooted processes by comparing /proc/pid/mountinfo and /proc/pid/mounts.
Both the above methods can be executed as unprivileged user, which alleviates the fact they are not totally reliable.
The last method, on the other hand, is reliable, but requires root privileges, and cgroup filesystem support in the kernel (since 2.6.24).
$ mkdir -p /dev/cgroup/ns
$ mount -t cgroup -o ns /dev/cgroup/ns
$ find /dev/cgroup/ns -name tasks xargs -l1 sed ':a;N;s/\n/,/g;ta'
The output here is a group of PID on separate lines for separate namespaces. The sed one-liner replaces all carriage returns in the tasks files with a comma./dev/cgroup/ns is specialized for namespaces cgroups. The directory itself contains a few files, most notably a tasks file containing a list of processes in the current set/cgroup (in our case, namespace). It also contains a subdirectory for each child namespace. Each of these subdirectories is similarly structured, so that it also contains a tasks list and possible subdirectories for its own children.
This means that not only can you spot different namespaces, but you can also spot how they relate to each other.
There are a few glitches, though. As with other cgroups subsystems, you can create new cgroups by creating a directory in the cgroup filesystem, but it happens not to create a new namespace in the case of the namespace subsystem. With all the kind of namespaces that now exist in Linux, this is not a big surprise, but there doesn t seem to be neither a feedback in the cgroup filesystem to know what kind of namespace is involved, nor a control to force creation of a really separate namespace. You also can t move processes from one mount namespace into another by writing a PID in the tasks file in a given namespace ; that just gives an Operation not permitted error.
On the other hand, contrary to the second method, chrooted processes are not separated out.
I will investigate further cgroups with PID, UTS and network namespaces and will post my findings.
The workshop I went to recently and the ICT Champions in general ( Website Under Development - release early, anyone?) seem to be funded by public and charity money. As I mentioned, the worst thing was all the needless promotion of certain companies products, so why are they promoting private products instead of being neutral and open?
That s bad for Britain, bad for the competing products that are co-produced by the Voluntary and Community Sector and bad for the VCS workers because it trains them in single-tool methods instead of transferable skills. Let the private sector pay for their own adverts! Why can t we stop making this mistake?
Which brings me to the other problem with that ICT advice: well-meaning common-sector businesses like ours can t update it autonomously, for two reasons:
I ve been working on a fun little project lately, trying to make a simple VoIP client which can be used on an Android device. It was fairly straightforward to port PJSIP SIP stack to it, however hooking it up with Android s audio system took a bit more fiddling. Once it was done, I just needed some JNI wrappers for the native functions, allowing me to call them from Java UI. Alltogether, it probably took about a month, most of the time spent chasing down incredibly stupid bugs, like trying to load a binary instead of dynamic library into Java :-). Finally, a couple of days ago I was able to build a primitive, but working application, which can successfully do SIP calls from my G1. The source code and the instructions on how to build it are available at http://android.wooyd.org.
A quick update before the big one about the 2009 Google Summer of Code.
I believe we had a great recruitment drive this year and we have a very good set of proposals to work with. We d like to thank everyone involved for their help. We re now ranking out student applications.
I promised elsewhere that I ll send out our shortlist of projects once Google sends us our preliminary slot allocation today but I misread the thread on the -mentors list and that count will only happen on Thursday, so we ll have to wait a bit more. That shortlist would only include projects, but not individual students. The idea is to give a heads up to everyone before committing to a group of projects and students. It is very important to inform the community as it increases visibility of the students work, giving them more help and support (and also avoids duplicating existing not yet publicized work!).
As far as mentors go, we should have all of our approximately 14 planned projects covered, except for 2. I m posting them here in case you could mentor or help find mentors for those projects. (The wiki pages are not really up to date, so please come on IRC and ask clarifications, see below)
Finish Petr Rockai s Adept 3.0 and bring a Qt4 Package Manager to Debian, with a different interface paradigm than Aptitude-gtk.
I work with log files a lot.
Most of the logfiles I work with are in a standard format of some kind, and most often they are rotated upon a daily basis. (Examples include syslog, qpsmtpd, and Apache logfiles.)
I wish there were a general purpose way to say "grep time-range pattern logfile".
Right now, for example, I've just deployed some changes upon a cluster of hosts. Now I want to see only messages that refer to a particular area of the codebase only those that occurred after 23:00 - which is when I did the commit/push/pull dance.
I've written a quick hack - tgrep (time-grep) - which allows simple before/equal/after/range grepping :
# show matching lines after 23:00PM tgrep \>23:00:00 -i subject /var/log/qpsmtpd/qpsmtpd.log # show matching lines in the interval 23:00PM 23:15PM tgrep 23:00:00-23:15:00 -i -r subject /var/log/qpsmtpd/If there is a common way of doing this "properly" then I'd love to be educated, failing that take it if it is useful (moreutils?) ObFilm: Chasing Amy
Date 30th March 2009
All being well the Debian Administration website now fully supports UTF-8.
This change was a long time coming, considering the amount of time the site has been live.
Most of the changes have been present for a while:
Next.