Search Results: "ag"

10 April 2009

Russell Coker: Fixing the Correct Network Bottleneck

The latest news in the Australian IT industry is the new National Broadband Network (NBN) plan [1]. It will involve rolling out Fiber To The Home for 90% of the population, the plan is that it will cost the government $43,000,000,000 making it the biggest government project. Kevin Rudd used Twitter to say Just announced biggest ever investment in Australian broadband - really exciting, infrastructure for the future [2]. Now whenever someone says that a certain quantity of a resource is enough you can expect someone to try and refute that claim by mentioning that Bill Gates supposedly stated that 640K is enough when referring to the RAM limits of the original IBM PC. As an aside, it s generally believed that Bill Gates actually didn t claim that 640K would be enough RAM, Wikiquote has him claiming to have never said any such thing [3]. He did however say that he had hoped that it would be enough for 10 years. I think that I needed that disclaimer before stating that I think that broadband speeds in Australia are high enough at the moment. In any computer system you will have one or more resources that will be limited and will be bottlenecks that limit the overall performance. Adding more of other resources will often make no difference to performance that a user might notice. On the machine I m using right now to browse the web the bottleneck is RAM. A combination of bloated web pages and memory inefficient web browsers uses lots of memory, I have 1.5G of RAM and currently there is 1.3G of swap in use and performance suffers because of it. It s not uncommon for the machine to page enough that the mouse cursor is not responsive while browsing the web. My options for getting faster net access on this machine are to add more RAM (it can t take more than 2G - so that doesn t gain much), to use more memory efficient web browsers and X server, and to simply buy a new machine. Dell is currently selling desktop machines with 2G of RAM, as they are 64bit systems and will therefore use more memory than 32bit systems for the same tasks they will probably give less performance than my 32bit machine with 1.5G of RAM for my usage patterns. Also the latest EeePC [4] ships with 1G of RAM as standard and is limited to a maximum of 2G, I think that this is typical of Netbook class systems. I don t use my EeePC for any serious work, but I know some people who do. Does anyone have suggestions on memory efficient web browsers for Linux? I m currently using Konqueror and Iceweasel (Firefox). Maybe the government could get a better return on their investment by spending a small amount of money sponsoring the development of free web browsers. A million dollars spent on optimising Firefox seems likely to provide good performance benefits for everyone. My wife s web browsing experience is bottlenecked by the speed of the video hardware in her machine (built-in video on a Dell PowerEdge T105 which is an ATI ES1000). The recent dramatic price reductions of large TFT monitors seem likely to make video performance more of an issue, and also increases the RAM used by the X server. Someone who has reasonably good net access at the moment will have an ADSL2+ connection and a computer that is equivalent to a low-end new Dell machine (which is more powerful than the majority of systems in use). In that case the bottleneck will be in the PC used for web browsing if you are doing anything serious (EG having dozens of windows open, including PDFs and other files that are commonly loaded from the web). If however a machine was used for simply downloading web pages with large pictures in a single session then FTTH would provide a real benefit. Downloading movies over the net would also benefit a lot from FTTH. So it seems to me that browsing the web for research and education (which involves cross-referencing many sites) would gain more of a benefit from new hardware (which will become cheap in a few years) while porn surfing and downloading movies would gain significantly from FTTH. The NBN will have the potential to offer great bi-directional speeds. The ADSL technology imposes a limit on the combination of upload and download speeds, and due to interference it s apparently technically easier to get a high download speed. But the upload speeds could be improved a lot by using different DSLAMS. Being able to send out data at a reasonable speed (20Mbit/s or more) has the potential to significantly improve the use of the net in Australia. But if the major ISPs continue to have terms of service prohibiting the running of servers then that won t make much difference to most users. Finally there s the issue of International data transfer which is slow and expensive. This is going to keep all affordable net access plans limited to a small quota (20G of downloads per month or less). It seems to me that the best way of spending taxpayer money to improve net access would be to provide better connectivity to the rest of the world through subsidised International links. Brendan makes an interesting point that the NBN is essentially a subsidy to the entertainment industry and that copyright law reform should be a higher priority [5].

9 April 2009

Manoj Srivastava: Manoj: kernel-package: Add support for kernel image debug package

A new version of kernel-package in Incoming at the time of writing adds support for creating a package which contains the Linux kernel debug image. This means the debugging information for the modules in the kernel image package, and the uncompressed vmlinux image. This builds on suggestions and code from Troy Heber, Theodore Y. Ts o, and Dann Frazier. As support for kexec/kdump support becomes more real, it will be very useful to be able to build kernels that have debugging information available, but not necessarily to install the debugging information on every single client system. The .ko object files are stripped of the debugging information before they are placed in the standard linux-image installation package. However, before that, a copy of the vmlinux and unstripped .ko files are saved in a debuginfo package, and everything except for the debugging information is removed from them using objcopy --keep-only-debug. This means that if someone ends up with a crash dump, they can send it to a support engineer and only the support engineer needs to install the debuginfo package and use the crash utility to debug the crash dump. It s also useful for developers, since the debuginfo information can be stored somewhere outside of /lib for storing its debug information, for example. This is useful for keeping the size of the root partition small, for those who keep / separate from /usr. The locations used are compatible also with SystemTap, which provides free software infrastructure to simplify the gathering of information about the running Linux system. This assists diagnosis of a performance or functional problem. SystemTap eliminates the need for the developer to go through the tedious and disruptive instrument, recompile, install, and reboot sequence that may be otherwise required to collect data. Also, I had to clear out some FUD about kernel-package from the ircbot dpkg on the OFTC Debian IRC channel, since someone had implied that kernel-package was some how obsolete. As can be seen, it is being actively developed, and features are being added apace.

Mike Hommey: SSH through jump hosts

I often need to connect to a server with ssh from another server because I don t have direct access. I even gave a small configuration example to use such jump hosts using ProxyCommands. A while ago, I got fed up to have to add new entries for each host I wanted to join with a jump server, especially when I only need these entries sporadicly, and decided to write a generic configuration. I ended up with this setup:
Host *%*
ProxyCommand ssh $(echo %h cut -d%% -f1) nc -w1 $(echo %h cut -d%% -f2) %p
The trick here is that you can use subshell expansions in a ProxyCommand. So, when I ssh to host1%host2 , the first subshell expansion returns host1 and the second host2 , this setup ends up being the equivalent of :
Host host1%host2
ProxyCommand ssh host1 nc -w1 host2 %p
which is quite similar to the setup from my previous post. Later on, I came up with an even more powerful implementation:
Host *%*
ProxyCommand ssh $(echo %h awk -F%% ' OFS="%%"; NF--; print $0 ') nc $(echo %h awk -F%% ' print $NF ') %p
Here, the first awk splits at the % characters and returns all fields except the last one, and the second awk returns only the last field. As a consequence, sshing to host1%host2%host3%host4 will have the first subshell expansion return host1%host2%host3 and the second host4 . The setup will then be equivalent to:
Host host1%host2%host3%host4
ProxyCommand ssh host1%host2%host3 nc -w1 host4 %p
The ssh in the ProxyCommand will, in turn, trigger the rule again so that the result is that host4 will be contacted from host3, which is contacted from host2 that we contacted from host1. In the meanwhile, I decided % was not that nice a separator, and switched to using /, which also allows for a nicer setup with the same recursive effect:
Host */*
ProxyCommand ssh $(dirname %h) nc -w1 $(basename %h) %p
Finally, since some remote hosts don t have nc installed, I usually copy it in my $HOME on these servers and changed my setup to:
Host */*
ProxyCommand ssh $(dirname %h) PATH=.:\$PATH nc -w1 $(basename %h) %p
The main drawback of this method is that the more jump hosts you use, the more your ssh traffic is encapsulated (recursively) in other ssh traffic. The advantage, though, is that you don t need to forward an agent onto untrusted servers to use ssh key authentication on any of the jump or final servers, nor to forward X11 or tunnel multiple times.

Martin Meredith: My Boss, with a little help, starts the conversion to Linux

Ok, so at work, pretty much the whole company uses Windows, of some sort. The web team, however, are pretty adamant that they use the best tools available for them, and we use Ubuntu because of this (because it s the easiest option that we all have the same distro, and as long as it s Linux, we don t mind) Anyway, at the moment we re working through the process of moving all our internal business applications from Visual Basic.NET (urgh!) to PHP + a web based app. The Product Owner for this is one of the bosses, who currently uses Windows Vista. He has, however, seen us all using Ubuntu and it s got him interested in using it. He s said that he wants to try using Ubuntu, and see whether he can get his work done on that, and slowly transition to it, as obviously, he needs to be able to use Windows for the .NET stuff (which sadly, doesn t work well under WINE). So, we set him up a Virtual Machine, and he got to grips with what he was doing, great. Except for the fact that he had to allocate memory to it, switching back and forth between windows and Ubuntu was a pain. He eventually gave up on this idea, and got a spare machine, a spare monitor, and set it up next to his machine. Still, however, it was a bit of a pain, he d have to switch position on his desk, start using a different keyboard and mouse, and i was all a bit of hassle. Recently, we ve gotten a big screen (which isn t actually that big!) for the Web Team to monitor the servers, our order download service, etc etc. We were showing our manager some of the stuff we could do with it, and he asked the question So, how do you control it - to which my response was to move my mouse off of the edge of my screen, and onto the big screen. So is that a third monitor? he asked. No I replied, it s another machine. Enter Synergy So this is the point where I explained to our manager exactly what synergy is. Trying to explain it however, is a bit of a hassle, so I ll try my best here. Synergy is an application that allows you to control another machine s keyboard and mouse from your own PC. It s a bit of a mix between a KVM switch, and Remote Desktop, but it doesn t require extra hardware, and you dont have to relay the video across the network to show it on your screen too. At this point, our boss got a glint in his eye Does it work in Windows? he asked. At this point, I didn t realise that he had a seperate machine for Ubuntu. I told him it did anyway, but I wasn t too sure whether it worked with Vista Lets give it a go anyway So we went over to his machine, and I installed quicksynergy on his ubuntu box, while he downloaded the Windows installer for Synergy. 5 minutes later, he was grinning like a maniac as he moved his mouse from one machine to another. Now he s happily using Ubuntu without any hassle, it s just a third screen to him - with different stuff on it. I think the moral here is, there are different ways to get people to be able to use Linux, and Synergy is one of those great tools that enable people to use Linux without having all the hassle that comes with switching over. I think also, a quote from m manager sums it up.
This is one of the best things I ve seen all year. I thought it d take forever to setup, but it was so quick. I can be so much more productive now
I hope that this will be one of the turning points for my manager, and we ll have another convert by the end of the year

Mohammed Sameer: Bye fglrx...

Thanks to all the hard work by the fglrx packaging team, DRI, MESA, Xorg, Radeon,... etc people!

I've been using fglrx with my laptop since I've had it:

01:00.0 VGA compatible controller: ATI Technologies Inc Radeon Mobility X1400

And to the people behind the Debian packaging..
* xserver-xorg-video-radeon from testing.
* libdrm-intel1 and libdrm2 from unstable
* libgl1-mesa-dri and libgl1-mesa-glx from experimental

vader:~# apt-get --purge remove fglrx-driver fglrx-kernel-src fglrx-kernel-2.6.26-1-686 fglrx-sourceReading package lists... Done
Building dependency tree
Reading state information... Done

The following packages will be REMOVED:
fglrx-driver* (9-2-2)
fglrx-kernel-2.6.26-1-686* (9-2-2+2.6.26-13)
fglrx-kernel-src* (9-2-2)
fglrx-source* (9-2-2)
0 upgraded, 0 newly installed, 4 to remove and 17 not upgraded.
After this operation, 38.8MB disk space will be freed.
Do you want to continue [Y/n]?

I probably lost the ability to suspend and resume but I don't usually do it with that laptop.

Now let's hope my lapop at work will be adapted easily too :)

UPDATE: Seems you lose OpenGL 2.0 if you use the free driver. If you are doing OpenGL stuff like me then you are in trouble ;- read more

MJ Ray: Switched over to LLU and ADSL-2+

A couple of months ago, I switched over to ThePhoneCoop s LLU package. The new service is good so far. The ADSL is blazingly fast (I wonder if it will remain so good) and the main drawbacks so far are a lack of reverse DNS (currently escalating that one up through tech support) and a slightly noisy dialling tone on the phone. I was initially switched to a dynamic IP address (which was a change I wasn t told about in advance), but got a static on request so we could restrict the VPNs a bit more. As usual, LUGgers were a source of help when it looked like going a bit wrong. Thanks to oddbloke for positive feedback about Buffalo routers (they work fine with OpenWRT, but not tried ADSL). Big thanks to Noodles for pointing out that the Buffalo router I linked wasn t an ADSL router, despite being listed as such, that the only ADSL chipset to have full open source support is the AR7 (but the wifi is dodgy) and that the Broadcom chipsets have 2.6 support but maybe with a binary blob. I m still not sure if a new router is necessary. The infernal Belkin seems to be working, but it s disconnecting and rebooting a lot more often. I seem to be on the TalkTalk/CPW network - anyone know of problems with Belkin routers? I ve not seen anything yet. Also, who else has unbundled and how was it for you?

Biella Coleman: From Open Source Software to Open Culture: Three Misunderstandings

A few weeks ago, I jotted down some thoughts about (false) expectations made on FLOSS/F/OSS, something that I wish I had more time to write about (teaching and other things seem to take most of my time these days). But before this year turned into the next one, I wanted to pass on a few additional (and similar though they enter into different territory) thoughts by O Reilly editor, Andy Oram: From Open Source Software to Open Culture: Three Misunderstandings

Biella Coleman: From Open Source Software to Open Culture: Three Misunderstandings

A few weeks ago, I jotted down some thoughts about (false) expectations made on FLOSS/F/OSS, something that I wish I had more time to write about (teaching and other things seem to take most of my time these days). But before this year turned into the next one, I wanted to pass on a few additional (and similar though they enter into different territory) thoughts by O Reilly editor, Andy Oram: From Open Source Software to Open Culture: Three Misunderstandings

8 April 2009

Brett Parker: Another day, another fab evening!

So, was supposed to end up in Hector's House playing pool with various people that I know, but all of 'em were out last night (erm, half of 'em with me) and didn't quite manage to make it... So after an hour of waiting for people to turn up I gave up and considered going back to my "local" (the hop poles) to have a swift beer or three... instead, it being a wednesday, and knowing that Band Aids was on at the Pav Tav, I wandered there... managed to get there before the first band played - they were Vier and were absolutely bloody fantastic. Stayed and watched/danced to the other 3 bands and all was well with the world. Now it's time for the sleep and the getting everything else ready for the big move on Friday (yay for working Good Friday... or something!)

Eddy Petrișor: About the protests in Moldavian Republic

Many people in the western world are completely unaware that in the Moldavian Republic there is an ongoing protest against the current Communist Party which is at power after the parliamentary elections held on Sunday. The protestants accuse the Communists of election fraud and there are numerous reports of dead people being recorded as voters, people which, by religion, chose not to vote, and other several people which couldn't have voted.

Tuesday the protests took a violent turn, apparently "aided" by some instigators which were reported to be seen as recording the protesters, taking notes and, generally, not fitting the picture. During the day the national television broadcasted absolutely no information on the events, and aired documentaries, cartoons and no news on any of the protests. To me, this looked very much alike what the Romanian Television aired during the Romanian Revolution in December '89 when during the fights in the streets of Bucharest the Television aired no images from the centre of Bucharest.

But, back to Moldavia, at the end of the day, sadly, the Parliament and Presidency buildings were devastated, making it easy for the Communist leaders to produce some disinformation propaganda based on real footage.

Yet again, sadly, but it seems under the protection of some of the aforementioned instigators, the flags of the EU and Romania were waved at the highest levels of the Parliament building during the time the protesters allegedly took over that building (there were several reports that the Police forces actually simply let them in, without actually needing/being forced to do that; also the passivity of the policemen seen in the footage from Romanian televisons was obvious).

As many Moldavian students live and study in Romania, some of them wanted to go home and take part at the protests happening in Chi in u. To their surprise, many of them were denied the right to enter their home country. Many, if not all, Romanian journalists were also denied access in the country accusing alternatively technical issues, lack of proper papers (although only the passport or the ID card was officially required), lack of safety, or simply refusing without reason.


Today, the Moldavian president Voronin, accused Romania as being behind the manifestations in Chi in u, Moldova's capital and said it will declare as a persona non grata the Ambasador of Romania in Moldova, and thus forcing him to leave Moldova within 24 hours. He went even further and said that visas will be required from now on for Romanian nationals. As soon as this was announced, it was one of the most invoked reasons by the Moldavian customs as a rejection reason, although, according to Moldavian Embassy officials in Romania, no rules or procedure were known for the release of visas for Romanian nationals.

Also, during Wenesday, Moldavian crossing points were opened and closed alternatively but incosistently accros the entire border for Moldavian nationals. Romanian nationals were denied in most cases (although some Romanian truck drivers were reported as allowed to cross). At some point during the day the Moldavian officials from the customs told the Moldavian students awating at the border that they are offered access on the condition they go direclty home with the buses provided by the Moldavian customs and were warned that going to Chi in u wasn't a smart choice for them. Some of the information from students which managed to enter their home country suggested that is advisable to not enter the country since they were going to be denied of the right to leave the country later, when they were to return to Romania to their studies (after Easter).

What's sadder is that the Moldavian voting system is built in such a way that the vote counting, in the end is done by Government officials behind closed doors, inside a Ministry. OTOH, the international official observed the things which happend in public, but did not see any of the dead voter lists (since they were not present when those votes were casted), and declared the vote as mostly legal and without major or significant incidents.


According to the media, before the elections, the Communists were credited with 35-37% of the votes in an independent poll, while in the exit poll conducted by the Communists themselves they were credited with 42-45% of the votes. In the end the last official partial results credited them with approximately 51% of the votes.

About Vladimir Voronin, it is a known fact that he has double citizenship, Moldavian and Russian and is a reserve general within the Russian Army and currently has a pretty nice pension from the Russian state.


All I can say is that, although the Moldavians' desire to have a real democracy is a legal and reasonable request, the western world chose, due to the international circumstances, to look the other way around and not to incommodate Russia in any way. Moreover, to accuse Romania to have unionist desires an act upon those desires is planly stupid since now Romania is part of the EU and Moldova uniting with Romania is simply impossible now, since Moldavia's economy would sunk even more the fragile and regressing Romanian economy and Romania would instantly fail the EU requirements which would mean disaster for us.

Marc 'Zugschlus' Haber: partition table gone, data still present

I just wanted to make an USB stick bootable and wondered why mkdiskimage -4 /dev/sda 0 32 64 complained about the disk having too many cylinders. After a few moments, it ocurred to me that since libata, the system hard disk has become sda and that the stick was sdb or sdc. One ctrl-C later, fdisk confirmed both counts: That I accidentally started mkdiskimaging my main system hard disk and that the partition table was already gone. A few hours later, the notebook is back in business without too much data loss. Lucky me. My notebook has an alibi installation of Windows XP in its first primary partition sized 4 GB. Since I aborted the mkdiskimage process early enough, I hoped that only the windows installation was hosed. The Linux system is entirely in LVM (with crypted LVs), and LVM hadn t noticed yet that its PV was gone. I reckoned that I still had a chance to rescue my data if I didn t reboot. So I hastily commandeered an USB hard disk (thanks, $CUSTOMER, for quickly supplying one), fdisked, pvcreated and vgextended it and started the pvmove. While this was happening, I theoretically was able to continue working, which I didn t because I was afraid of losing. About half an hour into the copy process, the USB disk chose to deregister itself and the pvmove aborted. Rebooting the disk made it reappear as sde (it was sdd before), and miraculously, a new pvmove call just worked. However, a good fraction of actions on the shell resulted in a lot of input/output errors on the screen. I guessed that this was the first sign of my data finally dying, but I let the pvmove continue. After the pvmove was finished (which was rather fast since the move of the crypted LV means that the crypted data is moved and not de- and recrypted), I had to recreate a /dev/sda2 partition with type 8e before the LVM tools allowed me to vgreduce the VG. I then zeroed out the first 10 GB of the internal disk and spent a few hours reinstalling and patching XP from scratch before I finally went through the pvcreate, vgextend, pvmove routine in the other direction. Then the big moment arrived: fsck of all LVs. I guess that the reason for the input/output errors was that /usr was completely hosed (no single file outside lost+found), but /home and all other file systems holding data where the way back to the (a week old) backup would have been quite painful were ok. I pulled /, /usr and /var from the backup to be consistent again, updated to current sid (avoiding KDE 4 for the time being) and could start working again. The only real loss was a few hours of work that were stored in /usr/local which died along with /usr, so the result of my stupidity was not very catastrophic but still a good lesson. And I have learned that LVM is a robust and resilient beast. Good work. Now I d like to know what happened to my /usr - I always had the impression that pvmove should not break data even when the target disk suddenly goes away...

Mike Hommey: Detecting per-process namespaces

Back with per-process namespaces. This time, I will give several methods to more or less reliably detect their use, some requiring root access, some not. We will, though, limit ourselves to mount namespaces , that is, not PID, UTS or network namespaces. The first method, which is based on the contents of /proc/pid/mounts should work on any kernel supporting namespaces (since 2.4.19):
$ md5sum /proc/[0-9]*/mounts awk -F [ /]* ns[$1] = ns[$1] ? ns[$1] , $3 : $3 END for (n in ns) print n, ns[n]
/proc/[0-9]* is used instead of /proc/* to avoid listing self, too. The awk script displays a summary : a md5 sum followed by the PID of all processes which /proc/self/mounts file has this md5 sum.
This method relies on the fact that the /proc/pid/mounts files may be different between namespaces, even when the same mount points are used because the ordering may differ. This method is obviously unreliable, except when all namespaces have different mount points. During my tests, it has been reliable to detect whether a process uses the same namespace as the init process, but different processes with different namespaces were reliably listed together as having the same namespace. The next method, based on the contents of /proc/pid/mountinfo will work on any kernel implementing this file (since 2.6.26):
$ md5sum /proc/[0-9]*/mountinfo awk -F [ /]* ns[$1] = ns[$1] ? ns[$1] , $3 : $3 END for (n in ns) print n, ns[n]
The script is essentially the same, only pointing at mountinfo instead of mounts.
This method is quite reliable, as /proc/pid/mountinfo does contain different information even when no mount point was changed within a namespace. Unfortunately, chrooted processes can have a different content while not being in a separate namespace. I ve not investigated further, but you may be able to spot chrooted processes by comparing /proc/pid/mountinfo and /proc/pid/mounts. Both the above methods can be executed as unprivileged user, which alleviates the fact they are not totally reliable. The last method, on the other hand, is reliable, but requires root privileges, and cgroup filesystem support in the kernel (since 2.6.24).
$ mkdir -p /dev/cgroup/ns
$ mount -t cgroup -o ns /dev/cgroup/ns
$ find /dev/cgroup/ns -name tasks xargs -l1 sed ':a;N;s/\n/,/g;ta'
The output here is a group of PID on separate lines for separate namespaces. The sed one-liner replaces all carriage returns in the tasks files with a comma.
You don t actually need root privileges for the last command, if the cgroup filesystem is already mounted. But there has been no normalization as to where the cgroup filesystems should be mounted yet, and they aren t mounted by default on most GNU/Linux distributions, I believe. Cgroups (Control groups) allow to put sets of processes together to give them special behaviour (cpu provisioning, memory limitations, namespaces, etc.). They also allow to do so in a hierarchical way, such that a cgroup can inherit the properties of its parent cgroup. As its name and the way it is mounted indicates, our /dev/cgroup/ns is specialized for namespaces cgroups. The directory itself contains a few files, most notably a tasks file containing a list of processes in the current set/cgroup (in our case, namespace). It also contains a subdirectory for each child namespace. Each of these subdirectories is similarly structured, so that it also contains a tasks list and possible subdirectories for its own children. This means that not only can you spot different namespaces, but you can also spot how they relate to each other. There are a few glitches, though. As with other cgroups subsystems, you can create new cgroups by creating a directory in the cgroup filesystem, but it happens not to create a new namespace in the case of the namespace subsystem. With all the kind of namespaces that now exist in Linux, this is not a big surprise, but there doesn t seem to be neither a feedback in the cgroup filesystem to know what kind of namespace is involved, nor a control to force creation of a really separate namespace. You also can t move processes from one mount namespace into another by writing a PID in the tasks file in a given namespace ; that just gives an Operation not permitted error. On the other hand, contrary to the second method, chrooted processes are not separated out. I will investigate further cgroups with PID, UTS and network namespaces and will post my findings.

David Welton: New LangPop.com stats, open source bits and pieces

I released some new statistics on LangPop.com. There are some new things:
  • The normalized results are now user configurable - you can put in your own weights for the various data sources (such as Amazon, Craigslist, and so on), and see how the results change.
  • Freshmeat.net fiddled with their web site, and statistics are not going to be available this month. Hopefully they will be able to correct the problem soon.
  • On the other hand, I wrote some code to grab the data from Ohloh, so there's something new to look at while we wait for Freshmeat to come back on line.
So there is a fair amount going on. Hopefully Freshmeat will come back soon, and I can also add a few fixes to the IRC statistics which I've been thinking about. In the process of doing the above, I also managed to do some open source work, albeit of a minor nature:
  • I updated LangPop.com to use the Flotr Javascript chart library instead of Plotr. So I also updated my Ruby code to integrate it. It's a fairly thin layer at this point, but it does what I need. It's available here: http://github.com/davidw/chartr/tree/master
  • I did some hacking on the Simile Timeline code, (used here on LangPop.com), correcting a bug, and also creating a system to install the whole thing with a simple command. Normally, it's kind of difficult to just use the Simile code on your own site, so I created an (admittedly hacky) way of making it easier to install. That code is here: http://github.com/davidw/standalone-timeplot/tree/master
Oh, yeah, and we moved from Innsbruck, Austria back to Padova, Italy!

David Welton: New LangPop.com stats, open source bits and pieces

I released some new statistics on LangPop.com. There are some new things:
  • The normalized results are now user configurable - you can put in your own weights for the various data sources (such as Amazon, Craigslist, and so on), and see how the results change.
  • Freshmeat.net fiddled with their web site, and statistics are not going to be available this month. Hopefully they will be able to correct the problem soon.
  • On the other hand, I wrote some code to grab the data from Ohloh, so there's something new to look at while we wait for Freshmeat to come back on line.
So there is a fair amount going on. Hopefully Freshmeat will come back soon, and I can also add a few fixes to the IRC statistics which I've been thinking about. In the process of doing the above, I also managed to do some open source work, albeit of a minor nature:
  • I updated LangPop.com to use the Flotr Javascript chart library instead of Plotr. So I also updated my Ruby code to integrate it. It's a fairly thin layer at this point, but it does what I need. It's available here: http://github.com/davidw/chartr/tree/master
  • I did some hacking on the Simile Timeline code, (used here on LangPop.com), correcting a bug, and also creating a system to install the whole thing with a simple command. Normally, it's kind of difficult to just use the Simile code on your own site, so I created an (admittedly hacky) way of making it easier to install. That code is here: http://github.com/davidw/standalone-timeplot/tree/master
Oh, yeah, and we moved from Innsbruck, Austria back to Padova, Italy!

MJ Ray: Why is public and charity money paying for the private sector s marketing?

The workshop I went to recently and the ICT Champions in general ( Website Under Development - release early, anyone?) seem to be funded by public and charity money. As I mentioned, the worst thing was all the needless promotion of certain companies products, so why are they promoting private products instead of being neutral and open? That s bad for Britain, bad for the competing products that are co-produced by the Voluntary and Community Sector and bad for the VCS workers because it trains them in single-tool methods instead of transferable skills. Let the private sector pay for their own adverts! Why can t we stop making this mistake? Which brings me to the other problem with that ICT advice: well-meaning common-sector businesses like ours can t update it autonomously, for two reasons:
  1. All the texts I saw were under various Non-Commercial and No Derivative licences which make them hazardous or unusable to IT support businesses.
  2. Particularly on Free and Open Source Software, the advice is at best misleading and unhelpful. This is very frustrating: based on our private work and success in explaining things, I m sure we could do better, but for various reasons, no-one is paying for us to publish that. Instead, someone s paying to publish statements like There s no such thing as open source written by someone from an association that includes Microsoft and Oracle.
Again, why is public and charity money paying for the private sector s marketing? Why is it published under blocking copyright terms, scattering broken glass across the path just built? Where can we get funding to publish our explanations?

Jurij Smakov: VoIP client for Android

I ve been working on a fun little project lately, trying to make a simple VoIP client which can be used on an Android device. It was fairly straightforward to port PJSIP SIP stack to it, however hooking it up with Android s audio system took a bit more fiddling. Once it was done, I just needed some JNI wrappers for the native functions, allowing me to call them from Java UI. Alltogether, it probably took about a month, most of the time spent chasing down incredibly stupid bugs, like trying to load a binary instead of dynamic library into Java :-). Finally, a couple of days ago I was able to build a primitive, but working application, which can successfully do SIP calls from my G1. The source code and the instructions on how to build it are available at http://android.wooyd.org.

John Goerzen: Goodbye

Jacob has lately been telling us gate gamma kassen so sick (Great Grandma Klassen is so sick), after his visit to the hospital, and it s true. Today started normal enough, but by 2:45 I got an email from my mom, saying that grandma had taken yet another turn for the worse; her pulse was racing, her temperature feverish, and her breathing shallow and difficult. The last grandchild that was going to be there made it, and got the last flicker of recognition from grandma. I wrapped some things up at work. Then I made the short drive from work to the nursing home, and got there just after 4. I went in to her room there, and it was full of family. Two of her children were singing her favorite hymns. I can never forget my uncle s deep bass voice as he stood at her bedside, holding her arm, while my aunt patted her head, both of them singing. Grandma s only remaining sibling, her younger brother Melvin, sat on the other side of the bed, looking on. Grandma s white table and chairs, which were in grandma and grandpa s house as long as I can remember, and followed grandma through all of the four other places she lived in the last years of her life, were in her room at the nursing home too. It always felt like home to be with grandma and those familiar things that she always took with her. Despite what was happening, I was glad she was back at home. My uncle motioned me over to her bedside, and I took her hand for a few minutes. It felt cold and weak; for the first time, she didn t grab my hand at all. A nurse came in to check oxygen saturation, but the machine wasn t able to get a reading due to poor circulation to her hand. She wasn t able to get a heart rate either because the heart was racing so fast. She discussed briefly whether the family wanted them to continue giving her oxygen, and they decided that they would, for her comfort. My uncle and aunt kept singing. I blew my nose and dabbed at my eyes, and there were hugs all around. And just a few minutes later, grandma peacefully stopped breathing, maybe 20 minutes after I had arrived. They went to get the nurse, who came back to listen to grandma s heart again, though we knew what she would find. She took the stethoscope off, and she almost lost her composure, but managed to say you can turn the oxygen off now. Several people gave grandma a last hug. After a few minutes with just the family, they told the nurses to go ahead and call the funeral home. That set off a lot of activity making arrangements for the next few days, funeral plans, and the like. I stepped back into grandma s room a few times, while the family was in the sitting area right outside it discussing. I looked around at the family photos on the wall, the old table and chairs, the recliner. Now, these are the things that were grandma s. It didn t feel like home anymore. Terah and Jacob were stuck at home Terah s car was at the mechanic for repair today. I called to give her the news. She wanted to come to be with the family, but I didn t really want to leave. She tried calling some friends to see if they could give her a lift to the mechanic, but not one of them was available. I talked to her again and suggested she just call the mechanic. She wasn t even halfway through describing the situation when he interrupted with, We ll have it at your place right away! Well, I don t expect you to have to do that, or you could certainly wait until you close. Nope, this isn t your average community, we ll bring it right over. You should be there. Grandma has enjoyed a simple life and had requested a simple death: no extraordinary measures at the end of life, no embalming either. So, by law, the burial must happen within 24 hours of death, and will be tomorrow. After all the arrangements, people realized nobody had supper yet. We went to the quickest available option pizza and ate there. It was paid for out of grandma s remaining money the last meal of so many that she provided for her family over the years. It was a happy meal. As I drove home, NPR news was on the radio. There were the same stories we hear all the time: the economy, the mideast, the president. Normally I m interested, but today I shut it off. Today is different. Tomorrow, for the second time in four years, I will help carry a grandparent s casket a few days before Easter. I ll end tonight with this photo. It was taken soon after Jacob was born. Grandma came to the hospital and held him. That smile sums her up perfectly. img_4965r.jpg

Obey Arthur Liu: Google Summer of Code at Debian: Update, need mentors!

A quick update before the big one about the 2009 Google Summer of Code. I believe we had a great recruitment drive this year and we have a very good set of proposals to work with. We d like to thank everyone involved for their help. We re now ranking out student applications. I promised elsewhere that I ll send out our shortlist of projects once Google sends us our preliminary slot allocation today but I misread the thread on the -mentors list and that count will only happen on Thursday, so we ll have to wait a bit more. That shortlist would only include projects, but not individual students. The idea is to give a heads up to everyone before committing to a group of projects and students. It is very important to inform the community as it increases visibility of the students work, giving them more help and support (and also avoids duplicating existing not yet publicized work!). As far as mentors go, we should have all of our approximately 14 planned projects covered, except for 2. I m posting them here in case you could mentor or help find mentors for those projects. (The wiki pages are not really up to date, so please come on IRC and ask clarifications, see below) Finish Petr Rockai s Adept 3.0 and bring a Qt4 Package Manager to Debian, with a different interface paradigm than Aptitude-gtk.
Petr said he would provide help with the existing codebase but can t mentor. Sune Vuorela from Debian KDE is ready to help with Qt4 related issues. Build Debian tools to create Debian images for Amazon EC2 and the free Eucalyptus implementation. Packaging of the Eucalyptus hosting framework is also possible.
For this project, we already have on board to help: Charles Plessy from Debian Med, Eric Hammond, developer of the existing vmbuilder Ubuntu tool for EC2 and Chris Grzegorczyk and Rich Wolski, from the Eucalyptus team. Plenty of people to get help from. Mentoring is a great experience! See this for what it entails. If we still can t find a mentor by the end of the week, I ll blast an announcement over at debian-devel@l.d.o along with the project shortlist. In the meantime, don t forget to idle on #debian-soc on irc.debian.org.

7 April 2009

Steve Kemp: I think you should let this one go

I work with log files a lot. Most of the logfiles I work with are in a standard format of some kind, and most often they are rotated upon a daily basis. (Examples include syslog, qpsmtpd, and Apache logfiles.) I wish there were a general purpose way to say "grep time-range pattern logfile". Right now, for example, I've just deployed some changes upon a cluster of hosts. Now I want to see only messages that refer to a particular area of the codebase only those that occurred after 23:00 - which is when I did the commit/push/pull dance. I've written a quick hack - tgrep (time-grep) - which allows simple before/equal/after/range grepping :
# show matching lines after 23:00PM
tgrep \>23:00:00 -i subject /var/log/qpsmtpd/qpsmtpd.log
# show matching lines in the interval 23:00PM 23:15PM
tgrep 23:00:00-23:15:00 -i -r subject /var/log/qpsmtpd/
If there is a common way of doing this "properly" then I'd love to be educated, failing that take it if it is useful (moreutils?) ObFilm: Chasing Amy

Steve Kemp: A mystery of open questions

Date 30th March 2009 All being well the Debian Administration website now fully supports UTF-8. This change was a long time coming, considering the amount of time the site has been live. Most of the changes have been present for a while: The only missing part was ensuring the at the text input by visitors/users was correctly decoded and treated as UTF-8. This was handled by updating changing the Perl CGI module to explicitly call charset appropriately. Since the code behind the site masks the database, memcached, and CGI handles behind singletons the change itself was pretty trivial: I made more changes this evening to tie it all together, and to ensure that my Database connection is always forced to use UTF but I think that wasn't so important. I hope this is vaguely useful the next time I have to fight with character sets & encodings. It is just all so nasty. Failing that these pages are vaguely useful: ObFilm: Run Lola Run

Next.

Previous.